Methods and systems for predicting response to PD-1 axis-directed therapeutics

Abstract

To predict a response to PD-1 axis directed therapeutics.SOLUTION: A scoring function is developed and used for identifying patients who might be responsive to a PD-1 axis directed therapy. The scoring function is obtained by extracting features from multiplex-stained sections, selecting features that correlate with a response to the therapy using a feature selection function, and fitting one or more of the selected features to a plurality of candidate scoring functions. A candidate scoring function showing the desired balance between predictive sensitivity and specificity may be selected for incorporation into a scoring system that includes at least an image analysis system.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This is an international application under the Patent Cooperation Treaty, which claims priority to U.S. Provisional Patent Application No. 62 / 739,828, filed October 1, 2018, and U.S. Provisional Patent Application No. 62 / 742,934, filed October 9, 2018, the contents of each of which are incorporated herein by reference in their entirety.

[0002] Reference to sequence listing submitted as an ASCII text file (.TXT) In accordance with the EFS-Web legal framework and 37 CFR § 1.821-825 (see MPEP § 2442.03(a)), a Sequence Listing in the form of an ASCII-compliant text file (Entitled "Sequence_Listing_3000022-004977_ST25.txt", created September 30, 2019, and 23,475 bytes in size) has been submitted contemporaneously with the present application, and the entire contents of the Sequence Listing are incorporated herein by reference. [Background technology]

[0003] FIELD OF THE INVENTION The present invention relates to the detection, characterization, and enumeration of biomarkers in tumor samples that are useful for predicting response to checkpoint inhibitor therapy.

[0004] 2. Description of Related Art Cancers can escape immune surveillance and eradication through upregulation of the programmed death 1 (PD-1) pathway and its ligand, programmed death-ligand 1 (PD-L1), in tumor cells and the tumor microenvironment. Blockade of this pathway with antibodies against PD-1 or PD-L1 has resulted in remarkable clinical responses in some cancer patients. However, identifying predictive biomarkers for patient selection remains a major challenge.

[0005] PD-L1 is the most widely used predictive biomarker for selecting patients for PD-1 axis-directed therapy. However, the results observed have been inconsistent. See Yi.

[0006] Mismatch repair (MMR) deficiency predicts solid tumor response to PD-1 blockade. See Le(I) and Le(II). However, not all patients with mismatch repair deficiency respond to PD-1 blockade treatment. Predictive value is limited due to the variable strength of the relationship between study and tumor type.

[0007] Recent studies suggest that the spatial arrangement and interactions between cancer cells and immune cells influence patient prognosis, survival, and response to treatment.

[0008] There is an increasing need to understand the tumor microenvironment and associated biomarkers to guide cancer immunotherapy. Summary of the Invention

[0009] The present disclosure relates generally to systems and methods for identifying and using novel biomarkers that predict the response of solid tumors to PD-1 axis-directed therapy.

[0010] In one embodiment, a method for developing a scoring function for predicting tumor response to a PD-1 axis-directed therapy is disclosed, the method comprising: (a) (a1) a set of digital images of tumor tissue samples obtained from a plurality of patients prior to treatment with a PD-1 axis-directed therapy, wherein at least one digital image for each patient is a digital image of a tissue section stained with a multiplex affinity histochemical stain for each of one or more epithelial markers, one or more immune cell markers, and one or more PD-1 axis pathway markers; and (a2) a set of digital images of tumor tissue samples obtained from a plurality of patients prior to treatment with a PD-1 axis-directed therapy, wherein at least one digital image for each patient is a digital image of a tissue section stained with a multiplex affinity histochemical stain for each of one or more epithelial markers, one or more immune cell markers, and one or more PD-1 axis pathway markers. (b) extracting a plurality of features from digital images of the multiply-stained tissue sections, (c) applying a feature selection function to the extracted plurality of features and the post-treatment response data to obtain a rank for each feature for strength of correlation with response to the PD-1 axis-directed therapy, (d) applying a modeling function to one or more of the ranked features and the post-treatment response data to generate a plurality of candidate models predicting response to checkpoint inhibitor therapy and testing each candidate model for concordance with the response, and (e) selecting the candidate model with the highest concordance with the response as a scoring function. In one embodiment, the multi-affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), and the features are selected from the group consisting of the features in the left column of Table 4. In one embodiment, the multiplex affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PanCK, PD-L1, PD1, CD8, and LAG3 (Panel 2), and the features are selected from the group consisting of features in the right column of Table 4. In one embodiment, the feature selection function is selected from the group consisting of ensemble feature selection methods (e.g., including Random Forest functions), filter methods (e.g., including mutual information-based functions, (mRMR) / correlation coefficient-based functions, and Relief-based functions), and / or embedded feature selection functions (such as elastic net / least absolute shrinkage or selection operator (LASSO) functions).In one embodiment, the candidate model is created using one or more of the top 25, top 20, top 15, top 10, top 9, top 8, top 7, top 6, top 5, top 4, or top 3 features identified by the feature selection function. In another embodiment, the candidate model uses at least one, at least two, at least three, at least four, or at least five features identified in the top 10 features of the feature selection function. In another embodiment, the candidate model includes at least one feature present in the top five features of at least two feature selection functions. In one embodiment, the modeling function is selected from the group consisting of quadratic discriminant analysis (QDA), linear discriminant analysis (LDA), support vector machine (SVM), and artificial neural network (ANN). In one embodiment, the PD-1 axis-directed therapy is a PD-1-specific monoclonal antibody or a PD-L1-specific monoclonal antibody. In one embodiment, the PD-1 axis-directed therapy is selected from the group consisting of pembrolizumab, nivolumab, atezolizumab, avelumab, durvalumab, cemiplimab, tislelizumab, and LY3300054.

[0011] In one embodiment, a method is provided for scoring a tumor sample for its likelihood of responding to a PD-1 axis-directed therapy, the method comprising: (a) obtaining a digital image of a tumor section from a tumor sample, the tumor section being stained with a multiple affinity histochemical stain for each of one or more epithelial markers, one or more immune cell markers, and one or more PD-1 axis pathway markers; (b) identifying a region of interest (ROI) within the digital image; (c) extracting one or more features from the ROI related to cells stained for each biomarker; and (d) applying a scoring function to a feature vector including the extracted feature(s) of (c) to generate a score indicative of the likelihood of the tumor responding to the PD-1 axis-directed therapy. In one embodiment, the ROI is derived from a digital image of a morphologically stained section of the tumor sample, the morphologically stained section and the multiple affinity histochemically stained sample being serial sections. In one embodiment, the ROI is identified by a user in the digital image of the morphologically stained section and automatically registered to the digital image of the multi-affinity histochemically stained section. In one embodiment, the multi-affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), the ROI is according to Table 3, and the feature comprises at least one feature selected from the group consisting of the features in the left column of Table 4. In one embodiment, the multi-affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), the ROI is according to Table 3, and the feature comprises at least one feature determined by ReliefF and / or Random Forest to be important in predicting patient response to PD-1 axis-directed therapy.In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), wherein the ROI is according to Table 3 and the feature comprises at least one feature determined by ReliefF and / or Random Forest to be one of the top 10 features most important for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), wherein the ROI is according to Table 3 and the feature comprises at least 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 of the top 10 features most important for predicting patient response to PD-1 axis-directed therapy, according to ReliefF and / or Random Forest. In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), wherein the ROI is an ROI according to Table 3, and the features comprise at least one feature selected from the group consisting of: the percentage of PD-L1+ macrophages in the stroma, the percentage of PD-L1+CD3+CD8- cells in the stroma, and the percentage of PD-L1+CD3+ cells in the stroma. In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), wherein the ROI is an ROI according to Table 3, and the features comprise each of the percentage of PD-L1+ macrophages in the stroma, the percentage of PD-L1+CD3+CD8- cells in the stroma, and the percentage of PD-L1+CD3+ cells in the stroma. In one embodiment, the multiplex affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PanCK, PD-L1, PD1, CD8, and LAG3 (Panel 2), the ROI is an ROI according to Table 3, and the feature is selected from the group consisting of the features in the right column of Table 4.In one embodiment, the multiple affinity histochemical stain comprises panel 2, the ROI is a ROI according to Table 3, and the feature comprises at least one feature in the right column of Table 4 that was determined by ReliefF and / or Random Forest to be important for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiple affinity histochemical stain comprises panel 2, the ROI is a ROI according to Table 3, and the feature comprises at least one feature in the right column of Table 4 that was determined by ReliefF and / or Random Forest to be one of the top 10 most important features for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiplex affinity histochemical staining comprises Panel 2, the ROI is an ROI according to Table 3, and the features include at least 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 features in the right column of Table 4 that were determined by ReliefF and / or Random Forest to be among the top 10 most important features for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiplex affinity histochemical staining comprises Panel 2, the ROI is an ROI according to Table 3, and the features comprise at least one feature selected from the group consisting of: maximum number of CD8+ / PD-1 low intensity cells within 20 μm of PD-L1+ cells in epithelial tumors, mean number of PD-1 low intensity CD8+ cells within a 20 μm radius of PD-L1+ cells, maximum value of Lag3 intensity in CD8+Lag3+ cells, mean number of PD-1+ cells within a 20 μm radius of PD-L1+ cells, and maximum value of Lag3+ intensity on CD8+ cells. In one embodiment, the multiplex affinity histochemical staining comprises Panel 2, the ROI is an ROI according to Table 3, and the features comprise each of the maximum number of CD8+ / PD-1 low intensity cells within 20 μm of PD-L1+ cells in epithelial tumors, the mean PD-1 low intensity CD8+ cells within a 20 μm radius of PD-L1 cells, the maximum value of Lag3 intensity in CD8+Lag3+ cells, the mean number of PD-1+ cells within a 20 μm radius of PD-L1+ cells, and the maximum value of Lag3+ intensity on CD8+ cells.In one embodiment, the scoring function is obtained from a modeling function selected from the group consisting of quadratic discriminant analysis (QDA), linear discriminant analysis (LDA), support vector machine (SVM), and artificial neural network (ANN). In one embodiment, the PD-1 axis-directed therapy is a PD-1-specific monoclonal antibody or a PD-L1-specific monoclonal antibody. In one embodiment, the PD-1 axis-directed therapy is selected from the group consisting of pembrolizumab, nivolumab, atezolizumab, avelumab, durvalumab, cemiplimab, tislelizumab, and LY3300054.

[0012] In one embodiment, a method for selecting a patient to receive a PD-1 axis-directed therapy is provided, the method comprising: (a) obtaining a digital image of a tumor section from a tumor sample, wherein the tumor section is stained with a multiplex affinity histochemical stain for each of one or more epithelial markers, one or more immune cell markers, and one or more PD-1 axis pathway markers; (b) identifying a region of interest (ROI) within the digital image; (c) extracting one or more features from the ROI related to cells stained for each biomarker; (d) applying a scoring function to a feature vector including the extracted feature(s) of (c) to generate a score, wherein the score indicates a likelihood that the tumor will respond to the PD-1 axis-directed therapy; (e) comparing the score to a predetermined cutoff value; and (f) selecting a patient to receive the PD-1 axis therapy or an alternative therapy based on the comparison of (e). In one embodiment, the ROI is derived from a digital image of a morphologically stained section of the tumor sample, and the morphologically stained section and the multi-affinity histochemically stained sample are serial sections. In one embodiment, the ROI is identified by a user in the digital image of the morphologically stained section and automatically registered to the digital image of the multi-affinity histochemically stained section. In one embodiment, the multi-affinity histochemical staining includes histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), the ROI is an ROI according to Table 3, and the feature includes at least one feature selected from the group consisting of the features in the left column of Table 4. In one embodiment, the multiplex affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), the ROI is an ROI according to Table 3, and the feature comprises at least one feature determined by ReliefF and / or Random Forest to be important in predicting patient response to PD-1 axis-directed therapy.In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), wherein the ROI is according to Table 3 and the feature comprises at least one feature determined by ReliefF and / or Random Forest to be one of the top 10 features most important for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), wherein the ROI is according to Table 3 and the feature comprises at least 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 of the top 10 features most important for predicting patient response to PD-1 axis-directed therapy, according to ReliefF and / or Random Forest. In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), wherein the ROI is an ROI according to Table 3, and the features comprise at least one feature selected from the group consisting of: the percentage of PD-L1+ macrophages in the stroma, the percentage of PD-L1+CD3+CD8- cells in the stroma, and the percentage of PD-L1+CD3+ cells in the stroma. In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), wherein the ROI is an ROI according to Table 3, and the features comprise each of the percentage of PD-L1+ macrophages in the stroma, the percentage of PD-L1+CD3+CD8- cells in the stroma, and the percentage of PD-L1+CD3+ cells in the stroma. In one embodiment, the multiplex affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PanCK, PD-L1, PD1, CD8, and LAG3 (Panel 2), the ROI is an ROI according to Table 3, and the feature is selected from the group consisting of the features in the right column of Table 4.In one embodiment, the multiple affinity histochemical stain comprises panel 2, the ROI is a ROI according to Table 3, and the feature comprises at least one feature in the right column of Table 4 that was determined by ReliefF and / or Random Forest to be important for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiple affinity histochemical stain comprises panel 2, the ROI is a ROI according to Table 3, and the feature comprises at least one feature in the right column of Table 4 that was determined by ReliefF and / or Random Forest to be one of the top 10 most important features for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiplex affinity histochemical staining comprises Panel 2, the ROI is an ROI according to Table 3, and the features include at least 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 features in the right column of Table 4 that were determined by ReliefF and / or Random Forest to be among the top 10 most important features for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiplex affinity histochemical staining comprises Panel 2, the ROI is an ROI according to Table 3, and the features comprise at least one feature selected from the group consisting of: maximum number of CD8+ / PD-1 low intensity cells within 20 μm of PD-L1+ cells in epithelial tumors, mean number of PD-1 low intensity CD8+ cells within a 20 μm radius of PD-L1+ cells, maximum value of Lag3 intensity in CD8+Lag3+ cells, mean number of PD-1+ cells within a 20 μm radius of PD-L1+ cells, and maximum value of Lag3+ intensity on CD8+ cells. In one embodiment, the multiplex affinity histochemical staining comprises Panel 2, the ROI is an ROI according to Table 3, and the features comprise each of the maximum number of CD8+ / PD-1 low intensity cells within 20 μm of PD-L1+ cells in epithelial tumors, the mean number of PD-1 low intensity CD8+ cells within a 20 μm radius of PD-L1 cells, the maximum value of Lag3 intensity in CD8+Lag3+ cells, the mean number of PD-1+ cells within a 20 μm radius of PD-L1+ cells, and the maximum value of Lag3+ intensity on CD8+ cells.In one embodiment, the scoring function is obtained from a modeling function selected from the group consisting of quadratic discriminant analysis (QDA), linear discriminant analysis (LDA), support vector machine (SVM), and artificial neural network (ANN). In one embodiment, the PD-1 axis-directed therapy is a PD-1-specific monoclonal antibody or a PD-L1-specific monoclonal antibody. In one embodiment, the PD-1 axis-directed therapy is selected from the group consisting of pembrolizumab, nivolumab, atezolizumab, avelumab, durvalumab, cemiplimab, tislelizumab, and LY3300054.

[0013] In one embodiment, a method of treating a patient suffering from a tumor is provided, the method comprising: (a) obtaining a digital image of a tumor section from a tumor sample, wherein the tumor section is stained in a multiplex affinity histochemical stain for each of one or more epithelial markers, one or more immune cell markers, and one or more PD-1 axis pathway markers; (b) identifying a region of interest (ROI) within the digital image; (c) extracting one or more features from the ROI for cells stained for each biomarker; (d) scoring the ROI; (c) fitting a matching function to a feature vector including the extracted feature(s) to generate a score, the score indicating the likelihood of the tumor responding to the PD-1 axis-directed therapy; (e) comparing the score to a predetermined cutoff value; and (f) administering a PD-1 axis-directed therapy to the patient if the score indicates the patient is likely to respond to the PD-1 axis-directed therapy, or administering a course of treatment to the patient that does not include a PD-1 axis-directed therapy if the comparison of (e) indicates the patient is unlikely to respond to both PD-1 axis-directed therapy. In one embodiment, the ROI is derived from a digital image of a morphologically stained section of the tumor sample, and the morphologically stained section and the multi-affinity histochemically stained sample are serial sections. In one embodiment, the ROI is identified by a user in the digital image of the morphologically stained section and automatically registered to the digital image of the multi-affinity histochemically stained section. In one embodiment, the multiplex affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), wherein the ROI is an ROI according to Table 3, and the feature comprises at least one feature selected from the group consisting of the features in the left column of Table 4.In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), wherein the ROI is an ROI according to Table 3 and the feature comprises at least one feature determined by ReliefF and / or Random Forest to be important for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), wherein the ROI is an ROI according to Table 3 and the feature comprises at least one feature determined by ReliefF and / or Random Forest to be one of the top 10 features most important for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiplex affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), the ROI is an ROI according to Table 3, and the features comprise at least 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 of the top 10 features important for predicting patient response to PD-1 axis-directed therapy by ReliefF and / or Random Forest. In one embodiment, the multiplex affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), the ROI is an ROI according to Table 3, and the features comprise at least one feature selected from the group consisting of: percentage of PD-L1+ macrophages in the stroma, percentage of PD-L1+CD3+CD8- cells in the stroma, and percentage of PD-L1+CD3+ cells in the stroma.In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PD-L1, CD8, CD3, CD68, and PanCK (Panel 1), the ROI is an ROI according to Table 3, and the features comprise each of the percentage of PD-L1+ macrophages in the stroma, the percentage of PD-L1+CD3+CD8- cells in the stroma, and the percentage of PD-L1+CD3+ cells in the stroma. In one embodiment, the multiple affinity histochemical staining comprises histochemical staining using biomarker-specific reagents for each of PanCK, PD-L1, PD1, CD8, and LAG3 (Panel 2), the ROI is an ROI according to Table 3, and the features are selected from the group consisting of the features in the right column of Table 4. In one embodiment, the multiple affinity histochemical stain comprises panel 2, the ROI is a ROI according to Table 3, and the feature comprises at least one feature in the right column of Table 4 that was determined by ReliefF and / or Random Forest to be important for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiple affinity histochemical stain comprises panel 2, the ROI is a ROI according to Table 3, and the feature comprises at least one feature in the right column of Table 4 that was determined by ReliefF and / or Random Forest to be one of the top 10 most important features for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the multiplex affinity histochemical staining comprises Panel 2, the ROI is an ROI according to Table 3, and the features include at least 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 features in the right column of Table 4 that were determined by ReliefF and / or Random Forest to be among the top 10 most important features for predicting patient response to PD-1 axis-directed therapy.In one embodiment, the multiplex affinity histochemical staining comprises Panel 2, the ROI is an ROI according to Table 3, and the features comprise at least one feature selected from the group consisting of: maximum number of CD8+ / PD-1 low intensity cells within 20 μm of PD-L1+ cells in epithelial tumors, mean number of PD-1 low intensity CD8+ cells within a 20 μm radius of PD-L1+ cells, maximum value of Lag3 intensity in CD8+Lag3+ cells, mean number of PD-1+ cells within a 20 μm radius of PD-L1+ cells, and maximum value of Lag3+ intensity on CD8+ cells. In one embodiment, the multiplex affinity histochemical staining comprises Panel 2, the ROI is an ROI according to Table 3, and the features comprise the maximum number of CD8+ / PD-1 low intensity cells within 20 μm of PD-L1+ cells in epithelial tumors, the mean number of PD-1 low intensity CD8+ cells within a 20 μm radius of PD-L1 cells, the maximum value of Lag3 intensity in CD8+Lag3+ cells, the mean number of PD-1+ cells within a 20 μm radius of PD-L1+ cells, and the maximum value of Lag3+ intensity on CD8+ cells. In one embodiment, the scoring function is obtained from a modeling function selected from the group consisting of quadratic discriminant analysis (QDA), linear discriminant analysis (LDA), support vector machine (SVM), and artificial neural network (ANN). In one embodiment, the PD-1 axis-directed therapy is a PD-1-specific monoclonal antibody or a PD-L1-specific monoclonal antibody. In one embodiment, the PD-1 axis-directed therapy is selected from the group consisting of pembrolizumab, nivolumab, atezolizumab, avelumab, durvalumab, cemiplimab, tislelizumab, and LY3300054.

[0014] In one embodiment, a method is provided that includes: (a) annotating a region of interest (ROI) on a digital image of a tumor specimen, the digital image being a digital image of a sample multi-affinity stained for PD-L1, CD8, CD3, CD68, and PanCK (Panel 1); (b) extracting one or more features from the ROI; and (c) fitting a scoring function to a feature vector including the feature(s) of (b), wherein the output of the scoring function is a value predictive of patient response to PD-1 axis-directed therapy. In one embodiment, the one or more features are determined to be important for predicting patient response to PD-1 axis-directed therapy by ReliefF and / or Random Forest. In one embodiment, the feature is determined to be one of the top 10 features most important for predicting patient response to PD-1 axis-directed therapy by ReliefF and / or Random Forest. In one embodiment, the at least one feature is selected from the group consisting of: the percentage of PD-L1+ macrophages in the stroma, the percentage of PD-L1+CD3+CD8- cells in the stroma, and the percentage of PD-L1+CD3+ cells in the stroma. In one embodiment, the feature vector comprises each of the percentage of PD-L1+ macrophages in the stroma, the percentage of PD-L1+CD3+CD8- cells in the stroma, and the percentage of PD-L1+CD3+ cells in the stroma. In one embodiment, the feature vector comprises at least 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 of the top 10 features identified by ReliefF and / or Random Forest as most important for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the scoring function is obtained by fitting a quadratic discriminant classification model to the selected features to predict response to treatment. In one embodiment, the treatment outcomes used to fit the quadratic discriminant classification model are grouped together in a configuration selected from the group consisting of PD vs. SD vs. PR+CR, PD vs. SD+PR+CR, and PD+SD vs. PR+CR.In one embodiment, the ROI is identified in a digital image of a first serial section of the test sample, the first serial section stained with hematoxylin and eosin, and the ROI is automatically registered to a digital image of at least a second serial section of the test sample, the second serial section stained with panel 1. In one embodiment, the method is implemented on a computer. In one embodiment, the PD-1 axis-directed therapy is a PD-1-specific monoclonal antibody or a PD-L1-specific monoclonal antibody. In one embodiment, the PD-1 axis-directed therapy is selected from the group consisting of pembrolizumab, nivolumab, atezolizumab, avelumab, durvalumab, cemiplimab, tislelizumab, and LY3300054.

[0015] In one embodiment, a method is provided that includes: (a) annotating a region of interest (ROI) on a digital image of a tumor specimen, the digital image being a digital image of a sample multi-affinity stained for PanCK, PD-L1, PD1, CD8, and LAG3 (Panel 2); (b) extracting one or more features from the ROI; and (c) fitting a scoring function to a feature vector including the feature(s) of (b), wherein the output of the scoring function is a value predictive of patient response to PD-1 axis-directed therapy. In one embodiment, the one or more features are determined by ReliefF and / or Random Forest to be important for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the one or more features are determined by ReliefF and / or Random Forest to be one of the top 10 features most important for predicting patient response to PD-1 axis-directed therapy. In one embodiment, the feature vector comprises at least one feature selected from the group consisting of "CD8+ / PD-1 low intensity cells within 20 μm of PD-L1+ cells in epithelial tumors," the mean number of PD-1 low intensity CD8+ cells within a 20 μm radius of PD-L1+ cells, the maximum value of Lag3 intensity in CD8+Lag3+ cells, the mean number of PD-1+ cells within a 20 μm radius of PD-L1+ cells, and the maximum value of Lag3+ intensity on CD8+ cells. In one embodiment, the feature vector includes each of the maximum numbers of CD8+ / PD-1 low intensity cells within 20 μm of a PD-L1+ cell in the epithelial tumor, and optionally further includes at least one or more additional features selected from the group consisting of the mean number of PD-1 low intensity CD8+ cells within a 20 μm radius of a PD-L1+ cell, the maximum value of Lag3 intensity on CD8+Lag3+ cells, the mean number of PD-1+ cells within a 20 μm radius of a PD-L1+ cell, and the maximum value of Lag3+ intensity on CD8+ cells.In one embodiment, the feature vector includes at least 1, 2, 3, 4, 5, 6, 7, 8, 9, or 10 features determined by ReliefF and / or Random Forest to be one of the top 10 features most important for predicting a patient's response to PD-1 axis-directed therapy. In one embodiment, the scoring function is obtained by fitting a quadratic discriminant classification model to the selected features to predict the response to treatment. In one embodiment, the treatment outcomes used to fit the quadratic discriminant classification model are grouped together in a configuration selected from the group consisting of PD vs. SD vs. PR+CR, PD vs. SD+PR+CR, and PD+SD vs. PR+CR. In one embodiment, the ROI is identified in a digital image of a first serial section of the test sample, the first serial section being stained with hematoxylin and eosin, and the ROI is automatically registered to a digital image of at least a second serial section of the test sample, the second serial section being stained with panel 2. In one embodiment, the PD-1 axis-directed therapy is a PD-1-specific monoclonal antibody or a PD-L1-specific monoclonal antibody. In one embodiment, the PD-1 axis-directed therapy is selected from the group consisting of pembrolizumab, nivolumab, atezolizumab, avelumab, durvalumab, cemiplimab, tislelizumab, and LY3300054.

[0016] In one embodiment, a system for predicting a patient's response to a PD-1 axis therapy is provided, the system comprising: a processor; and a memory coupled to the processor, the memory for storing computer-executable instructions that, when executed by the processor, cause the processor to perform operations comprising one or more of the methods for predicting a patient's response to a PD-1-directed therapy disclosed herein. In one embodiment, the system further comprises a scanner or microscope adapted to capture digital images of the tissue sample section and communicate the images to the computing device. In one embodiment, the system further comprises an automated slide stainer programmed to histochemically stain the tissue sample section with panel 1 or panel 2. In one embodiment, the system further comprises an automated hematoxylin and eosin stainer programmed to stain one or more serial sections of the section stained by the automated slide stainer. In one embodiment, the system further comprises a laboratory information system (LIS) for tracking sample and image workflow and diagnostic information, the LIS comprising a central database configured to receive and store information about tissue samples, the information including at least one of the following: processing steps performed on the tumor tissue sample; processing steps performed on digital images of sections of the tumor tissue sample; processing history of the tumor tissue sample and digital images; and one or more clinical variables (such as MMR status or MSI status) related to the likelihood that a patient will respond to therapy. In one embodiment, the PD-1 axis-directed therapy is a PD-1-specific monoclonal antibody or a PD-L1-specific monoclonal antibody. In one embodiment, the PD-1 axis-directed therapy is selected from the group consisting of pembrolizumab, nivolumab, atezolizumab, avelumab, durvalumab, cemiplimab, tislelizumab, and LY3300054.

[0017] In one embodiment, a non-transitory computer-readable storage medium is provided for storing computer-executable instructions for execution by a processor to perform operations, the operations including one or more of the methods for predicting a patient's response to a PD-1-directed therapy disclosed herein. In one embodiment, the PD-1 axis-directed therapy is a PD-1-specific monoclonal antibody or a PD-L1-specific monoclonal antibody. In one embodiment, the PD-1 axis-directed therapy is selected from the group consisting of pembrolizumab, nivolumab, atezolizumab, avelumab, durvalumab, cemiplimab, tislelizumab, and LY3300054.

[0018] The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the U.S. Patent and Trademark Office upon request and payment of the necessary fee. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a flow diagram illustrating an exemplary approach for deriving a scoring function as disclosed herein. [Figure 2] 1 is a flow diagram illustrating an exemplary approach for scoring a test sample using the scoring functions described herein. [Figure 3] 1 illustrates an exemplary scoring system that integrates the scoring functions described herein. [Figure 4A] 1 shows an exemplary flow diagram implemented on the image analysis system disclosed herein, in which an object identification function is performed on the entire image before an ROI generation program function is performed. [Figure 4B] 1 shows an exemplary flow diagram implemented on the image analysis system disclosed herein, where the object identification function is performed only on the ROI after the ROI generator function is performed. [Figure 5] 1 shows the multiple staining process using panel 1 as described in Example 1. [Figure 6] Panel 1 shows an exemplary slide stained with. [Figure 7] Panel 1 is used to show the severity ranking of all features in all cases stained. [Figure 8] Feature severity ranking for MMR-deficient cases stained using panel 1 is shown. [Figure 9] Immunohistochemical images of samples stained with Panel 1 from patients who demonstrated a complete response to PD-1 axis-directed therapy treatment. [Figure 10] 1 is an immunohistochemistry image of a sample stained with panel 1 from a patient who developed progressive disease after treatment with a PD-1 axis-directed therapy treatment. [Figure 11] 1 shows the staining and image analysis process used in Example II for panel 2. [Figure 12]The top 15 features of panel 2 were ranked using the ReliefF feature selection function. The features were: (1) the maximum number of PD-1 low intensity / CD8+ cells within a 20 μm radius of PD-L1+ / CD8+ cells, (2) the maximum PD-L1 intensity from PD-L1+ / CD8+ cells, (3) the ratio of the number of PD-1+ / PD-L1- / Lag3+ / CD8+ cells to CD8+ cells in the panCK- region, (4) the spatial variance of the number of PD-1 low intensity / CD8+ cells within a 20 μm radius of PD-L1+ cells, (5) the maximum PD-L1 intensity from all PD-L1+ cells, (6) the number of Lag3+ cells in the panCK- region, (7) the density of PD-L1+ / panCK- cells in the panCK- region, (8) the density of PD-L1+ / panCK- cells in the panCK- region, and (9) the density of PD-L1+ / panCK- cells in the panCK- region. ) the number of PD-L1+ cells in the panCK+ region, (9) the number of PD-L1+ cells in the panCK- region, (10) the number of PD-1+ cells in the panCK- region, (11) the maximum number of PD-1 low intensity / CD8+ cells within a 20 μm radius of PD-L1+ cells, (12) the average number of PD-1 low intensity / CD8+ cells within a 20 μm radius of PD-L1+ cells, (13) the maximum value of Lag3 intensity from CD8+ / Lag3+ cells, (14) the number of CD8+ cells in the panCK- region, and (15) the variance in the number of PD-1 low intensity / CD8+ cells within a 20 μm radius of PD-L1+ / CD8+ cells. [Figure 13]The top 15 features for panel 2 are ranked using the Random Forest feature selection function. The features are: (1) the average distance from a PD-L1+ cell to its nearest PD-1+ cell, (2) the maximum number of PD-1 low intensity / CD8+ cells within a 20 μm radius of a PD-L1+ / CD8+ cell, (3) the maximum number of PD-1 low intensity / CD8+ cells within a 20 μm radius of a PD-L1+ / panCK+ cell, (4) the average distance from a PD-L1+ cell to its nearest PD-1+ cell, (5) the ratio of the number of Lag3+ / CD8+ cells to CD8+ cells in the panCK- region, (6) the ratio of the number of PD-L1+ / CD8+ cells to CD8+ cells in the panCK- region, (7) the average number of PD-1 low intensity / CD8+ cells within a 10 μm radius of a PD-L1+ cell, ( (8) Variance in the number of PD-1 low intensity / CD8+ cells within a 10 μm radius of PD-L1+ cells, (9) Mean PD-1 intensity from all PD-1+ cells, (10) Minimum Lag3 intensity from all Lag3+ cells, (11) Density of PD-L1+ cells in the panCK- region, (12) Maximum number of PD-1 low intensity / CD8+ cells within a 10 μm radius of PD-L1+ / CD8+ cells, (13) Number of PD-1+ cells in the panCK+ region, (14) Ratio of the number of PD-1+ / PD-L1- / Lag3+ / CD8+ cells to CD8+ cells in the panCK- region, and (15) Maximum Lag3 intensity from Lag3+ / CD8+ cells. [Figure 14A] It contains figures showing the predicted spatial distribution of the number of PD-1+ cells within a 10 μm radius of a PD-L1+ cell (F1), the predicted mean number of PD-1 low intensity CD8+ cells within a 20 μm radius of a PD-L1+ cell (F2), and the predicted maximum value of Lag3 intensity in CD8+Lag3+ cells (F3). [Figure 14B] Scatter plots showing the predicted spatially dispersed number of PD-1+ cells within a 10 μm radius of PD-L1+ cells (F1) and the mean number of PD-1 low intensity CD8+ cells within a 20 μm radius of PD-L1+ cells (F2). [Figure 14C]Scatter plots showing the predicted spatial distribution of the number of PD-1+ cells within a 10 μm radius of a PD-L1+ cell (F1), the average number of PD-1 low intensity CD8+ cells within a 20 μm radius of a PD-L1+ cell (F2), and the maximum value of Lag3 intensity among CD8+Lag3+ cells (F3). [Figure 15] Exemplary immunohistochemical images of non-responders and responders are shown, along with graphical reconstructions. The locations of PD-L1+ cells (gray dots), PD-1-low cells within 20 μm of PD-L1+ cells (white dots), and PD-1+ cells within 10 μm of PD-L1+ cells (black dots) are shown. Frame (a) is a raw fluorescent image of a non-responder. Frame (b) is a graphical reconstruction of the white boxes from frame (a), showing the spatial relationship between PD-L1+ cells and PD-1-low cells within 20 μm of PD-L1+ cells in a non-responder. Frame (c) is a graphical reconstruction of the gray boxes from frame (a), showing the spatial relationship between PD-L1+ cells and PD-1+ cells within 10 μm of PD-L1+ cells in a non-responder. Frame (d) is a raw fluorescent image of a responder. Frame (e) is a graphical reconstruction of the white boxes from frame (d), showing the spatial relationship between PD-L1+ cells and PD-1-low cells within 20 μm of PD-L1+ cells in responders. Frame (f) is a graphical reconstruction of the gray boxes from frame (d), showing the spatial relationship between PD-L1+ cells and PD-1+ cells within 10 μm of PD-L1+ cells in responders. [Figure 16A] Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the ratio of the number of Lag3+ / CD8+ cells to total CD8+ cells in the panCK-negative area. [Figure 16B] Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the ratio of Lag3- / CD8+ cells to CD8+ cells in the panCK-negative region. [Figure 16C]Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the number of Lag3+ / panCK- cells divided by the panCK-negative area. [Figure 16D] Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the number of Lag3-positive cells in panCK-negative areas. [Figure 16E] Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on maximum Lag3 intensity in CD8+ cells. [Figure 16F] Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the number of Lag3+ cells in panCK-positive areas. [Figure 16G] Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the mean number of PD-1 / CD8+ cells within a 10 μm radius of PD-L1+ / panCK+ cells. [Figure 16H] Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the mean number of PD-1 / CD8+ cells within a 20 μm radius of PD-L1+ / panCK+ cells. [Figure 16I] Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the variance in the number of PD-1+ / CD8+ cells within a 20 μm radius of PD-L1+ / CD8+ cells. [Figure 16J] Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the variance in the number of PD-1+ / CD8+ cells within a 20 μm radius of PD-L1+ / panCK+ cells. [Figure 16K] Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the variance in the number of PD-1 low / CD8+ cells within a 10 μm radius of PD-L1+ cells. [Figure 16L]Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the maximum number of PD-1+ / CD8+ cells within a 20 μm radius of a PD-L1+ / CD8+ cell. [Figure 16M] Kaplan-Meier survival curves for prediction of overall survival after pembrolizumab treatment based on the maximum number of PD-1 / CD8+ cells within a 20-μm radius of PD-L1+ cells for each feature. The cohort was divided into two groups using the median of the feature distribution as the cutoff. DETAILED DESCRIPTION OF THE INVENTION

[0020] I. Definition Unless otherwise defined, the technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art.For example, see Lackie, DICTIONARY OF CELL AND MOLECULAR BIOLOGY, Elsevier (4th ed.2007); Sambrook et al., MOLECULAR CLONING, A LABORATORY MANUAL, Cold Springs Harbor Press (Cold Springs Harbor, NY1989).Term "a" or "an" is intended to mean "one or more".The terms "comprise", "comprises" and "comprises", when preceding a list of steps or elements, are intended to mean that the addition of further steps or elements is optional and not excluded.

[0021] Antibody: As used herein, the term "antibody" is used in the broadest sense and encompasses a variety of antibody structures, including, but not limited to, monoclonal antibodies, polyclonal antibodies, multispecific antibodies (e.g., bispecific antibodies), and antibody fragments, so long as they exhibit the desired antigen-binding activity.

[0022] Antibody fragment: "Antibody fragment" refers to a molecule other than an intact antibody that contains a portion of an intact antibody that binds the antigen to which the intact antibody binds. Examples of antibody fragments include, but are not limited to, Fv, Fab, Fab', Fab'-SH, F(ab')2; diabodies; linear antibodies; single-chain antibody molecules (e.g., scFv); and multispecific antibodies formed from antibody fragments.

[0023] Biomarker: As used herein, the term "biomarker" refers to any molecule or group of molecules found in a biological sample that can be used to characterize the biological sample or the subject from which the biological sample is obtained. For example, a biomarker can be a molecule or group of molecules whose presence, absence, or relative amount characterizes a particular cell or tissue type or condition, or characterizes a particular pathological condition or state, or indicates the severity of a pathological condition, the likelihood of progression or regression of a pathological condition, and / or the likelihood that a pathological condition will respond to a particular treatment. As another example, a biomarker can be a cell type or microorganism (such as a bacterium, mycobacterium, fungus, virus, etc.), or a substituent molecule or group of molecules thereof.

[0024] Biomarker-specific reagent: A specific detection reagent, such as a primary antibody, that can bind directly and specifically to one or more biomarkers in a cell sample.

[0025] Cell sample: As used herein, the term "cell sample" refers to any sample containing intact cells, such as a cell culture, a body fluid sample, or a surgical specimen taken for pathological, histological, or cytological interpretation.

[0026] Detection Reagent: A "detection reagent" is any reagent used to deposit a stain in proximity to a biomarker-specific reagent in a cell sample. Non-limiting examples include a biomarker-specific reagent (such as a primary antibody), a secondary detection reagent (such as a secondary antibody that can bind to a primary antibody), a tertiary detection reagent (such as a tertiary antibody that can bind to a secondary antibody), an enzyme that directly or indirectly associates with a biomarker-specific reagent, chemicals that react with such enzymes to result in the deposition of a fluorescent or chromogenic stain, wash reagents used between staining steps, etc.

[0027] Detectable moiety: A molecule or material capable of producing a detectable signal (such as visual, electronic, or other) that indicates the presence (i.e., qualitative analysis) and / or concentration (i.e., quantitative analysis) of a detectable moiety deposited on a sample. The detectable signal can be produced by any known or yet to be discovered mechanism, including absorption, emission, and / or scattering of photons (including photons of radio, microwave, infrared, visible, and ultraviolet frequencies). The term "detectable moiety" includes chromogenic, fluorescent, phosphorescent, and luminescent molecules and materials, catalysts (such as enzymes) that convert one substance to another to provide a detectable difference (by converting a colorless substance to a colored substance or vice versa, or by producing a precipitate or increasing the turbidity of the sample). In some examples, the detectable moiety is a fluorophore, belonging to several general chemical classes, including coumarin, fluorescein (or fluorescein derivatives and analogs), rhodamine, resorufin, luminophore, and cyanine. Additional examples of fluorescent molecules can be found in Molecular Probes Handbook - A Guide to Fluorescent Probes and Labeling Technologies, Molecular Probes, Eugene, OR, ThermoFisher Scientific, 11 thIn other embodiments, the detectable moiety is a molecule detectable via bright field microscopy, such as dyes including diaminobenzidine (DAB), 4-(dimethylamino)azobenzene-4'-sulfonamide (DABSYL), tetramethylrhodamine (DISCOVERY Purple), N,N'-biscarboxypentyl-5,5'-disulfonato-indo-dicarbocyanine (Cy5), and rhodamine 110 (rhodamine).

[0028] Feature: A value that indicates the level of a biomarker in a sample or the relationship between biomarkers. Examples include expression intensity (e.g., a 0+, 1+, 2+, 3+ scale), the number of cells positive for a biomarker, cell density (e.g., the number of biomarker-positive cells over the area of ​​the ROI, the number of biomarker-positive cells over the linear distance of the edges defining the ROI, etc.), pixel density (i.e., the number of biomarker-positive pixels over the area of ​​the ROI, the number of biomarker-positive pixels over the linear distance of the edges defining the ROI, etc.), the mean or median distance between cells expressing biomarker(s), etc. A feature can be a total or overall amount.

[0029] Histochemical detection: A process involving labeling a biomarker or other structure in a tissue sample with a biomarker-specific or detection reagent in a manner that allows for microscopic detection of the biomarker or other structure in the context of cross-sectional relationships between structures in the tissue sample. Examples include immunohistochemistry (IHC), chromogenic in situ hybridization (CISH), fluorescence in situ hybridization (FISH), silver in situ hybridization (SISH), and hematoxylin and eosin (H&E) staining of formalin-fixed, paraffin-embedded tissue sections.

[0030] Immune checkpoint molecule: A protein expressed by immune cells whose activation downregulates cytotoxic T cell responses. Examples include PD-1, TIM-3, LAG-4, and CTLA-4.

[0031] Immune escape biomarker: A biomarker expressed by tumor cells that helps the tumor evade a T-cell-mediated immune response. Examples of immune escape biomarkers include PD-L1, PD-L2, and IDO.

[0032] Immunological biomarker: a biomarker that characterizes or affects an immune response to abnormal cells, including, but not limited to, biomarkers expressed by, displayed by, or otherwise located on non-immune cellular structures (such as cell surface-expressed antigens, MHC-ligand complexes, and immune escape biomarkers) that indicate a particular class of immune cell (such as CD3), characterize an immune response (such as the presence, absence, or amount of a cytokine protein or a particular immune cell subtype(s)), or affect the type or extent of an immune cell response.

[0033] Monoclonal antibody: An antibody obtained from a population of substantially homogeneous antibodies, i.e., each individual antibody comprising the population is identical and / or binds the same epitope, excluding variant antibodies that contain, for example, naturally occurring mutations or that may arise during the production of a monoclonal antibody preparation, and such variants are generally present in minor amounts. In contrast to polyclonal antibody preparations, which typically include different antibodies directed against different determinants (epitopes), each monoclonal antibody of a monoclonal antibody preparation is directed against a single determinant on an antigen. Thus, the modifier "monoclonal" indicates the character of the antibody as being obtained from a substantially homogeneous population of antibodies and is not to be construed as requiring production of the antibody by any particular method. For example, monoclonal antibodies used in accordance with the present invention can be produced by a variety of techniques, including, but not limited to, hybridoma technology, recombinant DNA technology, phage display technology, and methods utilizing transgenic animals containing all or part of the human immunoglobulin loci, or a combination thereof.

[0034] Multiplex histochemical staining: A histochemical staining technique in which multiple biomarker-specific reagents that bind to different biomarkers are applied to a single section and stained with different colored stains.

[0035] PD-1 axis-directed therapy: A therapeutic agent that disrupts the ability of PD-1 to downregulate cellular activity. Exemplary PD-1 axis-directed therapies include PD-1-specific monoclonal antibodies (e.g., pembrolizumab, nivolumab, cemiplimab, and tislelizumab), PD-L1-specific monoclonal antibodies (e.g., atezolizumab, avelumab, durvalumab, and LY3300054), and PD-1 small molecule inhibitors (e.g., CA-170, others in preclinical development reviewed by Li & Tian).

[0036] Sample: As used herein, the term "sample" is intended to refer to any material obtained from a subject that can be tested for the presence or absence of a biomarker.

[0037] Secondary detection reagent: A specific detection reagent capable of specifically binding to a biomarker-specific reagent.

[0038] Section: as a noun, a thin section of a tissue sample suitable for microscopic analysis, usually cut using a microtome. As a verb, the process of producing a section.

[0039] Serial section: As used herein, the term "serial section" refers to any one of a series of sections cut successively by a microtome from a tissue sample. Two sections do not necessarily have to be consecutive sections from the same tissue to be considered "serial sections" of each other, but they should generally contain sufficiently similar tissue structures in the same spatial relationship so that the structures can be matched to each other after histological staining.

[0040] Single histochemical staining: A histochemical staining technique in which a single biomarker-specific reagent is applied to a single section and stained with a single color stain.

[0041] Specific detection reagent: Any composition of matter capable of specifically binding to a target chemical structure in the context of a cell sample. As used herein, the phrases "specific binding," "specifically binds to," or "specific for," or other similar phrases, refer to a measurable and reproducible interaction between a target and a specific detection reagent that determines the presence of the target in the presence of a heterogeneous population of molecules, including biological molecules. For example, an antibody that specifically binds to a target is an antibody that binds this target more readily, with greater affinity, avidity, and / or with longer duration than it binds to other targets. In one embodiment, the extent to which a specific detection reagent binds to an unrelated target is less than about 10% of the binding of the antibody to the target, as measured, for example, by radioimmunoassay (RIA). In certain embodiments, an antibody that specifically binds to a target has a dissociation constant (Kd) of 1 μM or less, 100 nM or less, 10 nM or less, 1 nM or less, or 0.1 nM or less. In another embodiment, specific binding can include, but does not require, exclusive binding.Exemplary specific detection reagents include nucleic acid probes specific for particular nucleotide sequences; antibodies and antigen-binding fragments thereof; and antibodies based on ADNECTIN (10th generation FN3 fibronectin-based scaffold, Bristol-Myers-Squibb Co.), AFFIBODY (scaffold based on the Z domain of protein A from S. aureus, Affibody AB, Solna, Sweden), AVIMER (domain A / LDL receptor-based scaffold, Amgen, Thousand Oaks, CA), dAb (VH or VL antibody domain-based scaffold, GlaxoSmithKline PLC, Cambridge, UK), DARPin (ankyrin repeat protein-based scaffold, Molecular Partners AG, Zurich, CH), ANTICALIN (lipocalin-based scaffold, Pieris AG, Freising, DE), NANOBODY (VHH (camel Ig)-based scaffold, Ablynx, NV), and other antibodies. N / V, Ghent, BE), TRANS-BODY (a transferrin-based scaffold, Pfizer Inc., New York, NY), SMIP (Emergent Biosolutions, Inc., Rockville, MD), and TETRANECTIN (a scaffold based on the C-type lectin domain (CTLD) of tetranectin, Borean Pharma A / S, Aarhus, DK). A description of such engineered specific binding structures is reviewed by Wurch et al., "Development of Novel Protein Scaffolds as Alternatives to Whole Antibodies for Imaging and Therapy: Status on Discovery Research and Clinical Validation," Current Pharmaceutical Biotechnology, Vol. 9, pp. 502-509 (2008), the contents of which are incorporated by reference.

[0042] Stain or Staining: When used as a noun, the term "stain" shall refer to any substance that can be used to visualize specific molecules or structures in a cell sample for microscopic analysis, including bright field microscopy, fluorescence microscopy, electron microscopy, etc. When used as a verb, the term "staining" shall refer to any process that results in the deposition of a staining agent onto a cell sample.

[0043] Subject: As used herein, the term "subject" or "individual" refers to a mammal. Mammals include, but are not limited to, domesticated animals (e.g., cows, sheep, cats, dogs, and horses), primates (e.g., humans and non-human primates such as monkeys), rabbits, and rodents (e.g., mice and rats). In certain embodiments, the individual or subject is a human.

[0044] Test Sample: A tumor sample obtained from a subject whose outcome is unknown at the time the sample is obtained.

[0045] Tissue sample: As used herein, the term "tissue sample" is intended to refer to a sample of cells that preserves the cross-sectional spatial relationships between cells that were present within the subject from which the sample was obtained.

[0046] Tumor sample: A tissue sample obtained from a tumor.

[0047] II. Biomarker Description CD3: CD3 is a cell surface receptor complex frequently used as a defining biomarker for cells of the T cell lineage. The CD3 complex is composed of four distinct polypeptide chains: the CD3 gamma chain, the CD3 delta chain, the CD3 epsilon chain, and the CD3 zeta chain. CD3 gamma and CD3 delta form heterodimers with CD3 epsilon (εγ and εδ heterodimers, respectively), whereas CD3 zeta forms a homodimer (ζζ homodimer). Functionally, the εγ, εδ, and ζζ homodimers form signaling complexes with the T cell receptor complex. Exemplary sequences for human CD3 gamma, delta, epsilon, and zeta chains (and isoforms and variants thereof) can be found under UniProt accession numbers P09693 (canonical amino acid sequence disclosed herein as SEQ ID NO: 1), P04234 (canonical amino acid sequence disclosed herein as SEQ ID NO: 2), P07766 (canonical amino acid sequence disclosed herein as SEQ ID NO: 3), and P20963 (canonical amino acid sequence disclosed herein as SEQ ID NO: 4), respectively. As used herein, the term "human CD3 protein biomarker" encompasses any CD3 gamma, delta, epsilon, and zeta chain polypeptides having the canonical human sequence and naturally occurring variants thereof that maintain the function of the canonical sequence. That is, εγ homodimers, εδ heterodimers, and ζζ homodimers comprising one or more of the CD3 gamma chain, CD3 delta chain, CD3 epsilon chain, and CD3 zeta chain polypeptides having the canonical human sequence and naturally occurring variants thereof that maintain the function of the canonical sequence, and any signaling complex comprising one or more of the foregoing CD3 homodimers or heterodimers.In some embodiments, a human CD3 protein biomarker-specific agent includes any biomarker-specific agent that specifically binds a structure (e.g., an epitope) within a CD3 gamma chain polypeptide (such as the polypeptide of SEQ ID NO: 1), a CD3 delta chain polypeptide (such as the polypeptide of SEQ ID NO: 2), a CD3 epsilon chain polypeptide (such as the polypeptide of SEQ ID NO: 3), or a CD3 zeta chain polypeptide (such as the polypeptide of SEQ ID NO: 4), or that binds to a structure (epitope) located within the εγ homodimer, εδ heterodimer, or ζζ homodimer.

[0048] CD8: CD8 is a heterodimeric, disulfide-linked transmembrane glycoprotein found in cytotoxic suppressor T cell subsets, thymocytes, certain natural killer cells, and within a subpopulation of myeloid cells. Exemplary sequences for the human alpha and beta chains of the CD8 receptor (and their isoforms and variants) can be found under Uniprot accession numbers P01732 (the canonical amino acid sequence disclosed herein as SEQ ID NO: 5) and P10966 (the canonical amino acid sequence disclosed herein as SEQ ID NO: 6), respectively. As used herein, the term "human CD8 protein biomarker" encompasses any CD8 alpha chain polypeptide having the canonical human sequence and naturally occurring variants thereof that maintain the function of the canonical sequence; any CD8 beta chain having the canonical human sequence and naturally occurring variants thereof that maintain the function of the canonical sequence; any dimer comprising a CD8 alpha chain polypeptide having the canonical human sequence and naturally occurring variants thereof that maintain the function of the canonical sequence; and / or a CD8 beta chain polypeptide having the canonical human sequence and naturally occurring variants thereof that maintain the function of the canonical sequence. In some embodiments, a human CD8 protein biomarker-specific agent includes any biomarker-specific agent that preferentially binds a structure (e.g., an epitope) within a CD8 alpha chain polypeptide (e.g., the polypeptide of SEQ ID NO: 5), a CD8 beta chain polypeptide (e.g., the polypeptide of SEQ ID NO: 6), or that binds to a structure (epitope) located within the CD8 dimer.

[0049] CD68: CD68 is a glycoprotein encoded by the CD68 gene located on chromosome 17 at position 17p13.1. CD68 protein is found within cytoplasmic granules in a variety of different blood and muscle cells and is frequently used as a biomarker for cells of the macrophage lineage, including monocytes, histiocytes, giant cells, Kupffer cells, and osteoclasts. An exemplary sequence of human CD68 (and its isoforms and variants) can be found in Uniprot accession number P34810 (the canonical amino acid sequence disclosed herein as SEQ ID NO: 7). As used herein, the term "human CD68 protein biomarker" encompasses any CD68 polypeptide having the canonical human sequence and naturally occurring variants thereof that maintain the function of the canonical sequence. In some embodiments, a human CD20 protein biomarker-specific agent encompasses any biomarker-specific agent that specifically binds a structure (epitope, etc.) within a human CD68 polypeptide (e.g., the polypeptide of SEQ ID NO: 7).

[0050] Pan-cytokeratin: As used herein, "pan-cytokeratin" and "PanCK" refer to any biomarker-specific reagent or group of biomarker-specific reagents that specifically bind to multiple cytokeratins sufficient to specifically stain epithelial tissue in a tissue sample. Exemplary pan-cytokeratin biomarker-specific reagents typically include either: (a) a single cytokeratin-specific reagent that recognizes an epitope common to multiple cytokeratins, where most epithelial cells in the tissue express at least one of the cytokeratins; or (b) a cocktail of biomarker-specific reagents that specifically reacts with multiple cytokeratins, where most epithelial cells in the tissue express at least one of the cytokeratins. Reference to a "cocktail" in this definition includes both a single composition containing each member of the plurality, or providing each member of the plurality as a separate composition, but staining the plurality with a single dye or a combination thereof. PanCK cocktails have been reviewed by NordiQC. In some embodiments, the PanCK biomarker-specific reagent comprises an antibody cocktail containing two or more of the antibody clones selected from the group consisting of 5D3, LP34, AE1, AE2, AE3, MNF116, and PCK-26. In one embodiment, the PanCK cocktail is selected from the group consisting of an AE1 and AE3 cocktail, an AE1, AE3, and 5D3 cocktail, and an AE1, AE3, and PCK26 cocktail. The AE1 and AE3 cocktail is commercially available from Agilent Technologies (catalog numbers GA05361-2, IS05330-2, IR05361-2, M351501-2, and M351529-2). The AE1, AE3, and 5D3 cocktail is commercially available from BioCare (catalog numbers CM162, IP162, OAI162, and PM162) and Abcam (catalog number ab86734). A cocktail of AE1, AE3, and PCK26 is available from Roche (catalog number 760-2135).

[0051] PD1: Programmed death 1 (PD-1) is a member of the CD28 family of receptors encoded by the PDCD1 gene on chromosome 2. An exemplary sequence for the human PD-1 protein (and its isoforms and variants) can be found in Uniprot accession number Q15116 (the canonical amino acid sequence disclosed herein in SEQ ID NO: 8). In some embodiments, a human PD-1 protein biomarker-specific agent includes any biomarker-specific agent that specifically binds a structure (such as an epitope) within a human PD-1 polypeptide (such as the polypeptide of SEQ ID NO: 8).

[0052] PD-L1: Programmed death-ligand 1 (PD-L1) is a type 1 transmembrane protein encoded by the CD274 gene on chromosome 9. PD-L1 acts as a ligand for PD-1 and CD80. An exemplary sequence for the human PD-L1 protein (and its isoforms and variants) can be found in Uniprot Accession No. Q9NZQ7 (the canonical amino acid sequence disclosed herein as SEQ ID NO: 9). In some embodiments, a human PD-L1 protein biomarker-specific agent includes any biomarker-specific agent that specifically binds a structure (such as an epitope) within a human PD-L1 polypeptide (such as the polypeptide of SEQ ID NO: 9).

[0053] LAG3: Lymphocyte activation gene 3 protein (LAG3) is a member of the immunoglobulin (Ig) superfamily encoded by the LAG3 gene on human chromosome 12. An exemplary sequence for the human LAG3 protein (and its isoforms and variants) can be found in Uniprot accession number P18627 (the canonical amino acid sequence disclosed herein in SEQ ID NO: 10). In some embodiments, a human LAG3 protein biomarker-specific agent includes any biomarker-specific agent that specifically binds a structure (such as an epitope) within a human LAG3 polypeptide (such as the polypeptide of SEQ ID NO: 10).

[0054] III. Generating the Scoring Function Figure 1 is a flow chart illustrating an exemplary approach for deriving the scoring functions disclosed herein. The scoring functions of the present methods and systems are generally derived from tumor samples derived from a cohort of patients prior to treatment with a PD-1 axis-directed therapy and for which outcome data (e.g., 3- or 5-year overall survival, progression-free survival, recurrence-free survival, progressive disease, stable disease, partial response, complete response, etc.) are available 101. A panel of biomarkers to be tested is selected, and samples from the cohort are stained for biomarkers 102 and imaged (often with morphologically stained serial sections) 103. Regions of interest (ROIs) are identified in the digital image(s) 104, and multiple biomarker features are extracted from the ROI(s) 105. The extracted features are evaluated by a feature selection function 106, which is used to identify features that correlate with response to the PD-1 axis. One or more of the selected features are modeled against the outcome using one or more modeling functions to identify candidate scoring functions, and optionally one or more cutoffs are selected to separate the cohort by score (e.g., into "likely responders" and "unlikely responders," or "likely responders" and "unlikely responders"), for example, by using ROC curves, and Kaplan-Meier curves to compare multiple groups. 107 Scoring function and cutoff combinations that show the desired separation between groups are then selected for inclusion in the scoring systems and methods described herein.

[0055] III.A. Samples and Sample Preparation for Generating Scoring Functions The scoring function is typically modeled on tissue sections obtained from a cohort of subjects afflicted with tumors and with known responses to PD-1 axis-directed therapy. 101 In some embodiments, the tumor is a solid tumor, such as a carcinoma, lymphoma, or sarcoma. In one embodiment, the tumor is a tumor of the skin, breast, head and / or neck, lung, upper gastrointestinal tract (including esophagus and stomach), female reproductive system (including tumors of the uterus, fallopian tube, and ovary), lower gastrointestinal tract (including tumors of the colon, rectum, and anus), renal-urogenital, exocrine, endocrine, renal, neurological, or lymphoid origin. In one embodiment, the subject is afflicted with melanoma, breast cancer, ovarian cancer, pancreatic cancer, head and neck cancer, lung cancer, esophageal cancer, gastric cancer, colorectal cancer (including cancer of the colon, rectum, and anus), prostate, urothelial carcinoma, or lymphoma. In specific embodiments, the tumor is non-small cell lung cancer, squamous cell carcinoma of the head and neck, Hodgkin's lymphoma, urothelial carcinoma, gastric cancer, renal cell carcinoma, hepatocellular carcinoma, or colon cancer.

[0056] The obtained samples 101 are typically tissue samples that have been processed in a manner compatible with histochemical staining, including, for example, fixation, embedding in a wax matrix (such as paraffin), and sectioning (such as with a microtome). No specific processing steps are required by the present disclosure, so long as the obtained samples are compatible with histochemical staining of the sample for the biomarkers of interest and generating digital images of the stained sample. In a specific embodiment, the scoring function is modeled using microtome sections of formalin-fixed, paraffin-embedded (FFPE) samples. Additionally, for purposes of generating the scoring function, the samples in cohort 101 are intended to be samples with a known outcome, such as disease recurrence, disease progression, death from disease, overall death, progressive disease, stable disease, partial response, and / or complete response.

[0057] III.B. Biomarker Panels When generating a scoring function, at least one section of the sample is stained with a panel of biomarker-specific reagents 102. The panel typically includes at least one epithelial marker-specific reagent (such as a Pan-CK-specific reagent), at least one immune cell-specific reagent (such as a CD3-, CD8-, and / or CD68-specific reagent), and at least one PD-1 axis biomarker-specific reagent (such as a PD-1-, PD-L1-, and / or PD-L2-specific reagent). In some embodiments, the panel can further include one or more additional immune checkpoint biomarker-specific reagents, such as a LAG3-specific reagent. In one embodiment, the biomarker-specific reagent panel is selected from the group consisting of Panel 1, which includes CD8, epithelial marker (EM), CD68, CD3, and PD-L1, and Panel 2, which includes CD8, epithelial marker (EM), PD-L1, PD-1, and LAG3. Examples of epithelial markers useful in Panels 1 and 2 include cytokeratins. In one embodiment, the epithelial markers are a set of cytokeratins stained by PanCK biomarker-specific reagents.

[0058] A panel of biomarker-specific reagents is used in conjunction with a set of appropriate detection reagents to produce biomarker-stained sections. Biomarker staining is typically achieved by contacting a sample section with the biomarker-specific reagents under conditions that promote specific binding between the biomarker and the biomarker-specific reagent. The sample is then contacted with a set of detection reagents, which interact with the biomarker-specific reagents to facilitate deposition of a detectable moiety in the vicinity of the biomarker, thereby generating a detectable signal localized to the biomarker. Washing steps are typically performed between the application of different reagents to prevent undesired nonspecific staining of the tissue. Biomarker-stained sections can optionally be additionally stained with a contrast agent (such as hematoxylin stain) to visualize macromolecular structures. In addition, serial sections of biomarker-stained sections may be stained with a morphological stain to facilitate identification of ROIs.

[0059] III.C.1. Labeling Schemes and Related Reagents The biomarker-specific reagent facilitates detection of the biomarker by mediating the deposition of a detectable moiety in the vicinity of the biomarker-specific reagent.

[0060] In some cases, the detectable moiety is directly attached to the biomarker-specific reagent and is therefore deposited on the sample when the biomarker-specific reagent binds to its target (generally referred to as direct labeling). Direct labeling often allows for more direct quantification, but often suffers from a lack of sensitivity. In other embodiments, deposition of the detectable moiety is brought about by the use of a detection reagent associated with the biomarker-specific reagent (generally referred to as indirect labeling). Indirect labeling is often more sensitive than direct labeling, especially when used in conjunction with dyes, because it increases the number of detectable moieties that can be deposited in proximity to the biomarker-specific reagent.

[0061] In some embodiments, an indirect method is used in which a detectable moiety is deposited via an enzymatic reaction localized on the biomarker-specific reagent. Enzymes suitable for such reactions are well known and include, but are not limited to, oxidoreductases, hydrolases, and peroxidases. Specific enzymes explicitly included are horseradish peroxidase (HRP), alkaline phosphatase (AP), acid phosphatase, glucose oxidase, β-galactosidase, β-glucuronidase, and β-lactamase. The enzyme can be directly bound to the biomarker-specific reagent or can be indirectly bound to the biomarker-specific reagent via a labeled conjugate. As used herein, "labeled conjugate" includes the following: (a) a specific detection reagent, and (b) An enzyme linked to a specific detection reagent, which reacts with a chromogenic substrate, signaling conjugate, or enzyme-reactive dye under suitable reaction conditions to result in the in situ generation and / or deposition of the dye on the tissue sample. In non-limiting examples, the specific detection reagent of the labeled conjugate can be a secondary detection reagent (such as a species-specific secondary antibody bound to a primary antibody, an anti-hapten antibody bound to a hapten-conjugated primary antibody, or a biotin-binding protein bound to a biotinylated primary antibody), a tertiary detection reagent (such as a species-specific tertiary antibody bound to a secondary antibody, an anti-hapten antibody bound to a hapten-conjugated secondary antibody, or a biotin-binding protein bound to a biotinylated secondary antibody), or other such preparations. The enzyme localized to the biomarker-specific reagent thus bound to the sample can then be used in a number of schemes to deposit a detectable moiety.

[0062] In some cases, the enzyme reacts with a chromogenic compound / substrate, specific non-limiting examples of which include 4-nitrophenylphospate (pNPP), Fast Red, bromochloroindolylphosphate (BCIP), nitroblue tetrazolium (NBT), BCIP / NBT, Fast Red, AP Orange, AP Blue, tetramethylbenzidine (TMB), 2,2'-azido-di-[3-ethylbenzothiazoline sulfonate] (ABTS), o-dianisidine, 4-chloronaphthol (4-CN), nitrophenyl-β-D-galactopyranoside, and the like. Examples of suitable anti-inflammatory agents include 5-bromo-4-chloro-3-indolyl-β-D-galactopyranoside (ONPG), o-phenylenediamine (OPD), 5-bromo-4-chloro-3-indolyl-β-galactopyranoside (X-Gal), methylumbelliferyl-β-D-galactopyranoside (MU-Gal), p-nitrophenyl-α-D-galactopyranoside (PNP), 5-bromo-4-chloro-3-indolyl-β-D-glucuronide (X-Gluc), 3-amino-9-ethylcarbazole (AEC), fuchsin, iodonitrotetrazolium (INT), tetrazolium blue, and tetrazolium violet.

[0063] In some embodiments, enzymes can be used in metallographic detection schemes. Metallographic detection methods include using an enzyme, such as alkaline phosphatase, in combination with a water-soluble metal ion and a redox-inactive substrate for the enzyme. In some embodiments, the substrate is converted by the enzyme to a redox-active agent, which reduces the metal ion, causing it to form a detectable precipitate. (See, e.g., U.S. Patent Application No. 11 / 015,646, filed December 20, 2004; PCT Publication No. 2005 / 003777; and U.S. Patent Application Publication No. 2004 / 0265922, each of which is incorporated herein by reference in its entirety.) Metallographic detection methods include using an oxidoreductase (such as horseradish peroxidase) with a water-soluble metal ion, an oxidizing agent, and a reducing agent to re-form a detectable precipitate. (See, e.g., U.S. Patent No. 6,670,113, incorporated herein by reference in its entirety.)

[0064] In some embodiments, enzymatic action occurs between the enzyme and the dye itself, and the reaction converts the dye from an unbound species to a species that is deposited on the sample. For example, the reaction of DAB with a peroxidase (such as horseradish peroxidase) oxidizes the DAB, causing it to precipitate.

[0065] In yet other embodiments, the detectable moiety is deposited via a signaling conjugate comprising a latent reactive moiety configured to react with an enzyme to form a reactive species capable of binding to a sample or other detection component. These reactive species can react with the sample near their generation, i.e., near the enzyme, but rapidly convert to a non-reactive species, so that the signaling conjugate is not deposited at a site distal to the site where the enzyme is deposited. Examples of latent reactive moieties include quinone methide (QM) analogs, such as those described in WO2015124703A1, and tyramide conjugates, such as those described in WO2012003476A2, each of which is incorporated herein by reference in its entirety. In some examples, the latent reactive moiety is directly conjugated to a dye, such as N,N'-biscarboxypentyl-5,5'-disulfonato-indo-dicarbocyanine (Cy5), 4-(dimethylamino)azobenzene-4'-sulfonamide (DABSYL), tetramethylrhodamine (DISCO Purple), and rhodamine 110 (rhodamine). In other examples, the latent reactive moiety is conjugated to one member of a specific binding pair, and the dye is linked to the other member of the specific binding pair. In other examples, the latent reactive moiety is linked to one member of a specific binding pair, and an enzyme is linked to the other member of the specific binding pair, and the enzyme (a) reacts with a chromogenic substrate to produce a dye, or (b) reacts with a dye to deposit a dye (e.g., DAB). Examples of specific binding pairs include the following: (1) biotin or a biotin derivative (such as desthiobiotin) linked to a latent reactive moiety, and a biotin-binding entity (such as avidin, streptavidin, deglycosylated avidin (such as NEUTRAVIDIN)), or a biotin-binding protein (such as CAPTAVIDIN) having a nitrated tyrosine at its biotin-binding site and linked to a dye or to a chromogenic substrate or to an enzyme reactive with a dye (e.g., peroxidase linked to a biotin-binding protein when the dye is DAB), and (2) A hapten linked to a latent reactive moiety and an anti-hapten antibody linked to a dye or to an enzyme reactive with a chromogenic substrate or reactive with a dye (e.g., peroxidase linked to a biotin-binding protein when the dye is DAB).

[0066] Non-limiting examples of combinations of biomarker-specific reagents and detection reagents are set forth and specifically included in Table 1. [Table 1] TIFF0007802843000002.tif250170 TIFF0007802843000003.tif246170 TIFF0007802843000004.tif238170 TIFF0007802843000005.tif255170 TIFF0007802843000006.tif255170 TIFF0007802843000007.tif191170 TIFF0007802843000008.tif168170 TIFF0007802843000009.tif115170 In a specific embodiment, the biomarker-specific reagents and specific detection reagents identified in Table 1 are antibodies. As will be appreciated by one of skill in the art, the detection scheme for each of the biomarker-specific reagents may be the same or different.

[0067] Non-limiting examples of commercially available detection reagents or kits containing detection reagents suitable for use with the methods of the present invention include the VENT ANA ultraView detection system (an enzyme-conjugated secondary antibody containing HRP and AP), the VENT ANA iVIEW detection system (a biotinylated anti-species secondary antibody and a streptavidin-conjugated enzyme), the VENTANA OptiView detection system (OptiView) (a hapten-conjugated anti-species secondary antibody and an anti-hapten tertiary antibody conjugated to an enzyme multimer), the VENTANA Amplification kit (an unconjugated secondary antibody that can be used with any of the previous VENTANA detection systems to amplify the number of enzymes at the site of primary antibody binding), the VENTANA OptiView Amplification systems (including an anti-species secondary antibody conjugated to a hapten, an anti-hapten tertiary antibody conjugated to an enzyme multimer, and a tyramide conjugated to the same hapten. In use, the secondary antibody is contacted with the sample, resulting in binding to the primary antibody. The sample is then incubated with the anti-hapten antibody to associate the enzyme with the secondary antibody. The sample is then incubated with tyramide, resulting in deposition of additional hapten molecules. The sample is then again incubated with the anti-hapten antibody, resulting in deposition of additional enzyme molecules. The sample is then incubated with a detectable moiety, resulting in pigmentation), VENTANA DISCOVERY, DISCOVERY OmniMap, DISCOVERY UltraMap anti-hapten antibody, secondary antibody, chromogen, fluorophore, and dye kits, each of which is manufactured by Ventana Medical Systems, Inc. (Tucson, Arizona), PowerVision, and PowerVision+ IHC Detection Systems (secondary antibodies that polymerize directly with HRP or AP into compact polymers with a high enzyme-to-antibody ratio), as well as the DAKO EnVision™+ System (enzyme-labeled polymers that bind to secondary antibodies).

[0068] III.C.2. Multiplex Labeling Schemes In one embodiment, the biomarker-specific reagents and detection reagents are applied in a multiplex staining method, in which the biomarker-specific reagents and detection reagents are applied in a manner that allows differential labeling of different biomarkers.

[0069] One way to achieve different labeling of different biomarkers is to select a combination of biomarker-specific reagents, detection reagents, and enzymes that does not result in off-target cross-reactivity between different antibodies or detection reagents (referred to as "combined staining"). For example, if secondary detection reagents are used, each secondary detection reagent can bind to only one of the primary antibodies used on the sections. For example, primary antibodies from different animal species (such as mouse, rabbit, or rat, and the resulting antibodies) can be selected, in which case a species-specific secondary antibody can be used. As another example, each primary antibody can contain a different hapten or epitope tag, and the secondary antibody is selected to specifically bind to the hapten or epitope tag. In addition, each set of detection reagents should be suitable for depositing different detectable entities on the sections, such as by depositing different enzymes near each biomarker-specific reagent. An example of such a preparation is shown in U.S. Patent No. 8,603,765. Such a setup has the potential advantage that each set of biomarker-specific reagents and associated specific binding reagents can be present on the sample simultaneously and / or staining can be performed with a cocktail of biomarker-specific reagents and detection reagents, thereby reducing the number of staining steps. However, such a setup may not always be feasible, given that reagents may cross-react with different enzymes and various antibodies may cross-react with each other, leading to aberrant staining.

[0070] Another way to achieve differential labeling of different biomarkers is to sequentially stain the sample for each biomarker. In such an embodiment, a first biomarker-specific reagent reacts with the section, followed by a secondary detection reagent reacting with the first biomarker-specific reagent and other detection reagents, resulting in the deposition of a first detectable entity. The section is then treated to remove the biomarker-specific reagent and associated detection reagents from the section, leaving the deposited stain in place. This process is repeated for subsequent biomarker-specific reagents. Examples of methods for removing biomarker-specific reagents and associated detection reagents include heating the sample in the presence of a buffer that elutes the antibody from the sample (referred to as the "heat kill method"), such as those disclosed by Stack et al., Multiplexed immunohistochemistry, imaging, and quantitation: A review, with an assessment of Tyramide signal amplification, multispectral imaging and multiplex analysis, Methods, Vol. 70, Issue 1, pp 46-58 (Nov. 2014), the contents of which are incorporated by reference, and PCT / EP 2016 / 057955.

[0071] As will be appreciated by those skilled in the art, combination and sequential staining methods can be combined. For example, if only a subset of primary antibodies are compatible with combination staining, the sequential staining method can be modified so that the antibodies compatible with combination staining are applied to the sample using combination staining, and the remaining antibodies are applied using sequential staining.

[0072] III.C.3. Counterstaining: If desired, biomarker-stained slides can be counterstained to assist in identifying morphologically relevant regions for identifying ROIs, either manually or automatically. Examples of counterstains include hematoxylin (stains blue to violet), methylene blue (stains blue), toluidine blue (stains nuclei deep blue and polysaccharides pink to red), nuclearfast red (also known as Kernectroth dye, stains red), and methyl green (stains green); non-nuclear chromogenic stains such as eosin (stains pink); fluorescent non-nuclear stains such as 4',6-diamino-2-phenylindole (DAPI, stains blue), propidium iodide (stains red), Hoechst stain (stains blue), Nuclear Green DCS1 (stains green), Nuclear Yellow (Hoechst S769121, stains yellow at neutral pH and blue at acidic pH), DRAQ5 (stains red), and DRAQ7 (stains red); and fluorophore-conjugated phalloidin (stains filamentous actin; color is due to the attached fluorophore).

[0073] III.C.4. Morphological staining of specimens In certain embodiments, it is also desirable to morphologically stain serial sections of the biomarker-stained slides. 102 These sections can be used to identify ROIs where scoring will occur. 103 Basic morphological staining techniques often rely on staining nuclear structures with one dye and cytoplasmic structures with a second dye. Many morphological stains are known, including, but not limited to, hematoxylin and eosin (H&E) staining and Lee stain (methylene blue and basic fuchsin). In specific embodiments, at least one serial section of each biomarker-stained slide is H&E stained. Any method of applying H&E staining can be used, including manual and automated methods. In one embodiment, at least one section of the sample is an H&E-stained sample stained using an automated staining system. Automated systems for performing H&E staining typically operate using one of two staining principles: batch staining (also known as "dip and dunk") or individual slide staining. Batch stainers generally use a vat or bath of reagent into which many slides are immersed simultaneously. Individual slide stainers, on the other hand, apply reagent directly to each slide, with no two slides sharing the same reagent aliquot. Examples of commercially available H&E stainers include the VENTANA SYMPHONY (individual slide stainer) and VENTANA HE 600 (individual slide stainer) H&E stainer series manufactured by Roche; the Dako CoverStainer (batch stainer) manufactured by Agilent Technologies; and the Leica ST4020 Small Linear Stainer (batch stainer), Leica ST5020 Multistainer (batch stainer), and Leica ST5010 Autostainer XL series (batch stainer) H&E stainers manufactured by Leica Biosystems Nussloch GmbH.

[0074] III.D. ROIs, Objects, and Features In one embodiment, one or more objects associated with the biomarkers of the panel are identified in a digital image of a sample stained with the biomarkers 104. The number of objects and / or the relatedness of different objects to each other are used to define the features to be evaluated for development of a scoring function. Non-limiting exemplary panels of potential objects that can be detected from each panel are set forth in Table 2 below. [Table 2]

[0075] In some embodiments, one or more regions of interest (ROIs) are also identified in the digital image of the biomarker-stained sample 104. The ROIs encompass biologically relevant locations in the tissue section from which relevant objects are identified for feature calculation. In one embodiment, the ROIs are morphological regions of the tumor-containing tissue section, such as the tumor region (TR), invasion front, and peritumoral (PT) region.

[0076] The ROI can be limited to the morphological region, expanded to include an area outside the morphological region (i.e., by extending the margins of the ROI a specified distance outside the morphological region), or restricted to a subregion of the morphological region (e.g., by shrinking the ROI a specified distance inside the perimeter of the morphological region, or by identifying an area within the ROI with a certain characteristic (such as the baseline density of a particular cell type)). In the case of a morphological region defined by an edge (such as an invasion front), the ROI can be defined, for example, as all points within a specified distance of any point of the edge, all points on one side of the edge within a specified distance of any point of the edge, the smallest geometric region (such as a circle, ellipse, square, rectangle, etc.) that encompasses the entire edge region, all points within a circle of a specified radius centered at the midpoint of the edge region, etc.

[0077] In some embodiments, the same ROI can be used for all sections and biomarkers. For example, a morphologically defined ROI can be identified in H&E-stained sections of a sample and used for all sections stained with a biomarker. In other embodiments, different ROIs can be used for different biomarkers. For example, an H&E-stained slide can be used to identify a specific morphological region, such as a tumor region, that is used as a first ROI. A second ROI or ROIs can then be identified in one of the biomarker-stained sections, for example, to identify a region having a class of cells at a certain threshold density (e.g., epithelial region vs. stromal region). The second ROI or ROIs can then be used for feature calculation.

[0078] Non-limiting examples of various ROIs are shown in Table 3. [Table 3]

[0079] In some embodiments, ROI is manually identified in digital image.For example, trained expert can manually outline one or more morphological area(s) (such as tumor area and / or invasion front) on the digital image of sample.Then, the outlined area(s) on the image can be used as ROI for calculating feature or as reference point for calculating ROI.

[0080] In other embodiments, the computer-implemented system can assist the user in annotating the ROI (referred to as "semi-automated ROI annotation"). For example, the user can outline one or more regions on a digital image, and the system will automatically convert it into a complete ROI. For example, if the desired ROI is a PI, PO, and / or PR region, the user can outline the tumor region and invasion front, and the system will automatically draw the user-defined PI, PO, and PR regions. In another embodiment, if the ROI is an EA or SA, the user can outline the tumor region and, optionally, the invasion front in the image, registering this to the biomarker-stained image, and the system will create the associated EA and SA ROIs by labeling all cells within a predetermined distance of EM+ cells as being within the EA and all cells beyond the predetermined distance as being within the SA. In another embodiment, the system can also apply pattern recognition functions using computer vision and machine learning to identify regions with similar morphological characteristics to the outlined and / or automatically generated region. Thus, for example, a tumor region can be annotated in a semi-automated manner by methods including: (a) a user annotates a tumor region in an H&E image of a sample by outlining the tumor region; and (b) The computer system applies pattern recognition functions to identify additional regions of the sample having morphological characteristics of the outlined region, where the total tumor region includes areas annotated by the user and areas automatically identified by the system. In another example, the PR, PI, and / or PO ROIs can be annotated in a semi-automated manner by methods including: (a) the user annotates tumor regions in the H&E image of the sample by outlining the tumor area and the invasion front; and (b) the computer system automatically defines a region(s) of PR, PI, and / or PO that encompasses all pixels within a defined distance of the annotated invasion front; and (c) The computer system applies a pattern recognition function to identify additional regions of the sample having morphological characteristics of the PI, PO, and / or PR regions identified by step (b). Many other arrangements can be used as well. If the generation of the ROI is semi-automated, the user may be given the option to modify the ROI annotated by the computer system, such as by enlarging the ROI, annotating regions of the ROI or objects within the ROI that are to be excluded from the analysis, etc.

[0081] In other embodiments, the computer system can automatically suggest ROIs without any direct input from the user (referred to as "automated ROI annotation"). For example, a previously trained tissue segmentation function or other pattern recognition function can be applied to an unannotated image to identify desired morphological regions to use as ROIs. The user can be given the option to modify the ROIs annotated by the computer system, such as enlarging the ROI or annotating regions of the ROI or objects within the ROI that are to be excluded from analysis.

[0082] One or more features are extracted from the ROI(s) and quantified to obtain feature values ​​for each sample 105. Exemplary features include, for example, the total number of objects within the ROI, the density of a particular object within the ROI, the spatial relationship between different objects within the ROI, the spatial distribution of a particular object within the ROI, the ratio of the number and / or density of different objects within the ROI, the ratio of the same object within different ROIs (e.g., the ratio of a particular cell in the EA ROI vs. the SA ROI or the ratio of a particular cell in the PI ROI vs. the PO ROI), the fraction of total objects of a larger ROI that fit within a smaller ROI that fit within the larger ROI (e.g., the fraction of a particular cell type in the TA ROI that fit within the EA, SA, PI, PO, or PT ROI). Specific exemplary features for each panel are set forth in Table 4. [Table 4] TIFF0007802843000013.tif251170TIFF0007802843000014.tif249170TIFF0007802843000015.tif249170TIFF0007802843000016.tif249170TIFF0007802843000017.tif249170TIFF0007802843000018.tif250170TIFF0007802843000019.tif198170Unless otherwise specified, the ROIs for the features in Table 4 are tumor areas.Unless otherwise specified, any densities listed in Table 4 are area densities (i.e., number of positive cells over the area of ​​the ROI). As used in Table 4, "PD1 low," "PD1 medium," and "PD1 high" refer to individual cells having low, medium, and high PD-1 staining intensities. In one embodiment, a "PD1 low" cell is a PD-1+ cell having a staining intensity in the bottom third of all PD-1+ cells measured across all samples tested, a "PD1 medium" cell is a PD-1+ cell having a staining intensity in the middle third of all PD-1+ cells measured across all samples tested, and a "PD1 high" cell is a PD-1+ cell having a staining intensity in the top third of all PD-1+ cells measured across all samples tested.

[0083] III.F. Scoring Function Modeling To identify a scoring function, features are modeled for their ability to predict the relative likelihood of responding to a PD-1 axis-directed course of treatment.

[0084] In one embodiment, features can be selected by running a feature selection function 106. Feature and outcome data for each member of the cohort are input into a feature selection function, which then uses the data to rank-order various features by their relative correlation with the desired outcome. Exemplary feature selection functions include aggregate feature selection functions (e.g., including Random Forest functions), filter method functions (e.g., including mutual information-based functions, (mRMR) / correlation coefficient-based functions, and relief-based functions), and / or embedded feature selection functions (such as elastic net / least absolute shrinkage or selection operator (LASSO) functions). In one embodiment, candidate models are created using the top 25, top 20, top 15, top 10, top 9, top 8, top 7, top 6, top 5, top 4, or top 3 features identified by the feature selection function. In another embodiment, the candidate model uses at least one, at least two, at least three, at least four, or at least five features identified in the top 10 features of at least two different feature selection functions. In another embodiment, the candidate model includes at least one feature present in the top 5 features of at least two feature selection functions. In one embodiment, a "responder" is considered a patient who has either a partial response or a complete response. In one embodiment, a "responder" is considered a patient who has stable disease, a partial response, or a complete response.

[0085] Candidate models are generated by inputting selected features and outcome data for each member of the cohort into a modeling function. The candidate model that best matches the response is selected as the scoring function. Exemplary modeling functions include quadratic discriminant analysis (QDA), linear discriminant analysis (LDA), support vector machine (SVM), and artificial neural network (ANN). In one embodiment, the candidate function is modeled solely based on features extracted from the digital image. In other embodiments, the candidate function includes other clinical variables such as age, sex, mismatch repair status, and / or microsatellite instability status. In one embodiment, the model is used to predict progressive disease after treatment versus stable disease after treatment versus a partial or complete response to therapy. In one embodiment, the model is used to predict the likelihood that a patient will have progressive disease after treatment versus the likelihood that the patient will have stable disease, a partial response, or a complete response to therapy. In one embodiment, the model is used to predict the likelihood that a patient will have progressive or stable disease after treatment versus the likelihood that the patient will have stable disease, a partial response, or a complete response to therapy.

[0086] Additionally, one or more stratification cutoffs can be selected to separate patients into "risk bins" by relative risk ("high risk" and "low risk," quartiles, deciles, etc.). 107 In one example, the stratification cutoffs are selected using a receiver operator characteristic (ROC) curve. The ROC curve allows the user to balance the sensitivity of the model (i.e., prioritizing capturing as many "positive" or "likely to respond" candidates as possible) with the specificity of the model (i.e., minimizing false positives for "likely to respond" candidates). In one embodiment, the cutoff is selected between likely and unlikely to return to a risk bin, and the selected cutoff balances sensitivity and specificity. In one embodiment, the stratification cutoff distinguishes between (a) patients likely to develop progressive disease after treatment and (b) patients likely to have stable disease, a partial response, or a complete response to therapy. In one embodiment, the stratification cutoff distinguishes between (a) patients who are likely to experience progressive disease after treatment, (b) patients who are likely to experience stable disease after treatment, and (c) patients who are likely to have a partial or complete response to therapy. In one embodiment, the stratification cutoff distinguishes between (a) patients who are likely to experience progressive or stable disease after treatment and (b) patients who are likely to have a partial or complete response to therapy.

[0087] The models can be implemented, if desired, using a computer-based statistical analysis software suite such as the R Project for statistical computing (available at https: / / www.r-project.org / ), SAS, MATLAB, among others.

[0088] IV. Scoring using the scoring function After the scoring function is modeled and the stratification cutoff options are selected, the scoring function can be applied to images of a test sample to calculate a score for the test sample. Figure 2 is a flow chart illustrating an exemplary approach for scoring a test sample using the scoring function described above. Tumor tissue sections are first obtained from a patient for whom PD-1 axis-directed therapy is being considered 201. The tissue sections are typically similar to the type of sample used to model the scoring function, except that the outcome is not yet known. At least one of the tissue sections is stained for a biomarker associated with the scoring function, and serial sections are stained with a morphological stain (e.g., H&E) if required for ROI selection 202. The stained sections are imaged 203, and one or more ROIs associated with the scoring function are annotated to the biomarker-stained image along with any objects used to calculate associated features 204. Relevant features are extracted from the ROIs, and feature values ​​for each feature are calculated 205. Next, a feature vector is assembled 206 containing all the variables used by the scoring function, and the scoring function is applied to the feature vector 207. In some cases, the variables are only features extracted from the ROI. In other cases, variables may be, for example, age, sex, mismatch repair status (such as whether the patient has deficient MMR (dMMR) or advanced MMR (pMMR)), microsatellite instability status (such as whether the patient has MSI), or other features extracted from the ROI. 高 Is it an MSI? 低 The output score can also be assigned to an associated risk bin if stratification cutoffs are used. The score can then be integrated into a diagnostic and / or treatment decision by a clinician, including, for example, by integrating the score with other clinical variables that may be important in determining whether to administer a PD-1 axis-directed therapy.

[0089] In one embodiment, the scoring function is integrated into a scoring system, an exemplary scoring system is shown in FIG.

[0090] The scoring system includes an image analysis system 300. The image analysis system 300 can include one or more computing devices, such as a desktop computer, a laptop computer, a tablet, a smartphone, a server, an application-specific computing device, or any other type(s) of electronic device(s) capable of performing the techniques and operations described herein. In some embodiments, the image analysis system 300 can be implemented as a single device. In other embodiments, the image analysis system 300 can be implemented as a combination of two or more devices that collectively accomplish the various functions discussed herein. For example, the image analysis system 300 can include one or more server computers and one or more client computers communicatively coupled to each other via one or more local area networks and / or a wide area network such as the Internet.

[0091] As shown in Figure 3, image analysis system 300 may include memory 314, processor 315, and display 316. Memory 314 may include any combination of any type of volatile or non-volatile memory, such as random access memory (RAM), read-only memory such as electrically erasable programmable read-only memory (EEPROM), flash memory, hard drives, solid-state drives, optical disks, etc. For purposes of simplicity, memory 314 is illustrated in Figure 3 as a single device, although it will be appreciated that memory 314 may also be located across two or more devices.

[0092] Processor 315 may include one or more processors of any type, such as a central processing unit (CPU), a graphics processing unit (GPU), a dedicated signal or image processor, a field programmable gate array (FPGA), a tensor processing unit (TPU), etc. For purposes of simplicity, processor 315 is illustrated in Figure 3 as a single device, but it will be appreciated that processor 315 may also be distributed across any number of devices.

[0093] The display 316 may be implemented using any suitable technology, such as LCD, LED, OLED, TFT, plasma, etc. In some implementations, the display 316 may be a touch-sensitive display (touch screen).

[0094] As shown in FIG. 3 , image analysis system 300 may also include an object identifier 310, a region of interest (ROI) generator 311, a user interface module 312, and a scoring engine 313. While these modules are illustrated in FIG. 3 as stand-alone modules, it will be apparent to those skilled in the art that each module may instead be implemented as several sub-modules, and that in some embodiments, any two or more modules may be combined into a single module. Furthermore, in some embodiments, system 100 may include additional engines and modules (e.g., input devices, network modules, and communication modules) not illustrated in FIG. 3 for the sake of brevity. Furthermore, in some embodiments, some of the blocks illustrated in FIG. 3 may be disabled or omitted. As discussed in more detail below, the functionality of some or all of the modules of system 100 may be implemented in hardware, software, firmware, or some combination thereof. Exemplary commercially available software packages useful for implementing the modules disclosed herein include VENTANA VIRTUOSO; Definiens TISSUE STUDIO, DEVELOPER XD, and IMAGE MINER; and Visopharm BIOTOPIX, ONCOTOPIX, and STEREOTOPIX software packages.

[0095] After acquiring an image, the image analysis system 300 can pass the image through the object identifier 310, which functions to identify and label relevant objects and other features in the image for subsequent scoring. From each image, the object identifier 310 can extract (or generate) multiple image features that characterize various objects in the image, as well as pixels representing the expression of biomarker(s). The extracted image features can include, for example, texture features such as Haralick features, bag-of-words features, etc. The values ​​of multiple image features can be combined into a high-dimensional vector, hereafter referred to as a "feature vector," that characterizes the expression of biomarkers associated with the features of the scoring function. For example, if M features are extracted for each object and / or pixel, each object and / or pixel can be characterized by an M-dimensional feature vector. The output of the object identifier 310 is effectively a map of the image that annotates the locations of objects and pixels of interest and associates the objects and pixels with feature vectors that describe the object or pixel.

[0096] For biomarkers that are scored based on the biomarker's association with a particular type of object (e.g., membrane, nucleus, cell), the features extracted by object identifier 310 may include features or feature vectors sufficient to classify objects in a sample as biomarker-positive or biomarker-negative objects of interest, and / or by the level or intensity of the object's biomarker staining. If a biomarker can be weighted differently depending on the type of object expressing it, the features extracted by object identifier 310 may include features relevant to determining the type of object associated with a biomarker-positive pixel. Thus, objects can then be classified based at least on biomarker expression (e.g., biomarker-positive or biomarker-negative cells) and, if relevant, object subtype (e.g., tumor cell, immune cell, etc.). If the degree of biomarker expression is scored independently of object association, the features extracted by object identifier 310 may include, for example, the location and / or intensity of biomarker-positive pixels. The exact features extracted from the image will depend on the type of classification function being applied and will be well known to those skilled in the art.

[0097] Examples of objects identified for a particular biomarker panel are set forth in Table 5 below. [Table 5] TIFF0007802843000021.tif26170

[0098] The image analysis system 300 may also pass the image through an ROI generator 311. The ROI generator 311 is used to identify one or more ROIs in the image for which the immune context score is calculated. If the object identifier 310 is not applied to the entire image, the one or more ROIs generated by the ROI generator 311 may also be used to define a subset of the image on which the object identifier 310 is run.

[0099] In one embodiment, the ROI generation program 311 can be accessed through the user interface module 312. An image of a biomarker-stained sample (or a morphologically stained serial section of a biomarker-stained sample) is displayed in the graphical user interface of the user interface module 112, and a user annotates one or more region(s) in the image to be considered as an ROI. In this example, the annotation of the ROI can take several forms. For example, the user can manually define the ROI (hereinafter referred to as "manual ROI annotation"). In other examples, the ROI generation program 311 can assist the user in annotating the ROI (hereinafter referred to as "semi-automated ROI annotation"). For example, the user can outline one or more regions on the digital image, which the system automatically converts into a complete ROI. For example, if the desired ROI is a tumor region, the user outlines the tumor region, and the system identifies similar morphological regions using, for example, computer vision and machine learning. As another example, a user can annotate edges in an image (e.g., by tracing a line defining a tumor's invasion front), and the ROI generation program 311 can automatically define an ROI based on the user-defined edges. For example, a user can annotate the edges of the invasion front or tumor region in the user interface module 312, and the ROI generation program 311 creates the ROI using the edges as a guide, for example, by drawing an ROI that encompasses all objects within a predefined distance within the edge (e.g., a PT ROI), or within a predefined distance on one side of the edge (e.g., a PO or PI ROI), or within a first predefined distance on a first side of the edge and a second predefined distance on a second side of the edge (e.g., a PT ROI whose inner and outer portions have different standard distances from the invasion front).

[0100] In other embodiments, the ROI generator 311 may automatically suggest an ROI without any direct input from the user (e.g., by applying a tissue segmentation function to an unannotated image), which the user can then choose to accept, reject, or edit as appropriate.

[0101] In some embodiments, the ROI generation program 311 can also include a registration function, whereby an ROI annotated in one section of a set of serial sections is automatically transferred to other sections in the set of serial sections. This function is particularly useful when H&E-stained serial sections are provided together with sections labeled with biomarkers. In such embodiments, a user can, for example, draw a tumor region in a digital image of the H&E-stained section. The ROI generation program 311 then registers the ROI from the H&E image to the image of the serial section stained with the biomarker, matching the tissue structure of the H&E image with the corresponding tissue structure in the serial section. Exemplary registration methods can be found, for example, in WO2013 / 140070 and US2016-0321809.

[0102] The object identifier 310 and the ROI generator 311 can be implemented in any order. For example, the object identifier 310 can be applied to the entire image first. The locations and features of the identified objects can be saved and recalled later when the ROI generator 311 is implemented. In such a configuration, scores can be generated by the scoring engine 313 immediately upon ROI generation. Such a flow diagram is shown in FIG. 3A. As seen in FIG. 4A, an image with a mixture of different objects is acquired (indicated by the dark ovals and dark diamonds). After the object identification task is implemented, all diamonds in the image are identified (indicated by the open diamonds). Once the ROI is attached to the image (indicated by the dashed line), only diamonds within the ROI region are included in the metric calculation for the ROI. A feature vector is then calculated that includes the feature values ​​and any additional values ​​used by the scoring function executed by the scoring engine 313. Alternatively, the ROI generator 311 can be implemented first. In this flow diagram, the object identifier 310 can be implemented only in the ROI (minimizing computation time) or even for the entire image (which would allow for on-the-fly adjustments without rerunning the object identifier 310). Such a flow diagram is shown in FIG. 4B. As seen in FIG. 4B, an image with a mixture of different objects is acquired (indicated by the dark ovals and dark diamonds). The ROI is attached to the image (indicated by the dashed lines), but no objects are yet labeled. After the object identification task is implemented in the ROI, all diamonds within the ROI are identified (indicated by the open diamonds) and included in the calculation of features for the ROI. A feature vector is then calculated that includes the feature(s) and any additional quantities used by the scoring function executed by the scoring engine 313. It may also be possible to implement the object identifier 310 and ROI generator 311 simultaneously.

[0103] After both the object identifier 310 and the ROI generator 311 are implemented, a scoring engine 313 is implemented. The scoring engine 313 calculates feature(s) for the ROI and, if used, calculates predetermined maximum and / or minimum cutoffs. A feature vector containing the calculated features and other variables used by the scoring function is assembled by the scoring engine, and the scoring function is applied to the feature vector.

[0104] Specific exemplary characteristics for each panel are set forth in Table 6. [Table 6] TIFF0007802843000023.tif255170TIFF0007802843000024.tif255170TIFF0007802843000025.tif255170TIFF00078028430 00026.tif255170TIFF0007802843000027.tif255170TIFF0007802843000028.tif255170TIFF0007802843000029.tif201170

[0105] 3, in some embodiments, image analysis system 300 can be communicatively coupled to image acquisition system 320. Image acquisition system 320 can acquire images of the sample and provide the images to image analysis system 300 for analysis and presentation to a user.

[0106] The image acquisition system 320 can include a scanning platform 325, such as a slide scanner, capable of scanning stained slides at 20x, 40x, or other magnifications to generate high-resolution whole-slide digital images, including, for example, a slide scanner. At a basic level, a typical slide scanner includes at least the following: (1) a microscope with an objective lens; (2) a light source (such as halogen, light-emitting diode, white light, and / or a multispectral light source, depending on the dye); (3) a robot for moving the slide (or moving the optics around the slide); (4) one or more digital cameras for image capture; and (5) a computer and associated software for controlling the robotics and manipulating, managing, and displaying the digital slides. Digital data for several different XY positions (and in some cases multiple Z-planes) on the slide are captured by the camera's charge-coupled device (CCD), and the images are combined to form a composite image of the entire scanned surface. A typical method for achieving this involves the following: (1) Tile-based scanning, which moves the slide stage or optics in very small increments to capture square image frames that slightly overlap adjacent squares, then automatically matches the captured squares to each other to create a composite image; (2) Line-based scanning, in which the slide stage moves in a single axis during acquisition to capture multiple composite image "strips." The image strips can then be aligned with each other to form a larger composite image. A detailed overview of various scanners (both fluorescent and brightfield) can be found in Farahani et al., Whole slide imaging in pathology: advantages, limitations, and emerging perspectives, Pathology and Laboratory Medicine Int'l, Vol. 7, pp. 23-33 (June 2015), the contents of which are incorporated by reference in their entirety. Examples of commercially available slide scanners include: 3DHistech PANNORAMIC SCAN II; DigiPath PATHSCOPE, Hamamatsu NANOZOOMER RS, HT, and XR; Huron TISSUESCOPE 4000, 4000XT, and HS; Leica SCANSCOPE AT, AT2, CS, FL, and SCN400; Mikroscan D2; Olympus VS120-SL; Omnyx VL4 and VL120; PerkinElmer LAMINA; Philips ULTRA-FAST SCANNER; Sakura Finetek VISIONTEK; Unic PRECICE 500 and PRECICE 600x; VENTANA ISCAN COREO and ISCAN HT; and Zeiss AXIO SCAN Z1. Other exemplary systems and features can be found, for example, in WO2011-049608 or U.S. Patent Application No. 61 / 533,114, entitled IMAGING SYSTEM, CASSETTE, AND METHODS OF USE THEREOF, filed September 9, 2011, the contents of which are incorporated by reference in their entirety.

[0107] Images generated by scanning platform 325 can be transferred to image analysis system 300 or to a server or database accessible by image analysis system 300. In some embodiments, the images can be transferred automatically over one or more local and / or wide area networks. In some embodiments, image analysis system 300 can be integrated with or included in other modules of scanning platform 325 and / or image acquisition system 320, in which case the images can be transferred to the image analysis system, for example, via memory accessible by both platform 325 and system 320. In some embodiments, image acquisition system 320 cannot be communicatively coupled to image analysis system 300, in which case the images can be stored on some type of non-volatile storage medium (e.g., a flash drive) and downloaded from the medium to image analysis system 300 or to a server or database communicatively coupled to the system. In any of the foregoing examples, image analysis system 300 may acquire images of biological samples, in which the samples may be fixed to slides and stained by a histochemical staining platform 323, in which the slides may be scanned by a slide scanner or another type of scanning platform 325. However, it will be recognized that in other embodiments, the techniques described below may also be applied to images of biological samples acquired and / or stained via other means.

[0108] The image acquisition system 320 can also include an automated histochemical staining platform 323, such as an automated IHC / ISH slide stainer. An automated IHC / ISH slide stainer typically includes at least the following: reservoirs for the various reagents used in the staining protocol, a reagent-dispensing unit in fluid communication with the reservoirs for dispensing the reagents onto the slides, a waste removal system for removing used reagents and other waste from the slides, and a control system for coordinating the operation of the reagent-dispensing unit and the waste removal system. In addition to performing the staining steps, many automated slide stainers can also perform steps incidental to staining (or are compatible with separate systems that perform such ancillary steps), including slide baking (to attach the sample to the slide), dewaxing (also called deparaffinization), antigen retrieval, counterstaining, dehydration and clearing, and coverslipping. Prichard, Overview of Automated Immunohistochemistry, Arch Pathol Lab Med., Vol. 138, pp. 1578-1582 (2014), the entire contents of which are incorporated herein by reference, describes several specific examples of automated IHC / ISH slide stainers and their various features, including the intelliPATH (Biocare Medical), WAVE (Celerus Diagnostics), DAKO OMNIS and DAKO AUTOSTAINER LINK 48 (Agilent Technologies), BENCHMARK (Ventana Medical Systems, Inc.), Leica BOND, and Lab Vision Autostainer (Thermo Scientific) automated slide stainers.Additionally, Ventana Medical Systems, Inc. is the assignee of several U.S. patents that disclose systems and methods for performing automated analyses, including U.S. Patent Nos. 5,650,327, 5,654,200, 6,296,809, 6,352,861, 6,827,901, and 6,943,029, and U.S. Patent Application Publication Nos. 20030211630 and 20040052685, each of which is incorporated herein by reference in its entirety. Commercially available staining units typically operate on one of the following principles: (1) Open-system individual slide staining, in which the slide is positioned horizontally and the reagent is dispensed as a paddle onto the surface of the slide containing the tissue sample (such as that implemented in the DAKO AUTOSTAINER Link 48 (Agilent Technologies) and intelliPATH (Biocare Medical) stainers); (2) Liquid overlay techniques, in which the reagent is either covered by or dispensed through an inert fluid layer placed on the sample (such as that implemented in the VENTANA BenchMark and DISCOVERY stainers); and (3) Capillary gap staining, in which the slide surface is placed adjacent to another surface (which may be another slide or a cover plate) to create a narrow gap through which the liquid reagent is drawn by capillary forces and maintained in contact with the sample (such as the staining principle used by the DAKO TECHMATE, Leica BOND, and DAKO OMNIS stainers). In some iterations of capillary gap staining, the liquid in the gap (such as on the DAKO TECHMATE and Leica BOND) does not mix. A variation of capillary gap staining, called dynamic gap staining, uses capillary forces to apply the sample to the slide and then moves parallel surfaces relative to one another to agitate the reagents during incubation, resulting in mixing of the reagents (such as the staining principle implemented on the DAKO OMNIS slide stainer (Agilent)). In moving gap staining, a movable head is placed on the slide. The underside of the head is separated from the slide by a first gap small enough to allow a liquid meniscus to form from the liquid on the slide during slide movement.A mixing extension having a lateral dimension less than the width of the slide extends from the underside of the movable head to define a second gap smaller than the first gap between the mixing extension and the slide. During translation of the head, the lateral dimension of the mixing extension is sufficient to generate lateral movement of liquid on the slide generally in a direction extending from the second gap to the first gap. See WO2011-139978 A1. The use of inkjet technology to deposit reagents on slides has recently been proposed. See WO2016-170008 A1. This list of staining techniques is not intended to be comprehensive, and any fully automated or semi-automated system for performing biomarker staining can be incorporated into the histochemical staining platform 323.

[0109] The image acquisition system 320 can also include an automated H&E staining platform 324. Automated systems for performing H&E staining typically operate on one of two staining principles: batch staining (also known as "dip and dunk") or individual slide staining. Batch stainers generally use a vat or bath of reagent into which many slides are immersed simultaneously. Individual slide stainers, on the other hand, apply reagent directly to each slide, with no two slides sharing the same aliquot of reagent. Examples of commercially available H&E stainers include the VENTANA SYMPHONY (individual slide stainer) and VENTANA HE 600 (individual slide stainer) H&E stainer series manufactured by Roche; the Dako CoverStainer (batch stainer) manufactured by Agilent Technologies; and the Leica ST4020 Small Linear Stainer (batch stainer), Leica ST5020 Multistainer (batch stainer), and Leica ST5010 Autostainer XL series (batch stainer) H&E stainers manufactured by Leica Biosystems Nussloch GmbH. The H&E staining platform 324 is typically used in an assembly line where morphologically stained serial sections of biomarker-stained section(s) are desired.

[0110] The scoring system may further include a laboratory information system (LIS) 330. The LIS 330 typically performs one or more functions selected from the following: recording and tracking processes performed on samples, slides, and images derived from the samples; instructing different components of the scoring system to perform specific processes on the samples, slides, and / or images and to track information about specific reagents applied to the samples and / or slides (lot number, expiration date, dispensed amount, etc.). The LIS 330 typically includes at least one database containing information about the samples, labels associated with the samples, slides, and / or image files (e.g., barcodes (including one-dimensional and two-dimensional barcodes), radio frequency ID (RFID) tags, alphanumeric codes affixed to the samples, etc.); and a communication device that reads labels on the samples or slides and / or communicates information about the slides between the LIS 330 and other components of the immune context scoring system. Thus, for example, a communication device may be located in each of the sample processing station, the automated histochemical stainer 323, the H&E staining platform 324, and the scanning platform 325. When the sample is first processed into sections, information about the sample (such as patient ID, sample type, processing to be performed on the section(s)) is entered into a communication device, and a label is created for each section made from the sample. At each subsequent station, a label is entered into the communication device (such as by scanning a bar code or RFID tag or manually entering an alphanumeric code), and the station is in electronic communication with a database, e.g., to instruct the station or station operator to perform a particular process on the section and / or record the processing being performed on the section.In scanning platform 325, scanning platform 325 may also code each image with a computer-readable label or code that correlates to the section or sample from which the image originates, such that when the image is transmitted to image analysis system 300, the image processing steps performed can be transmitted from the database of LIS 330 to the image analysis system and / or the image processing steps performed on the image by image analysis system 300 are recorded by the database of LIS 330. Commercially available LIS systems useful in the methods and systems of the present invention include, for example, the VENTANA Vantage Workflow system (Roche). [Example]

[0111] Characterization of PD-L1, CD8, CD3, CD68, and PanCK in the tumor microenvironment of gastrointestinal tumors in relation to patient mismatch repair status and anti-PD-1 treatment outcome using I.5Plex IHC and whole slide image analysis IA background technology There is a growing need to understand the tumor microenvironment to guide cancer immunotherapy. Multiplex immunohistochemistry (IHC) techniques enable characterization of the tumor microenvironment by detecting multiple biomarkers and their co-expression on a single slide while preserving the morphological characteristics of the tissue. Extracting information about the co-expression and spatial association of multiple biomarkers requires whole-slide image analysis algorithms tailored to each assay and its intended use. Cancers can evade immune surveillance and eradication through upregulation of the programmed death 1 (PD-1) pathway and its ligand, programmed death ligand 1 (PD-L1), in tumor cells and the tumor microenvironment. Blockade of this pathway with antibodies against PD-1 or PD-L1 has resulted in remarkable clinical responses in some cancer patients.

[0112] Mismatch repair (MMR) deficiency predicts the response of solid tumors to PD-1 blockade. However, not all patients with mismatch repair deficiency respond to PD-1 blockade treatment. To understand the differential response, we evaluated the tumor microenvironment by detecting PD-L1 expression in association with tumor cells and tumor-infiltrating immune cells.

[0113] IB specimens, staining, and image acquisition A cohort of 60 pretreatment (anti-PD-1 pembrolizumab) gastrointestinal tumor specimens with acceptable image and tissue quality for automated analysis was available for this study. After removing non-evaluable responses, 54 cases remained. Table 7 shows the breakdown of responses with respect to mismatch repair deficiency. [Table 7]

[0114] Samples were formalin fixed, paraffin embedded, sectioned and mounted on microscope slides.

[0115] Slides were stained in a multiplex format on a BenchMark ULTRA IHC / ISH automated slide stainer using fluorescent tyramide dye conjugates in a tyramide signal amplification procedure, as set forth in Table 8. [Table 8] The general concept of tyramide signal amplification is described in US Pat. No. 6,593,100. The staining procedure was essentially the same as that described by Zhang I. As shown in Figure 5, the stains were applied sequentially. After deposition of each stain, a heat kill step was applied, which includes the process described by Zhang I. An exemplary stained slide is shown in Figure 6. Serial sections of each sample were also stained for H&E using a VENTANA HE600 automated slide stainer.

[0116] IHC-stained slides were scanned on a Zeiss AxioScan Z1 slide scanner, and H&E-stained slides were scanned on a VENTANA ISCAN COREO slide scanner. All images were exported to DPath, a digital pathology image analysis software suite from Roche.

[0117] IC image annotation and ROI generation The ROIs for the tumor area and the tumor invasion front (when available) were annotated on the images by a pathologist. In addition, necrotic areas and other areas excluded from the analysis were annotated by the pathologist.

[0118] The DPath system automatically annotated epithelial tumor ROIs from aggregates of panCK+ cells and stromal regions. First, the tumor area was subdivided into tiles. For each tile, a panCK mask was first generated by labeling each panCK+ cell and recognizing connections between labeled cells. Next, post-processing was performed on the mask to correct for lymphocyte infiltration due to: (a) The morphological closure operation was performed using a disk-shaped structuring element with a radius of 10 pixels (resolution is 0.325 μm per pixel). (b) Any holes in the mask larger than 80 pixels (8.5 μm 2 ) are closed to generate a “hole-filled PanCK mask.” (c) Converting the hole-filled PanCK mask into polygons. Polygons from all tiles were then assembled to create polygons at the whole-slide level. The whole-slide polygons were then converted to a low-resolution mask (reduced in size by 3^3) and enlarged by 8.8 μm to generate the final mask for "+PanCK+ cell aggregates" at the whole-slide level.

[0119] In addition, an inner peritumoral ROI was automatically generated as the area 0.5 mm inside the tumor from the invasion front, and an outer peritumoral ROI was automatically generated as the area 0.5 mm outside the tumor from the invasion front.

[0120] IC feature calculation and data analysis The following features were calculated for each ROI: area density of all phenotypes; PD-L1 + panCK + Percentage of cells;PD-L1 + CD3 + Percentage of cells;PD-L1 + CD8 + Percentage of cells;PD-L1 + There is also CD3 + CD8 - Percentage of cells with closest PD-L1 + / CD68 + Descriptive statistics of CD8+ cell distance to neighbors; nearest PD-L1 + / panCK + Descriptive statistics of CD8+ cell distance to neighbors; nearest PD-L1 + / CD3 + Descriptive statistics of CD8+ cell distance to neighbors; nearest CD8 + PanCK to adjacent objects + Descriptive statistics of cell distances. CD8 + PD-L1 within 10 and 30 μm of the cell + / panCK + The average number of cells. A complete list of calculated features is in Table 9. [Table 9] TIFF0007802843000033.tif250170TIFF0007802843000034.tif238170TIFF0007802843000035.tif24417 0TIFF0007802843000036.tif250170TIFF0007802843000037.tif243170TIFF0007802843000038.tif95170

[0121] ReliefF feature selection was performed on features to determine the importance of each feature in classifying cases according to anti-PD-1 treatment outcome. The 10 most important features were then selected, and a quadratic discriminant analysis model was fitted to the most important features to predict treatment response. Treatment outcomes were grouped together in three configurations (PD = progressive disease, SD = stable disease, PR = partial response, CR = complete response): PD vs. SD vs. PR + CR, PD vs. SD + PR + CR, and PD + SD vs. PR + CR. Because the majority of the 54 specimens did not have a clear tumor invasion front, these features were excluded from the analysis of the inner and outer regions surrounding the tumor. Analyses for each configuration were performed using 1) all features, 2) tumor-only features, and 3) epithelial and stromal features only. This was done to remove highly correlated features to examine their impact on the final classification. Due to the small sample size, cross-validation was not performed, and reported classification results represent the classification accuracy of the training set.

[0122] Table 10 summarizes the classification accuracy from different configurations and different feature sets. [Table 10] The shaded cells show that for the third configuration (binary response), the multiplexed (Mpx) IHC data reaches 89% accuracy, while mismatch repair (MMR) status alone reaches 70%. Figure 7 shows the importance classification of all features across all cases. The following features were identified as most important: (1) The proportion of PD-L1+ macrophages in the interstitium (2) the proportion of PD-L1+ T helper cells in the stroma, and (3) The proportion of PD-L1+ T cells in the stroma.

[0123] Mismatch repair (MMR) deficiency has previously been shown to predict response to anti-PD-1 treatment. To determine whether multiplex (Mpx) IHC data could identify which MMR-deficient cases would respond to anti-PD-1 treatment, an analysis was performed focusing only on mismatch repair-deficient cases. Figure 8 shows the importance ranking of features in the analysis configuration for MMR-deficient cases. Figure 9 shows images of stained samples from patients who showed a complete response to treatment, while Figure 10 shows images of patients who developed progressive disease after treatment. Table 11 shows that using epithelium and stroma, Mpx IHC data can reach a classification accuracy of 92% in separating PD+SD from PR+CR. [Table 11] Table 12 shows the confusion matrix for the classification. Of the 37 failure cases, 34 were correctly identified by Mpx IHC data for responders and non-responders, with only 3 cases misclassified. In contrast, 53.7% of MMR failures responded to anti-PD-1 treatment. [Table 12]

[0124] II. Exploring the spatial interaction of PD-1 / PD-L1 to predict response to immunotherapy in gastrointestinal tumors by quantitative image analysis of automated multiplexed IHC II.A. Background technology Blockade of the PD-1 / L1 axis is an effective immunotherapy for some cancer patients. However, identifying predictive biomarkers for patient selection remains a major challenge. Current clinical practice based on PD-L1 expression levels by IHC and emerging biomarkers—tumor mutation burden and mismatch repair (MMR) status—is inadequate. Predictive value is limited due to the variable strength of the relationship between tumor type and tumor type. Recent studies suggest that the spatial arrangement and interactions between cancer cells and immune cells influence patient prognosis, survival, and response to treatment. Multiplex immunohistochemistry (IHC) tissue staining can provide a detailed characterization of the tumor microenvironment based on specific tumor and immune molecular features.

[0125] II.B. Samples, Staining, and Image Acquisition A cohort of 50 pre-treatment (anti-PD-1 pembrolizumab) patient gastrointestinal tumor specimens with acceptable image and tissue quality for automated analysis was available for this study. Table 13 shows the response breakdown. [Table 13] Samples were formalin fixed, paraffin embedded, sectioned and mounted on microscope slides.

[0126] An overview of the staining and image analysis is shown in Figure 11. Slides were stained in a multiplex format for PanCK, PD-L1, PD1, CD8, LAG3 on a BenchMark ULTRA IHC / ISH automated slide stainer using fluorescent tyramide dye conjugates in a tyramide signal amplification procedure, as identified in Table 14. [Table 14] The stains were applied sequentially as in Example I. After each stain deposition, a heat kill step was applied, including the process described by Zhang (I). Serial sections from each specimen were also stained for H&E using a VENTANA HE600 automated slide stainer.

[0127] Whole slides were scanned with a Zeiss AXIO Z1 scanner, and tumor areas were annotated by a pathologist. Image analysis was performed using Halo Hi-Plex software. MatLab computer vision, image processing, and machine learning toolboxes were used to (a) reconstruct graphs of each cell type from their spatial locations in the Halo output csv files, (b) generate quantitative quantities characterizing the interactions between different cellular signals, (c) classify and mine the most predictive feature combinations associated with anti-PD-1 response, and (d) build and optimize predictive models based on the selected features.

[0128] II.C. Image Annotation and ROI Creation Tumor area ROIs and tumor invasion fronts (when available) were annotated on the images by a pathologist. The DPath system automatically annotated epithelial tumor ROIs from aggregates of panCK+ cells and stromal regions as described in Example I.

[0129] II.C. Feature Computation and Data Analysis Each of the features in Table 16 was analyzed for each image and classified by both ReliefF and Random Forest. [Table 16] TIFF0007802843000045.tif255170TIFF0007802843000046.tif255170TIFF0007802843000047.tif255170TIFF0007802843000048.tif243170

[0130] The ranking of the top 15 features from each tier of ReliefF and Random Forest is shown in Figures 12 and 13, respectively. Of the 190 features analyzed, both ReliefF and Random Forest rank the following feature in the top two for predicting response to treatment: "maximum number of CD8+ / PD-1 low intensity cells within 20 μm of PD-L1+ cells in epithelial tumors." Quadratic discriminant analysis (QDA) using 5-fold cross-validation yields a prediction accuracy of 85%. When combined with "mean number of PD-1+ cells within a 20 μm radius of PD-L1+ cells" and "maximum Lag3+ intensity of CD8+ cells," the accuracy reaches 90.2%, independent of MMR status (see Figure 14 and Tables 17 and 18). [Table 17] [Table 18] Other features have lower accuracy (e.g., 60-70%).

[0131] Figures 14A-14C show: (a) predicted spatial distribution of PD-1+ cells within a 10 μm radius of a PD-L1+ cell; (b) predicted spatial distribution of PD-L1+ cells within a 10 μm radius of a PD-L1+ cell; + PD-1 within a 20 μm radius of the cell 低強度 CD8 + predicted mean number of cells, (c) CD8 + Lag3 + Predicted maximum Lag3 intensity in cells, and (d) scatter plot showing (a) versus (b).

[0132] Figure 15 shows exemplary IHC images of non-responders, along with graphical reconstructions of PD-L1 + cells (gray dots), PD-1 within 20 μm of PD-L1+ cells 低 cells (white dots), and PD-1 within 10 μm of PD-L1+ cells. +The location of the cells (black dots) is shown. As can be seen, there are similar PD-L1+ cells in non-responders and responders. However, responders show a significant difference in the PD-L1+ cells in both responders and non-responders. + CD8+PD-1 within 20 μm of the cell 低 In addition, the presence of PD-1 cells within 10 μm of PD-L1+ cells was higher in responders than in non-responders. + The cells are more evenly distributed.

[0133] II.D. Survival analysis Overall survival (OS) data were available for a subset of 46 patients. Survival analyses were performed for each of the variables in Table 14. The following variables significantly predicted survival benefit: (a) Lag3 in the panCK-negative region + / CD8 + Cell and total CD8 + (b) Lag3 in the panCK-negative region - / CD8 + Cells and CD8 + Ratio of cells to the number of cells, (c) Lag3 + / panCK - (d) Number of Lag3-positive cells divided by the number of panCK-negative cells in the panCK-negative area; (e) CD8 + (f) Maximum Lag3 intensity in cells, (g) Number of Lag3+ cells in panCK-positive areas, and (h) PD-L1 + / panCK + PD-1 within a 10 μm radius of the cell 中 / mean number of CD8+ cells, (h) PD-L1 + / panCK + PD-1 within a 20 μm radius of the cell 中 / mean number of CD8+ cells, (i) PD-L1 + / CD8 + PD-1 within a 20 μm radius of the cell 中 / CD8+ cell count variance, (j)PD-L1 + / panCK + PD-1 within a 20 μm radius of the cell 中 / CD8+ cell count variance, (k)PD-L1 + PD-1 within a 10 μm radius of the cell 低 / CD8+ cell count variance, (l) PD-L1 + / CD8 + PD-1 within a 20 μm radius of the cell 中 / maximum number of CD8+ cells, and (m)PD-L1 + PD-1 within a 20μ radius from the cell 中 / maximum number of CD8+ cells. For each feature, the cohort was divided into two groups using the median of the feature's distribution as the cutoff. Kaplan-Meier survival curves are depicted in Figures 16A-16M.

[0134] IV. Exemplary Image Analysis System and Clinical Workflow In clinical practice, the scoring function can be integrated into prognostic analysis to make treatment decisions. After tumor biopsy or surgical resection, a representative tissue block representing a cross-section of the tumor from the patient's tumor specimen is selected for analysis. At least three 4 μm thick sections are cut from this tissue block and transferred to glass slides. The sections are stained as follows: 1. IHC negative control (i.e., a staining protocol using primary antibody diluent instead of primary antibody), 2. Multiplex IHC including at least PD-L1, PD1, CD8, and LAG3 primary antibodies, and 3.H&E. All sections are scanned with a slide scanner. Images are transferred to a digital pathology system along with slide metadata. Slide metadata includes tumor sample identification and slide staining (H&E, IHC, or negative control), which can be either entered by the user when scanning the slide or automatically obtained from a laboratory information system. The digital pathology system uses the slide metadata to trigger automatic calculation of one or more features from Table 9 or Table 16. For example, the features include at least the maximum number of CD8+ / PD-1 low intensity cells within 20 μm of PD-L1+ cells in epithelial tumors, and optionally further include "the average number of PD-1+ cells within a 20 μm radius of PD-L1+ cells" and "the maximum value of Lag3+ intensity on CD8+ cells."

[0135] In a digital pathology system, a pathologist or expert observer opens a digital image of an H&E slide in viewing software and identifies the relevant morphological area to be scored. The user then annotates the tumor using annotation tools provided by the viewing software. Typically, a tumor is defined by creating one or more contours and identifying them as the tumor's border. To do this, the user creates additional contours that intersect with the tumor's border. The intersections define the start and end of sections on the tumor's border that are involved in the invasive process. The new contours are identified as the invasive margin.

[0136] The user then triggers automatic transfer of the annotations onto adjacent IHC slides. The digital pathology system provides a registration function that transfers annotations onto adjacent slides, taking into account the position, orientation, and local deformation of the tissue section. The user opens the IHC slide image in the viewer software and controls the location of the automatically registered annotations. The viewer software provides tools for modifying and editing the annotations, if necessary. Editing functions include moving the annotations, rotating the annotations, and locally modifying the annotation outline. The user further inspects the IHC slide image in the viewer software for tissue, staining, or image artifacts. The user delineates such artifact areas with annotations and identifies the artifacts to exclude from analysis.

[0137] In the digital pathology system, a user can select one or more IHC slides and trigger the generation of a report. The user can obtain a quality control report that can include the following components: 1. Low- to medium-resolution images showing all tissues on the slide 2. Overlay contours and / or transparent color regions on the same low- to medium-resolution image to indicate morphological regions of interest, such as tumor margins, and also overlay regions annotated for exclusion from analysis on this image. 3. The same low- to medium-resolution images with automatically generated small rectangular markers indicating the location of the high-resolution FOV for quality control. 4. High-resolution FOV 5. Each of the high-resolution FOVs overlaid with markers indicating the presence of each cell phenotype as determined by automated cell counting. As an option, markers indicating morphological regions of interest and cells from the automated cell count can also be displayed in the viewer software.

[0138] The user reviews the quality control data and decides to accept or reject the case. For accepted cases, the digital pathology system reports quantitative readouts that are passed to a scoring module. These quantitative readouts can include: 1. Area (mm) of each morphological region of interest 2 ). 2. Number of cells in each morphological region of interest. 3. Descriptive statistics describing the spatial distribution of and / or spatial relationships between cells of different phenotypes in each morphological region of interest. Additional information about the sample can be input into the digital pathology system, such as MMR status, the subject's previous exposure to therapy (such as chemotherapy, radiotherapy, and / or targeted therapy), tumor score using the TNM staging system, and / or overall tumor stage, clinical variables (patient age, tumor side, number of collected lymph nodes, and gender), and the additional information can be used by the system to, for example, select and apply an appropriate scoring function to the image. Additionally or alternatively, a user can select an appropriate scoring function based on such criteria or other criteria. The scoring module calculates a score based on the extracted features, and the score can be reported as a raw numerical value. Additionally, a binning function can be applied to the score to assign patients to risk bins (e.g., by applying a cutoff between populations based on "likely to respond" or "unlikely to respond" to checkpoint inhibitors) and / or population stratification bins (e.g., quartiles or decile bins based on score), and / or feature selection functions to stratify the scores. The clinician reviews the report and discusses the results with the patient who makes a decision about the results based on the clinical pathologist, which can then be used to make a treatment decision for the patient.

[0139] References Barrera et al.,Computer-extracted features relating to spatial arrangement of tumor infiltrating lymphocytes to predict response to nivolumab in non-small cell lung cancer(NSCLC).ASCO Annual Meeting 2018:Abstract#:12115. Le et al., PD-1 Blockade in Tumors with Mismatch-Repair Deficiency, N Engl J Med.2015 Jun 25;372(26):2509-20 (``Le(I)''). Le et al., Mismatch-repair deficiency predicts response of solid tumors to PD-1 blockade, Science, 10.1126 / science.aan6733 (2017) (“Le(II)”). Li & Tian, ​​Development of small-molecule immune checkpoint inhibitors of PD-1 / PD-L1 as a new therapeutic strategy for tumor immunotherapy, J. of Drug Targeting, DOI: 10.1080 / 1061186X.2018.1440400 (published online February 20, 2018). Nordic Immunohistochemical Quality Control, CK-Pan run 47 (2016), available at http: / / www.nordiqc.org / downloads / assessments / 82_85.pdf (last accessed October 4, 2018) (“NordiQC”). Topalian, Suzanne L., et al. "Safety, activity, and immune correlates of anti-PD-1 antibody in cancer." New England Journal of Medicine 366.26(2012):2443-2454。 Wang, et al., Prediction of recurrence in early stage non-small cell lung cancer using computer extracted nuclear features from digital H&E images., Scientific Reports 7.1(2017):13543。 Woodcock-Mitchell et al. Immunolocalization of keratin polypeptides in human epidermis using monoclonal antibodies. J Cell Biol. 1982;95(2):580-588。 Yi et al., Biomarkers for predicting efficacy of PD-1 / PD-L1 inhibitors, Mol Cancer. 2018;17:129. (Published online on August 23, 2018) Zhang et al. "Automated 5-plex fluorescent immunohistochemistry with tyramide signal amplification using antibodies from the same species" J Immunother Cancer. 2015;3(Suppl 2):P111("Zhang(I)"). Zhang et al.「An automated 5-plex fluorescent immunohistochemistry enabled characterization of PD-L1 expression and tumor infiltrating immune cells in lung and bladder cancer specimens.」Cancer Research 2016,76(14 Supplement):5117(「Zhang(II)」)。

Claims

1. 1. A PD-1 axis-directed therapeutic agent for use in a method of treating a patient having a tumor, the method comprising administering the PD-1 axis-directed therapeutic agent to the patient if a level of the tumor characteristic indicative of the tumor being likely to respond to the PD-1 axis-directed therapeutic agent has been identified in the patient, wherein identifying the level of the tumor characteristic indicative of the tumor being likely to respond to the PD-1 axis-directed therapeutic agent comprises scoring a tumor sample obtained from the patient for likelihood of response to the PD-1 axis-directed therapeutic agent, wherein the scoring comprises: (a) obtaining a digital image of a tumor section from the tumor sample, wherein the tumor section is stained with a multiplex affinity histochemical stain for each of PD-L1, CD8, CD3, CD68, and an epithelial marker (EM); (b) extracting features from a region of interest (ROI) within the image; The feature is (a) Interstitial PD-L1 ＋ / CD3 ＋ Cell count and interstitial CD3 ＋ Ratio to the total number of cells, (b) Stromal PD-L1 ＋ / CD8 ＋ Number of cells and interstitial CD8 ＋ Ratio to the total number of cells, (c) Interstitial PD-L1 ＋ / CD68 ＋ Cell number and interstitial CD68 ＋ Ratio to the total number of cells, (d) Stromal PD-L1 ＋ / CD3 ＋ / CD8 － Cell count and interstitial CD3 ＋ CD8 － Ratio to the total number of cells, (e) Epithelial CD8 ＋ PD-L1 closest to the cell ＋ / CD68 ＋ the average distance between cells, (f) epithelial PD-L1 within the ROI ＋ / CD3 ＋ / CD8 － cell density, (g) Epithelial PD-L1 ＋ / CD3 ＋ Cell counts and epithelial CD3 ＋ Ratio to the total number of cells, (h) the ratio of the number of epithelial PD-L1+ / CD8+ cells within an EM+ ROI or tumor ROI to the total number of epithelial CD8+ cells within said EM+ ROI or tumor ROI; (i) the ratio of the number of epithelial PD-L1+ / CD68+ cells within the ROI to the total number of epithelial CD68+ cells within the ROI; (j) the ratio of the number of epithelial PD-L1+ / CD3+ / CD8− cells in the ROI to the total number of epithelial CD3+ / CD8− cells in the ROI; (k) the ratio of the number of epithelial CD3+ / CD8− cells within the EM+ ROI or tumor ROI to the total number of epithelial CD3+ cells within the EM+ ROI or tumor ROI; (l) PD-L1 ＋ / CD8 ＋ PD-1 within a radius of 20 μm from the cell 低 / CD8 ＋ The maximum number of (m) CD8 ＋ Maximum cellular PD-L1 intensity, (n) PD-1 in the epithelial region ＋ / PD-L1 － / Lag3 ＋ / CD8 ＋ Cells and CD8 ＋ The ratio of cells to the number of cells (o) PD-L1 ＋ PD-1 within a radius of 20 μm from the cell ＋ the spatially distributed number of cells, (p) the maximum value of all cellular PD-L1 intensities; (q) EM ＋ the number of Lag3-positive cells within the ROI, (r) PD-L1 in EM-ROI ＋ / EM － cell density, (s)EM ＋ PD-L1 within ROI ＋ number of cells, (t) PD-L1 in EM-ROI ＋ number of cells, (u) PD-1 in EM-ROI ＋ number of cells, (v) PD-L1 ＋ PD-1 within a radius of 20 μm from the cell 低 / CD8 ＋ The maximum number of cells, (w) PD-L1 ＋ PD-1 within a radius of 20 μm from the cell 低 / CD8 ＋ The average number of cells, (x) CD8 ＋ Lag3 ＋ Maximum Lag3 intensity from the cell, (y)EM ＋ CD8 in ROI ＋ number of cells, (z) PD-L1 ＋ / CD8 ＋ PD-1 within a radius of 20 μm from the cell 低 / CD8 ＋ Variance in cell numbers, (aa) PD-L1 ＋ PD-1 closest to the cell ＋ average distance to cells, (bb) PD-L1 ＋ / EM ＋ PD-1 within a radius of 20 μm from the cell 低 / CD8 ＋ The maximum number of cells, (cc) PD-L1 ＋ PD-1 closest to the cell ＋ standard deviation of distance to cells, (dd)EM ＋ Lag3 within ROI ＋ / CD8 ＋ Cell count and CD8 ＋ Ratio to the number of cells, (ee)EM ＋ PD-L1 within ROI ＋ / CD8 ＋ Cell count and CD8 ＋ Ratio to the number of cells, (ff)PD-L1 ＋ PD-1 within a radius of 10 μm from the cell 低 / CD8 ＋ The average number of cells, (gg)PD-L1 ＋ PD-1 within a radius of 10 μm from the cell 低 / CD8 ＋ Variance in cell numbers, (hh) All PD-1 ＋ the mean PD-1 intensity from the cells; (ii) All Lag3 ＋ The minimum Lag3 intensity from the cells, (jj)EM ＋ PD-L1 within ROI ＋ / EM ＋ cell density, (kk)PD-L1 ＋ PD-1 within a radius of 10 μm from the cell 低 / CD8 ＋ The maximum number of cells, (ll) EM ＋ PD-1 within the ROI ＋ number of cells, (mm) PD-1 in EM-ROI ＋ / PD-L1 － / Lag3 ＋ / CD8 ＋ Cells and CD8 ＋ The ratio of cells to the number of cells (nn) CD8 ＋ / Lag3 ＋ Maximum Lag3 intensity from a cell, (oo)EM － Lag3 within ROI ＋ / CD8 ＋ Cell count and CD8 ＋ the ratio to the number of cells, and (pp)EM ＋ Number of Lag3-positive cells within the ROI A PD-1 axis-directed therapeutic agent selected from the group consisting of:

2. The PD-1 axis-directed therapeutic agent for use according to claim 1, wherein the PD-1 axis-directed therapeutic agent is a PD-1-specific monoclonal antibody or a PD-L1-specific monoclonal antibody.

3. 2. The PD-1 axis-directed therapeutic agent for use according to claim 1, wherein the PD-1 axis-directed therapeutic agent is selected from the group consisting of pembrolizumab, nivolumab, atezolizumab, avelumab, durvalumab, cemiplimab, tislelizumab, and LY3300054.

4. The feature quantity is (i) CD8 within the ROI ＋ Cells and the closest PD-L1 ＋ / CD68 ＋ the average distance between cells, (ii) CD8 within the ROI ＋ Cells and the closest PD-L1 ＋ / CD3 ＋ the average distance between cells, (iii) The epithelial cells within the ROI and the closest CD8 ＋ the average distance between cells, (iv) CD8 ＋ PD-L1 within the ROI within 10 μm of the cell ＋ number of epithelial cells, (v) CD8 ＋ PD-L1 within the ROI within 30 μm of the cell ＋ number of epithelial cells, (vi) CD8 within the ROI within 10 μm of an epithelial cell ＋ number of cells, (vii) CD8 within the ROI within 30 μm of an epithelial cell ＋ number of cells, (viii) PD-L1 within the ROI ＋ / CD3 ＋ cell density, (ix) PD-L1 within the ROI ＋ / CD3 ＋ / CD8 － cell density, (x) PD-L1 within the ROI ＋ / CD8 ＋ cell density, (xi) PD-L1 within the ROI ＋ / CD68 ＋ cell density, (xii) PD-L1 within the ROI ＋ epithelial cell density, (xiii) CD3 within the ROI ＋ cell density, (xiv) CD8 within the ROI ＋ cell density, (xv) CD68 within the ROI ＋ cell density, (xvi) the density of epithelial cells within the ROI; (xvii) PD-L1 within the ROI ＋ the ratio of the area occupied by epithelial cells to the total area of ​​the ROI; (xviii) the ratio of the area occupied by epithelial cells within the ROI to the total area of ​​the ROI; (xix) PD-L1 within the ROI ＋ the ratio of the number of epithelial cells to the total number of epithelial cells within the ROI; (xx) PD-L1 within the ROI ＋ / CD3 ＋ The number of cells and CD3 ＋ Ratio to the total number of cells, (xxi) PD-L1 within the ROI ＋ / CD3 ＋ / CD8 － The number of cells and CD3 ＋ / CD8 － Ratio to the total number of cells, (xxii) PD-L1 within the ROI ＋ / CD8 ＋ The number of cells and CD8 ＋ Ratio to the total number of cells, (xxiii) PD-L1 within the ROI ＋ / CD68 ＋ The number of cells and CD68 in the ROI ＋ Ratio to the total number of cells, (xxiv) PD-L1 within the ROI ＋ / CD3 ＋ / CD8 － The number of cells and CD3 ＋ / CD8 － Ratio to the total number of cells, (xxv) CD3 within the ROI ＋ / CD8 － The number of cells and CD3 ＋ the ratio to the total number of cells, and (xxvi) CD3 ＋ Total area occupied by cells is selected from the group consisting of The scoring (c) applying a scoring function to a feature vector including the features to generate a score indicative of the likelihood that the tumor will respond to a PD-1 axis-directed therapy; wherein the scoring function is derived from a cohort of patients for which outcome data is available prior to treatment with the PD-1 axis-directed therapy.

5. The ROIs are classified into tumor ROI, stromal ROI, epithelial marker positive (EM) ROI, and ＋ ) ROI, epithelial marker negative (EM － 5. The PD-1 axis-directed therapeutic agent for use according to claim 4, wherein the PD-1 axis-directed therapeutic agent is selected from the group consisting of a peritumoral inner (PI) ROI, a peritumoral outer (PO) ROI, and a peritumoral region (PR) ROI.

6. The ROI is the stromal ROI or the EM － and the feature vector is (xx) PD-L1 within the ROI ＋ / CD3 ＋ The number of cells and CD3 ＋ Ratio to the total number of cells, (xxii) PD-L1 within the ROI ＋ / CD8 ＋ The number of cells and CD8 ＋ Ratio to the total number of cells, (xxiii) PD-L1 within the ROI ＋ / CD68 ＋ The number of cells and CD68 in the ROI ＋ the ratio to the total number of cells, and (xxiv) PD-L1 within the ROI ＋ / CD3 ＋ / CD8 － The number of cells and CD3 ＋ / CD8 － Ratio to total number of cells The PD-1 axis-directed therapeutic agent for use according to claim 5, comprising one or more features selected from the group consisting of:

7. The feature vector is within the stromal ROI or the EM － PD-L1 within ROI ＋ / CD68 ＋ The number of cells and the number of cells in the stromal ROI or the EM － CD68 in the ROI ＋ Ratio to the total number of cells, within the stromal ROI or the EM － PD-L1 within ROI ＋ / CD3 ＋ / CD8 － The number of cells and the number of cells in the stromal ROI or the EM － CD3 in ROI ＋ / CD8 － the ratio to the total number of cells, and within the stromal ROI or the EM － PD-L1 within ROI ＋ / CD3 ＋ The number of cells and the number of cells in the stromal ROI or the EM － CD3 in ROI ＋ Ratio to total number of cells 7. The PD-1 axis-directed therapeutic agent for use according to claim 6, comprising each of:

8. The ROI is the tumor ROI or the EM ＋ and the feature vector is (i) Said EM ＋ CD8 in ROI or tumor ROI ＋ PD-L1 closest to the cell ＋ / CD68 ＋ the average distance between cells, (ix) PD-L1 within the ROI ＋ / CD3 ＋ / CD8 － density, (xx) PD-L1 within the ROI ＋ / CD3 ＋ The number of cells and CD3 ＋ Ratio to the total number of cells, (xxii) the EM ＋ PD-L1 within ROI or tumor ROI ＋ / CD8 ＋ Number of cells and EM ＋ CD8 in ROI or tumor ROI ＋ Ratio to the total number of cells, (xxiii) PD-L1 within the ROI ＋ / CD68 ＋ The number of cells and CD68 in the ROI ＋ Ratio to the total number of cells, (xxiv) PD-L1 within the ROI ＋ / CD3 ＋ / CD8 － The number of cells and CD3 ＋ / CD8 － the ratio to the total number of cells, and (xxv) the EM ＋ CD3 within the ROI or within the tumor ROI ＋ / CD8 － Number of cells and EM ＋ CD3 within the ROI or within the tumor ROI ＋ Ratio to total number of cells The PD-1 axis-directed therapeutic agent for use according to claim 5, comprising one or more features selected from the group consisting of:

9. the ROI is derived from a digital image of a morphologically stained section of the tumor sample; the morphologically stained section and the multiple affinity histochemically stained sample are serial sections; or 5. The PD-1 axis-directed therapeutic for use according to claim 4, wherein the ROI is identified by a user in the digital image of the morphologically stained section and automatically registered to the digital image of the multi-affinity histochemically stained section.

10. 10. The PD-1 axis-directed therapeutic agent for use according to any one of claims 4 to 9, wherein the scoring function is derived from a modeling function selected from the group consisting of quadratic discriminant analysis (QDA), linear discriminant analysis (LDA), support vector machine (SVM), and artificial neural network (ANN).

11. the scoring function is a QDA model that is fitted to the selected features to predict response to treatment, and the treatment outcome used to fit the QDA model is: Progressive disease (PD) vs. stable disease (SD) vs. partial response (PR) + complete response (CR), PD vs. SD+PR+CR, and PD+SD vs. PR+CR 11. The PD-1 axis-directed therapeutic for use according to claim 10, grouped together in a configuration selected from the group consisting of:

12. comparing the score obtained according to any one of claims 4 to 11 with a predetermined cut-off value; and if the comparison indicates that the patient is likely to respond to the PD-1 axis-directed therapy, administering the PD-1 axis-directed therapy to the patient.

Images