Systems and methods for quantification of digitized pathology slides

Abstract

A method for determining a raw score for a pathology slide from a tissue sample includes receiving, by a regression system, a plurality of first slide features corresponding to the pathology slide; calculating, by the regression system, one or more second slide features corresponding to the pathology slide based on the plurality of first slide features; and determining, by the regression system, a raw score based on one or more features of an accumulated feature set including the plurality of first slide features and the one or more second slide features.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to and benefit of U.S. Provisional Application No. 63 / 396,142, filed August 8, 2022 ("SYSTEM AND METHOD FOR QUANTIFICATION OF DIGITIZED PATHOLOGY SLIDES"), the entire contents of which are incorporated herein by reference.

[0002] Field One or more aspects of some embodiments according to the present disclosure relate to quantifying pathology slides. [Background technology]

[0003] background The human immune system utilizes T cells to help fight infections and other diseases, including cancer. PD-L1 is a transmembrane protein that suppresses immune responses by binding to two receptors on T cells, programmed cell death-1 (PD-1) and B7.1. One approach to fighting cancer is to block the PD-L1 protein, which can prevent cancer cells from inactivating T cells via both PD-1 and B7.1. PD-L1 testing can help doctors determine whether a patient is likely to benefit from anticancer drugs known as immune checkpoint inhibitors. These inhibitors prevent PD-1 / PD-L1 interaction from occurring. Thus, T cells can proceed with their attack on tumor cells without receiving a "stop" signal from the PD-L1 protein.

[0004] The PD-L1 (SP142) assay is an immunohistochemistry (IHC) assay that utilizes an anti-PD-L1 rabbit monoclonal primary antibody to recognize the programmed cell death ligand 1 (PD-L1) protein. This assay was developed to identify patients most likely to respond to treatment with immune checkpoint inhibitors. However, studies have shown that there is significant variability between pathologists in the assessment of PD-L1 SP142 immunohistochemistry as a percentage, as well as in PD-L1 SP142 status (positive vs. negative). Two factors may contribute to this high interobserver variability: 1) the amplification of the assay, and 2) the cumbersome and complex manual scoring guidelines.

[0005] The above information disclosed in this Background section is intended to enhance understanding of the background art only, and therefore, the information described in this Background section does not necessarily constitute prior art. Summary of the Invention

[0006] overview Aspects of embodiments of the present disclosure are directed to a cell-based scoring system that utilizes artificial intelligence (AI) to quantify digitized slides from patient samples (e.g., PD-L1 SP142 digitized slides) and predict an overall slide score percentage and therefore patient status (e.g., whether the patient is likely to benefit from a particular cancer drug).

[0007] According to some embodiments of the present disclosure, a method for determining a raw score of a pathology slide from a tissue sample is provided, the method including: receiving, by a regression system, a plurality of first slide features corresponding to the pathology slide; calculating, by the regression system, one or more second slide features corresponding to the pathology slide based on the plurality of first slide features; and determining, by the regression system, a raw score based on one or more features of an accumulated feature set including the plurality of first slide features and the one or more second slide features.

[0008] In some embodiments, the pathology slide is stained with PD-L1 SP142.

[0009] In some embodiments, the plurality of first slide features comprises at least one of an area of a tumor region on the pathology slide, a number of stained immune cells on the pathology slide, a number of unstained immune cells on the pathology slide, a number of stained tumor cells on the pathology slide, a number of unstained tumor cells on the pathology slide, a number of other cells on the pathology slide, and a total number of cells on the pathology slide.

[0010] In some embodiments, calculating the one or more second slide features includes calculating at least one of a field of view (FOV) area score, a cell area score, and a cell number score.

[0011] In some embodiments, the FOV area score is expressed as:

[0012] TIFF2025526652000002.tif15170

[0013] where the mean size of IC+ cells represents the mean size of stained immune cells, and the number of IC+ cells represents the number of stained immune cells in the pathology slide.

[0014] In some embodiments, the cell area score is expressed as:

[0015] TIFF2025526652000003.tif15170

[0016] where the average size of IC+ cells represents the average size of stained immune cells, the number of IC+ cells represents the number of stained immune cells in the pathology slide, and the tumor area represents the area of the tumor region corresponding to the pathology slide.

[0017] In some embodiments, the cell count score is expressed as:

[0018] TIFF2025526652000004.tif14170

[0019] where the number of IC+ cells represents the number of stained immune cells in the pathology slide, and the total number of cells represents the total number of cells in the pathology slide.

[0020] In some embodiments, determining the raw score includes providing one or more features of the accumulated feature set to a trained regression model configured to correlate raw score values to values of the one or more features, and estimating, by the trained regression model, a raw score corresponding to the one or more features.

[0021] In some embodiments, the regression system includes a trained machine learning model configured to correlate one or more features of the accumulated feature set to a raw score.

[0022] In some embodiments, the trained machine learning model comprises one of a K-nearest neighbor (KNN) model, a support vector machine (SVM) model, a random forest (RF) model, and a multi-layer perceptron (MLP) model.

[0023] In some embodiments, the method further comprises comparing the raw score to a threshold to determine the effectiveness of a treatment for the patient associated with the tissue sample.

[0024] In some embodiments, the method further includes receiving, by a classifier, an image of a pathology slide; classifying, by the classifier, each cell of the plurality of cells captured in the image by identifying a cell of each cell of the plurality of cells and assigning a cell type from among the plurality of cell types to each of the plurality of cells; and generating, by the classifier, a plurality of first slide features based on the classification of each cell.

[0025] In some embodiments, the classifier comprises a convolutional neural network.

[0026] According to some embodiments of the present disclosure, there is provided a method for determining a raw score of a pathology slide from a tissue sample, the method including: receiving an image of the pathology slide by a cell-based scoring system including processing circuitry and a memory; classifying each cell of a plurality of cells captured in the image by providing the image to a cell-based scoring system classifier, the classifier being configured to identify each cell of the plurality of cells and assign a cell type from among a plurality of cell types to each of the plurality of cells; generating, by the cell-based scoring system, a plurality of first slide features based on the classification of each cell; and determining, by the cell-based scoring system, a raw score based on one or more features of an accumulated feature set including the plurality of first slide features.

[0027] In some embodiments, generating the plurality of first slide features includes counting the number of cells assigned to each cell type of the plurality of cells, and generating the plurality of first slide features based on the number of cells assigned to each cell type.

[0028] In some embodiments, the method further includes receiving, by a cell-based scoring system, an area of the tumor region corresponding to the image, wherein the accumulated feature set further includes the area of the tumor region.

[0029] In some embodiments, the plurality of first slide features comprises at least one of: a number of stained immune cells in the pathology slide; a number of unstained immune cells in the pathology slide; a number of stained tumor cells in the pathology slide; a number of unstained tumor cells in the pathology slide; a number of other cells in the pathology slide; and a total number of cells in the pathology slide.

[0030] In some embodiments, the method further includes calculating, by the cell-based scoring system, one or more second slide features corresponding to the pathology slide based on the plurality of first slide features, wherein the accumulated feature set further includes the one or more second slide features.

[0031] In some embodiments, calculating the one or more second slide features includes calculating at least one of a field of view (FOV) area score, a cell area score, and a cell number score.

[0032] In some embodiments, determining the raw score includes providing one or more features of the accumulated feature set to a trained regression model configured to correlate raw score values to values of the one or more features, and estimating, by the trained regression model, a raw score corresponding to the one or more features.

[0033] In some embodiments, the method further comprises comparing the raw score to a threshold to determine the effectiveness of a treatment for the patient associated with the tissue sample.

[0034] According to some embodiments of the present disclosure, there is provided a cell-based scoring system for determining a raw score of a pathology slide from a tissue sample, the cell-based scoring system including: a classifier including a convolutional neural network configured to receive an image of the pathology slide, identify each cell of a plurality of cells captured in the image, classify each cell of the plurality of cells by assigning a cell type from among a plurality of cell types to each of the plurality of cells, and generate a plurality of first slide features based on the classification of each cell; a cell-based feature generator configured to calculate one or more second slide features corresponding to the pathology slide based on the plurality of first slide features; and a regressor configured to determine the raw score based on one or more features of an accumulated feature set including the plurality of first slide features and the one or more second slide features. [Brief explanation of the drawings]

[0035] Non-limiting and non-exhaustive embodiments according to the present disclosure are described with reference to the following figures, in which like reference numerals refer to like parts throughout the various views unless otherwise specified.

[0036] [Figure 1] FIG. 1 is a flow diagram illustrating various actions that may occur in a pathological situation or environment, according to some embodiments.

[0037] [Figure 2A] FIG. 1 illustrates the process of manually scoring a PD-L1 assay according to an example of the prior art. [Figure 2B] FIG. 1 illustrates the process of manually scoring a PD-L1 assay according to an example of the prior art. [Figure 2C] FIG. 1 illustrates the process of manually scoring a PD-L1 assay according to an example of the prior art. [Figure 2D] FIG. 1 illustrates the process of manually scoring a PD-L1 assay according to an example of the prior art.

[0038] [Figure 3A] FIG. 1 is a block diagram illustrating a cell classifier, according to some embodiments of the present disclosure.

[0039] [Figure 3B] 1 is a labeled image identifying different types of cells detected by a cell classifier, according to some embodiments of the present disclosure.

[0040] [Figure 4] FIG. 1 is a block diagram of a cell-based scoring system including a cell classifier and a regressor, according to some embodiments of the present disclosure.

[0041] [Figure 5] FIG. 1 is a flow diagram illustrating a process for determining a raw score for a pathology slide from a tissue sample using cell-based classification data corresponding to the pathology slide, according to some embodiments of the present disclosure.

[0042] [Figure 6] FIG. 1 is a flow diagram illustrating a process for determining a raw score of a pathology slide from a tissue sample according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0043] Detailed Description Aspects of some exemplary embodiments are described in more detail below with reference to the accompanying drawings, in which like reference numerals refer to like elements throughout. However, the present invention may be embodied in a variety of different forms and should not be construed as limited to only the embodiments illustrated herein. Rather, these embodiments are provided as examples so that this disclosure will be thorough and complete, and will fully convey the aspects and features of the present invention to those skilled in the art. Accordingly, processes, elements, and techniques that are not necessary for those skilled in the art to fully understand the aspects and features of the present invention may not be described. Unless otherwise noted, like reference numerals indicate like elements throughout the accompanying drawings and specification, and therefore, descriptions thereof will not be repeated. In the drawings, relative sizes of elements, layers, and regions may be exaggerated for clarity.

[0044] Pathology is a medical field that attempts to facilitate the diagnosis and treatment of disease by studying patient tissue, cell, and fluid samples. In many applications, tissue samples are collected from patients and processed into a form that can be analyzed by a physician (e.g., a pathologist), often under magnification, to diagnose and characterize associated disease states based on the tissue sample.

[0045] 1 is a flow diagram illustrating various operations that may occur in a pathology environment or system 100. For example, if a treating physician or healthcare provider identifies a patient for whom analysis of a tissue or bodily fluid sample may be beneficial in diagnosing or treating a medical condition, the tissue or bodily fluid sample may be collected in operation 102. Patient identification information may be collected and matched to the patient's sample, and the sample may be placed in a sterile container and / or collection medium for further processing.

[0046] The sample may then be transported to a pathology receiving laboratory in operation 104, where it may be received, sorted, organized, and labeled along with other samples from other patients for further processing.

[0047] In operation 106, the sample may be further processed as part of a grossing operation. For example, individual tissue samples or specimens may be sectioned for embedding and subsequent cutting onto slides.

[0048] The sample or specimen may then be mounted or deposited onto one or more glass slides in operation 108. Preparing the slides may include applying one or more reagents or stains to the sample, for example, to improve the visibility of or contrast between different portions of the sample.

[0049] In some cases, during or after the reagent or staining process, multiple slides may be organized or collected into a case or folio in operation 110. The case may be carefully labeled with, for example, individual patient identification information.

[0050] Between operations 102 and 110, in operation 112, the sample, specimen, or slide may be transported within or between medical facilities (e.g., between a doctor's office and a laboratory) or may be stored between processing operations.

[0051] Once sample and slide processing is complete and the pathologist is ready to review the sample, the slide and / or a case holding multiple slides corresponding to the patient may be transported back to the pathologist in operation 112. In operation 114, the pathologist may review the slide under magnification, for example, using a microscope. Individual slides may be placed under the objective lens of the microscope, and the pathologist may operate and adjust the microscope and slide as they review the tissue or bodily fluid.

[0052] Once the pathologist has completed review of the slide, the pathologist may attempt to render a medical finding or diagnosis in operation 116. Meanwhile, the sample or slide may be transported again to a longer-term storage facility in operation 112. In some cases, the sample or slide may be transported again to another physician before or after a storage period for further analysis, a second opinion, etc.

[0053] An example of the operations outlined above might occur in a pathology setting, where a pathologist identifies patients (e.g., breast cancer patients) who are likely to respond to treatment with an immune checkpoint inhibitor by manually analyzing and scoring a PD-L1 (SP142) assay. This immunohistochemistry (IHC) assay utilizes an anti-PD-L1 antibody (e.g., rabbit monoclonal anti-PD-L1 clone SP142) to recognize the programmed cell death-ligand 1 (PD-L1) protein in patient tissue samples.

[0054] 2A-2D show the process of manually scoring a PD-L1 (e.g., PD-L1 SP142) assay according to an example of the prior art.

[0055] 2A shows a slide 202 containing a section of a patient tissue sample containing tumor cells 202. The slide 202 may be stained with hematoxylin and eosin (H&E), which produces a staining pattern that reveals the general layout and distribution of cells, distinguishes between different types of tissue, and provides a general overview of the structure of the tissue sample. A pathologist may identify viable tumor areas from the H&E slide 202.

[0056] FIG. 2B shows an immunohistochemistry (IHC) slide 204 adjacent to a section from the H&E slide 202, which includes a section of a tissue sample stained with PD-L1 protein (e.g., PD-L1 SP124 protein). A pathologist can determine the presence of immune reactivity from the IHC slide 204. The tissue sections from the IHC slide and the H&E slide may be in close proximity (e.g., approximately 2 μm apart) and therefore may have substantially the same cellular morphology. As such, the H&E slide 202 may be used by the pathologist to help identify tumor areas on the IHC slide 204 of interest. Once the target areas are identified on the IHC slide 204, the pathologist identifies any dark spots (e.g., brown spots) that may be present on the slide 204, indicating staining of cells (e.g., tumor or immune cells) with the PD-L1 protein. If none are found, this is a negative sample, and the patient's status is identified as negative, i.e., the patient is unlikely to respond to treatment (e.g., with an immune checkpoint inhibitor). However, if dark spots (or PD-L1 staining) are observed, the pathologist must determine whether the dark spots are due to tumor cells (e.g., as a result of PD-L1 protein binding to tumor cell membranes) or immune cells (e.g., as a result of PD-L1 protein binding to immune cells such as T cells). Pathologists can distinguish immune cells (ICs) from tumor cells based at least on the shape and size of the cells. If the concentration of PD-L1 protein-stained immune cells is sufficiently high, the patient's condition may be identified as positive, meaning that the patient is likely to respond to treatment. This is because PD-L1 protein binding to immune cells (e.g., T cells) prevents the immune cells from attacking tumor cells. A specimen with a sufficiently high number of PD-L1-bound immune cells (i.e., assay-positive immune cells) is likely to respond well to treatment.

[0057] Figure 2C shows pathologist-highlighted tumor regions 208 and assay-positive IC regions 210 (e.g., regions of immune cells stained with the PD-L1 biomarker), which are also visually identified and manually highlighted by the pathologist. With these regions highlighted, the pathologist then mentally merges / aggregates the assay-positive IC regions to estimate the percentage of tumor region 208 occupied by the assay-positive IC regions and determine a raw IC percentage score (also referred to as a "slide score"). This score may be formally expressed as:

[0058] TIFF2025526652000005.tif19170

[0059] Figure 2D visualizes the thought process a pathologist must go through to arrive at this score. Each tissue may have an associated score threshold (e.g., 1%, 5%, 10%, 20%) above which the patient's status becomes positive. For example, the cutoff for breast cancer may be 1%. Thus, a score of 1% or above indicates a high likelihood that the patient will respond positively to treatment, while a score below 1% indicates a low likelihood that the patient will respond to treatment.

[0060] The complex manual slide scoring process shown in Figures 2A-2D is tedious, time-consuming, and highly inaccurate, involving a great deal of guesswork by the pathologist. This results in significant inter- and intra-observer variability in results, especially when scores are borderline close to the cutoff / threshold. This inaccuracy can lead to misdiagnosis of the patient's condition and, ultimately, to an incorrect procedural diagnosis, potentially resulting in poor or adverse patient outcomes.

[0061] Thus, embodiments of the present disclosure are directed to a cell-based scoring system that can reliably, repeatedly, and accurately determine a raw slide score (e.g., a PD-L1 Sp142 whole slide IC score). Additionally, embodiments of the present disclosure are directed to a cell classifier that can identify and count different types of cells within a tissue sample. Data generated by the cell classifier not only assists the cell-based scoring system in arriving at a slide score for a given sample, but can also assist researchers in formulating better hypotheses regarding the efficacy of a particular treatment regimen and / or testing various hypotheses.

[0062] For example, by knowing the location (e.g., xy location) and type of each cell in a tissue sample, the average distance between tumor cells in the sample and their nearest immune cells can be calculated. Such information may be relevant, for example, to why positive patients do not respond to a particular treatment. Many other relevant features can also be extracted from the raw data provided by the cell classifier, which can help researchers better explore hypotheses regarding therapeutic agents.

[0063] Figure 3A is a block diagram illustrating a cell classifier 300, according to some embodiments of the present disclosure. Figure 3B shows a labeled image identifying different types of cells detected by the cell classifier 300, according to some embodiments of the present disclosure.

[0064] Referring to FIG. 3A , in some embodiments, a cell classifier 300 receives an input image 302, which may be an image of a stained tissue sample (e.g., an image of an IHC slide), detects cells within the input image 302 (also referred to as a field of view (FOV)), and generates cell classification data 304 corresponding to the detected cells. The classification data 304 may include the type and location of each cell within the input image (e.g., a digitized red-green-blue (RGB) image) 302. The data 304 may further include the number of each identified cell type. According to some embodiments, the cell types classified by the cell classifier 300 may include stained immune cells (IC+), unstained immune cells (IC−), stained tumor cells (TC+), unstained tumor cells (TC−), stained macrophages (macrophage+), unstained macrophages (macrophage−), and / or other cells (not IC, TC, or macrophages). However, embodiments of the present disclosure are not limited thereto, and the cell classifier 300 may be trained to identify any suitable type of cell. FIG. 3B shows an example where cell classifier 300 identifies and labels different cells within an IHC slide.

[0065] In some embodiments, the cell classifier 300 includes a neural network (e.g., a convolutional neural network) 310 capable of performing cell detection and cell classification. The neural network 310 may include several layers, each of which performs a convolution operation on an input feature map (IFM) 312 via the application of a kernel / filter to generate an output feature map that serves as the input feature map 312 for subsequent layers. In the first layer of the neural network 310, the input feature map may be the input image 302.

[0066] The neural network 310 referred to in this disclosure may, according to some examples, be a convolutional neural network (ConvNet / CNN), which takes in an input image and assigns importance to various aspects / objects in the image (e.g., via learnable weights and biases) to distinguish one from another. However, embodiments of the present disclosure are not limited thereto. For example, the neural network 310 may be a recurrent neural network (RNN) involving convolution operations, etc.

[0067] The deep learning model of neural network 310 may be trained by providing neural network 310 with many examples of FOV 302 (e.g., over 100,000 samples) and corresponding annotated data including the location of each cell on the FOV (e.g., each cell's x-y location on the FOV) and each cell's type (e.g., label). The annotated cell type (e.g., cell label) may be one of "IC+," "IC-," "TC+," "TC-," "macrophage+," "macrophage-," and "other cells." A visualization of this annotated cell data is shown in Figure 3B, where each cell is marked with a colored shape corresponding to its cell type.

[0068] Some or all of the cell classification data 304 may be used by a regressor to determine a raw score (e.g., PD-L1 Sp142 whole slide IC score) for a given input image / FOV 302 and / or may be used by researchers in exploring various hypotheses regarding the efficacy of a particular treatment regimen.

[0069] FIG. 4 shows a block diagram of a cell-based scoring system 400 including a cell classifier 300 and a regressor 330, according to some embodiments of the present disclosure.

[0070] According to some embodiments, the cell-based scoring system 400 is a sample / slide scoring system configured to determine a raw score of a pathology slide from a tissue sample based on an image (e.g., a digitized image) of the slide. The cell classifier 300 receives an image 302 of the pathology slide and classifies each cell captured in the image. As described above with respect to FIG. 3, the cell classifier 300 may perform the classification by identifying each cell in the image 302 and assigning the cell an appropriate cell type.

[0071] In some embodiments, the cell-based scoring system 400 includes a feature generator (e.g., a cell-based feature generator) 320 configured to generate a plurality of first slide features (e.g., a plurality of extracted features) based on the classification of each cell. The feature generator 320 and the regressor 330 may be collectively referred to as a regression system 315. The feature generator 320 may generate the first features by counting the number of cells assigned to each cell type in the plurality of cells and then generating features based on the number of cells assigned to each cell type. For example, the first slide features may include at least one of the number of IC+ cells, the number of IC− cells, the number of TC+ cells, the number of TC− cells, the number of other cells identified in the image 302, and the total number of identified cells. In some embodiments, the first slide features may also include the area of tumor regions in the image 302 of the pathology slide, and the feature generator 320 may determine the area of the tumor regions based on the annotated image 303 that identifies (e.g., outlines) viable tumor regions 303a. The tumor regions 303a in the annotated image 303 may be generated (e.g., drawn / highlighted) by one or more pathologists. For example, a pathologist may outline (e.g., by feature generator 320) a viable tumor region 303a in the image 302, from which the area of the tumor region can be ascertained. In some examples, the tumor regions in the annotated image 303 may represent a consensus outline (e.g., an average) of the regions identified by several pathologists.

[0072] In some embodiments, the feature generator 320 also determines one or more second slide features (e.g., multiple calculated features) corresponding to the pathology slide image 302 based on the first slide features. The second slide features may include a field of view (FOV) area score, a cell area score, and a cell count score calculated based on the first slide features. For example, the FOV area score may be expressed as:

[0073] TIFF2025526652000006.tif19170

[0074] where the area occupied by all cells in the slide is calculated by multiplying the average size of each cell type by the number of corresponding cells and summing the results across all cell types. In some cases, the average size of immune cells may be used as an estimate of the size of cells in the "other cells" category.

[0075] Furthermore, the cell area score can be expressed as:

[0076] TIFF2025526652000007.tif19170

[0077] where the average size of the immune cells is provided to the regressor 330 or may be a value known by the regressor 330 (e.g., a fixed value).

[0078] Furthermore, the cell count score can be expressed as:

[0079] TIFF2025526652000008.tif19170

[0080] The second slide feature may represent a rough estimate of the actual slide score (e.g., PD-L1 Sp142 whole slide IC score) as defined above by Equation 1. The first and second slide features form a cumulative feature set, one or more of which are utilized by the cell-based scoring system (e.g., regressor 330) to determine the raw slide score 306.

[0081] According to some embodiments, the regressor 330 includes a regression model 340 that bridges the gap between the cell classification data generated by the cell classifier 300 and the raw slide score 306. The regressor 330 is trained via a regression algorithm to learn the relationship (e.g., a linear or nonlinear relationship) between one or more features of the accumulated feature set and the raw slide score. Thus, once trained, the regressor 330 can predict the raw slide score of a pathology slide based on the values of one or more features provided by the feature generator 320. The term "regression" is used informally herein as a general name for a mathematical modeling task in which the output is a real number. Regression models may include K-nearest neighbor (KNN) models, support vector machine (SVM) models, random forest (RF) models, multilayer perceptron (MLP) models, etc. For each model, deep learning may be used to identify the corresponding set of input features that are most relevant to the algorithm's prediction of the raw slide score. For example, if the regressor 330 includes a KNN machine learning model, it may use the tumor area, FOV area score, cell area score, and cell count score from the accumulated feature set to predict a raw slide score. Furthermore, if the regressor 330 includes an SVM machine learning model, it may use the tumor area, the number of IC+, IC-, TC+, TC-, and other cells, and the total number of cells in the pathology slide from the accumulated feature set to predict a raw slide score. Furthermore, if the regressor 330 includes an RF machine learning model, it may use the tumor area, the number of IC+ cells, and the number of IC- cells in the pathology slide from the accumulated feature set to predict a raw slide score. However, embodiments of the present disclosure are not limited thereto, and the regressor 330 may use any suitable set of input features from the accumulated feature set when estimating / predicting a slide score.

[0082] In some embodiments, the regressor 330 may be a specialized or general-purpose AI and is trained using training data and an algorithm, such as a backpropagation algorithm. The training data may include many examples of one or more features from an accumulated feature set and corresponding consensus slide scores from several pathologists.

[0083] The regressor 330 may include a set of weights for each parameter of a linear regression model, or the regressor 330 may include a set of weights for the connections between neurons of a trained neural network. In some embodiments, one or more features from the accumulated feature set are provided to the regressor 330 as values (e.g., input features) to the input layer of the regressor 330, and this value (or a set of intermediate values) is propagated forward through the regressor 330 to generate an output, which is the raw slide score.

[0084] According to some embodiments, the cell-based scoring system 400 includes a controller 350 for controlling the operation of the classifier 300, the feature generator 320, and the regressor 330, and further includes a memory 360 (e.g., on-logic die memory) for temporarily storing the input image 302, the output or intermediate results of the neural network 310 (e.g., output feature maps generated by layers of the neural network 310), and the regression model 340. In some examples, the memory 360 may be an embedded magnetoresistive random access memory (eMRAM), a static random access memory (SRAM), or the like.

[0085] To evaluate the performance of the cell-based scoring system 400 (i.e., the combination of the classifier 300 and the regressor 330), a representative benchmark set containing 100 cases closely aligned with the prevalence of breast cancer specimens was assembled. Three pathologists independently provided whole-slide scores for all 100 cases in the benchmark set, and the median of the three scores was calculated as the consensus score for each case in the benchmark set. Table 1 shows the confusion matrix for the representative benchmark set, highlighting the ability of the cell-based scoring system 400 to predict slide scores and correctly classify patients as negative (i.e., slide scores below the 1% threshold) or positive (i.e., slide scores above the 1% threshold).

[0086] [Table 1]

[0087] As shown in the example in Table 1, the number of true positive (TP) and true negative (TN) classifications by the cell-based scoring system 400 represented 98% of the benchmark cases, with only 2 percent of the positive cases incorrectly (e.g., incorrectly) labeled as negative (i.e., false negative (FN)) and no false positives (FP).

[0088] Performance metrics of the cell-based scoring system 400 in representative benchmark tests are shown in Table 2 below.

[0089] [Table 2]

[0090] The metrics in Table 1 are defined as follows:

[0091] TIFF2025526652000011.tif11170

[0092] TIFF2025526652000012.tif11170

[0093] TIFF2025526652000013.tif11170

[0094] TIFF2025526652000014.tif11170

[0095] TIFF2025526652000015.tif14170

[0096] A desirable effect of using a regressor in cell-based scoring system 400 is that the regressor can compensate for inherent biases or correct systematic errors that may be present in cell classifier 300. For example, cell classifier 300 may systematically undercount the number of IC+ cells. However, because regressor 330 is trained on data from cell classifier 300, the errors and biases are learned by regressor 330 and are not translated into erroneous slide scores. In other words, any errors in the output of cell classifier 300 are compensated for by regressor 330 and do not necessarily propagate through regressor 330. Thus, cell-based scoring system 400 can produce accurate results.

[0097] According to various embodiments of the present disclosure, the cell-based scoring system 400 is implemented using one or more processing or electronic circuits configured to perform various operations as described above. Types of electronic circuits may include central processing units (CPUs), graphics processing units (GPUs), artificial intelligence (AI) accelerators (e.g., vector processors that may include vector arithmetic logic units configured to efficiently perform operations common to neural networks, such as dot products and softmaxes), field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), digital signal processors (DSPs), etc. For example, in some circumstances, aspects of embodiments of the present disclosure are implemented in program instructions stored in non-volatile computer-readable memory that, when executed by an electronic circuit (e.g., a CPU, a GPU, an AI accelerator, or a combination thereof), perform the described operations. The operations performed by the cell-based scoring system 400 may be performed by a single electronic circuit (e.g., a single CPU, a single GPU, etc.) or may be distributed among multiple electronic circuits (e.g., multiple GPUs, or a CPU in conjunction with a GPU). The multiple electronic circuits may be local to each other (e.g., located on the same die, located in the same package, or located in the same embedded device or computer system) and / or may be remote from each other (e.g., in communication over a network such as a local personal area network such as Bluetooth®, communication over a local area network such as a local wired and / or wireless network, and / or communication over a wide area network such as the Internet, where some operations are performed locally and other operations are performed on a server hosted by a cloud computing service). One or more electronic circuits that operate to implement the cell-based scoring system 400 may be referred to herein as a computer or computer system, which may include a memory that stores instructions that, when executed by the one or more electronic circuits, implement the systems and methods described herein.

[0098] FIG. 5 is a flow diagram illustrating a process 500 for determining a raw score for a pathology slide from a tissue sample using cell-based classification data corresponding to the pathology slide, according to some embodiments of the present disclosure.

[0099] In some embodiments, the regression system 315 receives (S502) a plurality of first slide features corresponding to an image (e.g., a digitized image) 302 of a pathology slide. The pathology slide may be stained with PD-L1 SP142.

[0100] The regression system 315 calculates one or more second slide features corresponding to the pathology slide based on the plurality of first slide features (S504), which may include calculating at least one of a field of view (FOV) area score, a cell area score, and a cell count score defined in Equations 2-4.

[0101] The regression system 315 determines a raw score based on one or more features of an accumulated Feature Set including a plurality of first slide features and one or more second slide features (S506), which may include providing the one or more features of the accumulated Feature Set to a trained regression model 340 configured to correlate raw score values to values of the one or more features, and estimating, via the trained regression model 340, a raw score corresponding to the one or more features.

[0102] The cell-based scoring system 400 may then compare the raw score to a threshold value (e.g., 1%) to determine the effectiveness of a treatment for the patient associated with the tissue sample.

[0103] FIG. 6 is a flow diagram illustrating a process 600 for determining a raw score of a pathology slide from a tissue sample, according to some embodiments of the present disclosure.

[0104] In some embodiments, the cell-based scoring system 400 receives an image (e.g., a digitized image) 302 of a pathology slide (S602). The cell-based scoring system 400 classifies each cell of the plurality of cells captured in the image 302 by providing the image to a classifier 300 configured to identify each cell of the plurality of cells and assign a cell type from among a plurality of cell types to each of the plurality of cells (S604). The cell-based scoring system 400 then generates a plurality of first slide features based on the classification of each cell (S606) and determines a raw score based on one or more features of an accumulated feature set including the plurality of first slide features (S608). Generating the plurality of first slide features may include counting the number of cells assigned to each cell type of the plurality of cells and generating the plurality of first slide features based on the number of cells assigned to each cell type. This may further include receiving an area of the tumor region 303a corresponding to the image 302 and adding (e.g., including) the area to the accumulated feature set. The cell-based scoring system 400 may determine the raw score by providing one or more features of the accumulated feature set to a trained regression model 340 configured to correlate raw score values to values of one or more features, and having the trained regression model 340 estimate a raw score corresponding to the one or more features.

[0105] The cell-based scoring system 400 may then compare the raw score to a threshold value (e.g., 1%) to determine the effectiveness of a treatment for the patient associated with the tissue sample.

[0106] Thus, as described above, cell-based scoring systems have the ability to ascertain cell-level data from digitized images of pathology slides and convert this cell-level data into a slide score (e.g., PD-L1 SP142 whole-slide score). Cell-based scoring systems are constructed to not only predict a whole-slide score but also provide valuable cell-level information that pharmaceutical companies can use for drug development and discovery, as well as for exploring various hypotheses. The rapid and accurate results generated by cell-based scoring systems eliminate the need for time-consuming and tedious manual scoring by pathologists, which is plagued by inaccuracies and large inter- and intra-observer variability.

[0107] Although some of the examples provided herein describe the processing of pathology slides stained with PD-L1 SP142, embodiments of the present disclosure are not so limited and the cell-based scoring system of the present disclosure may operate on pathology slides stained with any suitable biomarker.

[0108] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the invention. As used herein, the singular forms "a" and "an" are intended to include the plural forms unless the context clearly suggests otherwise. It will be further understood that the terms "comprises," "comprising," "includes," and "including," when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. The term "and / or," as used herein, includes any and all combinations of one or more of the associated listed items. Phrases such as "at least one of," when preceding a list of elements, modify the entire list of elements and not the individual elements of the list.

[0109] Terms such as "first," "second," and "third" may be used herein to describe various elements, components, regions, layers, and / or sections, but it is understood that these elements, components, regions, layers, and / or sections should not be limited by these terms. These terms are used to distinguish one element, component, region, layer, or section from another element, component, region, layer, or section. Thus, a first element, component, region, layer, or section described below may be referred to as a second element, component, region, layer, or section without departing from the spirit and scope of the present invention.

[0110] As used herein, the terms "substantially," "about," and similar terms are used as terms of approximation, not as terms of degree, and are intended to account for inherent variations in measurements or calculations that will be recognized by those of ordinary skill in the art. Furthermore, the use of "may" when describing embodiments of the present invention refers to "one or more embodiments of the present invention." As used herein, the terms "use," "using," and "used" may be considered synonymous with the terms "utilize," "utilizing," and "utilized," respectively.

[0111] Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by those skilled in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and / or this specification, and should not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0112] While aspects of certain exemplary embodiments of the system and method for quantifying pathology slides using a cell-based scoring system have been described and illustrated herein, various modifications and variations may be implemented as understood by those skilled in the art without departing from the spirit and scope of the embodiments according to the present disclosure. Accordingly, it should be understood that the pathology slide manufacturing system and method according to the principles of the present disclosure may be embodied in other ways than those specifically described herein. The present disclosure is also defined in the following claims and their equivalents.

Claims

1. 1. A method for determining a raw score of a pathology slide from a tissue sample, comprising: receiving, by a regression system, a plurality of first slide features corresponding to the pathology slide; calculating, by the regression system, one or more second slide features corresponding to the pathology slide based on the plurality of first slide features; determining, by the regression system, the raw score based on one or more features of an accumulated Feature Set including the plurality of first slide features and the one or more second slide features; A method comprising:

2. 2. The method of claim 1, wherein the pathology slide is stained with PD-L1 SP142.

3. the plurality of first slide features comprising: the area of the tumor region of the pathology slide; the number of stained immune cells in the pathology slide; the number of unstained immune cells in the pathology slide; the number of stained tumor cells in the pathology slide; the number of unstained tumor cells in the pathology slide; the number of other cells in the pathology slide; and the total number of cells in the pathology slide; The method of claim 1 , comprising at least one of:

4. Calculating the one or more second slide features comprises: Calculating at least one of a field of view (FOV) area score, a cell area score, and a cell number score. The method of claim 1 , comprising:

5. The FOV area score is expressed as: where the mean size of IC+ cells represents the mean size of stained immune cells, and the number of IC+ cells represents the number of stained immune cells in the pathology slide. The method of claim 4.

6. The cell area score is expressed as: where the average size of IC+ cells represents the average size of stained immune cells, the number of IC+ cells represents the number of stained immune cells in the pathology slide, and the tumor area represents the area of the tumor region corresponding to the pathology slide. The method of claim 4.

7. The cell number score is expressed as follows: where the number of IC+ cells represents the number of stained immune cells in the pathology slide, and the total number of cells represents the total number of cells in the pathology slide. The method of claim 4.

8. determining the raw score, providing the one or more features of the accumulated Feature Set to a trained regression model configured to correlate raw score values to values of the one or more features; estimating the raw scores corresponding to the one or more features by the trained regression model; and The method of claim 1 , comprising:

9. The method of claim 1 , wherein the regression system comprises a trained machine learning model configured to correlate the one or more features of the accumulated Feature Set to the raw score.

10. 10. The method of claim 9, wherein the trained machine learning model comprises one of a K-nearest neighbor (KNN) model, a support vector machine (SVM) model, a random forest (RF) model, and a multi-layer perceptron (MLP) model.

11. 10. The method of claim 1, further comprising comparing the raw score to a threshold to determine the effectiveness of a treatment for a patient associated with the tissue sample.

12. receiving, by a classifier, an image of the pathology slide; identifying, with the classifier, each cell of the plurality of cells captured in the image and classifying each cell of the plurality of cells by assigning a cell type from among a plurality of cell types to each of the plurality of cells; generating, with the classifier, the plurality of first slide features based on the classification of each cell; The method of claim 1 further comprising:

13. The method of claim 12 , wherein the classifier comprises a convolutional neural network.

14. 1. A method for determining a raw score of a pathology slide from a tissue sample, comprising: receiving an image of the pathology slide by a cell-based scoring system including processing circuitry and memory; classifying, by the cell-based scoring system, each cell of a plurality of cells captured in the image by providing the image to a classifier of the cell-based scoring system, the classifier being configured to identify each cell of the plurality of cells and assign a cell type from among a plurality of cell types to each of the plurality of cells; generating a plurality of first slide features based on the classification of each cell by the cell-based scoring system; determining, by the cell-based scoring system, the raw score based on one or more features of an accumulated Feature Set that includes the plurality of first slide features; A method comprising:

15. generating the plurality of first slide features; counting the number of cells assigned to each cell type of said plurality of cells; generating the plurality of first slide features based on the number of cells assigned to each cell type; 15. The method of claim 14, comprising:

16. and receiving, by the cell-based scoring system, an area of the tumor region corresponding to the image; the accumulated feature set further comprises the area of the tumor region.

15. The method of claim 14.

17. the plurality of first slide features the number of stained immune cells in the pathology slide; the number of unstained immune cells in the pathology slide; the number of stained tumor cells in the pathology slide; the number of unstained tumor cells in the pathology slide; the number of other cells in the pathology slide; and the total number of cells in the pathology slide; The method of claim 14 , comprising at least one of:

18. calculating, by the cell-based scoring system, one or more second slide features corresponding to the pathology slide based on the plurality of first slide features; the accumulated feature set further includes the one or more second slide features.

15. The method of claim 14.

19. Calculating the one or more second slide features comprises: Calculating at least one of a field of view (FOV) area score, a cell area score, and a cell number score.

20. The method of claim 18, comprising:

20. determining the raw score, providing the one or more features of the accumulated Feature Set to a trained regression model configured to correlate raw score values to values of the one or more features; estimating the raw scores corresponding to the one or more features by the trained regression model; and 15. The method of claim 14, comprising:

21. 15. The method of claim 14, further comprising comparing the raw score to a threshold to determine the effectiveness of a treatment for a patient associated with the tissue sample.

22. 1. A cell-based scoring system for determining a raw score of a pathology slide from a tissue sample, comprising: receiving an image of the pathology slide; classifying each cell of the plurality of cells by identifying each cell of the plurality of cells captured in the image and assigning a cell type from among a plurality of cell types to each of the plurality of cells; generating a plurality of first slide features based on the classification of each cell; a classifier including a convolutional neural network configured as follows: a cell-based feature generator configured to calculate one or more second slide features corresponding to the pathology slide based on the plurality of first slide features; a regressor configured to determine the raw score based on one or more features of an accumulated feature set including the plurality of first slide features and the one or more second slide features; A cell-based scoring system comprising:

Images