Systems and methods for quantification of digitized pathology glass

Abstract

The present invention provides a method of determining an original score of a pathological slide from a tissue sample, the method comprising: receiving, by a regression system, a plurality of first slide characteristics corresponding to the pathological slide; calculating, by the regression system, one or more second slide features corresponding to the pathological slide based on the plurality of first slide features; and determining, by the regression system, the original score based on one or more features in a cumulative feature set including the plurality of first slide features and the one or more second slide features.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 396,142 (“SYSTEM AND METHODFOR QUANTIFICATION OF DIGITIZED PATHOLOGY SLIDES”), filed on August 8, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] One or more aspects according to some embodiments of the present disclosure relate to quantifying pathology slides. Background Art

[0004] The body's immune system uses T cells to help fight infections and other diseases, including cancer. PD-L1 is a transmembrane protein that downregulates immune responses by binding to two receptors on T cells: programmed death-1 (PD-1) and B7.1. One way to fight cancer is to block the PD-L1 protein, which may prevent cancer cells from inactivating T cells through both PD-1 and B7.1. PD-L1 testing helps doctors determine if a patient might benefit from cancer drugs called immune checkpoint inhibitors. This class of inhibitor drugs prevents the PD-1 / PD-L1 encounter from occurring. Therefore, without receiving a "stop" signal from the PD-L1 protein, T cells can continue to attack tumor cells.

[0005] The PD-L1 (SP142) assay is an immunohistochemistry (IHC) assay that utilizes an anti-PD-L1 rabbit monoclonal primary antibody to recognize the programmed death ligand 1 (PD-L1) protein. The assay was developed to identify patients most likely to respond to treatment with immune checkpoint inhibitors. However, studies have shown that there is significant inter-pathologist variability in the evaluation of PD-L1 SP142 immunohistochemistry as a percentage and in PD-L1 SP142 status (positive vs. negative). Two factors may contribute to this high inter-observer variability: 1) the assay is amplified and 2) the manual scoring guidelines are cumbersome and complex.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background technology, and therefore the information discussed in this Background section does not necessarily constitute prior art. Summary of the invention

[0007] Various aspects of embodiments of the present disclosure relate to a cell-based scoring system that utilizes artificial intelligence (AI) to quantify digitized slides from patient samples (e.g., PD-L1 SP142 digital slides) and predict the overall slide score percentage, and therefore predict the patient's status (e.g., whether the patient is likely to benefit from a particular cancer drug).

[0008] According to some embodiments of the present disclosure, a method for determining a raw score of a pathology slide from a tissue sample is provided, the method comprising: receiving, by a regression system, a plurality of first slide features corresponding to the pathology slide; calculating, by the regression system, one or more second slide features corresponding to the pathology slide based on the plurality of first slide features; and determining, by the regression system, the raw score based on one or more features in a cumulative feature set including the plurality of first slide features and the one or more second slide features.

[0009] In some embodiments, the pathology slide is stained with PD-L1 SP142.

[0010] In some embodiments, the plurality of first slide features include at least one of: the area of ​​the tumor region of the pathology slide; the number of stained immune cells of the pathology slide; the number of unstained immune cells of the pathology slide; the number of stained tumor cells of the pathology slide; the number of unstained tumor cells of the pathology slide; the number of other cells of the pathology slide; and the total number of cells of the pathology slide.

[0011] In some embodiments, calculating the one or more second slide characteristics comprises calculating at least one of a field of view (FOV) area fraction, a cell area fraction, and a cell count fraction.

[0012] In some embodiments, the FOV area fraction is expressed as:

[0013]

[0014] The average size of IC+ cells represents the average size of stained immune cells, and the number of IC+ cells represents the number of stained immune cells of the pathology slide.

[0015] In some embodiments, the cell area fraction is expressed as:

[0016]

[0017] Wherein the average size of IC+ cells represents the average size of stained immune cells, the number of IC+ cells represents the number of stained immune cells of the pathology slide, and the area of ​​the tumor represents the area of ​​the tumor region corresponding to the pathology slide.

[0018] In some embodiments, the cell count fraction is expressed as:

[0019]

[0020] Wherein the number of IC+ cells represents the number of stained immune cells of the pathology slide, and the total number of cells represents the total number of cells of the pathology slide.

[0021] In some embodiments, determining the raw score includes: providing the one or more features in the cumulative feature set to a trained regression model, the trained regression model being configured to correlate the raw score value with the values ​​of the one or more features; and estimating, by the trained regression model, the raw score corresponding to the one or more features.

[0022] In some embodiments, the regression system includes a trained machine learning model that is configured such that the one or more features in the accumulated feature set are correlated with the raw score.

[0023] In some embodiments, the trained machine learning model includes one of a K-nearest neighbor (KNN) model, a support vector machine (SVM) model, a random forest (RF) model, and a multi-layer perceptron (MLP) model.

[0024] In some embodiments, the method further comprises comparing the raw score to a threshold value to determine the efficacy of a treatment for a patient associated with the tissue sample.

[0025] In some embodiments, the method further includes: receiving, by a classifier, an image of the pathology slide; classifying, by the classifier, each of the plurality of cells by identifying each of the plurality of cells captured in the image and assigning a cell type from a plurality of cell types to each of the plurality of cells; and generating, by the classifier, the plurality of first slide features based on the classification of each cell.

[0026] In some embodiments, the classifier comprises a convolutional neural network.

[0027] According to some embodiments of the present disclosure, a method for determining a raw score of a pathology slide from a tissue sample is provided, the method comprising: receiving, by a cell-based scoring system comprising a processing circuit and a memory, the pathology slide image; classifying, by the cell-based scoring system, each of a plurality of cells captured in the image by providing the image to a classifier of the cell-based scoring system, the classifier being configured to identify each of the plurality of cells and being configured to assign a cell type from a plurality of cell types to each of the plurality of cells; generating, by the cell-based scoring system, a plurality of first slide features based on the classification of each cell; and determining, by the cell-based scoring system, the raw score based on one or more features in a cumulative feature set comprising the plurality of first slide features.

[0028] In some embodiments, generating the plurality of first slide characteristics comprises: counting a number of cells of each cell type assigned to the plurality of cells; and generating the plurality of first slide characteristics based on the number of cells assigned to each cell type.

[0029] In some embodiments, the method further comprises: receiving, by the cell-based scoring system, an area of ​​a tumor region corresponding to the image, wherein the accumulated feature set further comprises the area of ​​the tumor region.

[0030] In some embodiments, the plurality of first slide features include at least one of: the number of stained immune cells of the pathology slide; the number of unstained immune cells of the pathology slide; the number of stained tumor cells of the pathology slide; the number of unstained tumor cells of the pathology slide; the number of other cells of the pathology slide; and the total number of cells of the pathology slide.

[0031] In some embodiments, the method further comprises: calculating, by the cell-based scoring system, one or more second slide features corresponding to the pathology slide based on the plurality of first slide features, wherein the cumulative feature set further comprises the one or more second slide features.

[0032] In some embodiments, calculating the one or more second slide characteristics comprises calculating at least one of a field of view (FOV) area fraction, a cell area fraction, and a cell count fraction.

[0033] In some embodiments, determining the raw score includes: providing the one or more features in the cumulative feature set to a trained regression model, the trained regression model being configured to correlate the raw score value with the values ​​of the one or more features; and estimating, by the trained regression model, the raw score corresponding to the one or more features.

[0034] In some embodiments, the method further comprises comparing the raw score to a threshold value to determine the efficacy of a treatment for a patient associated with the tissue sample.

[0035] According to some embodiments of the present disclosure, a cell-based scoring system for determining a raw score of a pathology slide from a tissue sample is provided, the cell-based scoring system comprising: a classifier comprising a convolutional neural network, the classifier being configured to: receive an image of the pathology slide; classify each of a plurality of cells captured in the image by identifying each cell and assigning a cell type from a plurality of cell types to each of the plurality of cells; generate a plurality of first slide features based on the classification of each cell; a cell-based feature generator being configured to calculate one or more second slide features corresponding to the pathology slide based on the plurality of first slide features; and a regressor being configured to determine the raw score based on one or more features in a cumulative feature set comprising the plurality of first slide features and the one or more second slide features. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Non-limiting and non-exhaustive embodiments according to the present disclosure are described with reference to the following figures, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified.

[0037] Figure 1 is a flow chart illustrating various operations that may occur in a pathology context or environment according to some embodiments;

[0038] FIG. 2A to FIG. 2D The process of manually scoring PD-L1 assays based on examples from the related art is demonstrated.

[0039] Figure 3A is a block diagram showing a cell sorter according to some embodiments of the present disclosure.

[0040] Figure 3B Labeled images identifying different cell types detected by a cell classifier are shown according to some embodiments of the present disclosure.

[0041] Figure 4 A block diagram of a cell-based scoring system including a cell classifier and a regressor according to some embodiments of the present disclosure is shown.

[0042] Figure 5 is a flow chart showing a process of determining a raw score for a pathology slide from a tissue sample using cell-based classification data corresponding to the pathology slide, according to some embodiments of the present disclosure.

[0043] Figure 6 is a flow chart showing a process of determining a raw score for a pathology slide from a tissue sample according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0044] Hereinafter, the aspects of some exemplary embodiments will be described in more detail with reference to the accompanying drawings, in which the same reference numerals always refer to the same elements. However, the present invention can be embodied in various different forms and should not be construed as being limited to the embodiments shown herein alone. On the contrary, these embodiments are provided as examples so that the present disclosure will be thorough and complete and will fully convey the aspects and features of the present invention to those skilled in the art. Therefore, processes, elements and techniques that are unnecessary for those of ordinary skill in the art to fully understand the aspects and features of the present invention may not be described. Unless otherwise stated, the same reference numerals throughout the drawings and written descriptions represent the same elements, and therefore descriptions thereof will not be repeated. In the drawings, for clarity, the relative sizes of elements, layers and zones may be exaggerated.

[0045] Pathology is a medical discipline that attempts to aid in the diagnosis and treatment of disease by studying tissue, cell, and fluid samples from patients. In many applications, tissue samples can be collected from patients and processed into a form that can be analyzed by a physician (e.g., a pathologist), typically under magnification by a physician, to diagnose and characterize the relevant medical condition based on the tissue sample.

[0046] Figure 1 1 is a flow chart illustrating various operations that may occur in a pathology environment or pathology system 100. For example, when a treating physician or medical provider identifies a patient for whom analysis of a tissue or fluid sample may be beneficial in diagnosing or treating a medical condition, a tissue or fluid sample may be collected at operation 102. The identity of the patient may be collected and matched to the patient's sample, and the sample may be placed in a sterile container and / or collection medium for further processing.

[0047] The sample may then be transported to a pathology registration laboratory at operation 104 where the sample may be received, sorted, organized, and labeled for further processing along with other samples from other patients.

[0048] The sample may be further processed as part of a grossing operation at operation 106. For example, individual tissue samples or specimens may be cut into smaller sections for embedding and subsequent cutting for assembly onto slides.

[0049] The sample or specimen may then be mounted or deposited on one or more glass slides at operation 108. Preparation of the slides may involve applying one or more reagents or stains to the sample, for example, to improve visibility of or contrast between different portions of the sample.

[0050] In some cases, several slides may be assembled or collected in a box or folio during reagent or staining processing or after processing is complete at operation 110. For example, the box may be carefully labeled with identification information for individual patients.

[0051] Between each of operations 102 and 110, at operation 112, samples, specimens, slides may be transported within or between medical facilities (eg, between a physician's office and a laboratory), or may be stored between processing operations.

[0052] Once processing of the sample and slides is complete and the pathologist is ready to examine the sample, the slides corresponding to the patient and / or the box holding multiple slides may again be transported to the pathologist at operation 112. The pathologist may examine the slides, for example, under magnification using a microscope, at operation 114. Individual slides may be placed under the objective of the microscope, and the microscope and slides may be manipulated and adjusted as the pathologist examines the tissue or fluid.

[0053] Once the pathologist has completed his examination of the slide, the pathologist may attempt to form a medical opinion or diagnosis at operation 116. Meanwhile, the sample or slide may once again be shipped to a long-term storage facility at operation 112. In some cases, the sample or slide may be shipped again to other physicians for further analysis, second opinions, etc., either before or after a certain storage period.

[0054] An example of the above operation can be performed in a pathology setting, where a pathologist identifies patients (e.g., breast cancer patients) who may respond to treatment with immune checkpoint inhibitors by manually analyzing and scoring a PD-L1 (SP142) assay. This immunohistochemistry (IHC) assay utilizes an anti-PD-L1 antibody (e.g., rabbit monoclonal anti-PD-L1 clone SP142) to identify the programmed death ligand 1 (PD-L1) protein in a patient's tissue sample.

[0055] FIG. 2A to FIG. 2D The process of manually scoring PD-L1 (e.g., PD-L1 SP142) assays according to examples in the related art is shown.

[0056] Figure 2AShown is a slide 202 comprising a section of a patient tissue sample containing tumor cells 202. The slide 202 can be stained with hematoxylin and eosin (H&E), which produces a staining pattern that reveals the general arrangement and distribution of cells, distinguishes different tissue types, and provides a general overview of the structure of the tissue sample. A pathologist can identify viable tumor areas from the H&E slide 202.

[0057] Figure 2B An immunohistochemistry (IHC) slide 204 is shown, which includes a slice of a tissue sample adjacent to a slice of a tissue sample of an H&E slide 202 and has a PD-L1 protein (e.g., PD-L1SP124 protein) applied thereto. A pathologist can determine the presence of immunoreactivity from the IHC slide 204. Tissue slices from the IHC and H&E slides may be very close (e.g., may be about 2 μm apart) and therefore may have substantially the same cell morphology. Therefore, a pathologist can use the H&E slide 202 to help identify the tumor area to be focused on the IHC slide 204. Once the target area is identified on the IHC slide 204, the pathologist identifies any dark spots (e.g., brown spots) that may be present on the slide 204, which indicates staining of cells (e.g., tumor or immune cells) by the PD-L1 protein. If not found, this is a negative sample and the patient status is identified as negative, that is, the patient is unlikely to respond to treatment (e.g., using immune checkpoint inhibitors). However, if dark spots (or PD-L1 staining) are observed, the pathologist must then determine whether the dark spots are attributed to tumor cells (e.g., due to the binding of PD-L1 protein to tumor cell membranes) or to immune cells (e.g., due to the binding of PD-L1 protein to immune cells, such as T cells). Pathologists can distinguish between immune cells (IC) and tumor cells based at least on cell shape and size. When the concentration of immune cells stained with PD-L1 protein is high enough, the patient status can be identified as positive, that is, the patient has a high probability of responding to treatment. This is because when PD-L1 protein binds to immune cells (e.g., T cells), it prevents immune cells from attacking tumor cells. Specimens with sufficiently high PD-L1-bound immune cells (i.e., assay-positive immune cells) may respond well to treatment.

[0058] Figure 2CTumor regions 208 highlighted by a pathologist are shown, as well as assay-positive IC regions 210 (e.g., regions of immune cells stained with the PD-L1 biomarker) also visually identified and manually highlighted by the pathologist. With these regions highlighted, the pathologist then mentally combines / aggregates the assay-positive IC regions to estimate the percentage of the tumor region 208 occupied by the assay-positive IC regions to determine a raw IC percentage score (also referred to as a "slide score"). This score may be formally expressed as

[0059]

[0060] Figure 2D It is a visualization of the mental process that must be performed by the pathologist to arrive at this score. Each tissue may have an associated score threshold (e.g., 1%, 5%, 10%, 20%) above which the patient's status becomes positive. For example, the cutoff for breast cancer may be 1%. Therefore, a score at or above 1% indicates that the patient may respond positively to treatment, while a score below 1% indicates that the patient is unlikely to respond to treatment.

[0061] FIG. 2A to FIG. 2D The complex manual slide scoring process presented is cumbersome and time consuming, involving a great deal of guesswork by the pathologist, which can be highly inaccurate. This causes significant inter- and intra-observer variability in the results, especially in borderline cases where the scores are close to the critical value / threshold. This can lead to poor or harmful patient outcomes when such inaccuracy leads to an erroneous determination of the patient's status, and therefore to an incorrect therapeutic diagnosis.

[0062] Thus, aspects of the present disclosure relate to a cell-based scoring system that can reliably, repeatedly, and accurately determine raw slide scores (e.g., PD-L1 Sp142 whole slide IC scores). In addition, aspects of the present disclosure relate to cell classifiers that can identify and count different cell types within a tissue sample. The data generated by the cell classifier can not only assist the cell-based scoring system in obtaining a slide score for a given sample, but can also assist researchers in forming better hypotheses and / or testing various hypotheses about the efficacy of a particular treatment plan.

[0063] For example, by knowing the location (e.g., xy position) and type of each cell in a tissue sample, the average distance between a tumor cell in the sample and its closest immune cell can be calculated. Such information may be relevant to why a positive patient did not respond to a particular treatment, for example. Many other relevant features can also be extracted from the raw data provided by the cell classifier, which can assist the searcher in better exploring hypotheses about therapeutic drugs.

[0064] Figure 3A is a block diagram showing a cell sorter 300 according to some embodiments of the present disclosure. Figure 3B Labeled images identifying different cell types detected by the cell classifier 300 are shown according to some embodiments of the present disclosure.

[0065] refer to Figure 3A In some embodiments, the cell classifier 300 receives an input image 302, which may be an image of a stained tissue sample (e.g., an image of an IHC slide), detects cells within the input image 302 (also referred to as a field of view (FOV)), and generates cell classification data 304 corresponding to the detected cells. The classification data 304 may include the type and location of each cell in the input image (e.g., a digitized red-green-blue (RGB) image) 302. The data 304 may further include a count of each identified cell type. According to some embodiments, the cell types classified by the cell classifier 300 may include stained immune cells (IC+), unstained immune cells (IC-), stained tumor cells (TC+), unstained tumor cells (TC-), stained macrophages (macrophage+), unstained macrophages (macrophage-), and / or other cells (which are not IC, TC, or macrophages). However, embodiments of the present disclosure are not limited thereto, and the cell classifier 300 may be trained to identify any suitable cell type. Figure 3B An example is shown in which the cell sorter 300 has identified and labeled different cells in an IHC slide.

[0066] In some embodiments, the cell classifier 300 includes a neural network (e.g., a convolutional neural network) 310 capable of cell detection and cell classification. The neural network 310 may include a plurality of layers, each of which performs a convolution operation on an input feature map (IFM) 312 via application of a kernel / filter to generate an output feature map, which is used as an input feature map 312 of a subsequent layer. In the first layer of the neural network 310, the input feature map may be an input image 302.

[0067] According to some examples, the neural network 310 mentioned in the present disclosure may be a convolutional neural network (ConvNet / CNN), which may receive an input image, assign importance to various aspects / objects in the image (e.g., via learnable weights and biases), and be able to distinguish one from another. However, embodiments of the present disclosure are not limited thereto. For example, the neural network 310 may be a recurrent neural network (RNN) with a convolution operation, etc.

[0068] The deep learning model of neural network 310 can be trained by providing neural network 310 with many instances of FOV 302 (e.g., more than one hundred thousand samples) and corresponding annotated data including the location of each cell on the FOV (e.g., xy location) and the type of each cell (e.g., label). The annotated cell type (e.g., cell label) can be one of "IC+", "IC-", "TC+", "TC-", "Macrophage+", "Macrophage-", and "Other cells". The visualization of this annotated cell data is shown in FIG. Figure 3B , where each cell is labeled with a colored shape corresponding to the cell type.

[0069] Some or all of the cell classification data 304 may be used by a regressor to determine a raw score (e.g., PD-L1 Sp142 whole slide IC score) for a given input image / FOV 302, and / or may be used by a researcher to explore various hypotheses about the efficacy of a particular treatment plan.

[0070] Figure 4 A block diagram of a cell-based scoring system 400 including a cell classifier 300 and a regressor 330 is shown according to some embodiments of the present disclosure.

[0071] According to some embodiments, the cell-based scoring system 400 is a sample / slide scoring system that is configured to determine a raw score for a pathology slide based on an image (e.g., a digitized image) of a tissue sample. The cell classifier 300 receives an image 302 of a pathology slide and classifies each of the cells captured in the image. As described above with respect to FIG. 3 , the cell classifier 300 can do this by identifying each cell in the image 302 and assigning the appropriate cell type to the cell.

[0072] In some embodiments, the cell-based scoring system 400 includes a feature generator (e.g., a cell-based feature generator) 320 configured to generate a plurality of first slide features (e.g., a plurality of extracted features) based on the classification of each cell. The feature generator 320 and the regressor 330 may be collectively referred to as a regression system 315. The feature generator 320 may generate a first feature by counting the number of cells of each cell type assigned to the plurality of cells and then generating a first feature based on the number of cells assigned to each cell type. For example, the first slide feature may include at least one of the number of IC+ cells, the number of IC- cells, the number of TC+ cells, the number of TC- cells, the number of other cells identified within the image 302, and the total number of identified cells. In some embodiments, the first slide feature may also include the area of ​​the tumor zone within the image 302 of the pathology slide, and the feature generator 320 may determine the area based on the annotated image 303 that identifies (e.g., depicts) a viable tumor region 303a. The tumor region 303a in the annotated image 303 may be generated (e.g., drawn / highlighted) by one or more pathologists. For example, a pathologist may outline a viable tumor region 303a in the image 302, from which the area of ​​the tumor region may be determined (e.g., by the feature generator 320). In some examples, the tumor region of the annotated image 303 may represent a consensus depiction (e.g., an average) of the region identified by multiple pathologists.

[0073] In some embodiments, the feature generator 320 also determines one or more second slide features (e.g., a plurality of calculated features) corresponding to the image 302 of the pathology slide based on the first slide features. The second slide features may include a field of view (FOV) area fraction, a cell area fraction, and a cell count fraction calculated based on the first slide features. For example, the FOV area fraction may be expressed as:

[0074]

[0075] The area occupied by all cells in the slide is calculated by summing the average size of each cell type across all cell types multiplied by the number of cells of that type. In some examples, the average immune cell size can be used as an estimate of the size of cells belonging to the "other cells" category.

[0076] Furthermore, the cell area fraction can be expressed as:

[0077]

[0078] Here, the average size of the immune cells may be a value (eg, a fixed value) provided to or known by the regressor 330 .

[0079] Alternatively, the cell count fraction can be expressed as:

[0080]

[0081] The second slide feature may represent a rough estimate of the actual slide score (e.g., PD-L1 Sp142 whole slide IC score), which is defined by Equation 1 above. The first and second slide features form a cumulative feature set, one or more of which is utilized by a cell-based scoring system (e.g., regressor 330) to determine a raw slide score 306.

[0082] According to some embodiments, the regressor 330 includes a regression model 340 that bridges the gap between the cell classification data generated by the cell classifier 300 and the original slide score 306. The regressor 330 is trained via a regression algorithm to learn the relationship (e.g., linear or nonlinear relationship) between one or more features in the cumulative feature set and the original slide score. Therefore, once trained, the regressor 330 can predict the original slide score of the pathology slide based on the value of one or more features supplied by the feature generator 320. Here, the term "regression" is informally used as a general name for mathematical modeling tasks whose output is a real number. Regression models may include K nearest neighbor (KNN) models, support vector machine (SVM) models, random forest (RF) models, and multi-layer perceptron (MLP) models, among others. For each model, deep learning may be employed to identify a corresponding set of input features that are most relevant to the algorithm's prediction of the original slide score. For example, when the regressor 330 includes a KNN machine learning model, it may use the tumor area, FOV area fraction, cell area fraction, and cell count fraction from the cumulative feature set to predict the original slide score. Further, when the regressor 330 includes an SVM machine learning model, it may use the tumor area, the number of IC+, IC-, TC+, TC-, and other cells, and the total number of cells in the pathology slide from the cumulative feature set to predict the original slide score. In addition, when the regressor 330 includes an RF machine learning model, it may use the tumor area, the number of IC+ cells, and the number of IC- cells in the pathology slide from the cumulative feature set to predict the original slide score. However, embodiments of the present disclosure are not limited thereto, and the regressor 330 may use any suitable set of input features from the cumulative feature set to perform its estimation / prediction of the slide score.

[0083] According to some embodiments, the regressor 330 may be a dedicated AI or a general AI and is trained using training data and an algorithm such as a back propagation algorithm. The training data may include many instances of one or more features from the accumulated feature set and corresponding consensus slide scores from multiple pathologists.

[0084] The regressor 330 may include a set of weights for each of the parameters of the linear regression model, or the regressor 330 may include a set of weights for the connections between neurons of a trained neural network. In some embodiments, one or more features from the accumulated feature set are supplied to the regressor 330 as values ​​(e.g., input features) to an input layer of the regressor 330, and these values ​​(or a set of intermediate values) are forward propagated through the regressor 330 to generate an output, where the output is a raw slide score.

[0085] According to some embodiments, the cell-based scoring system 400 includes a controller 350 for controlling the operation of the classifier 300, the feature generator 320, and the regressor 330, and further includes a memory 360 (e.g., a logic on-die memory) for temporarily storing the input image 302, the output or intermediate results of the neural network 310 (e.g., the output feature maps generated by the layers of the neural network 310), and the regression model 340. In some examples, the memory 360 can be an embedded magnetoresistive random access memory (eMRAM), a static random access memory (SRAM), etc.

[0086] To evaluate the performance of the cell-based scoring system 400 (i.e., the combination of classifiers and regressors 300 and 330), a representative benchmark of 100 cases closely following the prevalence of breast cancer specimens was assembled. Three pathologists independently provided the entire slide score for all 100 cases in the benchmark set, and the median of the three scores was calculated as the consensus score for each case in the benchmark set. Table 1 is a confusion matrix for a representative benchmark set, which highlights the ability of the cell-based scoring system 400 to predict the slide score and correctly classify the patient as negative (i.e., the slide score is less than the 1% threshold) or positive (i.e., the slide score is greater than or equal to the 1% threshold).

[0087] Table 1:

[0088]

[0089] As shown in the example of Table 1, the number of true positive (TP) and true negative (TN) classifications performed by the cell-based scoring system 400 identified 98% of the benchmark cases, with only two percent of the positive cases being incorrectly (e.g., erroneously) labeled as negative (i.e., false negatives (FN)), and no false positives (FP).

[0090] Performance metrics of the cell-based scoring system 400 on representative benchmarks are presented in Table 2 below.

[0091] Table 2:

[0092]

[0093] The metrics in Table 1 are defined as follows:

[0094]

[0095]

[0096] The desired effect of using a regressor in the cell-based scoring system 400 is that it allows compensation for inherent biases or corrections for systematic errors that may exist in the cell classifier 300. For example, the cell classifier 300 may systematically undercount the number of IC+ cells. However, since the regressor 330 is trained based on data from the cell classifier 300, errors and biases are learned by the regressor 330 and do not translate into incorrect slide scores. In other words, any errors in the output of the cell classifier 300 are compensated by the regressor 330 and are not necessarily propagated through the regressor. Therefore, the cell-based scoring system 400 is able to produce accurate results.

[0097] According to various embodiments of the present disclosure, the cell-based scoring system 400 is implemented using one or more processing circuits or electronic circuits configured to perform various operations as described above. The types of electronic circuits may include a central processing unit (CPU), a graphics processing unit (GPU), an artificial intelligence (AI) accelerator (e.g., a vector processor, which may include a vector arithmetic logic unit configured to efficiently perform operations common to neural networks (such as dot products and softmax)), a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a digital signal processor (DSP), etc. For example, in some cases, various aspects of the embodiments of the present disclosure are implemented with program instructions stored in a non-volatile computer-readable memory, wherein when executed by an electronic circuit (e.g., a CPU, a GPU, an AI accelerator, or a combination thereof), the program instructions perform the described operations. The operations performed by the cell-based scoring system 400 may be performed by a single electronic circuit (e.g., a single CPU, a single GPU, etc.), or may be distributed between multiple electronic circuits (e.g., multiple GPUs or a CPU in combination with a GPU). The multiple electronic circuits may be local to each other (e.g., on the same die, within the same package, or within the same embedded device or computer system) and / or may be remote to each other (e.g., via a network such as a local personal area network such as The computer system 400 may communicate over a local area network, such as a local wired and / or wireless network, and / or over a wide area network, such as the Internet, in which case some operations are performed locally and other operations are performed on servers hosted by a cloud computing service. The one or more electronic circuits that operate to implement the cell-based scoring system 400 may be referred to herein as a computer or computer system, which may include a memory storing instructions that, when executed by the one or more electronic circuits, implement the systems and methods described herein.

[0098] Figure 5 is a flow chart illustrating a process 500 of determining a raw score for a pathology slide from a tissue sample using cell-based classification data corresponding to the pathology slide, according to some embodiments of the present disclosure.

[0099] In some embodiments, the regression system 315 receives a plurality of first slide features corresponding to an image (eg, digitized image) 302 of a pathology slide (S502). The pathology slide may be stained with PD-L1 SP142.

[0100] The regression system 315 calculates one or more second slide features corresponding to the pathology slide based on the plurality of first slide features (S504). This may include calculating at least one of the field of view (FOV) area fraction, cell area fraction, and cell count fraction defined in equations 2 to 4.

[0101] The regression system 315 determines a raw score based on one or more features in the cumulative feature set including the plurality of first slide features and the one or more second slide features (S506). This may include providing the one or more features in the cumulative feature set to a trained regression model 340, the trained regression model being configured to relate raw score values ​​to values ​​of the one or more features, and estimating, by the trained regression model 340, a raw score corresponding to the one or more features.

[0102] The cell-based scoring system 400 may then compare the raw score to a threshold value (eg, 1%) to determine the efficacy of a treatment for a patient associated with the tissue sample.

[0103] Figure 6 is a flow chart showing a process 600 of determining a raw score for a pathology slide from a tissue sample, according to some embodiments of the present disclosure.

[0104] In some embodiments, the cell-based scoring system 400 receives an image (e.g., a digitized image) 302 of a pathology slide (S602). The cell-based scoring system 400 classifies each of a plurality of cells captured in the image 302 by providing the image 302 to a classifier 300, the classifier being configured to identify each of the plurality of cells and being configured to assign a cell type from a plurality of cell types to each of the plurality of cells (S604). The cell-based scoring system 400 then generates a plurality of first slide features based on the classification of each cell (S606), and determines a raw score based on one or more features in a cumulative feature set including the plurality of first slide features (S608). Generating a plurality of first slide features may include counting the number of cells assigned to each cell type of the plurality of cells, and generating a plurality of first slide features based on the number of cells assigned to each cell type. It may further include receiving an area of ​​a tumor region 303a corresponding to the image 302 and adding (e.g., including) it to the cumulative feature set. The cell-based scoring system 400 can determine a raw score by providing one or more features in the cumulative feature set to a trained regression model 340, the trained regression model being configured to relate raw score values ​​to values ​​of the one or more features; and estimating, by the trained regression model 340, a raw score corresponding to the one or more features.

[0105] The cell-based scoring system 400 may then compare the raw score to a threshold value (eg, 1%) to determine the efficacy of a treatment for a patient associated with the tissue sample.

[0106] Thus, as described above, the cell-based scoring system has the ability to determine cell-level data from digitized images of pathology slides and the ability to convert the cell-level data into a slide score (e.g., PD-L1 SP142 whole slide score). The cell-based scoring system is constructed in such a way that it not only predicts the whole slide score, but also provides valuable cell-level information that can be used by pharmaceutical companies for drug development and discovery and to explore various hypotheses. The rapid and accurate results generated by the cell-based scoring system eliminate the need for time-consuming and tedious manual scoring by pathologists, which is subject to inaccuracies and significant inter- and intra-observer variability.

[0107] Although some of the examples provided herein describe processing pathology slides stained with PD-L1 SP142, embodiments of the present disclosure are not limited thereto, and the cell-based scoring system of the present disclosure may operate on pathology slides stained with any suitable biomarker.

[0108] The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used herein, the singular forms "one" and "a kind of" are also intended to include plural forms, unless the context clearly indicates otherwise. It should be further understood that the terms "comprises", "comprising", "includes" and "including" when used in this specification specify the existence of stated features, integers, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, parts and / or their groups. As used herein, the term "and / or" includes any combination and all combinations of one or more of the associated listed items. Statements such as "at least one of..." modify the entire element list when located before the element list and do not modify the individual elements in the list.

[0109] It should be understood that although the terms "first", "second", "third", etc. may be used herein to describe various elements, components, regions, layers and / or sections, these elements, components, regions, layers and / or sections should not be limited by these terms. These terms are used to distinguish one element, component, region, layer or section from another element, component, region, layer or section. Therefore, without departing from the spirit and scope of the present invention, the first element, component, region, layer or section described below may be referred to as a second element, component, region, layer or section.

[0110] As used herein, the terms "substantially," "about," and similar terms are used as terms of approximation rather than as terms of degree, and are intended to account for the inherent deviations in measured or calculated values ​​that would be recognized by one of ordinary skill in the art. Further, the use of "may" when describing embodiments of the present invention refers to "one or more embodiments of the present invention." As used herein, the terms "use," "using," and "used" may be considered synonymous with the terms "utilize," "utilizing," and "utilized," respectively.

[0111] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those of ordinary skill in the art to which the present invention belongs. It should be further understood that terms (such as those defined in commonly used dictionaries) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and / or this specification, and should not be interpreted in an idealized or overly formal sense, unless explicitly defined as such herein.

[0112] Although aspects of some example embodiments of systems and methods for quantifying pathology slides using a cell-based scoring system have been described and illustrated herein, as will be appreciated by those of ordinary skill in the art, various modifications and variations may be implemented without departing from the spirit and scope of embodiments according to the present disclosure. Therefore, it should be understood that the pathology slide manufacturing system and method according to the principles of the present disclosure may be embodiments other than those specifically described herein. The present disclosure is also defined in the appended claims and their equivalents.

Claims

1. A method of determining a raw score of a pathology slide from a tissue sample, the method comprising: receiving, by a regression system, a plurality of first slide features corresponding to the pathology slide; calculating, by the regression system, one or more second slide features corresponding to the pathology slide based on the plurality of first slide features; as well as The raw score is determined by the regression system based on one or more features in a cumulative feature set including the plurality of first slide features and the one or more second slide features.

2. The method of claim 1, wherein the pathology slide is stained using an immunohistochemical assay.

3. The method of claim 1, wherein the plurality of first slide features comprises at least one of: the area of ​​the tumor region of the pathology slide; the number of stained immune cells of the pathology slide; the number of unstained immune cells of the pathology slide; the number of stained tumor cells of the pathology slide; the number of unstained tumor cells on the pathology slide; the number of other cells on the pathology slide; and The total number of cells on the pathology slide.

4. The method of claim 1 , wherein calculating the one or more second slide characteristics comprises: At least one of the field of view (FOV) area fraction, the cell area fraction, and the cell count fraction is calculated.

5. The method of claim 4, wherein the FOV area fraction is expressed as: Wherein the average size of IC+ cells represents the average size of stained immune cells, and the number of IC+ cells represents the number of stained immune cells of the pathology slide.

6. The method of claim 4, wherein the cell area fraction is expressed as: Wherein the average size of IC+ cells represents the average size of stained immune cells, the number of IC+ cells represents the number of stained immune cells of the pathology slide, and the area of ​​the tumor represents the area of ​​the tumor region corresponding to the pathology slide.

7. The method of claim 4, wherein the cell count fraction is expressed as: Wherein the number of IC+ cells represents the number of stained immune cells of the pathology slide, and the total number of cells represents the total number of cells of the pathology slide.

8. The method of claim 1 , wherein determining the raw score comprises: providing the one or more features in the accumulated feature set to a trained regression model, the trained regression model configured to relate raw score values ​​to values ​​of the one or more features; as well as The raw scores corresponding to the one or more features are estimated by the trained regression model.

9. The method of claim 1, wherein the regression system comprises a trained machine learning model configured to correlate the one or more features in the cumulative feature set with the raw score.

10. The method of claim 9, wherein the trained machine learning model comprises one of a K-nearest neighbor (KNN) model, a support vector machine (SVM) model, a random forest (RF) model, and a multi-layer perceptron (MLP) model.

11. The method according to claim 1, further comprising: The raw score is compared to a threshold value to determine the efficacy of a treatment for the patient associated with the tissue sample.

12. The method according to claim 1, further comprising: receiving, by a classifier, an image of the pathology slide; classifying, by the classifier, each of the plurality of cells by identifying each of the plurality of cells captured in the image and assigning a cell type from a plurality of cell types to each of the plurality of cells; as well as The plurality of first slide features are generated by the classifier based on the classification of each cell.

13. The method of claim 12, wherein the classifier comprises a convolutional neural network.

14. A method of determining a raw score of a pathology slide from a tissue sample, the method comprising: receiving an image of the pathology slide by a cell-based scoring system including processing circuitry and memory; classifying, by the cell-based scoring system, each cell in a plurality of cells captured in the image by providing the image to a classifier of the cell-based scoring system, the classifier being configured to identify each cell in the plurality of cells and being configured to assign a cell type from a plurality of cell types to each cell in the plurality of cells; generating, by the cell-based scoring system, a plurality of first slide features based on the classification of each cell; as well as The raw score is determined by the cell-based scoring system based on one or more features in a cumulative feature set that includes the plurality of first slide features.

15. The method of claim 14, wherein generating the plurality of first slide features comprises: counting the number of cells of each cell type assigned to the plurality of cells; as well as The plurality of first slide features are generated based on the number of cells assigned to each cell type.

16. The method according to claim 14, further comprising: An area of ​​a tumor region corresponding to the image is received by the cell-based scoring system, wherein the accumulated feature set further includes the area of ​​the tumor region.

17. The method of claim 14, wherein the plurality of first slide features comprises at least one of: the number of stained immune cells of the pathology slide; the number of unstained immune cells of the pathology slide; the number of stained tumor cells of the pathology slide; the number of unstained tumor cells on the pathology slide; the number of other cells on the pathology slide; and The total number of cells on the pathology slide.

18. The method of claim 14, further comprising: One or more second slide features corresponding to the pathology slide are calculated by the cell-based scoring system based on the plurality of first slide features, wherein the cumulative feature set further includes the one or more second slide features.

19. The method of claim 18, wherein calculating the one or more second slide characteristics comprises: At least one of the field of view (FOV) area fraction, the cell area fraction, and the cell count fraction is calculated.

20. The method of claim 14, wherein determining the raw score comprises: providing the one or more features in the accumulated feature set to a trained regression model, the trained regression model configured to relate raw score values ​​to values ​​of the one or more features; as well as The raw scores corresponding to the one or more features are estimated by the trained regression model.

21. The method of claim 14, further comprising: The raw score is compared to a threshold value to determine the efficacy of a treatment for the patient associated with the tissue sample.

22. A cell-based scoring system for determining a raw score of a pathology slide from a tissue sample, the cell-based scoring system comprising: A classifier comprising a convolutional neural network, the classifier being configured to: receiving an image of the pathology slide; classifying each of the plurality of cells by identifying each of the plurality of cells captured in the image and assigning a cell type from a plurality of cell types to each of the plurality of cells; generating a plurality of first slide features based on the classification of each cell; a cell-based feature generator configured to calculate one or more second slide features corresponding to the pathology slide based on the plurality of first slide features; and A regressor is configured to determine the raw score based on one or more features in a cumulative feature set including the plurality of first slide features and the one or more second slide features.

Images