Distance-based tissue state determination

Abstract

The present invention relates to an image analysis method for determining the biomedical state of a tissue sample, the method comprising: - receiving (202) a digital image (118) of a tissue sample; - Identifying the number and location of type A cells (504) and type B cells (506) (204); - Obtaining the observed relative distribution (708, 706) (206); - obtaining a reference relative distribution (710, 714) of expected distances between reference type A cells and reference type B cells (208); - calculating the proximity score (718, 720) as the difference between the reference relative distribution and the observed relative distribution (210); - calculating (212) a combined score (912) comprising the proximity score and the density of type A cells and / or the density of type B cells; - using the combined score to determine the biomedical state of the tissue sample (214) and / or outputting the combined score (216).

Description

[Technical Field]

[0001] The present invention relates to the field of image analysis, and more particularly to image-based classification of biomedical conditions in tissue samples. [Background technology]

[0002] Today, several different image analysis methods are used to automatically determine biomedical characteristics of tissue samples. For example, image analysis is used to automatically identify cells, to segment digital pathology images into background and tissue regions, and to automatically identify and / or quantify biomedical conditions, such as specific diseases.

[0003] An example is the detection of biomarkers, e.g., tumor markers, in the diagnosis of leukemia, breast cancer, colon cancer, and other types of cancer. This involves labeling cells with antibodies against proteins that are specifically expressed or overexpressed in particular types of cells, e.g., tumor cells or other types of cells. By selecting the appropriate antibodies, different types of cells can be accurately determined.

[0004] Immunophenotyping is a technique commonly used for basic science research and laboratory diagnostic purposes. It uses antibodies specifically directed against a particular target molecule to identify the "phenotype" of a cell based on the expression level of one or more proteins. The cellular phenotype of one or more cells within a tissue can also be used to gain insight into the biomedical state of the tissue, for example, whether the tissue is healthy, a primary tumor, or a metastatic tissue.

[0005] A common problem associated with many image analysis algorithms currently used to analyze digital pathology images is that the expression level of a particular protein alone may enable the correct identification of cells of a particular cell type, but may not be sufficient to accurately, reproducibly, and automatically determine the biomedical status of the tissue.

[0006] In their paper "Measuring multiple parameters of CD8+ tumor-infiltrating lymphocytes in human cancers by image analysis" (J Immunother Cancer. 2018 Mar 6;6(1):20. doi:10.1186 / s40425-018-0326-x), Steele KE et al. characterize the immune response to cancer based on the number and location of CD8+ tumor-infiltrating lymphocytes (TILs). The strength of the immune response is considered a parameter with potential prognostic, pharmacodynamic, and predictive potential.

[0007] However, the ability of known approaches to provide accurate quantitative data in a manner that also addresses spatial aspects of cell distribution remains limited. Furthermore, the subjectivity and variability of methods used to examine spatial relationships performed by multiple laboratories hinders the ability to compare histopathological results. Summary of the Invention

[0008] The object of the present invention is to provide an improved method and image analysis system for determining the biomedical state of a tissue sample as specified in the independent claims. Embodiments of the invention are set out in the dependent claims. The embodiments of the invention can be freely combined with one another if they are not mutually exclusive.

[0009] In one aspect, the present invention relates to an image analysis method for determining a biomedical condition of a tissue sample, the method being implemented in an image analysis system, the method comprising: - receiving a digital image of the tissue sample; - analyzing the received image to identify the number and location of type A cells and type B cells observed in an area of ​​the received image, where type A and type B are different cell types; - analyzing the locations of type A and type B cells in the region to obtain an observed relative distribution, the observed relative distribution indicating an observed distance between type A cells and type B cells in the region; - obtaining a reference relative distribution, the reference relative distribution indicating an expected distance between reference type A cells and reference type B cells; - Calculating the proximity score as the difference between the reference relative distribution and the observed relative distribution; - calculating a combination score, the combination score including a proximity score and including a density of type A cells and / or a density of type B cells; - using the combined score to determine the biomedical state of the tissue sample and / or outputting the combined score to a user to enable the user to determine the biomedical state of the tissue sample. Includes.

[0010] This feature can be advantageous because, in contrast to state-of-the-art methods, the spatial distribution of different cell types is not subjectively determined by a pathologist. Rather, the observed relative distribution is compared to a reference relative distribution, i.e., a distribution representing the expected distance between reference type A cells and reference type B cells in a region. The expected distance can be, for example, the distance expected assuming that type A cells and / or type B cells are distributed according to a specific predetermined mathematical distribution, such as a Poisson distribution. Alternatively, the expected distribution can be determined experimentally based on the analysis of type A and type B cells in one or more other tissue samples with known biomedical conditions. By defining a reference relative distribution and calculating a proximity score as the difference between the reference and observed relative distributions, a reproducible measure of the relative spatial location of two different cell types in a tissue sample is provided. The proximity score can provide a reproducible quantitative measure of the deviation of the observed relative distribution / distance of type A and type B cells from the expected relative distribution / distance of type A and B cells.

[0011] The proximity score can be thought of as representing the "delta" between the observed relative distribution and the reference relative distribution. The proximity score integrates the spatial difference in the distance between type A cells and type B cells represented in the observed relative distribution and the reference relative distribution. Preferably, the proximity score is a single numerical data value that integrates distance information obtained from multiple type A and type B cells. For example, the "delta" can be calculated as the difference in radius as shown in FIG. 7 or the difference in area as shown in FIG. 8.

[0012] In a further advantageous aspect, the proximity score can be a highly aggregated data value because it can be a single data value obtained by processing multiple observed and expected type A and type B cell distances. Therefore, it becomes possible to consider the information encoded in the spatial distribution of hundreds, or even thousands, of cells, which can be used to reproducibly determine the biomedical state of a particular tissue. The information encoded in the spatial location of cells is complex, and their interpretation is also complex. By providing a single score value calculated as a function of the observed and expected type A cell-B cell distances, a parameter value is provided that encodes a wealth of biomedical information and, therefore, may have high predictive relevance for various predictive tasks, such as determining the biomedical state of a tissue or tissue region.

[0013] In a further advantageous embodiment, the proximity score is combined with additional information, namely, the density of type A and / or type B cells in the image region, to provide a final, combined score. The combined score can be advantageous because it provides both "relative" information (i.e., the deviation of the observed A-cell-B cell distance from the expected A-cell-B cell distance) and "absolute" information, namely, the density of type A or B cells in the interrogated image region. The combination of absolute and relative information can be beneficial because it can resolve differences in the biomedical state of a tissue sample that cannot be identified based on proximity scores alone or on the density of one of the cell types alone. For example, there may be multiple different biological tissue states that have the same proximity score but can be distinguished based on differences in the absolute density of type B or type A cells. Additionally or alternatively, there may be multiple different biomedical tissue states that have the same density of type B cells (or type A cells) but can be distinguished based on differences in proximity scores.

[0014] Embodiments of the present invention allow one to identify a hypothesis that a particular biomedical condition, e.g., a particular tumor type and stage, causes the observed distribution of cells to deviate from an expected distribution (e.g., a random distribution such as a Poisson distribution), and easily explore this hypothesis by calculating a proximity score. If the proximity score is zero or close to zero, then essentially there is no difference between the observed relative distribution and the expected reference relative distribution, and the hypothesis can be accepted. For example, the hypothesis can be that when a tissue sample is a tumor tissue sample taken from a tumor of a particular type and stage, the tumor microenvironment causes the observed relative distribution of tumor cells (type A cells) and immune cells (type B cells) to deviate from a random reference distribution.

[0015] Thus, the combined score may provide a highly aggregated parameter that has high predictive power and may enable reproducible characterization of spatial information related to the relative distribution of type A and type B cells in tissue. Proximity scores and combined scores may have the additional advantage that they can be used to characterize the relative distribution of any two different cell types and, therefore, can be used to automatically determine biomedical conditions related to many different biological problems. Thus, a highly reproducible, non-subjective yet generic score value is provided that can be used to automatically, reproducibly, and accurately determine the biomedical condition of a tissue sample.

[0016] Embodiments of the present invention may allow for avoidance of discrepancies when results of IHC biomarker analyses performed by multiple laboratories are compared, allowing for better inter-laboratory harmonization.

[0017] Furthermore, it was observed that the combined score is able to capture more complex histopathological information than state-of-the-art image analysis based methods for determining biomedical tissue status.

[0018] According to an embodiment, cells of one of the two cell types having a higher cell density in the image area are referred to as type A cells, and cells of the other cell type are referred to as type B cells.

[0019] According to a preferred embodiment, the distance between type A cells and type B cells is measured starting from the type A cells, and the density used to calculate the combined score is the density of type B cells in the image area.

[0020] According to embodiments, the density of reference type A cells is the same as or similar to the density of observed type A cells in the image region. For example, the reference type A cells can be simulated in the same image region where the type A cells were observed, such that the number of simulated reference type A cells is the same as or similar to the number of observed type A cells in the image region (e.g., + / - 10%). Furthermore, the density of reference type B cells is the same as or similar to the density of observed type B cells in the image region. For example, the reference type B cells can be simulated in the same image region where the type B cells were observed, such that the number of simulated reference type B cells is the same as or similar to the number of observed type B cells in the image region (e.g., + / - 10%).

[0021] Alternatively, the reference type A cells and reference type B cells are cells observed in a reference tissue sample. The reference tissue sample is selected such that the number and density of type A cells and type B cells in the reference tissue sample can be expected to be similar to the density of type A cells and type B cells in the tissue sample shown in the received image. This can ensure that differences in the observed and reference relative distributions are caused by factors other than cell density.

[0022] According to an embodiment, obtaining the observed relative distribution includes, for each identified type A cell (e.g., tumor cell) observed in the image region: a) selecting said type A cell as the center of a circle with a radius of 0; b) Increasing the radius by one step to generate an increased cycle; c) determining the number of B cells contained in the cycles generated in step b); d) storing in a storage medium the current radius of the circle associated with the number of type B cells determined in step b), and repeating steps b), c), and d) until a termination criterion is reached; and e) selecting one of the unselected type A cells and continuing a) with said newly selected type A cell until all type A cells have been selected.

[0023] The associated radius and number of observed type B cells obtained for each identified type A cell and radius is stored as the observed relative distribution.

[0024] For example, termination criteria can be a maximum radius, a maximum number of steps, reaching a maximum execution time, or a maximum number of iterations. This feature can be advantageous because a stepwise increase in radius can cover a wide range of cell-cell distances with relatively low computational effort. For example, steps can be selected such that the radius increases by a size of 1 to 100 μm, e.g., 10 μm, at each step. The radius can be increased, for example, until a predetermined maximum radius, e.g., about 500 μm or about 200 μm, is reached. By selecting the size, and therefore the number of steps and cell counting operations, a discrete data set can be obtained that reflects the observed relative spatial distribution of type A and type B cells.

[0025] According to an embodiment, the observed relative distribution is calculated based on the bivariate Ripley's K(t) function of observed type A and type B cells in the image domain.

[0026] For example, Ripley's K function was described by Philip M. Dixon in Volume 3, pp 1796-1803 of the Encyclopedia of Environmetrics (ISBN 0471 899976) Edited by Abdel H. El-Shaarawi and Walter W. Piegorsch, John Wiley & Sons, Ltd, Chichester, 2002. A generalization of Ripley's K(t) function to several types of points (multivariate spatial point processes) is This is described in the above document as equation (11) according to TIFF0007749461000001.tif15170.

[0027] This generalization can be advantageous because the Ripley K(t) function allows for recording and modeling the relative spatial distribution of two or more point patterns in two-dimensional space. It is relatively fast and is an integral part of several computational tools, e.g., mathematical software such as "R."

[0028] According to an embodiment, the observed relative distribution is According to TIFF0007749461000002.tif13170, TIFF0007749461000003.tif7170 is calculated as: - During the ceremony, TIFF0007749461000004.tif7170 is the bivariate Ripley's K(t) function, - where i is the occurrence of the object type of the "observed type A cell"; - where j is the occurrence of the object type "observed B-cell"; - During the ceremony, TIFF0007749461000005.tif7170 is the density of observed type B cells (number per image area); - During the ceremony, TIFF0007749461000006.tif6170 is the total number of observed type A cells identified within the image area; - During the ceremony, TIFF0007749461000007.tif6170 is the total number of observed type B cells identified within the image area; - During the ceremony, TIFF0007749461000008.tif6170 is a cell that is an observed type A cell (also considered the "currently investigated cell"); where r is the stepwise increasing radius located at the center of the observed A-type cell; - where t is the sum of u, r, and A function for TIFF0007749461000009.tif6170 that counts the number of observed type B cells in a circle of radius r around an observed type A cell u in the image region; - During the ceremony, TIFF0007749461000010.tif6170 means "Across all of the u's that are observed type A cells"; - where E is the expectation of t obtained over all u.

[0029] In general, the "expectation" E of t obtained over all u describes the expected value of a variable (in this case t). For example, the expected value E can be calculated as the average, e.g., the probability-weighted average, of all possible values ​​of t obtained over all u. Intuitively, E is like an "average outcome." For example, E of t obtained over all u can be calculated as the average t obtained over all u.

[0030] As used herein, an "average" is a single number taken as representative of a list of numbers. The "average" can be, for example, the arithmetic mean, i.e., the sum of the numbers divided by the number being averaged. Alternatively, the average can be the median or mode. The average is a measure of central tendency of a set of numbers.

[0031] According to an embodiment, the observed relative distribution is calculated based on the derivative of the bivariate Ripley's K(t) function of type A and type B cells observed in the image region.

[0032] According to one example, the derivative bivariate Ripley's K(t) function is a linear or non-linear transformation of the bivariate Ripley's K(t) function. In particular, the derivative bivariate Ripley's K(t) function can be a linear transformation of the bivariate Ripley's K(t) function. A linear transformation can be, for example, According to TIFF0007749461000011.tif15170, For TIFF0007749461000012.tif6170 It could be the square root of TIFF0007749461000013.tif7170.

[0033] TIFF0007749461000014.tif7170 is also called the "L function" or "Besag's L function." A linear transformation approximately stabilizes the variance of the estimate K when the process is Poisson, and the benchmark value TIFF0007749461000015.tif7170 Convert to TIFF0007749461000016.tif7170.

[0034] According to an embodiment, the reference relative distribution is calculated based on the bivariate Ripley's K(t) function of simulated type A cells and simulated type B cells in the image region.

[0035] According to an embodiment, obtaining the reference relative distribution comprises: - computer simulating a distribution of simulated reference type A cells, wherein the density of the simulated reference type A cells is the same as the density of observed type A cells in the image region; - simulating, by a computer, a distribution of simulated reference B cells, wherein the density of the simulated reference B cells is the same as the density of observed B cells in the image region; - calculating a reference relative distribution as a function of the computer-simulated distribution of simulated reference type A cells and simulated reference type B cells, the reference relative distribution indicating the distance between the simulated reference type A cells and the simulated reference type B cells in the region. Includes.

[0036] Computational generation of simulated reference type A cells and simulated reference type B cells in an image region (a "simulation" of type A and B cells and their distributions based on a predetermined, expected distribution) may have the advantages that no additional (typically rare) tissue is required as a reference, and that, if the number of simulated reference type A and reference type B cells is sufficiently large, the distributions of reference type A and reference type B cells are guaranteed to accurately represent the expected distributions. Thus, simulation of a "virtual" tissue image region containing the expected (e.g., Poisson) distribution of simulated reference type A and reference type B cells is computationally inexpensive and does not require the acquisition and staining of one or more reference tissue samples to provide an experimental reference base and individual reference images. Furthermore, simulation of reference type A and type B cells is highly flexible, allowing the simulated distributions to be arbitrarily modified. This may allow for "fine-tuning" biomedical hypotheses and for accurately determining complex bivariate codistribution patterns that do not follow a Poisson distribution. For example, a complex codistribution pattern could correspond to a high fraction of reference type A cells with reference type B neighbors in exactly two concentric "belts" around the reference type A cells. Such complex patterns may rarely be observed in nature. However, simulations make it possible to generate "idiotypic" codistributions that may rarely be observed in their "ideal" form in real tissue samples, but which may nevertheless be useful for formulating and confirming biological hypotheses related to the spatial distribution of cells in tissue samples.

[0037] According to an embodiment, the reference relative distribution is derived from a simulated distribution of reference type A and reference type B cells, whereby the simulated reference type A cell distribution and / or the simulated reference type B cell distribution may be, for example, a Poisson distribution.

[0038] According to a preferred embodiment, the distribution of the simulated reference type A cells is a Poisson distribution and the distribution of the simulated reference type B cells is a Poisson distribution.

[0039] The Poisson distribution is a discrete probability distribution that represents the probability of a specific number of events occurring in a fixed time or space interval when these events occur at a known, constant rate and independent of the time since the last event. The Poisson distribution can also be used for the number of events over other specific intervals, such as distances, areas, or volumes. If the occurrence of any particular type of event does not affect the time probability of future events, i.e., if observed events occur independently of one another, a reasonable assumption is that the number of events occurring in a day follows a Poisson distribution. Thus, by generating a Poisson-distributed set of simulated type A cells in an image region and a Poisson-distributed set of simulated type B cells in an image region, and by determining the reference relative distributions of simulated reference type A and reference type B cells, the hypothesis that the distribution of type A cells is completely independent of the distribution of reference type B cells, and vice versa, can be simulated and tested for the tissue sample currently under investigation.

[0040] According to embodiments, the density of simulated reference type A cells in a region is identical or highly similar to the density of observed type A cells in the region of the image. Here, the term "highly similar" means that the number of simulated reference type A cells in the image region is + / - 10% identical to the number of observed type A cells in the region of the image. The number of simulated reference type B cells in the region is identical or highly similar to the number of observed type B cells in the region of the image.

[0041] This can be advantageous because it ensures that the total number and density of simulated reference type A and type B cells in the image region are identical to or highly similar to the respective numbers and densities of the observed cells. Therefore, any deviation of the observed relative distribution from the reference relative distribution is due only to differences in the relative spatial distribution of cells of the two cell types, not due to any deviation in the number or density of the simulated cells. Thus, while the reference relative distribution may be based on simulated reference type A and type B cells, the simulation is strictly constrained to the observed densities of type A and type B cells in the tissue sample shown in the image region. This can make it possible to perform tissue-sample-specific simulations that can accurately determine any deviations in the relative spatial distribution of cells of the two cell types in the tissue sample compared to the expected, simulated distribution. Differences in spatial distribution from the simulated reference distribution are not affected by any other distribution factors that may be relevant when using a reference tissue sample to experimentally create a reference relative distribution.

[0042] According to an embodiment, calculating the reference relative distribution as a function of the computer-simulated distributions of the reference type A and reference type B cells comprises: For each of the randomly distributed simulated reference type A cells (e.g., tumor cells), perform the following steps: a) selecting the simulated reference type A cell as the center of a circle with a radius of 0; b) Increasing the radius by one step to generate an increased cycle; c) determining the number of simulated reference B-type cells (e.g., immune cells) contained in the cycles generated in step b); d) storing in a storage medium the current radius of the circle associated with the number of simulated reference B cells determined in step b), and repeating steps b), c), and d) until a termination criterion is reached; and e) selecting one of the unselected simulated reference type A cells and continuing step a) using said newly selected simulated reference type A cell until all simulated reference type A cells have been selected. will be carried out This includes:

[0043] The method further includes providing the associated radius and number of the simulated reference B cells as a reference relative distribution.

[0044] The advantage of this stepwise calculation of the number of reference type B cells in stepwise increasing circles around each reference type A cell may have the advantages already described for the similar determination of the radius and number of type B cells stored during the calculation of the observed relative distribution.

[0045] According to an embodiment, the reference relative distribution is According to TIFF0007749461000017.tif15170, TIFF0007749461000018.tif7170 is calculated as: - During the ceremony, TIFF0007749461000019.tif7170 is the K(t) function of bivariate replay, - where i is the occurrence of the object type of the "simulated reference type A cell"; - where j is the occurrence of the object type of the "simulated reference B-cell"; - During the ceremony, TIFF0007749461000020.tif7170 is the density of simulated reference B cells (number per image area); - During the ceremony, TIFF0007749461000021.tif6170 is the total number of simulated reference type A cells in the image area; - During the ceremony, TIFF0007749461000022.tif6170 is the total number of simulated reference B cells in the image area; - During the ceremony, TIFF0007749461000023.tif6170 is the simulated reference type A cell; where r is the stepwise increasing radius located at the center of the simulated reference A-type cell; - where t is the sum of u, r, and A function for TIFF0007749461000024.tif6170 that counts the number of simulated reference type B cells in a circle of radius r around a simulated reference type A cell u in the image region; - During the ceremony, TIFF0007749461000025.tif6170 means "across all of u, the simulated reference A-type cells"; - where E is the expected value of t over all u.

[0046] As defined above, the "expected value" E of t obtained over all u describes the expected value of a variable (in this case t). For example, the expected value E can be calculated as the average, e.g., probability-weighted average, of all possible values ​​of t obtained over all u. For example, E of t obtained over all u can be calculated as the average t obtained over all u.

[0047] According to an embodiment, the reference relative distribution is calculated based on the derivative of the bivariate Ripley's K(t) function of the simulated type A cells and the simulated type B cells. According to one example, the derivative bivariate Ripley's K(t) function is a linear or non-linear transformation of the bivariate Ripley's K(t) function. In particular, the derivative bivariate Ripley's K(t) function can be a linear transformation of the bivariate Ripley's K(t) function. A linear transformation can be, for example, According to TIFF0007749461000026.tif15170, For TIFF0007749461000027.tif6170 It could be the "L function" or "Besag's L function" calculated as the square root of TIFF0007749461000028.tif6170.

[0048] According to an embodiment, the reference relative distribution and / or the observed relative distribution can be: It is calculated as a version of Ripley's K(t) function as described in equation (12) of the above-mentioned article by Philip M. Dixon, according to TIFF0007749461000029.tif10170, where: TIFF0007749461000030.tif7170 is a graph of radius d within the research area. ik,jl is the fraction of the circumference of a circle centered at the kth position of process I with

[0049] According to an embodiment, obtaining the reference relative distribution includes calculating a plurality of initial reference relative distributions according to any one of the embodiments and examples described herein, and the method includes calculating an average reference relative distribution from the plurality of initial reference relative distributions and using the average reference relative distribution as the reference relative distribution.

[0050] Preferably, the proximity score is calculated as the difference ("delta") between the average reference relative distribution and the observed relative distribution. This includes the difference between parameter values ​​associated with the average reference relative distribution and the observed relative distribution. For example, this difference could be the difference in radii associated with the two distributions shown in Figure 7, or the difference in area shown in Figure 8.

[0051] For example, a number n of initial reference relative distributions can be calculated, where n is preferably greater than 10, more preferably greater than 30, e.g., 39. An "average relative distribution" is then calculated for each of a plurality of radii used to count type B cells in the spatial vicinity of type A cells defined by the circles having each respective radius (or vice versa) by calculating the average number of observed type B cells in the circle of each respective radius. For example, if n=5 and the four investigated radii are 10 μm, 30 μm, 60 μm, and 100 μm, a first average type B cell count is calculated for the 10 μm radius from the five values ​​obtained from each of the n initial relative distributions, a second average type B cell count is calculated for the 30 μm radius from the five values ​​obtained from each of the n initial relative distributions, and so on until an average type B cell count has been calculated for each investigated radius.

[0052] According to some embodiments, for each radius (or each of a plurality of other predetermined distances defining the spatial neighborhood of type A cells used to calculate the relative distributions), not only is the average reference type B cell count (or reference type A cell count, if the number of type A cells in the spatial neighborhood of reference type B cells is examined) calculated, but also the minimum and maximum reference type B cell counts in any one of the initial reference relative distributions and for any one of the plurality of radii / distances examined. The average reference relative distribution is used to calculate a proximity score, and the minimum / maximum cell counts are used to calculate a confidence band, where the minimum / maximum values ​​represent the boundaries of the confidence band and the width of the confidence band indicates the "significance region" in the plot. If the observed relative distribution is within the confidence band, the observed relative distribution is considered to be identical to the reference relative distribution or at least not significantly different from it.

[0053] For example, n=39 initial reference relative distributions can be calculated based on simulations of Poisson-distributed reference cells of type A and type B, respectively. This ensures that the density of the simulated reference type A cells is identical to the density of "real" type A cells observed in the image region of the tissue sample, and the cell density of the simulated reference type B cells is identical to the density of "real" type B cells observed in the image region. The simulation is performed 39 times for a given density of type A and type B cells to obtain a "p-value" that combines the observed relative distribution with 1 / (39 + 1) = 0.025 = alpha / 2, where alpha is the predetermined significance level. A 2D plot is then created, with one dimension representing cell distance and the other representing cell number. The average reference relative distribution, obtained by averaging the 39 initial reference distributions, is then plotted on the 2D plot. Additionally, "confidence bands" are plotted. Here, the lower boundary of the confidence band is calculated as the minimum cell number observed at a particular distance (e.g., radius) across all 39 initial reference relative distributions, and the upper boundary of the confidence band is calculated as the maximum cell number observed at a particular distance (e.g., radius) across all 39 initial reference relative distributions. Thus, for each distance / radius r around cells of a particular cell type (e.g., type A cells), the minimum, maximum, and average observed cell numbers of each other cell type (e.g., type B cells) from all 39 simulations are plotted. Furthermore, the observed relative distributions are plotted on a 2D plot. This can be useful because it allows the user to intuitively assess whether the observed relative distribution is inside or outside the confidence band surrounding the average reference relative distribution. When the observed reference relative distribution is outside the confidence band, the observed reference relative distribution differs from the average reference relative distribution at a predetermined significance level of alpha, where alpha can be, for example, .05 (0.025 two-sided).

[0054] The 2D plot described above may also be referred to as a "significance plot." An example of a significance plot is shown in FIG. 7. The significance plot may include a first dimension representing the radius of a circle drawn around each tumor cell and its nearest immune cells in the image region, and a second dimension representing the number of immune cells located within the circle. The number of immune cells is normalized by the density of immune cells and tumor cells. The plot may include a confidence band around the reference relative distribution, which indicates a confidence interval for the number of immune cells observed for each radius value according to the plotted reference relative distribution. According to a preferred embodiment, the confidence band represents a 95% confidence band derived from 39 simulations (it can be shown that these 39 simulations correspond to a 95% confidence level for a specific type of biomedical condition determination task). The observed relative distribution is also plotted on the significance plot. The observed relative distribution may be graphically represented as a curve visually indicating the number of immune cells observed and identified to occur in each circle around the observed tumor cell, normalized by the density of immune cells and tumor cells.

[0055] According to other embodiments, the reference relative distribution is not obtained by simulating a particular cell distribution, but is instead obtained experimentally from one or more other tissue samples of known biomedical conditions and respective digital pathology images of said samples.

[0056] According to another embodiment, obtaining the reference relative distribution comprises: - receiving further digital images for each of one or more further tissue samples, each further tissue sample being derived from tissue in a known biomedical state; the further digital images may also be referred to as "reference images" and the further tissue samples may also be referred to as "reference samples"; - analyzing each received further image to identify the number and location of observed reference type A cells and observed reference type B cells in a region of the received further image; - analyzing the locations of observed reference type A and observed reference type B cells in the region of each further image to obtain an observed reference relative distribution, the observed reference relative distribution indicating an observed distance between the reference type A cells and the reference type B cells observed in the region of the further received image; and - Using the observed reference relative distribution as the reference relative distribution Includes.

[0057] Thus, the generation of the reference relative distribution described in the above embodiment is not based on a computer simulation, but on an evaluation of the relative spatial distribution of reference type A and reference type B cells in one or more other tissue samples with a known biomedical condition. This may be advantageous when appropriate simulation software is not available, or when a mathematical function for generating a specific relative distribution representing a known biomedical condition has not yet been defined and implemented in the software. For example, when the biomedical condition, such as the type and stage of cancer or the level of immune cell infiltration of a particular tumor tissue, has been clearly identified by one or more pathologists, it may be more convenient to use the experimental information implicitly contained in such additional tissue samples to provide the reference relative distribution, rather than performing a computer simulation. For example, when the relative spatial distribution of two different cell types in a tissue sample with a known biomedical condition does not follow a Poisson distribution, it may be easier to simply determine the relative distribution of cells of the two cell types to obtain a reference relative distribution representing the "hypothesis" that a tissue sample with a similar relative distribution also has the biomedical condition of one or more other tissue samples from which the experimentally determined reference relative distribution is derived.

[0058] The experimentally determined reference relative distribution can be obtained, for example, from a tissue sample of the same organism "under investigation" as the tissue sample from which the observed relative distribution was obtained using image analysis methods. For example, the "other tissue sample" can be a tissue sample adjacent to the tissue sample shown in the digital image from which the observed relative distribution was obtained. Alternatively, the other tissue sample can be from another organism than the "under investigation," preferably from a species that is the same as or closely related to the "under investigation" organism. It is merely necessary that the biomedical state of the other organism be known, in order to ensure that the reference relative distribution obtained from said other organism represents a specific, known biomedical state.

[0059] The reference relative distribution may also be experimentally determined from each of a plurality of other sample tissues having the same known biomedical condition. An average reference relative distribution is then obtained by calculating the average of each of the experimentally determined initial reference relative distributions. According to some embodiments, obtaining the average reference relative distribution includes performing curve fitting.

[0060] The reference relative distribution or initial reference relative distribution can be obtained from one or more other tissue samples via image analysis in the same manner as the observed relative distribution was obtained.

[0061] According to one embodiment, a pathologist can manually annotate the biomedical condition of a reference tissue sample on an image of said reference tissue sample from a patient. A reference relative distribution is obtained based on image analysis of the image of the reference tissue sample. A plurality of tissue samples adjacent to the reference tissue sample are obtained from the patient. A separate digital image can be obtained from each of the plurality of other adjacent tissue samples, which is automatically analyzed to calculate the observed relative distribution. By comparing the observed relative distribution of each of the adjacent tissue samples to the reference relative distribution of the reference tissue sample, it can be determined whether the patient's tissue from which the plurality of tissue samples, including the reference sample, were derived exhibits a homogeneous or heterogeneous spatial relationship and distribution of type A and type B cells.

[0062] According to an embodiment, calculating the proximity score as the difference between the reference relative distribution and the observed relative distribution comprises: - providing a predetermined number representing a predetermined minimum number of B-type cells (e.g., "1"); - Within the observed relative distribution, the observed radius r o-最小 and identify the observation radius r o-最小 Within the observed relative distribution, the observed radius r is the radius that, when drawn around each observed type A cell, defines a circle that contains, on average, a given number of observed type B cells. o-最小 Identifying; - Within the reference relative distribution, the reference radius r r-最小 and identifying the reference radius r r-最小 is a radius that defines a circle that, when drawn around each of the type A cells observed in the further digital images of the simulated or further tissue sample to provide the reference relative distribution, contains on average a predetermined number of type B cells observed in the further digital images of the simulated or further tissue sample to provide the reference relative distribution, and r-最小 Identifying; - Proximity score is calculated by the observation radius r o-最小 and the reference radius r r-最小 Preferably, the function is the difference between the observation radius and the reference radius. Includes.

[0063] For example, the predetermined number representing the predetermined minimum number of B type cells can be any number greater than 0. Preferably, the predetermined minimum number is 1.

[0064] This feature can be advantageous because the complexity of the information contained in the observed relative distribution and in the reference relative distribution is reduced in some initial steps to two specific distance measurements, i.e., two specific radii, and in further steps to a proximity score value. In the simplest case, the proximity score is calculated as the difference between the two radii. However, the calculation of the proximity score can also be more complex, including, for example, additional normalization steps, steps for performing error correction, steps for multiplying the difference between the two radii by an error correction factor, etc. The proximity score can be a numerical value.

[0065] According to an alternative embodiment, calculating the proximity score as the difference between the reference relative distribution and the observed relative distribution comprises: - defining a first distance threshold (t1); for example, the first distance threshold may have a value between 10 μm and 20 μm, for example 15 μm; - defining a second distance threshold (t2); for example, the second distance threshold may have a value between 30 μm and 40 μm, for example 35 μm; - plotting the reference relative distribution and the observed relative distribution, each in the form of a curve, on a plot, a first dimension of the plot representing a radius r used to determine the observed and reference relative distributions, and a second dimension of the plot representing the number of type B cells that are closer to type A cells than r (or vice versa); - identifying a first region in the plot, the first region defined by first and second thresholds, a baseline corresponding to a second dimension value of zero, and a curve segment of the observed relative distribution between the first and second thresholds; - identifying a second region in the plot, the second region defined by the first and second thresholds, a baseline corresponding to a second dimension value of zero, and a curve segment of the reference relative distribution between the first and second thresholds; - calculating a proximity score as a function of the first region and the second region, the function being in particular the difference between the first region and the second region; Includes.

[0066] In some embodiments, the proximity scores are normalized within a predetermined scale range. This may allow the proximity scores obtained for the tissue sample currently under investigation to be mapped to a predetermined proximity score scale. The scale with the score values ​​may be displayed to the user, for example via a display on an image analysis system, and may allow the user to quickly and intuitively assess how much the relative distribution of type A and type B cells in the tissue sample currently under investigation is derived from a reference relative distribution of type A and type B cells, thereby allowing the user to quickly and intuitively assess whether the reference relative distribution represents a random distribution or other type of expected relative distribution.

[0067] The difference in the radii at which a given minimum number of B cells is observed indicates the deviation of the observed distribution from the expected, eg, random, relative distribution of A and B cells.

[0068] According to some embodiments, the Ripley's K of the observed relative distribution is multiplied by the cell density of type B cells observed in the image region. The Ripley's K of the reference relative distribution is then multiplied by the cell density of type B cells observed in an image region of the same size in an image of the simulated or reference tissue sample to obtain a reference cell count of type B cells in the vicinity of type A cells. The difference between the observed and reference B cell counts is then calculated.

[0069] According to some embodiments, the method graphically represents the observed relative distribution as a curve of the observed distribution in a 2D plot, the first dimension of which represents the radius r(Ro) and the second dimension of which represents the number of observed B cells relative to said radius, and the 2D plot is displayed on a display device of the image analysis system.

[0070] Additionally or alternatively, the method graphically represents the reference relative distribution as a curve of the reference distribution in a 2D plot, the first dimension of which represents the radius r (Rs) and the second dimension of which represents the number of simulated type B cells associated with said radius.

[0071] Preferably, the curves of the observed distribution and the curves of the reference distribution are plotted in the same 2D plot, which makes it easy for the user to compare the two treatments.

[0072] A 2D plot including the curve of the observed distribution and / or the curve of the reference distribution is then displayed on a display device of the image analysis system. Preferably, confidence bands are also calculated from the minimum and maximum values ​​of a plurality of simulated or experimentally obtained initial reference relative distributions, and the confidence bands are also plotted in the 2D plot such that the confidence bands surround the curve of the reference distribution.

[0073] This can be advantageous because the 2D plot allows even users with limited statistical knowledge to intuitively and quickly assess whether the observed relative distribution differs significantly from a given reference relative distribution.

[0074] According to an embodiment, type A cells are tumor cells and / or type B cells are immune cells. Preferably, the relative distribution is obtained by counting the number of observed or simulated immune cells in each spatial vicinity of tumor cells. This approach may be preferable for most tumor types where tumor cells have a greater density than immune cells, as it can ensure that a sufficient number of "count results" are obtained. The number of "count results" corresponds to the number of tumor cells, and is often "zero" if immune cells are very rare in the tissue.

[0075] In other embodiments, the type B cells are tumor cells and / or the type A cells are immune cells. Preferably, the relative distribution is obtained by counting the number of observed or simulated tumor cells in the spatial vicinity of each of the immune cells. This approach may be preferable for some tumor types where immune cells have a greater density than tumor cells or are for other reasons a better starting point for determining the number of cells of other cell types in their respective neighborhoods.

[0076] According to an embodiment, the tissue sample is a tumor tissue sample, and determining the biomedical state of the tissue sample comprises determining the infiltration state of the tumor tissue shown in areas of the received digital image that contain immune cells.

[0077] This can be advantageous because determining the infiltration status is an important prognostic parameter for characterizing tumors and selecting appropriate treatment options. The density of immune cells and tumor cells alone is not sufficient to distinguish, for example, tumor tissues exhibiting an "excluded" infiltration pattern from tumor tissues exhibiting an "infiltrated" infiltration pattern. Similarly, the spatial proximity of tumor cells and immune cells alone is not sufficient to distinguish, for example, tumor tissues exhibiting an "excluded" infiltration pattern from tumor tissues exhibiting an "excluded" infiltration pattern from tumor tissues exhibiting an "exhausted" infiltration pattern. However, a combined score including density and spatial proximity information allows for the distinction between many different infiltration states, such as "infiltration," "excluded," and "exhausted." Therefore, the calculation of the combined score described herein is particularly useful in determining the immune cell infiltration status of tumor tissue. A pre-immunotherapy immune set point value can be very useful for achieving truly personalized therapy. The combined score may enable better characterization of tumor patients to predict sensitivity to drugs with different mechanisms of action and to evaluate response to a given immunotherapy, providing decision support regarding the need for additional drugs for continuation of treatment or shifting to another drug with a different mechanism of action.

[0078] The proximity value can be a numerical value that can be represented by a specific point on a continuous scale, which provides greater granularity and sensitivity than the currently used manually defined classifications. This can be very beneficial, since the majority of cases cannot be clearly assigned to a specific, extreme state like "invasion," "depletion," or "elimination," but rather belong to a moderately inflammatory state, and the difference between a slightly inflamed tumor and a slightly eliminated tumor can be very difficult to identify and characterize in a clear, reproducible, and non-subjective manner.

[0079] For example, tumor cells can be identified in digital images of tissue samples by staining the tissue sample with dyes specific for one or more tumor markers (e.g., dyes that selectively stain cytokeratin, BRAC1, etc.) and applying image analysis methods to identify cells expressing the biomarkers as tumor cells. In some cases, tissue can be stained with dyes specific for immune cells, and image analysis methods are performed such that non-immune cells expressing tumor or proliferation markers are selectively identified as tumor cells. For example, cells that express the biomarker KI67 but do not express the lymphocyte biomarker CD8 (or CD3) are identified as tumor cells.

[0080] In some cases, it has been observed that not all tumor cells express Ki67, however, Ki67+ cells provide a representative sample of tumor cells and provide accurate results.

[0081] According to embodiments, immune cells are identified as cells that express a biomarker that is selectively expressed in immune cells. For example, cytotoxic T cells can be identified as CD8A+ cells.

[0082] According to some embodiments, the region in the received digital image from which cells are identified includes a subregion that has been manually or automatically annotated as a "tumor region." In this case, preferably, only cells expressing a proliferation biomarker, such as Ki67, and / or a tumor marker further within this "tumor region" are identified as "tumor cells." Furthermore, only cells expressing an immune cell-specific biomarker further within this "tumor region" are identified as "tumor cells." This can ensure that biomedical specificity of tissues outside the tumor region does not affect the analysis results, whether the relative spatial distribution of tumor cells and immune cells is the same as the expected relative distribution.

[0083] According to some embodiments, the method further includes graphically representing the combined scores as symbols in a 2D score plot. A first dimension of the 2D score plot represents the proximity scores, and a second dimension of the 2D score plot represents the density of type B cells or the density of type A cells. Preferably, the density of type B cells is plotted, where type B cells are immune cells and type A cells are tumor cells. The 2D score plot is output on a display of an image analysis system to enable a human to determine the biomedical state of the tissue sample. Additionally or alternatively, the determination of the biomedical state of the tissue sample is performed by the image analysis system by automatically identifying the biomedical state of the tissue sample within a limited set of predetermined biomedical states or within a predetermined continuous spectrum of biomedical states. The identified biomedical state is then output, e.g., displayed, as a determined biomedical state.

[0084] Preferably, the density represented by one dimension of the 2D score plot is the observed density of one of the two types of cells having the lower density, e.g., type B cells. Preferably, type B cells are one of the two cell types whose number is counted in the vicinity of each type A cell.

[0085] The determined biomedical state of the tissue can be identified in or mapped to a continuous spectrum of biomedical states. Additionally or alternatively, the biomedical state can be one of a limited set of predetermined discrete biomedical tissue states.

[0086] According to some embodiments, determining the immune cell infiltration status of a tumor sample comprises dividing the tumor into different predefined infiltration statuses, e.g., "depleted," "eliminated," or "inflamed." For example, a threshold combined score value is derived from experimental data of multiple tissue samples, and the resulting threshold combined score can be used to classify a tissue sample represented in an image field into one of the predefined infiltration statuses.

[0087] In another embodiment, determining the infiltration state includes identifying the infiltration state of the tissue sample on a predetermined continuous spectrum representing the grade of infiltration in the tumor. The extremes of the spectrum represent a highly inflamed tumor, whereas the extremes represent a complete lack of immune cells. In this case, one of a predetermined limited set of infiltration states is not identified, providing a continuous score that allows mapping the tumor in the image area depicted on the spectrum between the most extreme infiltration states.

[0088] Tumor infiltration by immune cells is driven by the presentation of mutated tumor antigens in the context of MHC on the tumor cell surface. Tumor antigens are recognized as non-self and are eliminated primarily by natural killer cells and cytotoxic T cells. Elimination relies on complex signaling pathways that may be altered or disrupted by the tumor. This balance between tumor growth and the immune system's attempts to kill tumor cells has been described as the elimination-balance-escape mechanism. Current developments in immunotherapy hold great potential for altering the dismal prognosis of many cancer patients by shifting the tumor immune microenvironment in favor of tumor cell elimination.

[0089] While there have been many proposals in the field to measure tumor immune phenotypes and describe the state of the elimination-equilibrium-escape balance, no consensus has been reached. A major hurdle to overcome is characterized by a lack of a deeper understanding of how the theory of the continuum between elimination and escape can be translated into biologically meaningful information presented in histological tumor samples. Manual scoring systems suggest the number of classes into which the spectrum should be divided, and there is generally no agreement on how to find thresholds or objective definitions for these categories. Embodiments of the present invention make it possible to overcome these obstacles and provide an automated, calibrated, and personalized measurement of tumor infiltration status based on both the presence and distribution of tumor-infiltrating immune cells. Thus, embodiments of the present invention make it possible to obtain information about biological trends, such as when the balance of the immune system is focused on eliminating tumor escape and when the immune system is generally activated or silenced.

[0090] According to an embodiment, the image analysis method includes receiving digital images of tissue samples from each of a plurality of different patients; calculating a combined score for each of the patients using each received image according to any one of the preceding claims; and graphically representing the biomedical condition of each patient on a 2D score plot by a symbol specific to the individual biomedical condition, wherein the position of each patient's symbol in the plot depends on the density of B cells and the proximity score calculated for the patient.

[0091] This can be advantageous because it provides a plot that allows the user to quickly and intuitively determine the biomedical status, e.g., immune cell infiltration status, of a tissue sample based on the x and y coordinates of the points in the plot, where the points represent tissue samples and the x and y coordinates of the plot represent the proximity score and the observed density of immune B cells in the tissue sample.

[0092] Immune infiltration in tumors is the host's response to foreign substances in the body. It is caused by the presence of tumor antigens (mutated proteins) presented by tumor cells. If the immune system is suppressed or if the tumor downregulates tumor antigen expression, the tumor may lack immune cells. In practice, the amount of immune infiltration is seen as a spectrum ranging from no infiltration to severe infiltration.

[0093] According to an embodiment, the identified biomedical condition is: - "Inflammation", where "inflammation" refers to an immunological tissue condition in which immune cells have a significantly increased cell density in all compartments of the tumor, indicating massive infiltration of immune cells into the tumor tissue, where the higher the observed immune cell density in an image area and the higher the proximity score, the more likely the tissue sample is classified as "inflamed". - "Exclusion", where "exclusion" refers to an immunological tissue condition in which immune cells are present in a tissue sample but are prevented from closely juxtaposing tumor cells, such that the immune cells are concentrated at the invasive margin and / or intratumoral stroma but separated from the tumor cells, where the higher the observed immune cell density in an image area and the lower the proximity score, the more likely the tissue sample is classified as "exclusion". - "Depleted", where "depleted" refers to an immunological tissue condition in which immune cells have zero or near-zero cell density in tumorous tissue regions, where the lower the observed immune cell density in an image region, the more likely the tissue sample is classified as "depleted" regardless of the high proximity score. is selected from the group comprising:

[0094] The applicant has observed that many tumors display images that do not completely fit into either of the above extreme categories. Therefore, calculating a combined score value is advantageous because it is a combination of two continuous numerical scores (proximity score, cell density), thereby providing an accurate prediction / determination of the biomedical status of tissue based on two important features. Density indicates whether the organism can induce an immune response. Proximity score indicates whether the tumor is immunogenic and adapted to attract immune cells. If not, the tumor may have developed a means to evade immune cells (e.g., HLA-upregulation or cytokine production, which either renders immune cells dormant or stimulates regulatory immune subsets).

[0095] According to an embodiment, if the biomedical condition of the tissue sample is determined to be in an infiltrative "inflammation" state, the image analysis method includes outputting, via a user interface, a treatment recommendation to prescribe a drug that acts as a checkpoint inhibitor, e.g., anti-PD-L1 / PD-1, CTL4.

[0096] Additionally or alternatively, if the biomedical status of the tissue sample is determined to be "clear" of the invasive status, the image analysis method includes outputting, via a user interface, a treatment recommendation for prescribing a drug adapted to attract immune cells proximal to the tumor cells. For example, the drug may be a bispecific antibody directed against tumor cells. An example of such a bispecific antibody is CEA-TCB, which simultaneously binds one arm to CD3 on T cells and two arms to CEA on tumor cells, thereby bringing the T cells into close proximity with the cancer cells. This results in T cell activation and subsequent tumor cell death. Another example of an appropriate drug recommended in the case of an "cleared" invasive status may be a drug that disrupts tumor escape mechanisms, such as a drug that blocks HLA-G or upregulates MHC I.

[0097] Additionally or alternatively, if the biomedical state of the tissue sample is determined to be "depleted" of infiltration, the image analysis method includes outputting, via the user interface, a treatment recommendation to prescribe IL-2, a drug adapted to generally boost the immune system.

[0098] According to an embodiment, the shapes of tumor cells and immune cells in an image region are stored in the form of points in a spatial database. The simulated and measured distances are determined using spatial database operations provided by the spatial DBMS, in particular the WITHIN_DISTANCE() and DISTANCE() operations. This can be advantageous because the relative distributions can be significantly improved. This can be particularly relevant when many identical relative distributions need to be simulated to provide a confidence band of a desired width. For example, the SDO_GEOM functionality of the Oracle Spatial DBMS can be used.

[0099] According to another embodiment, the scores are calculated outside the DB using R.

[0100] According to an embodiment, the expected distribution indicates the distance between the reference type A cells and the reference type B cells that is expected based on the assumption of a Poisson distribution of type A cells and type B cells in the region. Preferably, this assumption is further based on the same densities of the observed type A cells and the reference type A cells and the same densities of the observed type B cells and the reference type B cells.

[0101] In a further aspect, the present invention relates to a method for determining the effectiveness of a drug for treating a particular type of cancer, the method comprising: - receiving a plurality of first digital images; ,each The first image shows the number of individual organisms with a particular type of cancer. before the organism is treated with the drug. receiving a plurality of first digital images showing the tissue sample; - calculating a first combination score from each image region of the first digital image according to any one of the preceding claims, wherein the biomedical condition is an immune cell infiltration status of a tumor, type A cells are tumor cells, and type B cells are cancer cells; - receiving a plurality of second digital images; ,each The second image shows the individual organisms with a particular type of cancer. after the organism has been treated with the drug. receiving a plurality of second digital images showing the tissue sample; - calculating a second combination score of any one of the preceding claims from each image region of the second digital image, wherein the biomedical condition is an invasive condition, the type A cells are tumor cells, and the type B cells are cancer cells; - displaying a 2D score plot on a display device, wherein each first combination score is represented by a first symbol in the 2D score plot and each second combination score is represented by a second symbol in the 2D score plot, the positions of the first and second symbols in the plot depend on the immune cell densities observed in the image region of each image and the proximity scores calculated for the image region of each image, and wherein first and second symbols representing the same organism are visually linked in the 2D plot score from any one of the first symbols to its individually linked second symbol to visualize any shift in x and / or y coordinates. Includes.

[0102] This can be advantageous because the above approach allows for visual representation of the effectiveness of a particular drug tested on multiple different patients, thereby visualizing the drug's effect on the spatial distribution of immune cells and tumor cells. It is already difficult to visually represent the modification of the relative spatial distribution of cells in a single patient. Here, the response of multiple patients to a particular drug is visualized in terms of the modification of the relative distribution of tumor and immune cells, allowing medical professionals to easily assess whether the drug had an effect and what type of biomedical condition (e.g., tumor infiltration condition) was promoted or suppressed by the drug. Visual association can be performed, for example, by connecting symbols (e.g., circles, triangles, etc.) from the same patient with lines.

[0103] In a further aspect, the present invention relates to a storage medium comprising computer-interpretable instructions that, when executed by a processor, cause the processor to perform an image analysis method according to any one of the embodiments and examples described herein.

[0104] In a further aspect, the present invention relates to an image analysis system for determining a biomedical condition of a tissue sample, the system comprising: - receiving a digital image of the tissue sample; - analyzing the received image to identify the number and location of type A cells and type B cells observed in an area of ​​the received image, where type A and type B are different cell types; - analyzing the locations of type A and type B cells in the region to obtain an observed relative distribution, the observed relative distribution indicating an observed distance between type A cells and type B cells in the region; - obtaining a reference relative distribution, the reference relative distribution indicating an expected distance between a reference type A cell and a reference type B cell; - Calculating the proximity score as the difference between the reference relative distribution and the observed relative distribution; - calculating a combination score, the combination score including a proximity score and including a density of type A cells and / or a density of type B cells; - using the combined score to determine the biomedical state of the tissue sample and / or outputting the combined score to a user to enable the user to determine the biomedical state of the tissue sample. The present invention includes a processor and computer-interpretable instructions configured to cause the processor to execute instructions for performing a method including:

[0105] As used herein, a "relative distribution" is a function that describes the spatial dispersion between two types of objects. According to a preferred embodiment, the "relative distribution" is a function that captures the distance of a second type of object from a first type of object. According to a preferred embodiment, the "relative distribution" quantifies the spatial dispersion in a manner that is invariant to rotation and reflection. Several concise measures of spatial dispersion can be defined, for example, using the covariance matrix of the object's coordinates. The trace, determinant, and maximum eigenvalue of the covariance matrix can be used as measures of spatial dispersion. An example of a measure of spatial dispersion that is not based on the covariance matrix is ​​the average distance between nearest neighbors. A further preferred measure of spatial dispersion is Ripley's K function.

[0106] As used herein, a "reference relative distribution" is a relative distribution that captures the degree of spatial dispersion between two types of objects (e.g., reference type A cells and reference type B cells) and is used as a reference. A "reference" is a kind of "standard" or "baseline" against which other information, such as another relative distribution of two object types, can be compared. A reference relative distribution can be obtained by computational simulation based on the expected distribution of objects, e.g., based on a Poisson distribution, or can be obtained experimentally from the relative distribution of objects observed in one or more reference samples. A reference relative distribution can represent a "baseline." This allows the relative distribution of different cell types in a tissue sample under investigation to be compared with the relative distribution of the same cell types in one or more reference tissue samples with known biomedical conditions. Comparing the observed relative distribution with the reference relative distribution can make it possible to determine whether the relative distribution of cells in a tissue sample deviates significantly from the relative distribution of the cell types in the simulated or experimentally determined reference relative distribution.

[0107] As used herein, an "observed relative distribution" is a relative distribution that captures the degree of spatial dispersion between two types of objects (e.g., two cell types) within the tissue sample represented in the digital image currently being analyzed.

[0108] As used herein, a "proximity score" is a score value calculated as a function of the observed relative distribution and the reference relative distribution. Preferably, the proximity score is calculated by calculating the difference between one or more distances or distance measurements coded in the observed relative distribution and the reference relative distribution. According to an embodiment of the present invention, the score value is a numerical data value. According to an embodiment of the present invention, the score value is calculated as the difference (delta) between the reference distribution and the observed distribution. For example, the score value can be calculated as the difference between two distances r1 and r2, where r1 can be the distance from type A cells at which a certain number of type B cells are observed on average according to the observed relative distribution, and r2 can be the distance from type A cells (simulated or experimentally determined) at which a certain number of type B cells are observed on average (simulated or experimentally determined) according to the reference relative distribution. Alternatively, the difference can be the difference between two regions. One region is defined by two predetermined distance thresholds, a baseline of "0 observed occurrences," and the interval of the reference relative distribution curve between the two distance thresholds, and the second region is defined by two predetermined distance thresholds, the baseline, and the interval of the observed relative distribution curve between the two distance thresholds.

[0109] As used herein, "tumor-immunophenotyping" or "spatial immune infiltration phenotyping" is a method for dividing tumors into different groups or mapping them onto a continuous or discrete spectrum based on the grade of tumor inflammation. The two extremes of the spectrum represent highly inflamed tumors versus tumor tissue that is completely devoid of immune cells.

[0110] As used herein, "Ripley's K" is a spatial descriptive statistic that can be used to generate reference relative distributions and / or observed relative distributions. More specifically, it is a measure of spatial dispersion between two types of objects. Ripley's K function was described by Philip M. Dixon in Volume 3, pp 1796-1803 of Encyclopedia of Environmetrics (ISBN 0471 899976) Edited by Abdel H. El-Shaarawi and Walter W. Piegorsch, John Wiley & Sons, Ltd, Chichester, 2002. Ripley's K(t) function (also referred to as "K function" or "Ripley's K") is a mathematical expression of the expected value of the distance between type A cells and type B cells.

[0111] As used herein, a "biomedical state" is any disease, metabolic, genomic, proteomic, related, or morphological state of a tissue that is known in the context of biology and / or medicine, or that is of predictive or therapeutic use. For example, the "biomedical state" of a tissue may relate to the degree of infiltration by immune cells of tumor tissue, or may relate to the degree of differentiation of tumor tissue or tissue of a young, developing organism.

[0112] Embodiments of the invention will now be described in more detail, by way of example only, with reference to the drawings in which: [Brief explanation of the drawings]

[0113] [Figure 1] FIG. 1 is a block diagram of an image analysis system. [Figure 2] FIG. 1 is a flow diagram of an image analysis method for determining the biomedical condition of a tissue sample. [Figure 3] Images of three tumor tissue samples with different degrees of immune cell infiltration and a schematic diagram of individual cell distribution are shown. [Figure 4] Figure 3 shows the immune cell and tumor cell densities of the three tissue samples, respectively. [Figure 5]1 illustrates the use of radius to define spatial cell neighborhoods and to calculate the relative distribution of cells of different cell types. [Figure 6] Shown are the observed relative distributions obtained from images showing "invaded" and "excluded" infiltration states. [Figure 7] Two 2D plots are shown, each containing the observed relative distribution, the reference relative distribution, the confidence bands, and the "delta" used as the proximity score. [Figure 8] 10 shows an alternative, area-based analysis of 2D plots to determine the "delta" used as a proximity score. [Figure 9] Fluorescence images of two tumor tissue samples with "excluded" or "infiltrated" immune cell infiltration status are shown. [Figure 10] A 2D score plot containing clusters of combined scores is shown, where each combined score represents one representative from multiple different tissue samples. [Figure 11] 1 shows a logarithmic 2D score plot with points representing combined score values ​​obtained from multiple patients. [Figure 12] 1 shows a logarithmic 2D score plot with pairwise connected points representing combined score values ​​obtained from multiple patients before and after treatment. [Figure 13] A logarithmic 2D score plot containing three clusters of combined scores obtained from multiple patients is shown. [Figure 14] A logarithmic 2D score plot containing five clusters of combined scores obtained from multiple patients is shown. DETAILED DESCRIPTION OF THE INVENTION

[0114] FIG. 1 is a block diagram of an image analysis system 100 according to an embodiment of the present invention. The system includes one or more processors 104, a main memory 106, and a non-volatile storage medium 108. The storage medium includes one or more application programs or modules 110, 114, 112, 116, 120, and 122 configured to perform one or more data processing tasks, such as automatically detecting cells of different cell types (module 110), measuring cell-cell distances (module 112), simulating the distribution of cells at a given density (module 116), plotting various scores and curves (module 122), calculating proximity scores and combination scores (module 120), and automatically detecting or predicting a biomedical condition of a tissue sample based on the combination scores (module 114). The various modules can be combined in any manner in one or more application programs. For example, the functionality provided by the modules can be combined into a single application program. Alternatively, some functions can be implemented by separate application programs. For example, simulation of a given distribution can be performed by a mathematical program such as R, distance calculations can be performed by interacting with or within a spatial DBMS, and identification of cells of a particular cell type can be implemented by digital pathology image analysis software.

[0115] The storage medium 108, e.g., an electromagnetic hard disk drive, may contain one or more digital pathology images 118, each representing a tissue sample. For example, the digital images 118 may be monochrome images or multi-channel images. For example, the images 118 may be RGB images. The images 118 may be bright-field microscopy images or fluorescent images. Typically, the images are multi-channel images obtained by fluorescent microscopy from a tissue sample stained with dyes specific for one or more biomarkers having different colors.

[0116] The cell type detection module can be configured to automatically detect tumor cells and immune cells and store the location and type of the automatically detected cells in a spatial database. Identification of cells and their respective cell types can involve connected component analysis and edge detection routines to identify pixel blobs representing cells. Information coded in color or other image features can be used to identify cell types. Color information coded in the digital image 118 typically indicates a specific biomarker or set of specific biomarkers. For example, a specific set of cytokeratins can be stained with an appropriate antibody that binds to a fluorescent dye or a colored dye. Multiple monochrome images 118 can be obtained from a multispectral fluorescence image of a specific tissue sample by applying a color deconvolution algorithm.

[0117] For example, cells that express specific tumor or proliferation markers but do not express immune cell-specific markers can be considered tumor cells. Cells that express immune cell-specific markers, such as CD8A, can be identified as immune cells.

[0118] As another example, Ventana Medical Systems' primary antibody Anti-Pan Keratin "AE1 / AE3 / PCK26" can be used to stain poorly differentiated malignant tumors. The anti-Pan keratin antibody set "AE1 / AE3 / PCK26" specifically binds to an antigen located in the cytoplasm of simple and complex epithelial cells. This is a mouse monoclonal antibody cocktail raised against epitopes found in human epidermal keratins reported by Woodcock-Mitchell et al. This antibody cocktail reacts with the 56.5 kD, 50 kD, 50 kD, 48 kD, and 40 kD cytokeratins of the acidic subfamily and the 65-67 kD, 64 kD, 59 kD, 58 kD, 56 kD, and 52 kD cytokeratins of the basic subfamily.

[0119] According to an embodiment, one or more digital images 118 are generated by a color deconvolution algorithm or are directly stored on a storage medium after capturing them with a camera or other image capture device. The image analysis system may optionally include an image capture device, such as a camera (not shown).

[0120] Additionally, system 100 is coupled to or includes a display 102, such as an LCD display, which is used to display digital images 118 of various patient tissue samples, to display various plots including density, relative distribution, and combined score values, and / or to display the results of cluster analyses of biomedical tissue conditions automatically identified by the combined scores obtained from particular tissue samples.

[0121] Figure 2 is a flow diagram of an image analysis method for determining the biomedical status of a tissue sample. This method is described below using tumor cells and immune cells as examples of different cell types, and the immune cell infiltration status of a tumor tissue sample as an automatically determined biomedical tissue status. The method may enable the automatic stratification of patients with respect to their potential response to immunotherapy using spatially-informed image analysis. However, this is only one example of how the calculation of the combination score described herein from the relative distributions of two different cell types can be used in the context of biology and medicine to rapidly, automatically, and accurately determine the biomedical status of a particular tissue. The difference in the distributions of two cell types from an expected reference distribution can be used to determine and subclassify many other diseases or physiological conditions and / or to identify appropriate treatment modalities or drugs.

[0122] The method may be implemented in, and for example, performed by, the image analysis system 100 shown in Figure 1. In a first step 202, the image analysis system 100 receives a digital image of a tissue sample. For example, the digital image may be received directly from an image acquisition device, such as a microscope, or may be received over a network, such as the Internet, or may be read from a non-volatile storage medium 108.

[0123] Next, in step 204, the image analysis system analyzes the received image to identify the number and location of type A cells (e.g., tumor cells) and type B cells (e.g., immune cells) observed in the region of the received image. Image analysis may include application of various image processing and analysis techniques such as color deconvolution, image segmentation, blob detection, connected component analysis, feature extraction, future clustering, etc.

[0124] Next, in step 206, the image analysis system analyzes the location of type A and type B cells in the region to obtain an observed relative distribution, which indicates the observed distance between type A and type B cells in the region. Additionally, the image analysis system determines the densities of observed tumor cells and immune cells, respectively, in the image region.

[0125] Next, in step 208, the image analysis system obtains a reference relative distribution. The reference relative distribution can be obtained, for example, by performing multiple simulations to generate expected distributions of tumor cells and immune cells, respectively, and determining the distance between the different types of simulated cells. The expected distribution can be, in particular, a random distribution, for example, a Poisson distribution, which assumes the same density of tumor cells and immune cells as observed in the image region.

[0126] Next, in step 210, a proximity score is calculated as the difference between the reference relative distribution and the observed relative distribution. As used herein, the phrase "calculating the difference between the reference relative distribution and the observed relative distribution" means that the proximity score is calculated as a function of the observed and relative distributions, where the function includes at least one operation of calculating the difference between one or more "observed" distances coded in the observed relative distribution and one or more "reference" distances coded in the reference relative distribution.

[0127] Next, in step 212, the image analysis system combines the proximity score with the density of immune cells measured in the received digital image 118 to obtain a "combined score."

[0128] Next, in step 214, the image analysis system uses the combined score to automatically determine the biomedical condition of the tissue sample, such as a particular invasive state. Additionally or alternatively, in step 216, the image analysis system displays the resulting combined score for the tissue sample to the user on the image analysis system's display 102, thereby allowing the user to visually ascertain the current biomedical condition of the tissue and draw corresponding conclusions regarding disease progression and appropriate treatment options. For example, the combined score for the tissue sample can be represented as a data point in a 2-D score plot, as shown in Figures 9, 10-13.

[0129] According to a preferred embodiment, the observed relative distribution and the reference relative distribution are calculated using Ripley's K function.

[0130] According to alternative embodiments, a simpler approach can also be used. For example, the image analysis system 100 can receive 202 a digital image 118 of a tissue sample. The image analysis system analyzes 204 the received image to identify CD8+ immune cells and Ki-67+ tumor cells in the image or in specific subregions of the image. For example, the subregions can be image tiles of a predetermined size or automatically detected tumor tissue regions. For each identified tumor cell, the image analysis system determines 206 the distance of its most recent immune cells from that tumor cell. After the distances of all tumor cells to their individual most recent immune cells have been determined, the total distances can correspond to and represent the "observed relative distribution" of tumor cells and immune cells in the tissue. In a next step, the identified densities of tumor cells and immune cells in the image or tumor tissue region are determined and used to simulate Poisson distributions of simulated tumor cells and simulated immune cells, respectively. The densities of the simulated tumor cells and simulated immune cells are thereby identical to the observed densities of tumor cells and immune cells in the tissue. The distance of each of the simulated immune cells from all simulated tumor cells is then determined 208 and represented as the "reference relative distribution." A proximity score is then calculated 210 as a function of the determined distances. For example, the distance at which, on average, every tumor cell contains one immune cell in its neighborhood can be determined for both the observed relative distribution and the reference relative distribution, and the difference between these two can be used as the proximity score. The image analysis system then provides a combined score 212 that includes the proximity score and the immune cell density observed in the image 118, and uses the combined score 214 to automatically determine the biomedical state of the tissue (e.g., infiltration state) and / or display the combined score on the image analysis system's display 102, for example, on a 2-D score plot shown in Figures 9, 10-13, to allow a user to visually assess the current biomedical state of the tissue sample.

[0131] Figure 3 shows images of three tumor tissue samples with different degrees of immune cell infiltration and a schematic diagram of individual cell distribution.

[0132] Image 302 shows a digital pathology image of tumor tissue with an "infiltrated" state, i.e., the tumor tissue is densely infiltrated with immune cells. The relative spatial distribution of immune cells (triangles) and tumor cells (dots) in the "infiltrated" tumor tissue is shown in the schematic diagram 308 below.

[0133] Image 304 shows a digital pathology image of tumor tissue that has been "excluded," meaning that the tumor has created a kind of "border" that prevents immune cells from migrating into the tumor cells. The relative spatial distribution of immune cells and tumor cells in the "excluded" tumor tissue is shown in the schematic diagram 310 below.

[0134] Image 306 shows a digital pathology image of tumor tissue with a "depleted" / "desired" state, i.e., the tumor tissue is essentially free of immune cells. The relative spatial distribution of immune cells and the relative spatial distribution of tumor cells in the "depleted" tumor tissue is shown in the schematic diagram 312 below.

[0135] The three images and distributions shown are clearly representative of the three different biomedical states described above. However, in most cases, tissue samples fall somewhere between the two different states. As a result, currently available methods are unable to provide a reliable, objective, and reproducible way to determine the biomedical state of tissue samples that do not clearly fall into one of the three categories.

[0136] Figure 4 shows the immune cell and tumor cell densities of the three tissue samples from Figure 3. Additionally, Figure 4 includes plots 402, 404, and 406 showing the immune cell density (CD8A+) and tumor cell density (Ki-67+) for each condition. This figure shows that immune cell density is an important prognostic parameter for distinguishing "depleted" tissue from the other two tissue types, but is insufficient to distinguish between "excluded" and "infiltrated" tissue samples.

[0137] Figure 5 illustrates the use of radius to define spatial cell neighborhoods and calculate the relative distribution of cells of different cell types. The image shows six tumor cells 504, represented by small circles, and multiple immune cells, represented by small triangles. To obtain the observed relative distribution between immune cells and tumor cells in the image, each identified tumor cell is used as the center of a circle with radius r. Radius r defines the circular neighborhood. By gradually increasing the radius of the circle around the tumor cell and counting the number of immune cells contained in each circle, a cumulative measurement of the number of immune cells surrounding the tumor cell can be obtained. The number of steps and the respective radii can be freely selected as a compromise between computational resource consumption and accuracy. For example, each step may increase the radius by 1 μm, 5 μm, or 10 μm, etc. The relationship between each radius and the number of immune cells counted in the circular neighborhood defined by the radius can be expressed by a variety of different mathematical formulas and distributions. Preferably, Ripley's K function is used to describe the relative spatial distribution of immune cells and tumor cells obtained by each stepwise increase in radius.

[0138] Figure 6 shows the observed relative distributions obtained from images showing "invaded" and "excluded" infiltration states. The upper part of Figure 6 shows a scheme of the spatial distribution of immune cells and tumor cells, as already shown and described in Figure 3. The lower part of Figure 6 shows plots 602, 604, and 606, each with a y-axis representing the result of the Ripley's K function, K(r), and an x-axis representing the radius (increased stepwise). In the example shown, the radius increases by 1 μm. Thus, one data point is obtained for each micrometer on the x-axis. The curves in the plots are obtained by curve fitting the obtained data points in the plots. As can be seen in the invaded case, the results obtained by the Ripley's K function are positively correlated with increasing radius r. However, in the "excluded" case, there is almost no increase in K(r) with increasing r. In the "excluded" case, there is a slight increase in K(r) with increasing r. Therefore, although the relative spatial distribution may not always be sufficient to distinguish between excluded and depleted states, it can clearly distinguish between excluded or depleted and invaded states. Thus, the combination of proximity scores encoding differences in relative spatial distribution with density information may allow for unambiguous identification of the current biomedical state of tissue across a complex continuum of conditions.

[0139] FIG. 7A shows a 2D plot 702 in which the y-axis represents the number of immune cells observed in a circular vicinity around a tumor cell defined by the circle radius r, and the x-axis represents the radius r.

[0140] The first plot 702 shows the observed mean number of immune cells within a circle of radius r around tumor cells of different radii in tissue samples with an "invasion" state. Curve 706 represents the observed relative distribution and was obtained by curve fitting each data point in plot 702.

[0141] A second plot 704 shows the observed mean number of immune cells within a circle of radius r around tumor cells of different radii in tissue samples with a "removed" state. Curve 708 represents the observed relative distribution and was obtained by curve fitting each data point in plot 704.

[0142] FIG. 7B shows 2D plots 702, 704 supplemented with additional information.

[0143] Plot 702 is supplemented with an average reference relative distribution 714. The average reference relative distribution was obtained by generating multiple (e.g., 40) initial reference relative distributions. Each initial relative distribution is calculated by simulating a Poisson distribution of simulated tumor cells with the same cell density as the observed tumor cells, simulating a Poisson distribution of simulated immune cells with the same cell density as the observed immune cells, and determining a distance measurement (e.g., immune cell count obtained in a circular vicinity of the tumor cells using radius r) that provides information about the relative spatial distribution of immune cells and tumor cells. For each of the investigated radii (r = 1 μm, 2 μm, ..., 98 μm, 99 μm, 100 μm) and for each of the 40 simulations, the average number of immune cells in the circle of radius r around the tumor cells is determined. Then, for each of the investigated radii, the minimum, average, and maximum number of immune cells in the circle of radius r around the tumor cells are determined. The average numbers are plotted as data points to generate a curve 714 based on a curve fit. Curve 714 represents the reference relative distribution based on the simulation. The minimum number is plotted for each r data point to generate a lower edge of the confidence band 716 by curve fitting the data points. The maximum number is plotted for each r data point to generate an upper edge of the confidence band 716 by curve fitting the data points. The confidence band 716 represents the region within which any observed relative distribution is assumed not to differ significantly from the reference relative distribution 714.

[0144] "Delta=19" is the proximity score obtained by identifying a first point RP in the reference relative distribution 714 where the average number of immune cells in the tumor cell environment is "1," by identifying a second point OP in the observed relative distribution 706 where the average number of immune cells in the circular tumor cell environment is "1," and by subtracting the x-values ​​(radii) of the first and second points. The resulting difference ("radius delta" or "distance delta") is used as the proximity score for the tissue sample. A user can easily infer that the observed relative distribution 706 is significantly different from the predicted / reference relative distribution 714 from the fact that the second point OP is outside the confidence band 716.

[0145] Plot 704 was supplemented with an average reference relative distribution 710. The average reference relative distribution was obtained by generating multiple (e.g., 40) initial reference relative distributions. Each initial relative distribution was calculated by simulating a Poisson distribution of simulated tumor cells having the same cell density as the observed tumor cells in the tissue represented in plot 704, as described above for plot 702. Curve 710 represents the reference relative distribution based on the simulation. A minimum number is plotted for each r data point to generate the lower edge of a confidence band 712 by curve fitting the data points. A maximum number is plotted for each r data point to generate the upper edge of a confidence band 712 by curve fitting the data points. The confidence band 712 represents the region within which any observed relative distribution is expected not to differ significantly from the reference relative distribution 710.

[0146] "Delta = 32" is a proximity score obtained by identifying a first point RP in a reference relative distribution 710 where the average number of immune cells in the tumor cell environment is "1", identifying a second point OP in an observed relative distribution 708 where the average number of immune cells in a circular tumor cell environment is "1", and subtracting the x-values (radii) of the first and second points. The resulting difference ("delta of the radius") is used as the proximity score for the tissue sample. From the fact that the second point OP is outside the confidence band 714, the user can easily infer that the observed relative distribution 708 is significantly different from the predicted / reference relative distribution 710.

[0147] Figure 8 shows a plot 750 that is a modified version of the plot 702 shown in Figure 7B. The plot 750 can be used to automatically or manually determine an alternative, region-based "delta" that is used as a proximity score.

[0148] The plot 750 shows an average reference relative distribution 714 and an observed relative distribution 706. Further, the plot​​​​​​​​​​​The overlapping area between the first and second regions is indicated by a shaded "XXX".

[0152] Here, a "delta value" can be calculated as the difference or ratio of the size of the first region and the second region. This "delta value" (also called "region size delta") is used as a proximity score for the tissue sample. A user can easily infer that the observed relative distribution 706 is significantly different from the reference relative distribution 714 from the location of the curve section of curve 706 that is far outside the gray confidence band around curve 714.

[0153] Figure 9 shows fluorescent images 804 and 806 of tumor tissue samples with an "excluded" invasion status. Tumor samples from clinical trials were formalin-fixed, cut into 2.5-μm-thick sections, and stained with a dual-chromogenic assay for CD8 and Ki-67. The stained slides were scanned and imported into a digital pathology form, where a pathologist manually annotated the tumor, normal tissue, and necrotic regions. Areas with artifacts were excluded. The images were subjected to automated whole-slide image analysis using software logic adapted to automatically detect CD8-Ki-67 double-positive cells, CD8-negative Ki-67-negative T cells, and Ki-67-positive and CD8-negative tumor cells. Detection accuracy was checked by a pathologist before the results were entered into a database. Only objects detected in the annotated tumor regions were used for further analysis. The densities of CD8+ T cells, CD8+ / Ki-67+ cells, CD8+ / Ki-67- cells, and CD8- / Ki-67+ tumor cells were calculated. This type of fluorescence image can be used to stratify patients, as shown in the following figure.

[0154] Figure 10 shows a 2D score plot containing 80 combined scores divided into three groups, each representing one of the 80 different patients from whom tissue samples were taken.

[0155] The x-axis of the 2-D score plot 902 represents the logarithm of the density of immune cells (CD8A+ cells). The y-axis represents the proximity score. Cluster 906 represents tumor tissues that are "infiltrated" with immune cells. Cluster 908 represents tumor tissues determined to have a "depleted" infiltration state. Cluster 910 represents tumor tissues determined to have a "excluded" infiltration state. As can be inferred from the 2-D score plot, immune cell density is an important parameter for distinguishing between infiltrated and depleted tissue samples. However, density alone may not be sufficient to distinguish between "infiltrated" and "excluded" tissue samples. However, the additional dimension (y-axis), representing the proximity score, distinguishes between "excluded" and "infiltrated" tumor samples.

[0156] The thick line 904 indicates that the combination of immune cell density and proximity scores may allow for improved differentiation of biological tissue states over density information alone.

[0157] Figure 11 shows a logarithmic 2D score plot with points representing combined score values ​​obtained from multiple patients. Generally, it can be inferred that increased immune cell density correlates with increased immune cell and tumor cell engagement. However, there are significant differences between different patients and even between different tissue samples from the same patient. These differences may be used to determine the current biomedical state of the tissue and identify appropriate treatment options.

[0158] 12 shows a logarithmic 2D score plot with pairwise connected points representing combined score values ​​obtained from multiple patients before and after treatment. Each arrow connects two data points representing the combined score of a tissue sample obtained from a patient before the patient was treated with a drug and the combined score of another sample obtained from the same patient days or weeks after the patient was treated with the drug. For example, the drug may be a drug that enhances the immune system. The drug may be a drug administered during immunotherapy of a patient.

[0159] From the plot, it can be inferred that in response to treatment of patients with a particular drug, immune cell and tumor cell engagement significantly increased in almost all patients (most arrows point upward in the plot). Furthermore, immune cell density increased in almost all patients in response to drug application (most arrows point from left to right). However, several patients showed no response or even a decrease in immune cell number and / or immune cell-tumor engagement.

[0160] Thus, by obtaining repeated combined scores over time from one or many patients, it becomes possible to detect trends in disease progression or other physiological conditions. Often, biologically driven mechanisms are reflected in tissues as relative changes in the distribution of different cell types. Applicant has observed that, in addition to immune cell density, relative changes in the distribution between tumor cells and immune cells provide meaningful information about where a patient is in the tumor's elimination-equilibrium-escape balance, as well as an indication of the direction of the process over time toward more elimination or more escape.

[0161] Figure 13 shows a logarithmic 2D score plot containing the combined score values ​​obtained from multiple patients. The patients are grouped into three clusters based on the location of their respective combined score values ​​on the plot. The combined score therefore allows the invasive status of a particular patient's tumor to be mapped onto a continuous scale representing different invasive states of the tumor. Samples and patients can be grouped into three different clusters, although more detailed classifications are also possible, as shown in Figure 14. Cluster 972 represents the "invasive" status, cluster 974 represents the "eliminating" status, and cluster 976 represents the "exhausting" status.

[0162] Figure 14 shows a logarithmic 2D score plot containing five clusters of combined scores obtained from multiple patients. In addition to the three different biomedical conditions represented by each cluster in Figure 13, Figure 14 also includes combined score clusters for "marginally inflamed" and "moderately inflamed" conditions.

Claims

1. 1. An image analysis method for determining a biomedical condition of a tissue sample, implemented by an image analysis system (100), comprising: - receiving (202) a digital image (118) of a tissue sample; - analyzing (204) the received image to identify the number and location of type A cells (504) and type B cells (506) observed in the area of ​​the received image, where type A and type B are different cell types; - analyzing (206) the locations of type A and type B cells in the region to obtain an observed relative distribution (708, 706), the observed relative distribution indicating the observed distance between type A cells and type B cells in the region; - obtaining (208) a reference relative distribution (710, 714), which indicates the expected distance between reference type A cells and reference type B cells, the expected distance being the distance that would be expected assuming that type A cells and / or type B cells are distributed in a region of the image according to a certain predetermined mathematical distribution; - calculating (210) the proximity score (718, 720) as the difference between the reference relative distribution and the observed relative distribution; - calculating (212) a combination score (912), the combination score including the proximity score and including the density of type A cells and / or the density of type B cells; and - Using the combined score to determine the biomedical state of the tissue sample (214) and / or outputting the combined score to a user to enable the user to determine the biomedical state of the tissue sample (216). A method comprising:

2. Observed relative distributions can be obtained by: For each identified type A cell observed in the image field: a) selecting said type A cell as the center of a circle with radius 0; b) Increasing the radius by one step to produce an increased circle; c) determining the number of B-type cells contained in the circle generated in step b); d) storing in a storage medium the current radius of the circle associated with the number of type B cells determined in step c) and repeating steps b), c) and d) until a termination criterion is reached; and e) selecting one of the unselected type A cells and performing steps a), b), c) and d) using the newly selected type A cell until all type A cells have been selected; and - Providing the relative radius and number of observed B cells as the observed relative distribution 2. The image analysis method of claim 1, comprising:

3. Obtaining a reference relative distribution (208) includes: - simulating by a computer a distribution of simulated reference type A cells in the region, wherein the number of simulated reference type A cells is the same as the identified number of observed type A cells in the image region; - simulating by a computer a distribution of simulated reference type B cells in the region, wherein the number of simulated reference type B cells is the same as the identified number of type B cells observed in the image region; and - calculating a reference relative distribution as a function of the computer-simulated distribution of reference type A and reference type B cells, the reference relative distribution indicating the distance between the simulated reference type A cells and the simulated reference type B cells in the region; 3. The image analysis method according to claim 1 or 2, comprising:

4. 4. The image analysis method of claim 3, wherein the distribution of the simulated reference type A cells is a Poisson distribution and the distribution of the simulated reference type B cells is a Poisson distribution.

5. Calculating the reference relative distribution as a function of the computer-simulated distribution of the reference type A and reference type B cells comprises: For each of the randomly distributed simulated reference type A cells: a) selecting the simulated reference type A cell as the center of a circle with a radius of 0; b) Increasing the radius by one step to produce an increased circle; c) determining the number of simulated reference B cells contained in the circle generated in step b); d) storing in a storage medium the current radius of the circle associated with the number of simulated reference B cells determined in step c), and repeating steps b), c) and d) until a termination criterion is reached; and e) selecting an unselected one of the simulated reference type A cells and continuing steps a), b), c) and d) using the newly selected simulated reference type A cell until all simulated reference type A cells have been selected; and - Providing the relative radius and number of simulated reference B cells as a reference relative distribution 5. The image analysis method according to claim 3 or 4, comprising:

6. below: - calculating a plurality of initial reference relative distributions according to any one of claims 3 to 5; - calculating an average reference relative distribution from a plurality of initial reference relative distributions; and - Use the average reference relative distribution as the reference relative distribution 6. The image analysis method according to claim 3, further comprising:

7. Calculating 210 the proximity score as the difference between the reference relative distribution and the observed relative distribution is as follows: - providing a predetermined number representing a predetermined minimum number of B-type cells; - within the observed relative distribution, the observed radius r ｏ－最小 and identifying the observation radius r ｏ－最小 Within the observed relative distribution, there is an observation radius r, which is the radius that, when drawn around each observed type A cell, defines a circle that contains, on average, a given number of observed type B cells. ｏ－最小 Identifying; - Reference radius r within the reference relative distribution ｒ－最小 and identifying a reference radius r ｒ－最小 is a reference radius r within the reference relative distribution, which is a radius that, when drawn around each of the reference type A cells, defines a circle that contains, on average, a predetermined number of reference type B cells to provide the reference relative distribution. ｒ－最小 Identifying the - Proximity score is calculated by the observation radius r ｏ－最小 and the reference radius r ｒ－最小 , where the function is the difference between the observation radius and the reference radius, ｏ－最小 and the reference radius r ｒ－最小 Calculating as a function of 7. The image analysis method according to claim 1, comprising:

8. below: - graphically representing the observed relative distribution as an observed distribution curve in a 2D plot, where a first dimension represents the radius r (Ro) of circles around the observed type A cells and a second dimension represents the number of observed type B cells within each said circle; and displaying the 2D plot on a display device of the image analysis system; and / or - graphically representing the reference relative distribution as a reference distribution curve in a 2D plot, the first dimension of which represents the radius r (Rs) of a circle around the reference type A cells and the second dimension of which represents the number of reference type B cells within said circle; and displaying the 2D plot on a display device of the image analysis system.

8. The image analysis method of claim 1, further comprising:

9. 9. The image analysis method according to claim 1, wherein type A cells are tumor cells and type B cells are immune cells.

10. 10. The image analysis method of claim 1, wherein the tissue sample is a tumor tissue sample and determining the biomedical state of the tissue sample comprises determining the infiltration state of the tumor tissue shown in areas of the received digital image that contain immune cells.

11. below: - graphically representing the combined score as symbols in a score 2D plot, where a first dimension of the 2D plot represents the proximity score and a second dimension of the 2D plot represents the density of type B cells or type A cells; outputting the 2D score plot on a display of the image analysis system (214) to enable a human to determine the biomedical state of the tissue sample; and / or - determining the biomedical state of the tissue sample by automatically identifying the biomedical state of the tissue sample as a selected one of a limited set of predetermined biomedical states or from a predetermined continuous spectrum of biomedical states by an image analysis system, and outputting the identified biomedical state as a determined biomedical state; 11. The image analysis method of claim 1, further comprising:

12. below: - receiving digital images of tissue samples from each of a plurality of different patients; - calculating a combined score for each patient using each received image according to claim 11; and - graphically representing each patient's biomedical condition on a 2D score plot by a respective biomedical condition-specific symbol, the position of each patient's symbol in the plot depending on the B-cell density and proximity scores calculated for the patient; 12. The image analysis method of claim 11, further comprising:

13. The identified biomedical condition is: - "inflammation", an immunological tissue state with a marked increase in the cellular density of immune cells, indicating that immune cells are massively infiltrating the tumor tissue of the tumor in all compartments of the tumor; - "exclusion", which indicates an immunological tissue state in which immune cells are present in the tissue sample but are prevented from coming into close contact with the tumor cells, thereby concentrating them at the invasive margin and / or intratumoral stroma, but separated from the tumor cells; and - "Depleted", which refers to an immunological tissue state in which immune cells in the tumor tissue area have zero or near-zero cell density.

10. The image analysis method of claim 9, selected from the group comprising:

14. below: - if the tissue sample is classified as a tumor infiltration type "inflammation", outputting, via a user interface, a treatment recommendation for prescribing a drug acting as a checkpoint inhibitor; - if the tissue sample is classified as a tumor infiltration type "exclusion", outputting, via a user interface, a treatment recommendation for prescribing a drug adapted to attract immune cells close to the tumor cells; and - if the tissue sample is classified as "exhausted" in tumor infiltration status, outputting a treatment recommendation via the user interface to prescribe drugs adapted to boost the immune system in general; 14. The image analysis method of claim 1, further comprising:

15. 15. The image analysis method of claim 1, wherein the density of the reference type A cells is the same as the density of type A cells observed in the image region, and the density of the reference type B cells is the same as the density of type B cells observed in the image region.

16. 1. A method for determining the effectiveness of a drug for treating a particular type of cancer, comprising: receiving a plurality of first digital images, each first image showing a tissue sample of an individual organism having a particular type of cancer before said organism has been treated with a drug; - calculating a first combination score from each image region of the first digital image according to any one of claims 1 to 15, wherein the biomedical condition is an immune cell infiltration condition of a tumor, type A cells being tumor cells and type B cells being immune cells; receiving a plurality of second digital images, each second image showing a tissue sample of an individual organism having a particular type of cancer after said organism has been treated with a drug; - calculating a second combination score from each image region of the second digital image according to any one of claims 1 to 15, wherein the biomedical condition is a tumor infiltration condition, the type A cells are tumor cells, and the type B cells are immune cells; and - displaying a 2D score plot on a display device, wherein each first combined score is represented by a first symbol in the 2D score plot, each second combined score is represented by a second symbol in the 2D score plot, the position of the first and second symbols in the plot depends on the immune cell density observed in the image region of the individual images and the proximity score calculated for the image region of the individual images, and wherein first and second symbols representing the same organism are visually linked in the 2D score plot from any one of the first symbols to its individually linked second symbol in order to visualize any shift in x and / or y coordinates. A method comprising:

17. 16. A storage medium comprising computer-interpretable instructions which, when executed by a processor, cause the processor to perform the image analysis method of any one of claims 1 to 15.

18. An image analysis system (100) for determining a biomedical condition of a tissue sample, the image analysis system (100) including computer-interpretable instructions that cause a processor executing the instructions to: - receiving (202) a digital image (118) of a tissue sample; - analyzing the received image (204) to identify the number and location of type A cells (604) and type B cells observed in a region of the received image, where type A and type B are different cell types; - analyzing (206) the locations of type A and type B cells in the region to obtain an observed relative distribution, the observed relative distribution indicating the observed distance between type A cells and type B cells in the region; - obtaining a reference relative distribution (208), wherein the reference relative distribution indicates the expected distance between reference type A cells and reference type B cells; - calculating (210) a proximity score as the difference between the reference relative distribution and the observed relative distribution; - calculating a combination score (212), wherein the combination score includes a proximity score and includes the density of type A cells and / or the density of type B cells; and - Using the combined score to determine the biomedical state of the tissue sample (214) and / or outputting the combined score to a user to enable the user to determine the biomedical state of the tissue sample (216). A system comprising computer-interpretable instructions configured to cause execution of instructions for performing a method comprising:

19. 1. A data processing system for determining a biomedical state of a tissue sample, the system including a spatial database management system (DBMS) and an application program; The spatial DBMS: - observation space data, comprising positions of type A cells (604) and type B cells observed in a region of the digital image (118) of the tissue sample, where type A and type B are different cell types; - reference spatial data, comprising the locations of reference type A cells and reference type B cells; Including, The application program: - analyzing the observed spatial data to obtain an observed relative distribution, the observed relative distribution indicating the observed distances between type A cells and type B cells in the region, wherein the application program is configured to use spatial operations provided by the spatial DBMS to calculate the observed distances; - Identifying the number of A and B cells; - analyzing the reference spatial data to obtain a reference relative distribution, the reference relative distribution indicating an expected distance between an identified number of reference type A cells and reference type B cells in the region, wherein the application program is configured to use spatial operations provided by the spatial DBMS to calculate the expected distance, and the number of reference type A cells and reference type B cells is the same as the identified number of type A cells and type B cells, respectively; - calculating (210) a proximity score as the difference between the reference relative distribution and the observed relative distribution; - calculating a combination score (212), wherein the combination score includes a proximity score and includes the density of type A cells and / or the density of type B cells; and - Using the combined score to determine the biomedical state of the tissue sample (214) and / or outputting the combined score to a user to enable the user to determine the biomedical state of the tissue sample (216). instructing a data processing system to Data processing system.

Images