Computational System for Multiparameter Cells and Intracellular Image Data in Situ or In Vitro: Pathology Spatial Analysis Platform

Abstract

SOLUTION: A computational system pathology spatial analysis platform includes: (i) a spatial heterogeneity quantification component configured for generating a global quantification of spatial heterogeneity among cells of varying phenotypes in multi-parameter cellular and subcellular imaging data; (ii) a microdomain identification component configured for identifying a plurality of microdomains for tissue samples based on the global quantification; and (iii) a weighted graph component configured for constructing a weighted graph for the multi-parameter cellular and subcellular imaging data, the weighted graph having a plurality of nodes and a plurality of edges each being located between a pair of the nodes. In the weighted graph, each node is a particular one of the microdomains and the edge between each pair of microdomains in the weighted graph is indicative of a degree of similarity between the pair of the microdomains.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to digital pathology, and in particular to a comprehensive computational system pathology space analysis (CSPSA) computer platform that can integrate, visualize, and model high-dimensional cell and intracellular resolution image data in situ or in vitro.

Background Art

[0002] Digital pathology refers to the acquisition, storage, and display of histologically stained tissue samples, and initially attracted attention in niche applications such as second opinion telepathology, interpretation of immunohistochemistry, and intraoperative telepathology. Usually, a large amount of patient data composed of 3 to 50 slides is generated from biopsy specimens and visually evaluated microscopically by pathologists, but with digital technology, it can be evaluated by looking at a high-resolution monitor. Due to the inclusion of manual work, the current workflow practice is time-consuming, error-prone, and subjective.

[0003] Cancer is a heterogeneous disease. In hematoxylin-eosin (H&E) stained histological images, heterogeneity is characterized by the presence of various histological structures such as carcinoma in situ, invasive carcinoma, adipose tissue, blood vessels, and normal ducts. Furthermore, in many malignant tumors, molecular and cellular heterogeneity is a prominent feature between tumors in different patients, between different sites of neoplasia in the same patient, and within a single tumor. Intratumor heterogeneity involves phenotypic subpopulations of cancer cell clones and other cell types that constitute the tumor microenvironment (TME). These cancer cell clonal subpopulations and other cell types include localized myeloid-derived stromal stem cells and progenitor cells, subclasses of immunoinflammatory cells that are either tumor-promoting or tumor-lethal, cancer-associated fibroblasts, endothelial cells, and pericytes. The cellular microorganism (TME) can be viewed as a developing ecosystem in which cancer cells engage in heteromorphic interactions with other cell types, utilizing available resources to proliferate and survive. From this perspective, spatial interactions between cell types within the TME (i.e., spatial heterogeneity) are considered to be one of the main factors in disease progression and treatment resistance. Therefore, clarifying the spatial heterogeneity within the TME is essential for correctly determining specific disease subtypes and identifying the optimal treatment course for individual patients.

[0004] To date, intratumor heterogeneity has been investigated using three main approaches. The first approach involves taking core samples from specific regions of the tumor and measuring the population mean. Sample heterogeneity is measured by analyzing multiple cores within the tumor using multiple techniques such as whole exome sequencing, epigenetics, proteomics, and metabolomics. The second approach involves "single-cell analysis" using the above methods, RNASeq, imaging, or flow cytometry after separating cells from the tissue. The third approach utilizes the spatial resolution of optical microscopy imaging to maintain spatial context and measure cellular biomarkers in situ in combination with molecularly specific labeling. Biomarkers can identify cell type, activation state (e.g., phosphorylation of target proteins), and intracellular function. While each of these approaches exhibits a certain level of effectiveness, they also have various drawbacks and limitations.

[0005] Furthermore, one of the biggest problems in evaluating the clinical significance of tumor heterogeneity is the lack of advanced tools for spatial analysis of multi-parameter cellular and intracellular image data. [Overview of the project]

[0006] In one embodiment, a method is provided for analyzing disease progression from multiparameter cellular and intracellular image data obtained from multiple tissue samples from multiple patients or several multicellular in vitro models. The method of the present invention includes the steps of: generating an overall quantification of spatial heterogeneity between cells of several different predetermined phenotypes in the multiparameter cellular and intracellular image data; identifying a plurality of microdomains for the plurality of tissue samples based on the overall quantification, wherein each microdomain is associated with one of the individual tissue samples; and constructing a weighted graph for the multiparameter cellular and intracellular image data. The weighted graph has a plurality of nodes and a plurality of edges, each of which lies between pairs of the nodes, wherein in the weighted graph, each node is a specific one of the microdomains, and the edges between each pair of microdomains in the weighted graph indicate the degree of similarity between those pairs of microdomains.

[0007] In another embodiment, a system is provided for analyzing disease progression from multiparameter cellular and intracellular image data obtained from multiple tissue samples from multiple patients or several multicellular in vitro models. The system of the present invention includes a processing device comprising: (i) a spatial heterogeneity quantification component configured to produce an overall quantification of spatial heterogeneity between cells of several different predetermined phenotypes in multiparameter cellular and intracellular image data; (ii) a microdomain identification component configured to identify multiple microdomains for multiple tissue samples based on the overall quantification, wherein each microdomain is associated with one of the individual tissue samples; and (iii) a weighted graph component that constructs a weighted graph for multiparameter cellular and intracellular image data, wherein the weighted graph has multiple nodes and multiple edges, each of which is located between pairs of nodes, wherein in the weighted graph, each node is a specific one of microdomains, and the edges between each pair of microdomains in the weighted graph indicate the degree of similarity between those pairs of microdomains.

[0008] In yet another embodiment, a method is provided for generating a spatially-based representation of heterocellular communication from multiparameterized cell and intracellular image data obtained from multiple tissue samples from multiple patients or several multicellular in vitro models. The method of the present invention includes the steps of: generating a quantification of spatial heterogeneity between several different predetermined phenotypic cells in multiparameterized cell and intracellular image data, and identifying microdomains for one of the several tissue samples based on the quantification, which includes performing phenotyping on the multiparameterized cell and intracellular image data to identify the several different predetermined phenotypes; and constructing a communication graph for the microdomains. Each phenotype is a node in the communication graph, and the edges between each pair of phenotypes in the communication graph indicate the influence of one phenotype in the pair on the presence of the other phenotype in the pair.

[0009] In yet another embodiment, a computer system is provided for generating a spatial information-based representation of heterocellular communication from multiparameter cell and intracellular image data obtained from multiple tissue samples from multiple patients or several multicellular in vitro models. The system of the present invention includes a processing device comprising: (i) a spatial heterogeneity quantification component configured to generate a quantification of spatial heterogeneity between several different predetermined phenotypic cells in multiparameter cell and intracellular image data, which performs cell phenotyping on the multiparameter cell and intracellular image data to identify specific different predetermined phenotypes; (ii) a microdomain identification component configured to identify a plurality of microdomains for several tissue samples based on the quantification, where each microdomain is associated with one of the several tissue samples; and (iii) a communication network component configured to construct a communication graph for one selected of the plurality of microdomains, where each phenotype is a node in the communication graph, and the edges between each pair of phenotypes in the communication graph indicate the influence of one phenotype in the pair on the presence of the other phenotype in the pair.

[0010] In yet another form, a method is provided for creating a personalized medical strategy for a specific patient, wherein the specific patient is one of several patients, and the several patients are associated with multiparameterized cellular and intracellular image data obtained from several tissue samples from the said several patients. The method of the present invention includes the steps of: identifying a plurality of microdomains in one of several tissue samples associated with a specific patient, wherein the plurality of microdomains are based on multiparameterized cellular and intracellular image data; generating a heterogeneous intercellular communication network for each of the plurality of microdomains, wherein each heterogeneous intercellular communication network includes a representation of heterogeneous intercellular communication based on the spatial information of the microdomains; quantifying the spatial and temporal interdependencies of the microdomains based on the heterogeneous intercellular communication network; and formulating a medical strategy for a specific patient based on the quantified results.

[0011] In yet another embodiment, a method is provided for representing the temporal evolution of a disease in a particular patient, where the particular patient is one of several patients, and the several patients are associated with multiparameterized cellular and intracellular image data obtained from several tissue samples from several patients. The method of the present invention comprises the steps of generating a geometric representation of the disease landscape for a particular patient, the geometric representation comprising several points, each point of the geometric representation being (i) based on one particular of several tissue samples associated with a selected patient and describing the disease status of the particular patient at a particular point in time, (ii) based on a microdomain in one particular of several tissue samples based on multiparameterized cellular and intracellular image data, and (iii) comprising an intercellular communication network for the microdomain, which includes a representation of intercellular communication networks for the microdomain based on spatial information. [Brief explanation of the drawing]

[0012] [Figure 1]Figure 1 is a schematic diagram of an exemplary computational system pathology spatial analysis (CSPSA) platform for in situ or in vitro multi-parameter cell and intracellular image data, based on one embodiment of the disclosed concept. [Figure 2] Figure 2 is a schematic diagram of an exemplary hyperplexed image stack. [Figure 3] Figure 3 is a flowchart illustrating a method for analyzing tumor progression / progression from multi-parameter cellular and intracellular image data obtained from multiple tumor sections from a patient cohort, and this method can be implemented on the CSPSA platform of Figure 1 based on exemplary embodiments of the disclosed concept. [Figure 4] Figure 4 is a schematic diagram of an exemplary overall spatial map. [Figure 5] Figure 5 is a schematic diagram of each of the predetermined dominant biomarker intensity patterns in exemplary embodiments of the disclosed concept. [Figure 6] Figure 6 is a schematic diagram of a cell space-dependent image based on one specific exemplary embodiment of the disclosed concept. [Figure 7] Figure 7 is a schematic diagram of an exemplary microdomain of an exemplary tissue section based on one exemplary embodiment of the disclosed concept. [Figure 8] Figure 8 is a schematic diagram of a weighted microdomain graph configured for multi-parameter cells and intracellular image data, based on an exemplary embodiment of the disclosed concept. [Figure 9] Figure 9 is a flowchart illustrating a method for generating a spatial information-based representation of interspecies cell communication from multiparameter cells and intracellular image data, based on another exemplary embodiment of the disclosed concept. [Figure 10] Figure 10 is a schematic diagram of an exemplary communication graph based on one exemplary embodiment of the disclosed concept. [Figure 11]Figure 11 is a schematic diagram of a specific communication graph generated based on a particular exemplary embodiment in which the phenotype is tumor cell, lymphocyte, macrophage, stroma, and necrosis. [Figure 12] Figure 12 is a schematic diagram of a personalized medicine strategy based on one aspect of the disclosed concept. [Figure 13] Figure 13 is a schematic diagram of an exemplary cancer landscape based on one aspect of the disclosed concept. [Modes for carrying out the invention]

[0013] In this specification, the singular forms of "aru" (aru) and "sono" (sono) include references to the plural unless the context clearly indicates otherwise.

[0014] In this specification, the statement that two or more parts or components are “joined” means that, insofar as a connection occurs, the parts are joined or operate together directly or indirectly, i.e., through one or more intermediate parts or components.

[0015] In this specification, the term "several" means one or an integer greater than one (i.e., multiple).

[0016] In this specification, the terms “component” and “system” are intended to refer to computer-related entities that are either hardware, a combination of hardware and software, software, or running software. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. For example, both an application running on a server and the server may be components. One or more components may reside within a process and / or an execution thread, and components may be localized on one computer and / or distributed across two or more computers. While several methods of displaying information to the user are illustrated and described with specific figures or graphs as screenshots, those skilled in the art will recognize that various other alternative means may be employed.

[0017] As used herein, the term "multiparameter cellular and intracellular image data" means data obtained by generating a plurality of images from several sections of a tissue, and providing information on a plurality of measurable parameters at the cellular or intracellular level in those sections of the tissue. The multiparameter cellular and intracellular image data may be created by several different imaging techniques, such as, for example, (1) transmission light imaging by IHC using several biomarkers, (2) immunofluorescence imaging including both multiplex imaging (1 to 7 biomarkers) and hyperplex imaging (more than 7 biomarkers), (3) toponome imaging, (4) matrix-assisted laser desorption / ionization mass spectrometry imaging (MALDI MSI), (5) complementary spatial imaging, such as FISH, MxFISH, FISHSEQ, or CyTOF, (6) multiparameter ion beam imaging, (7) in vitro imaging of experimental models, but is not limited thereto. In addition, for example, but not limited to, multiparameter cellular and intracellular image data may be generated by generating several biomarker images from several sections of a tissue by labeling each section of the tissue with a plurality of different biomarkers.

[0018] In this specification, the term “spatial map” means a representation of several quantified spatial statistics that show relationships between cells of different phenotypes in a set of multi-parameter cellular and intracellular imaging data, e.g., a representation of spatial statistics in a collection of data and / or in a visually perceptible form. For example, but not limited to, a spatial map may be a pointwise mutual information (PMI) map generated by the method described in PCT application PCT / US2016 / 036825 and U.S. Patent Application Publication 2018 / 0204085, both of which are titled “Systems and Methods for Finding Regions of Interest in Hematoxylin and Eosin (H&E) Stained Tissue Images and Quantifying Intratumor Cellular Spatial Heterogeneity in Multiplexed / Hyperplexed Fluorescence Tissue Images,” the disclosures of which are incorporated herein by reference.

[0019] In this specification, the term “microdomain” means a spatial arrangement (or motif) of phenotypically distinct cells in a tissue sample, which arises from spatial intratumoral heterogeneity and is associated with one or more outcome-specific variables (e.g., time to recurrence). Microdomains may be identified from multi-parameter cellular and intracellular imaging data according to any of several known or hereafter developed methods. Such methods include, by reference, U.S. Provisional Application No. 62 / 675,832, “Predicting the Recurrence Risk of Cancer Patients From Primary Tumors with Multiplexed Immunofluorescence Biomarkers and Their Spatial Correlation Statistics,” and PCT Application No. PCT / US2019 / 033662, “System and Method for Predicting the Risk of Cancer Recurrence From Spatial Multi-Parameter Cellular and Sub-Cellular Imaging Data for Tumors by Identifying Emergent Spatial Domain Networks Associated With Recurrence,” which are incorporated herein by reference, and / or the methods described in Spagnolo, et al., “Platform for Quantitative Evaluation of Spatial Intratumoral Heterogeneity in Multiplexed Fluorescence Images, CancerRes. 2017 Nov 1;77(21):e71-e74,” and / or the public domain THRIVE(Tumor) described in the aforementioned references by Spagnolo et al. This method is implemented in, but is not limited to, the Heterogeneity Research Interactive Visualization Environment (HETRO) software.In that method, a map of the spatial organization of cancer recurrence in multiplexed tissue samples is constructed using the spatial resolved correlation between biomarkers as covariates of a multivariable survival model of prognostic data (e.g., recurrence). These maps depict microdomains associated with the progression of recurrence and metastasis.

[0020] For example, terms related to directions used herein, such as up, down, left, right, upper, lower, front, back, and their derivatives, relate to the directions of the elements shown in the drawings and do not limit the claims unless explicitly stated.

[0021] Hereinafter, the disclosed concepts will be described in terms of many specific details for the purpose of explanation to provide a complete understanding of the present invention. However, it will be apparent that the disclosed concepts can be implemented without these specific details without departing from the spirit and scope of the present invention.

[0022] The disclosed concepts provide a comprehensive computational system pathology space analysis (CSPSA) platform that can integrate, visualize, and / or model high-dimensional in situ cell and intracellular resolution image data. The CSPSA platform of the disclosed concepts combines a core set of existing tools for cell phenotyping and spatial analysis with an advanced toolset for inferring spatial heterocellular communication (intercellular) and intracellular communication patterns and, in the case of cancer, constructing an evolutionary lineage tree of tumors (or other diseases) from pre-cancerous origins to the endpoints of metastasis. By combining these tools, it is possible to apply them to both research and clinical purposes, such as quantifying spatial intratumoral heterogeneity in tissue samples and in vitro models and correlating it with prognostic data, constructing diagnostic and prognostic diagnostic applications, designing individualized treatment strategies and drug discovery. Furthermore, this platform can be enhanced by adding region-specific genomics and spatial transcriptomics data to in situ cell and intracellular high-definition images.

[0023] Figure 1 is a schematic diagram of an exemplary Computational System Pathology Spatial Analysis (CSPSA) platform 5 for in situ multiparameter cell and intracellular image data based on embodiments of the disclosed concept, where various methods described herein can be implemented. As shown in Figure 1, the CSPSA platform 5 is a computer device configured to receive and store specific multiparameter cell and intracellular image data 10 (for example, relating to a large patient cohort including recurrent and non-recurrent cancer tumors, or relating to individual patients or individual tumors) and to process the data 10 as described herein. In non-limiting and exemplary embodiments, the multiparameter cell and intracellular image data 10 is generated using multiplexed or hyperplexed immunofluorescence imaging, but it will be understood that other imaging techniques, such as those described elsewhere herein, can also be used. For example, the multiparameter cell and intracellular image data 10 may be based on an exemplary hyperplex image stack 12 relating to all patient data in a cohort, as shown in Figure 2.

[0024] The CSPSA platform 5 may, but is not limited to, a PC, laptop computer, tablet computer, smartphone, or other suitable computer device configured to perform the functions described herein. The CSPSA platform 5 includes an input device 15 (such as a keyboard), a display 20 (such as an LCD), and a processing unit 25. A user can provide input to the processing unit 25 using the input device 15, and the processing unit 25 provides output signals to the display 20 so that the display 20 can display information to the user (e.g., spatial maps or other spatially dependent images, microdomain images, weighted graph images, and / or communication network graph images), as described in detail herein. The processing unit 25 comprises a processor and memory. The processor is, for example, a microprocessor (μP), a microcontroller, an application-specific integrated circuit (ASIC), or other suitable processing device interfaced with the memory. Memory may be one or more of various types of internal and / or external storage media, such as RAM, ROM, EPROM, EEPROM, FLASH®, or other devices that provide storage registers, such as computer-readable media, and may be volatile or non-volatile memory. Memory stores several routines that can be executed by the processor, including routines for implementing the disclosed concepts as described herein.In particular, the processing device 25 includes, as described herein, a spatial heterogeneity quantification component 30 configured to generate an overall quantification of intercellular spatial heterogeneity of several different predetermined phenotypes in multiparameter cell and intracellular image data 10; a microdomain identification component 35 configured to identify multiple microdomains for multiple tumor sections from the multiparameter cell and intracellular image data 10 based on the overall quantification generated by the spatial heterogeneity quantification component 30, as described herein; a weighted graph component 40 configured to construct a weighted graph of the multiparameter cell and intracellular image data 10, as described herein; and a communication network component 45 configured to construct a communication network graph for the multiparameter cell and intracellular image data 10, as described herein.

[0025] Figure 3 is a flowchart illustrating a method for analyzing tumor progression / progression from multi-parameter cellular and intracellular image data 10 obtained from multiple tumor sections of a patient cohort, the method which can be implemented on the CSPSA platform 5 based on an exemplary embodiment of the disclosed concept. However, it is intended to be illustrative only, and it will be understood that the steps of the method shown in Figure 3 may be implemented in other configurations and / or platforms.

[0026] The method begins in step 50, in which the spatial heterogeneity quantification component 30 results in a global quantification of spatial heterogeneity in the multiparameter cell and intracellular image data 10. In an exemplary embodiment, the global quantification obtained in step 50 is a global spatial map of the multiparameter cell and intracellular image data 10. In a particular embodiment, the global spatial map is a global PMI map 52, as shown in Figure 4, which is generated by the method described in PCT application PCT / US2016 / 036825 and U.S. Patent Application Publication 2018 / 0204085, which are referred to elsewhere herein and are part of this specification by reference. In this exemplary embodiment, the multiparameter cell and intracellular image data 10 is created by labeling each section of tissue with several different biomarkers (e.g., ER, PR, and HER2) to generate several biomarker images from several sections of tissue. As described in the aforementioned application, the PMI map 52 is first generated by performing cell segmentation on the multi-parameter cell and intracellular image data 10 (i.e., on each of its “slides”). Any of several known or future-developed suitable cell segmentation algorithms may be employed. Then, spatial location and biomarker intensity data are obtained for each cell, and each cell is assigned to one of a predetermined phenotype (each phenotype is a predetermined dominant biomarker intensity pattern) based on the biomarker intensity configuration of the cell. Figure 5 shows schematic representations 56 of each of the predetermined dominant biomarker intensity patterns, referenced 1 to 8, in an exemplary embodiment. In the exemplary embodiment, each schematic representation 56 is provided in a unique color so that the schematic representations can be easily distinguished from one another. The cell assignments described herein and the schematic representations shown in Figure 5 can be used to generate a cell space-dependent image that visually shows the heterogeneity of the target tissue sample. Figure 6 shows a cell space-dependent image 58 based on one particular exemplary embodiment of the disclosed concept. As shown in Figure 6, the cell spatial dependence image 58 shows the spatial dependence between cells in the subject's slide using a schematic representation 56.Next, a spatial network is constructed to represent the composition of the intensity patterns of key biomarkers in the subject's slides. Then, the heterogeneity of the subject's slides is quantified by generating a PMI map 52 as shown in Figure 4. In an exemplary embodiment, the spatial network and PMI map 52 are generated as follows.

[0027] A network is constructed for the subject's slides to represent the spatial configuration of biomarker patterns in the biomarker images (i.e., tissue / tumor samples) of the subject's slides. The construction of a spatial network for tumor samples essentially links the intensity data (network nodes) and spatial data (network edges) of cellular biomarkers. The premise in network construction is that cells have the ability to communicate with nearby cells up to a certain limit, e.g., up to 250 μm, and that the ability of cells to communicate within that limit depends on the distance between cells. Therefore, the probability distribution in the exemplary embodiment is calculated for the distance between a cell in the subject's slide and its 10 nearest neighbor cells. A hard limit is selected based on the median value of this distribution multiplied by 1.5 (to estimate the standard deviation), and cells in the network are connected only within this limit. The edges between cells in the network are then weighted by the distance between adjacent cells.

[0028] Next, in an exemplary embodiment, pointwise mutual information (PMI) is used to assess the association between each pair of biomarker patterns in the dictionary, and thus between different cellular phenotypes, for the subject's slides. This metric captures both common linear and nonlinear statistical associations, while previous studies have used linear metrics such as Spearman's rho coefficient. Once the PMI is calculated for each pair of biomarker patterns, all measures of association in the subject's slide data are displayed in the PMI map 52. The use of PMI in this embodiment is illustrative, and other methods for defining spatial relationships, such as those described in Spagnolo DM, Al-Kofahi Y, Zhu P, Lezon TR, Gough A, Stern AM, Lee AV, Ginty F, Sarachan B, Taylor DL, Chennubhotla SC, Platform for Quantitative Evaluation of Spatial Intratumoral Heterogeneity in Multiplexed Fluorescence Images, Cancer Res. 2017 Nov 1, and Nguyen, L., Tosun, B., Fine, J., Lee, A., Taylor, L., Chennubhotla, C. (2017), Spatial statistics for segmenting histological structures in H&E stained tissue images, IEEE Trans Med Imaging. 2017 Mar 16, may be adopted with respect to the disclosed concepts.

[0029] An exemplary PMI map 52 shows the distribution of different cellular phenotypes within the microenvironment of the subjects' slides. Specifically, the entries in PMI map 52 indicate how frequently a particular spatial interaction between two phenotypes (referenced by row and column numbers) occurs in the dataset compared to interactions predicted by a random (or background) distribution across all phenotypes. Entries of a first color, such as red, indicate strong spatial associations between phenotypes, while entries of a second color, such as black, indicate a lack of colocalization (weak spatial associations between phenotypes). Other colors may be used to indicate other associations. For example, a PMI entry colored with a third color, such as green, indicates an association no better than the random distribution of cellular phenotypes across the entire dataset. Furthermore, PMI map 52 can represent anti-associations with entries shown in a fourth color, such as blue (for example, if phenotype 1 rarely occurs spatially near phenotype 3).

[0030] Referring again to Figure 3, following step 50, the method then proceeds to step 55. In step 55, the microdomain identification component 35 identifies and locates one or more microdomains in each tumor section in the multi-parameter cell and intracellular image data 10 (as described elsewhere in this specification, for example) based on the overall quantification obtained in step 50. For example, certain cancer cell types and certain immune cell types coexisting with them may have a high spatial co-occurrence value. Using a combination of cancer and immune cells as a seed point, microdomains can be generated in any tissue section by growing a spatial network of cells around a seed based on a distance cutoff (e.g., about 100 cells) by mining the overall quantification (e.g., a spatial map) and propagating the phenotypic associations. Such exemplary microdomains 62 are shown in the exemplary tissue section 64 shown in Figure 7.

[0031] Next, in step 60, for each microdomain identified in step 55, a local quantification of spatial heterogeneity (e.g., a local spatial map such as a local PMI map) is obtained. Furthermore, each of the local quantifications determined for each tissue section is used to define the degree of similarity between pairs of microdomains in the tissue section. In particular, in exemplary embodiments, given two microdomains A and B, (i) the difference in the relative abundance of cells in microdomain A and microdomain B, and (ii) their local quantifications (e.g., PMI) are obtained. A and PMI B A distance function is defined, which is a combination of the difference between local spatial maps (such as, or, as described herein, other methods may be used). This procedure is repeated for all pairs of microdomains across all multi-parameter cell and intracellular image data 10. This process determines a weighted similarity for each pair of microdomains in each tissue section.

[0032] Next, in step 65, a weighted microdomain graph 66, as illustrated in Figure 8, is constructed for the multi-parameter cell and intracellular image data 10. Each node 68 in the weighted microdomain graph 66 is one of the microdomains generated in step 55, and the edges 72 connecting each pair of microdomains are weighted by their determined similarity (e.g., the weighted similarity values ​​described above). The weighted microdomain graph 66 may then be displayed on the display 20.

[0033] Next, as shown in step 70 of Figure 3, further analysis of the weighted microdomain graph 66 can be used to understand the progression of phenotypes during the metastasis process. While ground truth labels exist for each tissue section of multiparameterized cells and intracellular image data 10, originating from either relapsed (R) or non-relapsed (NR) patients, it should be noted that a priori knowledge is not available to determine which microdomains best distinguish between the two categories. In an exemplary embodiment, labels R and NR are transferred from the tissue section to the corresponding microdomains. Probability propagation is then repeated several times to improve the labels, and cliques of microdomains in the weighted microdomain graph 66 that are uniformly R or NR are isolated. With improved reliability of label assignment, a betweenness centrality measure can be used to identify the microdomains most commonly transferred in the NR-to-R pathway. To mimic the evolutionary trajectory, a random walk of the weighted microdomain graph 66 can be performed, and it is known that some microdomains are found only in either observed or unobserved metastasis data. By comparing the relative probabilities of paths from NR to R, a class of events significant to the transition can be identified. Next, using the mean first traverse time of the random walk, microdomains prone to transition can be identified as a set of nodes where the probability of the random walk reaching either R or NR is 0.5. Note that microdomains with potential for transition are midpoints in the evolutionary trajectory from NR to R. Note that in the exemplary embodiment, the steps described above are performed by components of the CSPSA platform 5, and the results are displayed to the user on the CSPSA platform 5 display 20. Thus, this aspect of the disclosed concept provides the ability to define, identify, and compare phenotypic evolutions significant to the transition, as described herein.Furthermore, cancer is merely one example of a field to which the disclosed concepts can be applied; it will be understood that these concepts can also be applied to other disease tissues, particularly neurodegenerative diseases, metabolic diseases, and inflammatory diseases.

[0034] Figure 9 is a flowchart illustrating a method in this embodiment for generating a spatial information-based representation of heterocellular communication from multi-parameter cell and intracellular image data 10 that may be obtained from several tumor sections from a patient, which may be implemented on the CSPSA platform 5 according to another exemplary embodiment of the disclosed concept. However, it is intended to be illustrative only, and it will be understood that the steps of the method shown in Figure 9 may be implemented in other configurations and / or platforms.

[0035] The method begins in step 75, in which the spatial heterogeneity quantification component 30 results in the quantification of spatial heterogeneity in the multiparameter cell and intracellular image data 10, as described in detail elsewhere in this specification. As stated elsewhere in this specification, the spatial heterogeneity quantification performed in step 75 includes performing cell phenotyping on the multiparameter cell and intracellular image data 10 to identify different predetermined phenotypes. In exemplary embodiments, the quantification obtained in step 75 is a spatial map of the multiparameter cell and intracellular image data 10. In certain embodiments, the spatial map is generated by the method described elsewhere in this specification and by reference in PCT application PCT / US2016 / 036825 and U.S. Patent Application Publication 2018 / 0204085. In this exemplary embodiment, the multiparameter cell and intracellular image data 10 may be generated by generating several biomarker images from several sections of tissue by labeling each section of tissue with several different biomarkers (e.g., ER, PR, and HER2). In non-limiting and exemplary embodiments, multi-parameter cell and intracellular image data 10 are generated using multiplexed or hyperplexed immunofluorescence imaging, but it will be understood that other imaging techniques, such as those described elsewhere herein, may also be used.

[0036] Next, in step 80, the microdomain identification component 35 identifies and locates one or more microdomains in each of the tumor sections of the multi-parameter cell and intracellular image data 10, as described herein, based on the quantification obtained in step 75. Then, in step 85, one or more of the identified microdomains are selected. The method then proceeds to step 90.

[0037] In step 90, the communication network component 45 constructs a communication graph for each of the selected microdomains. The communication graph may be displayed on the display 20. An exemplary communication graph 92 is shown in Figure 10. As shown in Figure 10, in the exemplary embodiment, each phenotype is a node 94 in the communication graph 92, and the edges 96 between each pair of phenotypes (each pair of nodes 94) in the communication graph 92 indicate the influence of one phenotype of the pair on the presence of the other phenotype of the pair. In the exemplary embodiment, each phenotype (each node 94) in the communication graph 92 is represented by a data vector obtained from the multiparameter cell and intracellular image data 10. The edges 96 of the communication graph 92 in the exemplary embodiment are also generated (after removing confounding effects of phenotypes other than the paired phenotypes) by determining the value of the linear or nonlinear correlation coefficient between the data vectors of the paired phenotypes to establish a numerical relationship between the paired phenotypes for each paired phenotype. In addition, for each of the numerical relationships, orientation is preferably established based on several receptor-ligand databases of relevant biomarkers. In this embodiment, the edges 96 between each pair of nodes 94 indicate the determined numerical relationship and the determined direction. In an exemplary embodiment, the edges 96 may be colored to represent a specific correlation coefficient value (e.g., the degree of positive or negative correlation).

[0038] In addition, in exemplary embodiments, each data vector of each node 94 is a vector of expression values ​​of several predetermined biomarkers, where these predetermined biomarkers are specific biomarkers used when generating multi-parameter cell and intracellular image data 10. For example, each vector of expression values ​​may be a biomarker intensity pattern of the predetermined biomarker determined in step 75. Furthermore, each data vector may be analyzed and biologically interpreted to determine several activation states 98 for each phenotype (each node 94), which are included in the communication graph 92. For example, the biomarker ALDH1 is a surrogate of stemness in tumor cells, and the biomarker PD-L1 determines the mutation burden.

[0039] Figure 11 is a schematic diagram of a specific communication graph generated based on a particular exemplary embodiment in which the phenotype (node ​​94) is tumor cell, lymphocyte, macrophage, stroma, and necrosis. The effects of various edges 96 and various activation states 98 are also shown.

[0040] One of the mapping outcomes described herein is the prediction of disease progression pathways based on the local tumor microenvironment (TME), followed by a list of known molecular targets along those pathways. Subsequently, machine learning tools (e.g., Balestra Web) can be used to predict drug target interactions based on spatial relationships. This could lead to the development of new therapies as well as the reuse of existing drugs. Furthermore, several exemplary clinical applications (e.g., applications of the CSPA platform 5) that can incorporate various aspects of the disclosed concepts are described in detail below, including: (1) drug discovery and personalized medicine strategies using spatial modulation computational system pathology; (2) cancer / disease landscapes, which are geometric representations of multiparametric readouts of a patient's cancer / disease state; and (3) applications of the CSPA platform for in vitro models for basic research and clinical translation.

[0041] In certain exemplary embodiments, the disclosed concepts can be used in drug discovery and personalized medicine strategies using spatially modulated computational system pathology. More specifically, spatial intratumoral heterogeneity is a key feature in determining the temporal evolution of cancer and patient fate. This heterogeneity is reflected in the diversity of heterogeneous intercellular communication networks embedded within microdomains, so the resulting systems biology is patient-dependent. Current therapeutic strategies are designed for the average patient and therefore do not reflect patient-specific systems biology. Patient-specific systems pathology begins with identifying microdomains / regions of interest, elucidating the characterization of their underlying communication networks, and accurately quantifying the interdependencies of microdomains both spatially and temporally. This knowledge is also essential for new drug design strategies. These strategies can also be enhanced by utilizing region-specific genomics.

[0042] Figure 12 is a schematic diagram of a personalized medicine strategy based on this aspect of the disclosed concept, ranging from tissue sections to personalized therapeutic procedures. From left to right, Figure 12 shows: i) multi-parameter imaging at cell / intracellular resolution, ii) identification of regions of interest (microdomains) within the image, iii) reconstruction of intra-microdomain communication networks using weights of specific connections and spatial relationships between cells / microdomains, iv) combining information to extract precise systems biology, and v) defining a personalized therapeutic strategy.

[0043] As shown in Figure 12, in the illustrated exemplary embodiment, it was found that microdomain 1 and microdomain 2 of the sample were phenotypically different for each individual patient. A phenotyping algorithm as described herein is applied to this tissue sample. Next, a heterogeneous intercellular network is constructed for each microdomain as described herein. Prognostic tests applied separately to microdomain 1 and microdomain 2 reveal that both microdomains have a low risk of relapse. However, when those microdomains are in close proximity, the risk of relapse increases dramatically, suggesting that information flows between microdomain 1 and microdomain 2. Thus, in this aspect of the disclosed concept, the heterogeneous intercellular network informs which pathways are activated and what relationship exists between the two networks. For example, WNT signaling is upregulated in microdomain 2 but downregulated in microdomain 1, and TGFβ is upregulated in microdomain 1 but downregulated in microdomain 2.

[0044] In this aspect of the disclosed concept, pathways and microdomain networks are identified, and therefore, the flow of information that must be suppressed to slow the progression of cancer in a particular individual is also further identified. This information can be used to develop personalized medicine strategies. In another specific exemplary embodiment, the disclosed concept can be used to accurately describe the temporal evolution of a disease, such as cancer, in a particular patient. Thus, the disclosed concept defines a cancer landscape, which is a geometric representation of a multiparametric readout of a patient's cancer state. Each point in the cancer landscape represents the patient's cancer state, such as precancerous, early stage, or invasive. The pathways followed in this landscape describe the temporal evolution of the disease in that particular patient. Current treatments use a cancer landscape averaged across a patient population. Unless the profile of an individual patient perfectly matches the average profile, predictions made with average models may be inaccurate.

[0045] Figure 13 shows an exemplary cancer landscape, including a geometric representation, where the location on the landscape represents a specific exemplary patient state. Figure 13 uses the progression of colorectal cancer as an example, with indentations labeled 1 representing precancerous states, 2 representing small polyp states, 3 representing large polyp states, and 4 representing invasive colorectal cancer states. Arrows within the geometric representation indicate the path a patient takes from precancerous to invasive cancer. This model allows us to infer the dynamics of cancer progression.

[0046] According to one embodiment of this disclosed concept, a retrospective patient cohort is required to construct a comprehensive library of the microdomains described herein regarding the evolution of tissue from precancerous to metastatic. Specific xenocellular communication networks, as described herein, are associated with each microdomain. In addition, each xenocellular communication network has a systems biology model in the form of a system of ordinary differential equations. This system of ordinary differential equations defines the kinetics of the cancer landscape. In prospective studies, these kinetics can help predict the temporal evolution of microdomains from precancerous states to metastatic. Furthermore, the kinetic model can serve as a substitute for a synthetic tissue development model.

[0047] In yet another specific exemplary embodiment, the disclosed concept may be used in combination with in vitro models for basic research and clinical translation. More specifically, in the two specific applications of the disclosed concept described above, the platform of the disclosed concept was applied to in-situ hyperplex single-cell resolution solid tumor images. In this embodiment, the same platform is applied to image data from in vitro microphysiological models. Multicellular in vitro models enable the study of spatiotemporal cellular heterogeneity and heterocellular communication that summarize human tissues, applicable to investigating the mechanisms of disease progression in vitro, testing drugs, and characterizing the structural composition and content of these models for use in transplantation.

[0048] In vitro models take the form of 2D cultures, 3D spheroids, organoids, and biomimetic microphysiological systems. The goal of these systems was to approach physiologically appropriate biomimetics. 2D cultures are cell cultures that grow by surface attachment. They develop a 2D communication network, so information exchange is limited to two dimensions. 3D spheroids are clusters of cells that grow spatially. The cells grow in an artificial environment, but they better mimic in vivo growth conditions because the cells can interact and grow in all directions. Organoids are very small but self-organizing three-dimensional tissue cultures. They are usually made from stem cells. Organoids can be constructed to reproduce most of the complexity of an organ, or they can be induced to represent selected aspects of an organ, such as the generation of only specific types of cells. Recreating organ function means recreating systems biology and xenocellular communication under controlled conditions in vitro, even if only partially. Biomimetic microphysiological systems represent the first step in mimicking organ function in relation to organoids. 3D microfluidic channel cell culture chips mimic all the activity, mechanics, and physiological responses of an entire organ. This allows for a more accurate representation of systems biology and intercellular communication networks.

[0049] In one or more aspects of this particular embodiment, spatiotemporal cellular heterogeneity and heterocellular communication may be monitored by constructing an image timeline and comparing the inferred network progression in the model. The level of complexity of in vitro systems, from 2D cultures to biomimetic MPS, reflects the complexity of the developable systems biology models. While applicable to all in vitro models, biomimetic models constructed by hierarchically or bioprinting various cell types in appropriate 3D tissues will benefit most from defining spatial relationships within the model. The same type of systems pathology defined in the two specific applications of the disclosed concept described earlier will also be valid here. The goal is to demonstrate that the in vitro model reflects the histological and spatial relationships identified in the in situ tissue / organ under consideration and that systems biology can be reproduced in vitro. The model may be established and studied in a “normal” healthy state, or it may be established as a disease model using cells from patients with disease and / or induced pluripotent stem cells (iPSCs) from patients that have been differentiated and matured to the cell type required for the model. At different time points, after selected treatments, the model is fixed, implanted, sectioned, and labeled, just like patient tissue. Hyperplex imaging is then applied to the labeled tissue sections. The tissue sections obtained from the model may then be subjected to computational system pathology analysis of the concepts disclosed herein.

[0050] Finally, while we have described image data obtained from tumor sections, it should be understood that this disclosed concept can also be applied to image data obtained from other types of tissue sections, as well as to image data obtained from unsectioned samples using imaging techniques that can penetrate solid unsectioned samples.

[0051] In the claims, symbols placed in parentheses should not be construed as limiting the claims. The words “equip” or “include” do not preclude the existence of elements or processes other than those described in the claims. In an apparatus claim listing several means, some of these means may be embodied by a single piece of hardware. The word “are” preceding an element does not preclude the existence of multiple such elements. In any apparatus claim listing several means, some of these means may be embodied by a single piece of hardware. The mere fact that elements are described in different dependent claims does not indicate that these elements cannot be used in combination.

[0052] While the present invention has been described in detail with regard to embodiments currently considered to be the most practical and preferred embodiments, it should be understood that such details are for that purpose only, and the present invention is not limited to the disclosed embodiments, but rather intended to include modifications and equivalent configurations that fall within the spirit and scope of the appended claims. For example, it should be understood that, to the extent possible, one or more features of any embodiment can be combined with one or more features of any other embodiment.

Claims

1. A method for representing the temporal evolution of a disease in a specific patient, wherein the specific patient is one of several patients, and the several patients are associated with multi-parameter cellular and intracellular image data obtained from several tissue samples from the several patients. A method comprising the step of generating a geometric representation of a disease landscape for a particular patient, wherein the geometric representation includes a plurality of points, each point of the geometric representation being (i) based on one particular tissue sample of several related to the particular patient that describes the disease status of the particular patient at a particular point in time, (ii) based on a microdomain in one particular tissue sample of several related tissue images that are based on multiparameter cells and intracellular image data, and (iii) including an intercellular communication network for the microdomain that includes a representation of intercellular communication based on spatial information for the microdomain.

2. The method according to claim 1, wherein each intercellular communication network is associated with a systems biology model that includes a system of ordinary differential equations.

3. The method according to claim 2, wherein, for each intercellular communication network, the system of ordinary differential equations defines the kinetics of the disease landscape for predicting the time evolution of the disease landscape for the particular patient.

4. The method according to claim 1, wherein each microdomain is identified by a step of generating a quantification of spatial heterogeneity between several different predetermined phenotypic cells in the multiparameterized cell and intracellular image data, and identifying the microdomain based on the quantification, the step of performing cell phenotyping on the multiparameterized cell and intracellular image data to identify the several different predetermined phenotypics.

5. The method according to claim 1, wherein each intercellular communication network comprises a communication graph of the microdomains, each phenotype is a node in the communication graph, and the edges between each pair of phenotypes in the communication graph indicate the influence of one phenotype in the pair on the presence of the other phenotype in the pair.

Images