System and method for spatial analysis of multiplex immunohistochemical tissue images

Abstract

A system and method for identifying differences in tissue architectures or cell types is provided. A user can utilize the methods described herein to perform multiplexed immunohistochemistry on a tissue sample to generate a multiplexed immunohistochemical image. The multiplexed immunohistochemical image can be used to identify a plurality of single marker channels each identifying a cell within the multiplexed immunohistochemical image. From the single channels a visual representation of the X and Y positional coordinated for each cell within the image is determined and a cell type is simultaneously assigned to each cell based on the intensity of the signal generated from fluorescently labeled antibody bound to a target. Based upon the cell type and the X and Y position of the cells a hexagonal heatmap is generated and subsequently output as an image that represents at least a cell density and a cell-to-cell association within the tissue sample. These hexagonal heatmaps are an easy visual representation for researchers to evaluate and enhance the understandings of tissue pathology.

Description

SYSTEM AND METHOD FOR SPATIAL ANALYSIS OF MULTIPLEX IMMUNOHISTOCHEMICAL TISSUE IMAGESCROSS-REFERENCE TO RELATED PATENT APPLICATIONS

[0001] The present application claims benefit of priority to U.S. Provisional Patent Application No. 63 / 604,806, filed November 30, 2023, which is incorporated by reference for all purposes.BACKGROUND

[0002] Tissue imaging analysis is a fundamental tool in pathology and biological research, enabling the identification of differences in tissue architectures and cell types compared to normal tissues. Recent advances in multiplex analysis have empowered researchers to visualize an array of biomarkers, offering a wealth of information about the tissue microenvironment. While simple imaging analysis in multiplex studies can provide valuable data on target cell density' and biomarker levels, it can sometimes yield false or misleading readings, especially when dealing with sparsely or densely distributed target cell populations within the tissue. For instance, when assessing immune cell activity- in the tumor microenvironment, measuring T cell density' is a common practice. However, retying solely on cell density’ readings may obscure critical information about the spatial distribution of T cells. They may be spread throughout the entire tumor area, concentrated at the tumor-stroma boundary, or densely distributed within the stroma with a gap between the tumor and stromal regions. Remarkably, all of these scenarios could yield the same T cell density if the cell count and tissue area are identical. Yet, the localization of T cells in these instances carries entirely different implications.

[0003] Embodiments address these and other problems, individually or collectively.SUMMARY

[0004] Embodiments herein provide for methods and systems for identifying differences in tissue architectures or cell types. The methods described herein are to perform multiplexed immunohistochemistry on a tissue sample to generate a multiplexed immunohistochemical image. The multiplexed immunohistochemical image can be used to identity’ a plurality- of single marker channels each identifying a cell within the multiplexed immunohistochemical image. From the single channels, a visual representation of the X and Y positional coordinated for each cell within the image is determined and a cell type is simultaneouslyassigned to each cell based on the intensity of the signal generated from fluorescently labeled antibody bound to a target. Based upon the cell type and the X and Y position of the cells, a hexagonal heatmap is generated and subsequently output as an image that represents at least a cell density and a cell-to-cell association within the tissue sample. These hexagonal heatmaps are visual representations that help to evaluate and enhance the understandings of tissue pathology. As such, embodiments provide semi-automatic visualization and measurement methods and systems for analyzing spatial relationships of single cell types and cell-cell associations using XY coordinates extracted from multiplex immunohistochemistry (mIHC) images of patient-derived cancer tissues.

[0005] Embodiments provide for a novel method for identifying tissue architectures or cell ty pes that may be semi-automated to generate cell density hexagonal heatmap (HHM) plots and cell-cell association HHM plots, complete with measurements, using cell segmentation data. Embodiments described herein compare the spatial relationships between two or more cell types within the tissue.

[0006] One embodiment is directed to a method for identifying differences in tissue architectures or cell types, the method comprising: performing multiplexed immunohistochemistry on a tissue sample to generate a multiplexed immunohistochemical image. Subsequently, the method includes identifying, from the mIHC, a plurality of single marker channels each identifying a cell within the multiplexed immunohistochemical image. The method includes the step of generating, from the plurality of single marker channels, a visual representation of X and Y positional coordinates for each cell within the multiplexed immunohistochemical image and assigning a cell type to each cell based on an intensity of a signal generated from a color, wherein the color is correlated with the single marker channels. And finally, generating, based on the cell type and the X and Y positional coordinates, a hexagonal heatmap of cell ty pes and outputting the hexagonal heatmap as an image, wherein the hexagonal heatmap represents at least a cell density and a cell-to-cell association within the tissue sample.

[0007] Another embodiment is directed to a computer system comprising one or more processor and one or more memory storing computer-readable instructions that, upon execution by the one or more processors, configure the computer system to perform the methods described herein.

[0008] Another embodiment is directed to a method for identifying differences in tissue architectures or cell types, the method comprising performing a cellular recognition technique on a tissue sample to generate an image. Subsequently, the method includes identifying, from the image, a plurality of single marker channels each identifying a cell within the image. The method includes the step of generating, from the plurality of single marker channels, a visual representation of X and Y positional coordinates for each cell within the image and assigning a cell type to each cell based on the single marker channels, wherein the multiplexed immunohistochemical image of the tissue sample includes at least two different cell types. The method includes determining cell-to-cell associations among a plurality of cells within the images based on the cell type and the X and Y positional coordinates. And finally, generating, based on the cell type and the X and Y positional coordinates, a hexagonal heatmap of cell types and outputting the hexagonal heatmap as an image, wherein the hexagonal heatmap represents at least a cell density and cell-to-cell associations within the tissue sample. In some embodiments, the cellular recognition technique may include spatial- transcriptomics, FISH, mass spectrometry, or any other imaging technique that may identify a single cell in a tissue sample.

[0009] These and other embodiments are described in further detail below.BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG. 1 shows two patterns for cell density measurements having the same cell density with different cell associations, according to various embodiments.

[0011] FIG. 2 shows a flow chart of the method for identifying differences in tissue architectures or cell types, according to various embodiments.

[0012] FIG. 3A shows an example panel of markers and associated fluorescent dyes used for performing multiplexed immunohistochemistry (mIHC), according to various embodiments.

[0013] FIG. 3B shows an example mIHC image collected using the panel described in FIG. 3A for collecting a signal unmixed mIHC image for one tissue microarray (TMA), according to various embodiments.

[0014] FIG. 3C shows example IHC images for individual markers (CD3, CD20, FOXP3, Ki67, PanCK, and HLADR) separated via filters to singly express each marker, according to various embodiments.

[0015] FIG. 3D shows an example cell dot plot with distributions of different types of cells classified from the mIHC image collected in FIG. 3B, according to various embodiments.

[0016] FIG. 3E shows an example cell dot plot of T-cells (TC: CD3+ / Ki67- and TCp: CD3+ / Ki67+) classified from the mIHC image collected in FIG. 3B, according to various embodiments.

[0017] FIG. 3F shows an example hexagonal heatmap plot identifying T-cell density at each hexagonal microlesion with heatmap indicator, according to various embodiments.

[0018] FIG. 4A shows example mIHC images of 4 different tumor microarrays of human lung tumor stained for markers from FIG. 3A, according to various embodiments.

[0019] FIG. 4B shows example expression of CD3 + cells (T-cells) on each TMA, according to various embodiments.

[0020] FIG. 4C shows example cell plots for T-cells on each TMA core from FIG. 4A, according to various embodiments.

[0021] FIG. 4D shows example hexagonal heatmap plots of T-cell’s (cells / hexagon) for each TMA core, according to various embodiments.

[0022] FIG. 4E shows example violin plots for the density of T-cells per Hexagon from FIG. 4D, according to various embodiments. (**** indicates a pO.OOOl for t-test.)

[0023] FIG. 4F shows an example histogram plot comparing the frequency distributions of cell densities at each hexagonal microlesion for each TMA core, according to various embodiments.

[0024] FIG. 5A shows example expression of CD20+ cells in each TMA core, according to various embodiments.

[0025] FIG. 5B shows example hexagonal heatmaps plots of the CD20+ expression in each TMA core, according to various embodiments.

[0026] FIG. 5C shows example expression of FOXP3 cells in each TMA core, according to various embodiments.

[0027] FIG. 5D shows example hexagonal heatmaps plots of the FOXP3 expression in each TMA core, according to various embodiments.

[0028] FIG. 5E shows example expression of CK cells in each TMA core, according to various embodiments.

[0029] FIG. 5F shows example hexagonal heatmaps plots of the CK cells in each TMA core, according to various embodiments.

[0030] FIG. 6 A shows an example of the T-cell density in a hexagonal heatmap from the TMA-a core representing a 1 : 1 ratio of the hexagonal microlesion, according to various embodiments.

[0031] FIG. 6B shows an example of the T-cell density in a hexagonal heatmap from the TMA-a core representing a 1 :2. 1 ratio of the hexagonal microlesion, according to various embodiments.

[0032] FIG. 6C shows an example of the T-cell density in a hexagonal heatmap from the TMA-a core representing a 1 :3. 1 ratio of the hexagonal microlesion, according to various embodiments.

[0033] FIG. 6D shows an example of the T-cell density in a hexagonal heatmap from the TMA-a core representing a 1 :3.4 ratio of the hexagonal microlesion, according to various embodiments.

[0034] FIG. 6E shows a representative violin plot of the hexagonal heatmap ratios indicating the cells / hexagon. according to various embodiments.

[0035] FIG. 6F shows a representative violin plot of the normalized hexagonal heatmap ratios indicating the cells / hexagon, according to various embodiments.

[0036] FIG. 7A shows an example hexagonal heatmap of the TMA-a core measured at 10 microns, according to various embodiments.

[0037] FIG. 7B shows an example hexagonal heatmap of the TMA-a core measured at 20 microns, as compared to various embodiments.

[0038] FIG. 7C shows an example hexagonal heatmap of the TMA-a core measured at 30 microns, as compared to various embodiments.

[0039] FIG. 7D shows an example hexagonal heatmap of the TMA-a core measured at 60 microns, as compared to various embodiments.

[0040] FIG. 7E shows an example plot of the associations / hexagonal at each representative measurement size, according to various embodiments.

[0041] FIG. 7F shows an example histogram plot comparing the frequency distributions of cell densities at each hexagonal microlesion measurement size for the TMA-a core, according to various embodiments.

[0042] FIG. 8 A shows an example TMA-a core separated by CK cells (a), T-cells (b), and TREG cells (c) including hexagonal heatmaps of Ck cells compared to T-cells (d) and T-cells to TREG cells (e), according to various embodiments.

[0043] FIG. 8B shows an example TMA-b core separated by CK cells (a), T-cells (b), and TREG cells (c) including hexagonal heatmaps of Ck cells compared to T-cells (d) and T-cells to TREG cells (e), according to various embodiments.

[0044] FIG. 8C shows an example TMA-c core separated by CK cells (a), T-cells (b), and TREG cells (c) including hexagonal heatmaps of Ck cells compared to T-cells (d) and T-cells to TREG cells (e), according to various embodiments.

[0045] FIG. 8D shows an example TMA-d core separated by CK cells (a), T-cells (b), and TREG cells (c) including hexagonal heatmaps of Ck cells compared to T-cells (d) and T-cells to TREG cells (e), according to various embodiments.

[0046] FIG. 8E shows an example violin plot of the association of CK cells to T cells of FIG. 8A-8D, according to various embodiments.

[0047] FIG. 8F shows an example violin plot of the association of T-cells cells to TREG cells of FIG. 8A-8D, according to various embodiments.

[0048] FIG. 9A show s an example of breast cancer tissue (tissue #1) stained with the multiplex panel for IHC imaging (a), the hexagonal heatmap of Ck cells associated with T- cells determined from (a), and T-cells associated with TREG cells determined from (a), according to various embodiments.

[0049] FIG. 9B shows an example of breast cancer tissue (tissue #2) stained with the multiplex panel for IHC imaging (a), the hexagonal heatmap of Ck cells associated with T- cells determined from (a), and T-cells associated with TREG cells determined from (a), according to various embodiments.

[0050] FIG. 9C shows an example of breast cancer tissue (tissue #3) stained with the multiplex panel for IHC imaging (a), the hexagonal heatmap of Ck cells associated with T- cells determined from (a), and T-cells associated with TREG cells determined from (a), according to various embodiments.

[0051] FIG. 9D shows representative violin plots of Ck+ cells associated with T-cells of the three separate breast cancer tissue samples, according to various embodiments. (““ indicates a pO.OOOl for t-test.)

[0052] FIG. 9E shows representative violin plots of T-cells associated with TREG cells of the three separate breast cancer tissue samples, according to various embodiments. (’**’ indicates a pO.OOOl for t-test.)

[0053] FIG. 10A shows example breast cancer tissue areas visualized according to the cell Epe classification based on transcriptomics markers, according to various embodiments.

[0054] FIG. 10B shows example Morisita-Hom index (MHI) square grid plots for tissue area 1 (a and c) and tissue area 2 (b and d) comparing the epithelial cells to epithelial cells (control group, a and b) and epithelial cells to cytotoxic T cells (CTL) (c and d), according to various embodiments.

[0055] FIG. 10C shows hexagonal grid plots of example Morisita-Hom index (MHI) square grid plots for tissue area 1 (a and c) and tissue area 2 (b and d) comparing the epithelial cells to epithelial cells (control group, a and b) and epithelial cells to cytotoxic T cells (CTL) (c and d), according to various embodiments.

[0056] FIG. 10D shows example spatial proximity score analysis (SPS) using hexagonal grids to visualize proximity-based cell associations for tissue area 1 (a and c) and tissue area 2 (b and d) comparing the epithelial cells to epithelial cells (control group, a and b) and epithelial cells to cytotoxic T cells (CTL) (c and d), according to various embodiments.

[0057] FIG. 11 illustrates an exemplary computer system, in accordance with various embodiments.DETAILED DESCRIPTION

[0058] Prior to discussing specific embodiments, some terms may be described in detail.

[0059] A ‘"memory” may refer to any suitable device or devices that may store electronic data. A suitable memory may comprise a non-transitory computer readable medium that stores instructions that can be executed by a processor to implement a desired method. Examples of memories may comprise one or more memory chips, disk drives, etc. Such memories may operate using any suitable electrical, optical, and / or magnetic mode of operation.

[0060] A “processor” may refer to any suitable data computation device or devices. A processor may comprise one or more microprocessors working together to accomplish a desired function. The processor may include a CPU that comprises at least one high-speed data processor adequate to execute program components for executing user and / or systemgenerated requests. The CPU may be a microprocessor such as AMD's Athlon, Duron and / or Opteron; IBM and / or Motorola's PowerPC; IBM's and Sony's Cell processor; Intel's Celeron, Itanium, Pentium, Xeon, and / or XScale; and / or the like processor(s).

[0061] A “region of interest (ROI)” or “regions of interest (ROIs)” may refer to a volume over which statistics can be calculated. As an example, an ROI may be a volume of tissue over which total or average biomarker activity can be calculated. An ROI, or ROIs, may be used to estimate or relate a statistic to a statistic corresponding to a larger volume. For example, a number of ROIs in different volumes of tissue may be used to estimate biomarker activity throughout the entire tissue.

[0062] The term “antibody” includes, but is not limited to, synthetic antibodies, monoclonal antibodies, recombinantly produced antibodies, multispecific antibodies (including bi-specific antibodies), human antibodies, humanized antibodies, chimeric antibodies, single-chain Fvs (scFv), Fab fragments, F(ab') fragments, disulfide-linked Fvs (sdFv) (including bi-specific sdFvs). and anti-idiotypic (anti-Id) antibodies, and epitope-binding fragments of any of the above. The antibodies provided herein may be monospecific, bispecific, trispecific or of greater multi-specificity. Multispecific antibodies may be specific for different epitopes of a polypeptide or may be specific for both a polypeptide as well as for a heterologous epitope, such as a heterologous polypeptide or solid support material.

[0063] A “cell density” may refer to calculation of the population of cells in a region of interest. Cell density shows vary little variation within a given cell type.

[0064] A ‘"tissue microarray (TMA)" may be formed of paraffin blocks in which up to 1000 separate tissue cores are assembled in array fashion to allow multiplex histological analysis. TMAs are commonly employed for multiplex immunohistochemical analysis. They may be used to analyze the expression of proteins simultaneously in multiple individual tissue samples on one slide.

[0065] The term “cell-to-cell association” or “cell associations” may refer cells that are in close approximation to one another. For example, cell associations may refer to cells that are within a defined space to one another. In some embodiments, the distance may be within 1 pm to the identified single cell. Cell-to-cell associations may be determined by measuring the distance between two cell types and may be presented as an average within a microlesion.

[0066] Embodiments relate to methods and systems for imaging analyses designed to visualize, measure, and compare microlesional cell densities and cell-cell associations within tissue regions of interest (ROIs). The methods described herein involve the utilization of a cellular identification imaging technique. For example, the cellular identification imaging technique can include microscopic images comprising multiplex immunohistochemistry (mIHC) images, fluorescence in-situ hybridization (FISH) assay, spatial transcriptomic images, or mass spectrometry imaging. In some embodiments, the mIHC images may be obtained from tissue microarray (TMA) cores (e.g., 1.3 mm-sized TMA cores) extracted from human lung cancer tissues. In some embodiments, the images may be obtained from spatial- transcriptomics data generated via a gene panel for identifying the cell types in a breast cancer sample. In some embodiments, the microscopic images are pre-classified to identify different cell types, providing the spatial coordinates of individual cells within the microscopic tissue images and their corresponding cell types. In some embodiments, the spatial coordinates of individual cells within the microscopic tissue images and their corresponding cell types are utilized in an algorithm (e.g., a Python-based script) to create hexagonal heatmap microlesions (HHMs), w hich visually represent the levels of cell densify across the tissue. The HHMs may be further evaluated to measure and compare microlesional cell densities and their distributions based on a plurality of (e.g., four) TMA cores.

[0067] Embodiments further determine cell-cell associations, specifically between tumor cells and T cells, as well as T cells and TREG cells, within the TMA cores. In some embodiments, the techniques described herein may be applied to relatively larger breast cancer tissue images (e.g., 2.8 x 2.8 mm and 2.8 x 4.2 mm) to assess cell-cell associationsbetween tumor cells and T cells, as well as between T cells and TREG cells. The methods and systems described herein affirm the capability of transforming classified microscopic tissue images into visual representations of microlesional cell densities and cell-cell associations, allowing for meaningful comparisons between different images. In some embodiments, the system described herein may determine a score for comparing a tissue area from a larger tissue sample. In some embodiments, the score may represent a cell-to-cell association, a number of cells presenting a similar cell type, or a signature determined by the user.

[0068] Various embodiments are discussed in connection with performing multiplexed immunohistochemistry with a panel of markers. For example, described herein are methods and systems for performing multiplexed immunohistochemistry on lung cancer tissue samples. It may be understood by one skilled in the art that the techniques described herein may be used on any panel of markers for a disease or condition including, but not limited to, other types of cancers.

[0069] In some embodiments, the techniques described herein may be used on any tissue biopsy collected from a human. For example, the tissue biopsy may be from any human body part including, but not limited to, lung, prostate, colon, breast, cervix, bladder, brain, stomach, esophagus, neck, kidney, liver, pancreas, testicle, skin, thyroid, or uterus. In some embodiments, the tissue biopsy may be from a person who has or is suspected of having cancer.

[0070] Embodiments provide systems and methods for identifying differences in tissue architectures or cell types. These techniques can be used on any number of biomarkers, tissue biopsy samples, and any disease or condition to accurately output a heatmap (e.g., hexagonal heatmap) for visualization, measurement, and comparison of the microlesional cell densities. In some embodiments, the tissue biomarker may be a protein. RNA, or DNA. In embodiments where the protein is identified, the method may include using multiplexed immunofluorescence as further described below. In embodiments where RNA is identified, the method may include using highly multiplexed gene panels as further described below.

[0071] FIG. 1 shows two patterns 100, 110 for cell density measurements having the same cell density with different cell associations, according to various embodiments. For example, FIG. 1 demonstrates the difference between patterns used for measuring cell densities in anROI 102. These various patterns may result in similar or same outcomes for cell density while simultaneously cell-cell associations determined from the two separate patterns would result in a different outcome. Specifically, pattern one 100 demonstrates that a close cell-to- cell association (within the dotted circle representing the ROI 102) has a cell density that is equivalent to pattern two 110 while the specific cell-to-cell associations are different in pattern two 1 10 than in pattern one. Thus, FIG. 1 provides a clear representation for understanding the necessity of the methods and systems described herein.

[0072] FIG. 2 shows a flow chart 200 of the method for identifying differences in tissue architectures or cell types, according to various embodiments. The method for identifying differences in tissue architectures or cell types includes a first step 202 of performing multiplexed immunohistochemistry (mIHC) on a tissue sample to generate a multiplexed immunochemical image. Steps and methods for performing mIHC may be known to those skilled in the art. For example, a tissue sample may be prepared from a patient having or suspected of having a disease wherein a biopsy is taken from the patient to obtain the tissue sample. The tissue sample may be initially prepared for mIHC imaging, such as, fixation, dehydration, embedding, and sectioning. In some embodiments, the tissue sample may be snap-frozen including the steps of submerging the tissue sample in liquid nitrogen followed by cutting using a cryostat to produce the tissue slices for imaging. Upon obtaining a microscopic image of the tissue sample, cell segmentation may be performed on the microscopic image to generate the mIHC image. The microscopic image comprises a plurality of signal intensities corresponding to a plurality of fluorescently labeled antibody targets.

[0073] It may be understood that the methods described herein are agnostic of the tissue samples and / or the biomarker panel used for detection. It may be understood that the tissue sample is isolated from a patient having or suspected of having a disease condition wherein a biopsy is taken to characterize or diagnose a disease. For example, a biopsy may include a lymph node or other solid organ biopsy for the detection of cancer. The antibody panel used for labeling the cells within the biopsy may be selected from any commercially available antibody. In some embodiments, the antibodies may be non-commercially available antibodies, such as ones synthesized by researchers for specific conditions and biomarkers. The first antibody may be specific to the target antigen. Additionally, the second antibody selectively binds to the first antibody. The second antibody is conjugated to the detecting modality. In some embodiments, the detecting modality is a fluorophore. In someembodiments, the antibody panel selected for mIHC comprises a plurality of detecting modalities having different detection wavelengths. In some embodiments, the method may include a panel of primer or probes designed to target specific sequences of interest. For example, a commercially available gene panel may be employed to target specific disease characteristics in a given tissue sample. The primers or probes may be labeled with a fluorescence tag for visualization. In some embodiments, the probe panel may be a gene panel for spatial transcriptomics analysis. In yet other embodiments, the panel may be designed for other imaging techniques including FISH assays. In yet other embodiments, the imaging data generated may not include an imaging modality such as matrix-assisted laser desorption / ionization (MALDI).

[0074] The tissue sample may be loaded into an imaging device, such as an imaging device or a multispectral imaging scope. In some embodiments, the digitized multiplex-image (e.g., the microscopic image) is analyzed for each individual cell on imaging platforms including but not limited to, inform, HALO, QuPath, or Xenium platforms. In some embodiments, prior to obtaining the mIHC image, cell segmentation is performed to separate out each cell identified in the tissue sample ROI. The resulting images may be further processed by methods described below to further characterize the mIHC image.

[0075] At block 204, the mIHC image may be utilized to identify a plurality of single marker channels each identifying a cell within the multiplexed immunohistochemical image. In some embodiments, the analysis platforms are capable of applying filters to an mIHC image to separate out each fluorescent channel. For example, the techniques involve analyzing only a specific wavelength that corresponds to a single target of interest from the plurality of biomarkers. The single channels data may be further manipulated to extract data that may be further interpreted to evaluate cell positions.

[0076] At block 206, the identified cells from the plurality of single marker channels are designated with an X and Y positional coordinate for each independent cell within the mIHC image and displayed via a visual representation. In some embodiments, the visual representation is a cell dot plot, wherein each dot is the identified cells. Additionally, the cell dot plot may have a color indicator to designate the cell of interest different from the plurality of cells in the image. For example, the cells identified as TREG cells may be depicted as a first color within the cell dot plot while all other cells are second color, thus providing a 2D representation of cell locations within an ROI. In some embodiments, the cell dot plot mayalternatively include multiple colors, for example, using the TREG cells above as a first color, T-cells may be designated as a second color while all other cells are a third color. This information may provide a quick analysis of the cells within the ROI. The information provided in the cell dot plot may be used to assign a cell type to each cell based upon the intensity of the signal generated from a color, wherein the color is correlated with the single marker channels.

[0077] At block 208, the method further includes assigning a cell type to each cell based on the single marker channels, wherein the multiplexed immunohistochemical image of the tissue sample includes at least two different cell types. From the single marker channels the cells are identified according to the wavelength of emitted light by the fluorescent tag conjugated to the antibody that is specific for a single target. For example, a plurality of markers may be used to target TREG cells, T-cells, or NK cells independently from one another. The single channel may correlate to one of the biomarkers of interest and be used to identity’ the cells within the mIHC image. For example, a fluorescence intensity’ associated with TREG cells may be depicted as a first color wherein the intensity of the color correlates to the number of TREG cells in the ROI. For example, a cell ty pe may be assigned to a cell based on an intensity’ of a signal generated from a color correlated with the single marker channel. In some embodiments, block 208 may be performed simultaneously with block 206. In some embodiments, block 206 may proceed block 208 (e.g., the cell type may be identified at the same time as the separation of the signal channels to generate the cell dot plot, or the cell may first be identified before a dot plot can be generated).

[0078] At block 210, the method further includes generating, based upon the X and Y positional coordinates, a hexagonal heatmap of cell types. In some embodiments, the hexagonal heatmap may be generated to show multiple characteristics and different cell architectures. For example, the hexagonal heatmap may be generated to depict cell density, cell identification, cell-to-cell associations, frequency distributions, or other cell features.

[0079] At block 212, the method may further include the step of determining cell-to-cell associations among a plurality of cells within the multiplex immunohistochemical image based on the cell type and the X and Y positional coordinates. In some embodiments, the cell ty pe may be identified by the multiplexed immunohistochemical analysis methods to identify singles cells. In a second step, the cells may be assigned positional X and Y coordinates. The X and Y coordinates may be used to determine the cells that are within a defined space andnear a second cell (e.g., cell-to-cell association). For example, a first cell being a CK cell may be in close proximity7to a T-cell. In such configurations, the method may include identifying the location of all CK cells in a single hexagon and further identifying a second cell type in the same hexagon. The hexagon may be assigned a color that may be generated based on the number of the first cells near, or associated with, the second cell. In some embodiments, the cell-to-cell association may be determined by measuring the distance between two cell types and presenting it as the average microlesional association within each hexagon.

[0080] At block 214, the method may further include the step of outputting the hexagonal heatmap as an image. The hexagonal heatmap represents at least a cell density and cell-to-cell associations within the tissue sample, i.e., the hexagonal heatmap enables determining cell- to-cell associations among a plurality of cells within the mIHC image based upon the cell type and the X and Y positional coordinates. The cell-to-cell associations may be displayed as a hexagonal heatmap. The hexagonal heatmap may further include values within each hexagon, each value corresponding to the number of cell associations within the hexagon. In some embodiments, the hexagon may be varied in size, by the user or based on a predetermined rule or condition, to evaluate different aspects of the tissue. For example, an embodiment for the cell-to-cell associations within one ROI may require a hexagon size of 30 pm while a second ROI may require a hexagon size of 60 pm. In some embodiments, the cell-to-cell associations may further include identifying a distance among the plurality of cells within the multiplex immunohistochemical image.

[0081] In some embodiments, the user may select a hexagon size of interest and an ROI size in a TMA and the system may output a value representing the length of time required to perform the methods described herein. For example, a user may change the distance measurement from one cell to another. For example, a user may want to evaluate an ROI for cell associations that are within 1 pm to 100 pm away from one another (e.g., a linear distance of 1 pm to 100 pm from a first cell to a second cell). Subsequent to the input distance, the number of cells of a second cell type within the predetermined distance from the first cell of the first type is calculated. The value may be incorporated into a value on a representation of the first cell on the hexagonal heatmap and also associated with a color. In some embodiments, changing both the hexagon size as well as the distance from one cell to another may impact the data represented on the heatmap. For example, decreasing the hexagon size may mean a reduced number of cells within a region and therefore reduced cell-to-cell associations within the region. However, by decreasing the hexagon size, the data may provide more insights on cell-to-cell associations on a single cell level thus providing data that may also be useful to the user. In some embodiments, the ROI may be varied (e.g., whether single cell associations or a TMA that is 1.3 mm x 1.3 mm in size is being analyzed). The ROI may further be limited by the imaging platform utilized by the user.

[0082] In some embodiments, the technique described above in FIG. 2 may be employed for identifying differences in tissue architectures or cell types, via methods other than mIHC. The method for identifying differences in tissue architecture or cell types includes a first step of performing a cellular recognition technique on a tissue sample to generate an image. Steps and methods for performing the cellular recognition technique may be known to those skilled in the art. For example, a tissue sample may be prepared from a patient having or suspected of having a disease wherein a biopsy is taken from the patient to obtain the tissue sample. The tissue sample may be initially prepared for the cellular recognition technique.

[0083] The tissue sample may be loaded into an imaging device, such as an imaging device or a multispectral imaging scope. In some embodiments, the digitized multiplex-image is analyzed for each individual cell on imaging platforms including but not limited to, inform, HALO, QuPath, or Xenium platforms. In some embodiments, prior to obtaining the image, cell segmentation may be performed to separate out each cell identified in the tissue sample ROI. The resulting images may be further processed by methods described below to further characterize the image.

[0084] The image may be utilized to identify a plurality of single marker channels each identifying a cell within the image. In some embodiments, the analysis platforms are capable of applying filters to an image to separate out each fluorescent channel. For example, the techniques involve analyzing only a specific wavelength that corresponds to a single target of interest from the plurality of biomarkers. The single channels data may be further manipulated to extract data that may be further interpreted to evaluate cell positions. In embodiments not including a tagging modality, the image may be generated based at least in part on the biomarker abundance in the sample.

[0085] The identified cells or biomarkers in the sample from the plurality of single marker channels may be designated with an X and Y positional coordinate for each independent cell within the image and displayed via a visual representation. In some embodiments, the visualrepresentation is a cell dot plot, wherein each dot is the identified cells. Additionally, the cell dot plot may have a color indicator to designate the cell of interest different from the plurality of cells in the image. For example, the cells identified as Epithelial cells (EP) may be depicted as a first color within the cell dot plot while all other cells are a second color, thus providing a 2D representation of cell locations within an ROI. In some embodiments, the cell dot plot may alternatively include multiple colors, for example, using the EP cells above as a first color, CTL cells may be designated as second color while all other cells are a third color. In some embodiments, each cell of interest may be designated an alternate color. For example, the image generated may include more than 10 colors. This information may provide a quick analysis of the cells within the ROI.

[0086] In some embodiments, the method may include additional steps for assigning a cell type to each cell based on the single marker channels. From the single marker channels the cells are identified according to the wavelength of emitted light by the fluorescent tag conjugated to the probes or primers in the gene panel that is specific for a single target. For example, a plurality of markers may be used to target TREG cells, T-cells, or NK cells, EP cells, MEP cells, hTC cells, MAST cells, pDC cells and any other cell in a sample tissue area independently from one another. The single channel may correlate to one of the biomarkers of interest and be used to identify the cells within the image. For example, a fluorescence intensity associated with EP cells may be depicted as a first color wherein the intensity of the color correlates to the number of EP cells in the ROI. For example, a cell type may be assigned to a cell based on an intensity of a signal generated from a color correlated with the single marker channel. In some embodiments, the above method may be performed simultaneously with other steps outlined in the method. In some embodiments, the method steps may be carried our in an alternate order as compared to the method depicted in FIG. 2 (e.g., the cell type may be identified at the same time as the separation of the signal channels to generate the cell dot plot, or the cell may first be identified before a dot plot can be generated).

[0087] The method may include an additional step of generating, based upon the X and Y positional coordinates, a hexagonal heatmap of cell types. In some embodiments, the hexagonal heatmap may be generated to show multiple characteristics and different cell architectures. For example, the hexagonal heatmap may be generated to depict cell density, cell identification, cell-to-cell associations, frequency distributions, or other cell features.

[0088] The method may further include the step of determining cell-to-cell associations among a plurality of cells within the image based upon the cell type and the X and Y positional coordinates and subsequently outputting the hexagonal heatmap as an image. The hexagonal heatmap represents at least a cell density and cell-to-cell associations within the tissue sample. The cell-to-cell associations may be displayed as a hexagonal heatmap. The hexagonal heatmap may further include values within each hexagon, each value corresponding to the number of cell associations within the hexagon. In some embodiments, the hexagon may be varied in size, by the user or based on a predetermined rule or condition, to evaluate different aspects of the tissue. For example, an embodiment for the cell-to-cell associations within one ROI may require a hexagon size of 30 pm while a second ROI may require a hexagon size of 60 pm. In some embodiments, the cell-to-cell associations may further include identifying a distance among the plurality of cells within the multiplex immunohistochemical image. In some embodiments, the determining the number of cells per grid unit size may include identifying the plurality of cells within the image, and subsequently determining, based upon a size of the grid, a total distribution of cells in a single hexagon. Additionally, the value may be incorporated onto the hexagonal heatmap, wherein the value is indicative of the number of cells in the single hexagon. For example, the hexagon may identify the number of cells of a relative type as compared to other cell types in a sample.

[0089] The method may may include scaling each hexagon within the hexagonal heatmap to generate a visual representation of the cell-to-cell association or cell density. For example, the scaling may include comparing the values of a plurality of hexagons to one another to produce a scale that may be interpreted as a visual representation onto each hexagon within the hexagonal heatmap. In some embodiments, the methods described herein may be carried out using Morisita-hom index or spatial proximity score analysis. In some embodiments, the user may determine the most suitable analysis method based at least on the size of the grid, the time required to perform the analysis, or the level of detail required for analysis.

[0090] In some embodiments, the user may select a hexagon size of interest and an ROI size in a TMA and the system may output a value representing the length of time required to perform the methods described herein. For example, a user may change the distance measurement from one cell to another. In some embodiments, a user may want to evaluate an ROI for cell associations that are within 1 pm to 100 pm away from one another (e.g., alinear distance of 1 jam to 100 jam from a first cell to a second cell). Subsequent to the input distance, the number of cells of a second cell type within the predetermined distance from the first cell of the first type is calculated. The value may be incorporated into a value on a representation of the first cell on the hexagonal heatmap and also associated with a color. In some embodiments, changing both the hexagon size as well as the distance from one cell to another may impact the data represented on the heatmap. For example, decreasing the hexagon size may mean a reduced number of cells within a region and therefore reduced cell- to-cell associations within the region. However, by decreasing the hexagon size, the data may provide more insights on cell-to-cell associations on a single cell level thus providing data that may also be useful to the user. In some embodiments, the ROI may be varied (e.g., whether single cell associations or a TMA that is 1.3 mm x 1.3 mm in size is being analyzed). The ROI may further be limited by the imaging platform utilized by the user. In some embodiments, the ROI may be an average of a given cell number in the ROI. For example, an ROI includes a higher number of EP cells as compared to CTL cells, the ROI may be assigned a value correlating to the EP cells (e.g., the higher proportion of cells is given the ROI designation). The method may be performed in a whole area analysis wherein the hexagonal ROI may not be showing single cell-to-cell association and may be the average cell-to-cell association for the given ROI. For example, if there is more cell-to-cell association for EP to CTL cells as compared to EP to MAST cells, the ROI may be designated with a score corresponding to EP to CTL cells. IN some embodiments, if the ROI is small, the ROW may indicate individual cell-to-cell associations. One skilled in the art may understand the parameters of the method described herein may be altered to view single cell associations to total cell area associations.

[0091] Various multiplex analysis techniques have been developed to comprehensively extract information from tissue samples, especially those derived from patients. These analyses typically involve identifying cell types, assessing gene or protein expression levels, and measuring cell density within the tissue. However, the challenge lies in the heterogeneity of the tissue microenvironment within diseased or defective areas, which complicates tissue comparisons. Additionally, variations in tissue architecture, size, and structure further hinder accurate comparisons betw een different tissues. Therefore, it is important to understand the tissue architecture at the target tissue lesion and assign it as the complex of microlesions to acquire parameters of microlesional cell density and cell-cell associations. This allowscomparison of tissues not by cell density of a certain tissue size but by the microlesional parameters even if the tissue sizes are different.

[0092] The imaging analysis described below was conducted using Python code, leveraging cell segmentation data obtained from multiplex immunohistochemistry (IHC) results. The tissue samples are pre-classified for cell types and their respective locations, determined through imaging platforms. FIG. 3A shows an example panel of markers and associated fluorescent dyes used for performing multiplexed immunohistochemistry (mIHC), according to various embodiments. As shown in FIG. 3A, an immune multiplex panel 300 is used to visualize specific cell types, including T cells (CD3+ cells). B cells (CD20+ cells), regulatory T cells (TREG cells; FOXP3+ / CD3+ cells), tumor cells (panCK+ cells), and other marker-negative cells, within patient-derived cancerous tissues. The techniques described herein are used to visualize and compare microlesional cell densities and cell-cell associations, particularly those involving tumor cells with T cells, T cells with TREG cells (as demonstrated below), and tumor cells with B cells. To demonstrate the effectiveness of this methodology, FIG. 3B illustrates a first multiplexed immunohistochemical analysis performed on the sample to generate a first mIHC image 302 based on the immune multiplex panel 300. As shown in FIG. 3C, the mIHC may be separated into single marker channels for CD3, CD20, FOXP3, Ki67, PanCK, and HLADR (304, 306, 308, 310, 312, and 314 respectively). However, the methodology can be applied to any imaging platform capable of generating datasets with cell types (or marker positive / negative designations) and X and Y coordinates acquired from scanned images. The output data for microlesion cell density or cell-cell associations between tw o cell types are represented as hexagonal heatmap microlesions, allowing for visualization and comparisons within tissues or between multiple tissues. Additionally, the recorded parameters measured for each microlesion in each tissue can be subjected to various statistical comparisons, such as violin plots for simple measurement comparisons and histograms for frequency -based comparisons.

[0093] Visualizing and comparing the tissue distributions of a single cell ty pe is straightforward when using single-plex images or when dealing with a single cell type in multiplex analysis. However, the complexity arises when attempting to compare the spatial relationships between two or more cell types within the tissue. While it is possible to compare the cell densities of at least tw o cell types, this approach does not reveal whether these cell ty pes are spatially associated with each other. The techniques described herein provide theuser with the potential cell-cell associations by measuring the distance between two cell types and presenting it as the average microlesional association. A measurement range of 30 microns was used across the studies described in FIGs. 3A-10D, which corresponds to approximately 3-5 cells from the center cell. To evaluate the effect on distance variations, a range from 10 microns (1-2 cells range) to 60 microns (about a 10-cell range) was selected. The results indicate that the distribution of hexagonal heatmap microlesions remains consistent across these different range settings, but the measurement values for each microlesion increase with a wider range. Since immune cells are known to actively survey the tissue, these range settings can serve as a predictive tool for immune cell-related cell-cell associations. Notably, shorter-range analyses provide a higher likelihood of capturing these associations. In cases involving more static cell types, (e g., epithelial cells), it may be necessary' to use a minimum range parameter, such as 10 microns, to accurately assess cellcell associations.

[0094] Embodiments use an exemplary algorithm (e.g., Python-based tool) that automatically' generates cell density hexagonal heatmap (HHM) plots and cell-cell association HHM plots, complete with measurements, using cell segmentation data (for example stored in a data file, such as an Excel file). This data file includes details on individual cells within the tissue, such as tissue name, cell type, and XY coordinates. In a test involving 73 TMA cores, the algorithm can automatically measure cell-cell associations for CK+ cells with T cells in approximately 3-5 minutes per TMA core, depending on the number of CK+ cells and T cells. Moreover, this imaging analysis tool may be used to analyze datasets from different types of tissue images, such as chromogenic IHC images, multiplex IHC images, spatial-transcriptomics data, mass spectroscopy imaging data, fluorescence in-situ hybridization (FISH) assay data, spatial transcriptomic data, enabling the detection of vanous parameters for a wide range of research applications. While the figures illustrate application of the techniques on human cells, but any biological object with X- and Y- coordinates can be analyzed using techniques described herein (e g. the positions of RNA expression from transcriptomics data).

[0095] Embodiments provide a comparative analysis of cell-cell associations, focusing on interactions between CK+ cells and T cells, as well as between T cells and TREG cells. T cells' activity in attacking CK+ cells or being suppressed by TREG cells within breast cancer tissue can be evaluated using the hexagonal heatmap representing at least a cell density andcell-to-cell associations within the tissue sample. By comparing the original multiplex IHC image with cell-cell association hexagonal heatmap (HHM) images, embodiments allow for identification of different patterns of T cell activity, categorization of tissues as immune hot, immune cold, or immune desert. Embodiments provide a powerful asset for comprehensive visualization and analysis of tissues with microlesions, enabling a deeper understanding of their characteristics. Additionally, embodiments facilitate efficient comparisons between multiple tissue images simultaneously, enhancing its utility in research and diagnostics.Transforming multiplex images into digitized locational map

[0096] As shown in FIG. 3B, exemplary lung cancer TMA cores were stained with a multiplex IHC panel 300 for visualizing human markers for identification of various cancer cell types using a multispectral imaging scope (e.g., Polaris). . The exemplary markers illustrated in FIGs. 3A-3F include CD3, CD20, FOXP3, Ki67, Cytokeratins and HLA-DR. This kind of digitized multiplex-image is usually analyzed for each individual cell on an imaging platform (e.g., inForm, HALO, QuPath imaging platforms) to identify cell classification data, including categorized cell types based on stained markers, cell counts, cell densities, and the XY coordinates of individual cells. For example, the single-plex IHC images for each human marker 304, 306, 308, 310, 312. 314 are shown in FIG. 3C. An algorithm is used to visualize each cell’s location as a dot plot 316 for multiple cell types as shown in FIG. 3D or as a dot plot 318 for single cell type (e.g., T cells) as shown in FIG. 3E. However, the plot alone as shown in FIG. 3D or FIG. 3E does not provide an interpretation for the local density of the cell type. Therefore, embodiments include converting the dot plots to an hexagonal heatmap (HHM) image 320 as shown in FIG. 3F to visualize the local density measurement as hexagonal heatmap microlesions (HHMs) for T cells’ distribution. The HHM image 320 provides the cell density’ information 322 on each hexagon which can be used to compare tissues. The HHM image 320 may be used to identify' a certain cell type distribution in the tissue.Comparing T cell densities in different TMA cores

[0097] To assess the suitability of HHM for comparing cell densities across distinct tissue samples, T cells (CD3+ cells, FIG. 4B) in four TMA cores 400, 402, 404, and 406 as shown in Figure 4A were examined. T-cells are indicated in the images as a white dot (CD3) and DAPI stained DNA is indicated as the grey sections w ithin the cores (400-414). Dot plots(416. 418, 420. and 422) were utilized to visualize T cells, labeled as, TC: CD3+ / K167- and TCp: CD3+ / Ki67+ from each of the four core samples. Evaluating the TMA cores for CD3+ biomarker demonstrated that tissue TMA-a (400) exhibits notably higher T cell density, whereas TMA-d (406) showcases comparatively lower T cell density within each microlesion. To better visualize the profile differences of the TMA cores in FIG. 4B. the TMA cores were converted to cell dot plots (416, 418, 420, and 422) (FIG. 4C). Additionally, as can be observed in FIG. 4C, TMA-b (418) core exhibited a notably higher cell density' localized in the upper left quadrant as compared to the relatively equal distribution of cells in TMA-a (416). HHMs were generated to visualize the T cell density observed (424. 426, 428, and 430). Additionally included in each respective hexagon is a value associated with the density distribution measured in each respective hexagon of FIG. 4D. These microlesions were subject to comparison using the T cell density dataset obtained from each HEX. A violin plot (432) (FIG. 4E) contrasting an average T cell density within the HEX highlighted a significantly lower cell density in microlesions of TMA-d (406) and a markedly higher cell density in TMA-a (400). Within these TMAs, TMA-a (400), -b (402), and -c (404) revealed a broad spectrum of T cell densities across microlesions. As shown in FIG. 4F, the histogram visualization (434) illustrated that TMA-a predominantly features HEXs with elevated T cell density (T cells / HEX). whereas TMA-d is characterized by a greater proportion of HEXs with lower T cell density. To rigorously compare these histogram datasets, the Kolmogorov- Smirnov Test and / or Mann-Whitney U Test were used. Consequently, the data classifying cells, generated through our imaging platform, can be effectively assessed, visually- represented, and statistically analyzed using HHM and its associated scores.Visualizing other cell types

[0098] To assess the ability for the methods described herein to be applied to any cell biomarkers, the TMAs were evaluated for the expression of CD20, FOXP3, and CK. For example, to assess the TMA cores for the expression of cells other than T-cells, CD20 (B-cell marker; FIG. 5A (500, 502, 504, and 506)), FOXP3 (TREG; Regulatory- T cell marker; FIG. 5C (516, 518, 520, and 522)) and Pan-CK (CK; epithelial / tumor cell marker; FIG. 5E (532, 534, 536, and 538)) biomarkers were imaged. Specifically, for CD20, the four TMA cores w ere stained according the mIHC panel above, and the single channel was isolated for CD20 (500, 502, 504, and 506). The single channels were converted into HHMs to identify the location of the single marker in the TMA (508, 510. 512, and 514). Classified cell locationswere interpreted into hexagonal heatmaps for each other respective marker FOXP3 (FIG. 5D (524, 526, 528, 530)), and CK (FIG. 5F (540, 542, 544, and 546)). The results demonstrate the differential expression of each marker in the respective TMA cores. For example, based upon the derived HHM, TMA-a (508) shows relatively high expression of CD20 as compared to TMA-d (514). When addressing other markers such as FOXP3 (FIGS. 5C-5D), the TMA cores (516, 518, 520, and 522) have a drastically different distribution of FOXP3 expressing cells as visualized in the HHMs (524, 526, 528, and 530). Similarly, the same observations may be made with respect to Ck cells (FIGS. 5E-5F). For example, TMA cores (534, 536, and 538). when converted to HHMs (542, 544, and 546) demonstrate a similar distribution of epithelial / tumor marker CK. while TMA-a (532 and its corresponding HHM (540) demonstrates a more concentrated distribution of CK marker in the upper half of the core. The hexagonal heatmaps clearly demonstrate the relatively higher expression CK in all TMA cores as compared to the expression of CD20. The results demonstrate that the methods described herein are capable of visualizing other cell types other than only T-cells.Size of hexagonal microlesions

[0099] The parameters used in Figure 3 were based on the default settings, which were specifically suited for the analysis of four TMA cores. How ever, it's essential to acknowdedge that tissue sizes can differ significantly, both in terms of dimensions and cell densities. T cell density in TMA-a (Figure 8A(e) 808) was used to compare different sizes of hexagonal microlesion. Microlesions were evaluated according to the ratios 1 : 1 (default setting) 600, 1 :2.1 (602), 1 :3.1 (604) and 1:3.4 (606) (FIGs. 6A-6D) to compare the visualization and the associated measurements. For example, in hexagon a (600) having a 1: 1 ratio the corresponding hexagons all relatively stay below a value of 25, while converting that ratio to 1 :3.4 (606), the value in some of the hexagons peaked to as high as 90. This implies that having a larger ratio (1 :3.4 (606) may provide an overall picture of a distribution on a large scale, if finer detail is required a narrow er ratio may be beneficial. However, as show n in FIG. 6E, HHMs showed a similar cell density pattern (610), even though their cell count (608) per hexagon is different, likely because of the differences in the hexagon size. Thus, the size of the hexagonal microlesion can be modified depending on tissue size or visual presentation purpose and supply similar cell density information. Alternatively, FIGs. 7A-7D illustrate that comparing different ranges of hexagonal sizes (10 (700), 20 (702), 30 (704) and 60 (706) microns)) show ed similar positive HEX distributions with different associations / HEX (708)(FIG. 7E). For example, the larger the size of the Hexagonal heatmap map. the larger the associations / Hex were observed. Evaluating the histogram plot (710) of the different sized hexagons further demonstrated the variability in the frequency of the cells / hex as the size of the hexagon increased. Briefly, at 10 microns (700). the number of cells / hex was predominantly zero, while at 60 microns (706), the number of cells / hex was spread from 10 to 20 cells / hex (FIG. 7F). The results further demonstrate the parameters of the method described herein may be changed according to the data to be interpreted.The analysis for spatial cell-cell associations

[0100] It is important to know if different types of cells are associating in the microlesion especially for understanding immune cell activities in tumor microenvironment. HHMs can also be used for visualizing cell-cell associations. Initial tests were conducted to evaluate associations between tumor cells (CK+ cells) and T cells (CD3+ cells) (Figures 8A-D(d)) and between T cells and TREG cells (CD3+ / FOXP3 cells) (Figures 8A-D(e)) in same TMA cores used in Figure 3. To predict the cell-cell associations, the cell counts from one cell type to the other cell type within 30 microns were measured, and then the average of the cell counts per each hexagonal microlesion was visualized as HHM. Although the range for the analysis was set as 30 microns , it may be understood by one of ordinary skill in the art that the range could also be modified. FIGs. 8A-D(a-c) (800, 802, 804, 810, 812, 814, 820, 822, 824, 830, 832, and 834) show each marker positive (CK, T cells, TREG) cell’s dot plot in each of the TMA cores. Cell-to-cell associations, based on inputs in the algorithm to generate HHMs, were determined for both CK vs T cells (806, 816, 826, and 836) and T cells vs TREG (808, 818, 828, and 838) (FIGs. 8A-D(d-e)). By comparing these visualizations, cell-to-cell associations in each microlesion can be easily captured. In these comparisons, HHM (CK vs T cells: CK<>T cells, 806. 816, 826, and 836) can be used to predict the T cell attack on tumor cells, and HHM (T cells vs TREG: T cells<>TREG, 808, 818, 828, and 838) can be used for visualizing the suppression status of T cells in each microlesion. Comparing the measurements of association / HEX (840) for each TMA core shows TMA-a has significantly higher associations per hexagon, and TMA-d has the lowest association between CK+ cells and T cells (FIG. 8E). Interestingly, the associations between T cells and TREG (842) is higher in TMA-a and -c and lower in TMA-b and -d (FIG. 8F). Thus, these parameters can be measured and supplied for hexagonal microlesions which can be used for comparing cell-cell associations at each microlesion.Analysis for the larger tissue area

[0101] As provided above, in embodiments where proteins are identified, the method may include using highly multiplexed immunohistochemical analysis. This spatial-imaging analysis is not limited to the small tissue area like TMA (about 1.3mm diameter), larger tissues can also be analyzed. FIGs. 9A-C illustrate three breast cancer tissues stained for same mIHC panel 300 as previously described in FIG. 3A. Two tissues (Tissue #1 (900) and #2 (906)) were 2.8 x 2.8mm, and one tissue (Tissue #3 (912)) was 2.8 x 4.2mm. mIHC images for each tissue sample (900, 906, and 912) were generated and were subsequently classified as cell segmentation result recording cell types (CK+, T cells, B cells, TREG and others) and compared for cell-cell association HHM (CK vs T cells (902, 908, and 914) and T cells vs TREG (904, 910, and 916)). As shown in FIG. 9Da, b, the HHM (CK vs T cells (902, 908, and 914)) measurements show that Tissue #1 900 and #3 912 have significantly higher cellcell associations between CK+ cells and T cells compared with Tissue #2 906. Although Tissue #1 900 and #3 912 showed similar HHM scores, Tissue #1 900 has more T cell infiltration in DC1S lesions, and Tissue #2 906 has less infiltration. Similarly, as shown in FIG. 9Ea, b, the HHM measurements comparing cell-cell associations between T cells and TREG cells (904, 910, and 916) reveal significantly higher levels in Tissue #1 900 and #3 912, and lower levels in Tissue #2 906. This indicates that T cells in Tissue #1 900 and #3 912 may be suppressed by TREGs. However, the Tissue #1 900 has much less T cell-TREG cell associations where cell-cell associations are high for CK+ cells with T cells in DCIS, suggesting that T cells inside of DCIS lesion may be attacking DCIS cells in Tissue #1 900. On the other hand, Tissue #2 906 has some DCIS lesions with almost zero infiltration of T cell but has other lesions with high cell-cell associations between CK+ cells and T cells where cell-cell associations for T cells with TREG cells are also high, suggesting that this tissue may have a heterogeneity in T cell activity against CK+ cells. Thus, visualizing various cell activities with HHM plots allows to locate the differences easily by visual impressions and measured microlesional parameters, but it is still important to compare the measurement results with the actual tissue image.Analysis for large tissue areas using special transcriptomics

[0102] As provided above, in embodiments where RNA is identified, the method may include using highly multiplexed gene panels. The spatial imaging analysis is not limited toonly mIHC panels as demonstrated above. FIGS. 10A-10D illustrate different techniques to visualize proximity-based cell associations for tissue areas in breast cancer tissue.

[0103] According to various embodiments, the heatmap described above may be generated as a grid heatmap using any size / shape of grid elements. For example, a hexagon heatmap may be generated using hexagon grid elements, or a square heatmap may be generated using square grid elements.

[0104] FIG. 10A illustrates the breast cancer tissue post transcriptomics analysis and further identifying the areas (Area 1, 1002 and Area 2, 1004) utilized in experiments below. FIGS. 10B-10C illustrate an exemplary technique based on Morisita-Hom index (MHI) using hexagonal grids to visualize proximity -based cell associations for tissue areas in breast cancer tissue. In an initial step, spatial transcriptomics was performed on a breast cancer tissue sample. The transcriptomics analysis was performed on a 1 Ox Xenium platform with a 280- gene panel, and cel types were identified through cell-type-specific markers. FIG. 10A illustrates a plot map according to the transcriptomics analysis. The plot map shows the classified cell types within the tissue, with each cell visualized according to its cell type classification based on the transcriptomic markers. Key cell types shown are epithelial / cancer cells (EP), myoepithelial cells (MEP), helper T cells (hTC), cytotoxic T cells (CTL), macrophages (Mac), conventional dendritic cells (eDC), B cells (BC), regulatory' T cells (TREG), mast cells (MAST), natural killer cells (NK), and others. Two regions, denoted as Area 1 1002 and Area 2 1004. were selected for comparative analysis of cell associations shown in FIGS. 10B-10D.

[0105] FIG. 10B illustrates Morisita-Hom index (MHI) for cell associations within annotated areas being visualized via square grids (1006, 1008, 1010, 1012). FIG. 10C illustrates Morisita-Hom index (MHI) for cell associations within annotated areas being visualized via hexagonal grids (1014, 1016, 1018. and 1020). In embodiments shown in FIGs. 10B and 10C, the MHI was computed for two cell-type comparisons: EP vs EP (1006, 1008, 1014, and 1016) (control group, FIGS. lOBa-b and lOCa-b); and EP vs CTL (1010, 1012, 1018, and 1020) (FIGS. lOBc-d and lOCc-d). Local similarity' between cell ty pes, indicating the degree of co-occurrence between cell types, is represented by a color gradient within each grid. The MHI scores for each image are displayed besides the images. Square grids (e.g., exemplary side length 60 pm, and exemplary area 3600 pm2) and HEX grids (e.g., exemplary side length 37 pm , and exemplary area 3557 pm2) were used to ensure comparability acrossgrid types. One of ordinary skill in the art will appreciate that the grid shapes and sizes are exemplary, and provided for illustrative purposes. Other grid shapes and sizes may be used to perform various embodiments. FIGs. 10A-10C show similar MHI scores between the two grid configurations for EP vs CTL in both Area 1 1002 and Area 2 1004. Specifically, the MHI area for the EP vs CTL group of FIG. lOBc-d is 0.286 for Area 1 1002 and 0.034 for Area 2 1004 and the hexagonal grids of FIG. lOCc-d is 0.310 for Area 1 1002 and 0.048 for Area 2 1004. The results of the MHI scoring highlight the fault that may be present when using the MHI for performing cell-to-cell associations on a larger scale such as grids measuring 60 pm. Specifically, MHI may be used for determining cell-to-cell associations when a grid size is smaller, such as 10 pm, however the computational load may be to high and thus alternative options may be required to accurately assess the cellular environment. The results also highlight that changing the grid boundaries (e.g., from square to hexagonal) alone does not impact the values represented in the area segmented for analysis.

[0106] FIG. 10D illustrates spatial proximity score analysis (SPS) using hexagonal grids to visualize proximity-based cell associations for tissue areas in breast cancer tissue. SPS analysis is performed on the hexagonal grids to visualize proximity -based cell associations for both Area 1 (1002) and Area 2 (1004). The SPS is calculated between each epithelial cell (EP) (1022 and 1024) and neighboring EP or CTL cells (1026 and 1028) within a predetermined distance (e.g., 30 microns). Each grid’s SPS scores can be displayed as HEX plot with metrics: “positive grids,” “grid average,” and “total average.” The positive grids are the number of grids with positive SPS scores. The grid average denotes the average SPS within grids containing positive scores. The total average denotes the overall average SPS for individual cells. FIG. 10D illustrates that the EP vs CTL cells resulted in 89 positive grids for Area 1 1002 with a grid average of 1.73 and a total average of 0.70. While for Area 2 1004, there were 10 positive grids with a grid average of 0.06 and a total average of 0.05. The positive grid areas provide an accurate assessment of the cell-to-cell associations of the target cell types. In FIG. 10D (c) for example, the 89 positive grids identify the locations in the tissue section where there is a strong presence of CTL cells to EP cells. Alternatively, for Area 2 1004 of FIG. 10D (d), there were 10 positive grids identified via the methods described above. In the parameters used for assessment, the grid average refers to the average of the associations between two cell types (center cells and target cells) within each grid, calculated within a predetermined proximity (e.g., the proximity is adjustable). As usedherein, the total average represents the overall average of associations between the two cell types across all grids (from center cells to target cells). While these metrics individually may not fully identify the trends of the entire tissue, combining them with visual information or the positive grids parameter shown in FIG. 10D provides insights into whether the associations are localized or reflect a broader trend within the tissue. The results demonstrate the capabilities of using the methods described above for not only mIHC imaging but also spatial transcriptomics. The methods described herein may be used for any number of visualization techniques, including but not limited to, FISH assays, mIHC, spatial transcriptomics, and mass spectrometry imaging.

[0107] A portion of the subsystems or components of an exemplary computer apparatus are shown in FIG. 11. The subsystems shown in FIG. 11 are interconnected via a system bus 1105. The subsystems such as a printer 1104, keyboard 1105, fixed disk 1109 (or other memory comprising computer-readable media), monitor 1106, which is coupled to a display adapter 1111, and others are shown. Peripherals and input / output (I / O) devices, which couple to I / O controller 1101. can be connected to the computer system by any number of means known in the art, such as serial port 1107. For example, serial port 1107 or external interface 1110 can be used to connect the computer apparatus to a wide area netw ork such as the Internet, a mouse input device, or a scanner. The interconnection via system bus allows the central processor 1103 to communicate with each subsystem and to control the execution of instructions from system memory 1102 or the fixed disk 1109, as well as the exchange of information between subsystems. The system memory 1102 and / or the fixed disk 1109 may embody computer-readable medium.

[0108] The methods described herein may be carried out by a computing device for identifying differences in tissue architectures or cell types. Any of the computer systems mentioned herein may utilize any suitable number of subsystems interconnected via a system bus 1105. In some embodiments, a computer system includes a single computer apparatus, where the subsystems can be components of the computer apparatus. In other embodiments, a computer system can include multiple computer apparatuses, each being a subsystem, with internal components. For example, the computing device may have a central processor 1103, a system bus 1105, a system memory 1102, and a monitor 1106. The computing device may be communicatively coupled to the imaging platform to provide semi-automatic mIHC. For example, the user may input the necessary hexagon sizes, the distance from one cell toanother, and load the prepared slides onto the imaging platform. Subsequent to loading the slide and inputting values, the user may be able to allow the central processor 1103 to execute one or more operations described herein for determining a cell architecture or cell type. Is it also possible for the components to be distributed in a cloud computing system or grid computing system.

[0109] A computer system can include a plurality of the components or subsystems, e.g., connected together by external interface or by an internal interface. In some embodiments, computer systems, subsystems, or apparatuses can communicate over a network. In such instances, one computer can be considered a client and another computer a server, where each can be part of a same computer system. A client and a server can each include multiple systems, subsystems, or components.

[0110] It should be understood that any of the embodiments can be implemented in the form of control logic using hardware (e.g., an application specific integrated circuit or field programmable gate array) and / or using computer software with a generally programmable processor in a modular or integrated manner. As used herein a central processor 1103 includes a single-core processor, multi-core processor on a same integrated chip, or multiple processing units on a single circuit board or networked. Based on the disclosure and teachings provided herein, a person of ordinary7skill in the art will know and appreciate other ways and / or methods to implement embodiments using hardware and a combination of hardware and software. The central processor 1103 can execute one or more operations for identifying differences in tissue architecture or cell types. The central processor 1103 can include one processing device or multiple processing devices. Non-limiting examples of the processor include field-programmable gate array (“FPGA”), an application-specific integrated circuit (“ASIC”), a processor, a microprocessor, etc.

[0111] The processor is communicatively coupled to the system memory 1 102 via the system bus 1105. The system memory 1 102 may include any type of memory device that retains stored information when powered off. Non-limiting examples of the system memory71102 include electrically erasable and programmable read-only memory7(“EEPROM”), flash memory, or any other type of non-volatile memory. In some examples, at least some of the system memory 1102 can include a non-transitory7computer-readable medium from which the processor can read instructions. A computer-readable medium can include electronic, optical, magnetic, or other storage devices capable of providing the processor with computer-readable instructions or other program code. Non-limiting examples of a computer-readable medium include (but are not limited to) magnetic disk(s), memory chip(s), read-only memory (ROM), random-access menior.' (“RAM”), an ASIC, a configured processing device, optical storage, or any other medium from which a computer processing device can read instructions. The instructions can include processing device-specific instructions generated by a compiler or an interpreter from code written in any suitable computer-programming language, including, for example, Java, C, C++, C#, Objective-C, Swift, or scripting language such as Perl or Python using, for example, conventional or object-oriented techniques. The computer readable medium may be any combination of such storage or transmission devices.

[0112] Such programs may also be encoded and transmitted using carrier signals adapted for transmission via wired, optical, and / or wireless networks conforming to a variety of protocols, including the Internet. As such, a computer readable medium according to an embodiment may be created using a data signal encoded with such programs. Computer readable media encoded with the program code may be packaged with a compatible device or provided separately from other devices (e.g., via Internet download). Any such computer readable medium may reside on or within a single computer product (e.g., a hard drive, a CD, or an entire computer system), and may be present on or within different computer products within a system or network. A computer system may include a monitor, printer or other suitable display for providing any of the results mentioned herein to a user.

[0113] In some examples, the computing device includes a monitor 1106. The monitor 1106 can represent one or more components used to output data. Examples of the display device can include a liquid-cry stal display (LCD), a computer monitor, a touch-screen display, etc.

[0114] Any of the methods described herein may be totally or partially performed with a computer system including one or more processors, which can be configured to perform the steps. Thus, embodiments can involve computer systems configured to perform the steps of any of the methods described herein, potentially with different components performing a respective steps or a respective group of steps. Although presented as numbered steps, steps of methods herein can be performed at a same time or in a different order. Additionally, portions of these steps may be used with portions of other steps from other methods. Also, all or portions of a step may be optional. Additionally, and of the steps of any of the methods can be performed with modules, circuits, or other means for performing these steps.

[0115] The specific details of particular embodiments may be combined in any suitable manner without departing from the spirit and scope of embodiments. However, other embodiments may involve specific embodiments relating to each individual aspect, or specific combinations of these individual aspects. The above description of exemplary embodiments has been presented for the purpose of illustration and description. It is not intended to be exhaustive or to limiting to the precise form described, and many modifications and variations are possible in light of the teaching above. The embodiments were chosen and described in order to best explain the embodiments and its practical applications to thereby enable others skilled in the art to best utilize the embodiments in various embodiments and with various modifications as are suited to the particular use contemplated.

[0116] A recitation of '‘a”, “an” or “the” is intended to mean “one or more” unless specifically indicated to the contrary'. The use of “or” is intended to mean an “inclusive or, and not an “exclusive or” unless specifically indicated to the contrary.

Claims

WHAT IS CLAIMED IS:

1. A computer implemented method for identifying differences in tissue architectures or cell types, the method comprising: performing multiplexed immunohistochemistry on a tissue sample to generate a multiplexed immunohistochemical image; identifying a plurality of single marker channels each identifying a cell within the multiplexed immunohistochemical image; generating, from the plurality of single marker channels, a visual representation of X and Y positional coordinates for each cell within the multiplexed immunohistochemical image; assigning a cell type to each cell based on the single marker channels, wherein the multiplexed immunohistochemical image of the tissue sample includes at least two different cell types; generating, based on the cell type and the X and Y positional coordinates, a hexagonal heatmap of cell types; determining cell-to-cell associations among a plurality of cells within the multiplex immunohistochemical image based on the cell type and the X and Y positional coordinates; and outputting the hexagonal heatmap as an image, wherein the hexagonal heatmap represents at least a cell density and the cell-to-cell associations within the tissue sample.

2. The computer implemented method of claim 1 , further comprising: representing each cell-to-cell association on the hexagonal heatmap using a value, wherein the value is a number representing the cell-to-cell association for a given cell among the plurality of cells.

3. The computer implemented method of claim 2, wherein the determining the cell-to- cell associations comprises: identifying a distance among the plurality of cells within the multiplex immunohistochemical image; determining, based upon the distance, a number of cells of a second cell type within a predetermined distance of a first cell of a first cell type; andincorporating the value on a representation of the first cell on the hexagonal heatmap, wherein the value is indicative of the number of cells of the second cell type within the predetermined distance of the first cell.

4. The computer implemented method of claim 1 , further comprising: selecting a hexagonal size, wherein the hexagonal size ranges from 1 um to 100 um; selecting a size of a region of interest, wherein the size of a region of interest ranges from 1 um to 3 mm, wherein a plurality7of cells are within the region of interest; and outputting, based on the hexagonal size and the size of the region of interest, a hexagonal heatmap, wherein the hexagonal heatmap comprises a value of a number of cell- to-cell associations.

5. The computer implemented method of claim 1, wherein the hexagonal heatmap is used to visually display data related to cells within a region of interest of the tissue sample, wherein the data comprises a cell identification, or a frequency distribution.

6. The computer implemented method of claim 1 , wherein the step of performing multiplexed immunohistochemistry comprises: obtaining a microscopic image of the tissue sample, wherein the microscopic image comprises a plurality of signal intensities corresponding to a plurality7of fluorescently labeled antibody targets; and running cell segmentation on the microscopic image to generate the multiplexed immunohistochemical image.

7. The computer implemented method of claim 1, wherein the visual representation is a cell dot plot.

8. A computer system comprising: one or more processors; and one or more memory7storing computer-readable instructions that, upon execution by the one or more processors, cause the computer system to: perform multiplexed immunohistochemistry on a tissue sample to generate a multiplexed immunohistochemical image;identify a plurality of single marker channels each identifying a cell within the multiplexed immunohistochemical image; generate, from the plurality of single marker channels, a visual representation of X and Y positional coordinates for each cell within the multiplexed immunohistochemical image; assign a cell type to each cell based on the single marker channels; generate, based on the cell type and the X and Y positional coordinates, a hexagonal heatmap of cell types; determine a cell-to-cell associations among a plurality of cells within the multiplex immunohistochemical image based on the cell type and the X and Y positional coordinates; and output the hexagonal heatmap as an image, wherein the hexagonal heatmap represents at least a cell densify and cell-to-cell associations within the tissue sample.

9. The computer system of claim 8, wherein the execution of the computer-readable instructions further causes the computer system to: represent each cell-to-cell association on the hexagonal heatmap using a value, wherein the value is a number representing cell-to-cell associations for a given cell among the plurality of cells.

10. The computer system of claim 9, wherein the computer-readable instructions to determine the cell-to-cell associations comprises: identifying a distance among the plurality of cells within the multiplex immunohistochemical image; determining, based upon the distance, a number of cells of a second cell type within a predetermined distance of a first cell of a first cell type; and incorporating the value on a representation of the first cell on the hexagonal heatmap, wherein the value is indicative of the number of cells of the second cell type within the predetermined distance of the first cell.

11. The computer system of claim 8. wherein the execution of the computer-readable instructions further causes the computer system to: select a hexagonal size, wherein the hexagonal size ranges from 1 um to 100 um;select a size of a region of interest, wherein the size of a region of interest ranges from 1 urn to 3 mm, wherein a plurality of cells are within the region of interest; and output, based on the hexagonal size and the size of the region of interest, a hexagonal heatmap, wherein the hexagonal heatmap comprises a value of a number of cell- to-cell associations.

12. The computer system of claim 8, wherein the computer-readable instructions to perform multiplexed immunohistochemistry further comprises: obtaining a microscopic image of the tissue sample, wherein the microscopic image comprises a plurality of signal intensities corresponding to a plurality of fluorescently labeled antibody targets; and running cell segmentation on the microscopic image to generate the multiplexed immunohistochemical image.

13. The computer system of claim 8. wherein the visual representation is a cell dot plot.

14. A computer implemented method for identifying differences in tissue architectures or cell types, the method comprising: performing a cellular recognition technique on a tissue sample to generate an image; identifying a plurality of cells within the image; generating a visual representation of X and Y positional coordinates for each cell within the image; assigning a cell type to each cell; generating, based on the cell type and the X and Y positional coordinates, a grid heatmap of cell types; determining cell-to-cell associations among a plurality of cells within the image based on the cell type and the X and Y positional coordinates; and outputting the grid heatmap as an image, wherein the grid heatmap represents at least a cell density and cell-to-cell associations within the tissue sample.

15. The computer implemented method of claim 14, wherein the cellular recognition technique comprises spatial-transcriptomics, fluorescence in-situ hybridization (FISH), or mass spectrometry.

16. The computer implemented method of claim 14. further comprising: representing each cell-to-cell association on the grid heatmap using a value, wherein the value is a number representing the cell-to-cell association for a given cell among the plurality of cells.

17. The computer implemented method of claim 14, wherein the determining the cell- to-cell associations comprises: identifying a distance among the plurality7of cells within the image; determining, based upon the distance, a number of cells of a second cell type within a predetermined distance of a first cell of a first cell ty pe; and incorporating a value on a representation of the first cell on the grid heatmap, wherein the value is indicative of the number of cells of the second cell type within the predetermined distance of the first cell.

18. The computer implemented method of claim 14, wherein the determining the cell- to-cell associations comprises: identifying the plurality7of cells within the image; determining, based upon a size of a grid element of the grid heatmap, a total distribution of cells in a single grid element; and incorporating a value on the grid heatmap, wherein the value is indicative of a number of cells in the single grid element.

19. The computer implemented method of claim 14. further comprising: scaling each grid element within the grid heatmap to generate a visual representation of the cell-to-cell association or cell densify; and outputting the visual representation onto each grid within the grid heatmap.

20. The computer implemented method of claim 14, wherein the determining the cell- to-cell associations is performed via Morisita-hom index or spatial proximity7score analysis.

Images