Scalable and highly accurate context-guided segmentation of histological structures, including tubules / glands and lumens, tubule / glandular clusters, and individual nuclei, in full-slide images of tissue samples from a spatial multi-parameter cell / intracellular imaging platform.

Abstract

To provide a method and system that segment one or more histological structures in a tissue image represented by multi- parameter cellular and intracellular imaging data.SOLUTION: A method of segmenting one or more histological structures in a tissue image represented by multi-parameter cellular and intracellular imaging data includes: receiving coarsest level image data on a tissue image, in which the coarsest level image data corresponds to the coarsest level of a multiscale representation of first data corresponding to the multi-parameter cellular and intracellular imaging data; further dividing the coarsest level image data into a plurality of non-overlapping superpixels; assigning, to each superpixel, a probability belonging to the one or more histological structures using some pre-trained machine learning algorithms to create a probability map; extracting an estimation boundary of the one or more histological structures by applying a contour algorithm to the probability map; and generating an accurate boundary of the one or more histological structures using the estimation boundary.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] <Government Contract> This invention was made with government support under grant #CA204826 awarded by the National Institutes of Health (NIH). The government has certain rights in this invention.

[0002] <Technical Field> The present invention relates to digital pathology, and more particularly to scalable and high-precision context-guided segmentation of histological structures including, but not limited to, ducts / glands and lumens, clusters of ducts / glands, and individual nuclei in multi-parameter cellular and sub-cellular imaging data of several stained tissue images such as whole slide images obtained from several patients or several in vitro multicellular models.

Background Art

[0003] The histopathological examination of diseased tissue is essential for the diagnosis and grading of diseases. Currently, pathologists usually make diagnostic decisions (e.g., the malignancy or severity of a disease) based on the visual interpretation of histopathological structures in the transmitted light images of diseased tissue. Such decisions are mostly subjective and, especially in atypical situations, a high level of disagreement occurs.

[0004] In addition, digital pathology is gaining momentum in applications such as second opinion telepathology, interpretation of immunohistochemistry, and intraoperative telepathology. Usually, digital pathology consists of several tissue slides, a large amount of patient data representing them is generated, the slides are displayed on a high-resolution monitor and evaluated by pathologists. Due to the inclusion of manual work, the current workflow practice in digital pathology is time-consuming, error-prone, and subjective.

Summary of the Invention

[0005] In one embodiment, a method is provided for segmenting one or more histological structures in a tissue image represented by multiparameter cell-intracellular imaging data. The method includes receiving the coarsest level of image data of the tissue image, the coarsest level of image data corresponding to the coarsest level of a multiscale representation of first data corresponding to multiparameter cell-intracellular imaging data. The method further includes dividing the coarsest level of image data into a plurality of non-overlapping superpixels; creating a probability map by assigning each superpixel a probability of belonging to one or more histological structures using several pre-trained machine learning algorithms; applying a contour algorithm to the probability map to extract estimated boundaries for one or more histological structures; and using the estimated boundaries to obtain accurate boundaries for one or more histological structures. In one exemplary embodiment, the multiscale representation includes a Gaussian multiscale pyramid decomposition, the multiparameter cell / intracellular imaging data includes stained tissue image data, receiving the coarsest level of image data for the tissue image includes receiving the coarsest level of normalized constituent stain image data for the stained tissue image, the coarsest level of normalized constituent stain image data relating to a specific constituent stain of the stained tissue image and corresponding to the coarsest level of the Gaussian multiscale pyramid decomposition of the first data corresponding to the stained tissue image data, and splitting the coarsest level of image data into multiple superpixels includes splitting the coarsest level of normalized constituent stain image data into multiple superpixels.

[0006] In one embodiment, a computer system is provided for segmenting one or more histological structures in a tissue image represented by multi-parameter cell / intracellular imaging data. The system includes a processing unit, which comprises several components configured to carry out the above method. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a schematic diagram of an exemplary digital pathology system for segmenting histological structures from multi-parameter cell and intracellular imaging data, based on an exemplary embodiment of the disclosed concept. [Figure 2A] Figure 2A is a flowchart showing a method for segmenting histological structures based on a specific exemplary embodiment of the disclosed concept. [Figure 2B] Figure 2B is a flowchart showing a method for segmenting histological structures based on specific exemplary embodiments of the disclosed concept. [Figure 3] Figure 3 shows a non-limiting, exemplary H&E-stained tissue image that can be processed by the disclosed concept, illustrating a color deconvolution step in an exemplary embodiment of the disclosed concept. [Figure 4] Figure 4 illustrates the normalization of staining intensity in the steps of an exemplary embodiment of the disclosed concept. [Figure 5] Figure 5 illustrates the Gaussian multiscale pyramid decomposition step of an exemplary embodiment of the disclosed concept. [Figure 6] Figure 6 illustrates the decomposition of image data into superpixels based on an exemplary embodiment of the disclosed concept. [Figure 7] Figure 7 illustrates an exemplary pair of superpixels based on an exemplary embodiment of the disclosed concept. [Figure 8] Figure 8 illustrates an exemplary probability map and an exemplary image showing the application of a region-based active contour algorithm based on an exemplary embodiment of the disclosed concept. [Modes for carrying out the invention]

[0008] In this specification, the singular forms of "aru" (aru) and "sono" (sono) include references to the plural unless the context clearly indicates otherwise.

[0009] In this specification, the statement that two or more parts or components are “joined” means that, insofar as a connection occurs, the parts are joined or operate together directly or indirectly, i.e., through one or more intermediate parts or components.

[0010] In this specification, the term "several" means one or an integer greater than one (i.e., multiple).

[0011] In this specification, the terms “component” and “system” are intended to refer to computer-related entities that are either hardware, a combination of hardware and software, software, or running software. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. For example, both an application running on a server and the server may be components. One or more components may reside within a process and / or an execution thread, and components may be localized in one computer and / or distributed across two or more computers. While several methods of displaying information to the user are illustrated and described herein with specific figures or graphs as screenshots, those skilled in the art will recognize that various other alternative means may be employed.

[0012] In this specification, the term “multiparameter cell / intracellular imaging data” means data obtained by generating several images from a section of tissue that provides information about multiple measurable parameters at the cellular and / or intracellular levels in the section of tissue. Multiparameter cell / intracellular imaging data may be created by several different imaging modalities, including, but not limited to, transmitted light (e.g., combinations of H&E and / or IHC (one or more biomarkers)), fluorescence, immunofluorescence (including, but not limited to, antibodies and nanobodies), multiplexing and / or high-multiplexing of live cell biomarkers, and electron microscopy. Targets include, but are not limited to, tissue samples (human or animal) and in vitro models of tissues and organs (human or animal).

[0013] In this specification, the term “superpixel” means a concatenated patch or group of two or more pixels having similar image statistics as defined in a suitable color space (e.g., RGB, CIELAB, or HSV).

[0014] In this specification, the term "non-overlapping superpixel" means a superpixel whose boundary does not overlap with any of its neighboring superpixels.

[0015] In this specification, "Gaussian multiscale pyramidal decomposition" means repeatedly applying a Gaussian filter to an image, smoothing and subsampling it by 2 in the x and y directions.

[0016] In this specification, the term "region-based active contour algorithm" refers to any active contour model that takes image gradients into account to detect the boundaries of an object.

[0017] As used herein, the term "context-ML model" refers to a machine learning algorithm that can take into account the neighborhood information of superpixels.

[0018] As used herein, the term "staining-ML model" refers to a machine learning algorithm that can take into account the staining intensity of superpixels.

[0019] As used herein, the term "probability map" refers to a set of pixels having probability values in the range from 0 to 1, and these probability values represent the positional probability of whether a pixel is within a specific histological structure.

[0020] For example, terms related to directions used herein, such as up, down, left, right, upper, lower, front, back, and their derivatives, relate to the directions of the elements shown in the drawings and do not limit the scope of the claims unless explicitly stated.

[0021] Hereinafter, the disclosed concepts will be described in terms of many specific details for the purpose of explanation to provide a complete understanding of the innovation that is the subject matter. However, it will be apparent that the disclosed concepts can be implemented without these specific details without departing from the spirit and scope of the present invention.

[0022] The concepts disclosed herein, as described in more detail in relation to various exemplary embodiments, provide a novel approach to identifying and characterizing the morphological properties of histopathological structures. An early application of such tools is to perform scalable, high-precision context-guided segmentation of histological structures, including, for example, tubules / glands and lumens, tubule / gland clusters, and individual nuclei, in images of tissue samples (e.g., full-slide images), based on spatial multi-parameter cellular / intracellular imaging data representing such images. In this particular non-limiting application of the disclosed concepts, as described in more detail herein, hematoxylin and eosin (H&E) imaging data are employed as multi-parameter cellular / intracellular imaging data, and color deconvoluted hematoxylin imaging data is used to segment tubules / glands and lumens, tubule / gland clusters, and individual nuclei. However, this is merely illustrative, and it will be understood that the disclosed concepts may be employed to segment other histological structures using other types of data. For example, connective tissue may be segmented using color deconvolved eosin image data obtained from H&E image data. Further possibilities are conceivable within the scope of the disclosed concepts.

[0023] The disclosed concepts relate to and improve upon the subject matter described in U.S. Patent Application No. 15 / 577,838 (published as 2018 / 0204085), “Systems and Methods for Finding Regions of Interest in Hematoxylin and Eosin (H&E) Stained Tissue Images and Quantifying Intratumor Cellular Spatial Heterogeneity in Multiplexed / Hyperplexed Fluorescence Tissue Images,” the disclosure of which is incorporated herein by reference. The disclosed concepts differ from the subject matter of the aforementioned application in at least two respects. First, the disclosed concepts fall into the semi-supervised or weakly supervised category in that user input is present in at least one step of the machine learning algorithm. Second, the disclosed concepts work best when the boundaries of the region of interest (ROI) are given by rough estimation. The concepts disclosed herein, as described in detail, clarify such rough boundaries.

[0024] Figure 1 is a schematic diagram of an exemplary digital pathology system 5 built and configured for automated segmentation of histological structures from multiparameter cell / intracellular imaging data, based on exemplary embodiments of the concepts disclosed herein. As seen in Figure 1, System 5 is a computer device built and configured to generate and / or receive multiparameter cell / intracellular imaging data (labeled 25 in Figure 1) and to process the data as described herein in order to segment the histological structures in tissue images represented by the multiparameter cell / intracellular imaging data 25. System 5 may, but is not limited to, a PC, laptop computer, tablet computer, or other suitable computer device built and configured to perform the functions described herein.

[0025] System 5 includes an input device 10 (such as a keyboard), a display 15 (such as an LCD), and a processing unit 20. A user can provide input to the processing unit 20 using the input device 10, and the processing unit 20 provides output signals to the display 15, enabling the display 15 to display information to the user as described in detail herein. The processing unit 20 comprises a processor and memory. The processor is, for example, a microprocessor (μP), a microcontroller, an application-specific integrated circuit (ASIC), or other suitable processing device that interfaces with the memory. The memory may be one or more of various types of internal and / or external storage media, such as computer-readable media, such as RAM, ROM, EPROM, EEPROM, FLASH®, or others that provide storage registers, in the case of data storage such as the internal storage area of ​​a computer, and may be volatile or non-volatile memory. The memory stores several routines that can be executed by the processor, including routines for implementing the disclosed concepts as described herein. In particular, the processing device 20 includes a histological structure segmentation component 30, which is configured to identify and segment histological structures (but not limited to tubules / glands and lumens, tubule / gland clusters, and individual nuclei, etc.) in several tissue images (e.g., H&E stained image data) represented by multi-parameter cell / intracellular imaging data 25 obtained from various imaging modalities, as described herein in various embodiments.

[0026] Figures 2A and 2B are flowcharts showing a method for segmenting histological structures according to a specific exemplary embodiment of the disclosed concept. The method shown in Figures 2A and 2B may be carried out, for example, in System 5 of Figure 1 described above, and the method is described as such for illustrative purposes. In addition, in the specific non-exclusive exemplary embodiments shown in Figures 2A and 2B, the multi-parameter cell / intracellular imaging data used is H&E stained image data of a tissue sample, the histological structures to be segmented are based on hematoxylin image data, and include tubules / glands and lumens, tubule / gland clusters, and individual nuclei. Hereinafter, it will be understood that the specific embodiments shown in Figures 2A and 2B and described herein are intended to be illustrative only and not limiting. It will be understood that other types of multi-parameter cell / intracellular imaging data may be used in connection with the disclosed concept.

[0027] Referring to Figure 2A, the method begins in step 100, in which the processing unit 20 of System 5 generates and / or receives multi-parameter cell-intracellular imaging data representing the H&E-stained tissue image to be processed. A non-limiting exemplary H&E-stained tissue image 35 that can be processed by the disclosed concept is shown in Figure 3 for illustrative purposes. In addition, in the exemplary embodiment, the multi-parameter cell-intracellular imaging data generated and / or received in step 100 is in RGB format.

[0028] Next, in step 105, the multi-parameter cell / intracellular imaging data (i.e., H&E stained tissue image data in RGB format) generated and / or received in step 100 is color deconvoluted to individual staining intensities (hematoxylin and eosin) to create hematoxylin image data and eosin image data for the H&E stained tissue image to be processed. Figure 3 illustrates the color deconvolution in step 105 by showing a hematoxylin image 40 represented by the hematoxylin image data and a resulting eosin image 45 represented by the eosin image data resulting from the color deconvolution of the H&E stained tissue image 35.

[0029] The method then proceeds to step 110, in which the staining intensity of the hematoxylin image data is normalized against a reference dataset to generate normalized hematoxylin image data. The staining intensity normalization in step 110 is performed so that variations in staining intensity are normalized for downstream processing. In an exemplary embodiment, to set up the reference dataset, a batch of whole slide images (WSI) is first color deconvolved into hematoxylin staining intensity images and eosin staining intensity images. From this batch, a random number of 1Kx1K images are trimmed and used to create a cumulative intensity histogram of hematoxylin channels. The test WSI first undergoes a color deconvolution operation. Next, histogram equalization is performed to match the intensity histogram of hematoxylin channels with the histogram of the reference dataset. The staining intensity normalization in step 110 is shown in Figure 4. Figure 4 shows the original hematoxylin channel 50 from a different (different) exemplary full-slide image and the normalized hematoxylin channel 55 from the same exemplary full-slide image. As a result, the intensity of the normalized hematoxylin channel 55 now matches that of the reference image dataset.

[0030] Next, in step 115, a Gaussian multiscale pyramidal decomposition (a form of pyramidal representation) is performed on the normalized hematoxylin image data to generate a multiscale representation of the normalized hematoxylin image data. The multiscale representation created in step 115 has n levels, L1...L n It includes L1, which is level data representing the full resolution level of the decomposition, and L n is level data representing the coarsest level of resolution. By constructing a Gaussian pyramid, the computational load when detecting histological structures from the entire slide image according to the disclosed conceptual method can be reduced. Figure 5 shows a Gaussian multiscale pyramidal resolution of an exemplary normalized hematoxylin channel 55 shown in Figure 4. In the exemplary embodiment, the image size is reduced from the original resolution of 30K × 50K to the coarsest level of 1K × 1.5K. Also in the exemplary embodiment, the image size is halved at each level of the resolution hierarchy.

[0031] The method then proceeds to step 120 in Figure 2B. In step 120, the coarsest level data L n In exemplary embodiments, this is divided into non-overlapping superpixels, which are sets of connected pixels having similar intensity (gray) values. As can be understood, this can be done in several ways. In the simplest approach, a normal distribution of noise is assumed, with a mean of zero and a standard deviation of sigma. For example, for an image with 256 grayscale levels per pixel, the standard deviation of noise is typically assumed to be 4 grayscale levels. This value may be set by the end user. In exemplary embodiments, this is done using a simple linear iterative clustering (SLIC) algorithm, some of which are known in the art. Figure 6 shows the coarsest level of data L obtained from an exemplary hematoxylin image 40 described herein. nThe results of step 120 when performed on [the specified image] are shown. In the exemplary embodiment, the image is segmented into approximately 5K superpixels. This is recommended for 1K×1K hematoxylin channel images because the computation is fast and effective for downstream processing. In addition, in the exemplary embodiment, a 2D Delaunay triangulation of the superpixel centroids is performed to identify spatial neighbors for each superpixel.

[0032] In addition, based on one aspect of the disclosed concept, several machine learning algorithms / models are trained to predict which superpixels belong to a particular histological structure, which is a tubule / gland, in an exemplary embodiment. Thus, following step 120, the method proceeds to step 125, where each superpixel is assigned a probability of belonging to a histological structure, such as a tubule / gland, in the illustrated exemplary embodiment, using several pre-trained machine learning algorithms. As a result, the coarsest level of data L n A probability map is created.

[0033] In a non-limiting exemplary embodiment of the disclosed concept, several trained machine learning algorithms employed in step 125 comprise a context-ML model (such as a context-support vector machine (SVM) model or a context-logistic regression (LR) model) and a stain-ML model (such as a stain-support vector machine (SVM) model or a stain-logistic regression (LR) model) for predicting superpixels belonging to the structure of the problem. In this exemplary embodiment, the RGB color histograms of the superpixels and their neighbors are used as feature vectors. Specifically, in this embodiment, two models, namely a context-ML model and a stain-ML model, are constructed and trained (in a monitored manner). Each of these models is described in more detail below.

[0034] In an exemplary embodiment, for a context-ML model such as a context-SVM model, 2000 pairs of adjacent superpixels from 10 different images in a reference image dataset are randomly selected for the training set of the exemplary embodiment. Ground truth is collected (i.e., user input is required) by displaying pairs of superpixels on a screen and asking a subject (e.g., an experienced / specialized pathologist) whether none of the displayed superpixels belong to a tube, or whether one or both belong to a tube. For illustrative purposes, Figure 7 shows three such exemplary pairs of superpixels in exemplary images (labeled A, B, and C), where pair A has 0 superpixels present in the tube (class label 0), pair B has 1 superpixel in the tube (class label 1), and pair C has 2 superpixels in the tube (class label 2). This ground truth is used as class labels to train the context ML model. In an exemplary embodiment, the color histograms (i.e., pixel values ​​for R, G, and B colors) of each superpixel pair and their first neighbor are used as feature vectors. For each test superpixel pair, the ML model returns the probability that the pair has superpixels that do not both belong to the tube, or that one or both belong to the tube. In each case, the superpixel pair is assigned to the category with the highest probability. Note that this does not determine the actual identity of the superpixels in the structure. Instead, a second model, staining-ML described below, is applied for that purpose.

[0035] For stain-ML models, such as the stain-SVM model in exemplary embodiments, ground truth is collected separately. In particular, the subject (e.g., an experienced / specialized pathologist) is asked to classify whether a particular superpixel has “no staining,” “light staining,” “moderate staining,” “heavy staining,” or “unknown” (i.e., user input is again required). Since some structures, such as tubes, are amorphous, one way to detect the boundaries of a structure is to carefully observe the change in staining color as one moves from the inside of the tube into the surrounding connective tissue. Using this information, it is possible to identify superpixels that may be part of a tube. Figure 7 shows four such superpixels D, E, F, and G, classified as “no staining,” “light staining,” “moderate staining,” and “heavy staining,” respectively, for exemplary images. Superpixels are assigned to a category with the highest probability.

[0036] Therefore, in step 125 of this particular exemplary embodiment, the two machine learning models described above (context-ML model and stain-ML model) are used on the coarsest level of data L n The non-overlapping superpixels are sequentially applied (step 120) to create a probability map for identifying superpixel pairs that are likely to be inside the structure in question (a tube in this exemplary embodiment). In the exemplary embodiment, all moderately to heavily stained superpixels are identified as being inside the tube. In other words, the context-ML model and the stain-ML model together assign a conditional probability to each superpixel that it belongs to the structure in question.

[0037] Once the probability map is created in step 125 as described above, the method proceeds to step 130. In step 130, a rough estimate of the boundaries of the histological structure is extracted by applying the region-based active contour algorithm to the probability map. Exemplary probability map 60 and exemplary image 65 demonstrating the application of the region-based active contour algorithm are provided in Figure 8.

[0038] Next, the method proceeds to step 135, where the newly obtained estimation is used to obtain segmentation of the structure in the full-resolution image. Specifically, step 135 involves sequentially refining the boundary of the histological structure from coarse to fine by upsampling the structural boundary from level K+1 to level K, and then initiating region-based active contouring at level K using the upsampled boundary. In an exemplary embodiment, the upsampling first involves simply doubling the coordinates known at K+1 to determine the boundary coordinates from level K+1 to level K, and then interpolating between the boundary pixels at level K.

[0039] Therefore, in the exemplary methods shown in Figures 2A and 2B, superpixels are utilized only at the coarsest level of the pyramid. As explained, region-based contouring is performed on the probability map of identified superpixels. However, at successive finer scales, superpixels are not required, and region-based active contouring is performed directly on the relevant stained-separated images.

[0040] In one particular embodiment of this exemplary embodiment, the region-based active contour algorithm used is the Chan-Vese segmentation algorithm, which separates the foreground (tube) from the background (the rest of the image). The cost function of the active contour varies depending on the difference in the mean values ​​of hematoxylin staining between the foreground and background regions. For example, two superpixels that are likely to be inside a tube will have nearly identical staining (from moderate to dark), and their boundaries will be iteratively merged by active contour optimization.

[0041] To construct "clusters" of tubes, region-based active contouring may be performed on probability maps returned by context-ML and stain-ML models. In an exemplary embodiment, the probability maps assign non-zero probabilities to regions bridging tubes, and the region-based active contouring model performed on the probability maps does a better job of depicting tube clusters. To segment tubes from the entire WSI, tubes and tube clusters are first identified from the lowest-resolution pyramid image using the steps described above. These results are recursively upsampled, and region-based active contouring is re-run at each level of the hierarchy to refine the upsampled tube boundaries. In an exemplary embodiment, the active contour image consists of masks showing pixels inside and outside the tube boundaries. The active contour image and the hematoxylin image are upsampled together.

[0042] The disclosed concepts may be used to identify nuclei within a tube. Once the tube is identified, superpixel segmentation is performed in the region belonging to the tube. Next, a staining-ML model may be performed to further separate moderately stained and heavily stained superpixels within the tube. The heavily stained superpixels would correspond to the locations of nuclei within the tube. To identify nuclei outside the tube, a similar model may be developed to construct a feature vector of superpixels (histograms of each channel: red, blue, and green) without neighbors in the first layer. Without the mean histogram of superpixels and its first layer, all heavily stained superpixels corresponding to nuclei inside and outside the tube would be identified.

[0043] In the exemplary embodiment described in relation to Figures 2A and 2B, the color deconvolution step (105) and the staining intensity normalization step (110) are performed before the Gaussian multiscale pyramidal decomposition is performed (step 115). In another embodiment, the order of these steps is reversed. In particular, in an alternative embodiment of the disclosed concept, after the multi-parameter cell / intracellular imaging data is received (step 100), the Gaussian multiscale pyramidal decomposition is first performed on the entire H&E-stained tissue image. Next, the color deconvolution in step 105 and the staining intensity normalization in step 110 are performed only on the coarsest level image data to generate normalized hematoxylin image data only for the coarsest level image data. Subsequently, steps 120 to 130 are performed using the normalized hematoxylin image data only for the coarsest level image data to generate the probability map and estimated boundaries of histological structure as described above. The boundaries are then sequentially refined using step 135 as described.

[0044] Again, while certain embodiments of the disclosed concept use color deconvoluted hematoxylin image data to distinguish tubules / glands and lumens, tubule / gland clusters, and individual nuclei, it should be noted that this is merely illustrative, and it is understood that the disclosed concept can be employed to distinguish other histological structures using other types of data. For example, but not limited to, connective tissue may be segmented using color deconvoluted eosin image data (different from color deconvoluted hematoxylin image data) obtained from H&E image data. Further possibilities are conceivable within the scope of the disclosed concept.

[0045] Furthermore, the above-disclosed conceptual explanation is based on and utilizes in-situ multi-parameter cell and intracellular imaging data. However, it should be understood that this does not imply limitation. Rather, it should be understood that the disclosed concepts can be used in conjunction with in vitro microphysiological models for basic research and clinical translation. Multicellular in vitro models enable the study of spatiotemporal cellular heterogeneity and heterocellular communication, summarizing human tissues applicable to investigating the mechanisms of disease progression in vitro, testing drugs, and characterizing the structural composition and content of these models for use in transplantation.

[0046] In the claims, symbols placed in parentheses should not be construed as limiting the claims. The words “equip” or “include” do not preclude the existence of elements or steps other than those described in the claims. In an apparatus claim listing several means, some of these means may be embodied by a single piece of hardware. The word “are” preceding an element does not preclude the existence of multiple such elements. In any apparatus claim listing several means, some of these means may be embodied by a single piece of hardware. The mere fact that elements are described in different dependent claims does not indicate that these elements cannot be used in combination.

[0047] While the present invention has been described in detail with regard to embodiments currently considered to be the most practical and preferred embodiments, it should be understood that such details are for that purpose only, and the present invention is not limited to the disclosed embodiments, but rather intended to include modifications and equivalent configurations that fall within the spirit and scope of the appended claims. For example, it should be understood that, to the extent possible, one or more features of any embodiment can be combined with one or more features of any other embodiment.

Claims

1. A method for processing tissue images represented by multi-parameter cell / intracellular imaging data, Receiving the first image data based on the aforementioned multi-parameter cell / intracellular imaging data, Decomposing the first image data into multiple non-overlapping superpixels, Using several pre-trained machine learning algorithms, a probability map is created by assigning the probability of belonging to one or more histological structures to each non-overlapping superpixel. Characterizing one or more morphological characteristics of the one or more histological structures based on the probability map, Methods that include...

2. The method according to claim 1, further comprising creating a multiscale representation of the multi-parameter cell / intracellular imaging data including full-resolution image data and the first image data, wherein the resolution of the first image data is smaller than the resolution of the full-resolution image data.

3. The method according to claim 2, wherein the first image data is the coarsest level of image data in the multiscale representation of the multiparameter cell / intracellular imaging data.

4. The method according to claim 1, wherein characterizing one or more morphological properties of the one or more histological structures includes segmenting the one or more histological structures, and segmenting the one or more histological structures includes extracting estimated boundaries of the one or more histological structures by applying a contour algorithm to the probability map, and using the estimated boundaries to obtain accurate boundaries of the one or more histological structures.

5. The method according to claim 4, wherein the contour algorithm is a region-based active contour algorithm.

6. The method according to claim 1, wherein the aforementioned pre-trained machine learning algorithms are several supervised machine learning algorithms that have been pre-trained based on user input.

7. The method according to claim 6, wherein the aforementioned pre-trained machine learning algorithms include a context-ML model and a stain-ML model applied to the plurality of non-overlapping superpixels.

8. The method according to claim 1, wherein each non-overlapping superpixel is a connected group of two or more pixels having similar intensity or image statistics.

9. A non-temporary computer-readable medium storing one or more programs that, when executed by a computer, include instructions causing the computer to perform the method described in claim 1.

10. A computer system for segmenting one or more histological structures in a tissue image represented by multi-parameter cell / intracellular imaging data, Receiving the first image data based on the aforementioned multi-parameter cell / intracellular imaging data, Decomposing the first image data into multiple non-overlapping superpixels, Using several pre-trained machine learning algorithms, a probability map is created by assigning the probability of belonging to one or more histological structures to each non-overlapping superpixel, Characterizing one or more morphological characteristics of the one or more histological structures based on the probability map, A system comprising a processing unit including several components configured to perform a specific task.

11. The system according to claim 10, wherein some of the components are further configured to create a multiscale representation of the multi-parameter cell / intracellular imaging data, including full-resolution image data and the first image data, the resolution of the first image data being lower than the resolution of the full-resolution image data.

12. The system according to claim 11, wherein the first image data is the coarsest level of image data in the multiscale representation of the multiparameter cell / intracellular imaging data.

13. Characterizing one or more morphological properties of the one or more histological structures comprises segmenting the one or more histological structures, the segmenting of the one or more histological structures comprises extracting estimated boundaries of the one or more histological structures by applying a contour algorithm to the probability map, and using the estimated boundaries to obtain accurate boundaries of the one or more histological structures, the system according to claim 10.

14. The system according to claim 13, wherein the contour algorithm is a region-based active contour algorithm.

15. The system according to claim 10, wherein the aforementioned pre-trained machine learning algorithms are several supervised machine learning algorithms that have been pre-trained based on user input.

16. The system according to claim 15, wherein the aforementioned pre-trained machine learning algorithms include a context-ML model and a stain-ML model applied to the plurality of non-overlapping superpixels.

17. The system according to claim 10, wherein each non-overlapping superpixel is a connected group of two or more pixels having similar intensity or image statistics.

Images