Automatic fluorescence imaging and single cell segmentation

Abstract

To provide a system and a method for automated, unsupervised, parameter-free segmentation of a single cell and other object in an image generated by a fluorescence microscope.SOLUTION: A method for improving both initial image quality and automatic segmentation on an image is typically performed on a digital image by a computer executing an appropriate software stored in a memory or by a processor.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 62 / 985,539, filed March 5, 2020, the entire disclosure of which is incorporated herein by reference.

[0002] Technical Field The present disclosure relates to the field of analysis of digital images, and in particular to the automated analysis of digital images produced by fluorescence microscopes to perform segmentation of single cells or other objects in digital images produced by fluorescence microscopes. [Background technology]

[0003] Flow cytometers (FCM) and fluorescence-activated cell sorters (FACS) are currently some of the primary tools for cell characterization. Both systems utilize fluorescent antibodies or other fluorescent probes (fluorophores) to tag cells with specific characteristics of interest and then detect the fluorescent light to locate the target cells. Both systems are widely used in both biomedical research and clinical diagnostics and can be used to study physical properties of cells (e.g., cell size and shape) as well as biochemical properties (e.g., cell cycle distribution and DNA content). Flow cytometry has also become a valuable clinical tool for monitoring the progression of certain cellular diseases.

[0004] Information about cells of interest is typically acquired optically. FCM / FACS involves the introduction of specific tagged probes that fluoresce at specific wavelengths when exposed to specific light inputs. By connecting the tagged probes to the target of interest, it is possible to identify the individual cells to which the probes are attached. It is important to recognize that in FCM / FACS, the measured signal is not pure. Fluorescence is typically detected optically by a device such as a photomultiplier tube (PMT), which receives the light from the marker as well as background signal (light pollution) and autofluorescence from the cells. Furthermore, even the specific light from the marker can be acquired from nonspecifically bound markers present in the sample. This is in addition to the specifically bound markers attached to target molecules on or within the cells, which is the value of interest.

[0005] For determining the desired result, only the signal of the specifically bound marker is relevant, and the rest of the measured signal is considered to be "noise" or error. This error prevents a specific signal from being measured with the desired sensitivity, and when the intensity of the specific signal is low and the intensity of the noise is relatively high, the specific signal may "disappear" in the noise, leading to a false negative result. Therefore, the amount of noise (signal-to-noise ratio) affects the detection sensitivity of the measurement. Furthermore, noise can also affect the distinction between positive and negative cell populations, as the measured positive and negative values ​​may become indistinguishable.

[0006] The main reasons for measurement errors are background signals (stains) within the small dynamic range of the detection device, high cellular autofluorescence signals, nonspecific binding of markers to cells, and variations that occur during the staining process (e.g., different marker concentrations, marker labels, or staining conditions). While it is desirable to reduce nonspecific background signals as much as possible, FCM / FACS-based methods have limited possibilities for reducing background signals. In principle, the basic background signal of an FCM / FACS device can only be reduced by avoiding the accumulation or detection of interfering signals.

[0007] To eliminate background signals from detection devices and cellular autofluorescence, currently known methods typically use a control sample in which the cells under investigation are not treated with a fluorophore, and subtract the background and autofluorescence values ​​from this sample from the actual measurement run. However, this approach has many drawbacks. Comparing two different cell populations often introduces additional measurement errors. This is because the populations may differ in the density, age, expression intensity, etc. of the investigated marker. Furthermore, the fluorescence of sample materials (excluding cells) may vary between different samples due to variations in their manufacturing process or their composition. Finally, and often most importantly, the control sample value is typically an average calculated based on the control sample measurements and does not take into account the variations that exist between individual cells.

[0008] One of the main problems with FACS, and analyzing digital images of biological systems more generally, is the need for segmentation. Segmentation typically refers to the need to identify cell boundaries within the digital images of cells. Once acquired, digital images, such as those generated using these fluorophores, must be segmented so that individual cells can be identified and, for example, counted. Segmentation is typically performed using an algorithm. The watershed transform is one image processing technique that has been used to segment images of cells. With the watershed transform, the digital image can be modeled as a three-dimensional topological surface, where pixel intensity values ​​represent geographic heights. Thus, items associated with more fluorophores (such as cell nuclei) can be identified as peaks or "volcanoes," while cell walls can be the valleys between the peaks.

[0009] However, a problem with many segmentation algorithms is the variability in tissue structure between different tissue types and cells. Therefore, a particular algorithm may not produce accurate segmentations without adaptation or training to specific cell types. This variation can also make training automated segmentation systems very difficult. Algorithms can fail in both directions, causing images to be over-segmented (showing complete cells when only parts of cells are present) or under-segmented (showing single cells when multiple cells are present).

[0010] To combat the problem, alternative algorithms have been proposed, including those discussed in U.S. Patent No. 8,995,740, the entire disclosure of which is incorporated herein by reference. However, these systems often still suffer from issues due to the underlying image capture quality, preventing even the best algorithms from reaching their full potential, and even then, they often still suffer from topological structures (such as "atolls") that can be common in certain cell images.

[0011] This problem is particularly acute because training methods for automated cell analysis are typically trained on a wide variety of heterogeneous cells. Thus, cells with low autofluorescence and cells with high autofluorescence are typically calibrated against the same average control sample value (or training data), resulting in cells with low autofluorescence being more likely to be scored as false negatives and cells with high autofluorescence being more likely to be scored as false positives. Because all of the above can significantly impair the sensitivity of the detection method, it would be desirable to have available a method that overcomes the shortcomings of existing methods and enables more sensitive and reliable cell analysis.

[0012] U.S. Patent Application No. 15 / 708,221, the entire disclosure of which is incorporated herein by reference, provides systems and methods that can be used to more effectively cancel autofluorescence and background noise. Specifically, this is done by determining autofluorescence from the actual sample being tested, rather than from a generalized sample or training selection of materials. This provides a substantial improvement in signal-to-noise ratio. However, these systems and methods do not offer advanced calibration techniques that utilize high dynamic range image (HDRI) cameras, nor do they offer advanced processing techniques that can improve the accuracy of cell segmentation when images are so prepared. Summary of the Invention

[0013] The following is a summary of the invention to provide a basic understanding of some aspects of the invention. This summary is not intended to identify key or critical elements of the invention or to delineate the scope of the invention. The sole purpose of this section is to present some concepts of the invention in a simplified form as a prelude to the more detailed description that is presented later.

[0014] To address these and other problems in the art, described herein is a method for calibrating an imaging system for cellular imaging, the method including, among other things, providing an imaging system for imaging cells tagged with a fluorophore having a wide range of fluorescence wavelengths; performing a calibration for autofluorescence, the calibration including providing a sample of unstained cells having a wide range of autofluorescence wavelengths to the imaging system, illuminating the sample with an illumination source, and imaging the sample over a dynamic range that includes all of the wide range of fluorescence wavelengths and all of the wide range of autofluorescence wavelengths; and performing a calibration for chromatic aberration, the calibration including providing the sample of cells to the imaging system, illuminating the sample with an illumination source, acquiring a first image of the cellular sample, repositioning the imaging system relative to the cellular sample, and acquiring a new image of the cellular sample.

[0015] In one embodiment of the method, the method is performed by a digital computer including a memory having instructions for carrying out the method.

[0016] In one embodiment of the method, the minimum photon collection time of the imaging system is set separately for each wavelength in the dynamic range, such that the minimum photon collection time is sufficient for detection of all values ​​within the complete dynamic range.

[0017] In one embodiment of the method, the dynamic range includes all wavelength imaging system images.

[0018] In one embodiment of the method, the imaging system comprises a digital grayscale camera.

[0019] In one embodiment of the method, the camera is equipped with a filter set based on a fluorophore.

[0020] In one embodiment of the method, the camera is a high dynamic range imaging (HDRI) camera.

[0021] In one embodiment of the method, the camera generates high dynamic range (HDR) through exposure fusion to provide improved contrast.

[0022] In one embodiment of the method, the cells tagged with a fluorophore are a different type of cell than the unstained cells.

[0023] In one embodiment of the method, the sample of cells is a different type of cell than the sample of unstained cells.

[0024] In one embodiment of the method, the sample of cells is a sample of unstained cells.

[0025] Also described herein in one embodiment is a method for analyzing cellular images, the method including providing a sample of cells tagged with fluorophores having a wide range of fluorescence wavelengths, illuminating the sample with an illumination source, imaging the illuminated sample across the wide range of fluorescence wavelengths to create a sample image, subtracting a calibration image from the sample image to create a calibrated image, representing the image as a topological curve, wherein the height of the curve at each curve pixel represents the intensity of fluorescence at that image pixel in the calibrated image, searching the topological curve for groups of pixels having a height greater than a selected height, choosing a new height less than the selected height, repeating the search using the new height as the selected height, and for each identified group of pixels, following a convex boundary of the topological curve away from the group of pixels to an inflection point where the curvature on the convex side is reduced, and identifying the convex boundary, wherein the group of pixels is included as a cell in the calibrated image.

[0026] In one embodiment of the method, the method is performed by a digital computer including a memory having instructions for carrying out the method.

[0027] In one embodiment of the method, the calibration image is formed by a method that includes providing a sample of unstained cells having a wide range of autofluorescence wavelengths to an imaging system, illuminating the sample with an illumination source, and imaging the sample over a dynamic range that includes all of the wide range of fluorescence wavelengths and all of the wide range of autofluorescence wavelengths.

[0028] In one embodiment of the method, the minimum photon collection time of the imaging system is set separately for each wavelength in the dynamic range, such that the minimum photon collection time is sufficient for detection of all values ​​within the complete dynamic range.

[0029] In one embodiment of the method, the dynamic range includes all wavelength imaging system images.

[0030] In one embodiment, the method further includes performing a calibration for chromatic aberration on the imaging system before acquiring the sample image, the calibration including illuminating the cellular sample with an illumination source, acquiring a first image of the cellular sample from the imaging system at a first position, moving the imaging system to a second, different position relative to the cellular sample, acquiring a second image of the cellular sample, and positioning the imaging system at the first position if the first image is optimized compared to the second image in generating the sample image, and otherwise positioning the imaging system.

[0031] In one embodiment of the method, the imaging system comprises a digital grayscale camera equipped with a fluorophore-based filter set.

[0032] In one embodiment of the method, the imaging system comprises a high dynamic range imaging (HDRI) camera.

[0033] In one embodiment of the method, the imaging system generates high dynamic range (HDR) through exposure fusion to provide improved contrast.

[0034] In one embodiment of the method, the calibrated images are used to train a neural network. [Brief explanation of the drawings]

[0035] [Figure 1] 1 shows a flowchart of one embodiment of a method for automated image processing that can be used to prepare images for cell segmentation. [Figure 2] The sensitivity curve of a typical CCD chip is shown below. [Figure 3A] 1 shows a topological representation of two adjacent cells, where image acquisition was calibrated according to the first calibration. [Figure 3B]3B shows a topological representation of two adjacent cells from FIG. 3A without image acquisition calibrated according to the first calibration. [Figure 4A] Shows difficult-to-segment areas of FCM imaged with a standard low-dynamic-range camera. [Figure 4B] Shows the difficult-to-segment area of ​​Figure 4A imaged with a high dynamic range imaging (HDRI) camera. [Figure 5A] This shows the histogram of a low dynamic range camera, specifically a microscope camera with 8-bit dynamic range. [Figure 5B] 5B shows the histogram of a high dynamic range imaging (HDRI) camera for the same image as FIG. 5A. [Figure 6A] Nuclear stained liver sections are shown before subtraction of the corresponding autofluorescence images. [Figure 6B] The liver section from FIG. 6A is shown after subtraction of the corresponding autofluorescence image. [Figure 7] A portion of the image is shown that is particularly difficult to segment due to variations in shape and staining intensity. [Figure 8] 2D topological curves showing the identification of seeds and cells with different "water levels" using the inverse watershed methodology. [Figure 9] 1 shows a portion of an image in which seeds have been identified. DETAILED DESCRIPTION OF THE INVENTION

[0036] FIG. 1 provides a flowchart of one embodiment of a method for automated, unsupervised, parameter-free segmentation of single cells and other objects in images generated by a fluorescence microscope. This method would typically be performed on digital images by a computer or processor executing appropriate software stored in memory. However, in alternative embodiments, the method may be implemented through electromechanical hardware, such as, but not limited to, circuitry. The system and method would typically be provided as part of or in conjunction with the operation of an automated microscope equipped with a digital image sensor, such as a charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS). A computer in combination with such software or hardware designed for such comprises one embodiment of a system of the present invention, as do such elements in conjunction with other elements of cell or other biological tissue analysis.

[0037] Throughout this disclosure, the term "computer" generally describes hardware that implements functionality provided by digital computing technology, particularly computing functions associated with microprocessors. The term "computer" is not limited to any particular type of computing device, but is intended to include all computational devices, including, but not limited to, processing devices, microprocessors, personal computers, desktop computers, laptop computers, workstations, terminals, servers, clients, portable computers, handheld computers, smartphones, tablet computers, mobile devices, server farms, hardware appliances, minicomputers, mainframe computers, video game consoles, handheld video game products, and wearable computing devices, including, but not limited to, eyewear, wristwear, pendants, and clip-on devices.

[0038] As used herein, a "computer" should be an abstraction of the functionality provided by a single computing device equipped with hardware and accessories characteristic of a particular role of the computer. By way of example and not limitation, the term "computer" in the context of a laptop computer will be understood by those skilled in the art to include functionality provided by a pointer-based input device such as a mouse or trackpad, while the term "computer" used in the context of an enterprise-class server will be understood by those skilled in the art to include functionality provided by redundant systems such as RAID drives and dual power supplies.

[0039] It is also well known to those skilled in the art that the functionality of a single computer may be distributed across several individual machines. This distribution may be functional, as in the case where a particular machine performs a specific task, or it may be balanced, as in the case where each machine is capable of performing most or all of the functions of any other machine, and is assigned tasks at a given time based on its available resources. Thus, as used herein, the term "computer" may refer to a single, standalone, self-contained device, or multiple machines operating together or independently, including, but not limited to, a network server farm, a "cloud" computing system, software as a service, or other distributed or collaborative computer network.

[0040] Those skilled in the art will also understand that some devices not traditionally thought of as "computers" nevertheless exhibit "computer" characteristics in certain contexts. To the extent that such devices perform the functions of a "computer" described herein, the term "computer" includes such devices to that extent. This type of device includes, but is not limited to, network hardware, print servers, file servers, NAS and SAN, load balancers, and any other hardware capable of interacting with the systems and methods described herein for a traditional "computer."

[0041] Throughout this disclosure, the term "software" refers to code objects, program logic, command structures, data structures and definitions, source code, executable and / or binary files, machine code, object code, compiled libraries, implementations, algorithms, libraries, or any instructions or sets of instructions that are executable by a computer processor or that can be converted into an executable form by a computer processor, including, but not limited to, a virtual processor, or through the use of a run-time environment, virtual machine, and / or interpreter. Those skilled in the art will recognize that software may be hardwired or embedded in hardware, including, but not limited to, a microchip, and still be considered "software" within the meaning of this disclosure. For purposes of this disclosure, software includes, but is not limited to, instructions stored or storable in RAM, ROM, flash memory, BIOS, CMOS, mother and daughter board circuitry, hardware controllers, USB controllers or hosts, peripheral devices and controllers, video cards, audio controllers, network cards, Bluetooth and other wireless communication devices, virtual memory, storage devices and associated controllers, firmware, and device drivers. The systems and methods described herein are typically contemplated to employ computers and computer software stored on computer or machine-readable storage media or memory.

[0042] Throughout this disclosure, terms used herein to describe or refer to medium-bearing software, including but not limited to terms such as "medium," "storage medium," and "memory," may include or exclude ephemeral media such as signals and carrier waves.

[0043] Throughout this disclosure, the term “real-time” generally refers to software performance and / or response time within an operational timeframe that generally occurs virtually simultaneously with a reference event in a typical user's perception of the passage of time for a particular operational context. Those skilled in the art will appreciate that “real-time” does not necessarily mean that a system performs or responds immediately or instantaneously. For example, those skilled in the art will appreciate that when the operational context is a graphical user interface, “real-time” typically means a response time of approximately one second, with milliseconds or microseconds being preferred, being the actual time for at least some manner of response from the system. However, those skilled in the art will also appreciate that under other operational contexts, a system operating in “real-time” may exhibit delays longer than one second, such as when multiple devices and / or additional processing on or between particular devices, or network operations that may include multiple point-to-point round trips for data exchange between devices, are involved. Those skilled in the art will further appreciate the distinction between “real-time” execution by a computer system as compared to “real-time” execution by a human or multiple humans. Performing a particular method or function in real time may be impossible for a human, but possible for a computer. Even if a human or humans could ultimately produce the same or similar output as a computerized system, the time required would be longer than the time a consumer of the output would be willing to wait for the output, rendering the output worthless or meaningless, or the number and / or complexity of the calculations would cause the commercial value of the output to exceed the cost of producing it.

[0044] US Patent Application No. 15 / 708,221 is also relevant to the discussion herein, and the definitions therein are specifically incorporated by reference as the definitions of these terms as used herein.

[0045] This image analysis is typically performed in real time by a computer so that the results of the analysis can be readily used in both research and diagnostic or clinical settings, and for the selection of treatments for diseases indicated through image analysis. Furthermore, the process is typically automated, whereby the image analysis can be performed with either minimal or no human intervention. Specifically, the image analysis is typically performed by a computer performing the acts of acquiring images and / or evaluating the images for segmentation of cells within the images, without the need for a human to assist in the analysis. In most cases, the automated system will be further combined with other automated systems that can utilize the segmented images to provide a human user with further evaluation of the cells of interest, although this is by no means required.

[0046] The present systems and methods, in one embodiment, combine systems and methods for improving both the process of image acquisition and image processing (to provide improved input data for the image processing step) in the nature of cell segmentation. However, those skilled in the art will recognize that the image acquisition discussed herein can be used to provide improved images to traditional image processing systems and methods, and in alternative embodiments, traditional image acquisition systems can be used to provide data to the present image processing systems.

[0047] The image processing elements discussed herein are typically designed to handle high heterogeneity in input data associated with cell segmentation. Heterogeneity within a sample may be caused, for example, by different sample sources of underlying cells (e.g., brain, liver, spleen), sample quality, staining quality, or other factors. In other embodiments, the systems and methods herein can be used to evaluate images of subjects other than cells, which may have high or low heterogeneity. For example, the systems for image acquisition discussed herein can be used to improve the signal-to-noise ratio for nearly any fluorescence-based image. Furthermore, image segmentation need not provide for the detection of individual living cells, but may be used to detect other subcomponents of an image for which segmentation is determined to be useful. However, for ease of discussion, the present disclosure utilizes the imaging of cells (specifically living cells) and the segmentation of those images to detect individual cells as an exemplary embodiment.

[0048] As shown in Figure 1, the method and operation of the system typically first involves obtaining a sample of cells (101), which will typically be immobilized on a solid substrate. This may be performed by any method known to those of skill in the art, and all such known methods are incorporated herein by reference. U.S. Patent Application Nos. 15 / 708,221 and 13 / 126,116, the disclosures of which are incorporated herein by reference in their entireties, provide example embodiments of how such immobilization and sample preparation may be performed.

[0049] Once the sample is acquired (101), it is exposed to an image acquisition system (103). The image acquisition system will typically be a digital grayscale camera, and the camera will typically have a filter set selected based on the fluorophores used for imaging. The camera will typically be in the form of a high dynamic range imaging (HDRI) camera or a software algorithm that controls the camera to generate HDR through exposure fusion to provide improved contrast. The systems and methods discussed herein can utilize cameras in which HDRI is provided using multiple exposures or a single exposure, depending on the camera selected and the particular embodiment of the system and method being utilized.

[0050] The image acquisition system may first be calibrated (105), typically in two calibration actions (201) and (203). However, calibration (105) does not necessarily have to be performed during or coextensive with image acquisition (107). Alternatively, calibration (105) may be performed during system setup or maintenance, each day before imaging begins, or when needed. Thus, decision (104) may be made to either calibrate (105) the system or proceed directly to image acquisition (107). Furthermore, calibration (105) may be performed without a subsequent image acquisition (107), a scenario in which the system may typically cease operation after element (203).

[0051] In the first calibration (201), the image acquisition system is calibrated for autofluorescence of unstained cells and any inherent background signal using any light source that will be used during actual imaging of the stained sample. In one embodiment, the minimum photon collection time is set separately for each wavelength within the full dynamic range of the autofluorescence plus fluorescently labeled detector to provide sufficient signal for detection of all values. In other words, the imaging dynamic range can be selected to ensure that it includes any wavelength detectable by the camera and any autofluorescence or background signal detected at the specific wavelength for the fluorophore selected for staining.

[0052] This type of first calibration (201) is generally used to compensate for differences in sensitivity of the image acquisition system with respect to different wavelengths that the image acquisition system can detect and that may be present. The first calibration typically allows for determining what is background and what is autofluorescence of the sample based on current measurements, thus reducing the reliance on signal size to determine whether a signal is of interest or noise.

[0053] As an example, Figure 2 provides a sensitivity curve (401) for a typical CCD chip that may be used in an image acquisition system. In the chip of Figure 2, longer exposure times may be used at lower and higher wavelengths to ensure that the lower end of the dynamic range of the image acquisition system is capable of capturing the autofluorescence of any cells present in the sample within the dynamic range of the camera for each acquisition wavelength used. It should be recognized that any particular imaging run of the system or method may utilize any or all of the camera's available range. This first calibration (201) will typically be performed for all wavelengths (or a subgroup of wavelengths selected depending on sensitivity and time constraints) used in the acquisition run of this particular sample, referred to as the dynamic range.

[0054] In the second calibration (203), the image acquisition system is calibrated for chromatic aberration. Chromatic aberration (commonly referred to as the "rainbow effect") results in photons from a single location on the specimen projecting to different locations on the camera, depending on the wavelength of the photons and lens and / or chip design. If not compensated for, the x-, y-, and z-axis offsets between different wavelengths lead to blurring or offsets in the image, thus reducing the quality of the input data for the segmentation process. The second calibration (203) is typically performed by taking uncalibrated images in the first round. These images may be taken from stained or unstained specimens, depending on the embodiment. In the second and subsequent rounds, the x-, y-, and z-positions of the imager are changed while the specimen is held constant. The positional shift is typically a stepwise or similarly repeatable method to find the optimal x-, y-, and z-offsets between a single filter set used for different wavelengths. This calibration will be used during the actual imaging run.

[0055] Figures 3A and 3B show an example of how the second calibration (203) can improve image quality. Figure 3A shows two adjacent cells (501) and (503) as peaks in a topological representation (601) (with height as staining intensity) after the image acquisition system has been calibrated as contemplated by the second calibration. Figure 3B shows the same two adjacent cells (501) and (503) without the second calibration, illustrating the blurring caused by offset imaging. As can be seen, the blurring makes it more difficult to distinguish between two different cells.

[0056] After the second calibration (203) is complete, the image acquisition system will typically be considered calibrated and ready to begin image acquisition (107). Image acquisition (107) typically includes at least two, and possibly more, acquisition passes, followed by image correction (109) to remove artifacts from autofluorescence and background signals. The first acquisition pass (301) will generally be performed on the sample prior to staining.

[0057] The first collection pass (301) will typically be performed in a dynamic range that covers the complete dynamic range of autofluorescence at all wavelengths imaged, and thus throughout the dynamic range of the camera selected for this imaging. As will be apparent, this dynamic range will generally correspond to the dynamic range over which the first calibration (201) was performed. The first image collection (301) will also typically be performed using the same input light or lights that will be used in conjunction with subsequent collections.

[0058] After this first pass (301) is complete, the sample will be exposed to a fluorophore (typically in the form of a detection conjugate having a binder moiety and a fluorescent dye moiety) and stained as desired. The second pass (303) will essentially involve repeating the steps of the first image collection pass on the just-stained sample. Collection will typically be performed over a dynamic range that covers the full dynamic range of autofluorescence plus biomarker expression revealed by the fluorescently labeled detector (e.g., antibody, aptamer). For simplicity, the dynamic range of both the first image collection pass (301) and the second image collection pass (303) may be the same, although it should be appreciated that the first image collection pass (301) may utilize a smaller dynamic range since it is only seeking autofluorescence.

[0059] Figures 4A and 4B show examples of difficult-to-segment areas captured with a low-dynamic-range camera (Figure 4A) and an HDRI camera (Figure 4B). Histograms generated from the images of Figures 4A and 4B are shown in Figures 5A and 5B, revealing that only the HDRI image of Figure 4B can cover the full dynamic range of the fluorescence emitted by the sample, whereas a standard microscope camera with a typical 8-bit dynamic range can only capture values ​​with a dynamic range of approximately 200. This lack of dynamic range in Figure 4A results in a less sharp image, which creates artifacts and reduces the ability to segment cells within the image.

[0060] After the second image acquisition (303) is complete, the first pass image (301) will typically be subtracted from the second (and any other subsequent) pass image (303) in compensation (109). Methodologies for doing this are discussed in the above-referenced U.S. patent application Ser. No. 15 / 708,221, incorporated herein by reference. Subtraction helps eliminate illumination artifacts from the optical system that appear in the image and also reduces autofluorescence signals that obscure signals generated by the staining itself. Figure 6A shows a liver section with stained nuclei before compensation (109), and Figure 6B shows the same section after compensation (109) has been performed.

[0061] After correction (109) is complete, the image is typically considered optimized for single object or cell segmentation. Segmentation (111) can be performed using any system or method known to those skilled in the art. However, due to the high variability in object shape, sample, and staining quality, robust algorithms or machine learning approaches are typically preferred to ensure high sensitivity and specificity of object recognition. Figure 7 shows an image that is particularly difficult to segment due to the variability in shape and staining intensity.

[0062] In one embodiment, calibrated and optimized high dynamic range input image data can be acquired from a wide variety of sources using the systems and methods discussed in FIG. 1 , including correction (109). These images can then be used to train a neural network (or similar processing system) using supervised learning and backpropagation to recognize single objects and object boundaries in a manner known to those skilled in the art. Because the generation of such high-quality images aids in the elimination of artifacts and differences between sources, the images provided can be from heterogeneous sources with wide dynamic ranges and fluorescence characteristics. Alternatively, the neural network (or similar) can be provided with images from a more homogeneous source (e.g., only one type of cell), if desired. This neural network can then, in one embodiment, be used to perform segmentation (111).

[0063] In the embodiment of Figures 7 and 8, segmentation (111) is performed on the image using a form of the inverse watershed transform referred to herein as circular seed detection or "falling water level" analysis. Seed detection in conjunction with the traditional watershed transform is discussed, for example, in U.S. Patent No. 8,995,740, the entire disclosure of which is incorporated herein by reference.

[0064] In the circular seed detection system and method used herein, a topology analysis is performed similar to a watershed transform, in which the intensity of the fluorescence is used to indicate the "height" of each pixel, and thus peaks in the topology typically indicate target elements of the associated cell (e.g., cell nuclei). This is also contemplated in Figures 3A and 3B above. However, the watershed transform is prone to errors for objects such as "atolls" or "erupting volcanoes" (which may be objects like the seed (811) in Figure 8). Objects such as these may be generated, for example, by the quality of the staining or self-effacement. In fact, traditional watershed transforms require a perfect "volcano" shape to function.

[0065] The inverse watershed transform as contemplated herein works to slowly remove water instead of flooding the topology as in traditional watershed transforms. Figure 8 provides an example 2D topological curve from a portion of an image to illustrate the operation of the circular seed detection or inverse watershed transform process. First, a first intensity (801) is selected, which is the "high water mark." Any pixel group identified as above this intensity (801) will serve as a seed (811) and will be identified as belonging to an individual cell. Figure 9 provides an example of a seed (711) in an image.

[0066] Returning to FIG. 8 , a second, lower intensity (803) is then selected, and the process is repeated to identify an additional seed (813) that has just risen "above" the new "water level" (803) of this lower intensity (803). This reduction in intensity and search for a new seed would typically be repeated with the new, lower level selected in a consistent, stepwise fashion. In the embodiment of FIG. 8 , two additional levels (805) and (807) are selected, representing new seeds (815) and (817), respectively. FIG. 8 also shows level (809), which may correspond to the initial read of a more traditional watershed transformation. This level may produce a single seed (819) instead of three seeds (811), (813), and (815).

[0067] Each of these seeds (811), (813), (815), and (817) is presumed to have an associated cell whose boundary needs to be detected. Therefore, the seeds can be used as a source for determining the extent of the cells belonging to seeds (811), (813), (815), and (817). This is typically done by tracing the convex boundaries of the topological curves away from seeds (811), (813), (815), and (817) and attempting to find the outer edge of the "island" by searching for inflection points where the curvature decreases on the convex side. In Figure 8, this results in the segmentation of the left portion of the image into four target cells.

[0068] Although the present invention has been disclosed in connection with certain specific embodiments, this should not be considered as being limited to all of the details provided. It should be understood that modifications and variations of the described embodiments can be made without departing from the spirit and scope of the invention, and that other embodiments are encompassed by the present disclosure as can be appreciated by those skilled in the art.

[0069] Furthermore, it will be understood that any of the ranges, values, properties, or characteristics given for any single component of this disclosure, as provided throughout this specification, may, where compatible, be used interchangeably with any ranges, values, properties, or characteristics given for any of the other components of this disclosure to form an embodiment having the defined values ​​for each of the components. Furthermore, ranges provided for a genus or category may also apply to the species or members of the category within the genus, unless otherwise specified.

[0070] Finally, the modifier "generally" and similar modifiers used herein will be understood by those skilled in the art to accommodate discernible attempts to fit devices to the modified term, which may nevertheless not fit. This is because terms such as "circular" are purely geometric constructs, and real-world components are not truly "circles" in the geometric sense. Variations from geometric and mathematical descriptions are inevitable due to, among other things, manufacturing tolerances that result in shape variations, defects and imperfections, non-uniform thermal expansion, and natural wear. Furthermore, for each object, there exists a level of magnification at which geometric and mathematical descriptors fail due to the nature of matter. Thus, those skilled in the art will understand that the term "generally" and the relationships contemplated herein, regardless of the inclusion of such modifiers, encompass various variations from the literal geometric meaning of the term, taking these and other considerations into account.

Claims

1. 1. A method for analyzing a cell image, the method comprising: providing a sample of cells tagged with fluorophores having a wide range of fluorescence wavelengths; illuminating the sample with an illumination source; imaging the illuminated sample across the broad range of fluorescent wavelengths to generate a sample image; subtracting the calibration image from the sample image to create a calibrated image; representing the image as a topological curve, the height of the curve at each curve pixel representing the intensity of fluorescence at that image pixel in the calibrated image; searching the topological curve for groups of pixels having a height greater than a selected height; choosing a new height that is lower than the selected height; repeating the searching using the new height as the selected height; and For each identified pixel group, following a convex boundary of the topological curve away from the group of pixels to an inflection point where curvature decreases on the convex side; and identifying the convex boundary, wherein the group of pixels is included as a cell in the calibrated image.

2. The calibration image is providing a sample of unstained cells having a wide range of autofluorescence wavelengths to the imaging system; illuminating the sample with an illumination source; imaging the sample over a dynamic range that includes all of the broad fluorescence wavelengths and all of the broad autofluorescence wavelengths.

3. a minimum photon collection time of the imaging system is set separately for each wavelength within the dynamic range; The method of claim 2 , wherein the minimum photon collection time is sufficient for detection of all values ​​within the complete dynamic range.

4. The method of claim 2 , wherein the dynamic range includes all wavelengths of the imaging system image.

5. and performing a calibration for chromatic aberration on the imaging system prior to acquiring the sample image, the calibration comprising: illuminating the sample of cells with an illumination source; acquiring a first image of the cell sample from the imaging system at a first location; moving the imaging system to a second, different position relative to the sample of cells; acquiring a second image of the sample of cells; and When generating the sample image, positioning the imaging system at the first position when the first image is optimized compared to the second image; and If not, disposing the imaging system.

6. The method of claim 1 , wherein the imaging system comprises a digital grayscale camera equipped with a filter set based on the fluorophore.

7. The method of claim 1 , wherein the imaging system comprises a high dynamic range imaging (HDRI) camera.

8. The method of claim 1 , wherein the imaging system generates high dynamic range (HDR) by exposure blending to provide improved contrast.

9. The method of claim 1 , wherein the calibrated images are used to train a neural network.

Images