System and method for calculating autofluorescence contribution in multi-channel images

Abstract

Disclosed herein are systems and methods for estimating autofluorescence (AF) signals and other non-target signals in each channel of a multi-channel image of a biological sample stained with one or more fluorescent markers. In some embodiments, the estimated autofluorescence signal can then be subtracted or masked from the multi-channel image. In some embodiments, the multi-channel image with autofluorescence removed can then be used for further processing (e.g., image analysis, etc.).

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of the filing date of U.S. Provisional Patent Application No. 62 / 805,239, filed on February 13, 2019, the disclosure of which is incorporated herein by reference in its entirety. Background Art

[0003] Fluorescence imaging of biological cells and tissue samples is used to visualize the presence and expression level of specific antigens, such as using probes conjugated with antibodies and fluorescent dyes. Multiple proteins in a given tissue section can be visualized using probes targeting the specific antigen of interest and one or more histological dyes (e.g., DAPI, a nuclear counterstain). Other targets (e.g., RNA or DNA) can be visualized using fluorescent in situ hybridization and oligonucleotide-labeled fluorescent probes, respectively.

[0004] Fluorescence imaging of a dye involves exciting it with light of a first wavelength band or range of wavelengths and observing the light it emits in response to that excitation, for example, light emitted in a second wavelength band or range of wavelengths. The propensity of a fluorescent dye to emit light in response to excitation of a given wavelength is called its excitation spectrum. The distribution of wavelengths of fluorescent light emitted by a dye is called its emission spectrum.

[0005] When multiple dyes are used, they are typically selected to have different excitation spectra, emission spectra, or both, so that by careful selection of the excitation and emission wavelengths used, the dye being observed can be distinguished. Determining the dye associated with the observed emission becomes more difficult when the spectra of the various fluorescent dyes are not completely distinct in their excitation and emission spectra but rather overlap substantially.

[0006] Many samples exhibit intrinsic fluorescence emission. This means that when excited by light, the sample itself emits fluorescence in addition to the fluorescence emitted by the fluorescent dye used in conjunction with the antibody-conjugated probe or as a counterstain. This can further complicate the above determinations.

[0007] Multispectral imaging of fluorescent samples involves acquiring a series of images of the sample at different excitation wavelengths, emission wavelengths, or a combination of both. The various images are combined into an image cube, where two dimensions of the cube correspond to the spatial position in the sample and the third dimension corresponds to the spectrum associated with the various excitation and / or emission wavelengths. Summary of the invention

[0008] The present disclosure provides systems and methods for estimating autofluorescence (AF) signals and other non-target signals within each channel of a multi-channel image of a biological sample (e.g., a tissue section) stained with multiple fluorescent markers. In some embodiments, the estimated autofluorescence signal can be subtracted and / or masked from the multi-channel image. In some embodiments, the multi-channel image with autofluorescence removed can be used for further processing (e.g., image analysis, etc.).

[0009] Generally speaking, in order to obtain image data, tissue sections are processed so that target biomarkers (e.g., proteins, RNA fragments, or DNA fragments) are labeled with different fluorophores. When excited with light of an appropriately selected excitation spectrum, each type of fluorophore emits light of a specific emission spectrum (or color). In some embodiments, in addition to the target signal generated by the different fluorophores, the acquired image may also include non-target signals. It is believed that the main source of such non-target signals is autofluorescence, i.e., light emitted by the intrinsic fluorescence of molecules contained in the tissue. In some embodiments, other sources of non-target signals include red blood cells, dead cells, and reagents that assist in the slide making process, such as a sealing agent for fixing a coverslip to a slide with a tissue section.

[0010] Fluorescence multiplexing approaches have great potential to extract information about the expression, co-expression, and co-localization of multiple biomarkers in the context of tissue architecture, thereby preserving the spatial information and interrelationships of the biomarkers. Fluorescence multiplexing assays require imaging of multiple fluorescent markers, making imaging an essential component of the workflow. Different imaging modalities have been used for imaging and subsequent analysis of fluorescence multiplexing assays. Two main approaches involve the following steps:

[0011] (i) Detection using a multispectral camera. In this configuration, spectral information from multiple panels of the target is captured by a multispectral camera or detector, resulting in a multispectral image cube. The signal detected in each spectral channel is then a linear combination of the signals from all fluorophores and the AF.

[0012] (ii) Use filter cubes (excitation and emission filter sets) corresponding to each fluorophore and detect FL emission using a monochrome camera. This produces a multi-channel image of the sample, where each channel corresponds to one fluorophore. The excitation and emission filters are selected in a way that minimizes FL crosstalk caused by simultaneous detection of FL signals from more than one fluorophore within a single imaging channel. Nevertheless, it is believed that there will be some crosstalk between the fluorophores detected in the multi-channel image, which needs to be removed by subsequent analysis in order to accurately assess the intensity of each fluorophore.

[0013] In both methods, the fluorescence signal detected in each spectral channel comes from multiple fluorophores plus autofluorescence signals from tissue, mounting media, and slides. In some embodiments, linear spectral unmixing is used to separate the contributions of different fluorophores to the signal detected in each spectral channel. In order to reliably unmix and quantify spectral information, it is necessary to obtain a correct instance of the emission spectrum of each fluorophore, as well as a representative autofluorescence spectrum from an unstained sample (in the background to be used). Linear spectral unmixing requires a minimum number of spectral channels, which is equal to the number of all fluorescence sources (i.e., fluorophores plus autofluorescence).

[0014] The first imaging modality described above typically captures spectral information in a large number of spectral channels and therefore meets the requirement for a minimum number of spectral channels for linear spectral unmixing. However, the cost of such multispectral detection imaging systems, longer image acquisition times, and higher memory requirements for storing multispectral images may be factors that prevent the use of such modalities. If such imaging modalities are used, it is still desirable to capture images in a smaller number of detection channels to reduce acquisition time and storage space requirements.

[0015] In the case of a second imaging modality using one filter cube corresponding to each fluorophore, adding an additional filter cube to capture the autofluorescence signal may not always be feasible or effective. In this case, an analysis method is needed to estimate the autofluorescence signal based on a multi-channel image of the sample, where each channel corresponds to a fluorophore.

[0016] In both modalities, if the number of image channels is equal to or greater than the number of fluorescent markers in the sample, a linear spectral unmixing method can be used to unmix the contributions of fluorescent markers and autofluorescence to the fluorescence signal at each pixel of the multi-channel image. Due to acquisition time and image storage space considerations, an analysis method that can estimate the autofluorescence signal from a multi-channel or multispectral image using a smaller number of image channels is considered advantageous. In some embodiments, the systems and methods described herein advantageously allow for faster image acquisition times, reduced image storage requirements, and less fluorescence excitation of stained tissue during image acquisition. Since fluorophores are subject to photobleaching, which means that the yield of emitted fluorescence decreases over time and with each excitation, if a stained slide needs to be rescanned, acquisition with fewer excitations allows for more reproducible image acquisition. In some embodiments, the systems and methods presented herein can be applied to image acquisition with a minimum number of acquisition channels, particularly for acquisition using matched filters, where only one filter set is used for each target fluorophore signal. In view of the foregoing, in some embodiments, the present disclosure relates to systems and methods for determining the autofluorescence contribution within each channel of a multi-channel image of a biological sample stained with multiple fluorescent markers. In some embodiments, autofluorescence is removed while retaining as much of the specific fluorescent marker as possible.

[0017] One aspect of the present disclosure is a method for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the method comprising: independently calculating the autofluorescence contribution in each channel of each pixel of the multi-channel image based on one or more retrieved autofluorescence spectra, and subtracting the calculated autofluorescence contribution in each channel for the corresponding pixel from each measured pixel spectrum of the multi-channel image. In some embodiments, the autofluorescence contribution in each channel of each pixel of the multi-channel image is calculated from a single retrieved autofluorescence spectrum. In some embodiments, the single retrieved autofluorescence spectrum is derived from a measured signal corresponding to an unlabeled portion of a training multi-channel image of a biological sample containing one or more fluorescent markers. In some embodiments, the autofluorescence contribution in each channel of each pixel is calculated by independently fitting the single retrieved autofluorescence spectrum to each measured spectrum of the pixel of the multi-channel image. In some embodiments, fitting comprises (i) dividing the measured spectrum of the pixel of the multi-channel image by the single retrieved autofluorescence spectrum to provide a normalized measured spectrum of the pixel; (ii) calculating the minimum of the normalized measured spectrum of the pixel; and (iii) scaling the calculated minimum by the single retrieved autofluorescence spectrum.

[0018] In some embodiments, an autofluorescence contribution in each channel of each pixel of the multi-channel image is calculated from at least two retrieved autofluorescence spectra. In some embodiments, the at least two retrieved autofluorescence spectra are derived from an assay signal corresponding to an unlabeled portion of a training multi-channel image of a biological sample comprising one or more fluorescent markers. In some embodiments, the calculated autofluorescence contribution in each channel of each pixel of the multi-channel image is a maximum autofluorescence correction value independently identified for each channel of each pixel of the multi-channel image. In some embodiments, the maximum autofluorescence correction for each channel of each pixel of the multi-channel image is identified by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of the at least two retrieved autofluorescence spectra to the assay spectrum of the pixel; (ii) determining in each channel the maximum autofluorescence intensity value of the calculated at least two estimated autofluorescence corrections. In some embodiments, the at least two retrieved autofluorescence spectra include at least two autofluorescence low-high spectrum pairs.

[0019] Another aspect of the present disclosure is a system for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the system comprising: (i) one or more processors, and (ii) one or more memories, the one or more memories being coupled to the one or more processors, the one or more memories storing computer executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising the following steps: (a) independently calculating a maximum autofluorescence correction in each channel for each pixel of the multi-channel image, wherein the calculated maximum autofluorescence correction in each channel for each pixel is derived from at least two different retrieved autofluorescence spectra; and (b) subtracting the calculated maximum autofluorescence correction in each channel for each measured pixel of the multi-channel image from the spectrum of the pixel.

[0020] In some embodiments, the maximum autofluorescence correction for each channel for each pixel is calculated by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of the at least two retrieved autofluorescence spectra to the measured spectrum of the pixel; (ii) identifying the maximum autofluorescence intensity value of the calculated at least two estimated autofluorescence corrections in each channel. In some embodiments, each independent fit includes (i) dividing the measured spectrum of the pixel of the multi-channel image by one of the at least two retrieved autofluorescence spectra to provide a normalized measured spectrum for the pixel; (ii) calculating the minimum value of the normalized measured spectrum for the pixel; and (iii) scaling the calculated minimum value by one of the at least two retrieved autofluorescence spectra.

[0021] In some embodiments, the at least two retrieved autofluorescence spectra include at least two autofluorescence low-high spectrum pairs. In some embodiments, each independent fit includes (i) dividing a measured spectrum of a pixel of the multi-channel image by a first autofluorescence low spectrum of one of the at least two autofluorescence low-high spectrum pairs; (ii) calculating a minimum of the normalized measured spectrum of the pixel; and (iii) scaling the calculated minimum by the first autofluorescence high spectrum of a corresponding one of the at least two autofluorescence low-high spectrum pairs. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] For a general understanding of the features of the present disclosure, reference is made to the drawings, in which like reference numerals are used throughout to identify like elements.

[0023] Figure 1A A representative digital pathology system including an image acquisition device and a computer system according to some embodiments is shown.

[0024] Figure 1B Various modules that may be used in a digital pathology system or a digital pathology workflow according to some embodiments are listed.

[0025] Figure 2A A flow chart is listed showing the various steps for generating a multi-channel image with autofluorescence removed according to some embodiments.

[0026] Figure 2B A flow chart is listed showing the steps for calculating the autofluorescence contribution in each channel of a measured pixel spectrum using a single autofluorescence spectrum in accordance with some embodiments.

[0027] Figure 2C A flow chart is listed showing various steps for calculating the autofluorescence contribution in each channel of a measured pixel spectrum using two or more different autofluorescence spectra in accordance with some embodiments.

[0028] Figure 2D Depicted showing Figure 2C The output of each step of the method is depicted in the workflow.

[0029] Figure 2E A flow chart is listed showing various steps for calculating the autofluorescence contribution in each channel of a measured pixel spectrum using two or more autofluorescence spectra according to some embodiments.

[0030] Figure 2F Depicted showing Figure 2E The output of each step of the method is depicted in the workflow.

[0031] Figure 3ARegions within the multi-channel image corresponding to autofluorescence are shown.

[0032] Figure 3B shows a multi-channel image with autofluorescence removed, where Figure 3A The identified unwanted autofluorescence has been removed.

[0033] Figure 4 The measured pixel spectrum, the retrieved autofluorescence spectrum and the fitted autofluorescence spectrum are shown.

[0034] Figure 5A The measured pixel spectrum and the first and second retrieved autofluorescence spectra are shown.

[0035] Figure 5B The measured pixel spectrum, the first and second fitted autofluorescence spectra, and the maximum fitted autofluorescence spectrum are shown.

[0036] Fig. 6A High and low spectral pairs derived from the spectral grouping are shown.

[0037] Figure 6B Another example of high and low spectral pairs derived from spectral grouping is shown.

[0038] Figure 6C A number of autofluorescence spectra, obtained, for example, from a training image and before grouping, are shown.

[0039] Fig.6D Shows from Figure 6C Several different groupings of the spectral derived autofluorescence spectra.

[0040] Fig. 6E Shown are the grouping of autofluorescence spectra (upper left panel) and the spectral pairs derived therefrom (lower left panel).

[0041] Fig. 6F Shown are the grouping of autofluorescence spectra (upper left panel) and the spectral pairs derived therefrom (lower left panel).

[0042] Figure 6G A multi-channel image with autofluorescence removed is provided, wherein the calculated autofluorescence contribution (e.g., from Fig. 6E and Fig. 6F The spectra of the derived ones) are removed.

[0043] Fig. 7A Shown are multi-channel images before staining with multiple fluorescent molecules.

[0044] Figure 7B It shows that through Fig. 7A Exclusion mask derived by spectral multi-thresholding of the signal within.

[0045] Figure 7C Shows from Fig. 7A The exported inclusion mask.

[0046] Fig.7D Shown in the application Figure 7B and Figure 7C Multichannel image after removing autofluorescence using the exclusion mask and inclusion mask. DETAILED DESCRIPTION

[0047] It should also be understood that, unless indicated to the contrary, in any method claimed herein that includes more than one step or action, the order of the steps or actions of the method is not necessarily limited to the order in which the steps or actions of the method are described.

[0048] As used herein, the singular forms "a", "an", and "the" include plural referents unless the context clearly indicates otherwise. Similarly, the word "or" is intended to include "and" unless the context clearly indicates otherwise. The term "comprising" is defined as inclusive, such as "comprising A or B" means comprising A, B, or A and B.

[0049] As used herein in the specification and claims, "or" shall be understood to have the same meaning as "and / or" defined above. For example, when separating items in a list, "or" or "and / or" shall be interpreted as being inclusive, i.e., including at least one element from a number of elements or a list of elements, but also including more than one element, and optionally including additional unlisted items. Only terms indicating the contrary, such as "only one" or "exactly one", or "consisting of..." as used in the claims, will refer to the inclusion of exactly one element from a number of elements or a list of elements. In general, the term "or" as used herein shall be interpreted as indicating an exclusive alternative (i.e., "one or the other, but not both") only when preceded by an exclusive term such as "or", "one of", "only one", or "exactly one". "Substantially consisting of..." as used in the claims shall have the ordinary meaning used in the field of patent law.

[0050] The terms "including", "comprising", "having", etc. are used interchangeably and have the same meaning. Similarly, "including", "comprising", "having", etc. are used interchangeably and have the same meaning. Specifically, the definition of each term is consistent with the definition of "including" under ordinary U.S. patent law, so each term can be understood as an open term, which means "at least the following" and can also be interpreted as not excluding additional features, limitations, aspects, etc. Therefore, for example, "a device having components a, b and c" means that the device includes at least components a, b and c. Similarly, the phrase: "a method involving steps a, b and c" means that the method includes at least steps a, b and c. In addition, although the steps and processes may be outlined in a particular order herein, those skilled in the art will recognize that the order steps and processes may vary.

[0051] As used herein in the specification and claims, with respect to a list of one or more elements, the phrase "at least one" should be understood as at least one element selected from any one or more elements in the list of elements, but does not necessarily include at least one of each element specifically listed in the list of elements, nor does it exclude any combination of elements in the list of elements. In addition to the elements specifically identified in the list of elements to which the phrase "at least one" refers, the definition also allows other elements to be optionally present, whether or not they are related to the specifically identified elements. Thus, as a non-limiting example, "at least one of A and B" (or equivalently, "at least one of A or B", or equivalently, "at least one of A and / or B") may refer to at least one optionally including more than one A, but without B (and optionally including elements other than B) in one embodiment; in another embodiment, to at least one optionally including more than one B, but without A (and optionally including elements other than A); in yet another embodiment, to at least one optionally including more than one A, and at least one optionally including more than one B (and optionally including other elements), etc.

[0052] As used herein, the terms "biological sample," "tissue sample," "specimen," or similar terms refer to any sample including biomolecules (e.g., proteins, peptides, nucleic acids, lipids, carbohydrates, or combinations thereof) obtained from any organism, including viruses. Examples of other organisms include mammals (e.g., humans; therians, such as cats, dogs, horses, cows, and pigs; and laboratory animals, such as mice, rats, and primates), insects, annelids, arachnids, marsupials, reptiles, amphibians, bacteria, and fungi. Biological samples include tissue samples (e.g., tissue sections and puncture biopsies of tissues), cell samples (e.g., cytological smears, such as cervical smears or blood smears, or cell samples obtained by microdissection), or cell fractions, fragments, or organelles (e.g., obtained by lysing cells and separating their components by centrifugation or other means). Other examples of biological samples include blood, serum, urine, semen, feces, cerebrospinal fluid, interstitial fluid, mucus, tears, sweat, pus, biopsy tissue (e.g., obtained by surgical biopsy or needle biopsy), nipple aspirate, cerumen, breast milk, vaginal secretions, saliva, swabs (e.g., oral swabs), or any material containing biomolecules derived from the first biological sample. In certain embodiments, the term "biological sample" as used herein refers to a sample (e.g., a homogenized or liquefied sample) prepared from a tumor or a portion thereof obtained from a subject.

[0053] As used herein, the term "label" refers to an agent that can bind to, be internalized or otherwise absorbed by an analyte and be detected, for example, by shape, morphology, color, fluorescence, luminescence, phosphorescence, absorbance, magnetism or radioactive emission. Suitable examples of fluorescent labels include molecules such as fluorescein, coumarin, BODIPY dye, resorufin and rhodamine (many other examples of fluorescent molecules can be found in The Handbook—A Guide to Fluorescent Probes and Labeling Technologies, Molecular Probes, Eugene, OR).

[0054] As used herein, a "mask" is a derivative of a digital image, wherein each pixel in the mask is represented as a binary value, such as "1" or "0" (or "true" or "false"). By covering the digital image with the mask, all pixels of the digital image that map to a particular one of the binary values ​​of the mask pixel are hidden, removed, or otherwise ignored or filtered out in further processing steps applied to the digital image. For example, a mask can be generated from an original digital image by assigning all pixels of the original image having an intensity value above a threshold as either true or false, thereby creating a mask that will filter out all pixels covered by "false" masking pixels.

[0055] As used herein, the term "multi-channel image" refers to a digital image obtained from a biological tissue sample, in which different biological structures are simultaneously stained with specific stains, fluorescent dyes, quantum dots, chromogens, etc. (or any combination thereof), each of which fluoresces or is otherwise detectable in a different spectral band, thereby constituting one of the channels of the multi-channel image. The multi-channel image may include any number of channels, such as 2 channels, 3 channels, 4 channels, 5 channels, 6 channels, 9 channels, etc.

[0056] As used herein, the terms "multiplex," "multiplexed," or "multiplexing" refer to the simultaneous, substantially simultaneous, or sequential detection of multiple targets in a sample. Multiplexing can include identifying and / or quantifying a variety of different nucleic acids (e.g., DNA, RNA, mRNA, miRNA) and polypeptides (e.g., proteins), both individually and in any and all combinations.

[0057] As used herein, the term "slide" refers to any substrate of any suitable size on which a biological sample can be placed for analysis (e.g., a substrate made in whole or in part of glass, quartz, plastic, silicon, etc.), and more particularly refers to a "microscope slide" such as a standard 3 x 1 inch microscope slide or a standard 75 mm x 25 mm microscope slide. Examples of biological samples that can be placed on a slide include, but are not limited to, cytological smears, thin tissue sections (e.g., from a biopsy), and biological sample arrays, such as tissue arrays, cell arrays, DNA arrays, RNA arrays, protein arrays, or any combination thereof. Thus, in one embodiment, tissue sections, DNA samples, RNA samples, and / or proteins are placed at specific locations on the slide. In some embodiments, the term "slide" may refer to SELDI and MALDI chips, as well as silicon wafers.

[0058] As used herein, the term "staining", "staining" or similar terms generally refers to any treatment of a biological sample that detects and / or distinguishes the presence, location and / or amount (e.g., concentration) of a specific molecule (e.g., lipid, protein or nucleic acid) or a specific structure (e.g., normal or malignant cells, cytoplasm, nucleus, Golgi apparatus or cytoskeleton) in a biological sample. For example, staining can compare a specific molecule or a specific cell structure of a biological sample with the surrounding part, and the intensity of the staining can provide a determination of the amount of the specific molecule in the sample. Staining can be used not only with a bright field microscope, but also with other observation tools such as phase contrast microscopes, electron microscopes and fluorescence microscopes to assist in the observation of molecules, cell structures and organisms. Some staining performed by the system can make the outline of the cell clearly visible. Other staining performed by the system may rely on specific cell components (e.g., molecules or structures) that are stained and do not stain other cell components or stain other cell components relatively little. Examples of various types of staining methods performed by the system include, but are not limited to, histochemical methods, immunohistochemical methods, and other methods based on molecular reactions (including non-covalent binding interactions) such as hybridization reactions between nucleic acid molecules. Specific staining methods include, but are not limited to, primary staining methods (such as H&E staining, cervical staining, etc.), enzyme-linked immunohistochemistry methods, and in situ RNA and DNA hybridization methods, such as fluorescence in situ hybridization (FISH).

[0059] In some embodiments, the present disclosure relates to systems and methods for determining the autofluorescence contribution within each channel of a multi-channel image of a biological sample stained with two or more fluorescent markers, three or more fluorescent markers, four or more fluorescent markers, etc. In other embodiments, the present disclosure relates to systems and methods for determining the autofluorescence contribution within each channel of a multi-channel image of a biological sample stained with multiple fluorescent markers. In some embodiments, autofluorescence is removed while retaining as many specific fluorescent markers as possible.

[0060] In some embodiments, the present disclosure also provides systems and methods for generating multi-channel images of biological samples stained with multiple fluorescent markers, wherein the generated multi-channel images of the biological samples are corrected for autofluorescence, wherein the calculated autofluorescence contribution is removed (e.g., subtracted and / or masked) from the multi-channel images.

[0061] A multi-channel image consists of multiple images of a sample, each of which is captured at a certain set of excitation and / or emission wavelengths. Each set of excitation and / or emission wavelengths is called a "channel". The intensity of each image pixel across multiple channels of a multi-channel image thus forms a spectrum for that pixel. It is believed that the image intensity value in each channel of a multi-channel image is generated by the contribution of all fluorophores plus autofluorescence and other different sources such as tissue, fixative material, mounting medium, slide, and cover glass non-target signals.

[0062] At least some embodiments of the present disclosure relate to systems and methods for analyzing digital images captured from biological samples, including tissue samples, stained with one or more fluorescent markers. Figure 1A A digital pathology system 100 for imaging and analyzing samples is shown according to some embodiments. The digital pathology system 100 may include an imaging device 12 (e.g., a microscope or scanner) and a computer 14, whereby the imaging device 12 and the computer may be coupled together in a communicative manner (e.g., directly, or indirectly via a network 20). The computer system 14 may include a desktop computer, a laptop computer, a tablet computer, or the like, digital electronic circuitry, firmware, hardware, memory, computer storage media, a computer program or instruction set (e.g., the program is stored in the memory or storage media), one or more processors (including programmed processors), and any other hardware, software, or firmware modules or combinations thereof. For example, the Figure 1A The computing system 14 shown in may include a computer having a display device 16 and a housing 18. The computer may store digital images in binary form (stored locally, such as in a memory, on a server, or on another network-connected device). The digital image may also be divided into a matrix of pixels. The pixel may include a digital value of one or more bits defined by a bit depth. Those skilled in the art will recognize that other computer devices or systems may be utilized, and the computer system described herein may be connected to additional components (such as sample analyzers, other imaging systems, automated slide preparation equipment, etc.) in the form of communication. Some of these additional components and various available computers, networks, etc. will be further described herein. PCT Publication No. WO / 2018 / 189370 and U.S. Patent No. 9,310,302 describe devices for imaging biological samples stained with one or more fluorescent molecules, and the disclosures of these two documents are hereby incorporated herein by reference in their entirety.

[0063] In some embodiments, the imaging device can be a multispectral imaging (MSI) system or a fluorescence microscopy system. Generally speaking, MSI equips a computerized microscope-based imaging system for the analysis of pathological samples by accessing the spectral distribution of the image on a pixel layer. Although there are various multispectral imaging systems, these systems have one thing in common in operation, namely, the ability to form multispectral images. Multispectral images refer to images that capture image data at specific wavelengths or specific spectral bandwidths of the electromagnetic spectrum. These wavelengths can be selected by optical filters or using other instruments capable of selecting preset spectral components, including electromagnetic radiation with wavelengths beyond the visible light range, for example, infrared (IR).

[0064] The MSI system may include an optical imaging system, a portion of which includes a spectral selection system that is adjustable to define a predetermined number N of discrete optical bands. The optical system may be adapted for imaging a tissue sample that is illuminated by a broadband light source transmitted to an optical detector. In one embodiment, the optical imaging system may include a magnification system, such as a microscope, having a single optical axis that is generally spatially aligned with a single light output of the optical system. When the spectral selection system is adjusted or tuned (e.g., with a computer processor), the system forms a sequence of images of the tissue, thereby, for example, ensuring that images are captured in different discrete spectral bands. The device may additionally include a display that can display at least one visually perceptible image of the tissue from the captured image sequence. The spectral selection system may include an optical dispersive element, such as a diffraction grating, a set of optical filters, such as a thin film interference filter, or any other adapted to select a specific passband from the spectrum of light transmitted from the light source through the sample to the detector in response to user input or preprogrammed processor commands.

[0065] In an alternative embodiment, the spectrally selective system defines a plurality of light outputs corresponding to N discrete spectral bands. This type of system takes a transmitted light output from an optical system and spatially redirects at least a portion of the light output along N spatially distinct light paths so that the sample in an identified spectral band can be imaged onto a detector system along the light path corresponding to the identified spectral band.

[0066] Figure 1B The various modules utilized within the digital pathology system of the present disclosure are summarized. In some embodiments, the digital pathology system employs a computer device 100 or a computer-implemented method having one or more processors 103 and one or more memories 101 storing non-transitory computer-readable instructions for execution by the one or more processors to cause the one or more processors (103) to execute instructions (or store data) in one or more modules (e.g., modules 105 to 108).

[0067] refer to Figure 1B and Figure 2A In some embodiments, the system of the present disclosure includes: (a) an imaging module 102, which is suitable for generating a multi-channel image of a biological sample labeled with one or more fluorescent molecules (see, for example, step 200); (b) a reference spectrum retrieval module 105, which is configured to retrieve one or more reference spectra (e.g., reference spectra from training images) from a database (e.g., a database within a storage subsystem 104); (c) an autofluorescence contribution calculation module 106, which is suitable for deriving the amount of autofluorescence in the multi-channel image based at least on one or more retrieved reference spectra (see, for example, step 201); and (d) an image generation module 107, which generates at least an image with autofluorescence removed, i.e., a multi-channel image in which the autofluorescence has been at least partially removed (see, for example, step 202). In some embodiments, the system may also include a masking module 108 for generating certain exclusion and / or inclusion masks. In some embodiments, additional modules may be incorporated into the workflow as needed. For example, an image analysis module can be run to derive features or metrics from stained objects within a multi-channel image (see, e.g., PCT Publication No. PCT / US19 / 41257, the disclosure of which is incorporated herein by reference in its entirety). Similarly, a scoring module can be run to score the derived image features, such as scoring the percent positive, membrane or nuclear staining intensity, or providing an H-score (see, e.g., U.S. Publication No. 2015 / 0347702, the disclosure of which is incorporated herein by reference in its entirety).

[0068] In some embodiments, the systems and methods of the present disclosure utilize one or more stored autofluorescence spectra, such as autofluorescence spectra stored in a database within the storage subsystem 104, and which can be retrieved using the reference spectrum retrieval module 105. These autofluorescence spectra can be derived from stained samples (e.g., samples labeled with one or more fluorescent molecules) or unstained samples. In some embodiments, the autofluorescence spectra are derived from training images, such as stained images of biological samples, each image including one or more fluorescent markers. In some embodiments, the stained images include different groups of fluorescent molecules or different types of tissue or cellular structures. For example, multi-channel images of biological samples stained with multiple fluorescent markers can be imaged and unwanted emissions and / or excitations within the multi-channel images can be identified (see Figure 3A , where four "different types of autofluorescence to be removed" are shown, and where the spectrum of each different type of autofluorescence can be stored in a database for later retrieval). Identification of unwanted emission and / or excitation can be performed manually or in an automated manner (e.g., running an automated algorithm that identifies specific spectral wavelengths with specific intensities).

[0069] In some embodiments, the autofluorescence contribution within each channel of the obtained multi-channel image (e.g., a multi-channel image of a biological sample stained with one or more fluorescent markers) can be removed by an intensity subtraction method (using modules 105, 106, and 107). In other embodiments, the autofluorescence contribution within each channel of the multi-channel image can be removed by applying one or more exclusion and / or inclusion masks (using at least module 108). In other embodiments, the autofluorescence contribution within each channel of the multi-channel image can be removed by applying a combination of one or more intensity subtraction and masking methods. For example, a masking method can be applied to a first portion of the multi-channel image to remove some unwanted autofluorescence, and an intensity subtraction method can be applied after applying the mask (e.g., to the entire multi-channel image or only to the first portion) to remove additional contributions of autofluorescence within one or more channels. Similarly, a masking method can be applied to a first portion of the multi-channel image to remove some unwanted autofluorescence, and an intensity subtraction method can be applied to a second portion of the multi-channel image.

[0070] In some embodiments, the intensity subtraction method includes removing the estimated contribution of autofluorescence within each channel of each pixel within the multi-channel image. In other words, the intensity subtraction method described herein is applied independently to each pixel of the multi-channel image, and the end result is the autofluorescence contribution removed from each channel of the multi-channel image (which can be the entire multi-channel image, such as an entire slide scan, or one or more portions thereof).

[0071] In some embodiments, the autofluorescence contribution within each channel of the multi-channel image is derived from a single autofluorescence spectrum (e.g., an autofluorescence spectrum stored in subsystem 104). In some embodiments, a single autofluorescence spectrum can be retrieved from a database using reference spectrum retrieval module 105, and the single autofluorescence spectrum can be independently fitted to the measured spectrum of each pixel of the multi-channel image (method 1). In other words, a fitted autofluorescence spectrum can be generated for each pixel of the multi-channel image, wherein each fitted autofluorescence spectrum is independently generated by matching the single retrieved autofluorescence spectrum to the measured spectrum of one of the pixels of the multi-channel image. In some embodiments, the fitted autofluorescence spectrum independently generated for each pixel is subtracted from the measured spectrum of the particular pixel. This process is repeated for each pixel of the multi-channel image.

[0072] For example, Figure 4As shown, the autofluorescence spectrum retrieved from the database can be fitted within the measured spectrum of the pixel of the multi-channel image to provide a corresponding fitted autofluorescence spectrum. For example, for channel 2, where the normalized measured pixel intensity value is 1, and the value of the retrieved autofluorescence spectrum is about 0.2, the fitted spectrum for this particular pixel at channel 2 will have a value of about 0.05. In this example, the "fitting" process is repeated for each of channels 1 to 6. The fitted autofluorescence spectra of channels 1 to 6 are then subtracted from the measured spectrum of the pixel.

[0073] refer to Figure 2B In some embodiments, the process of "fitting" the autofluorescence spectrum to the measured spectrum includes normalizing the measured pixel spectrum and scaling the retrieved individual autofluorescence spectrum to fit within the normalized measured pixel spectrum. In some embodiments, a measured pixel spectrum of a pixel of an input multi-channel image is obtained (210). The measured pixel spectrum is divided by the retrieved autofluorescence spectrum to provide a normalized measured spectrum for a particular pixel (211), for example, by dividing the intensity value in each channel of the measured spectrum by the value in each corresponding channel of the autofluorescence spectrum. As described above, the autofluorescence spectrum can be retrieved from the storage subsystem 104 (step 214). In some embodiments, the retrieved autofluorescence spectrum is derived from a training multi-channel image of a biological sample stained with one or more fluorescent molecules. Next, a minimum value of the normalized measured spectrum is calculated (212). From the calculated minimum value, the autofluorescence contribution in each channel can be calculated, for example by multiplying the calculated minimum value by the retrieved autofluorescence spectrum (step 213). In some embodiments, the calculated autofluorescence contribution in each channel can be subtracted from the corresponding channel in the input measured pixel spectrum (202).

[0074] In other embodiments, the autofluorescence contribution within each channel of the obtained multi-channel image is derived from two or more different autofluorescence spectra, where such autofluorescence spectra can be retrieved from the storage subsystem 104 (Method 2). The process is similar to the process of fitting a single autofluorescence spectrum to a measured pixel spectrum described above, but differs in that (i) two or more different autofluorescence spectra are each individually fitted to each measured pixel spectrum; and (ii) the maximum of the fitted autofluorescence spectra is used to generate a maximum fitted autofluorescence spectrum. Thus, the generated maximum fitted autofluorescence spectrum includes the maximum autofluorescence contribution at each channel (wherein the maximum autofluorescence contribution at each channel is derived from the fitted autofluorescence spectrum at the channel having the maximum value). For example, if three spectra are fitted to the measured pixel spectrum, and the intensity values ​​in channel 4 of each fitted autofluorescence spectrum are approximately 0.25 (first fitted autofluorescence spectrum), approximately 0.15 (second fitted autofluorescence spectrum), and approximately 0.2 (third fitted autofluorescence spectrum), the intensity value of the first fitted spectrum will be used as a data point at channel 4 to generate the maximum fitted autofluorescence spectrum. This process is then repeated for each channel, such that the maximum intensity value data points are accumulated for each channel, and a maximum fitted autofluorescence spectrum can then be generated using the accumulated data points. It is this generated maximum fitted autofluorescence spectrum that is then subtracted from the corresponding measured pixel spectrum. This process is repeated for each pixel of the multi-channel image, i.e., a maximum fitted autofluorescence spectrum is generated for each pixel, and then the specific generated maximum fitted autofluorescence spectrum is subtracted from the corresponding measured pixel spectrum.

[0075] For example, and if Figure 5A As shown, two reference autofluorescence spectra may be retrieved from a database within the storage subsystem 104. Each of the retrieved first and second autofluorescence spectra may be fitted to the measured pixel spectrum, such as Figure 5B As shown, first and second fitted autofluorescence spectra are provided. For each channel, the maximum intensity value of all fitted autofluorescence spectra is identified, and the cumulative identified maximum intensity values ​​are used to generate the maximum fitted autofluorescence spectrum shown.

[0076] Figure 3A and Figure 3B The results of an image subtraction process using two or more autofluorescence spectra (i.e., using method 2) are shown. Figure 3A The autofluorescence spectra of the identified unwanted signals can each be fitted to each measured pixel spectrum in the multi-channel image, so that the autofluorescence contribution of each channel in each pixel of the multi-channel image can be estimated. The estimated contribution of autofluorescence of each channel can then be subtracted from the measured spectrum at each pixel, providing Figure 3B Multi-channel image with autofluorescence removed.

[0077] In some embodiments, the number of stored autofluorescence spectra that can be fit to the measured pixel spectrum can be in the range of between 2 to about 1000 spectra. In other embodiments, the number of stored autofluorescence spectra that can be fit to the measured pixel spectrum can be in the range of between 2 to about 500 spectra. In other embodiments, the number of stored autofluorescence spectra that can be fit to the measured pixel spectrum can be in the range of between 2 to about 250 spectra. In further embodiments, the number of stored autofluorescence spectra that can be fit to the measured pixel spectrum can be in the range of between 2 to about 125 spectra. In still further embodiments, the number of stored autofluorescence spectra that can be fit to the measured pixel spectrum can be in the range of between about 5 to about 75 spectra. In still further embodiments, the number of stored autofluorescence spectra that can be fit to the measured pixel spectrum can be in the range of between about 5 to about 50 spectra.

[0078] The process of fitting two or more different autofluorescence spectra within the measured pixel spectrum and generating the maximum fitted autofluorescence spectrum is described in Figure 2C and Figure 2D In some embodiments, a measured pixel spectrum of a pixel of an input multi-channel image is obtained (220). In some embodiments, and with reference to Figure 2D (See graph 1), offline measurement of the spectrum (MS) and autofluorescence spectrum (AF). Figure 2D In the example of , there are two such spectra, namely AF1 and AF2. The measured pixel spectrum is divided by the first retrieved autofluorescence spectrum (227) to provide a normalized measured spectrum (221) for the particular pixel, for example by dividing the intensity value in each channel of the measured spectrum by the intensity value in each corresponding channel of the autofluorescence spectrum. Next, the minimum value of the normalized measured spectrum is calculated (222). From this calculated minimum value, the autofluorescence correction in each channel can be calculated, for example by multiplying the calculated minimum value by the retrieved autofluorescence spectrum (step 223). With regard to Figure 2D (See graph 2), the measured spectrum (MS) is divided by the autofluorescence spectrum AF1, and then the minimum value is calculated. Correction 1 is then calculated by multiplying the calculated minimum value by AF1.

[0079] As shown in dashed box 224, the process of steps 221, 222 and 223 may be repeated for each retrieved autofluorescence spectrum (227). Figure 2D(See graph 3) The measured spectrum (MS) can be divided by the autofluorescence spectrum AF2 and then the minimum value can be calculated. Correction 2 is then calculated by multiplying the calculated minimum value by AF1. Thus, an autofluorescence correction (corrected spectrum) is calculated separately for each retrieved autofluorescence spectrum. Based on the sum of the calculated autofluorescence corrections, the autofluorescence contribution in each channel is calculated as the maximum autofluorescence correction value for each channel (225).

[0080] In some embodiments, the calculated autofluorescence contribution in each channel derived from the maximum autofluorescence correction in each channel can be subtracted from the corresponding channel in the input measured pixel spectrum. This process can be repeated (226) for each pixel of the input multi-channel image. Figure 2D (See curve Figure 4 ), the final correction can be determined as the maximum value among all corrections. Figure 2D 5, the final correction can then be subtracted from the measured spectrum (MS) to provide a spectrum with the AF removed.

[0081] In other embodiments, the autofluorescence contribution within each channel is derived from one or more groupings of autofluorescence spectra. In some embodiments, the autofluorescence spectra can be grouped according to similarity. As a non-limiting example, the autofluorescence spectra can be grouped according to the maximum wavelength or spectral band of the autofluorescence spectra. For example, the grouping of spectra can include those autofluorescence spectra with a maximum measured wavelength of about 610 to about 640 nm.

[0082] Within each grouping of autofluorescence spectra, a spectral pair may be generated. In some embodiments, the generated spectral pair includes (i) the generated autofluorescence spectrum having the highest intensity value for each channel; and (ii) the generated autofluorescence spectrum having the lowest intensity value for each channel (referred to herein as a "high and low spectral pair"). For example, Fig. 6A and Figure 6B The generated high and low spectral pairs derived from two different groupings of autofluorescence spectra are shown.Once the high and low spectral pairs are generated, all possible spectra between the high and low spectral pairs can be identified and removed, for example using the spectral fitting methods described herein.

[0083] In some embodiments, the spectral set includes 3 to about 200 collected autofluorescence spectra. In other embodiments, the spectral set includes 3 to about 100 collected autofluorescence spectra. In other embodiments, the spectral set includes 3 to about 50 collected autofluorescence spectra. For example, Figure 6C The collected spectra for six different image channels (each with a different stain or marker) are shown. Figure 6CThe autofluorescence spectra collected in the image are grouped into different groups (or clusters). The grouping can be performed manually or can be performed automatically (for example using a k-means clustering method). Fig.6D Different groups (or clusters) of autofluorescence spectra are shown, each group comprising multiple individual autofluorescence spectra. In some embodiments, high and low spectral pairs can be generated from these individual groupings (or clusters). In some embodiments, if grouping is performed, the low-high pair is simply the minimum and maximum of all spectra in the group in each channel. It is believed that the goal of grouping is to cover all collected spectra with the minimum possible removal, which is the lowest possible combined area between the low spectral curve and the high spectral curve.

[0084] Fig. 6E and Fig. 6F The generation of spectral pairs from grouped autofluorescence spectra is further illustrated, wherein grouped autofluorescence spectra containing unwanted autofluorescence are collected from a multi-channel image (see the upper left panel of each figure), whereby high and low spectral pairs are derived from the collected autofluorescence spectra (see the lower left panel of each figure). Fig. 6E and Fig. 6F The spectral pairs identified in determine the autofluorescence contribution within each channel of the multichannel image, which can then be subtracted from the acquired multichannel image to provide Figure 6G Multi-channel images with autofluorescence removed. In addition, Fig. 6E and Fig. 6F A collection of AF spectra is shown, which are then stored in a database, and from which they are subsequently retrieved.

[0085] The process of fitting two or more different autofluorescence spectra within the measured pixel spectrum and generating the maximum fitted autofluorescence spectrum is described in Figure 2E and Figure 2F In some embodiments, a measured pixel spectrum of a pixel of an input multi-channel image is obtained (230). In some embodiments, and with reference to Figure 2F (See graph 1), offline measurement of the spectrum (MS) and autofluorescence spectrum (AF). Figure 2F In the example of , there are two such spectra, namely AF1 and AF2.

[0086] The measured pixel spectrum is divided by a first retrieved autofluorescence low spectrum (234) (e.g., from a single high and low autofluorescence spectrum pair) to provide a normalized measured spectrum for the particular pixel (231). Next, a minimum value of the normalized measured spectrum is calculated (232). From this calculated minimum value, an autofluorescence correction in each channel can be calculated, for example by multiplying the calculated minimum value by the retrieved autofluorescence high spectrum (step 233). Figure 2F(See graph 2), the measured spectrum (MS) is divided by the autofluorescence spectrum AF1Low, and then the minimum value is calculated. Correction 1 is then calculated by multiplying the calculated minimum value by AF1High.

[0087] As shown in dashed box 234, the process of steps 231, 232 and 233 may be repeated for each retrieved pair of autofluorescence spectra. Figure 2F (See graph 3 and graph Figure 4 ), the measured spectrum (MS) can be divided by the autofluorescence spectrum AF2Low and then the minimum value can be calculated. Correction 2 is then calculated by multiplying the calculated minimum value by AF2High. Thus, the autofluorescence correction (correction spectrum) is calculated separately for each retrieved pair of autofluorescence spectra.

[0088] Based on the sum of the calculated autofluorescence corrections, the autofluorescence contribution in each channel is calculated as the maximum autofluorescence correction value for each channel (235). Figure 2F (See graph 5), the final correction can be determined as the maximum value among all corrections. Figure 2F 6, the final correction can then be subtracted from the measured spectrum (MS) to provide a spectrum with the AF removed.

[0089] This calculated autofluorescence contribution in each channel derived from the maximum autofluorescence correction in each channel can be subtracted from the corresponding channel in the input measured pixel spectrum. This process can be repeated (236) for each pixel of the input multi-channel image.

[0090] In some embodiments, one or more exclusion masks may be generated and applied to the input image to "mask" certain components or structures within the input multi-channel image. For example, given a particular spectrum, a pixel may be thresholded, and if the spectrum of the pixel falls within a predetermined range, it may be added to the exclusion mask. Any thresholding technique may be applied, including global thresholding, Otsu's method, and local thresholding. Other examples of thresholding techniques are described in U.S. Patent Publication Nos. 2017 / 0294017, 2018 / 0012355, and 2017 / 0337695 or PCT Publication No. WO / 2018 / 115055 (the disclosures of which are hereby incorporated by reference in their entirety), any of which may be applied to generate an exclusion mask. Figure 7B The figure shows the process from the input multi-channel image (see Fig. 7A ) An example of an exclusion mask derived from. In some embodiments, multiple exclusion masks can be generated and used to mask portions of an input multi-channel image in order to remove certain structures or components that contribute to autofluorescence.

[0091] In some embodiments, one or more inclusion masks may be generated.In some embodiments, the inclusion mask is generated based on the desired fluorescence that should be retained within the input multi-channel image. Figure 7C The figure shows the process from the input multi-channel image (see Fig. 7A )Exported instance containing mask grinding. Fig.7D The resulting multi-channel image with autofluorescence removed is shown, where both the exclusion mask and the inclusion mask are applied. For illustration purposes, the exclusion mask is Fig.7D is overlaid, i.e., to show where the autofluorescence has been removed.

[0092] Additional Embodiments

[0093] Additional Example 1. A method for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the method comprising: independently calculating the autofluorescence contribution in each channel of each pixel of the multi-channel image based on one or more retrieved autofluorescence spectra, and subtracting the calculated autofluorescence contribution in each channel for each measured pixel of the multi-channel image from the spectrum of the corresponding pixel.

[0094] Additional Example 2. A method as described in additional embodiment 1, wherein the autofluorescence contribution in each channel of each pixel of the multi-channel image is calculated from a single retrieved autofluorescence spectrum.

[0095] Additional Example 3. The method of additional embodiment 2, wherein the single retrieved autofluorescence spectrum is derived from a measured signal corresponding to an unlabeled portion of a training multi-channel image of the biological sample.

[0096] Additional Example 4. A method as described in Additional Example 3, wherein the training multi-channel image of the biological sample includes one or more fluorescent markers.

[0097] Additional Example 5. A method as described in additional embodiment 2, wherein the autofluorescence contribution in each channel of each pixel is calculated by independently fitting the single retrieved autofluorescence spectrum to each measured spectrum of the pixel of the multi-channel image.

[0098] Additional Example 6.A method as described in additional embodiment 5, wherein the fitting includes (i) dividing the measured spectrum of the pixel of the multi-channel image by the single retrieved autofluorescence spectrum to provide a normalized measured spectrum of the pixel; (ii) calculating the minimum value of the normalized measured spectrum of the pixel; and (iii) scaling the calculated minimum value by the single retrieved autofluorescence spectrum.

[0099] Additional Example 7. The method as in additional embodiment 1, wherein the autofluorescence contribution in each channel of each pixel of the multi-channel image is calculated from at least two retrieved autofluorescence spectra.

[0100] Additional Example 8. The method of additional embodiment 7, wherein the at least two retrieved autofluorescence spectra are derived from a measured signal corresponding to an unlabeled portion of a training multi-channel image of the biological sample.

[0101] Additional Example 9. A method as described in Additional Example 8, wherein the training multi-channel image of the biological sample includes one or more fluorescent markers.

[0102] Additional Example 10. A method as described in additional embodiment 8, wherein the calculated autofluorescence contribution in each channel of each pixel of the multi-channel image is a maximum autofluorescence correction value independently identified for each channel of each pixel of the multi-channel image.

[0103] Additional Example 11. A method as described in additional embodiment 10, wherein the maximum autofluorescence correction for each channel of each pixel of the multi-channel image is identified by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of the at least two retrieved autofluorescence spectra to the measured spectrum of the pixel; and (ii) determining the maximum autofluorescence intensity value of the calculated at least two estimated autofluorescence corrections in each channel.

[0104] Additional Example 12. A method as in additional embodiment 1, wherein the autofluorescence contribution in each channel of each pixel of the multi-channel image is calculated from at least two retrieved autofluorescence low-high spectral pairs.

[0105] Additional Example 13. A method as described in additional embodiment 12, wherein the calculated autofluorescence contribution in each channel of each pixel of the multi-channel image is a maximum autofluorescence correction value independently identified for each channel of each pixel of the multi-channel image.

[0106] Additional Example 14.A method as described in additional embodiment 13, wherein the maximum autofluorescence correction for each channel of each pixel is calculated by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of at least two retrieved autofluorescence low-high spectrum pairs to the measured spectrum of the pixel; and (ii) identifying the maximum autofluorescence intensity value of the calculated at least two estimated autofluorescence corrections in each channel.

[0107] Additional Example 15. A method as described in additional embodiment 14, wherein each independent fit includes (i) dividing the measured spectrum of the pixel of the multi-channel image by the first autofluorescence low spectrum of one of the at least two autofluorescence low-high spectrum pairs; (ii) calculating the minimum value of the normalized measured spectrum of the pixel; and (iii) scaling the calculated minimum value by the first autofluorescence high spectrum of the corresponding one of the at least two autofluorescence low-high spectrum pairs.

[0108] Additional Example 16. A system for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the system comprising: (i) one or more processors, and (ii) one or more memories, the one or more memories being coupled to the one or more processors, the one or more memories storing computer executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising the steps of independently calculating the autofluorescence contribution in each channel for each pixel of the multi-channel image based on one or more retrieved autofluorescence spectra, and subtracting the calculated autofluorescence contribution in each channel for each measured pixel of the multi-channel image from the spectrum of the corresponding pixel.

[0109] Additional Example 17. A method as described in additional embodiment 16, wherein the autofluorescence contribution in each channel of each pixel of the multi-channel image is calculated from a single retrieved autofluorescence spectrum.

[0110] Additional Example 18. The method of additional embodiment 17, wherein the single retrieved autofluorescence spectrum is derived from a measured signal corresponding to an unlabeled portion of a training multi-channel image of the biological sample.

[0111] Additional Example 19. A method as described in Additional Example 18, wherein the training multi-channel image of the biological sample includes one or more fluorescent markers.

[0112] Additional Example 20.A method as described in additional embodiment 17, wherein the autofluorescence contribution in each channel of each pixel is calculated by independently fitting the single retrieved autofluorescence spectrum to each measured spectrum of the pixel of the multi-channel image.

[0113] Additional Example 21. A method as described in additional embodiment 20, wherein the fitting includes (i) dividing the measured spectrum of the pixel of the multi-channel image by the single retrieved autofluorescence spectrum to provide a normalized measured spectrum for the pixel; (ii) calculating the minimum value of the normalized measured spectrum for the pixel; and (iii) scaling the calculated minimum value by the single retrieved autofluorescence spectrum.

[0114] Additional Example 22. A method as described in additional embodiment 16, wherein the autofluorescence contribution in each channel of each pixel of the multi-channel image is calculated from at least two retrieved autofluorescence spectra.

[0115] Additional Example 23. The method of additional embodiment 22, wherein the at least two retrieved autofluorescence spectra are derived from a measured signal corresponding to an unlabeled portion of a training multi-channel image of the biological sample.

[0116] Additional Example 24. A method as described in Additional Example 23, wherein the training multi-channel image of the biological sample includes one or more fluorescent markers.

[0117] Additional Example 25. A method as described in additional embodiment 23, wherein the calculated autofluorescence contribution in each channel of each pixel of the multi-channel image is a maximum autofluorescence correction value independently identified for each channel of each pixel of the multi-channel image.

[0118] Additional Example 26. A method as described in additional embodiment 25, wherein the maximum autofluorescence correction for each channel of each pixel of the multi-channel image is identified by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of the at least two retrieved autofluorescence spectra to the measured spectrum of the pixel; and (ii) determining the maximum autofluorescence intensity value of the calculated at least two estimated autofluorescence corrections in each channel.

[0119] Additional Example 27. A method as described in additional embodiment 16, wherein the autofluorescence contribution in each channel of each pixel of the multi-channel image is calculated from at least two retrieved autofluorescence low-high spectral pairs.

[0120] Additional Example 28.A method as described in additional embodiment 27, wherein the calculated autofluorescence contribution in each channel of each pixel of the multi-channel image is a maximum autofluorescence correction value independently identified for each channel of each pixel of the multi-channel image.

[0121] Additional Example 29. A method as described in additional embodiment 28, wherein the maximum autofluorescence correction for each channel of each pixel is calculated by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of at least two retrieved autofluorescence low-high spectrum pairs to the measured spectrum of the pixel; and (ii) identifying the maximum autofluorescence intensity value of the calculated at least two estimated autofluorescence corrections in each channel.

[0122] Additional Example 30. A method as described in additional embodiment 29, wherein each independent fit includes (i) dividing the measured spectrum of the pixel of the multi-channel image by the first autofluorescence low spectrum of one of the at least two autofluorescence low-high spectrum pairs; (ii) calculating the minimum value of the normalized measured spectrum of the pixel; and (iii) scaling the calculated minimum value by the first autofluorescence high spectrum of the corresponding one of the at least two autofluorescence low-high spectrum pairs.

[0123] Additional Example 31. A non-transitory computer-readable medium storing instructions for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, comprising independently calculating the autofluorescence contribution in each channel for each pixel of the multi-channel image based on one or more retrieved autofluorescence spectra, and subtracting the calculated autofluorescence contribution in each channel for each measured pixel of the multi-channel image from the spectrum of the corresponding pixel.

[0124] Additional Example 32. A method as described in additional embodiment 31, wherein the autofluorescence contribution in each channel of each pixel of the multi-channel image is calculated from a single retrieved autofluorescence spectrum.

[0125] Additional Example 33. The method of additional embodiment 32, wherein the single retrieved autofluorescence spectrum is derived from a measured signal corresponding to an unlabeled portion of a training multi-channel image of the biological sample.

[0126] Additional Example 34. A method as described in additional embodiment 33, wherein the training multi-channel image of the biological sample includes one or more fluorescent markers.

[0127] Additional Example 35.A method as described in additional embodiment 32, wherein the autofluorescence contribution in each channel of each pixel is calculated by independently fitting the single retrieved autofluorescence spectrum to each measured spectrum of the pixel of the multi-channel image.

[0128] Additional Example 36. A method as described in additional embodiment 35, wherein the fitting includes (i) dividing the measured spectrum of the pixel of the multi-channel image by the single retrieved autofluorescence spectrum to provide a normalized measured spectrum for the pixel; (ii) calculating the minimum value of the normalized measured spectrum for the pixel; and (iii) scaling the calculated minimum value by the single retrieved autofluorescence spectrum.

[0129] Additional Example 37. A method as described in additional embodiment 31, wherein the autofluorescence contribution in each channel of each pixel of the multi-channel image is calculated from at least two retrieved autofluorescence spectra.

[0130] Additional Example 38. The method of additional embodiment 37, wherein the at least two retrieved autofluorescence spectra are derived from a measured signal corresponding to an unlabeled portion of a training multi-channel image of the biological sample.

[0131] Additional Example 39. A method as described in additional embodiment 38, wherein the training multi-channel image of the biological sample includes one or more fluorescent markers.

[0132] Additional Example 40. A method as described in additional embodiment 38, wherein the calculated autofluorescence contribution in each channel of each pixel of the multi-channel image is a maximum autofluorescence correction value independently identified for each channel of each pixel of the multi-channel image.

[0133] Additional Example 41. A method as described in additional embodiment 40, wherein the maximum autofluorescence correction for each channel of each pixel of the multi-channel image is identified by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of the at least two retrieved autofluorescence spectra to the measured spectrum of the pixel; and (ii) determining the maximum autofluorescence intensity value of the calculated at least two estimated autofluorescence corrections in each channel.

[0134] Additional Example 42. A method as described in additional embodiment 31, wherein the autofluorescence contribution in each channel of each pixel of the multi-channel image is calculated from at least two retrieved autofluorescence low-high spectral pairs.

[0135] Additional Example 43.A method as described in additional embodiment 42, wherein the calculated autofluorescence contribution in each channel of each pixel of the multi-channel image is a maximum autofluorescence correction value independently identified for each channel of each pixel of the multi-channel image.

[0136] Additional Example 44. A method as described in additional embodiment 43, wherein the maximum autofluorescence correction for each channel of each pixel is calculated by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of at least two retrieved autofluorescence low-high spectrum pairs to the measured spectrum of the pixel; and (ii) identifying the maximum autofluorescence intensity value of the calculated at least two estimated autofluorescence corrections in each channel.

[0137] Additional Example 45. A method as described in additional embodiment 44, wherein each independent fit includes (i) dividing the measured spectrum of the pixel of the multi-channel image by the first autofluorescence low spectrum of one of the at least two autofluorescence low-high spectrum pairs; (ii) calculating the minimum value of the normalized measured spectrum of the pixel; and (iii) scaling the calculated minimum value by the first autofluorescence high spectrum of the corresponding one of the at least two autofluorescence low-high spectrum pairs.

[0138] Additional Example 46. A method for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the method comprising (a) independently calculating a maximum autofluorescence correction in each channel for each pixel of the multi-channel image, wherein the calculated maximum autofluorescence correction in each channel for each pixel is derived from at least two different retrieved autofluorescence spectra; and (b) subtracting the calculated maximum autofluorescence correction in each channel for each measured pixel of the multi-channel image from the spectrum of that pixel.

[0139] Additional Example 47. A non-transitory computer-readable medium storing instructions for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, comprising: (a) independently calculating a maximum autofluorescence correction in each channel for each pixel of the multi-channel image, wherein the calculated maximum autofluorescence correction in each channel for each pixel is derived from at least two different retrieved autofluorescence spectra; and (b) subtracting the calculated maximum autofluorescence correction in each channel for each measured pixel of the multi-channel image from the spectrum of that pixel.

[0140] Additional Example 48.A system for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the system comprising: (i) one or more processors, and (ii) one or more memories, the one or more memories being coupled to the one or more processors, the one or more memories storing computer executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising the following steps: (a) independently calculating a maximum autofluorescence correction in each channel for each pixel of the multi-channel image, wherein the calculated maximum autofluorescence correction in each channel for each pixel is derived from at least two different retrieved autofluorescence spectra; and (b) subtracting the calculated maximum autofluorescence correction in each channel for each measured pixel of the multi-channel image from the spectrum of that pixel.

[0141] Additional Example 49. A system as described in additional embodiment 48, wherein the maximum autofluorescence correction for each channel of each pixel is calculated in the following manner: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of the at least two retrieved autofluorescence spectra to the measured spectrum of the pixel; and (ii) identifying the maximum autofluorescence intensity value of the at least two estimated autofluorescence corrections calculated in each channel.

[0142] Additional Example 50. A system as described in additional embodiment 49, wherein each independent fit comprises (i) dividing the measured spectrum of the pixel of the multi-channel image by one of the at least two retrieved autofluorescence spectra to provide a normalized measured spectrum for the pixel; (ii) calculating a minimum value of the normalized measured spectrum for the pixel; and (iii) scaling the calculated minimum value by one of the at least two retrieved autofluorescence spectra.

[0143] Additional Example 51. The system of additional embodiment 49, wherein the at least two retrieved autofluorescence spectra include at least two autofluorescence low-high spectrum pairs.

[0144] Additional Example 52. A system as described in additional embodiment 51, wherein each independent fit comprises (i) dividing the measured spectrum of the pixel of the multi-channel image by the first autofluorescence low-spectrum of one of the at least two autofluorescence low-high spectrum pairs; (ii) calculating the minimum value of the normalized measured spectrum of the pixel; and (iii) scaling the calculated minimum value by the first autofluorescence high-spectrum of the corresponding one of the at least two autofluorescence low-high spectrum pairs.

[0145] Additional Example 53.A system for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the system comprising: (i) one or more processors, and (ii) one or more memories, the one or more memories being coupled to the one or more processors, the one or more memories storing computer executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising the steps of independently calculating the autofluorescence contribution in each channel for each pixel of the multi-channel image based on at least two retrieved autofluorescence spectra, and subtracting the calculated autofluorescence contribution in each channel for each measured pixel of the multi-channel image from the spectrum of the corresponding pixel, wherein the two or more retrieved autofluorescence spectra are derived from measured signals corresponding to an unlabeled portion of a training multi-channel image of a biological sample containing one or more fluorescent markers.

[0146] Additional Example 54. A non-transitory computer-readable medium storing instructions for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, comprising: independently calculating the autofluorescence contribution in each channel for each pixel of the multi-channel image based on at least two retrieved autofluorescence spectra, and subtracting the calculated autofluorescence contribution in each channel for each measured pixel of the multi-channel image from the spectrum of the corresponding pixel, wherein the two or more retrieved autofluorescence spectra are derived from measured signals corresponding to unlabeled portions of a training multi-channel image of a biological sample containing one or more fluorescent markers.

[0147] Additional Example 55. A method for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the method comprising independently calculating the autofluorescence contribution in each channel for each pixel of the multi-channel image based on at least two retrieved autofluorescence spectra, and subtracting the calculated autofluorescence contribution in each channel for each measured pixel of the multi-channel image from the spectrum of the corresponding pixel, wherein the two or more retrieved autofluorescence spectra are derived from measured signals corresponding to unlabeled portions of a training multi-channel image of the biological sample containing the one or more fluorescent markers.

[0148] All U.S. patents, U.S. patent application publications, U.S. patent applications, foreign patents, foreign patents, and non-patent publications mentioned in this specification and / or listed in the Application Data Sheet are incorporated herein by reference in their entirety. If necessary, various aspects of the embodiments may be modified to provide other further embodiments using concepts from various patents, applications, and publications.

[0149] Although the present disclosure has been described with reference to some illustrative embodiments, it should be understood that those skilled in the art can design many other modifications and embodiments within the spirit and scope of the principles of the present disclosure. More specifically, without violating the spirit of the present disclosure, within the scope of the above disclosure, the drawings and the appended claims, the components and / or arrangements of the subject combination arrangement may be reasonably varied and modified. In addition to the variations and modifications in the components and / or arrangements, alternative uses will also be apparent to those skilled in the art.

Claims

1. A method for removing autofluorescence contribution in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the method comprising: independently calculating the autofluorescence contribution in each channel for each pixel of the multi-channel image based on one or more acquired autofluorescence spectra, and subtracting the calculated autofluorescence contribution in each channel for each measured pixel of the multi-channel image from the spectrum of the corresponding pixel, wherein the autofluorescence contribution in each channel for each pixel of the multi-channel image is calculated from a single acquired autofluorescence spectrum, wherein the autofluorescence contribution in each channel for each pixel is calculated by independently fitting the single acquired autofluorescence spectrum to each measured spectrum of the pixel of the multi-channel image, and wherein the fitting comprises (i) dividing the measured spectrum of the pixel of the multi-channel image by the single acquired autofluorescence spectrum to provide a normalized measured spectrum for the pixel; (ii) calculating the minimum value of the normalized measured spectrum of the pixel; and (iii) scaling the calculated minimum by the single acquired autofluorescence spectrum.

2. The method of claim 1, wherein the single acquired autofluorescence spectrum is derived from a measured signal corresponding to an unlabeled portion of a training multi-channel image of a biological sample.

3. The method of claim 2, wherein the training multi-channel image of the biological sample comprises one or more fluorescent markers.

4. A method for removing autofluorescence contribution in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the method comprising: independently calculating the autofluorescence contribution in each channel for each pixel of the multi-channel image based on one or more acquired autofluorescence spectra, and subtracting the calculated autofluorescence contribution in each channel for each measured pixel of the multi-channel image from the spectrum of the corresponding pixel, wherein the autofluorescence contribution in each channel for each pixel of the multi-channel image is calculated from at least two acquired autofluorescence spectra, wherein the at least two acquired autofluorescence spectra are derived from measured signals of an unlabeled portion of a training multi-channel image corresponding to a biological sample, wherein the calculated autofluorescence contribution in each channel for each pixel of the multi-channel image is a maximum autofluorescence correction value independently identified for each channel for each pixel of the multi-channel image, and wherein the maximum autofluorescence correction for each channel for each pixel of the multi-channel image is identified by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of the at least two acquired autofluorescence spectra to the measured spectrum of the pixel; (ii) Determine in each channel the calculated maximum autofluorescence intensity value for at least two estimated autofluorescence corrections.

5. The method of claim 4, wherein the training multi-channel image of the biological sample comprises one or more fluorescent markers.

6. A method for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the method comprising: independently calculating the autofluorescence contribution in each channel for each pixel of the multi-channel image based on one or more acquired autofluorescence spectra, and subtracting the calculated autofluorescence contribution in each channel for each measured pixel spectrum of the multi-channel image, wherein the autofluorescence contribution in each channel for each pixel of the multi-channel image is calculated from at least two acquired autofluorescence low-high spectrum pairs, wherein the calculated autofluorescence contribution in each channel for each pixel of the multi-channel image is a maximum autofluorescence correction value independently identified for each channel for each pixel of the multi-channel image, and wherein the maximum autofluorescence correction for each channel for each pixel is calculated by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of the acquired at least two acquired autofluorescence low-high spectrum pairs to the measured spectrum of the pixel; and (ii) identifying in each channel at least two estimated autofluorescence-corrected maximum autofluorescence intensity values ​​calculated.

7. The method of claim 6, wherein each independent fit comprises (i) dividing the measured spectrum of the pixel of the multi-channel image by a first autofluorescence low spectrum of one of the at least two autofluorescence low-high spectrum pairs; (ii) calculating the minimum value of the normalized measured spectrum of the pixel; and (iii) scaling the calculated minimum by a first autofluorescence hyperspectrum of a corresponding one of the at least two autofluorescence hypo-hyperspectral pairs.

8. A system for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, the system comprising: (i) one or more processors, and (ii) one or more memories coupled to the one or more processors, the one or more memories storing computer executable instructions that, when executed by the one or more processors, cause the system to perform operations comprising: (a) independently calculating a maximum autofluorescence correction in each channel for each pixel of the multi-channel image, wherein the calculated maximum autofluorescence correction in each channel for each pixel is derived from at least two different acquired autofluorescence spectra; and (b) subtracting the calculated maximum autofluorescence correction in each channel for each measured pixel spectrum of the multi-channel image, wherein the maximum autofluorescence correction for each channel for each pixel is calculated by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of the at least two acquired autofluorescence spectra to the measured spectrum of the pixel; (ii) Identify in each channel at least two estimated autofluorescence-corrected maximum autofluorescence intensity values ​​calculated.

9. The system of claim 8, wherein each independent fit comprises (i) dividing the measured spectrum of the pixel of the multi-channel image by one of the at least two acquired autofluorescence spectra to provide a normalized measured spectrum of the pixel; (ii) calculating the minimum value of the normalized measured spectrum of the pixel; and (iii) scaling the calculated minimum value by said one of said at least two acquired autofluorescence spectra.

10. The system of claim 8, wherein the at least two acquired autofluorescence spectra include at least two autofluorescence low-high spectrum pairs.

11. The system of claim 10, wherein each independent fit comprises (i) dividing the measured spectrum of the pixel of the multi-channel image by a first autofluorescence low spectrum of one of the at least two autofluorescence low-high spectrum pairs; (ii) calculating the minimum value of the normalized measured spectrum of the pixel; and (iii) scaling the calculated minimum by a first autofluorescence hyperspectrum of a corresponding one of the at least two autofluorescence hypo-hyperspectral pairs.

12. A non-transitory computer readable medium storing instructions for removing autofluorescence contributions in one or more channels of a multi-channel image of a biological sample stained with one or more fluorescent markers, comprising: independently calculating an autofluorescence contribution in each channel for each pixel of the multi-channel image based on at least two acquired autofluorescence spectra, and subtracting the calculated autofluorescence contribution in each channel for each measured pixel of the multi-channel image from the measured spectrum of the corresponding pixel, wherein the two or more acquired autofluorescence spectra are derived from measured signals of an unlabeled portion of a training multi-channel image corresponding to a biological sample comprising one or more fluorescent markers, wherein the at least two acquired autofluorescence spectra are derived from measured signals of an unlabeled portion of a training multi-channel image corresponding to a biological sample, wherein the calculated autofluorescence contribution in each channel for each pixel of the multi-channel image is a maximum autofluorescence correction value independently identified for each channel for each pixel of the multi-channel image, and wherein the maximum autofluorescence correction for each channel for each pixel of the multi-channel image is identified by: (i) calculating at least two estimated autofluorescence corrections for each channel by independently fitting each of the at least two acquired autofluorescence spectra to the measured spectrum of the pixel; (ii) Determine in each channel the calculated maximum autofluorescence intensity value for at least two estimated autofluorescence corrections.

13. The non-transitory computer-readable medium of claim 12, wherein the training multi-channel image of the biological sample comprises one or more fluorescent markers.

Images