Hyperspectral imaging system using hybrid unmixing

Abstract

The hybrid unmixing (HyU) technique of the present disclosure may provide enhanced imaging of multiple fluorescent labels enabling longitudinal imaging of multiple fluorescent signals at reduced illumination intensity. The present disclosure generally relates to imaging systems. The present disclosure relates to hyperspectral imaging systems. The present disclosure further relates to hyperspectral imaging systems that generate unmixed color images of a target. The present disclosure further relates to hyperspectral imaging systems configured to provide enhanced imaging of a target using the HyU technique. The present disclosure further relates to hyperspectral imaging systems configured to provide enhanced imaging of multiple fluorescent labels enabling longitudinal imaging of multiple fluorescent signals at reduced illumination intensity using the HyU technique. The present disclosure further relates to hyperspectral imaging systems for use in diagnosing health conditions.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Provisional Patent Application No. 63 / 247,688, entitled "A Hyperspectral Imaging System with Hybrid Unmixing," filed on September 23, 2021 (Attorney Docket No. AMISC.022PR), the entire contents of which are incorporated herein by reference.

[0002] (Statement regarding research funded by the U.S. Government) This invention was made with government support under Grant No. DGE-1842487 awarded by the National Science Foundation Graduate Research Fellowship, and under Grant No. PR150666 awarded by the Department of Defense. The Government has certain rights in this invention.

[0003] The present disclosure generally relates to imaging systems. The present disclosure relates to hyperspectral imaging systems. The present disclosure further relates to hyperspectral imaging systems that generate an unmixed color image of a target. The present disclosure further relates to a hyperspectral imaging system configured to provide enhanced imaging of a target using a hybrid unmixing (HyU) technique. The present disclosure further relates to a hyperspectral imaging system configured to provide enhanced imaging of a multiplexed fluorescence label using a hybrid unmixing technique that enables longitudinal imaging of multiple fluorescence signals at reduced illumination intensity. The present disclosure further relates to a hyperspectral imaging system for use in diagnosing a health condition. [Background technology]

[0004] Expanded applications of fluorescence imaging in biomedical and biological research towards more complex systems and geometries may require tools capable of analyzing multiple components at widely varying time and length scales. A major challenge in such complex imaging experiments is to cleanly separate multiple fluorescent labels with overlapping spectra from each other and background autofluorescence without perturbing the sample with high levels of light. Therefore, efficient and robust analytical tools capable of quantitatively separating these signals are required.

[0005] In recent years, several high content imaging approaches have been elaborated to decode the complex and dynamic orchestration of biological processes. Fluorescence has become the reference technique for imaging due to its high contrast, high specificity, and multiple parameters. The continuous improvement of fluorescence microscopes and the ever-expanding palette of genetically encoded and synthesized fluorophores allow the labeling and observation of a large number of molecular species. Although such fluorescence techniques may offer the possibility to simultaneously track multiple labels in the same specimen using multiplexed imaging, these techniques fall short of the full potential. A standard fluorescence microscope may collect multiple images in sequence, employing different excitation and detection bandpass filters for each label.

[0006] Recently developed fluorescence techniques may enable large-scale multiplexing by utilizing sequential labeling of fixed samples, but are not suitable for in vivo imaging. These approaches may not be adequate to separate overlapping fluorescence emission signals, and the narrowband optical filters used to increase selectivity reduce the photon efficiency of imaging (Figures 7-8). These limitations limit the number of imaged fluorophores per sample (usually up to four) and risk exposing the specimen to damaging levels of the stimulating light. Such limitations have been significant obstacles for dynamic imaging, preventing in vivo and intravital imaging from realizing its full potential with broader implications for research / applications ranging from developmental biology, cancer research, and immunology to neuroimaging.

[0007] Hyperspectral Fluorescent Imaging (HFI) potentially overcomes the limitation of overlapping emissions by extending signal detection into the spectral domain. HFI captures a spectral profile from each (image) pixel, resulting in a hyperspectral cube (x, y, wavelength) of data that can be processed to infer the labels present at that pixel. Linear unmixing (LU) has been widely utilized to analyze HFI data and works well for bright samples that emit strong signals from fully characterized extrinsic fluorophores, such as fluorescent proteins and dyes. However, in vivo fluorescence microscopy is almost always limited in the number of photons collected per pixel (due to expression levels, biophysical fluorescence properties, and the sensitivity of the detection system), which reduces the quality of the acquired spectra.

[0008] A further challenge affecting the quality of the spectra is the presence of several forms of noise in the imaging of the sample. Two examples of instrumental noise can be photon noise and read noise.

[0009] Photon noise, also known as Poisson noise, can be an inherent property associated with statistical fluctuations between photon emission from a light source and detection. Poisson noise can be unavoidable when imaging fluorescent dyes and is more pronounced in the low-photon regime. Such noise can pose challenges especially in live and time-lapse imaging, where the power of the excitation laser is reduced to avoid photodamage to the sample, reducing the amount of fluorescent signal.

[0010] Read noise can arise from voltage fluctuations in microscopes operating in analog mode during the conversion of photons to digital levels of intensity and is a common effect in fluorescence imaging acquisition.

[0011] Most biological samples used for in vivo microscopy are labeled using extrinsic signals from fluorescent proteins or fluorescent probes, but often contain intrinsic signals (autofluorescence), which can contribute unwanted photons in the LU that are difficult to identify and account for.

[0012] The cumulative presence of noise can inevitably result in degradation of the acquired spectrum during imaging. As a result, the spectral separation by LU can often be poorly performed, and the signal-to-noise ratio (SNR) of the final unmix is ​​often reduced by the weakest signal among the detected signals.

[0013] Increasing the amount of laser excitation can partially overcome these challenges, but higher energy deposition in the sample can cause photobleaching and photodamage, affecting both the integrity of the live sample and the duration of observation.

[0014] Additionally, traditional unmixing strategies such as LU can be computationally intensive, requiring long analysis times and often slowing down interrogation.

[0015] The above potential degradations and shortcomings combine to reduce both overall multiplexing capacity and adoption of HFI multiplexing techniques.

[0016] (Related art references) The following publications are relevant art with respect to the background of this disclosure. The one or two digits in box brackets preceding each reference correspond to numbers in box brackets used in other parts of this disclosure. [1]Valm,AMet al.Applying systems-level spectral imaging and analysis to reveal the organelle interactome.Nature 546,162-167(2017). [2] Tsurui, H. et al.Seven-color fluorescence imaging of tissue samples based on fourier spectroscopy and singular value decomposition.Journal of Histochemistry and Cytochemistry 48,653-662(2000). [3] Amat, F. et al.Fast, accurate reconstruction of cell lineages from large-scale fluorescence microscopy data.Nature Methods 11,951-958(2014). [4]Ueno,T.& Nagano,T.Fluorescent probes for sensing and imaging.Nature Methods vol.8 642-645 Preprint at https: / / doi.org / 10.1038 / nmeth.1663(2011). [5]Lichtman,J.W.& Conchello,J.A.Fluorescence microscopy.Nature Methods vol.2 910-919 Preprint at https: / / doi.org / 10.1038 / nmeth817(2005). [6]Sinclair,M.B.,Haaland,D.M.,Timlin,J.A.& Jones,H.D.T.Hyperspectral confocal microscope.45,6283-6291(2006). [7]Jahr,W.,Schmid,B.,Schmied,C.,Fahrbach,F.O.& Huisken,J.Hyperspectral light sheet microscopy.Nature Communications 6,1-7(2015). [8]Truong,T.V,Supatto,W.,Koos,D.S.,Choi,J.M.& Fraser,S.E.Deep and fast live imaging with two-photon scanned light-sheet microscopy.8,(2011). [9]Chen,B.C.et al.Lattice light-sheet microscopy:Imaging molecules to embryos at high spatiotemporal resolution.Science(1979)346,(2014).

[10] Kredel,S.et al.mRuby,a bright monomeric red fluorescent protein for labeling of subcellular structures.PLoS ONE 4,1-7(2009).

[11] Sakaue-Sawano,A.et al.Visualizing Spatiotemporal Dynamics of Multicellular Cell-Cycle Progression.Cell 132, 487-498 (2008).

[12] Wade,O.K.et al.124-Color Super-resolution Imaging by Engineering DNA-PAINT Blinking Kinetics.Nano Letters 19,2641-2646(2019).

[13] Strauss,S.& Jungmann,R.Up to 100-fold speed-up and multiplexing in optimized DNA-PAINT.Nature Methods 17,789-791(2020).

[14] Costa,G.et al.Asymmetric division coordinates collective cell migration in angiogenesis.Nature Cell Biology 18,1292-1301(2016).

[15] Entenberg,D.et al.Setup and use of a two-laser multiphoton microscope for multichannel intravital fluorescence imaging.Nature Protocols 6,(2011).

[16] Sakaguchi,R.,Leiwe,M.N.& Imai,T.Bright multicolor labeling of neuronal circuits with fluorescent proteins and chemical tags.Elife 7,(2018).

[17] Zimmermann,T.,Rietdorf,J.& Pepperkok,R.Spectral imaging and its applications in live cell microscopy.FEBS Letters 546,87-92(2003).

[18] Paddock,S.Multi-Spectral Imaging and Linear Unmixing Add a Whole New Dimension to.Biotechniques 31,1272-1278(2001).

[19] Zimmermann,T.Spectral imaging and linear unmixing in light microscopy.Advances in Biochemical Engineering / Biotechnology 95,245-265(2005).

[20] Garini,Y.,Young,I.T.& McNamara,G.Spectral imaging:Principles and applications.Cytometry Part A vol.69 Preprint at https: / / doi.org / 10.1002 / cyto.a.20311(2006).

[21] Rakhymzhan,A.et al.Synergistic Strategy for Multicolor Two-photon Microscopy:Application to the Analysis of Germinal Center Reactions in Vivo.Scientific Reports 7,(2017).

[22] Bass,M.Handbook of Optics,vol 3.Geometric Optics,General Principles Spherical Surfaces,2nd ed.,Optical Society of America,New York(1995).

[23] Hamamatsu Photonics,K.K.P.T.H.PHOTOMULTIPLIER TUBES Basics and Applications FOURTH EDITION.(1994)

[24] Pawley,J.B.Confocal and two-photon microscopy:Foundations,applications and advances.Microscopy Research and Technique 59,(2002).

[25] Huang,F.et al.Video-rate nanoscopy using sCMOS camera-specific single-molecule localization algorithms.Nature Methods 10,(2013).

[26] Digman,M.A.,Caiolfa,V.R.,Zamai,M.& Gratton,E.The Phasor Approach to Fluorescence Lifetime Imaging Analysis.Biophysical Journal 94,L14-L16(2008).

[27] Fereidouni,F.,Bader,A.N.,Colonna,A.& Gerritsen,H.C.Phasor analysis of multiphoton spectral images distinguishes autofluorescence components of in vivo human skin.Journal of Biophotonics 7,589-596(2014).

[28] Scipioni,L.,Rossetta,A.,Tedeschi,G.& Gratton,E.Phasor S-FLIM:a new paradigm for fast and robust spectral fluorescence lifetime imaging.Nature Methods 18,542-550(2021).

[29] Ranjit,S.,Malacrida,L.,Jameson,D.M.& Gratton,E.Fit-free analysis of fluorescence lifetime imaging data using the phasor approach.Nature Protocols 13,1979-2004(2018).

[30] Shi,W.et al.Pre-processing visualization of hyperspectral fluorescent data with Spectrally Encoded Enhanced Representations.Nature Communications 11,1-15(2020).

[31] Keshava,N.& Mustard,J.F.Spectral unmixing.IEEE Signal Processing Magazine 19,44-57(2002).

[32] Dobigeon,N.,Altmann,Y.,Brun,N.& Moussaoui,S. Linear and Nonlinear Unmixing in Hyperspectral Imaging. in Data Handling in Science and Technology vol.30(2016).

[33] Zeiss,C.& Online,M.Introduction to Spectral Imaging and Linear Unmixing.Imaging 1,1-13(2010).

[34] Hedde,P.N.,Cinco,R.,Malacrida,L.,Kamaid,A.& Gratton,E.Phasor-based hyperspectral snapshot microscopy allows fast imaging of live,three-dimensional tissues for biomedical applications.Communications Biology 4,(2021).

[35] Cutrale,F.et al.Hyperspectral phasor analysis enables multi- plexed 5D in vivo imaging.Nature Publishing Group(2017)doi:10.1038 / nmeth.4134.

[36] Stringari,C.et al.Metabolic trajectory of cellular differentiation in small intestine by Phasor Fluorescence Lifetime Microscopy of NADH.Scientific Reports 2,(2012).

[37] Ranjit S. Datta R.,Dvornikov,A.& Gratton,E.Multicomponent Analysis of Phasor Plot in a Single Pixel to Calculate Changes of Metabolic Trajectory in Biological Systems.Journal of Physical Chemistry A 123,(2019).

[38] Jeong,S.et al.Time-resolved fluorescence microscopy with phasor analysis for visualizing multicomponent topical drug distribution within human skin.Scientific Reports 10,(2020).

[39] Haas,K.T.,Fries,M.W.,Venkitaraman,A.R.& Esposito,A.Single-Cell Biochemical Multiplexing by Multidimensional Phasor Demixing and Spectral Fluorescence Lifetime Imaging Microscopy.Frontiers in Physics 9,(2021).

[40] Lanzano,L.et al.Encoding and decoding spatio-temporal information for super-resolution microscopy.Nature Communications 6,(2015).

[41] Yao,Z.et al.Multiplexed bioluminescence microscopy via phasor analysis.Nature Methods 2022 19:7 19,893-898(2022).

[42] Depasquale,J.A.Actin Microridges.Anat Rec(Hoboken)301,2037-2050(2018).

[43] Okuda,K.S.,Hogan,B.M.,Cantelmo,A.R.& Hogan,B.M.Endothelial Cell Dynamics in Vascular Development:Insights From Live-Imaging in Zebrafish.11,(2020).

[44] Isogai,S.,Lawson,N.D.,Torrealday,S.,Horiguchi,M.& Weinstein,B.M.Angiogenic network formation in the developing vertebrate trunk.(2003)doi:10.1242 / dev.00733.

[45] Denk,W.,Strickler,J.H.& Webb,W.W.Two-photon laser scanning fluorescence microscopy.Science(1979)248,(1990).

[46] Zipfel,W.R.et al.Live tissue intrinsic emission microscopy using multiphoton-excited native fluorescence and second harmonic generation.Proceedings of the National Academy of Sciences 100,7075-7080(2003).

[47] Datta,R.et al.Interactions with stromal cells promote a more oxidized cancer cell redox state in pancreatic tumors.Science Advances 8,(2022).

[48] Ma,N.et al.Label-free assessment of pre-implantation embryo quality by the Fluorescence Lifetime Imaging Microscopy(FLIM)-phasor approach.Scientific Reports 9,(2019).

[49] Zipfel,W.R.et al.Live tissue intrinsic emission microscopy using multiphoton-excited native fluorescence and second harmonic generation.Proc Natl Acad Sci U S A 100,7075-7080(2003).

[50] Bird,D.K.et al.Metabolic mapping of MCF10A human breast cells via multiphoton fluorescence lifetime imaging of the coenzyme NADH.Cancer Research 65,(2005).

[51] Lakowicz,J.R.,Szmacinski,H.,Nowaczyk,K.& Johnson,M.L.Fluorescence lifetime imaging of free and protein-bound NADH.Proc Natl Acad Sci U S A 89,(1992).

[52] Skala,M.C.et al.In vivo multiphoton microscopy of NADH and FAD redox states,fluorescence lifetimes,and cellular morphology in precancerous epithelia.Proc Natl Acad Sci U S A 104,19494-19499 (2007).

[53] Sharick,J.T.et al.Protein-bound NAD(P)H Lifetime is Sensitive to Multiple Fates of Glucose Carbon.Scientific Reports 8,(2018).

[54] Stringari,C.et al.Phasor approach to fluorescence lifetime microscopy distinguishes different metabolic states of germ cells in a live tissue.Proc Natl Acad Sci U S A 108,(2011).

[55] Wagnieres,G.A.,Star,W.M.& Wilson,B.C.Invited Review ln Vivo Fluorescence Spectroscopy and Imaging for Oncological Applications.68,603-632(1998).

[56] Fevotte,C.& Dobigeon,N.Nonlinear hyperspectral unmixing with robust nonnegative matrix factorization.IEEE Transactions on Image Processing 24,(2015).

[57] Heslop,D.,von Dobeneck,T.& Hocker,M.Using non-negative matrix factorization in the“unmixing”of diffuse reflectance spectra.Marine Geology 241,63-78(2007).

[58] Paddock,S.W.Confocal Microscopy,Methods and Protocols, Second Edition.Humana Press 1075,388(2014).

[59] M.,W.The zebrafish book.(University of Oregon Press,1994).

[60] Trinh,L.A.et al.A versatile gene trap to visualize and interrogate the function of the vertebrate proteome.Genes and Development 25,2306-2320(2011).

[61] Shi,W.et al.Pre-processing visualization of hyperspectral fluorescent data with Spectrally Encoded Enhanced Representations.Nature Communications 11,1-15(2020).

[62] Parichy,D.M.,Ransom,D.G.,Paw,B.,Zon,L.I.& Johnson,S.L.An orthologue of the kit-related gene fms is required for development of neural crest-derived xanthophores and a subpopulation of adult melanocytes in the zebrafish,Danio rerio.Development(2000).

[63] Digman,M.A.,Dalal,R.,Horwitz,A.F.& Gratton,E.Mapping the number of molecules and brightness in the laser scanning microscope.Biophys J 94,2320-2332(2008).

[64] Dalal,R.B.,Digman,M.A.,Horwitz,A.F.,Vetri,V.& Gratton,E.Determination of particle number and brightness using a laser scanning confocal microscope operating in the analog mode.Microsc Res Tech 71,69-81(2008).

[65] Wagnieres,G.A.,Star,W.M.& Wilson,B.C.Invited Review ln Vivo Fluorescence Spectroscopy and Imaging for Oncological Applications.68, 603-632(1998).

[66] Cutrale,F.et al.Hyperspectral phasor analysis enables multi- plexed 5D in vivo imaging.Nature Publishing Group(2017)doi:10.1038 / nmeth.4134.

[67] Taylor,R.C.Experiments in physical chemistry(Shoemaker,David P.;Garland,Carl W.).Journal of Chemical Education 45,(1968).

[68] Fevotte,C.& Dobigeon,N.Nonlinear hyperspectral unmixing with robust nonnegative matrix factorization.IEEE Transactions on Image Processing 24,(2015).

[69] Parslow,A.,Cardona,A.& Bryson-Richardson,R.J.Sample drift correction following 4D confocal time-lapse Imaging.Journal of Visualized Experiments(2014)doi:10.3791 / 51086.

[70] Schindelin,J.et al.Fiji:An open-source platform for biological-image analysis.Nature Methods vol.9 Preprint at https: / / doi.org / 10.1038 / nmeth.2019(2012).

[71] Shimozono,S.,Iimura,T.,Kitaguchi,T.,Higashijima,S.I.& Miyawaki,A.Visualization of an endogenous retinoic acid gradient across embryonic development.Nature 496,363-366 (2013).

[72] Islam,M.S.,Honma,M.,Nakabayashi,T.,Kinjo,M.& Ohta,N.pH dependence of the fluorescence lifetime of FAD in solution and in cells.International Journal of Molecular Sciences 14,(2013).

[73] Islam,M.S.,Honma,M.,Nakabayashi,T.,Kinjo,M.& Ohta,N.pH dependence of the fluorescence lifetime of FAD in solution and in cells.International Journal of Molecular Sciences 14,(2013).

[74] Andrews,L.M.,Jones,M.R.,Digman,M.A.& Gratton,E.Spectral phasor analysis of Pyronin Y labeled RNA microenvironments in living cells.Biomedical Optics Express 4,(2013).

[75] Fereidouni,F.,Bader,A.N.& Gerritsen,H.C.Spectral phasor analysis allows rapid and reliable unmixing of fluorescence microscopy spectral images.Optics Express 20,(2012).

[76] Cutrale, F.et al.Hyperspectral phasor analysis enables multiplexed 5D in vivo imaging.Nature Methods 14,(2017).

[77] Stringari,C.et al.Phasor approach to fluorescence lifetime microscopy distinguishes different metabolic states of germ cells in a live tissue.Proc Natl Acad Sci U S A 108,13582-7(2011).

[78] Lanzano,L.et al.Encoding and decoding spatio-temporal information for super-resolution microscopy.Nature Communications 6,(2015).

[79] Fereidouni,F.,Bader,A.N.& Gerritsen,H.C.Spectral phasor analysis allows rapid and reliable unmixing of fluorescence microscopy spectral images.Opt Express 20,12729-12741(2012).

[80] Scipioni,L.,Rossetta,A.,Tedeschi,G.& Gratton,E.Phasor S-FLIM:a new paradigm for fast and robust spectral fluorescence lifetime imaging.Nature Methods 18,(2021).

[81] Malacrida,L.,Ranjit,S.,Jameson,D.M.& Gratton,E.The Phasor Plot:A Universal Circle to Advance Fluorescence Lifetime Analysis and Interpretation.Annual Review of Biophysics vol.50 Preprint at https: / / doi.org / 10.1146 / annurev-biophys-062920-063631(2021).

[82] Ranjit, S., Malacrida, L., Jameson, DM & Gratton, E. Fit-free analysis of fluorescence lifetime imaging data using the phasor approach.Nature Protocols 13,1979-2004(2018).

[83] Digman, MA, Caiolfa, VR, Zamai, M. & Gratton, E. The Phasor Approach to Fluorescence Lifetime Imaging Analysis.Biophysical Journal 94,L14-L16(2008).

[0017] The entire contents of each of the above publications are incorporated herein by reference. Summary of the Invention

[0018] Examples described herein generally relate to imaging systems. Some examples of the present disclosure also relate to hyperspectral imaging systems. Some examples further relate to hyperspectral imaging systems that generate unmixed color images of a target. Some examples further relate to hyperspectral imaging systems configured to provide enhanced imaging of a target using hybrid unmixing techniques. Some examples further relate to hyperspectral imaging systems configured to provide enhanced imaging of multiple fluorescent labels using hybrid unmixing techniques that enable longitudinal imaging of multiple fluorescent signals at reduced illumination intensity. Some examples further relate to hyperspectral imaging systems used in the diagnosis of health conditions.

[0019] In this disclosure, a hyperspectral imaging system may include an image forming system. The imaging system acquires detected radiation of a target, the detected radiation comprising at least two (target) waves, each target wave having a detected intensity and a different detected wavelength, forms a target image using the detected target radiation, the target image comprising at least two (image) pixels, each image pixel corresponding to one physical point on the target, forms at least one (intensity) spectrum for each image pixel using the (detected) intensity and (detected) wavelength of each target wave, and based on the intensity spectrum of each image pixel, converts the intensity spectrum of each image pixel using a Fourier transform into a complex-valued function, each complex-valued function having at least one real component and at least one imaginary component, forms one phasor point on a phasor plane for each image pixel by plotting the value of the real component against the value of the imaginary component, the value of the real component being hereinafter referred to as the real value and the value of the imaginary component being hereinafter referred to as the imaginary value, and The method may form a (phasor) histogram that includes (phasor) bins, with each (phasor) bin including at least one phasor point.

[0020] In the present disclosure, the imaging system may be (further) configured to aggregate the detected spectra belonging to image pixels of each phasor bin, generate a representative intensity spectrum for each phasor bin, unmix the representative intensity spectrum for the phasor bin by using an unmixing technique, thereby determining the abundance of each spectral endmember of the detected radiation, and assign a color to a corresponding image pixel of the target by using the representative intensity spectrum and the abundance of each spectral endmember in the detected intensities belonging to the image pixels to generate a representative intensity image of the target representing the abundance of each spectral endmember.

[0021] In the present disclosure, the intensity spectra aggregated in each phasor bin may have essentially similar or substantially the same spectral shape. Or, the intensity spectra aggregated in each phasor bin may have essentially similar or substantially the same spectral features. Such spectral features may include the detected spectral intensity and / or the detected wavelength of each detected spectrum. For example, when the detected intensity of each detected spectrum is normalized using a standard (e.g., the maximum detected intensity of the spectrum), the relative (normalized) detected intensity of all intensity spectra aggregated in the same bin may have essentially similar or substantially the same spectral shape. In one example of such a configuration of the imaging system, each detected spectrum belonging to an image pixel in the same bin may have at least two detected intensities and a detected wavelength for each detected intensity. In another example of such a configuration of the imaging system, the relative detected intensity value of each spectrum belonging to the same spectral bin may be substantially the same as the relative detected intensity value of other spectra aggregated in the same bin. Further, in another example of such an imaging system configuration, the system may discretize the phasor plane into discrete phasor plane regions (which may have similar or the same region size and / or similar or the same region shape) and treat the phasor points as phasor points that belong to essentially similar or substantially the same detected spectrum. Further, in another example of such an imaging system configuration, the system may form at least four phasor bins by discretizing the phasor plot along its real dimension and its imaginary dimension. For any such configuration, each phasor bin may have a phasor bin area on each phasor plot, where the phasor bin area may be 4 / (total number of phasor bins), where the total number of phasor bins may be the product of the number of discretizations along the real dimension of the phasor plot and the number of discretizations along the imaginary dimension of the phasor plot.

[0022] Adding or averaging these essentially similar or substantially identical detected intensity spectra effectively averages the intensity spectra to produce a representative (or average) intensity spectrum for that phasor position. Adding or averaging these substantially similar intensity spectra may be accomplished by any mathematically conventional or known method. That is, any adding or averaging mathematical technique that can result in a representative intensity spectrum is within the scope of this disclosure.

[0023] In the present disclosure, any (spectral) unmixing technique that may unmix the detected target emission, intensity spectrum, and / or representative intensity spectrum (any one or more of the target emission, intensity spectrum, and / or representative intensity spectrum) is within the scope of the present disclosure. The unmixing technique may be a linear unmixing technique. The unmixing technique may be a fully constrained least squares unmixing technique, a matrix inversion unmixing technique, a non-negative matrix factorization unmixing technique, a geometric unmixing technique, a Bayesian unmixing technique, a sparse unmixing technique, or any combination thereof.

[0024] The imaging system of the present disclosure may have a further arrangement for applying a denoising filter to reduce Poisson noise and / or instrumental noise of the detected radiation. The imaging system may also have a further arrangement for applying the denoising filter at least once to the real and / or imaginary components (real and / or imaginary components) of each complex-valued function to generate a denoised real value and a denoised imaginary value for each image pixel. The denoising filter may be applied after the imaging system transforms the formed intensity spectrum belonging to each image pixel into a complex-valued function using a Fourier transform and / or before the imaging system forms a phasor point on the phasor plane for each image pixel. The imaging system may also have a further arrangement for applying the denoising filter to the real component value and / or the imaginary component value after the imaging system forms a phasor point on the phasor plane for each image pixel. Using the denoised real values ​​as the real values ​​of each image pixel, and using the denoised imaginary values ​​of each image pixel as the imaginary values, one phasor point may be formed on the phasor plane for each image pixel.

[0025] The hyperspectral imaging system may further include an optical system. The optical system may include at least one optical component. The at least one optical component may include at least one optical detector. The at least one optical detector may detect one or more of absorbed, transmitted, refracted, reflected, and emitted electromagnetic radiation from at least one physical point on the target, thereby forming a target radiation, the target radiation may include at least two target waves, each target wave having an intensity and a different wavelength. The at least one optical detector may further include a configuration that can detect the intensity and wavelength of each target wave. The at least one optical detector may also include a further configuration that can transmit the detected target radiation and the detected intensity and detected wavelength of each target wave to an imaging system to be acquired. The imaging system may further include a control system, a hardware processor, a memory, and a display. The imaging system may further include a configuration that can display a representative image of the target on a display of the imaging system.

[0026] The unmixing technique of the present disclosure may be any unmixing technique. For example, the unmixing technique may be a linear unmixing technique. For example, the unmixing technique may be a fully constrained least squares unmixing technique, a matrix inversion unmixing technique, a non-negative matrix decomposition unmixing technique, a geometric unmixing technique, a Bayesian unmixing technique, a sparse unmixing technique, or any combination thereof.

[0027] The imaging system of the present disclosure may further be configured to apply a denoising filter to reduce Poisson noise and / or instrumental noise of the detected radiation. The denoising filter may be any denoising filter applied at least once. Each applied denoising filter may be the same denoising filter or a different denoising filter. The denoising filter may be applied, for example, to the intensity of the target radiation, the intensity of the intensity spectrum, the real and / or imaginary components of each complex-valued function, the intensity of the representative intensity spectrum, and / or combinations thereof. For example, the imaging system of the present disclosure may be configured to apply a denoising filter at least once to the real and / or imaginary components of each complex-valued function to generate a denoised real value and a denoised imaginary value for each image pixel. For example, the imaging system of the present disclosure may be configured to apply a denoising filter at least once to both the real and imaginary components of each complex-valued function to generate a denoised real value and a denoised imaginary value for each image pixel, the denoising filter being applied (1) after the imaging system converts the formed intensity spectrum belonging to each image pixel into a complex-valued function using a Fourier transform and / or (2) before the imaging system forms a phasor point on the phasor plane for each image pixel, and using the denoised real value as the real value of each image pixel and the denoised imaginary value as the imaginary value of each image pixel to form a phasor point on the phasor plane for each image pixel. For example, the imaging system of the present disclosure may be configured to apply a denoising filter to the real component value and / or the imaginary component value after the imaging system forms a phasor point on the phasor plane for each image pixel.

[0028] The imaging system of the present disclosure may have a further configuration capable of aggregating detected spectra belonging to image pixels of each phasor bin, where detected spectra belonging to image pixels of the same bin have substantially the same detected intensity and detected wavelength.

[0029] The imaging system of the present disclosure may have further configurations that may generate a representative image of a target using at least one harmonic of the Fourier transform. The at least one harmonic may be the first harmonic and / or the second harmonic. Such a system may also use only one harmonic. The only one harmonic may be the first harmonic or the second harmonic. Such a system may also use only the first harmonic or only the second harmonic.

[0030] In the present disclosure, the at least one optical component may further include at least one illumination source for illuminating the target, the illumination source generating illumination source radiation comprising at least one illumination wave. Such a system may also further include at least one illumination source generating illumination source radiation comprising at least two illumination waves, each illumination wave having a different wavelength.

[0031] In the present disclosure, the imaging system may further include a control system, a hardware processor, a memory, and a display.

[0032] In this disclosure, the imaging system may have further features that may display a representative image of the target on a display of the imaging system.

[0033] In the present disclosure, the imaging system may further include a control system, a hardware processor, a memory, and an information transmission system, which transmits the representative image of the target to the user in any manner. The information transmission system may transmit the representative image of the target to the user as an image, a numerical value, a color, a sound, a mechanical movement, a signal, or a combination thereof.

[0034] In the present disclosure, the at least one optical component may further include an optical lens, an optical filter, a dispersive optical system, or a combination thereof.

[0035] In the present disclosure, the detected target radiation may be fluorescent radiation.

[0036] A hyperspectral imaging system for generating a representative image of a target is disclosed herein. The hyperspectral imaging system may include an image forming system. The image forming system may be configured to acquire detected radiation of the target. The image forming system may be configured to form a target image using the detected target radiation, the target image including at least two image pixels, each image pixel corresponding to one physical point on the target. The image forming system may be configured to form at least one intensity spectrum for each image pixel. The image forming system may be configured to transform the intensity spectrum of each image pixel based on the intensity spectrum of each image pixel. The image forming system may be configured to form one phasor point on a phasor plane for each image pixel. The image forming system may be configured to form a phasor histogram including at least two phasor bins, each phasor bin including at least one phasor point. The image forming system may be configured to aggregate the detected spectra belonging to the image pixels of each phasor bin. The image forming system may be configured to generate a representative intensity spectrum for each phasor bin. The imaging system may be configured to unmix the representative intensity spectrum of the phasor bins using one or more unmixing techniques. The imaging system may be configured to determine the abundances of the spectral end-members in the representative intensity spectrum. The imaging system may be configured to generate a representative intensity image of the target representative of the abundances of the spectral end-members.

[0037] Disclosed herein is a method for generating a representative image of a target. The method may include forming at least one intensity spectrum for image pixels of the target image, the target image being based on the detected radiation. The method may include implementing a hyperspectral phasor system. The hyperspectral phasor system may be configured to form one phasor point on a phasor plane for each image pixel. The hyperspectral phasor system may be configured to form a phasor histogram including at least two phasor bins, each phasor bin including at least one phasor point. The hyperspectral phasor system may be configured to aggregate the detected spectra of the image pixels for the at least two phasor bins. The hyperspectral phasor system may be configured to generate at least one representative intensity spectrum for the at least two phasor bins. The method may further include implementing an unmixing system. The unmixing system may be configured to unmix the at least one representative intensity spectrum for the at least two phasor bins using one or more unmixing techniques. The method may further include generating a representative intensity image of the target based at least on the representative intensity spectrum and the detected intensities corresponding to the detected radiation.

[0038] Disclosed herein is a method for generating a representative image of a target. The method may include forming at least one intensity spectrum for image pixels of the target image, the target image being based on detected radiation. The method may include generating the at least one representative intensity spectrum based on phasor points on a phasor plane corresponding to the image pixels. The method may include unmixing the at least one representative intensity spectrum using one or more linear unmixing techniques. The method may include generating a representative intensity image of the target based on at least the unmixed representative intensity spectrum.

[0039] Any combination of the above features / configurations is within the scope of the present disclosure.

[0040] These and other components, steps, features, objects, benefits, and advantages will become apparent from review of the following detailed description of exemplary embodiments, the accompanying drawings, and the claims herein.

[0041] The drawings are exemplary embodiments. They do not depict all embodiments. Other embodiments may be used in addition or instead. Details that may be obvious or unnecessary may be omitted to save space or for a more efficient illustration. Some embodiments may be practiced with additional components or steps and / or without all of the components or steps shown. When the same numerals appear in different drawings, they refer to the same or similar components or steps.

[0042] This patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0043] The colors disclosed in the following brief description of the drawings and other parts of this disclosure refer to the color drawings and color photographs originally filed in U.S. Provisional Patent Application No. 63 / 247,688, entitled "A Hyperspectral Imaging System with Hybrid Unmixing," filed September 23, 2021, with Attorney Docket No. AMISC.022PR, the entire contents of which are incorporated herein by reference. [Brief description of the drawings]

[0044] [Figure 1]FIG. 1 shows a schematic of how systems and methods discussed herein, such as those describing Hybrid Unmixing (HyU), may enhance the analysis of multiplexed hyperspectral fluorescence signals in vivo. In this example, (A) a multicolor fluorescent biological sample (here a zebrafish embryo) is imaged in hyperspectral mode to collect the fluorescence spectrum of each voxel in the specimen. (B) HyU represents the spectral data as a phasor plot, which is a 2D histogram of real and imaginary Fourier components (at a single harmonic). (C) A spectral denoising filter may reduce Poisson and instrumental noise on the phasor histogram, providing a first signal improvement. (D) The phasor may act as an encoder, with each histogram bin corresponding to a number n of pixels, each with a relatively similar spectrum (E). Adding these spectra together effectively averages the spectrum for that phasor location. This denoising results in a sharper average spectrum for this set of pixels, ideally suited for analytical decomposition (F) through an unmixing algorithm. (G) Unmixing results in an image separated into its spectral components. Here, linear unmixing (LU) is used for unmixing, but HyU is compatible with any unmixing algorithm. Note that HyU may offer a significant reduction in data size and complexity of the LU (or any other unmixing) calculation because the calculation is applied to 104 histogram bins (D) rather than the approximately 107 voxels in the specimen (A). This dramatically reduces the number of calculations required for LU. [Diagram 2]We show that systems and methods discussed herein, such as those describing hybrid unmixing (HyU), can outperform conventional linear unmixing (LU) in both synthetic and live spectral fluorescence imaging. In this example, (A) hybrid unmixing (HyU) and (B) linear unmixing (LU) were tested using hyperspectral fluorescence simulations generated from four fluorescence signatures (emission spectra, Figure 11E). (C) Absolute mean squared error (MSE) shows that HyU can provide consistent reduction in error over a wide range of photons per spectrum (# photons / independent spectral component is here the result of four reference spectra combined). (D) The performance difference in MSE of HyU relative to LU persists when applying multiple phasor denoising filters (median filters from 0 to 5). Analysis of this synthetic data shows consistent improvement of HyU at low photon numbers, with more than a factor of two improvement when applying five denoising filters at a signal level of 16 photons per spectrum. Shaded areas in line plots represent 95% confidence intervals around the mean. (E) Unmixing of experimental data from four-color zebrafish shows increased contrast for HyU (left) compared to LU (right). Scale bar = 50 μm. (F, G) The increased precision is revealed by residual images of HyU and LU, showing the spatial distribution of unassigned signals after analysis of the data in E. Results show consistently lower residual values ​​for HyU (F) compared to LU (G). (H) Box plots of the residuals in F and G, with n = 1.05 × 106 pixels, present a value of 11% for HyU compared to 77% for LU, with * (p < 10-10). Center: median, Box: First / Third Quartile, Whisker: 1.5 times first, third quartile, Min / Max not shown. (I-L) A zoomed-in rendering of the HyU results (E, white box) clearly shows low levels of bleed-through (MP) between labels. A similar zoom-in of the LU results shows significantly worse performance.Note that areas with bright signal (white arrows in membrane J,N) bleed through to the other channels (M) and (O). Scale bar: 20 µm. Tetra-labeled specimens used in this example were Gt(cltca-citrine), Tg(ubiq:lyn-tdTomato, ubiq:Lifeact-mRuby; fli1:mKO2). [Diagram 3]We show that systems and methods discussed herein, such as those describing hybrid unmixing (HyU), can enhance unmixing for low signal in vivo multiplexing to achieve deeper volumetric imaging. In this example, (A) hybrid unmixing (HyU) volumetric renderings compared to (B) linear unmixing (LU) ones of a four-color zebrafish trunk section demonstrate increased contrast and reduced residuals in the HyU results, especially across deeper sections of the specimen. The four labels in the fish are Gt(cltca-citrine), Tg(ubiq:lyn-tdTomato, ubiq:Lifeact-mRuby, fli1:mKO2), labeling clathrin-coated pits (green), membranes (yellow), actin (cyan), and endothelium (magenta), respectively. (C,E) HyU results show improved spatial resolution and less bleed-through compared to (D,F) LU results. Scale bar: 20 μm. Observing a zoomed-in visualization of the surface area of ​​the specimen, the yellow signal distinctly marks the membrane, while the cyan signal clearly labels actin in (C) HyU. The same signals are not distinct in (D) LU due to multiple misassigned magenta pixels that bleed through and degrade the true signal in the other channel. Similarly, for a zoomed-in visualization of the perivascular area of ​​the embryo, in (E) HyU the yellow and magenta signals clearly distinguish the membrane and vasculature, while in (F) LU the result is compromised by a larger noise. (G,H) Intensity line plots of each of the four result signals for HyU (solid lines) and LU (dashed lines) demonstrate an improved profile with significantly reduced noise peaks in HyU compared to LU. Intensity is scaled by the maximum value of each unmixed channel. DL: digital level. (I) Box plots of relative residual values ​​as a function of z-depth for HyU and LU highlight the improvement in the unmixing results. HyU has an unmixing residual of 6.6% ± 5.3% compared to 58% ± 17% for LU. The average amount of residual is 9 times lower with HyU and the variance of the residual is narrower. For each z-slice, n = 5.2 × 105 pixels.Center: median, box: 1st / 3rd quartiles, whiskers: 1.5 times 1st, 3rd quartiles, min / max values ​​not shown. [Figure 4] We show that systems and methods discussed herein, such as those describing HyU, can reveal the dynamics of the developing vasculature by enabling multiplexed volumetric time-lapse. Hybrid unmixing (HyU) can overcome challenges in performing multiplexed volumetric time-lapse in vivo imaging of developing embryos. This example shows HyU rendering of the trunk section of (A) three-color zebrafish Gt(cltca-citrine);Tg(kdrl:mCherry, fli1:mKO2) at time point 0. (B) HyU unmix results enable quantitative analysis and segmentation, here an example showing the time evolution of segmented volumes of mCherry (vasculature, magenta), mKO2 (endothelial lymphatic system, yellow), and citrine (clathrin-coated pits, cyan). Box and line plots were generated using ImarisVantage, as described in the methods. (C1-4) Time-lapse imaging of the formation of the vasculature over 300 min at 0, 100, 200, and 300 min (zoomed-in rendering of the box in A). This example shows that HyU can provide good unmixing at low light levels to allow multiplexing to be used for the observation of expression in live embryos. [Diagram 5]We show that systems and methods discussed herein, such as those describing HyU, can enable the discrimination and unmixing of low-photon endogenous signals in conjunction with exogenous signals. In this example, (A) HyU results for a whole zebrafish embryo can provide a frame of reference for improved unmixing of exogenous signals as well as its increased sensitivity to enable the discrimination and unmixing of endogenous signals inherently present in a low-photon environment. (B) HyU results for the head region (box in A) can reveal the simplicity of discerning an unknown autofluorescence signal among multiple exogenous signals using the phasor method for tetragenic zebrafish Gt(cltca-citrine), Tg(ubiq:lyn-tdTomato, ubiq:Lifeact-mRuby, fli1:mKO2) imaged across multiple tiles. Scale bar: 80 μm. (C) Input spectra required to perform unmixing are easily identified on the phasor plot when (D) visualizing each spectrum as a spatial location. Phasor provides simplified identification and selection of independent and unexpected spectral components in the encoded HyU approach. Endogenous signals are known to have low photon emitted, leading to their inability to be unmixed using traditional unmixing algorithms. (E) A zoomed-in acquisition of the embryo's head region (box in A) displays the HyU unmixing results of many endogenous and exogenous signals when under very low photon output, an experimental condition that was previously very difficult to unmix. Scale bar: 70 μm. (F) The phasor plot representation provides eight independent fluorescent fingerprint locations that are easily identifiable. (G) Spectra corresponding to each of the eight independent spectral components are also provided as a reference. Colors in (F) match the renderings in (E) and (G). NADH bound (red), NADH free (yellow), retinoid (magenta), retinoic acid (cyan), reflectance (green), elastin (purple), and exogenous signals: mKO2 (blue), and mRuby (orange). All signals were excited with single-photon lasers at both 488 nm and 561 nm (A-D), or with a two-photon laser at 740 nm (E-G). [Figure 6] We show that systems and methods discussed herein, such as those describing HyU, can push the upper limits of live multiplexed volumetric time-lapse imaging of endogenous and exogenous signals. The increased sensitivity of HyU provides a facile solution to the challenging task of imaging time-lapse data at six time points (125 min) for both endogenous and exogenous signals in four transgenic zebrafish: Tg((cltca-citrine), (ubiq:Lifeact-tdTomato), (ubiq:Lifeact-mRuby), (fli1:mKO2)). (A)-(F) Volumetric rendering of HyU results for time points acquired at 25 min intervals can reveal high contrast and multiplexed labeling of NADH bound (red), NADH free (yellow), retinoid (magenta), retinoic acid (cyan), mKO2 (green), and autofluorescence from blood cells (blue) when excited at 740 nm. Additional exogenous signals of mKO2 (yellow), tdTomato (magenta), mRuby (cyan), citrine (green), and blood cell autofluorescence (blue) are also easily unmixed using HyU when the sample is excited at 488 / 561 nm. HyU may offer the ability to simultaneously multiplex nine signals in a live sample over an extended period of time, a task that has not been previously explored. Scale bar: 50 μm. [Figure 7]We show how the systems and methods discussed herein, such as those describing HyU, can reduce noise and signal bleed-through compared to traditional bandpass filter imaging. Imaging of four transgenic zebrafish Gt(cltca-Citrine), Tg(ubiq:lyn-tdTomato, ubiq:Lifeact-mRuby, fli1:mKO2) (same data as in Figure 3) was performed using (A) four-channel optical filter imaging and (B) multispectral imaging and hybrid unmixing (HyU) analysis. Bleed-through from the overlapping emission spectra of the fluorophores is present in A. This artifact is a result of the sharp spectral discretization imposed on the fluorescent signal by the optical filters, which fails to produce a clear distinction between Citrine (480 nm to 690 nm), mKO2 (525 nm to 690 nm), tdTomato (530 nm to 690 nm), and mRuby (560 nm to 690 nm). The fluorophores are well separated in B. The colors in both A and B represent citrine (green), mKO2 (yellow), tdTomato (magenta), and mRuby (cyan). [Figure 8] We show how systems and methods discussed herein, such as those describing HyU, can outperform current methods. True color rendering of 32-channel quadruple-labeled Gt(cltca-citrine), Tg(ubiq:lyn-tdTomato, ubiq:Lifeact-mRuby) zebrafish shows indistinguishability of multiple labels in the absence of analysis. (B) Optical filter imaging displays strong bleed-through across the four channels. (C) Classical unmixing provides increased contrast between labels while still being affected by erroneous reassignment of signals. (D) Hybrid unmixing enhances separation of spectrally and spatially overlapping signals. (E-H) Zoom-ins from the white boxes in A, B, C, D, respectively. Scale bars = 100 μm. [Figure 9]Comparison of synthetic data unmixing results at different SNRs that may demonstrate improved HyU performance. Ground truth photon masks of four independent fluorescent signals: (A) mKO2, (B) Citrine, (C) mRuby, and (D) tdTomato for synthetic data. (E) Maximum intensity projection (MIP) of a simulated 32-channel hyperspectral image generated from the four ground truth masks at low signal-to-noise ratio (SNR). In this case, up to 10 photons are simulated for each fluorescent component. (F-I) Grayscale representation of the maximum emission channel of each component based on their respective spectra. (J) LU and (K) HyU unmixing results for simulations with up to 10 report a degradation in performance. In very low SNR simulations (up to 5 photons for each component), both LU (I) and HyU (M) results degrade, but HyU maintains a 1.5x lower average MSE compared to LU. [Figure 10]Quantification of HyU vs. LU unmixing results for synthetic data highlighting improved HyU performance is shown. HyU performance is evaluated under several algorithm parameters and experimental conditions. (A) Relative MSE between HyU and LU calculated as a function of maximum input photons / spectrum over 5 denoising filter passes for HyU. Improvement increases with both the number of photons and the number of denoising filters, showing a significant difference above 7 photons / spectrum with a peak of 124%. Shaded regions represent 95% confidence intervals around the mean. (B) Absolute MSE from LU and HyU algorithms for the same synthetic data set with and without a beam splitter. The addition of an optical filter increases the MSE of LU by 8% on average compared to an average increase of 5% for HyU. N=1.05e6 pixels. Center: median, box: 1st / 3rd quartiles, whiskers: 1.5x 1st, 3rd quartiles, min / max not shown. (C) Average relative residuals of synthetic data with increasing levels of denoising, with and without a beam splitter. The average relative residual without a beam splitter with denoising (HyU-filter 1-HyU-filter 5) is 83% compared to 109% for LU. Without a denoising filter (filter 0), the average relative residual is 92.9%. A beam splitter was applied to this simulation and both mean squared error (MSE) and residual values ​​were calculated with and without a beam splitter. N=1.05e6 pixels. Center: median, box: 1st / 3rd quartiles, whiskers: 1.5x 1st, 3rd quartiles, min / max not shown. (D) Simulated spectra with and without a beam splitter and (E) simulated spectra without a beam splitter are shown. [Figure 11]Residual analysis of synthetic data is shown, which allows identifying locations with reduced algorithm performance. The simulated data from Figure 2 for four fluorescent labels (Citrine, mKO2, tdTomato, mRuby) are analyzed with LU and HyU. (A) Unmixing results for HyU and (B) LU. Residual image maps for (C) HyU and (D) LU results present areas with higher residuals (red) along the boundary between the labeled features of the sample and the background, where the signal-to-noise drops. The average residual value for LU (118%) is higher than for HyU (94%). (E) Residual phasor map shows higher residuals for background regions (arrows), consistent with the results in C, D. Jet color bar scale corresponds to C, D, and E. (F) Phasor residual intensity histogram maps the average photon count in each histogram bin in E from 0 to 50 photons, presenting a trend of decreasing relative residuals with photon number. (G) Average relative residual plot shows higher values ​​for LU compared to HyU with different denoising filters applied. Ground truth values ​​are also included for comparison. N=1.05e6 pixels. Center: median, box: 1st / 3rd quartiles, whiskers: 1.5x 1st, 3rd quartiles, min / max not shown. (H) Original phasor plot with 0 threshold and 5 denoising filters applied is presented. ROI (yellow circle) highlights background pixels in yellow in (I) average spectral intensity image. Noise from background and residuals can be significantly reduced with intensity thresholding. [Figure 12]A schematic of the residual calculation is shown. The image residual is the residual for an image (x, y). (A) A raw hyperspectral data cube of dimensions (x, y, λ). x, y are the spatial dimensions. λ is the wavelength range from the spectral channels on the detector. (D) A recovered model of dimensions (x, y, λ) is obtained from (B) the product of the recovery ratios (x, y, ch) and the independent spectra (ch, λ). ch is the number of the independent spectrum or unmixing component. (C) The residual is the difference between the recovered model and the raw data. (E-H) Same logic for the phasor residual, but instead of (x, y), the phasor dimensions consist of real and imaginary Fourier components (G, S). [Figure 13] Exemplary unmixing of tetragenic zebrafish with HyU and LU highlighting improvements in contrast and spatial features. Volumetric zoom-in view of a somite in the trunk region of a 10 dpf Gt(cltca-citrine), Tg(ubiq:lyn-tdTomato, ubiq:Lifeact-mRuby, fli1:mKO2) merges all channels in (A) HyU and (B) LU. (A-E) HyU presents a broad dynamic range of intensity with 1.11-fold higher average contrast than LU compared to (F-J) LU. In LU, bleed-through from (H) membrane labeling (arrowheads) is observed in (G) lymphatic channel (arrowhead) and (I) actin channel (arrow). This erroneous reassignment of intensity is absent in the corresponding HyU channels for (B) vasculature and (D) actin fibers (arrows) are cleanly unmixed. (K) Phasor residual distribution shows the distribution of relative residuals (%) and photon counts in phasor histogram bins. Residual distribution shows the distribution of relative residuals (%) and photon counts in histogram pixels for both (L) HyU and (M) LU. [Figure 14]Residual analysis of experimental data supporting the improved performance of HyU. Residual analysis for multispectral fluorescence data of 5 dpf four transgenic zebrafish Gt((cltca-citrine), Tg(ubq:lyn-tdTomato), (ubiq:Lifeact-mRuby), (fli1:mKO2)) in Figure 3. Unmixing results for (A) HyU and (B) LU, respectively. Residual image maps of z-average datasets for (C) HyU and (D) LU show lower residual values ​​for HyU, suggesting improved unmixing quality. Residual distribution relative to the original intensity at each pixel as a function of estimated photon counts per spectrum for (E) LU and (F) HyU. (G) Residual phasor map presents increased residual values ​​in background regions (arrows). Jet color map scale refers to C, D, and G. (H) Residual phasor histogram for HyU shows the distribution of residuals over a wide dynamic range of photons for the experimental data. (I) Raw phasors with 0 threshold applied and 5 rounds of denoising filters, ROI (yellow circle) highlights background pixels in (J) average spectrum image (first z-slice shown). [Figure 15]Application of denoising filters reveals improved results with lower residuals. (A) Residual image maps of HyU unmixing of tetragenic zebrafish Gt(cltca-citrine), Tg(ubiq:lyn-tdTomato, ubiq:Lifeact-mRuby, fli1:mKO2) containing one strong autofluorescence signal. Residual values ​​are calculated using different denoising filter times. 8 The average relative residual clearly decreases with increasing denoising filter times. (B) Residual phasor maps show a significant decrease in values ​​between 0 and 1 denoising filter times, maintaining statistically similar values ​​at higher denoising filter applications. (C) Phasor plots for different denoising filters. Phasors of raw data before denoising or thresholding present high count regions corresponding to noise connected to each of the detectors, as well as background regions. Phasor plot distribution highlights areas where higher pixel counts arise from background noise, which is consistent with the lower residual phasor map values ​​in B. (D) The average residual values ​​for the residual image map (A) and the residual phasor map (B) highlight that the improvement in the residuals is mainly focused on the first application of the denoising filter. With this initial denoising, the average relative residual is reduced from 69.8% to 46.8%, and further reduced to 42.6% after five denoising passes. The average relative residual for the phasors is reduced from 33.9% to 7.1% after one denoising filter application, and further reduced to 2.1% after five denoising passes. When applying a standard processing threshold of 250 digital levels (lower 0.38% intensity in 16-bit format), the average relative residual is reduced from 7.2% to 4.6%, and further reduced to 4.1% after five denoising filters. The average relative residual for the phasors is reduced from 10.1% to 2.6%, and further reduced to 1.1% after five denoising filters. The bars represent the variance of the relative residual values. [Figure 16]Comparison of LU and HyU residual images highlighting improved HyU performance. Residual image projections for LU and HyU of a 3D dataset of 3dfp tetra-transgenic zebrafish Gt(cltca-citrine), Tg(ubiq:lyn-tdTomato, ubiq:Lifeact-mRuby, fli1:mKO2) with an intensity threshold of 250. (A) The LU residual image map of a single slice (z=8 of 17 in the z-stack) provides an average relative residual of 35.9%, while (B) the corresponding map for HyU provides an average of 7.1%. (C) The residual phasor map for the z-stack presents an average relative residual of 1.1%. The reduction in residual for HyU is maintained across the z-stack, as shown in (D) average LU and (E) HyU residual image maps constructed from the average of the residuals across all z-slices. The average residual improvement of 4% for HyU is 5.3 times that of LU at 21%. [Figure 17]Residual maps that can facilitate the identification of independent spectral components are shown. Experimental fluorescence microscopy data often contain unexpected autofluorescence signals. Residual maps (Methods) provide additional information to account for these signals and to appropriately adjust the HyU analysis. (A) Mean intensity image with pixels pseudocolored in cyan (autofluorescence) and magenta (background) according to the ROI selection on the residual phasor map (B), calculated from the HyU of the four input spectra with a threshold of zero. The pseudocolored regions of the image match with the regions presenting high residual values ​​in the residual image map (C). By modifying the unmixing input to include the unexpected autofluorescence spectrum (cyan) and performing the residual phasor map selection, a background pseudocolored (magenta) image is generated (D). Including autofluorescence as an independent spectral component in the unmixing reduces the number of pixels corresponding to the autofluorescence signal (cyan ROI) in the residual phasor map (E), thereby matching with the residual image map (F) that no longer presents high residuals in the central part of the image. Increasing the threshold to 250 removes pixels with high residuals that correspond to the background and removes them from the (G) average intensity image, (H) residual phasor map, and (I) residual image map. [Figure 18] HyU analysis of a 36 hpf casper zebrafish is shown, demonstrating the feasibility of unmixing only endogenous signals. Casper zebrafish are transgenic zebrafish characterized by the absence of pigment. The dataset was acquired in two-photon spectral mode at 740 nm excitation. HyU unmixing was performed utilizing five pure endogenous signals measured in solution (Methods): (A) merged overview of all signals, (B) NADH bound, (C) NADH free (yellow), (D) retinoids (magenta), (E) retinoic acid (cyan) which appears primarily in the yolk sac, a known location where carotenoids are stored, transported and then metabolized to retinoic acid, (F) elastin (green) has a similar distribution within the zebrafish floor plate at this developmental stage, (G) phasor, (H) average spectrum from selection in G. [Figure 19] Figure 1 shows speed comparison and improvement plots of multiple unmixing algorithms in their original form versus coded HyU. (A) Computation time of multiple unmixing algorithms for both the original (per pixel) and HyU versions across a range of hyperspectral imaging dataset sizes. (B) Speed ​​improvement using the ratio of HyU to original unmixing demonstrates a significant increase in speed for all algorithms except LU across all input data sizes. (C, D) Computation time and speed improvement of the original and HyU versions of LU show that the original version of LU offers approximately 2x higher computation speed. Plots A-C use a logarithmic scale while plot D uses a linear scale for the y-axis. [Figure 20] Residuals for synthetic and experimental data are shown. (A) Data were simulated with the aim of covering a wide range of noise and allowing a thorough test of the algorithm's performance. In this example, a ground truth spectrum with 14 photons (dashed red line) is simulated considering multiple types of noise (dark green line). The simulated spectrum presents a cluttered shape with substantial presence of noise. During HyU analysis, in the encoding of the spectrum within a phasor bin, the simulated spectrum (dark green line) is averaged with similar spectra of multiple other pixels to produce (light green light). The spectrum recovered with hybrid unmixing (orange line) is similar to the ground truth. However, in the calculation of residuals, the absence of noise in the clean data is counted as residual due to the cluttered signal of the simulation (dark green line). (B) Spectrum from experimental data (from Figure 3) in a similar photon range (15-20 photons) for comparison. Due to the nature of the experimental data, without the ground truth, the same color code for the spectral lines is utilized. [Figure 21]HyU unmixing for low concentration signals using customized independent spectra. Results of unmixing endogenous and exogenous signals in four transgenic zebrafish: Gt(cltca-citrine), Tg(ubiq:lyn-tdTomato, ubiq:Lifeact-mRuby, fli1:mKO2) at single time points of the dataset presented in Figure 6 provide further information and highlight the weak expression of some exogenous signals in this dataset. (A) Input spectra for the endogenous signatures were obtained directly by endmember selection in the phasor plot. Input spectra for the exogenous signatures were obtained from other datasets of samples expressing those signatures individually and excited with two-photon at 740 nm, since these exogenous signals are not strongly expressed in this dataset. (B) Rendering of the unmixing results was auto-adjusted to show optimal contrast. Unmixing can be performed even with spectra from weak input signatures. (C) Histogram counts of each unmixed independent spectral signature demonstrate the lower signal of the exogenous fluorescent signature compared to the endogenous signature. The median values ​​for the mRuby and tdTomato channels are 57 and 77 digital levels, respectively, significantly lower than the median values ​​of the other signals. [Figure 22]Figure 1 shows the improvement in relative mean squared error (RMSE) for combinations of simulated fluorescence spectra highlighting the improvement in HyU performance over multiple rounds of denoising filters. Twelve matrices demonstrate the RMSE improvement of HyU over LU when unmixing ensembles of synthetic data with 2-8 exogenous labels (Y-axis of each matrix) as a function of the spatial overlap of these labels in the sample (X-axis of each matrix). In the matrices, 0% overlap represents a simulation with spatially distinct fluorophores, where each pixel corresponds to a single fluorophore, while a simulation with 100% overlap contains a randomized ratio of n fluorophores in every pixel. Each of the values ​​reported in the matrices is the average of 1024 x 1024 x 32 pixel simulations and shows the RMSE improvement of HyU over LU. Different columns in the figure report the RMSE improvement matrices with different rounds of denoising filters applied (0 times, 1 time, 3 times, 5 times) with a total number of photons per pixel of (A) 16, (B) 32, (C) 48. Without the denoising filter, the overall improvement in HyU is less than 8%. The denoising filter improves the RMSE by over 80%. The spectra used in this simulation are reported in Figure 26A. [Diagram 23]Figure 1 shows the RMSE improvement for a combination of simulated fluorescence and autofluorescence spectra highlighting the improvement in HyU performance over multiple rounds of denoising filters. Twelve matrices demonstrate the RMSE improvement of HyU over LU when unmixing a set of synthetic data with 2-8 exogenous and endogenous labels (Y-axis of each matrix) as a function of the spatial overlap of these labels in the sample (X-axis of each matrix). In the matrices, 0% overlap represents a simulation with spatially distinct fluorophores where each pixel corresponds to a single fluorophore, while a simulation with 100% overlap has a randomized ratio of n exogenous and endogenous fluorophores at every pixel. Each of the values ​​reported in the matrices is the average of 1024 x 1024 x 32 pixel simulations and shows the RMSE improvement of HyU over LU. The different columns in the figure report the RMSE improvement matrix with different times of the denoising filter applied (0, 1, 3, 5 times) with a total number of photons per pixel of (A) 16, (B) 32, (C) 48. Without the denoising filter, the overall improvement in HyU is less than 25%. The denoising filter improves the RMSE by more than 100%. The spectra utilized for this simulation are reported in Figure 26B. [Figure 24]Figure 1 shows the RMSE improvement for combinations of simulated fluorescence spectra highlighting the overall degradation in performance over a reduced number of spectral channels. Fifteen matrices demonstrate the RMSE improvement of HyU over LU when unmixing ensembles of synthetic data with 2-8 exogenous labels (Y-axis of each matrix) as a function of the spatial overlap of these labels in the sample (X-axis of each matrix). In the matrices, 0% overlap represents a simulation with spatially distinct fluorophores where each pixel corresponds to a single fluorophore, while a simulation with 100% overlap includes a randomized ratio of n fluorophores at every pixel. Each of the values ​​reported in the matrices is the average of 1024 x 1024 x 32 pixel simulations and shows the RMSE improvement of HyU over LU using three denoising filters. The columns in the figure represent the RMSE improvement matrix over increasing numbers of binned spectral channels (32, 16, 8, 6, 4) applied at a total number of photons per pixel of (A) 16, (B) 32, (C) 48. When utilizing 32 spectral channel data, the RMSE improvement reaches greater than the previously reported 80% for highly overlapping fluorophores. Successively increasing the binning across the wavelength dimension (thus decreasing the number of spectral channels) shows a slow downward trend in RMSE improvement until a four spectral channel matrix, where the RMSE improvement drops dramatically below 8%, especially for six or more labels. The spectra utilized for this simulation are reported in Figure 26A. [Diagram 25]Figure 1 shows the RMSE improvement for a combination of simulated fluorescence and autofluorescence spectra highlighting the overall performance degradation over a reduced number of spectral channels. Fifteen matrices demonstrate the RMSE improvement of HyU over LU when unmixing a set of synthetic data with 2-8 exogenous and endogenous labels (Y-axis of each matrix) as a function of the spatial overlap of these labels in the sample (X-axis of each matrix). In the matrices, 0% overlap represents a simulation with spatially distinct fluorophores where each pixel corresponds to a single fluorophore, while a simulation with 100% overlap includes a randomized ratio of n exogenous and endogenous fluorophores at every pixel. Each of the values ​​reported in the matrices is the average of 1024 x 1024 x 32 pixel simulations and shows the RMSE improvement of HyU over LU. The columns in the figure represent the RMSE improvement matrix using three denoising filters over increasing numbers of binned spectral channels (32, 16, 8, 6, 4) applied with a total number of photons per pixel of (A) 16, (B) 32, (C) 48. When utilizing 32 spectral channel data, the RMSE improvement reaches up to the previously reported 100% for highly overlapping fluorophores. Successively increasing the binning across the wavelength dimension (and thus decreasing the number of channels) shows a slow downward trend in RMSE improvement until a four spectral channel matrix, where the RMSE improvement drops dramatically below 25%, especially for three or more labels. The spectra utilized for this simulation are reported in Figure 26B. [Figure 26]Emission spectra of components in overlap simulations are shown. (A) Emission spectra from eight fluorophores including tdTomato, Citrine, mKO2, mCherry, GFP, Alexa610, DAPI, and CFP used in fluorescence (FL) overlap simulations. (B) Emission spectra from eight exogenous and endogenous fluorophores including tdTomato, Citrine, mKO2, mRuby, NADH-bound, NADH-free, retinol, and retinoic acid used in autofluorescence (autoFL) overlap simulations. [Figure 27] Pre-identified locations for common fluorophores on the phasor map are shown. (A) Pre-identified exogenous label locations (g,s) are represented on the phasor plot for the first harmonic and (B) the second harmonic. (C) Intrinsic label locations are further added on the phasor plot for the first harmonic and (D) the second harmonic. The second harmonic generally covers a larger portion of the phasor space compared to the first harmonic. However, for intrinsic signals, the pure autofluorescence spectral locations are on average more separated when utilizing the first harmonic. Details of the pure spectral sources of these locations are reported in the Methods - Independent Spectral Signatures. [Figure 28]Retinol and flavin adenine dinucleotide (FAD) autofluorescence in high magnification hindbrain regions of zebrafish embryos. (A) Phasor analysis reveals distinct autofluorescence spectral components (magenta dots) when imaging a wild-type zebrafish brain at 22 hpf at high magnification (pixel size = 0.078 × 0.078 µm) and high power utilizing two-photon excitation at 740 nm (Table S1). (B) Corresponding emission spectrum from phasor selection in A. The spectrum corresponding to the magenta phasor selection in A closely matches the spectral signal of FAD obtained from in vitro solutions (Methods-independent spectral signatures), accounting for changes in the local environment. (C) FAD unmixing channel highlights FAD clusters in the zebrafish head region. (D) Composite image rendering of the unmixing results for endogenous signals: NADH-bound, NADH-free, retinoid, retinoic acid, FAD, and elastin. [Figure 29]Exemplary phasor analysis for signal distortion in deep tissues. Images of different Z positions of a 3D (x, y, z) dataset of a 19 hpf Tg(ubiq:lyn-tdTomato) zebrafish were acquired from depths of 0 µm to 80 µm (relative to the dataset) for each of the 13 z slices. (A-D) Phasors calculated from single slices at depths of 0 µm, 26 µm, 52 µm, and 78 µm. (E-H) Corresponding average intensity images (across 32 spectral channels) for each z slice show the expected decrease in fluorescence intensity with depth. (I) Average spectra of pixels linked to phasor position bins for the tdTomato fluorescence signature for each of the four presented z slices (0, 26, 52, 78 µm) show a decrease in spectral area without a change in spectral shape, as shown by (J) normalizing the average spectra shown in I to the maximum value of each spectrum. This demonstrates that the spectral shape does not change across different depths (z-planes), while the overall intensity decreases. (K) Randomly selected spectra from the raw spectral images at 0 μm (blue) and 26 μm (red), five spectra for each image, and similar for (L) 52 μm (green) and 78 μm (yellow). Two yellow and three green spectra are not clearly visible due to low signal intensity. [Diagram 30]Figure 1 shows an exemplary comparison of HyU vs. hyperspectral phasor (HySP) results from spectrally overlapping and spatially dispersed samples. We present the results of unmixing using HyU and HySP on a spectrally overlapping and spatially dispersed dataset collected from triple-labeled transgenic zebrafish embryos obtained by injecting mRNA-encoding H2B-cerulean (cyan) into double transgenic embryos Gt(desm-citrine)ct122a / +, Tg(kdrl:eGFP) (magenta and yellow, respectively). (A-F) HyU unmixing results, and (G-L) HySP unmixing results renderings for the dataset. (F) Line profiles of HyU analysis results (B,H dashed lines) show the similarity of signals between the two methods for all channels in non-overlapping samples. (A,F) Volumetric images show the similarity between HyU and HySP results. This is further demonstrated for (B,H) single z-slice results for only (C,I) the Citrine channel, (D,J) the Cerulean channel, and (E,K) the mCherry channel. (F,L) Line profiles for the lines shown in B and H respectively also demonstrate similar results for HyU and HySP for non-spatially overlapping samples. [Diagram 31]1 shows an exemplary comparison of HyU vs. HySP results from a spectrally overlapped and spatially non-dispersed sample. Unmixing results using HyU and HySP on a spectrally overlapped and spatially non-dispersed dataset collected from 5 dpf double-labeled transgenic zebrafish embryos: Gt(cltca-citrine), Tg(fli1:mKO2), showing frequent combinations of signals in pixels across the dataset. (A-E) HyU unmixing results and (F-K) HySP unmixing results for the dataset. (A,F) Volumetric images show the expected signal overlap between channels in the HyU results and a clearer separation in the HySP results. This is further demonstrated for (C,H) the mKO2 channel, and (D,I) the citrine channel only, and (B,G) results in a single z-slice. (E, J) Line profiles of the lines shown in B and G, respectively, demonstrating the fractional nature of the HyU results compared to the winner-take-all analysis of HySP. [Diagram 32]Exemplary residual analysis of endogenous fluorescence signals of HyU and LU shows robust results for HyU unmixing. Residual analysis was performed on hyperspectral fluorescence data of 3 dpf tetragenic zebrafish Gt(cltca-citrine), Tg(ubq:lyn-tdTomato, ubiq:Lifeact-mRuby, fli1:mKO2) and the unmixing results are reported in Figure 5. Residual analysis results are shown for (A-B) LU and (C-F) HyU, respectively. Residual image maps (Methods) of the z-averaged datasets for LU and HyU in A and C, respectively, show lower residual values ​​for HyU, suggesting an improvement in the quality of unmixing. Residual distributions relative to the original intensity at each pixel as a function of estimated photon counts per spectrum are presented for LU and HyU in B and D, respectively, showing a distribution with lower relative residuals for HyU. (E) The residual phasor map (Methods) shows higher residual values ​​in regions corresponding to the edges of the phasor clusters. (F) The residual phasor histogram for HyU shows the distribution of residuals over a wide dynamic range of photons for the experimental data. [Diagram 33] An exemplary endmember spectrum selection process is shown. (A) Phasor map shows the spectral distribution of data from a single fluorescently labeled sample, in this case a transgenic Tg(ubiq:lyn-tdTomato) zebrafish at 18 hpf. (B) The average spectrum corresponding to the phasor bin selection (red dots in A) can be visualized using the software plots discussed herein. (C) The corresponding average spectrum with relative (top) and absolute (bottom) intensities. The save button allows the spectral data to be exported as a text file that can be reloaded to unmix other data. (D) Unmixing results. More step-by-step information is available in the README file associated with the software of this document. [Diagram 34]An exemplary relationship between spectral SNR and photons / spectrum is shown. A direct relationship between SNR and photons per spectrum is shown here using the calculation of the spectral SNR for various levels of photons per spectrum. Although the spectral SNR has a general trend of increasing values ​​with increasing photons per spectrum, it is not a truly monotonic function. This non-monotonicity demonstrates the limitations of SNR when analyzing spectral images. (A) Absolute spectral SNR and (B) relative spectral SNR follow the same trend of increasing values ​​with increasing photons per spectrum. However, relative spectral SNR better distinguishes the influence of different spectral shapes on SNR. Citrine, mKO2, mRuby, and tdTomato are each easily distinguished by the value of the slope of the regression in ascending order. Even with the same number of photons per spectrum, tdTomato has the spectral shape that provides the best SNR, while Citrine provides the lowest SNR. [Diagram 35] Exemplary endogenous fluorescence signatures in fresh mouse tissues are shown. Endogenous fluorescence signatures in fresh kidney tissues of 7-month-old Balb-c mice were imaged with two-photon excitation at 740 nm in a 150 μm deep volume. Despite the increasing scattering effects of this mammalian tissue with increasing depth, HyU is able to perform unmixing of endogenous fluorescence signals. (A) Volumetric rendering of the unmixing results of five endogenous fluorescence signatures shows results consistent with the literature, as seen in (C) (B-E) orthogonal views of the unmixed (x,y) cross-sections of the volume at 30 μm depth in the sample and their corresponding (B) (x,z) and (E) (y,z) projections. (D) The average autofluorescence signal per acquired spectral (x,y) section across the 150 μm depth of the volume shows a sharp decrease in intensity after 75 μm depth, as seen in the corresponding (y,z) projection in E. [Diagram 36]Exogenous fluorescence signatures in fixed mouse tissue. The performance of HyU was evaluated in imaging the fluorescence signal in highly scattering fixed kidney tissue of 7-month-old Balb-c mice with embedded Cy3 fluorescent beads (Methods) imaged with two-photon 850 nm excitation up to 150 μm depth. (A) Volumetric rendering of the unmixing results of signals from fixed autofluorescence (autoFL), Cy3 beads, background, and second harmonic generation (SHG). (B-E) Orthogonal views of the same volume for a single (x,y) plane of the volume at 90 μm depth with (C) cross-section (yellow hairline in C), (B) 18 μm (z,x) and (E) 4 μm cross-sections (y,z), respectively, showing cross-sections of the unmixed volume containing Cy3 beads at different depths up to 140 μm. (D) Average intensity values ​​per acquired (x,y) spectral image slice as a function of depth reveals a significant loss of fluorescence signal deeper than 110 μm. (F) The average spectrum for each z-plane containing pixels with Cy3 bead signal plotted in absolute intensity (digital level, DL), shows that the intensity decreases with depth, as made visible by the area under the spectrum. (G) The same average spectrum has been normalized and plotted in relative intensity to show the consistency of the spectral shape as a function of depth, with reference to Cy3 beads in solution (dashed line). [Figure 37] 1 illustrates an exemplary hyperspectral imaging system comprising an exemplary optical system and an exemplary image forming system. [Figure 38] FIG. 45 shows an exemplary hyperspectral imaging system with an exemplary optical system that is a fluorescence microscope that can generate an unmixed color image of a target by using an exemplary imaging system that includes features disclosed in, for example, FIGS. [Figure 39] FIG. 45 shows an exemplary hyperspectral imaging system with an exemplary optical system that is a multi-illumination wavelength microscope that can generate an unmixed color image of a target by using an exemplary imaging system that includes features disclosed in, for example, FIGS. [Diagram 40] FIG. 45 shows an exemplary hyperspectral imaging system with an exemplary optical system that is a multi-illumination wavelength device that can generate an unmixed color image of a target by using an exemplary imaging system that includes features disclosed in, for example, FIGS. [Diagram 41] FIG. 45 shows an exemplary hyperspectral imaging system with an exemplary optical system that is a multi-wavelength detection microscope that can generate an unmixed color image of a target by using an exemplary imaging system that includes features disclosed in, for example, FIGS. [Diagram 42] FIG. 45 shows an exemplary hyperspectral imaging system with an exemplary optical system that is a multi-illumination wavelength and multi-wavelength detection microscope that can generate an unmixed color image of a target by using an exemplary imaging system that includes features disclosed in, for example, FIGS. [Diagram 43] An exemplary hyperspectral imaging system with an exemplary optical system that is a multi-wavelength detection device is shown, which can generate an unmixed color image of a target by using an exemplary imaging system including features disclosed in, for example, FIGS. [Diagram 44] An exemplary hyperspectral imaging system with an exemplary optical system that is a multi-wavelength detection device is shown, which can generate an unmixed color image of a target by using an exemplary imaging system including features disclosed in, for example, FIGS. [Diagram 45] 1 illustrates exemplary features of an exemplary imaging system that may be used to generate an unmixed color image of a target. [Figure 46] 1 illustrates exemplary features of an exemplary imaging system that may be used to generate an unmixed color image of a target. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0045] Exemplary embodiments are now described. Other embodiments may be used in addition or instead. Details that may be obvious or unnecessary may be omitted to save space or for a more efficient presentation. Some embodiments may be practiced with additional components or steps and / or without all of the components or steps described.

[0046] The present disclosure generally relates to imaging systems. The present disclosure relates to hyperspectral imaging systems. The present disclosure further relates to hyperspectral imaging systems that generate unmixed color images of a target. The present disclosure further relates to hyperspectral imaging systems configured to provide enhanced imaging of a target using hybrid unmixing techniques. The present disclosure further relates to hyperspectral imaging systems configured to provide enhanced imaging of multiple fluorescent labels using hybrid unmixing techniques that enable longitudinal imaging of multiple fluorescent signals at reduced illumination intensity. The present disclosure further relates to hyperspectral imaging systems for use in diagnosing health conditions.

[0047] The present disclosure also relates to a hyperspectral imaging system for generating a representative image of a target. The hyperspectral imaging system may be configured to implement one or more hybrid unmixing technique(s). For example, one or more hardware computer processors may be configured to execute program instructions to cause the hyperspectral imaging system to perform one or more operations related to hybrid unmixing. Hybrid unmixing, such as those performed by the hardware computer processor and / or those performed by the hyperspectral imaging system, or operations related thereto, may be referred to herein, collectively or individually, as hybrid unmixing (HyU).

[0048] In the present disclosure, a hyperspectral imaging system may include an image forming system that acquires detected radiation of a target, the detected radiation including at least two (target) waves, each target wave having a detected intensity and a different detected wavelength, forms a target image using the detected target radiation, the target image including at least two (image) pixels, each image pixel corresponding to a physical point on the target, forms at least one (intensity) spectrum for each image pixel using the (detected) intensity and (detected) wavelength of each target wave, and performs a Fourier transform based on the intensity spectrum of each image pixel. Using an E-transform, the intensity spectrum of each image pixel may be converted into a complex-valued function, each complex-valued function having at least one real component and at least one imaginary component, and by plotting the value of the real component against the value of the imaginary component, one phasor point may be formed on the phasor plane for each image pixel, the value of the real component being hereinafter referred to as the real value and the value of the imaginary component being hereinafter referred to as the imaginary value, forming a (phasor) histogram comprising at least two phasor bins, each (phasor) bin comprising at least one phasor point.

[0049] In the present disclosure, the imaging system may (further) be configured to aggregate the detected spectra belonging to image pixels of each phasor bin, generate a representative intensity spectrum for each phasor bin, unmix the representative intensity spectrum for the phasor bin by using an unmixing technique, thereby determining the abundance of each spectral end-member of the detected radiation, and assign a color to a corresponding image pixel of the target by using the representative intensity spectrum and the abundance of each spectral end-member in the detected intensities belonging to the image pixels, thereby generating a representative intensity image of the target representing the abundance of each spectral end-member.

[0050] In the present disclosure, the intensity spectra aggregated for each phasor bin may have a relatively similar intensity spectrum or substantially the same intensity spectrum. Adding or averaging these substantially similar intensity spectra effectively averages the intensity spectra to generate a representative (or average) intensity spectrum for that phasor position. Adding or averaging these substantially similar intensity spectra may be accomplished by any mathematically conventional or known method. That is, any adding or averaging mathematical technique that can result in a representative intensity spectrum is within the scope of the present disclosure.

[0051] In this disclosure, the imaging system may also have a (further) configuration that may aggregate detected spectra belonging to image pixels of each phasor bin, and detected spectra belonging to image pixels of the same bin may have substantially the same detected intensity and detected wavelength.

[0052] In this disclosure, any (spectral) unmixing technique that can unmix the detected target emissions, intensity spectra, and / or representative intensity spectra is within the scope of this disclosure. The unmixing technique can be a linear unmixing technique. The unmixing technique can be a fully constrained least squares unmixing technique, a matrix inversion unmixing technique, a non-negative matrix decomposition unmixing technique, a geometric unmixing technique, a Bayesian unmixing technique, a sparse unmixing technique, or any combination thereof.

[0053] The imaging system of the present disclosure may have a further configuration for applying a denoising filter to reduce Poisson noise and / or instrumental noise of the detected radiation. The imaging system may also have a further configuration for applying the denoising filter at least once to both the real and imaginary components of each complex-valued function to generate a denoised real value and a denoised imaginary value for each image pixel. The denoising filter may be applied after the imaging system converts the formed intensity spectrum belonging to each image pixel into a complex-valued function using a Fourier transform and / or before the imaging system forms a phasor point on the phasor plane for each image pixel. The imaging system may also have a further configuration for applying the denoising filter to the real component value and / or the imaginary component value after the imaging system forms a phasor point on the phasor plane for each image pixel. The denoised real value may be used as the real value of each image pixel and the denoised imaginary value of each image pixel may be used as the imaginary value to form a phasor point on the phasor plane for each image pixel.

[0054] An exemplary HyU hyperspectral imaging system that can enhance the analysis of multiplexed hyperspectral fluorescence signals in vivo is shown in FIG. 1. In this example, a multicolor fluorescent biological specimen (here, a zebrafish embryo) can be imaged in hyperspectral mode to collect the fluorescence spectrum of each voxel in the specimen FIG. 1A. The fluorescence spectral data is converted to a phasor plot, which is a 2D histogram of real and imaginary Fourier components (at a single harmonic) (FIG. 1B). A spectral denoising filter can be applied to the values ​​of the phasor points to reduce Poisson and instrumental noise on the phasor histogram and provide signal improvement (FIG. 1C). The phasor transform can act as an encoder such that each histogram bin corresponds to the number n of (image) pixels, each with a relatively similar spectrum (FIG. 1D). Adding or averaging these (intensity) spectra effectively averages the (intensity) spectra to generate a representative intensity spectrum for that phasor location (FIG. 1E). Such generation of a representative intensity spectrum can further denoise the detected image radiation, which may be suitable for analytical decomposition through an unmixing algorithm (FIG. 1F). Unmixing may result in an image separated into its spectral components (FIG. 1G). Here, linear unmixing (LU) is used for unmixing, but HyU is compatible with any unmixing technique. HyU, for example, can be used with a computation time of about 10 7 voxels, not 10 4 Note that this can provide a significant reduction in the data size and complexity of the unmixing calculation, since the HyU approach can be applied to 10 histogram bins (FIG. 1D). This HyU approach can dramatically reduce the number of calculations required for traditional unmixing calculations.

[0055] The hyperspectral imaging system may further comprise an optical system. The optical system may include at least one optical component. The at least one optical component may include at least one optical detector. The at least one optical detector may detect electromagnetic radiation absorbed, transmitted, refracted, reflected, and / or emitted from at least one physical point on the target, thereby forming a target radiation, the target radiation may have a configuration including at least two target waves, each target wave having an intensity and a different wavelength. The at least one optical detector may have a further configuration that may detect the intensity and wavelength of each target wave. The at least one optical detector may also have a further configuration that may transmit the detected target radiation and the detected intensity and detected wavelength of each target wave to the imaging system to be acquired. The imaging system may further comprise a control system, a hardware processor, a memory, and a display. The imaging system may have a further configuration that may display a representative image of the target on the display of the imaging system.

[0056] An example of an exemplary hyperspectral imaging system including an optical system and an image forming system is shown generally in FIG. 37. The hyperspectral imaging system 10 may include an optical system 20, an image forming system 30, or a combination thereof. For example, the hyperspectral imaging system may include an optical system and an image forming system. For example, the hyperspectral imaging system may include an image forming system. Exemplary optical systems are shown in FIGS. 38-44. An exemplary configuration of the image forming system is shown in FIG. 45. An exemplary configuration of the hyperspectral imaging system is shown in FIG. 46.

[0057] Any of the exemplary optical systems shown and / or discussed herein may include at least one optical component. Examples of the at least one optical component are a detector ("optical detector"), a detector array ("optical detector array"), a light source illuminating a target ("illumination source"), a first optical lens, a second optical lens, an optical filter, a dispersive optical system, a dichroic mirror / beam splitter, a first optical filtering system disposed between the target and the at least one optical detector, a second optical filtering system disposed between the first optical filtering system and the at least one optical detector, or a combination thereof. For example, the at least one optical component may include at least one optical detector. For example, the at least one optical component may include at least one optical detector and at least one illumination source. For example, the at least one optical component may include at least one optical detector, at least one illumination source, at least one optical lens, at least one optical filter, and at least one dispersive optical system. For example, the at least one optical component may include at least one optical detector, at least one illumination source, a first optical lens, a second optical lens, and a dichroic mirror / beam splitter. For example, the at least one optical component may include at least one optical detector, at least one illumination source, an optical lens, a dispersive optics, and the at least one optical detector is an optical detector array. For example, the at least one optical component may include at least one optical detector, at least one illumination source, an optical lens, a dispersive optics, a dichroic mirror / beam splitter, and the at least one optical detector is an optical detector array. For example, the at least one optical component may include at least one optical detector, at least one illumination source, an optical lens, a dispersive optics, a dichroic mirror / beam splitter, and the at least one optical detector is an optical detector array, and the illumination source directly illuminates the target. These optical components may form, for example, the exemplary optical systems shown in Figures 38-44.

[0058] Any of the exemplary optical systems shown and / or discussed herein may include an optical microscope, an example of which may be a confocal fluorescence microscope, a two-photon fluorescence microscope, or a combination thereof.

[0059] At least one optical detector shown and / or discussed herein may have a configuration to detect electromagnetic radiation absorbed, transmitted, refracted, reflected, and / or emitted by at least one physical point on a target ("target radiation"). The target radiation may include at least one wave ("target wave"). The target radiation may include at least two target waves. Each target wave may have an intensity and a different wavelength. The at least one optical detector may have a configuration to detect the intensity and wavelength of each target wave. The at least one optical detector may have a configuration to transmit the detected target radiation to an imaging system. The at least one optical detector may have a configuration to transmit the detected intensity and wavelength of each target wave to an imaging system. The at least one optical detector may have any combination of these configurations.

[0060] The at least one optical detector shown and / or discussed herein may include a photomultiplier tube, a photomultiplier array, a digital camera, a hyperspectral camera, an electron multiplying charge coupled device, a Sci-CMOS, a digital camera, or a combination thereof. The digital camera may be any digital camera. The digital camera may be used with an active filter for detection of target radiation. The digital camera may also be used with an active filter for detection of target radiation, including, for example, luminescence, thermal radiation, or a combination thereof.

[0061] The target radiation shown and / or discussed herein may include electromagnetic radiation emitted by the target. The electromagnetic radiation emitted by the target may include luminescence, thermal radiation, or a combination thereof. Luminescence may include fluorescence, phosphorescence, or a combination thereof. For example, the electromagnetic radiation emitted by the target may include fluorescence, phosphorescence, thermal radiation, or a combination thereof. For example, the electromagnetic radiation emitted by the target may include fluorescence. The at least one optical component may further include a first optical filtering system. The at least one optical component may further include a first optical filtering system and a second optical filtering system. The first optical filtering system may be disposed between the target and the at least one optical detector. The second optical filtering system may be disposed between the first optical filtering system and the at least one optical detector. The first optical filtering system may include a dichroic filter, a beam splitter type filter, or a combination thereof. The second optical filtering system may include a notch filter, an active filter, or a combination thereof. The active filter may include an adaptive optics system, an acousto-optic tunable filter, a liquid crystal tunable bandpass filter, a Fabry-Perot interferometric filter, or a combination thereof.

[0062] At least one optical detector shown and / or discussed herein may detect target radiation at a wavelength in the range of 300 nm to 800 nm. At least one optical detector may detect target radiation at a wavelength in the range of 300 nm to 1300 nm.

[0063] At least one illumination source may generate electromagnetic radiation ("illumination source radiation"). The illumination source radiation may include at least one wave ("illumination wave"). The illumination source radiation may include at least two illumination waves. Each illumination wave may have a different wavelength. At least one illumination source may directly illuminate the target. In this configuration, there are no optical components between the illumination source and the target. At least one illumination source may indirectly illuminate the target. In this configuration, there is at least one optical component between the illumination source and the target. The illumination source may illuminate the target at each illumination wavelength by transmitting all illumination waves simultaneously. The illumination source may illuminate the target at each illumination wavelength by transmitting all illumination waves sequentially.

[0064] In the present disclosure, the illumination source may include a coherent electromagnetic radiation source, which may include a laser, a diode, a two-photon excitation source, a three-photon excitation source, or a combination thereof.

[0065] The illumination source radiation may include illumination waves having wavelengths in the range of 300 nm to 1300 nm. The illumination source radiation may include illumination waves having wavelengths in the range of 300 nm to 700 nm. The illumination source radiation may include illumination waves having wavelengths in the range of 690 nm to 1300 nm. For example, the illumination source may be a one-photon excitation source capable of generating electromagnetic radiation in the range of 300 to 700 nm. For example, such a one-photon excitation source may generate electromagnetic radiation that may include waves having wavelengths of about 405 nm, about 458 nm, about 488 nm, about 514 nm, about 554 nm, about 561 nm, about 592 nm, about 630 nm, or combinations thereof. In another example, the light source may be a two-photon excitation source capable of generating electromagnetic radiation in the range of 690 nm to 1300 nm. Such an excitation source may be a tunable laser. In yet another example, the light source can be a one-photon excitation source and a two-photon excitation source capable of generating electromagnetic radiation in the range of 300 nm to 1300 nm. For example, such a one-photon excitation source can generate electromagnetic radiation that can include waves having wavelengths of about 405 nm, about 458 nm, about 488 nm, about 514 nm, about 554 nm, about 561 nm, about 592 nm, about 630 nm, or combinations thereof. For example, such a two-photon excitation source can be capable of generating electromagnetic radiation in the range of 690 nm to 1300 nm. Such a two-photon excitation source can be a tunable laser.

[0066] The intensity of the illumination source radiation may not be greater than a certain level such that the target is not damaged by the illumination source radiation when the target is illuminated.

[0067] The hyperspectral imaging system may include a microscope. The microscope may be any microscope. For example, the microscope may be an optical microscope. Any optical microscope may be suitable for the system. An example of an optical microscope may be a two-photon microscope, a one-photon confocal microscope, or a combination thereof. An example of a two-photon microscope is disclosed in Alberto Diaspro "Confocal and Two-Photon Microscopy: Foundations, Applications and Advances" Wiley-Liss, New York, November 2001, and Greenfield Sluder and David E. Wolf "Digital Microscopy" 4th Edition, Academic Press, August 20, 2013. The entire contents of each of these publications are incorporated herein by reference.

[0068] An exemplary optical system comprising a fluorescence microscope 100 is shown in FIG. 38. The exemplary optical system may include at least one optical component. In this system, the optical components may include an illumination source 101, a dichroic mirror / beam splitter 102, a first optical lens 103, a second optical lens 104, and a detector 106. These optical components may form a fluorescence microscope 100. The exemplary system may be suitable for forming an image of a target 105. The light source may generate illumination source radiation 107. The dichroic mirror / beam splitter 102 may reflect the illumination wave to illuminate the target 105. As a result, the target may emit electromagnetic radiation (e.g., fluorescent light) 108 and reflect back the illumination source radiation 107. The dichroic mirror / beam splitter 102 may filter the illumination source radiation from the target and may substantially prevent the illumination source radiation reflected from the target from reaching the detector. Using these optical components, the detected image of the target and the measured intensity of the target radiation may generate an unmixed color image of the target using the system features / configurations of the present disclosure. For example, this unmixed color image of the target may be generated using any of the system features / configurations shown generally in Figures 45-46.

[0069] An exemplary optical system comprising a multi-illumination wavelength microscope 200 is shown in FIG. 39. This exemplary optical system may include at least one optical component. In this system, the optical components may include an illumination source 101, a dichroic mirror / beam splitter 102, a first optical lens 103, a second optical lens 104, and a detector 106. These optical components may form a hyperspectral imaging system comprising a fluorescence microscope, a reflected light microscope, or a combination thereof. This exemplary system may be suitable for forming an image of a target 105. The illumination source may generate illumination source radiation comprising multiple waves, each wave having a different wavelength. For example, the illumination source in this example may generate illumination source radiation comprising two waves, each having a different wavelength, 201 and 202. The light source may sequentially illuminate the target with each wavelength. The dichroic mirror / beam splitter 102 may reflect the illumination source radiation to illuminate the target 105. As a result, the target may emit and / or reflect back waves of electromagnetic radiation. In one embodiment, the dichroic mirror / beam splitter 102 may filter electromagnetic radiation from the target, substantially allowing the emitted radiation to reach the detector, and substantially preventing illumination source radiation reflected from the target from reaching the detector. In another example, the dichroic mirror / beam splitter 102 may transmit only reflected waves from the target, but substantially filter emitted waves from the target, thereby allowing only reflected waves from the target to reach the detector. In yet another example, the dichroic mirror / beam splitter 102 may transmit both reflected radiation and emitted radiation from the target, thereby allowing both reflected radiation and reflected radiation from the target to reach the detector. In this example, multiple waves, each having a different wavelength, may reach the detector. For example, the electromagnetic radiation reaching the detector may have two waves 203 and 204, each having a different wavelength. By using these optical components, the detected image of the target and the measured intensity of the target radiation may generate an unmixed color image of the target by using the system features / configurations of the present disclosure.For example, this unmixed color image of the target can be generated by using any of the system features / configurations shown generally in Figures 45-46.

[0070] Another exemplary hyperspectral imaging system comprising a multi-wavelength detection microscope 300 is shown in FIG. 40. This exemplary hyperspectral imaging system may include at least one optical component. In this system, the optical components may include a first optical lens 103, a dispersion optics 302, and a detector array 304. These optical components may form a hyperspectral imaging system comprising a fluorescent device, a reflective device, or a combination thereof. This exemplary system may be suitable for forming an image of a target 105. The target may emit and / or reflect waves 301 of electromagnetic radiation. In this example, at least one wave or at least two waves may reach the detector array. Each wave may have a different wavelength. The dispersion optics 302 may form a spectrally dispersed electromagnetic radiation 303. By using these optical components, the detected image of the target and the measured intensity of the target radiation may generate an unmixed color image of the target by using the system features / configurations of the present disclosure. For example, this unmixed color image of the target may be generated by using any of the system features / configurations shown generally in FIGS. 45-46.

[0071] Another exemplary hyperspectral imaging system comprising a multi-wavelength detection microscope 400 is shown in FIG. 41. This exemplary hyperspectral imaging system may include at least one optical component. In this system, the optical components may include an illumination source 101, a dichroic mirror / beam splitter 102, a first optical lens 103, a dispersion optics 302, and a detector array 304. These optical components may form a hyperspectral imaging system comprising a fluorescent device. This exemplary system may be suitable for forming an image of a target 105. The illumination source may generate illumination source radiation including at least one wave 107. Each wave may have a different wavelength. The light source may sequentially illuminate the target with each wavelength. The dichroic mirror / beam splitter 102 may reflect the illumination wave to illuminate the target 105. As a result, the target may emit a wave of electromagnetic radiation. The dichroic mirror / beam splitter 102 may substantially allow the emitted wave 301 to reach the detector array, but may filter the target radiation, thereby substantially preventing waves reflected from the target from reaching the detector array. In this example, the emitted radiation reaching the detector array may include multiple waves, each having a different wavelength. The dispersive optics 302 may form a spectrally dispersed electromagnetic radiation 303. By using these optical components, the detected image of the target and the measured intensity of the target radiation may generate an unmixed color image of the target by using the system features disclosed above. For example, this unmixed color image of the target may be generated by using any of the system features shown generally in Figures 45-46.

[0072] Another exemplary hyperspectral imaging system with multiple illumination wavelengths and multiple wavelength detection device 500 is shown in FIG. 42. This exemplary hyperspectral imaging system may include at least one optical component. In this system, the optical components may include an illumination source 101, a dichroic mirror / beam splitter 102, a first optical lens 103, a dispersion optics 302, and a detector array 304. These optical components may form a hyperspectral imaging system with a fluorescence microscope, a reflected microscope, or a combination thereof. This exemplary system may be suitable for forming an image of a target 105. The light source may generate an illumination wave that includes multiple waves, each wave may have a different wavelength. For example, the illumination source in this example may generate an illumination source radiation that includes two waves, each having a different wavelength, 201 and 202. The illumination source may sequentially illuminate the target with each wavelength. The dichroic mirror / beam splitter 102 may reflect the illumination radiation to illuminate the target 105. As a result, the target may emit and / or reflect back electromagnetic radiation. In one example, the dichroic mirror / beam splitter 102 may filter radiation from the target, substantially allowing only emitted radiation to reach the detector array, but substantially preventing radiation reflected from the target from reaching the detector array. In another example, the dichroic mirror / beam splitter 102 may transmit only reflected waves from the target, but substantially filter emitted waves from the target, thereby substantially allowing only reflected waves from the target to reach the detector array. In yet another example, the dichroic mirror / beam splitter 102 may substantially transmit both reflected waves and emitted waves from the target, thereby allowing both reflected waves from the target and the reflected beam to reach the detector. In this example, the beam reaching the detector array may have multiple waves, each having a different wavelength. For example, the beam reaching the detector array may have two waves 203 and 204, each having a different wavelength. The dispersive optics 302 may form a spectrally dispersed electromagnetic radiation 303.Using these optical components, the detected image of the target and the measured intensity of the target radiation may generate an unmixed color image of the target using the system features / configurations of the present disclosure. For example, this unmixed color image of the target may be generated using any of the system features / configurations shown generally in Figures 45-46.

[0073] Another exemplary optical system comprising a multi-wavelength detection device 600 is shown in FIG. 43. This exemplary optical system may include at least one optical component. In this system, the optical components may include an illumination source 101, a first optical lens 103, a dispersion optics 302, and a detector array 304. These optical components may form a hyperspectral imaging system comprising a fluorescence and / or reflectance device. This exemplary system may be suitable for forming an image of a target 105. The illumination source may generate illumination source radiation comprising at least one wave 107. Each wave may have a different wavelength. The light source may sequentially illuminate the target with each wavelength. As a result, the target may emit, reflect, refract, and / or absorb a beam 203 of electromagnetic radiation. In this example, the emitted, reflected, refracted, and / or absorbed beam reaching the detector array may include multiple waves, each having a different wavelength. The dispersion optics 302 may form a spectrally dispersed electromagnetic radiation 303. Using these optical components, the detected image of the target and the measured intensity of the target radiation may generate an unmixed color image of the target using the system features / configurations of the present disclosure. For example, this unmixed color image of the target may be generated using any of the system features / configurations shown generally in Figures 45-46.

[0074] Another exemplary optical system comprising a multi-wavelength detection device 700 is shown in FIG. 44. This optical system may include at least one optical component. In this system, the optical components may include an illumination source 101, a first optical lens 103, a dispersion optics 302, and a detector array 304. These optical components may form a hyperspectral imaging system comprising a fluorescence and / or reflectance device. This exemplary system may be suitable for forming an image of a target 105. The illumination source may generate illumination source radiation comprising at least one wave 107. Each wave may have a different wavelength. The light source may sequentially illuminate the target with each wavelength. As a result, the target may emit, transmit, refract, and / or absorb a beam 203 of electromagnetic radiation. In this example, the emitted, transmitted, refracted, and / or absorbed electromagnetic radiation reaching the detector array may include multiple waves, each having a different wavelength. The dispersion optics 302 may form a spectrally dispersed electromagnetic radiation 303. Using these optical components, the detected image of the target and the measured intensity of the target radiation may generate an unmixed color image of the target using the system features / configurations of the present disclosure. For example, this unmixed color image of the target may be generated using any of the system features / configurations shown generally in Figures 45-46.

[0075] In the present disclosure, the imaging system 30 may include a control system 40, a hardware processor 50, a memory system 60, a display 70, or a combination thereof. An exemplary imaging system is shown in FIG. 37. The control system may be any control system. For example, the control system may control an optical system. For example, the control system may control at least one optical component of the optical system. For example, the control system may control at least one optical detector to detect target radiation, detect the intensity and wavelength of each target wave, transmit the detected intensity and wavelength of each target wave to the imaging system, and display an unmixed color image of the target. For example, the control system may control the movement of the optical components, such as opening and closing an optical shutter, moving a mirror, etc. The hardware processor may include a microcontroller, a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. In one embodiment, all of the processing discussed herein is performed by one or more hardware processor(s). For example, the hardware processor may form a target image, perform phasor analysis, perform a Fourier transform of the intensity spectrum, apply a denoising filter, form a phasor plane, map back the phasor point(s), assign optional color(s), generate an unmixed color image of the target, and combine such configurations. The memory system may be any memory system. For example, the memory system may receive and store inputs from the hardware processor. These inputs may be, for example, a target image, target radiation, intensity spectrum, phasor plane, an unmixed color image of the target, and the like, or combinations of such configurations. For example, the memory system may provide output to other components of the imaging system, such as a processor and / or a display.These outputs may be, for example, a target image, a target emission, an intensity spectrum, a phasor plane, an unmixed color image of the target, or the like, or a combination of such configurations. The display may be any display. For example, the display may display a target image, an intensity spectrum, a phasor plane, an unmixed color image of the target, or the like, or a combination of such configurations. The imaging system 30 may be connected to the optical system 20 via a network. In some cases, the imaging system 30 may be located on a server that is remote from the optical system 20.

[0076] The imaging system may have a configuration that causes an optical detector to detect the target radiation and transmit the detected intensity and wavelength of each target wave to the imaging system.

[0077] The imaging system may have a configuration for acquiring detected target radiation including at least two target waves.

[0078] The imaging system may have a configuration for acquiring target radiation that includes at least two target waves, each wave having a different intensity and wavelength.

[0079] The imaging system can have a configuration for acquiring a target image, the target image including at least two pixels, each pixel corresponding to one physical point on the target.

[0080] The imaging system may have a configuration that uses the detected target radiation to form an image of the target (a "target image"). The target image may include at least one pixel. The target image may include at least two pixels. Each pixel corresponds to a physical point on the target.

[0081] The target image may be formed / acquired in any form. For example, the target image may have a visual form and / or a digital form. For example, the formed / acquired target image may be stored data. For example, the formed / acquired target image may be stored as data in a memory system. For example, the formed / acquired target image may be displayed on a display of an imaging system. For example, the formed / acquired target image may be an image printed on paper or any similar medium.

[0082] The imaging system may be configured to use the detected intensity and wavelength of each target wave to form at least one spectrum (an "intensity spectrum") for each pixel.

[0083] The imaging system may be configured to obtain at least one intensity spectrum for each pixel, the intensity spectrum comprising at least two intensity points.

[0084] The intensity spectrum may be formed / obtained in any form. For example, the intensity spectrum may have a visual form and / or a digital form. For example, the formed / obtained intensity spectrum may be stored data. For example, the formed / obtained intensity spectrum may be stored as data in a memory system. For example, the formed / obtained intensity spectrum may be displayed on a display of an imaging system. For example, the formed / obtained intensity spectrum may be an image printed on paper or any similar medium.

[0085] The imaging system may have a configuration for converting the formed intensity spectrum of each pixel using a Fourier transform into a complex-valued function based on the intensity spectrum of each pixel, each complex-valued function having at least one real component and at least one imaginary component.

[0086] The imaging system may be configured to apply a denoising filter at least once to both the real and imaginary components of each complex-valued function to produce a denoised real value and a denoised imaginary value for each pixel.

[0087] The imaging system may be configured to form a point (a "phasor point") on the phasor plane for each pixel by plotting the denoised real values ​​against the denoised imaginary values ​​of each pixel. The imaging system may form the phasor plane, for example, by using its hardware components, such as a control system, a hardware processor, a memory, or a combination thereof. The imaging system may display the phasor plane.

[0088] The phasor points and / or phasor planes may be formed / obtained in any form. For example, the phasor points and / or phasor planes may have a visual form and / or a digital form. For example, the formed / obtained phasor points and / or phasor planes may be stored data. For example, the formed / obtained phasor points and / or phasor planes may be stored as data in a memory system. For example, the formed / obtained phasor points and / or phasor planes may be displayed on a display of an image forming system. For example, the formed / obtained phasor points and / or phasor planes may be images printed on paper or any similar medium.

[0089] The imaging system may be configured to map back the phasor points to corresponding pixels on the target image based on the geometric location of the phasor points on the phasor plane. In this disclosure, the imaging system may be configured to map back the phasor plane to a corresponding target image based on the geometric location of each phasor point on the phasor plane. The imaging system may map back the phasor points, for example, by using its hardware components, such as a control system, a hardware processor, a memory, or a combination thereof.

[0090] The phasor points and / or phasor planes may be mapped back in any form. For example, the mapped back phasor points and / or phasor planes may have a visual form and / or a digital form. For example, the mapped back phasor points and / or phasor planes may be stored data. For example, the mapped back phasor points and / or phasor planes may be stored as data in a memory system. For example, the mapped back phasor points and / or phasor planes may be displayed on a display of an imaging system. For example, the mapped back phasor points and / or phasor planes may be images printed on paper or any similar medium.

[0091] The imaging system may be configured to assign arbitrary colors to corresponding pixels based on the geometric location of a phasor point on the phasor plane.

[0092] The unmixed color image may be formed in any form. For example, the unmixed color image may have a visual form and / or a digital form. For example, the unmixed color image may be stored data. For example, the unmixed color image may be stored as data in a memory system. For example, the unmixed color image may be displayed on a display of an image forming system. For example, the unmixed color image may be an image printed on paper or any similar medium.

[0093] The imaging system may be configured to display an unmixed color image of the target on a display of the imaging system.

[0094] The imaging system may have any combination of any of the configurations shown and / or described herein, such as those described above.

[0095] The imaging system may generate an unmixed color image of the target using at least one harmonic of the Fourier transform. The imaging system may generate an unmixed color image of the target using at least a first harmonic of the Fourier transform. The imaging system may generate an unmixed color image of the target using at least a second harmonic of the Fourier transform. The imaging system may generate an unmixed color image of the target using at least a first harmonic and a second harmonic of the Fourier transform.

[0096] The denoising filter may be any denoising filter. For example, the denoising filter may be such that the image quality is not impaired when the denoising filter is applied. For example, the detected electromagnetic radiation intensity of each pixel in the image may not change when the denoising filter is applied. An example of a suitable denoising filter may include a median filter.

[0097] An unmixed color image of the target may be formed with at least one spectral signal-to-noise ratio in the range of 1.2 to 50. An unmixed color image of the target may be formed with at least one spectral signal-to-noise ratio in the range of 2 to 50.

[0098] One exemplary embodiment of a hyperspectral imaging system is shown diagrammatically in FIG. 45. In this example, the imaging system may acquire an image of a target (401). The image may include at least two waves and at least two pixels. The system may form an image ("intensity spectrum") of the target using the intensity of each wave and the wavelength of the at least two waves (402). The system may transform the intensity spectrum of each pixel by using a Fourier transform 403, thereby forming a complex-valued function based on the detected intensity spectrum of each pixel. Each complex-valued function may have at least one real component 404 and at least one imaginary component 405. The system may apply a denoising filter 406 at least once to both the real and imaginary components of each complex-valued function. (Denoising may also be applied to the intensity of the target radiation and / or the intensity of the intensity spectrum before and / or after spectrum formation. Such configurations are within the scope of the present invention and are not shown in FIG. 45.) The system may thereby obtain denoised real and denoised imaginary values ​​for each pixel. The system may plot the denoised real values ​​against the denoised imaginary values ​​for each image pixel. The system may thereby form a point on the phasor plane (407). The system may form at least one additional point on the phasor plane by using at least one or more pixels of the image. The system may form a phasor histogram including at least two phasor bins, each phasor bin may include at least one phasor point (408). The system may aggregate the detected spectra belonging to the image pixels of each phasor bin (408). The system may generate a representative intensity spectrum for each phasor bin, for example, by averaging the intensities of the spectra belonging to the same phasor bin (409). The system may unmix the representative intensity spectrum for the phasor bin by using an unmixing technique, thereby determining the abundance of each spectral end member of the detected radiation (410).The system can determine the abundance of each spectral endmember in the representative intensity spectrum and the detected intensities belonging to the image pixels, and generates 411 a representative image of the target showing the abundance of each spectral endmember.

[0099] Another embodiment of a hyperspectral imaging system is shown diagrammatically in FIG. 46. In this example, the hyperspectral imaging system may further include at least one detector 106 or detector array 304. The imaging system may form an image of a target by using the detector or detector array (401). The image may include at least two waves and at least two image pixels. The system may form an image ("intensity spectrum") of the target using the intensity of each wave and the wavelength of the at least two waves (402). The system may transform the intensity spectrum of each pixel by using a Fourier transform 403, thereby forming a complex-valued function based on the detected intensity spectrum of each pixel. Each complex-valued function may have at least one real component 404 and at least one imaginary component 405. The system may apply a denoising filter 406 at least once to both the real and imaginary components of each complex-valued function. (Noise reduction may also be applied to the intensity of the target radiation and / or the intensity of the intensity spectrum before and / or after spectrum formation. Such configurations are within the scope of the present invention and are not shown in FIG. 46.) The system may thereby obtain denoised real and denoised imaginary values ​​for each pixel. The system may plot the denoised real values ​​against the denoised imaginary values ​​for each image pixel. The system may thereby form a point on the phasor plane (407). The system may form at least one additional point on the phasor plane by using at least one or more pixels of the image. The system may form a phasor histogram including at least two phasor bins, each phasor bin may include at least one phasor point (408). The system may aggregate the detected spectra belonging to the image pixels of each phasor bin (408). The system may generate a representative intensity spectrum for each phasor bin (409), for example, by averaging the intensities of spectra belonging to the same phasor bin.The system may use an unmixing technique to unmix the representative intensity spectrum of the phasor bins, thereby determining the abundance of each spectral end-member in the detected radiation (410). The system may determine the abundance of each spectral end-member in the representative intensity spectrum and the detected intensities belonging to the image pixels, and generate a representative image of the target representing the abundance of each spectral end-member (411).

[0100] In the present disclosure, the target can be any target. The target can be any target that has a particular color spectrum. For example, the target can be a tissue, a fluorescent genetic label, an inorganic target, or a combination thereof.

[0101] The system can be calibrated by assigning a color to each pixel using a reference. The reference can be any known reference. For example, the reference can be any reference where an unmixed color image of the reference is determined prior to generation of an unmixed color image of the target. For example, the reference can be a physical structure, a chemical molecule, a biological molecule, a change in physical structure and / or a biological activity (e.g., a physiological change) as a result of a disease.

[0102] The target radiation may include fluorescence. A hyperspectral imaging system suitable for fluorescence detection may include an optical filtering system. An example of an optical filtering system is a first optical filter that substantially reduces the intensity of the source radiation reaching the detector. The first optical filter may be disposed between the target and the detector. The first optical filter may be any optical filter. An example of the first optical filter may be a dichroic filter, a beam splitter type filter, or a combination thereof.

[0103] The hyperspectral imaging system suitable for fluorescence detection may further include a second optical filter. The second optical filter may be disposed between the first optical filter and the detector to further reduce the intensity of the source radiation reaching the detector. The second optical filter may be any optical filter. Examples of the second optical filter may be a notch filter, an active filter, or a combination thereof. Examples of active filters may be an adaptive optical system, an acousto-optical tunable filter, a liquid crystal tunable bandpass filter, a Fabry-Perot interference filter, or a combination thereof.

[0104] A hyperspectral imaging system may be calibrated by assigning a color to each pixel using a reference material. The reference material may be any known reference material. For example, the reference material may be any reference material where an unmixed color image of the reference material is determined prior to generation of an unmixed color image of the target. For example, the reference material may be a physical structure, a chemical molecule (i.e., a chemical compound), a physical structure change, and / or a biological activity (e.g., a physiological change) as a result of a disease. The chemical compound may be any chemical compound. For example, the chemical compound may be a biomolecule (i.e., a chemical compound).

[0105] The hyperspectral imaging system may be used to diagnose any health condition. For example, the hyperspectral imaging system may be used to diagnose any health condition in any mammal. For example, the hyperspectral imaging system may be used to diagnose any health condition in a human. Examples of health conditions may include disease, congenital deformities, disorders, wounds, injuries, ulcers, abscesses, and the like. The health condition may be associated with a tissue. The tissue may be any tissue. For example, the tissue may include skin. An example of a skin or tissue associated health condition may be a skin lesion. The skin lesion may be any skin lesion. An example of a skin lesion may be skin cancer, a scar, acne formation, a wart, a wound, an ulcer, and the like. Another example of a skin or tissue health condition may be the structure of the tissue or skin, such as the moisture level, oil, collagen content, hair content, and the like of the tissue or skin.

[0106] The target may include tissue. The hyperspectral imaging system may display an unmixed color image of the tissue. The health condition may cause a differentiation of the chemical composition of the tissue. This chemical composition may be related to chemical compounds such as hemoglobin, melanin, proteins (e.g., collagen), oxygenated water, or combinations thereof. Due to the differentiation of the chemical composition of the tissue, the color of the tissue affected by the health condition may appear different from the color of the tissue not affected by the health condition. Due to such color differentiation, the health condition of the tissue may be diagnosed. Thus, the hyperspectral imaging system may enable a user to diagnose, for example, a skin condition, regardless of the room lighting and skin pigmentation level.

[0107] For example, illumination source radiation delivered to biological tissue may undergo multiple scattering due to heterogeneity of biological structures and absorption by chemical compounds present in the tissue, such as hemoglobin, melanin, and water, as the electromagnetic radiation propagates through the tissue. For example, the absorption, fluorescence, and scattering properties of tissue may change during disease progression. Thus, for example, reflected, fluorescent, and transmitted light from tissue detected by the optical detectors of the hyperspectral imaging of the present disclosure may carry quantitative diagnostic information regarding tissue pathology.

[0108] The diagnostic information obtained by using the hyperspectral imaging system may determine the health of tissue. This diagnostic information may therefore enhance the clinical outcome of a patient, for example, before, during, and / or after surgery or treatment. The hyperspectral imaging system may be used to track the progression of a patient's health over time, for example, by determining the health of the patient's tissue. In the present disclosure, the patient may be any mammal. For example, the mammal may be a human.

[0109] The reference materials disclosed above may be used in the diagnosis of a health condition.

[0110] Hyperspectral imaging systems, including Hyperspectral Phasor (HySP), may apply a Fourier transform to convert all photons collected across the spectrum into one point in a two-dimensional (2D) phasor plot ("density plot"). The reduced dimensionality may work well in low SNR regimes compared to linear unmixing methods, where errors in each channel may contribute to the fitting results. In any imaging system, the number of photons emitted by the dye during a time interval may be a stochastic (Poisson) process, the signal (total digital count) may be scaled as the average number of acquired photons N, and the noise may be scaled as the square root of N√N. Such Poisson noise of fluorescence emission and detector read noise may be more noticeable at lower light levels. First, the errors on the HySP plot may be quantitatively evaluated. This information may then be used to develop a noise reduction approach that demonstrates that hyperspectral imaging systems, including HySP, are robust systems for resolving time-lapse hyperspectral fluorescence signals in vivo in low SNR regimes.

[0111] The following features are also within the scope of the present disclosure.

[0112] Multispectral fluorescence microscopy may be combined with hyperspectral phasor and linear unmixing to create a hybrid unmixing (HyU) technique. In some examples, dynamic imaging of multiple fluorescent labels in live developing zebrafish embryos and mouse tissues may demonstrate the capabilities of HyU. HyU may be more sensitive to low light levels of fluorescence and allow better multiplexed volumetric imaging over time with less bleaching compared to traditional linear unmixing approaches. HyU may also image both bright exogenous and dim endogenous labels simultaneously due to its high dynamic range. This technique may allow the investigation of cellular behavior, tagged components, and cellular metabolism within the same specimen, providing a powerful window into the coordinated complexity of biological systems.

[0113] Hybrid unmixing (HyU) technique(s) may resolve many of the challenges that have limited the broader acceptance of HFI for applications, such as in vivo imaging. HyU employs a phasor approach merged with traditional unmixing algorithms to more quickly and accurately resolve fluorescent signals from multiple exogenous and endogenous labels.

[0114] The phasor approach, a dimensionality reduction approach for both fluorescence lifetime and spectral imaging analysis, may provide HyU with benefits including spectral compression, noise removal, and computational reduction for both preprocessing and unmixing of HFI datasets.

[0115] Traditional phasor analysis may be fully supervised and may require manual selection of regions or points on a graphical representation of the transformed spectrum called a phasor plot.

[0116] As discussed herein, HyU can utilize phasor processing as an encoder to aggregate similar spectra and apply an unmixing algorithm such as LU to the aggregated similar spectra to provide unsupervised analysis of HFI data, thereby simplifying data processing and removing user subjectivity.

[0117] HyU, for example, may offer three advantages over conventional techniques: (1) improved unmixing over conventional LU, especially for low intensity images, e.g., down to 5 photons per spectrum, (2) simplified identification of independent spectral components, and (3) dramatically faster processing of large data sets, overcoming typical unmixing bottlenecks in in vivo fluorescence microscopy.

[0118] As discussed herein, HyU combines the best features of hyperspectral phasor analysis and unmixing techniques, which may result in faster computational speeds and more reliable results, especially at low light levels.

[0119] In this disclosure, the (intensity) spectrum may be unmixed by any technique. An example of an unmixing technique is the linear unmixing (LU) technique. Examples of unmixing techniques may include (1) fully constrained least squares, (2) matrix inversion, (3) non-negative matrix factorization, (4) geometric unmixing method, (5) Bayesian unmixing method, and (6) sparse unmixing method. For a review of such unmixing techniques, see, e.g., Jiaojiao Wei and and Xiaofei Wang “An Overview on Linear Unmixing of Hyperspectral Data,” Mathematical Problems in Engineering, Volume 2020, Article ID 3735403, pages 1 - 12, https: / / doi.org / 10.1155 / 2020 / 3735403. The entire contents of this publication are incorporated herein by reference. Such unmixing techniques are within the scope of this disclosure.

[0120] Example 1. Exemplary HyU The phasor approaches of the present disclosure can reduce the computational burden because they are compressive, for example reducing the 32 channels of an HFI spectrum plot to positions on a 2D histogram representing the real and imaginary Fourier components of the spectrum (FIGS. 1A, 1B). The different 32 channel spectra are represented as different positions on the 2D phasor plot, and mixtures of the two spectra are rendered at positions along the line connecting the pure spectra.

[0121] The spectral content of an entire 2D or 3D image set is rendered onto a single phasor plot, resulting in dramatic data compression from spectra for each voxel in the image set (e.g., up to gigavoxels or beyond) down to histogram values ​​on the phasor plot (e.g., megapixels).

[0122] In addition, because each "bin" on the phasor plot histogram corresponds to multiple voxels with very similar spectral profiles, the binning itself represents a spectral averaging process that reduces Poisson and instrumental noise (Figures 1C-E).

[0123] Poisson noise in the collected light is unavoidable in HFI unless the excitation is made so high that the statistics of the collected fluorescence produce hundreds or thousands of photons per spectral bin. The clear separation between the spectral phasor plot and its referenced imaging data allows denoising algorithms to be applied to the phasor plot with minimal degradation of image resolution.

[0124] LU or other unmixing approaches applied to spectra on phasor plots can provide a dramatic reduction in the computational load of large image datasets (Figure 1D). To understand this savings, consider the traditional approach of LU applied to image data at the voxel level (Figure 1A, Figure 1F). A time-lapse volumetric dataset of 512 x 768 x 17 (x, y, z) pixels across six time points (Table 1) may require, for example, 40 million operations. HyU may require only about 18,000 operations to unmix each bin on a phasor plot, for example, representing a savings of over 1000 times (Figure 1F, Figure 1G).

[0125] Example 2. Advantages of HyU over conventional LU In this example, to quantitatively evaluate the relative performance of HyU and conventional LU, they were analyzed against a synthetic hyperspectral fluorescence dataset created by computationally modeling the biophysics of fluorescence spectral emission and microscope performance (Figures 2A, 2B, and 9-11). The synthetic dataset was used to quantitatively evaluate the LU and HyU algorithm performance using metrics such as Mean Square Error (MSE) and unmixing residuals (see Figure 12 for both metrics; lower values ​​indicate better performance).

[0126] In addition to the computational efficiency mentioned above, HyU analysis shows a better ability to capture spatial features across a wide dynamic range of intensities compared to conventional LU, primarily due to the noise reduction created by processing in phasor space (Figure 2A,B). The improved accuracy is demonstrated by the lower MSE in comparing the results of HyU with conventional LU to the image ground truth.

[0127] The absolute MSE for HyU can be consistently up to 2x lower than that of conventional LU, especially at low and ultra-low fluorescence levels (Figure 2C). The MSE can be further reduced by the use of a denoising filter on the phasor plot, resulting in the superiority of HyU over conventional LU for HFI at low (5-20 photons / spectrum) and ultra-low (2-5 photons / spectrum) levels (Figure 2D).

[0128] To better characterize the performance on experimental data without ground truth, the unmixing residual can be defined as the difference between the original multichannel hyperspectral images and their unmixed results. The residual provides a measure of how closely the unmixed results reconstruct the original signal (Figure 9). The unmixing residual is inversely proportional to the performance of the algorithm, with low residuals indicating high similarity between the unmixed and original signals. Analysis of the unmixing residuals on synthetic data highlights the improved interpretation of spectral information in HyU, with a mean unmixing residual reduction of 21% compared to the standard (Figure 11C). The reduction in both MSE and mean unmixing residual for synthetic data demonstrates the performance of HyU over conventional LU and provides a baseline comparison when demonstrating performance improvements on experimental data.

[0129] Analysis of experimental data revealing relatively low unmixing residuals and a high dynamic range compared to conventional LU supports the enhanced performance of HyU. Data were acquired from four transgenic zebrafish embryos, Tg(ubiq:Lifeact-mRuby), Gt(cltca-Citrine), Tg(ubiq:lyn-tdTomato), and Tg(fli1:mKO2), labeling actin, clathrin, plasma membrane, and pan-endothelial cells, respectively (Figures 2E-L, 3, and 13-15).

[0130] HyU unmixing of the data shows minimal signal crosstalk between channels, while conventional LU presents significant bleed-through (Figure 2M-P). Consistent with the synthetic data, the unmixing residuals can be utilized as a key indicator for the quality of the analysis on experimental data, in the absence of ground truth.

[0131] Residual images (Figure 2F, G) show a striking difference in performance between HyU and conventional LU. The average relative residuals of HyU represent a 7-fold improvement in disentangling the fluorescence spectra compared to conventional LU (Figure 2H). By zooming in on the details (Figure 2, I-P) to highlight areas affected by bleed-through and difficult to unmix, the unmixed channels can be seen independently (Figure 2, I-L). With a 2-fold higher contrast than conventional LU, HyU reduces the bleed-through effect and produces images with sharper spatial features, leading to better interpretation of the experimental data (Figure 2K, L, and S13).

[0132] Applying HyU to another HFI dataset further highlights HyU's improvements in noise reduction and reconstruction of spatial features for low-photon unmixing (Figure 3, Figure 14). In a zoomed-in image of a single slice of the embryonic skin surface acquired in the trunk region, the HyU image does not correctly display endothelial cells and pan-endothelial (magenta) signal in the epidermis, an area that should be free of mKO2 signal (Figure 3C). In contrast, results from conventional LU show a visually distinct pan-endothelial signal across the entire tissue plane (Figure 3D). This erroneous estimation of the relative contribution of mKO2 fluorescence to conventional LU is likely due to the presence of noise that marshals the spectral profile. This is further depicted in the intensity profile of the mKO2 signal between HyU and LU, with much higher individual peaks from the noise demonstrated for LU (Figure 3G, bottom left). The intensity profiles for both enlarged cross sections of the volume (Figure 3C-F) provide a striking visualization of the HyU improvement. The line intensity profile in HyU exhibits reduced noise and more closely represents the expected distribution of signal (Figure 3G,H). The visible micropattern of actin on the epidermal membrane suggests that the improvements quantified in the synthetic data are maintained in the raw sample signal and the geometric pattern of the microridges. In contrast, results from conventional LU are characterized by noise corruption and the presence of mislocated signals, with high frequency intensity fluctuations that do not match both the labeling and the biological patterns.

[0133] HyU is more accurate, leading to more reliable unmixing results with significantly reduced unmixing residuals throughout the sample depth. The average residual for HyU is 9 times lower and the variance is 3 times narrower than that of conventional LU (Fig. 3I, Fig. 14). This reduction in residuals is consistent with increasing z-depth where HyU unmixing results stably maintain both lower residuals and variance on average. These reduced residuals correspond to a mathematically more accurate and more uniform decomposition of the signal, as shown by the distribution of residuals with respect to photons (Fig. 14E, Fig. 14F, Fig. 20).

[0134] Taking advantage of the increased sensitivity of HyU, we were able to overcome common challenges of multiplexed imaging, such as low photon yield and spectral crosstalk, and visualize dynamics in developing zebrafish embryos, such as tri-transgenic zebrafish embryos with labeled pan-endothelial cells, vasculature, and clathrin-coated pits (Tg(fli1:mKO2), Tg(kdrl:mCherry), Gt(cltca-citrine)). Multiplexing these spectrally close fluorescent proteins is made possible by the increased sensitivity of HyU at lower photon counts.

[0135] The improved performance at lower SNR allowed us to perform faster acquisitions and maintain high quality results while reducing photon damage through lower excitation laser powers and pixel dwell times (Figure 4). The reduced experimental requirements allow the field of view to be extended to tile larger volumes while still providing sufficient temporal resolution for developmental events, even with a high number of multiplexed fluorescent signals. The timelapse included simultaneous acquisition of clathrin, kdrl, and fli1, allowing visualization of the formation of ventral endothelial protrusions while tracking the development of vesicles and vasculature. HyU allows comparative quantification of spatiotemporal features, in this case the determination of volumetric changes over a long timelapse of 300 min (Figure 4B).

[0136] HyU provides the ability to combine information from intrinsic and extrinsic signals during live imaging of samples, both at a single time point (Figure 5) and at multiple time points (Figure 6). A graphical representation of the phasor allows the identification of an unexpected intrinsic fluorescence signature in tetragenic zebrafish embryos Gt(cltca-citrine), Tg(ubiq:lyn-tdTomato, ubiq:Lifeact-mRuby, fli1:mKO2), imaged with single photons (488 and 561 nm excitation) (Figure 5A-D). The elongated distribution on the phasor (Figure 5C) highlights the presence of an additional unexpected spectral signature, associated with strong sample autofluorescence (Figure 5D blue). HyU analysis of a sample containing this additional signal provides a separation of the contributions of five different fluorescence spectra with a residual error of 3.9% ± 0.3%.

[0137] HyU allows for reduced energy burden, tiled imaging of whole embryos without disrupting embryo development or depleting their fluorescent signal (Figure 5A). Faster, lower power imaging allows for subsequent reimaging of the same sample, as in the zoomed high-resolution acquisition of a head section (Figure 5B, Figure 5E). With the ability to unmix low-photon signals, HyU allows for imaging and decoding of endogenous signals that are inherently low light. Two-photon lasers are ideal for exciting and imaging blue-shifted endogenous fluorescence from samples. Here, the same four transgenic samples are imaged using excitation at approximately 740 nm to access both endogenous and exogenous signals (Figure 5E-G, Figure 27, and Example 23). HyU allows for unmixing of at least nine endogenous and transgenic fluorescent signals (Figure 5) and recovers fluorescence intensity from labels illuminated at suboptimal excitation wavelengths (Figure 5E). The spectrum of intrinsic fluorescence was obtained from in vitro measurements and values ​​reported in the literature. For this sample, the intrinsic signal arises mainly from events related to metabolic activity (NADH and retinoids), tissue structure (elastin), and illumination (laser reflection) (Figure 5E, Figure 28, Figure 32, and Example 23). These results demonstrate that the HyU of the present disclosure is a powerful tool to enable imaging and analysis of endogenous labels.

[0138] Using HyU capabilities, we can multiplex volumetric time-lapse of exogenous and endogenous signals by imaging the tail region of the same four transgenic zebrafish embryos. We can excite the exogenous labels at 488 / 561 nm and the endogenous signal with two photons at 740 nm and collect six tiled volumes over 125 min (Figure 6, Figures 15-17, Figure 21, and Example 23). In this example, HyU unmixing allows the distinction of nine signals and separates their contributions with sufficiently low requirements to allow repeated imaging of endogenous fluorescence, known for its low SNR.

[0139] The advantages of hybrid unmixing (HyU) over conventional linear unmixing (LU) in performing complex multiplexed interrogations are described herein. HyU can overcome the key challenge of separating multiple fluorescent and autofluorescent labels with overlapping spectra while minimally perturbing the sample with excitation light.

[0140] One exemplary advantage of HyU over conventional LU is its multiplexing capabilities when imaging in the presence of biological and instrumental noise, especially at low signal levels. The increased sensitivity of HyU improves multiplexing in photon-limited applications (Figure 2F-L), deeper volumetric acquisitions (Figure 3I, Figure 29), and signal-deficient imaging of autofluorescence (Figure 5E, Figure 6). Simulation results (Figure 2) demonstrate that HyU improves unmixing of simultaneously excited spatially and spectrally overlapping fluorophores. Improved robustness in low-photon imaging conditions reduces imaging requirements for excitation levels and detector integration times, enabling imaging with reduced phototoxicity. Live imaging on multicolor samples performed at high sampling frequencies allows for improved tiling, increasing the field of view while maximizing the use of finite fluorescent signals over time (Figure 3, Figure 4). Two-photon imaging of intrinsic and extrinsic signals suggests the ability of HyU to multiplex signals with large dynamic range differences (Figure 5), extending multiplexed volumetric imaging to the time dimension (Figure 6). Although improved, images are still compromised, especially at low signals (Figure 10), setting the reasonable utilization range beyond 8 photons / spectrum.

[0141] Ease of use and versatility are other key advantages of HyU over conventional LU, inherited from both the phasor approach and classical unmixing algorithms. Here, phasors act as spectral encoders to reduce the computational burden and consolidate similar spectral signatures into histogram bins in the phasor plot. This representation simplifies the identification of independent spectral signatures (Fig. 5 and Example 22) through both phasor plot selection and phasor residual mapping (Fig. 17), and accounts for unexpected intrinsic signals (Fig. 5, Fig. 6, Fig. 18, and Example 23) in a semi-automated manner, while allowing fully automated analysis by spectral libraries.

[0142] The simplicity of this approach is particularly useful in live imaging, where identifying independent spectral components is an open challenge due to the presence of intrinsic signals (Figure 18 and Example 22). High SNR reference spectra can be derived from other experimental data or directly identified on the phasor. Selection of a portion on the phasor plot allows visualization of the corresponding spectrum in the wavelength domain (Figures 5C, 5D, 5F, 5G, and 33). This intuitive versatility allows identification of both the number of unexpected signatures and their spectra, a task that was previously difficult to perform due to noise and the lack of global visualization tools.

[0143] In single-photon imaging (Figure 5A-D), HyU phasor allowed the discrimination of a fifth distinct spectral component arising from the general autofluorescence background, thereby improving the unmixing results. In two-photon imaging, HyU allowed the discrimination and multiplexing of eight highly overlapping signals with a wide dynamic range of intensities between endogenous and exogenous markers (Figure 5F, G). The combination of single-photon and two-photon imaging increased the number of multiplexed fluorophores to nine, taking into account that some of the exogenous labels are excited with two photons (Figure 6). Signal multiplexing can be further improved by implementing HyU on fluorescent dyes.

[0144] HyU performs better than the standard algorithm with and without the phasor noise reduction filter. Compared to conventional LU, the enhanced unmixing when such a filter is applied is demonstrated by a reduction in MSE of up to 21% (Figure 2C), with a 7-fold reduction in the average amount of residual error. Even in the absence of the phasor noise reduction filter, HyU performs up to 7.3% better than the standard (Figure 2D) based on the mean squared error of synthetic data unmixing. This base improvement is attributed to the averaging of similarly shaped spectra in each phasor histogram bin, which reduces the statistical variability within the spectra used for the unmixing calculation (Figure 1E). This averaging strategy works well for typical fluorescence spectra due to their broad and mostly unique spectral shapes.

[0145] In the absence of noise, for example, in ground truth simulations, conventional LU produces a 6x lower MSE than HyU (Figures 11B-11C, 12G). In these noise-free conditions, binning and averaging of the spectrum in the phasor histogram provides statistically independent error values ​​with respect to conventional LU without noise removal, suggesting results of similar quality.

[0146] HyU can be adapted to existing experimental pipelines to interface with various unmixing algorithms. Hybridization with iterative approaches such as non-negative matrix factorization, fully constrained and non-negative least squares was tested. Speed ​​tests using the iterative fitting unmixing algorithm demonstrate up to 500 times speed increase when applying the HyU compression strategy (Figure 19 and Example 24). Due to the initial computation overhead for encoding the spectrum with phasors, there is a 2 times speed reduction with HyU compared to standard LU. However, this can be improved with further optimization of the HyU implementation or by implementing a different type of encoding.

[0147] One limitation of HyU may stem from the mathematics of linear unmixing, where the linear equations representing the unmixed channels need to be solved for the unknown contributions of each fluorophore analyzed.

[0148] To obtain a better solution from these equations and to avoid indeterminate equation systems, the maximum number of spectra for unmixing may not exceed the number of acquired channels, which is typically 32 for commercial microscopes.

[0149] This number can be increased, but due to the broad and photon-poor nature of the fluorescence spectrum, acquisition of more channels may have a detrimental effect on the sample, imaging time, and intensity. Depending on the number of labels in the specimen of interest, expanding the number of labels to simultaneously unmix beyond 32 will likely require a spectral resolution upsampling strategy.

[0150] The HyU improvement is related to the presence of different types of signal disturbances and noise in microscopic images, such as stochastic emission, Gaussian, Poisson, and digital, as well as unidentified spectral signature sources, which affect the SNR in different ways (Figures 11B-11C, 12G, 34). In multiplexing fluorescence signals, HyU offers performance, quality, and speed improvements in the low signal regime. HyU improves over the previously disclosed phasor analysis (Figures 30, 31, and Example 25) and the current gold standard, conventional LU, under multiple experimental conditions (Figures 22-23) with a reduced number of channels (Figures 24-25) at low SNR, in the case of fluorescence signals, as well as in the case of a combination of multiplexed fluorescence and autofluorescence signals (Figure 26). HyU is prepared to be used in the context of in vivo imaging, collecting information from samples labeled at endogenous levels, even in the scattering of mammalian samples (Figures 35-36).

[0151] The results of this example quantitatively demonstrate that HyU, a phasor-based computational unmixing framework, may be well suited to address many of the challenges present in live imaging of multiplexed fluorescent labels. HyU's reduced requirements in the amount of fluorescent signals allow for a reduction in laser excitation burden and imaging time. These features of HyU may enable multiplexed imaging of biological events with longer duration, higher speed, and lower phototoxicity, while providing access to information-rich imaging across different spatiotemporal scales. The reduced requirements of HyU make it fully compatible with any commercial and common microscope capable of spectral detection, easing access to the technology.

[0152] This disclosure provides examples that demonstrate the robustness, simplicity, and improvements of HyU in identifying both new and known spectral signatures, as well as the significant improvements in unmixing output, providing a much-needed tool for delving into many of the issues that still surround research using live imaging.

[0153] Example 3. Zebrafish sample preparation Transgenic zebrafish lines were crossed for multiple generations to obtain embryos with multiple combinations of transgenes. All lines were maintained as heterozygous for each transgene. Embryos were screened for expression patterns of individual fluorescent proteins using a fluorescent stereomicroscope (Axio Zoom, Carl Zeiss) prior to imaging experiments. Tg(ubiq:Lifeact-mRuby) lines were separated from Tg(ubiq:lyn-tdTomato) lines by distinguishing spatially and spectrally overlapping signals using a confocal microscope (LSM780, Carl Zeiss).

[0154] For in vivo imaging, 5–6 zebrafish embryos were fixed at 18–72 hpf and placed in 1% UltraPure low melting agarose (catalog no. 16520-050, Invitrogen) solution and prepared in 30% Danieau (17.4 mM NaCl, 210 M KCl, 120 M MgSO4 7H2O, 180 M Ca(NO3)2, 1.5 mM HEPES buffer, pH 7.6) with 0.003% PTU and 0.01% tricaine in imaging dishes (catalog no. D5040P, WillCo Wells) with a number 1.5 coverslip bottom. After solidification of the agarose at room temperature (1–2 min), the imaging dishes were filled with 30% Danieau solution and 0.01% tricaine at 28.5 °C.

[0155] Example 4. Characterization of fluorescent silica beads One fluorescent silica bead solution (Nanocs, Inc.) labeled with Cy3 (Si500-S3-1, 0.5 mL, 1% solids, Lot No. 1608BRX5) was characterized for its spectral fluorescence emission and physical size.

[0156] A 10-fold dilution of the beads in PBS was placed on a number 1.5 imaging cover glass and spectrally characterized using the spectral mode on a Zeiss LSM780 laser confocal scanning microscope equipped with a 32-channel detector using a 40x / 1.1W LD C-Apochromat Korr UV-VIS-IR lens to excite fluorescence from the beads using a 740 nm two-photon laser with a 690 nm low-pass filter to separate excitation and emission. Spectra obtained from multiple beads with the same label were averaged to generate the reference spectrum (dashed line) reported in Figure 36G. The size and concentration of the fluorescent silica beads were determined by nanoparticle tracking analysis (NTA) on a Nanosight NS300 (Malvern Panalytical). Samples were run five times and the results were averaged for the final size and concentration values ​​reported.

[0157] Example 5. Mouse sample preparation For autofluorescence measurements, mouse organ samples were collected from Balb-c mice. After euthanasia, organs were excised, washed with phosphate buffered saline (PBS) to remove residual blood, and kept in PBS until imaging preparation. Organs were sectioned for imaging internal architecture and mounted on glass imaging dishes with sufficient PBS to avoid dehydration of the samples. After imaging, all samples were fixed in 10% neutral buffered formalin solution at 4°C.

[0158] Mouse organ samples were collected from Balb-c mice for ex vivo bead characterization in tissues. After euthanasia, organs were excised and washed in PBS, followed by incubation in 10% buffered formalin for at least 24 hours. Kidneys were then removed from fixative and sectioned into smaller approximately 5×5×5 mm pieces for imaging. A working solution of fluorescent silica beads (Nanocs, Inc.) labeled with Cy3 (Si500-S3-1, 0.5 mL, 1% solids, Lot No. 1608BRX5) and previously characterized was prepared using a 10-fold dilution of the fluorescent beads from a stock concentration of the fluorescent beads. Beads were injected into the sample using 50 ul of the solution loaded into a 0.5 mL syringe with a 28 g needle. The kidney sections were then placed into an imaging dish containing a small amount of PBS to hydrate the sample before imaging.

[0159] Example 6. Image Acquisition Images were acquired with a Zeiss LSM780 laser confocal scanning microscope equipped with a 32-channel detector using a 40x / 1.1W LD C-Apochromat Korr UV-VIS-IR lens at 28°C.

[0160] Gt(cltca-citrine), Tg(ubiq:lyn-tdTomato), Tg(fli1::mKO2), and Tg(ubiq:Lifeact-mRuby) samples were imaged simultaneously with 488 nm and 561 nm laser excitation for citrine, tdTomato, mKO2, and mRuby. A narrow 488 nm / 561 nm dichroic mirror was used to separate excitation and fluorescence emission. Samples were imaged with a 740 nm two-photon laser to excite autofluorescence, and a 690 nm low-pass filter was used to separate excitation and emission.

[0161] Samples of mouse kidney tissue were imaged with two-photon excitation at 740 nm or 850 nm using a 690+ nm low pass filter during 37° C. incubation.

[0162] For all samples, detection was performed over the full available range (410.5-694.9 nm) with spectral binning of 8.9 nm. Example 7. Detailed description of imaging parameters used for all images in this disclosure. [Table 1] [Table 2] [Table 3] [Table 4]

[0163] (Example 8. Hyperspectral fluorescence image simulation) This model simulates spectral fluorescence emission by generating a probability distribution of photons with a profile equivalent to a pure reference spectrum (as described in Example 22). The effect of photon starvation commonly observed in microscopy is synthetically obtained by manually reducing the number of photons in this probability distribution. Detection, Poisson, and signal propagation noise are then added to generate a 32-channel fluorescence emission spectrum that closely resembles that obtained in a microscope. The simulation includes accurate integration of the dichroic mirror and imaging setup.

[0164] Simulations that match the experiment. To quantify the performance of HyU vs. LU on experimentally acquired microscopy data, synthetic data were generated where each input spectrum was organized with an intensity distribution obtained from the experimental data. An analogue to the photon count rate was calibrated based on existing literature. The real data was discretized into photons to generate a realistic photon mask with a biologically relevant signal distribution. This provided intensities and ratios that matched those acquired from the microscope while allowing the effects of photon starvation to be controlled.

[0165] Overlapping Simulations. Simulations were included to quantify the performance of HyU versus conventional LU with respect to the number of spectral combinations. These simulations were created with artificial intensity distributions such that a simulation with X% overlap and n fluorophores has a certain percentage X of pixels with a randomized ratio of the n input spectra. As an example, for a simulation with 6 fluorophores and 50% overlap, the simulated data set has 50% of the pixels containing a randomized combination of the 6 fluorophores, while the remaining pixels contain a single fluorophore. This allowed us to investigate the impact of increasing the number of spectral combinations on the compressive properties of the phasor method for HyU.

[0166] Example 9. Image Analysis: Independent Spectral Signatures Independent spectral fingerprints can be obtained from samples, solutions, literature, or through a spectrum viewer website (Thermo Fisher, BD Spectrum Viewer, Spectrum Analyzer). The fluorescence signals used herein were obtained by imaging a single labeled sample in an area known morphologically and physiologically to express a specific fluorescence. See FIG. 27. For each data set, a phasor plot was calculated. 32-channel spectral fingerprints were extracted from the phasor bins at the count-weighted average positions of the phasor clusters. These fingerprints were compared to literature fingerprints and manually corrected to reduce noise. Further description of the method for identifying new components can be found in Example 22 and FIG. 17, FIG. 23.

[0167] Elastin spectra were obtained experimentally and compared to literature for autofluorescence signal. Spectra for nicotinamide adenine dinucleotide (NADH) free, NADH bound, retinoic acid, retinol, and flavin adenine dinucleotide (FAD) were acquired from in vitro solutions using a microscope. NADH free of B-nicotinamide adenine dinucleotide (Sigma-Aldrich, St. Louis, MO, no. 43420) in phosphate buffered saline (PBS) solution. NADH bound from B-nicotinamide adenine dinucleotide and L-lactate dehydrogenase (Sigma-Aldrich, no. 43420, no. L3916) in PBS. Retinoic acid from a solution of retinoic acid (Sigma-Aldrich, no. R2625) in dimethyl sulfoxide (DMSO). Retinol from a solution of retinol synthesis (Sigma-Aldrich, no. R7632) in DMSO. FAD from flavin adenine dinucleotide disodium salt hydrate (Sigma-Aldrich, no. F6625) in PBS.

[0168] (Example 10. Phasor Analysis) For each pixel in the data set, the Fourier coefficients of its normalized spectrum define coordinates in the phasor plane (G(n), S(n)), where:

number

[0169] In the formula, λ s and λ f are the start and end wavelengths, respectively, I is the measured intensity, c is the number of spectral channels (32 in this case), and n is the harmonic order. The first harmonic (n=1) is utilized for the autofluorescence signal based on the sparsity of the independent spectral components, and the second harmonic (n=2) is utilized for the fluorescence signal. A two-dimensional histogram with dimensions (S, G) is applied to the phasor coordinates to group pixels with similar spectra within a single square bin. This process can be defined as phasor encoding.

[0170] (Example 11. Linear Unmixing) The hypothesis for linear unmixing in this work is that given i independent spectral fingerprints (fp), each collected spectrum (I(λ)) is a linear combination of the fp, with each fp contribution (R) summing to unity.

number

[0171] In the formula, R i is the ratio, W i are the weights and N is the noise. The acquired spectra are collected into an original spectral cube with shape (t,z,c,y,x) with t as time, c as channel, and x,y,z as spatial dimensions.

[0172] i spectral vectors fp imust be provided to the unmixing function. We assume that all fp have equal weights and that the noise N has low values. Under these conditions, R i is obtained by applying Jacobian Matrix Inversion.

number

[0173] In the pixel-wise linear unmixing implementation in this work, the inverse Jacobian matrix is ​​applied to the acquired spectrum at each pixel with dimensions (t, z, c, y, x). The resulting ratios of each spectral vector are assembled in the form of a ratio cube with shape (t, z, i, y, x), where x, y, z, t are the spatial and temporal dimensions of the original image, respectively, and i is the number of input spectral vectors. The ratio cube (t, z, i, y, x) is multiplied with the integral of the intensity over the channel dimensions of the original spectral cube with shape (t, z, y, x) to obtain the final resulting dataset with shape (t, z, i, y, x).

[0174] (Example 12. Hybrid Unmixing - Linear Unmixing) In a hybrid unmixing implementation, the inverse Jacobian matrix is ​​applied to the average spectrum of each phasor bin with dimensions (c, s, g), where g and s are the phasor histogram sizes and c is the number of spectral channels acquired. The average spectrum in each bin is calculated by using the phasor as an encoding to refer each original pixel spectrum to a bin. The resulting ratios of each component channel are assembled in the form of a phasor bin ratio cube with shape (i, s, g), where i is the number of input independent spectra fp (linear unmixing sections). This phasor bin ratio cube is then referenced to the original image shape, forming a ratio cube with shape (t, z, i, y, x), where x, y, z, t are the original image dimensions. The ratio cube is multiplied with the integral of the intensity over the channel dimensions of the original spectral cube with shape (t, z, y, x) to obtain a final result dataset with shape (t, z, i, x, y).

[0175] (Example 13. Unmixing algorithm used for speed comparison) The unmixing algorithms used for the speed comparison with the HyU algorithm (Figure 19) were plugged into the unmixing step of the analysis pipeline and sourced as follows: Non-negative constrained least squares and fully constrained least squares from pysptools.abundance_maps (https: / / pysptools.sourceforge.io / abundance_maps.html). Robust non-negative matrix factorization. 10 The python implementation is obtained from (https: / / github.com / neel-dey / robust-nmf).

[0176] Example 14. Data visualization Rendering of the final result dataset was performed using Imaris 9.5-9.7. In Figures 2-3, contrast settings (min, max, gamma) for each channel were set to be equal to provide a reasonable comparison between HyU and LU results. Gamma was set to 1, no minimum threshold was applied, and the max value for each channel was set to 1 / 3 of the maximum intensity. Images were rendered using Maximum Intensity Projection (MIP) and, for improved viewing, digitally resampled in the z direction while maintaining a fixed xy ratio to attenuate gaps created from sparsely sampling z direction on the microscope.

[0177] Example 15. Box plot generation All box plots were generated using standard plotting methods. The center line corresponds to the median, the lower box boundary corresponds to the first quartile, and the upper box boundary corresponds to the third quartile. The lower and upper whiskers correspond to 1.5 times the lower and upper interquartile ranges of the first and third quartiles, respectively.

[0178] Example 16. Time-lapse registration A customized python script (Supplementary Code) was first utilized to pad the number of z-slices across multiple time points to obtain volumes of equal size. Data were registered using the "Correct 3D drift" plug (https: / / imagej.net / Correct_3D_Drift) in FIJI (https: / / imagej.net / Fiji).

[0179] Example 17. Time-lapse statistics Box plots and line plots for the time lapse were generated using ImarisVantage in Imaris 9.5-9.7. Box plot elements follow the same guidelines as described above. Line plots are connected box plots for each time point, with the solid line representing the median and the shaded areas representing the first and third quartiles.

[0180] Example 18. Mean Square Error For synthetic data, ground truth is available for comparison of unmixing fidelity between HyU and LU. Due to the arbitrary nature of intensity values ​​in microscopy data, fp contributions, or ratios, were used for quantification. Mean squared error (MSE) is used to judge the quality of the ratios in synthetic data. MSE can be defined as the squared difference between the ratios recovered by the unmixing algorithm (r unmix) and the ground truth ratio (r) divided by the total number of pixels (n).

number

[0181] To simplify the comparison between different unmixing algorithms, the relative mean squared error (RMSE) can be defined as:

number

[0182] The RMSE is a measure of the improvement in MSE when using HyU compared to traditional LU.

[0183] Example 19. Residuals For experimental data, in the absence of ground truth, the performance of the results returned by the unmixing algorithm is quantified with the following measures: the average relative residual, the residual image map, the residual phasor map, and finally, the residual intensity histogram.

[0184] The residual (R) is calculated as follows: About Pictures:

number

number

[0185] The spectral intensity difference between the unmixed image and the original image for each pixel or phasor bin depends on the following description of the intensity image (I), where:

number

[0186] The original spectrum (I raw image) is the combination of each independent spectral component (fp) with its ratio (r) plus noise (N). The recovered spectrum is obtained by multiplication of each corresponding individual component with its recovered ratio (r unmixed).

[0187] The relative residual (RR) is calculated as the sum of the residual values ​​across the C channels, normalized to the sum of the original intensity values ​​across the C channels (eg, C=32).

number

[0188] The average relative residual (Fig. 11) provides a single comparison value for evaluating the performance of different processing methods on the same data, such as applying multiple filters, applying various thresholds, and varying the number of components estimated. The average relative residual (RRmean) is defined as the average of the relative residuals for every pixel in the image or every phasor bin in the phasor histogram. About Pictures:

number

number

[0189] Residual image maps visualize the residual value for each pixel of an image (Figure 10). Regions with higher residual values ​​appear to characterize parts of the dataset where an increased amount of noise or unexpected spectral signatures are present.

[0190] The residual image map (Rimagemap(x,y)) projects the relative residual (RR) cube onto a 2D image geometry for each voxel, providing an estimated visualization of the algorithm ratio recovery performance in the spatial context of the original image.

number

[0191] Residual phasor maps visualize the residuals for each bin of the phasor histogram (Figure 10). These maps allow insight into where HyU unmixing results have degraded performance in the phasor domain and show the phasor locations of unexpected additional spectral components (Figure 11).

number

[0192] Residual Intensity Histogram R Intensity Histogram(p,rr) (Fig. 3g, Fig. 3h, and Fig. 10d) calculates the distribution of relative residuals related to the intensity across all pixels or all phasor bins. Higher residuals appear to be present in areas with lower signal strength and SNR, providing degraded performance. For images:

number

number

[0193] where p is the bin of histogram P, rr is the bin of RR, and sf is the factor that converts the number of photons to a digital intensity level.

[0194] Example 20. Image contrast Image contrast is a measure of the distinguishability of detail against the background. Percent contrast can refer to the relationship between the highest and lowest intensities in an image.

number

[0195] Here, the average signal intensity (I s ) is the average of the top 20% intensities in the image. The background average intensity (I B ) is the average of the bottom 20% image intensities.

[0196] Example 21. Spectral signal-to-noise ratio Since each synthetic data set has a ground truth, the SNR can be calculated by comparing the simulated images to the ground truth. Since these are hyperspectral images, the definition of SNR can be extended to the wavelength dimension of the data and the term spectral SNR can be used. Two types of spectral SNR can include absolute spectral SNR and relative spectral SNR.

[0197] The spectral SNR can be calculated for each single spectrum simulation as follows: First, take the absolute value of the difference between the ground truth intensity and the simulated intensity for each pixel and channel. Then calculate the average value over all pixels in each channel. Finally, take the sum over all of the channels and divide by 32 for absolute SNR or by the number of channels with signal for relative SNR. The number of channels with signal is calculated by checking if there is a statistically significant number of pixels in a single channel with pixel SNR values ​​greater than zero.

number

[0198] In the formula, i gnd is the intensity per pixel per channel for the ground truth data, and i sim is the intensity per pixel per channel for the simulated (noisy) data, P is the total number of pixels, and C is the number of channels with signal.

[0199] Example 22. Spectra and new component identification using HyU Identification of independent spectral components has been an adversity to unmixing hyperspectral data. First, collected spectra may be distorted by reduced SNR. Second, excitation of endogenous signals causes uncertainty in biological samples. Advantageously, HyU simplifies this process by adapting the phasor approach and achieving a semi-automated or fully automated process for spectrum identification and selection. In HyU, spectra can be loaded from existing libraries, substantially automating the analysis process. Pre-identified cursors are generated from common fluorophores such as mKO2, tdTomato, mRuby, and Citrine. Obtaining fluorescence spectra from experimental samples has several advantages compared to utilizing spectra from existing libraries, as they account for a large number of experimental and instrumental settings. Imaging settings such as different types of lenses or optical filters (Figures 10C-10D), along with factors in the sample's microenvironment such as pH or temperature, have the potential to modify the fluorescence spectral emission. In the presence of unexpected fluorescence signals, spectra can also be selected and visualized directly from the phasor. Phasors facilitate the identification of unexpected independent components and their differentiation from multiple system noises. A noise-free spectrum appears as a single point on the phasor plot, while spectra affected by instrumental and electronic noise appear primarily as Gaussian distributions centered on the original spectral signal. Conversely, random noise across multiple spectral channels will not produce a clustered aggregation of spectra on the phasor. A constant spectral noise with a different spectrum (e.g., a constant optical leakage into the system) would produce a different phasor cluster and could be selected for unmixing. The phasor plot representation is a 2D histogram and provides insight into the frequency of occurrence of these signals. These unexpected independent components in the sample often appear as "tails" on the phasor distribution (Figure 17C).In the exemplary HyU graphical interface, clicking on a phasor visualizes the spectrum in a small region (9x9 bins by default, size adjustable from the interface) of the phasor histogram (Figure 1D). The example shown and / or discussed in Figures 15-17 identifies five different endmembers on the phasor (Figure 16C) and visualizes their spectra identifying Citrine, mRuby, Td-Tomato, mKO2, and one strong autofluorescence signature. The use of the residual phasor map (Figure 17B) allows the identification of regions in the phasor with large amounts of residuals that likely correspond to missing endmembers in the unmixing. The residual image map (Figure 17C) provides a quick overview of the residuals in the image data for identification of the location in the dataset of the missing endmembers.

[0200] Example 23. HyU in autofluorescence data Cellular metabolism is a key regulator of cell function and plays an essential role in the development of many diseases. Understanding of cellular metabolic pathways is critical for the development and evaluation of novel therapeutic and diagnostic methods. Several metabolites have been reported in the literature to be fluorescent and change their spectra according to their biochemical makeup. For example, measurement of NADH in its free and bound states is possible thanks to the shift in emission spectrum when NADH is bound to enzymes such as Lactate Dehydrogenase (LDH). Similarly, retinol and retinoic acid are known to have different autofluorescence spectra. A map of phasor positions for typical autofluorescence from pure solutions is reported in Figure 27B. Imaging autofluorescence data for cellular metabolism requires consideration of complex and dynamic changes in metabolic pathways that can occur over a wide range of times, from seconds to years. These autofluorescence signals are often weak in nature and do not replenish quickly after photobleaching. Laser power can be reduced to avoid rapid autofluorescence spectral signal bleaching and to reduce photodamage. Additional factors known to affect emission spectra include pH and temperature, the per-pixel concentration of fluorophores, excitation power, developmental stage and region of the sample imaged. An example of the latter is reported in Figure 18, where the signal in the sample presents strong local differences. An example of the effect of different two-photon excitation powers and different levels of per-pixel concentration is reported in Figures 5-6. In these images, samples of similar developmental stages are imaged with different pixel sizes (lateral resolutions of 0.259 μm and 0.923 μm) and a resolution of approximately 4.7·10 -6 mW / mm 2 The images were taken using laser powers (4% and 3% at 740 nm two-photon) resulting in a laser power density of 1.4 10. This different laser power does not excite some lower concentration endogenous fluorophores, in this case mRuby, which is visible in Figure 5 but not in Figure 6. In both of these images, FAD is not excited in any measurable amount, whereas in Figure 6, FAD is excited in any measurable amount, whereas in Figure 7, FAD is excited in any measurable amount, whereas in Figure 8, FAD is excited in any measurable amount. -3 mW / mm 2In Fig. 28, the FAD contribution is measurable and unmixed. HyU is well suited for the analysis of intrinsically low autofluorescence due to its ability to operate at low SNR. Fig. 18 shows the unmixing of multiple autofluorescence signals based on spectra acquired from in vitro solutions. Fig. 23, Fig. 25 present a simulated overview of the improvement of HyU over linear unmixing for autofluorescence data as a function of the number of labels, the percentage of pixels containing a mixture of fluorophores, the number of applied denoising filters, and the number of channels under different levels of signal-to-noise ratio.

[0201] (Example 24. Reducing computational costs during unmixing) The advantage of HyU is speed. HyU provides significant speedups compared to other pixel-based unmixing algorithms. The exception is in the case of traditional LU vs. HyU due to the highly optimized computational implementation of the functions utilized in traditional LU. This speedup occurs because the unmixing is performed at the phasor histogram level, where a single bin corresponds to many image pixels. For algorithms other than standard LU, HyU provides up to 500x speed improvement with comparable coding language and computing hardware, processing 2GB in under 100 seconds (Figure 19).

[0202] This improvement provides a solution to two outstanding image analysis challenges in fluorescence multiplexing: first, the increase in size of HFI data due to continuously higher throughput and resolution microscopes, which scales with the number of spectral channels, and second, the number of data sets due to experimental reproducibility and biological variability.

[0203] Example 25. Improvements of HyU over standard phasor analysis Linearity of the combination is a common assumption of most of the spectral analysis algorithms in hyperspectral fluorescence imaging (HFI). Each pixel is assumed to contain a linear combination of independent spectral signatures, or endmembers, contained in the sample. This assumption requires knowledge, or identification, of the independent spectra in the sample. In standard linear unmixing algorithms, the extraction of relative spectral amounts (ratios) is performed pixel by pixel, at the expense of computational cost. For lower signal-to-noise ratio (SNR) spectra, the disturbed experimental signal complicates the detection of the spectral endmembers and reduces the accuracy of the ratio determination. However, these standard unmixing algorithms have the advantage of being unsupervised, with the possibility to automate the analysis process.

[0204] The phasor approach has become a popular dimensionality reduction approach for both fluorescence lifetime and spectral imaging analysis. Phasors are used for preprocessing HFI datasets. 3For both unmixing and unmixing, phasor analysis offers important advantages including spectral compression, noise removal, and computational reduction. Phasor analysis overcomes the challenges of low SNR data analysis that limit standard unmixing algorithms and provides a multiplexed solution to the need. The phasor transform is, in principle, a lossy encoder that carries a reduction in the percentage of information compared to the original clean data. In imaging of fluorescent signals, where the signal-to-noise ratio is often reduced to lower orders of magnitude, the encoding loss is less relevant compared to the noise in the fluorescent signal. This fundamental advantage of increasing the SNR in noisy data makes the phasor method a valuable tool for fluorescence microscopy, both lifetime and spectral fluorescence microscopy. This point has been reported by several groups using phasors and has recently been nicely illustrated in the work of Scipioni et al. Standard phasor analysis is fully supervised and requires the manual selection of regions or points in a graphical representation of the transformed spectrum, called a phasor plot. Each selection of a region in the phasor plot associates pixels containing similar spectra to the same fluorophore, forming an output channel containing the wavelength integral of the intensity with a single ratio value. This "winner takes all" approach is suitable when the fluorophores for each single excitation light are spectrally overlapping and spatially dispersed (Figure 30), but requires separate acquisition of different excitation wavelengths to demultiplex the spatially and spectrally overlapping fluorophores (Figure 31).

[0205] HyU uses the phasor transform to group pixels with similar spectral shapes within each phasor histogram bin. This approach maintains the advantages of compression, denoising and ease of identification of clean end-member fluorescence spectra. However, HyU improves the robustness of the analysis. The denoised signals are kept in the hybrid phasor and wavelength domains and can therefore be unmixed with a number of standard unmixing algorithms (Figure 19), such as linear unmixing or fully constrained least squares. These standard unmixing approaches operate unsupervised and provide each pixel with a ratio for a set of spectral signals, which can overcome some of the limitations of phasors, but generally do not perform well in experimental conditions with reduced and impaired signals such as fluorescence, and require enormous computational time for high spectral count data sets. HyU provides wavelength-based denoised spectra that allow these standard algorithms to outperform their typical pixel-by-pixel applications in quality of results (Figures 22-25) due to cleaner and better defined fluorescence spectra in each phasor bin, and in general, in speed, due to phasor dimensionality reduction. HyU works well for single excitation lights when fluorophores are spectrally overlapping, both when spatially dispersed or colocalized, providing a ratio of each currently unmixed independent spectrum. HyU can have reasonable performance for up to eight different fluorophores per data set, for each single excitation wavelength. In experiments with a carefully selected palette of labels, where an octuple of fluorophores can be excited by a single wavelength, in five sequential acquisitions (one for each excitation light), with an instrument capable of spectral acquisition at five standard and well-spectrally separated excitation wavelengths, HyU could, in principle, unmix 40 signals. However, this performance decreases with the number of channels (Figures 24-25), showing a slight degradation with eight channels and a limitation with four channels.

[0206] Unless otherwise specified, all measurements, ratings, locations, dimensions, sizes, and other specifications set forth in this specification, including those in the following claims, are approximate and not precise and are intended to have a reasonable range consistent with the function to which they pertain and the customary practice in the art to which they pertain.

[0207] As will be understood by those skilled in the art, for any or all purposes, including, in particular, in terms of providing a written description, all ranges disclosed herein also encompass any and all possible subranges and combinations of those subranges. Any recited range can be readily recognized as fully descriptive and allowing for the same range to be broken down into at least equal halves, thirds, quarters, fifths, tenths, etc. As a non-limiting example, each range discussed herein can be readily broken down into a lower third, middle third, and upper third, etc. As will also be understood by those skilled in the art, all words such as "up to," "at least," "greater than," "less than" and the like refer to a range that includes the recited number and can then be broken down into subranges as discussed herein. Finally, as will be understood by those skilled in the art, a range includes each individual member. Thus, for example, a group having 1-3 items refers to a group having 1, 2, or 3 items. Similarly, a group having 1-5 items refers to a group having 1, 2, 3, 4, or 5 items, etc.

[0208] While various aspects and embodiments are disclosed herein, other aspects and embodiments will be apparent to those of skill in the art. The various aspects and embodiments disclosed herein are for illustrative purposes and are not intended to be limiting.

[0209] All references cited herein, including, but not limited to, published and unpublished applications, patents, and literature references, are incorporated herein in their entirety by reference for the subject matter referenced, and made a part of this specification. To the extent that the publications and patents or patent applications incorporated by reference conflict with the disclosure contained herein, the present specification is intended to supersede and / or supersede such conflicting material.

[0210] In this disclosure, the indefinite article "a" and the phrases "one or more" and "at least one" are synonymous and mean "at least one."

[0211] Relative terms such as "first" and "second" may be used only to distinguish one entity or action from another, without necessarily requiring or implying any actual relationship or order between them. "Comprises," "comprising," and any other variations thereof, when used in connection with a list of elements within the specification or claims, are intended to indicate that the list is not exclusive and that other elements may be included. Similarly, an element preceded by "a" or "an" does not, without further constraints, exclude the presence of additional elements of the same type.

[0212] With respect to the use of virtually any plural and / or singular term herein, those of skill in the art can interpret the plural to the singular and / or the singular to the plural as appropriate to the context and / or application. The various singular / plural permutations may be expressly set forth herein for clarity.

[0213] The phrase "means for," when used in the claims, is intended and should be interpreted to encompass the corresponding structures and materials described, and their equivalents. Similarly, the phrase "step for," when used in the claims, is intended and should be interpreted to encompass the corresponding acts described, and their equivalents. The absence of these phrases from a claim means that the claim is not intended to, and should not be limited to, these corresponding structures, materials, or acts, or their equivalents.

[0214] In at least some of the embodiments described above, one or more elements used in an embodiment may be used interchangeably in another embodiment, unless such substitution is technically feasible. Those skilled in the art will appreciate that various other omissions, additions, and modifications may be made to the methods and structures described herein without departing from the scope of the claimed subject matter. All such modifications and variations are intended to be within the scope of the disclosed subject matter.

[0215] The scope of protection is limited only by the scope of the claims which follow, which scope is intended to be broad in accordance with the ordinary meaning of the terms used in the claims when interpreted in light of this specification and the ensuing prosecution history, unless a specific meaning is stated, and should be interpreted to encompass all structural and functional equivalents.

[0216] Those skilled in the art will generally understand that the terms used herein, and particularly in the appended claims (e.g., the body of the appended claims), are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including, but not limited to," the term "having" should be interpreted as "having at least," the term "includes" should be interpreted as "including, but not limited to," etc.). It will be further understood by those skilled in the art that if a particular number of introduced claim recitations are intended, such intention will be expressly recited in the claim, and that in the absence of such recitation, no such intention is present. For example, to aid in understanding, the following appended claims can be introduced with the introductory phrases "at least one" and "one or more." However, the use of such phrases should not be construed as implying that the introduction of a claim statement by the indefinite article "a" or "an" limits any particular claim that includes such an introduced claim statement to an embodiment that includes only one such statement, and the same claim still remains true in the case of a definite article used to introduce a claim statement, even if the same claim includes the introductory phrase "one or more" or "at least one" and the indefinite article "a" or "an" (e.g., "a" and / or "an" should be interpreted to mean "at least one" or "one or more"). In addition, even if a specific number of introduced claim statements is explicitly recited, one of ordinary skill in the art will recognize that such recitation should be interpreted to mean at least the recited number (e.g., the bare recitation of "two statements" without other modifiers means at least two statements, or more than two statements).Furthermore, in instances where a convention similar to "at least one of A, B, and C, etc." is used, such structure is generally intended in the sense that one of ordinary skill in the art would understand the convention (e.g., "a system having at least one of A, B, and C" includes, but is not limited to, systems having only A, only B, only C, both A and B, both A and C, both B and C, and / or both A, B, and C, etc.). In instances where a convention similar to "at least one of A, B, or C, etc." is used, such structure is generally intended in the sense that one of ordinary skill in the art would understand the convention (e.g., "a system having at least one of A, B, or C" includes, but is not limited to, systems having only A, only B, only C, both A and B, both A and C, both B and C, and / or both A, B, and C, etc.). It will be further understood by those skilled in the art that disjunctions and / or phrases indicating two or more alternative terms, whether in the specification, claims, or drawings, should be understood to contemplate the possibility of including one of the terms, either of the terms, or both of the terms. For example, the phrase "A or B" will be understood to include the possibilities of "A" or "B" or "A and B."

[0217] In addition, where features or aspects of the disclosure are described in terms of a Markush group, those skilled in the art will recognize that the disclosure is also thereby described in terms of any individual members or subgroups of members of the Markush group.

[0218] The claims are not intended, and should not be construed, to encompass subject matter that does not comply with the requirements of 35 U.S.C. §§ 101, 102, or 103. Any unintended coverage of such subject matter is hereby excluded. Except as merely set forth in this paragraph, nothing described or illustrated is intended, or should be construed, to result in the dedication to the public of any element, step, feature, object, benefit, advantage, or equivalent, whether or not recited in the claims.

[0219] The Abstract is provided to aid the reader in quickly ascertaining the nature of the technical disclosure. The Abstract is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, various features of the foregoing detailed description have been grouped together in various embodiments to streamline the disclosure. This method of disclosure should not be interpreted as requiring the claimed embodiments to have more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. As such, the following claims are incorporated herein, with each claim standing on its own as separate claim subject matter.

Claims

1. A hyperspectral imaging system comprising an image formation system for generating a representative image of an object using a hybrid unmixing technique, said hybrid unmixing technique hereinafter referred to as HyU; The image forming system includes: acquiring detected radiation of the object, wherein the detected radiation includes at least two object waves, each object wave having a detected intensity and a different detected wavelength; forming an object image using the detected object radiation, wherein the object image comprises at least two image pixels, each image pixel corresponding to a physical point on the object; forming at least one intensity spectrum for each image pixel using the detected intensity and the detected wavelength of each target wave; converting the intensity spectrum of each image pixel into a complex-valued function using a Fourier transform based on the intensity spectrum of each image pixel, wherein each complex-valued function has at least one real component and at least one imaginary component; forming one phasor point on the phasor plane for each image pixel by plotting the value of said real component against the value of said imaginary component, wherein said real component value is hereinafter referred to as the real value and said imaginary component value is hereinafter referred to as the imaginary value; forming a phasor histogram including at least two phasor bins, where each phasor bin includes at least one phasor point; aggregating the detected spectra belonging to the image pixels for each phasor bin; generating a representative intensity spectrum for each phasor bin; unmixing the representative intensity spectrum of the phasor bins by using an unmixing technique, thereby determining the abundance of each spectral end-member of the detected radiation; determining the abundance of each spectral endmember in the representative intensity spectrum and the detected intensities belonging to the image pixels; A hyperspectral imaging system that generates a representative intensity image of the object, representing the abundance of each spectral endmember.

2. the hyperspectral imaging system further comprising an optical system; the optical system comprises at least one optical component; the at least one optical component comprises at least one optical detector; the at least one optical detector: detecting one or more of electromagnetic radiation absorbed, transmitted, refracted, reflected, and emitted from at least one physical point on the object, thereby forming the object radiation, the object radiation including at least two object waves, each object wave having an intensity and a different wavelength; Detect the intensity and wavelength of each target wave; 10. The hyperspectral imaging system of claim 1, further comprising a configuration for transmitting the detected target radiation and the detected intensity and detected wavelength of each target wave to the imaging system to be acquired.

3. 10. The hyperspectral imaging system of claim 1, wherein the imaging system further comprises a control system, a hardware processor, memory, and a display, the imaging system further configured to display the representative image of the object on a display of the imaging system.

4. The hyperspectral imaging system of claim 1 , wherein the unmixing technique is a linear unmixing technique.

5. 2. The hyperspectral imaging system of claim 1, wherein the unmixing technique is a fully constrained least squares unmixing technique, a matrix inversion unmixing technique, a non-negative matrix decomposition unmixing technique, a geometric unmixing technique, a Bayesian unmixing technique, a sparse unmixing technique, or any combination thereof.

6. 10. The hyperspectral imaging system of claim 1, wherein the imaging system further comprises: applying a noise reduction filter to reduce Poisson noise and / or instrumental noise in the detected radiation.

7. 2. The hyperspectral imaging system of claim 1, wherein the imaging system is further configured to apply a denoising filter at least once to the real and / or imaginary components of each complex-valued function to produce a denoised real value and a denoised imaginary value for each image pixel.

8. the image forming system, applying a denoising filter at least once to the real and / or imaginary components of each complex-valued function to generate a denoised real value and a denoised imaginary value for each image pixel, wherein the denoising filter is applied after the imaging system converts the formed intensity spectrum belonging to each image pixel into the complex-valued function using the Fourier transform and / or before the imaging system forms one phasor point on the phasor plane for each image pixel; 2. The hyperspectral imaging system of claim 1, further comprising: forming one phasor point on the phasor plane for each image pixel using the denoised real value as the real value for each image pixel and the denoised imaginary value for each image pixel as the imaginary value.

9. 2. The hyperspectral imaging system of claim 1, further configured to apply a denoising filter to the value of the real component and / or the value of the imaginary component after the imaging system forms one phasor point on the phasor plane for each image pixel.

10. 2. The hyperspectral imaging system of claim 1, wherein the imaging system has a further configuration for aggregating the detected spectra belonging to the image pixels of each phasor bin, and the detected spectra belonging to the same phasor bin have essentially similar or substantially the same spectral shape.

11. the imaging system further comprising: aggregating the detected spectra belonging to the image pixels of each phasor bin; each detected spectrum belonging to the image pixels of the same bin has at least two detected intensities and a detected wavelength for each detected intensity; 2. The hyperspectral imaging system of claim 1, wherein the relative detected intensity value of each spectrum belonging to the same spectral bin is substantially the same as the relative detected intensity value of other spectra aggregated in the same bin.

12. the image forming system, forming at least two phasor bins by discretizing the phasor plot along its real dimension and its imaginary dimension, where each phasor bin has a phasor bin area on each phasor plot; The hyperspectral imaging system of claim 1 , further comprising: aggregating the detected spectra belonging to the image pixels of each phasor bin.

13. the image forming system, forming at least four phasor bins by discretizing the phasor plot along its real dimension and its imaginary dimension, wherein each phasor bin has a phasor bin area on each phasor plot, the phasor bin area being 4 / (total number of phasor bins), the total number of phasor bins being the product of the number of discretizations along the real dimension of the phasor plot and the number of discretizations along the imaginary dimension of the phasor plot; The hyperspectral imaging system of claim 1 , further comprising: aggregating the detected spectra belonging to the image pixels of each phasor bin.

14. The hyperspectral imaging system of claim 1 , wherein the imaging system uses at least one harmonic of the Fourier transform to generate the representative image of the object.

15. The hyperspectral imaging system of claim 1 , wherein the imaging system uses at least the first and / or second harmonic of the Fourier transform to generate the representative image of the object.

16. 10. The hyperspectral imaging system of claim 1, wherein the imaging system uses only the first harmonic or only the second harmonic of the Fourier transform to generate the representative image of the object.

17. 10. The hyperspectral imaging system of claim 1, wherein the imaging system uses only the first and second harmonics of the Fourier transform to generate the representative image of the object.

18. 10. The hyperspectral imaging system of claim 1, wherein the at least one optical component further comprises at least one illumination source for illuminating the object, the illumination source generating illumination source radiation comprising at least one illumination wave.

19. 10. The hyperspectral imaging system of claim 1, further comprising at least one illumination source, the illumination source generating illumination source radiation comprising at least two illumination waves, each illumination wave having a different wavelength.

20. The hyperspectral imaging system of claim 1 , wherein the image forming system further comprises a control system, a hardware processor, a memory, and a display.

21. The hyperspectral imaging system of claim 1 , wherein the imaging system further comprises: a display of the representative image of the object on a display of the imaging system.

22. 10. The hyperspectral imaging system of claim 1, wherein the image forming system further comprises a control system, a hardware processor, a memory, and an information transmission system, the information transmission system transmitting the representative image of the object to a user in any manner.

23. 10. The hyperspectral imaging system of claim 1, wherein the image forming system further comprises a control system, a hardware processor, a memory, and an information transmission system, wherein the information transmission system transmits the representative image of the object to a user as an image, a number, a color, a sound, a mechanical movement, a signal, or a combination thereof.

24. The hyperspectral imaging system of claim 1 , wherein the at least one optical component further comprises an optical lens, an optical filter, a dispersive optical system, or a combination thereof.

25. The hyperspectral imaging system of claim 1 , wherein the detected target radiation is fluorescent radiation.

26. A hyperspectral imaging system comprising an image forming system for generating a representative image of an object, the image forming system comprising: the image forming system, acquiring detected radiation of the object; forming an object image using the detected object radiation, wherein the object image comprises at least two image pixels, each image pixel corresponding to a physical point on the object; forming at least one intensity spectrum for each image pixel; transforming the intensity spectrum of each image pixel based on the intensity spectrum of each image pixel; For each image pixel, form one phasor point on the phasor plane; forming a phasor histogram comprising at least two phasor bins, where each phasor bin comprises at least one phasor point; aggregating the detected spectra belonging to the image pixels for each phasor bin; generating a representative intensity spectrum for each phasor bin; unmixing the representative intensity spectra of the phasor bins using one or more unmixing techniques; determining the abundance of spectral endmembers in the representative intensity spectrum; generating a representative intensity image of the object, representing the abundance of the spectral endmembers; A hyperspectral imaging system configured to:

27. A method of operating a hyperspectral phasor system having an imaging system for generating a representative image of an object, comprising: the imaging system forming at least one intensity spectrum for image pixels of an image of the object, wherein the image of the object is based on the detected radiation; the image forming system, For each image pixel, form one phasor point on the phasor plane; forming a phasor histogram including at least two phasor bins, each phasor bin including at least one phasor point; aggregating the detected spectra of the image pixels of the at least two phasor bins; generating at least one representative intensity spectrum for the at least two phasor bins; the imaging system unmixing the at least one representative intensity spectrum of the at least two phasor bins using one or more unmixing techniques; generating a representative intensity image of the object based on at least the representative intensity spectrum and detected intensities corresponding to the detected radiation, the imaging system; A method of operating a hyperspectral phasor system, comprising:

28. A method of operating an imaging system to generate a representative image of an object, comprising: the imaging system forming at least one intensity spectrum for image pixels of an image of the object, wherein the image of the object is based on the detected radiation; generating at least one representative intensity spectrum based on phasor points on a phasor plane corresponding to the image pixels; the imaging system unmixing the at least one representative intensity spectrum using one or more linear unmixing techniques; generating a representative intensity image of the object based on at least the unmixed representative intensity spectrum, A method of operating an imaging system, comprising:

Images