Systems and methods for hyperspectral imaging

Abstract

Systems and methods for hyperspectral imaging are described. A hyperspectral imaging system includes: a sample holder configured to hold a sample; a light source configured to emit excitation light having one or more wavelengths; a two-dimensional imaging array; and an optical system. The optical system includes: a first dispersive element or a first diffraction element; and a second dispersive element or a second diffraction element. Wherein the optical system is configured to use the first dispersive element or the first diffraction element to construct the excitation light into a spectrally dispersed two-dimensional excitation pattern at a focal plane in the sample or a conjugate plane of the focal plane; illuminate the sample in the focal plane using an excitation pattern having one or more wavelengths dispersed along a first lateral direction; collect emission light from the sample; spectrally disperse the collected emission light using the second dispersive element or the second diffraction element; and focus the collected emission light to provide to the imaging array so that the collected emission light is spectrally dispersed along a second lateral direction, wherein the first lateral direction is different from the second lateral direction.

Description

[0001] This application is a divisional application of the invention patent application with an international application date of May 27, 2017, Chinese application number 201780032763.6, and invention name “System and method for 4-D hyperspectral imaging”.

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] This application claims the benefit of U.S. Provisional Application No. 62 / 342,252, filed on May 27, 2016, and U.S. Non-Provisional Application No. 15 / 607,457, filed on May 27, 2017, and incorporates the contents of each of these by reference. Technical Field

[0004] The present disclosure relates generally to the field of imaging and microscopy systems and methods. More specifically and without limitation, the disclosed embodiments relate to systems and methods for hyperspectral imaging using a spatial light modulator and a dispersive element. Background Art

[0005] Hyperspectral imaging is a remote sensing spectroscopy method that is often used for non-medical applications such as material identification, astronomy, surveillance, and geophysical applications. It is an "imaging spectroscopy" method that combines the spatial resolution of imaging with the chemical specificity of spectroscopy. The light collected from an object is dispersed into a spectrum of individual wavelengths or narrow spectral bands and detected on a two-dimensional (2-D) imaging sensor as a set of images, each representing one wavelength or spectral band. Therefore, the data set collected by a hyperspectral imaging system is often called a hyperspectral cube, represented in three dimensions (3-D): two spatial directions (x and y) and one spectral dimension (λ).

[0006] Hyperspectral imaging is an emerging technology for medical applications, such as disease diagnosis and surgery. Biological tissues have intrinsic and extrinsic optical features that can reflect their chemical, biophysical and / or morphological properties, such as endogenous fluorescence and exogenous fluorescence. Hyperspectral imaging can be used to study physiological and / or pathological changes in living tissue or tissue segmentation, and further provide information about tissue health or disease. For example, hyperspectral imaging can replace biopsy as a digital pathology tool. Typically, frozen segmentation biopsies are used to obtain information about tissue samples for tumor surgeons to make decisions during surgery. This type of biopsy may take about 10 to 20 minutes and uses a rapid staining protocol, which makes the feature definition worse than standard staining methods. By imaging the autofluorescence from tissue samples and parsing the spectral characteristics of the autofluorescence, hyperspectral imaging can be used as a faster tissue pathology analysis tool that allows for higher diagnostic accuracy than frozen segmentation biopsies. In addition, hyperspectral imaging can be used as an experimental tool in research and clinical trials, such as applications in immunohistochemistry (IHC) staining and fluorescence in situ hybridization (FISH), in which molecules labeled with various fluorophores with different spectral characteristics target specific proteins and nucleic acids.

[0007] The information in the spectral dimension of a hyperspectral cube typically reflects the intensity of light within a wavelength range emitted by a fluorophore or other type of optical label when excited at a given wavelength. However, if the light intensity is measured as a function of both the excitation wavelength and the emission wavelength, the optical properties of fluorophores, optical labels, fluorophore-labeled molecules, and / or biological tissues can be more accurately identified. In addition, in many medical applications, various types of intrinsic or extrinsic labels with different spectral characteristics can be used, including fluorophores, Raman labels, photoluminescent labels, or quantum dots (QDs). Measuring the combined excitation and emission spectra can enhance the ability to distinguish, identify, and characterize different labels of a given type or various types of labels.

[0008] Acquiring a hyperspectral imaging dataset with both an excitation spectrum and an emission spectrum is typically extremely time-consuming because it is necessary to measure the intensity of the light emitted at multiple excitation and emission wavelengths. For example, many wide-field 2-D fluorescence images can be acquired with different filters on the collection path, each filter transmitting a narrow spectral band in the emission spectrum. To acquire additional excitation spectra, this process then needs to be repeated using many different excitation lasers at different excitation wavelengths. In addition, if one narrow-band filter is used at a time, most of the collected fluorescence will be discarded. As a result, this process is inefficient and can even result in photobleaching of fluorophores in the sample, rendering them unable to fluoresce forever. Therefore, a fast, efficient, and automated method and system are needed to acquire a hyperspectral imaging dataset with both an excitation spectrum and an emission spectrum. Summary of the invention

[0009] Embodiments of the present disclosure include systems and methods for implementing hyperspectral imaging that allows acquisition of hyperspectral imaging datasets having both excitation spectra and emission spectra. Advantageously, exemplary embodiments allow for rapid, efficient, and automated acquisition of four-dimensional (4-D) hyperspectral imaging datasets, including two spatial dimensions (horizontal x and vertical y), one excitation spectral dimension (λ a ) and an emission spectrum dimension (λ b ).

[0010] According to another exemplary embodiment of the present disclosure, a hyperspectral imaging system is described. The system may include a sample holder configured to hold a sample, an illumination system, and a detection system.

[0011] The illumination system may include a light source configured to emit excitation light having one or more wavelengths and a first set of optical elements. The first set of optical elements may include a first spatial light modulator (SLM), at least one lens, and at least one dispersive element. The illumination system may be configured to construct the excitation light into a predetermined two-dimensional pattern at a conjugate plane of a focal plane in the sample, spectrally disperse the constructed excitation light in a first lateral direction, and illuminate the sample with the excitation pattern having one or more wavelengths dispersed in the first lateral direction.

[0012] The detection system may include a two-dimensional imaging device and a second set of optical elements. The second set of optical elements may include at least one lens and at least one dispersive element. The detection system may be configured to spectrally disperse the emission light collected from the sample in a second lateral direction and image the spectrally dispersed emission light to the imaging device.

[0013] According to another exemplary embodiment of the present disclosure, a method for hyperspectral imaging is described. The method includes the following steps: providing a light source that emits excitation light having one or more wavelengths; constructing the excitation light from the light source into a predetermined two-dimensional pattern at a conjugate plane of a focal plane in a sample by a first spatial light modulator (SLM); spectrally dispersing the constructed excitation light in a first lateral direction by a first dispersive element; irradiating the sample with the excitation pattern having one or more wavelengths dispersed in the first lateral direction; spectrally dispersing the emission light collected from the sample in a second lateral direction by a second dispersive element; and imaging the spectrally dispersed emission light to an imaging device.

[0014] According to another exemplary embodiment of the present disclosure, a method for configuring a microscope to obtain a hyperspectral imaging data set of a sample is described. The method includes the following steps: providing a light source that emits excitation light having one or more wavelengths; constructing the excitation light from the light source into a predetermined two-dimensional pattern at a conjugate plane of a focal plane in the sample through a first spatial light modulator (SLM); spectrally dispersing the constructed excitation light in a first lateral direction through a first dispersive element; irradiating the sample with an excitation pattern having one or more wavelengths dispersed in the first lateral direction; spectrally dispersing the emission light collected from the sample in a second lateral direction through a second dispersive element; and imaging the spectrally dispersed emission light to an imaging device.

[0015] Additional features and advantages of the disclosed embodiments will be set forth in part in the following description, and in part will be apparent from the description, or may be learned by practicing the disclosed embodiments. The features and advantages of the disclosed embodiments will be realized and obtained by the elements and combinations particularly pointed out in the appended claims.

[0016] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments, as claimed.

[0017] The accompanying drawings constitute a part of this specification. The drawings illustrate several embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosed embodiments, as set forth in the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a diagram of an exemplary scheme for acquiring a hyperspectral imaging dataset according to an embodiment of the present disclosure.

[0019] Figure 2 is a diagram of another exemplary scheme for acquiring a hyperspectral imaging dataset according to an embodiment of the present disclosure.

[0020] Figure 3 is a schematic representation of an exemplary hyperspectral imaging system according to an embodiment of the disclosure.

[0021] Figure 4 is a schematic representation of another exemplary hyperspectral imaging system according to an embodiment of the present disclosure.

[0022] Figure 5 is a schematic representation of another exemplary hyperspectral imaging system according to an embodiment of the present disclosure.

[0023] Figure 6 is a schematic representation of another exemplary hyperspectral imaging system according to an embodiment of the present disclosure.

[0024] Figure 7 is a schematic representation of another exemplary hyperspectral imaging system according to an embodiment of the present disclosure.

[0025] Figure 8 is a schematic representation of another exemplary hyperspectral imaging system according to an embodiment of the present disclosure.

[0026] Fig. 9 is a schematic representation of another exemplary hyperspectral imaging system according to an embodiment of the present disclosure.

[0027] Fig.10 is a schematic representation of another exemplary hyperspectral imaging system according to an embodiment of the present disclosure.

[0028] Fig.11 is a schematic representation of another exemplary hyperspectral imaging system according to an embodiment of the present disclosure.

[0029] Fig.12 is a schematic representation of an exemplary diffractive element according to an embodiment of the present disclosure.

[0030] Fig.13 is a schematic representation of another exemplary diffractive element according to an embodiment of the present disclosure.

[0031] Fig.14 is a flow chart of an exemplary method for hyperspectral imaging according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] The disclosed embodiments relate to systems and methods for implementing hyperspectral imaging that allows acquisition of hyperspectral imaging data sets with both excitation spectra and emission spectra. Embodiments of the present disclosure can be implemented using a microscope having one or more 2-D imaging devices (e.g., CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) sensors or cameras), such as a fluorescence microscope, a confocal microscope (with confocality along at least one dimension), a transmission microscope, or a reflection microscope. Alternatively, according to embodiments of the present disclosure, an optical system can be constructed using suitable optical elements.

[0033] Rather than using the time-consuming process of acquiring a hyperspectral cube for each excitation wavelength, embodiments of the present disclosure allow acquisition of 2-D images of emission spectra corresponding to more than one excitation wavelength for a subset of regions on a sample. Multiple 2-D images can be acquired and computationally reconstructed to obtain a 4-D hyperspectral imaging dataset of the sample.

[0034] According to one aspect of the present disclosure, excitation light having one or more wavelengths can be used to excite fluorophores in a sample. The excitation light can be generated by a polychromatic light source that emits light having one or more wavelengths. In some embodiments, the polychromatic light source can have a continuous spectrum. For example, the polychromatic light source can be a broadband light source, such as a supercontinuum laser, a white light source (e.g., a high-pressure mercury lamp, a xenon lamp, a halogen lamp, or a metal halide lamp), or one or more LEDs. In other embodiments, the polychromatic light source can have a discrete spectrum. For example, the polychromatic light source can be a combination of pulsed or continuous "single wavelength" lasers that emit light having a narrow spectrum.

[0035] According to one aspect of the present disclosure, the excitation light emitted by the light source can be constructed to excite a subset of regions on the sample with an excitation pattern using a spatial light modulator (SLM). To construct the excitation light, the SLM can modulate the phase or amplitude of the excitation light by selectively activating or switching its pixels. In some embodiments, the SLM can be selected from a group of SLMs including a digital micromirror device (DMD), a diffractive optical element, a liquid crystal device (LCD), and a liquid crystal-on-silicon (LCOS) device.

[0036] According to one aspect of the present disclosure, the constructed excitation light can be spectrally dispersed in a first lateral direction (e.g., vertical direction y or horizontal direction x). The spectral dispersion of the excitation light can separate or split one or more wavelengths of the excitation light spectrum in the first lateral direction. In some embodiments, at least one dispersive element can be used to spectrally disperse the excitation light before the excitation light illuminates the sample with an excitation pattern. At least one dispersive element can be a diffraction grating or a prism, or a combination of one or more prisms. Thus, a spectrally dispersed excitation pattern can be generated to illuminate areas on the sample at different spatial locations.

[0037] Fluorophores or other types of optical tags in a sample can be excited by the excitation light that illuminates the sample. When they relax to their ground state, the fluorophores or optical tags can emit light in a range of wavelengths (called the emission spectrum). Fluorophores or optical tags can have different emission spectra corresponding to different wavelengths of the excitation light.

[0038] As described herein, fluorophores are used as exemplary optical labels in the present disclosure. The description of reference fluorophores is equally applicable to other types of optical labels consistent with embodiments of the present disclosure. For example, excitation light emitted from a light source can also excite other types of optical labels, which can emit light with emission spectra when excited. Therefore, the fluorescence and fluorescence emission spectra used in the description of the present disclosure can also be used to represent the emission light and emission spectra of other optical labels.

[0039] According to one aspect of the present disclosure, fluorescence emitted by a fluorophore excited by excitation light in a given region of a sample can be spectrally dispersed in a second lateral direction (e.g., horizontal direction x or vertical direction y). At least one dispersive element can be used to disperse the fluorescence spectrum into a fluorescence emission spectrum corresponding to the excitation wavelength in the given region. The fluorescence emission spectrum of a subset of regions on the sample can be acquired as a 2-D image by a 2-D imaging device in a single exposure.

[0040] According to one aspect of the present disclosure, fluorescence excitation and emission spectra of all areas on a sample or field of view can be acquired by scanning a spectrally dispersed excitation pattern in first and second lateral directions and acquiring a 2-D image of the fluorescence emission spectrum at each spatial position of the excitation pattern.

[0041] In some embodiments, the excitation pattern is scanned across the sample or field of view by modulating the pixels of the SLM. In other embodiments, an xy translation stage can be used to laterally scan the excitation pattern across the sample or field of view by moving the sample or the diffraction grating in the first and second lateral directions. The translation stage can be a motorized translation stage, a piezoelectric translation stage, or any suitable stage that allows lateral linear motion.

[0042] Advantageously, a 4-D hyperspectral imaging dataset can be computationally reconstructed from 2-D images of the emission spectrum, each 2-D image corresponding to the excitation pattern at a different spatial location on the sample.

[0043] In some aspects, the systems and methods according to the present disclosure allow for confocal optical segmentation. This can allow acquisition of hyperspectral imaging datasets for multiple focal planes along the axial direction of the sample. According to one aspect of the present disclosure, a hyperspectral imaging dataset for a focal plane can be acquired by implementing one or more optical pinholes at a plane conjugate to the selected focal plane. The optical pinhole can be one or more spatial pinholes, or a programmable artificial pinhole formed by the pixels of the second SLM.

[0044] Advantageously, the degree of confocality can be adjusted as desired by changing the size and / or spacing of the artificial pinholes formed by the SLM. In addition, the pinhole pattern is formed by the SLM by selectively modulating or switching its pixels to match the excitation pattern of the excitation light. The pinhole pattern can advantageously allow confocal imaging of multiple areas on the sample that are simultaneously illuminated by the excitation pattern. This can increase the speed and / or throughput of acquiring hyperspectral imaging data sets on the sample in the focal plane compared to conventional confocal microscopy using sequential point-by-point scanning.

[0045] Reference will now be made in detail to embodiments and aspects of the present disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numerals will be used throughout the drawings to refer to the same or like parts.

[0046] As described herein, in order to illustrate light of different wavelengths or frequencies, dotted textures of different densities are used in the drawings. Higher densities correspond to longer wavelengths or lower frequencies of light. In addition, vertical and horizontal directions are used as examples to illustrate the first and second lateral directions. Alternatively, the horizontal direction can be the first lateral direction and the vertical direction can be the second lateral direction. As described herein, any two suitable different directions or a pair of non-parallel (e.g., orthogonal) directions can be used as the first and second lateral directions.

[0047] Example protocol for acquiring a hyperspectral imaging dataset

[0048] Figure 1 An exemplary scheme for acquiring a hyperspectral imaging dataset used by the methods and systems of the present disclosure is illustrated. Figure 1 As shown, excitation light having a discrete spectrum is irradiated onto the sample in an excitation pattern 100. The excitation pattern 100 may include a 2-D array of excitation points. For example, Figure 1A portion of an exemplary 2-D array of circular excitation spots is shown. Additional spots of the array may be located above, below, to the left, and / or to the right of the exemplary array (not shown). As described herein, the appropriate size of the array and the appropriate shape, size, and / or spacing of the spots may be predetermined based on the application.

[0049] The discrete spectrum of the excitation light includes multiple discrete wavelengths or multiple narrow spectral bands. Therefore, when the excitation light is spectrally dispersed along a given lateral direction by the dispersion element, the excitation pattern 100 can be spectrally dispersed so that light of different wavelengths is guided to different positions in the given lateral direction. For example, Figure 1 As shown, the excitation pattern 100 may include a plurality of scanning units 110, for example, a 2-D array of scanning units 110. When the excitation light is spectrally dispersed along a vertical direction, each scanning unit 110 may include a plurality of excitation points 112a, 112b, 112c, 112d, 112e, and 112f, which are vertically offset from each other and correspond to different excitation wavelengths of the excitation light generated by the spectral dispersion.

[0050] The vertical spacing between the excitation points may or may not be uniform and may be predetermined by various factors, such as the excitation wavelength, the size of the points, and the amount of dispersion of the excitation light. The total number of vertically dispersed excitation points in the scanning unit 110 may depend on the number of discrete wavelengths or narrow spectral bands of the excitation light.

[0051] To generate an excitation spectrum for a given spatial position on the sample, the sample can be scanned in the vertical direction as Figure 1 The spectrally dispersive excitation pattern 100 shown allows excitation points corresponding to different excitation wavelengths to be shifted to the given spatial position. For example, when the excitation pattern 100 is shifted in the vertical direction, the area in each scanning unit 110 previously illuminated by the excitation points 112a, 112b, 112c, 112d, 112e, and 112f can be illuminated by different excitation points among these excitation points. For example, by shifting the excitation pattern 100 at one excitation point, the area previously illuminated by the excitation point 112b is illuminated by the excitation point 112a, the area previously illuminated by the excitation point 112b is illuminated by the excitation point 112c, the area previously illuminated by the excitation point 112e is illuminated by the excitation point 112d, the area previously illuminated by the excitation point 112f is illuminated by the excitation point 112e, and the area previously illuminated by the excitation point 112a is illuminated by the excitation point 112f shifted from the scanning unit (not shown) located above.

[0052] The area within each scanning unit 110 can be scanned by shifting the spectrally dispersed excitation pattern 100 in the vertical and horizontal directions. For example, by shifting the excitation pattern 100 in the vertical direction over the length of the scanning unit 110, a given area in the scanning unit 110 can be illuminated by different excitation points corresponding to different excitation wavelengths of the light source. By shifting the excitation pattern 100 vertically and / or horizontally (e.g., based on the desired vertical and / or horizontal resolution) over the length of the scanning unit 110 in a continuous manner or at predetermined intervals, each given area in the scanning unit 110 can be illuminated by a different excitation point.

[0053] like Figure 1 As shown, the excitation points of the excitation pattern 100 are spaced apart in the horizontal direction at a given period. Advantageously, the periodic spacing of the excitation points in the horizontal direction allows for the measurement of the fluorescence emission spectra of the excited regions on the sample. For example, the emitted fluorescence from a given region illuminated by the excitation points can be spectrally dispersed in the horizontal direction without overlapping with the emitted fluorescence of another region. Therefore, the fluorescence emission spectra of multiple regions simultaneously illuminated by the excitation points can be generated and acquired by a 2-D imaging sensor or device.

[0054] Figure 1 A 2-D image 200 of a fluorescence emission spectrum is shown, where the excitation wavelength (λ a ) is expressed in the vertical direction, and the emission wavelength (λ b ) is expressed in the horizontal direction. Figure 1 The figure shows that in the 2-D image 200, the areas excited by the excitation points 112a, 112b, 112c, 112d, 112e and 112f corresponding to different excitation wavelengths can generate different fluorescence emission spectra 212a, 212b, 212c, 212d, 212e and 212f, which extend in the horizontal direction and are correspondingly offset from each other in the vertical direction. Therefore, multiple fluorescence emission spectra can be acquired in the 2-D image 200, each emission spectrum corresponding to an excitation point of the excitation pattern 100 at a different spatial position on the sample.

[0055] As described above, different areas in each scanning unit 110 can be illuminated by spatially shifting the excitation pattern 100 laterally in the vertical and horizontal directions. At each spatial position of the excitation pattern 100, a fluorescence emission spectrum of the illuminated area can be acquired on the 2-D image 200. Therefore, multiple 2-D images 200 of fluorescence emission spectra can be acquired corresponding to a series of excitation patterns 100 that are laterally shifted from each other.

[0056] By combining the datasets of the acquired 2-D image 200, a fluorescence excitation-emission matrix (EEM) can be acquired for each pixel or spatial location in the 2-D image 200. The fluorescence EEM can record or display the fluorescence intensity as a function of multiple excitation wavelengths and emission wavelength ranges. Therefore, a 4-D hyperspectral imaging dataset of a sample having both an excitation spectrum and an emission spectrum can be collected and reconstructed from the acquired 2-D image 200.

[0057] Figure 2 Another exemplary scheme for obtaining a hyperspectral imaging dataset by the method and system of the present disclosure is illustrated. Figure 2 As shown in FIG. 1 , when the excitation light has a continuous spectrum and is spectrally dispersed along the vertical direction by the dispersion element, the focus point of the excitation light can be spectrally dispersed into a continuous excitation line along the vertical direction. In this case, as Figure 2 As shown, the excitation pattern 100 may be spectrally dispersed into a 2-D array of excitation lines 114, each of which represents a wavelength range of excitation light that is vertically offset in a sequential manner.

[0058] Figure 2 A portion of an exemplary 2-D array of excitation lines 114 is shown, showing a 3×3 2-D array. Other excitation lines of the array may be located above, below, to the left, and / or to the right of the exemplary array (not shown). As described herein, a suitable size of the array, suitable shape, size, and / or spacing of the excitation lines may be selected according to a given application.

[0059] The area within each scanning unit 110 can be similarly scanned as described above by shifting the spectrally dispersed excitation pattern 100 in the vertical and horizontal directions. An array of fluorescence emission spectra 214 corresponding to the array of excitation lines 114 of the excitation pattern 100 can be similarly acquired on the 2-D image 200. Each fluorescence emission spectrum 214 in the 2-D image 200 corresponds to a continuous strip on the sample illuminated by the excitation line 114 of the excitation pattern 100.

[0060] exist Figure 2 In the illustrated scheme, in some embodiments, when the spectrally dispersed excitation pattern 100 is shifted in the vertical direction, after the vertical shift, the area along the excitation line 114 excited at a first wavelength can then be excited at a second wavelength that is longer or shorter than the first wavelength. Therefore, by vertically shifting the excitation pattern 100 in a continuous manner or at a predetermined pitch (e.g., based on the desired vertical resolution), the area illuminated by the excitation line 114 in each scanning unit 110 can be illuminated by a different excitation wavelength of the light source. Similar to Figure 1In the illustrated scheme, other areas on the sample in each scanning unit 110 can be illuminated by different excitation lines by shifting the excitation pattern 100 in the horizontal direction.

[0061] As described herein, the area on the sample illuminated by the excitation pattern 100 can be substantially determined by the size and shape of the excitation spots or lines of the excitation pattern 100. The size and shape of the excitation spots or lines can be determined by many factors of the optical system, including the size and shape of the pixels of the SLM, the magnification of the optical system, and the degree of spectral dispersion of the excitation light.

[0062] The horizontal and / or vertical spatial spacing between excitation points or lines of the excitation pattern 100 can be predetermined based on various factors, such as the excitation wavelength, the size of the sample, the field of view of the optical system, the desired measurement throughput, spatial resolution and / or speed, and the amount of spectral dispersion of the excitation light and / or emitted fluorescence.

[0063] For example, the spatial spacing between the excitation points or lines in the vertical direction can be predetermined based on the spectral dispersion of the excitation light, so that the excitation points or lines do not overlap in the vertical direction. The spatial spacing between the excitation points or lines in the horizontal direction can be predetermined based on the range of the fluorescence emission spectrum in the horizontal direction, so that the fluorescence emission spectra do not overlap each other in the horizontal direction.

[0064] In some embodiments, the horizontal and / or vertical periodicity of the arrays of excitation points of different wavelengths may be the same. In other embodiments, the horizontal and / or vertical periodicity of the arrays of excitation points of different wavelengths may be different. In some cases, different spatial periods may be convenient for computationally reconstructing a 4-D hyperspectral imaging dataset, for example, where the SLM is placed at the Fourier plane of the sample to generate the excitation pattern 100, as further described below.

[0065] The embodiments described below with reference to schematic representations of optical systems and / or components relate to systems and methods for implementing the above-described schemes for acquiring 4-D hyperspectral imaging datasets. The schematic representations should be understood as not being drawn to scale.

[0066] Exemplary Optical Systems and Components

[0067] Figure 3 is a schematic representation of an exemplary hyperspectral imaging system 300. In some embodiments, system 300 may be a fluorescence microscope, a transmission microscope, or a reflection microscope, or a confocal fluorescence microscope (having confocality along at least one dimension). Embodiments of the present disclosure may be applied to other suitable microscopy techniques for performing hyperspectral imaging.

[0068] like Figure 3As shown, system 300 may include an illumination system and a detection system, each having multiple components. The illumination system may include a light source 310, a first SLM 320a, at least one lens (e.g., lens 330a), and at least one dispersive element (e.g., dispersive element 340a). The detection system may include a 2-D imaging device 380, at least one lens (e.g., lens 330b), and at least one dispersive element (e.g., dispersive element 340b). Depending on its layout, geometry, and / or application, system 300 may further include a beam splitter 350, an objective lens 360, a polarizer 390a, and / or a sample holder 370 for placing a sample to be imaged. System 300 may further include other optical elements, such as a reflector, a beam dump, and the like.

[0069] As described herein, the optical axis of system 300 can define the path that excitation light and emitted fluorescent light from a sample propagate through system 300 .

[0070] In lighting systems, such as Figure 3 As shown, light source 310 emits excitation light 402, which is directed to SLM 320a. Excitation light 402 can be collimated and / or expanded using one or two lenses (not shown). SLM 320a can construct excitation light 402 by modulating the phase or amplitude of excitation light 402 by selectively actuating or switching pixels of excitation light 402. At least a portion of the pixels of SLM 320a reflect excitation light 402 and direct it along the optical axis of system 300.

[0071] like Figure 3 As shown, the reflected excitation light 404 is transmitted through the lens 330a and the dispersive element 340a. The lens 330a can collimate the reflected excitation light 404 along the optical axis. The dispersive element 340a spectrally disperses the reflected excitation light 404 along a first lateral direction. For example, the dispersive element 340a can cause a small wavelength-dependent angular deflection to the reflected excitation light 404. The spectrally dispersed excitation light 406 can be reflected by the beam splitter 350 and directed to the objective lens 360. The objective lens 360 then focuses the spectrally dispersed excitation light 406 onto a sample placed on the sample holder 370.

[0072] In the detection system, Figure 3As shown, fluorescence emitted by excited fluorophores in the sample is collected and / or collimated by objective lens 360. Fluorescence 408 is transmitted through beam splitter 350 and dispersive element 340b along the optical axis of system 300. Dispersive element 340b spectrally disperses fluorescence 408 along a second lateral direction as described above. Spectrally dispersed fluorescence 410 is transmitted through lens 330b and acquired by 2-D imaging device 380. Imaging device 380 can be placed approximately one focal length away from lens 330b so that lens 330b can image and focus spectrally dispersed fluorescence 410 onto a 2-D sensor of imaging device 380.

[0073] Other configurations of system 300 are possible using additional optical elements, such as mirrors, lenses, etc., as described further below.

[0074] The functions and working principles of the various components of system 300 are described in detail below.

[0075] light source

[0076] As described above, the light source 310 can have a continuous spectrum or a discrete spectrum. The light source 310 can be a white light source, such as a supercontinuum laser, or a combination of "single wavelength" lasers with discrete narrow spectral bands. In some embodiments, the excitation light 402 emitted by the light source 310 can be directed directly to the SLM 320a. In other embodiments, the excitation light 402 can be collimated and / or expanded by a lens before being incident on the SLM 320a. Additionally or alternatively, the excitation light 402 can be diffused using a diffuser or despeckle element to reduce the speckle effect of coherent illumination.

[0077] In some embodiments, the light source 310 is operably connected to a controller (not shown) having a processor and a computer-readable medium storing instructions or operating steps. When executed by the processor, these instructions or steps modulate the operating state of the light source 310. For example, the processor can activate or deactivate the light source 310, modulate the duration of the pulse when the light source 310 is a pulsed light source, and / or switch or tune the emission wavelength of the light source 310.

[0078] Spatial light modulator for modulating the excitation light

[0079] As described above, to configure excitation light 402 for illuminating a sample with excitation pattern 100, SLM 320a may modulate the amplitude or phase of excitation light 402 by selectively modulating pixels of excitation light 402 between operating states.

[0080] Amplitude Modulation

[0081] In some embodiments, the amplitude of the excitation light 402 can be modulated by the SLM 320a. For example, the SLM 320a can be a digital micromirror device (DMD) having an array of multiple micromirrors (not shown). These mirrors can be individually actuated to switch between two operating positions, an "on" position and an "off" position. When the micromirrors are configured to be in the "on" position, the excitation light 402 is reflected to propagate along the optical axis as reflected excitation light 404 directed to the sample. When the micromirrors are configured to be in the "off" position, the excitation light 402 is reflected in a direction that deviates from the optical axis and is not directed to the sample (not shown). In some embodiments, the excitation light 402 reflected by the "off" micromirror can be directed to other optical elements, such as a mirror or a beam collector (not shown).

[0082] In some embodiments, the micromirror is square with side lengths ranging from about a few microns to about 10 μm. Other shapes and sizes of micromirrors are also possible and can be used as appropriate. A DMD is typically able to change or alternate the "on" and "off" positions of a micromirror very quickly.

[0083] In some embodiments, a single micromirror of a DMD may be referred to as a single pixel. In other embodiments, multiple micromirrors may be referred to as a single pixel. For example, a group of closely adjacent micromirrors may be referred to as a single pixel and may be modulated or actuated to the same position.

[0084] The amplitude modulation pattern can be formed by the micromirrors or pixels of the DMD in the "on" position. The amplitude modulation pattern can be imaged onto the sample as the excitation pattern 100 through the lens 330a and the objective lens 360. For example, the lens 330a is used as a tube lens and combined with the objective lens 360 to form an imaging configuration. The DMD is placed on a conjugate plane of the sample or approximately one focal length before the lens 330a. Depending on the focal lengths of the lens 330a and the objective lens 360, the excitation pattern 100 can be an enlarged or reduced image of the amplitude modulation pattern.

[0085] In other embodiments, in order to modulate the amplitude of the excitation light 402, the SLM 320a can be a liquid crystal device (LCD) or a liquid crystal on silicon (LCOS) device. The pixels of the SLM 320a can create an amplitude modulation pattern by manipulating the polarization of the light incident on the pixel. Similar to the DMD, the LCD or LCOS device can be placed on a conjugate plane of the sample. The pixels of the LCD or LCOS device can be electrically modulated between an "on" state and an "off" state in a pixel-by-pixel manner. The "on" pixel can rotate the orientation of the linearly polarized light by approximately 90°, while the "off" pixel does not perform the rotation. In this case, a first linear polarizer (not shown) can be used to linearly polarize the excitation light 402. A second linear polarizer or polarizing beam splitter (polarizing beam splitter, PBS) (not shown) can be used to transmit the excitation light 404 reflected by the "on" pixel and block the excitation light 402 reflected by the "off" pixel.

[0086] A disadvantage of using SLM 320a to modulate the amplitude of excitation light 402 is the loss of light during modulation. This is because most of the pixels of SLM 320a are typically in the "off" state. Therefore, most of the excitation light 402 is diverted away from the optical axis, does not reach the sample, and is therefore lost. An excitation light recycling system can be used to reduce this loss by redirecting excitation light that leaves the optical axis back to the optical axis, as described further below.

[0087] Phase Modulation

[0088] In some embodiments, the phase of the excitation light 402 may be modulated by the SLM 320a. The SLM 320a may be a reflective LCD or LCOS device. Figure 4 is a schematic representation of an exemplary system 300 using an LCD or LCOS device as an SLM 320a. Figure 4 As shown, an LCD or LCOS device can be placed near the conjugate plane 322 of the sample. A custom phase modulation pattern can be formed by the pixels of the LCD or LCOS device. The phase modulation pattern can create an array of off-axis lens phase distributions. The wavefront of the excitation light 402 can then be modulated by the phase modulation pattern and form a preliminary excitation pattern (e.g., a diffraction pattern) at the conjugate plane 322. The preliminary excitation pattern can be an enlarged or reduced image of the excitation pattern 100 and can include an array of focus points.

[0089] The conjugate plane 322 may be located at a short distance outside the SLM 320a. The focal plane of the focal point of the preliminary excitation pattern may be wavelength-dependent. Therefore, the excitation light 402 of different wavelengths may not all be focused on the conjugate plane 322. In some embodiments, the focal plane of the central wavelength of the excitation light 402 is approximately at the conjugate plane 322. The preliminary excitation pattern formed at or near the conjugate plane 322 is then imaged onto the sample as the excitation pattern 100 by the lens 330a and the objective lens 360. Although the excitation patterns 100 of different wavelengths may have slightly different focal planes in this configuration, modulating the phase of the excitation light 402 increases the efficiency of using the excitation light 402 compared to amplitude modulation.

[0090] In other embodiments, the LCD or LCOS device can be placed on an aperture plane, which can be a conjugate plane of the back aperture of the objective lens 360 or a Fourier plane of the sample. For example, an exemplary configuration of the system 300 can have two tube lenses (not shown) placed between the SLM 320a and the objective lens 360. The first tube lens can be located approximately one focal length behind the SLM 320a. The second tube lens can be located approximately two focal lengths behind the first tube lens. The objective lens 360 can be located approximately one focal length behind the second tube lens.

[0091] The pixels of the LCD or LCOS device can form a custom phase modulation pattern to modulate the wavefront of the excitation light 402. When the LCD or LCOS device reflects the excitation light 402, the phase at different positions of the wavefront of the reflected excitation light 404 can be selectively changed according to the phase modulation pattern. In some embodiments, the pixels of the LCD or LCOS device can be electrically modulated between an "on" state and an "off" state in a pixel-by-pixel manner. If the pixels of the LCD or LCOS device are in the "on" state, they can change the phase of the reflected light by changing the optical path length of the light traveling in the liquid crystal; and if they are in the "off" state, they may not change the phase of the reflected light. This allows the phase modulation pattern formed by the pixels to be digitally customized as needed. In other embodiments, the pixels of the LCD or LCOS device can have multiple states or adjustment levels (e.g., 256 levels) and can be individually modulated to the desired state or level. Advantageously, increasing the adjustment state or level of the pixel increases the continuity of the adjustment of the phase modulation pattern, thereby increasing the adjustment of the phase of the excitation light 402.

[0092] Phase modulation can present wavelets of the reflected excitation light 404 with different directions and / or phases. When the reflected excitation light 404 propagates along the optical axis, each of the tube lens and the objective lens 360 can perform a Fourier transform on the wavefront of the reflected excitation light 404. A diffraction pattern can then be formed on the focal plane of the objective lens 360. When irradiated on the sample, the diffraction pattern is referred to herein as the excitation pattern 100.

[0093] In the above configuration, because the phase modulation pattern is at or approximately at the Fourier plane of the sample, the phase modulation pattern is the inverse Fourier transform of the desired excitation pattern 100 impinged on the sample. Because the Fourier transform includes a scaling factor that is proportional to the wavelength of the light, the spatial period of the array of excitation points of different wavelengths in the excitation pattern 100 can be different. For example, longer wavelengths will diffract at larger angles, which can translate into larger spatial periods. This can result in the corresponding array of fluorescence emission spectra acquired in the 2-D image 200 having different spatial periods. Custom computer algorithms can be used to generate time-varying phase modulation patterns for scanning across a field of view and / or for computationally reconstructing a 4-D hyperspectral imaging dataset from such a 2-D image dataset.

[0094] Advantageously, modulating the phase of the excitation light 402 can allow it to propagate with substantially uniform intensity in the near field of the LCD or LCOS device, thereby reducing the loss of light. The modulated excitation light can then form a customizable or programmable excitation pattern 100 on the sample in the far field. Therefore, modulating the phase of the excitation light 402 to create the excitation pattern 100 can significantly improve the illumination efficiency of the system 300 by reducing the loss of the excitation light 402, compared to modulating the amplitude of the excitation light 402 as described above. In addition, increasing the continuity of the phase modulation of the excitation light 402 can further increase the diffraction efficiency of the LCD or LCOS device, thereby increasing the illumination efficiency of the system 300.

[0095] Alternatively, the LCD or LCOS device used to modulate the amplitude or phase of the excitation light 402 can be a transmissive device implemented along the optical axis. The geometry of the illumination system can be appropriately designed so that the amplitude or phase modulation pattern formed by the pixels of the device can modulate the amplitude or phase of the excitation light 402 as described above.

[0096] Regardless of whether the SLM 320a modulates the amplitude or phase of the excitation light 402, the excitation pattern 100 can be programmed and customized as desired by modulating the pixels of the SLM 320a between two or more operating states or levels on a pixel-by-pixel basis. In addition, by scanning or shifting the modulation of the pixels of the SLM 320a, the excitation pattern 100 can be translated or shifted in a given spatial direction, such as horizontally or vertically. This advantageously allows the excitation pattern 100 to be scanned across the field of view of the system 300 without using an xy translation stage to move the sample and / or sample holder 370.

[0097] In some embodiments, the excitation light 402 may be directed to the SLM 320a at a predetermined angle relative to the plane of the SLM 320a, depending on the type and modulation characteristics of the pixels of the SLM 320a. The predetermined angle may depend on the type of the SLM 320a and / or the geometry of the system 300. In some cases, when the SLM 320a is a reflective SLM that modulates the phase of the excitation light 402, the excitation light 402 may be directed to the SLM 320a at an angle such that the reflected excitation light 404 propagates along the optical axis of the system 300. In other cases, when the SLM 320a is a DMD, the excitation light 402 may be directed to the DMD at an angle such that the excitation light 404 reflected by the "on" micromirror propagates along the optical axis.

[0098] In some embodiments, SLM 320a is operably connected to a controller (not shown) having a processor and a computer-readable medium storing instructions or operating steps. When executed by the processor, these instructions or steps modulate the operating states of the pixels of SLM 320a to form a desired excitation pattern 100 and / or to translate the excitation pattern 100 in a desired spatial direction over a predetermined distance across the field of view.

[0099] Lenses and Objectives

[0100] The various lenses of the system 300, such as lenses 330a and 330b, can be achromatic, such as achromatic doublets or triplet lenses, to limit or reduce the effects of chromatic aberration and / or spherical aberration of the system. In addition, the objective lens 360 of the system 300 can be achromatic. Alternatively or additionally, the objective lens 360 can be an infinity-corrected objective, so that the objective lens 360 can form a desired focus (e.g., a focal point or a focal pattern) of a collimated light beam entering from its rear aperture. The use of an achromatic lens and / or an achromatic or infinity-corrected objective can allow excitation light 402 of different wavelengths to have at least approximately the same focus in the sample. In addition, the use of an achromatic lens and / or an achromatic objective can allow fluorescence of different wavelengths from a focal plane in the sample to similarly form a focused image at the imaging device 380. Therefore, the use of an achromatic lens and / or an achromatic objective can improve the quality of the 2-D image 200 of the fluorescence emission spectrum, thereby improving the quality of the reconstructed hyperspectral imaging data set.

[0101] Dispersion Components

[0102] Dispersive elements 340a and 340b may be diffraction gratings or prisms, such as non-deviant prisms (e.g., Amici prisms or double Amici prisms). The types of dispersive elements 340a and 340b may be the same or may be different. The degree of dispersion caused by dispersive elements 340a and 340b may be the same or different and may be predetermined based on various factors, such as the spectral range of the excitation light and the fluorescence, the size of the sample or field of view, the size of the imaging device 380, the desired spectral resolution, and the application of the system 300.

[0103] In some embodiments, the degree of dispersion caused by the dispersive elements 340a and 340b can be adjustable. For example, the dispersive element 340a can be a pair of double Amici prisms placed along the optical axis of the system 300. At least one of the pair of double Amici prisms can rotate relative to the other around the optical axis. The rotation of the double Amici prisms relative to each other can allow continuous control of the amount and / or angular orientation of the spectral dispersion of the excitation light 402. Similarly, the dispersive element 340b can be a pair of double Amici prisms, allowing continuous control of the amount and / or angular orientation (e.g., dispersion angle) of the spectral dispersion of the fluorescence 408.

[0104] Excitation light blocking

[0105] Because the intensity of the excitation light 402 can be several orders of magnitude stronger than the fluorescence 408, the excitation light 402 reflected and / or scattered by the sample and / or the sample holder 370 can enter the detection system and affect the detection or acquisition of the fluorescence emission spectrum by the imaging device 380. Therefore, embodiments of the present disclosure can reduce or block the propagation of the excitation light 402 into the detection system, as described below.

[0106] In some embodiments, beam splitter 350 can be used to reject or block excitation light 402 from propagating into the detection system. For example, beam splitter 350 of system 300 can be a dichroic beam splitter, a polarizing beam splitter (PBS), or another suitable type of beam splitter.

[0107] When the light source 310 or the excitation light 402 has a discrete spectrum with one or more discrete wavelengths or narrow spectral bands, the beam splitter 350 can be a dichroic beam splitter that selectively reflects and transmits light according to its wavelength. For example, the beam splitter 350 can be a multiband dichroic beam splitter with multiple cut-off wavelengths and passbands. The multiband dichroic beam splitter can be selected to substantially reflect the wavelength of the excitation light 402 and substantially transmit the wavelength of the fluorescence 408. In this case, some wavelengths of the fluorescence 408 that are the same as or close to the wavelength of the excitation light 402 can be substantially blocked and thus can have a substantially reduced intensity in the 2-D image 200 acquired by the imaging device 380.

[0108] Alternatively or additionally, when the light source 310 or the excitation light 402 has a discrete spectrum, a set of corresponding notch filters or a single multi-notch filter (not shown) can be added to the detection system along the optical axis. The notch filter or multi-north filter can selectively reflect discrete wavelengths or narrow spectral bands of the excitation light 402, thereby blocking the excitation light 402 from reaching the imaging device 380. Similarly, some wavelengths of the fluorescence 408 that are the same or close to the wavelength of the excitation light 402 can be substantially blocked by the notch filter and can therefore have a substantially reduced intensity in the 2-D image 200 acquired by the imaging device 380.

[0109] When the light source 310 or the excitation light 402 has a continuous spectrum, the beam splitter 350 can be a long-pass dichroic beam splitter that reflects at least a portion of the wavelengths of the excitation light 402 and transmits at least a portion of the wavelengths of the fluorescence 408. The spectrum of the excitation light 402 is typically from the ultraviolet light to the visible light spectrum, and the spectrum of the fluorescence 408 is typically from the visible light to the near-infrared light spectrum. Therefore, the long-pass dichroic beam splitter can block the wavelengths of the excitation light 402 and transmit the wavelengths of the fluorescence 408. However, in some cases, the spectrum of the excitation light 402 and the spectrum of the fluorescence 408 can both include short wavelengths to long wavelengths, and they can overlap, for example, in the visible light spectrum. In this case, the long-pass dichroic beam splitter can block at least some of the fluorescence 408 in the visible light spectrum, for example, in applications where the blocked spectrum of the fluorescence 408 contains desired spectral information, which may not be suitable for rejecting the excitation light 402.

[0110] Regardless of the type of spectrum of the light source 310 or the excitation light 402 (whether discrete or continuous), in some embodiments, the polarizer 390a and the beam splitter 350 can be used in combination to block the excitation light 402 from entering the detection system, thereby blocking it from propagating toward the imaging device 380. For example, the beam splitter 350 can be a polarizing beam splitter (PBS) that reflects light whose vibration orientation is aligned with the transmission axis of the polarizer 390a.

[0111] For example, polarizer 390a can be placed at any suitable position along the optical axis to linearly polarize excitation light 402. The PBS can be selected to reflect light having the same vibration orientation as the polarized excitation light and to transmit light having a vibration orientation perpendicular to the polarized excitation light. Thus, most of the excitation light collected by objective lens 360 will be reflected from the PBS and will not reach imaging device 380. In some cases, both the sample and objective lens 360 can depolarize the reflected and / or scattered excitation light to a small extent, thereby undesirably allowing some excitation light to be transmitted through the PBS and into the detection system.

[0112] 2-D imaging devices

[0113] The imaging device 380 may include a suitable 2-D sensor located at an image plane conjugate to the selected focal plane in the sample. The sensor may be implemented as a CMOS sensor, a CCD sensor, a 2-D silicon avalanche photodiode (APD) array, an electron-multiplied CCD (EMCCD), an enhanced CCD, or other suitable types of 2-D sensors.

[0114] The imaging device 380 is operably connected to a controller or a computing device (not shown) that controls its operation. For example, the controller (not shown) may have a processor and one or more computer-readable media storing instructions or operating steps. When executed by the processor, the instructions or operating steps may operate the exposure of the imaging device 380, acquire a 2-D image 200, and / or store a data set of the 2-D image 200 in a memory. The computer-readable medium may also store instructions or operating steps that, when executed by the processor, perform data processing of the acquired 2-D image data set and / or reconstruct a 4-D hyperspectral image data set based on the 2-D image data set.

[0115] The system 300 may advantageously have additional technical features and capabilities to enhance its functionality and performance, as described in detail below.

[0116] Time resolution

[0117] In some embodiments, time-resolving capabilities can be advantageously added to system 300 to allow for fluorescence lifetime imaging (FLIM) or time-resolved fluorescence spectroscopy. For example, a pulsed light source (such as a supercontinuum laser) can be used as light source 310 in conjunction with a 2-D imaging device 380 (such as an intensified CCD camera or an electro-optical streak camera) with picosecond to nanosecond time gating capability. Alternatively, a conventional 2-D CCD or CMOS sensor can be used in combination with an electro-optical shutter. In some embodiments, a modulated electron-multiplied fluorescence lifetime imaging microscope (MEM-FLIM) camera can be used in combination with a modulated light source 310 (e.g., a pulsed light source).

[0118] The lifetime of fluorophores or fluorescent molecules in a sample can be calculated from the acquired time-resolved 2-D image of the fluorescence emission spectrum at each spatial position in the field of view. This adds another dimension of information to the hyperspectral imaging dataset, providing additional information about the fluorophores or fluorescent molecules in the sample.

[0119] Because FLIM excites fluorophores with temporally short excitation pulses, the FLIM capability of system 300 can substantially reject scattered and / or reflected excitation light by discarding signals near zero latency. This can advantageously reduce or minimize the effects of scattered and / or reflected excitation light in the acquired fluorescence signal (e.g., 2-D image 200).

[0120] Fluorescence polarization

[0121] In some embodiments, system 300 can advantageously allow for fluorescence polarization (or anisotropy) measurements to obtain additional information about fluorophores or fluorescent molecules in a sample. The relationship between the polarization of the excitation light and the subsequently detected emitted fluorescence can be used to analyze and study various chemical and / or physical processes of molecules in a sample, such as rotational diffusion, binding interactions, and orientation.

[0122] To increase the ability to measure fluorescence polarization, e.g. Figure 5 As shown, system 300 may include a polarizer 390a and an optical element, such as a wave plate or a polarizer. For example, the optical element may be an achromatic half-wave plate (HWP) 390b. Polarizer 390a may be a linear polarizer located in the illumination system to generate linearly polarized excitation light, for example, vertically polarized light. Individual fluorophores in the sample are preferentially excited by the linearly polarized excitation light, depending on the orientation of their absorption dipoles. Fluorescence emitted from the sample may be partially depolarized due to the random orientation, diffusion, and / or rotation of the fluorophores.

[0123] The beam splitter 350 may be a polarization beam splitter (PBS) that substantially transmits horizontally polarized light and reflects vertically polarized light. Figure 5 As shown, the excitation light vertically polarized by polarizer 390a can be reflected by the PBS and then propagated toward HWP 390b. HWP 390b can be placed in a suitable position, such as before the beam splitter 350 along the optical axis. In this case, for example, HWP 390b can rotate the vibration orientation of both the linearly polarized excitation light and the fluorescence collected from the sample by approximately twice the angle between the vertical axis and the fast axis of the HWP. Rotating HWP 390b around the optical axis will advantageously rotate the vibration orientation of both the excitation light and the fluorescence. Beam splitter 350 can substantially block the polarized excitation light and transmit at least a portion of the polarized fluorescence to be acquired by the imaging device 380.

[0124] Depending on the application, such fluorescence polarization assays can be performed in the steady state or with time-resolved measurements, such as utilizing FLIM capabilities as described above.

[0125] The measurement of fluorescence polarization (or anisotropy) adds another dimension of information to the hyperspectral imaging dataset acquired by the system 300. This additional dimension of information can supplement the other dimensions of the hyperspectral imaging dataset with information about the local chemical and physical environment of the fluorophore-labeled molecules in the sample, such as molecular mass and molecular orientation. The enhanced hyperspectral imaging dataset acquired by the system 300 can further improve the diagnostic accuracy of physiological or pathological changes in the sample.

[0126] Excitation light recycling system

[0127] As described above, modulating the amplitude of the excitation light 402 using the SLM 320a (e.g., DMD or LCD) to generate the excitation pattern 100 results in light loss because most of the excitation light 402 is diverted away from the optical axis and does not reach the sample. Therefore, in some embodiments, the system 300 may advantageously include an excitation light recycling system 500 to improve the efficiency of utilizing the excitation light 402. The recycling system 500 may redirect the excitation light that leaves the optical axis back to the optical axis toward the sample, as described below.

[0128] Reflection-based solution

[0129] In some embodiments, the excitation light recycling system 500 uses Figure 6 The reflection-based solution shown. The recycling system 500 may include one or more lenses and reflectors. For example, the recycling system 500 may include a lens 330c, a plane mirror 510, a first concave mirror 520a, and a second concave mirror 520b. The concave mirror may be replaced by a combination of a plane mirror and a lens.

[0130] The excitation light 402 may be collimated before passing through the lens 330c. The lens 330c may focus the collimated excitation light 402 to a focal point 312a of a reflector 520a, which is located at a given plane 312 that is close to or aligned with the plane of the SLM 320a. The excitation light 402 then expands as it propagates from the focal point 312a to the reflector 520a. The reflector 520a may re-collimate the excitation light 402 and direct it to the SLM 320a.

[0131] As described above, when the SLM 320a is a DMD, a small portion of the excitation light 402 may be reflected by the "on" pixels along the optical axis toward the lens 330a, while the remainder (e.g., off-axis excitation light 403) is reflected by the "off" pixels at different angles and away from the optical axis. The reflector 520b may be configured to intercept the off-axis excitation light 403 and reflect it back to a point 312b very close to the focal point 312a. The spacing between the point 312b and the focal point 312a may be just large enough to allow the edge of the reflector 510 to intercept the returning off-axis excitation light 403 without substantially blocking the original excitation light 402. The reflector 510 may then direct the off-axis excitation light 403 back to the SLM 320a via a path that is nearly aligned with the original path. In this configuration, the excitation light 402 may be incident on the SLM 320a multiple times through multiple reflections between the reflectors, thereby recycling the off-axis excitation light 403 back to the optical axis.

[0132] As stated in this article, Figure 6The three paths shown for recycling of off-axis excitation light 403 are exemplary only. Multiple or infinite recycling paths are possible.

[0133] Some design considerations of the system 300 are discussed below. In some cases, the recycled off-axis excitation light 403 may be slightly divergent. For each recycled path of the off-axis excitation light 403 propagating in the recycling system 500, because the off-axis excitation light 403 does not return to the focal point 312a, the off-axis excitation light 403 will have a slightly different angle than the original excitation light 402 when it arrives at the SLM 320a. The angular difference (or divergent angle) can be defined as Δθ=Δx / f, where "Δx" is the spacing between the focal point 312a and the point 312b, and "f" is the focal length of the mirror 520a (or lens) used to re-collimate the off-axis excitation light reflected by the mirror 510. Δx can be at least greater than any unusable rough edge of the mirror 510 and greater than the diffraction-limited spot size of the excitation light 402. Depending on the values ​​of Δx and f, Δθ may be less than 1 degree. This small degree of angular difference (or divergent angle) may not affect the formation of the excitation pattern 100.

[0134] In some cases, when SLM 320a is a DMD, the DMD may have a diffraction effect on the reflected excitation light 404. For example, a single micromirror of the DMD may have a side length of about 10 μm. A typical divergence angle of the reflected excitation light 404 caused by diffraction of the micromirror array may be about λ a / 10μm, where λ a is the wavelength of the excitation light 402. Therefore, the divergence angle may be approximately less than one radian, such as 1 / 20 radian, or less than a few degrees, such as 3 degrees. Therefore, most of the excitation light 404 reflected by the "on" pixels or micromirrors of the DMD from different recycling paths in the recycling system 500 may overlap and propagate along the optical axis, thus not affecting the formation of the excitation pattern 100.

[0135] In some cases, the reflected excitation light 404 from different recycling paths in the recycling system 500 may exhibit optical interference. For light sources 310 with discrete wavelengths or narrow spectral bands, this interference may cause the reflected excitation light 404 to have unstable intensity when focused on the sample. Additional optical components can be added to control the relative phases of the excitation light 403 propagating in different recycling paths to reduce the optical interference effect. However, this may complicate the design of the system 300. Therefore, Figure 6The illustrated reflection-based scheme for the recycling system 500 may be more suitable for systems 300 with a light source 310 having a broadband spectrum, such as a white light source. For such a system 300, the effects of interference may impose very rapid small oscillations on the spectrum of the reflected excitation light 404. Fluorophores typically have spectrally broad absorption features, allowing these oscillations to average out during excitation. Therefore, when the light source 310 has a broadband spectrum, optical interference effects may have little effect on the acquired fluorescence emission spectrum.

[0136] Polarization-based solutions

[0137] In order to solve the above technical problems for recycling the excitation light 402 having discrete wavelengths or narrow spectral bands, in some embodiments, the excitation light recycling system 500 can use Figure 7 The polarization-based scheme shown.

[0138] like Figure 7 As shown, the polarization-based recycling system 500 may include one or more optical components. For example, the recycling system 500 may include an optical isolator 530, a polarization beam splitter (PBS) 540, a quarter-wave plate (quarter-waveplate, QWP) 550, and one or more mirrors, such as a first mirror 510a and a second mirror 510b. In some embodiments, the optical isolator 530 may include a linear polarizer, or may be optionally replaced by a linear polarizer.

[0139] In this scheme, the optical isolator 530 allows the excitation light 402 to propagate in only one forward direction. The excitation light 402 may be linearly polarized light, or may become linearly polarized light after passing through the optical isolator 530. The linearly polarized excitation light after passing through the optical isolator 530 is referred to as the excitation light 420. The PBS 540 may be configured to transmit light having a vibration orientation parallel to the vibration orientation of the excitation light 420, and to reflect light having a vibration orientation orthogonal to the vibration orientation of the excitation light 420. For example, the excitation light 420 may be horizontally polarized or have a vibration orientation in a horizontal direction. The PBS 540 may transmit horizontally polarized light and reflect vertically polarized light. Therefore, the excitation light 420 is transmitted through the PBS 540 and propagates toward the SLM 320a.

[0140] The following description of the polarization-based approach of the recycling system 500 uses horizontally polarized excitation light 420 as an example. Embodiments of the polarization-based approach of the recycling system 500 are equally applicable to linearly polarized excitation light 420 having any vibrational orientation.

[0141] As described above, when the SLM 320a is a DMD, a small portion of the excitation light 420 can be reflected by the "on" micromirror of the DMD along the optical axis toward the lens 330a, while the off-axis excitation light 403 reflected by the "off" pixel is turned away from the optical axis. The reflector 510a can be configured to intercept the off-axis excitation light 403 and reflect it back to the "off" pixel on the DMD. The off-axis excitation light 403 can pass through the QWP 550, the first time to propagate toward the reflector 510a, and the second time to be guided back to the DMD by the reflector 510a, which rotates the vibration orientation of the off-axis excitation light 403 by 90°. For example, horizontally polarized excitation light 403 can be changed to vertical polarization after passing through the QWP 550 twice. The vertically polarized excitation light is then reflected by the "off" micromirror of the DMD to the PBS 540.

[0142] Because the vertically polarized excitation light reflected to PBS 540 has a vibration orientation orthogonal to the vibration orientation of the horizontally polarized excitation light 420, it is reflected by PBS 540 and directed to reflector 510b. Without changing its vibration orientation, reflector 510b and PBS 540 then reflect the vertically polarized excitation light back to the DMD, where the "on" micromirror then reflects the vertically polarized excitation light along the optical axis toward lens 330a. The "off" micromirror reflects the vertically polarized excitation light, which is transmitted through QWP 550 and reflector 510a twice again and becomes horizontally polarized. This horizontally polarized excitation light will pass through PBS 540, but will not propagate back to light source 310 due to optical isolator 530.

[0143] In the polarization-based scheme of the recycling system 500 described above, because the QWP 550 rotates the vibration orientation of the off-axis excitation light 403 by 90°, the excitation light 404 reflected toward the optical axis, including the portion of the off-axis excitation light 403 that is recycled, will have an orthogonal polarization. In this case, the beam splitter 350 may appropriately be a multi-band dichroic beam splitter with multiple cut-off wavelengths and passbands, rather than a polarization beam splitter. As described above, the multi-band dichroic beam splitter may be selected so that wavelengths of the excitation light 402 having a discrete spectrum are substantially reflected and wavelengths of the emitted fluorescent light 408 are substantially transmitted. Therefore, this polarization-based scheme may work better in a system 300 using a light source 310 having a discrete wavelength or narrow spectral band, such as a combination of a group of lasers operating at discrete wavelengths.

[0144] Confocal optical segmentation capability

[0145] As described above, system 300 can allow confocal optical sectioning, which allows the depth of the focal plane in the sample to be selected. The depth of the focal plane can be selected by introducing one or more optical pinholes in a plane conjugate to the selected focal plane.

[0146] Figure 8 is a schematic representation of an exemplary system 300 that allows confocal optical segmentation. Figure 8 As shown, system 300 may include a first SLM 320a for generating the excitation pattern 100 as described above, a second SLM 320b for confocal optical segmentation, at least one additional mirror 510c, one or more tube lenses (e.g., 330d and 330e), and a z-axis translation stage or tunable liquid lens (not shown). SLM 320b may be of similar type and features as described above for SLM 320a. For example, the pixels of SLM 320b may be individually modulated in the same manner as described for SLM 320a.

[0147] SLM 320b can be placed near a plane conjugate to the focal plane at a desired depth in the sample along the optical axis. For example, lens 330b and objective lens 360 can form an imaging configuration. Figure 8 As shown, lens 330b can be located behind objective lens 360, and SLM 320b can be located about one focal length behind lens 330b. The space between the rear aperture of objective lens 360 and lens 330b is a collimation space, which can be adjusted as needed based on various factors (such as the geometry of system 300 and the desired position of the minimum beam aperture). In some embodiments, lens 330b is placed about one focal length behind objective lens 360.

[0148] The pixels of the SLM 320b can be selectively actuated or switched to an "on" or "off" state to form a pinhole pattern on the sample that matches or is conjugate to the excitation pattern 100. The pinhole pattern can include multiple artificial optical pinholes at conjugate planes and reject out-of-focus fluorescence from the sample. Thus, the out-of-focus fluorescence does not pass through the detection system and is substantially removed or eliminated from the acquired 2-D image 200.

[0149] The size and spacing of the artificial pinholes in the pinhole pattern are programmable and can be customized based on the magnification of the imaging configuration formed by the objective lens 360 and the lens 330b. In some cases, the pinhole pattern can include a plurality of "on" pixels of elongated shape to allow simultaneous acquisition of fluorescence emitted from multiple locations on the sample (e.g., areas excited by the excitation spots 112a-112f). In other cases, the pinhole pattern can include an array of "on" pixels that match the size of the excitation lines or excitation spots in the excitation pattern 100.

[0150] The fluorescence 412 reflected by the "on" pixels of SLM 320b is then imaged by tube lenses 330d and 330e to imaging device 380. For example, mirror 510c can be placed at an appropriate position along the optical axis and used to direct the fluorescence 412 reflected by the "on" pixels to the tube lens. Tube lens 330d can be located approximately one focal length outside the image produced by lens 330b (e.g., approximately one focal length behind SLM 320b) so that it re-collimates the fluorescence from the sample. Imaging device 380 can be located approximately one focal length behind tube lens 330e, or at a conjugate plane of SLM 320b. Because the fluorescence is collimated in the space between tube lenses 330d and 330e, the distance between tube lenses 330d and 330e can be adjusted as needed. In some embodiments, tube lens 330e may be approximately two focal lengths behind tube lens 330d such that a plane intermediate tube lenses 330d and 330e is conjugate with the exit pupil of system 300 .

[0151] By digitally changing and / or laterally shifting the excitation pattern 100 and the matching pinhole pattern on the SLM 320b accordingly, the entire field of view can be scanned to acquire a confocal imaging dataset. By further scanning the field of view of the entire sample, the entire sample can be scanned to acquire a complete confocal imaging dataset of the sample.

[0152] In some embodiments, imaging device 380 may be appropriately tilted to reduce aberrations, thereby improving the quality of the acquired 2-D image data set. This is at least because the "on" pixels of SLM 320b direct fluorescent light 412 at an angle that is not perpendicular to the surface plane of SLM 320b, so that the image plane formed by tube lenses 330d and 330e may be tilted. Aberrations caused by this tilting effect may be compensated by appropriately tilting imaging device 380. Aberrations may be further reduced if the dispersion angle of dispersive element 340b is adjusted to be parallel to the rotation axis of tilted imaging device 380.

[0153] In order to change or select the depth of the focal plane, in some embodiments, the sample holder 370 can be mounted on a z-axis translation stage. The desired depth of the focal plane can be selected by moving the sample holder 370 along the optical axis using the z-axis translation stage. Alternatively, the objective lens 360 can be mounted on a z-axis translation stage, and the desired depth of the focal plane can be selected by moving the objective lens 360 along the optical axis. As described herein, the z-axis translation stage can also include xy translation capabilities to span the field of view of the sample movement system 300 in the lateral direction. In other embodiments, the desired depth of the focal plane can be selected by tuning the focus of a tunable liquid lens (not shown) placed behind the objective lens 360. In addition, the z-translation stage or the tunable liquid lens can be controlled by a computer program to achieve automatic focusing.

[0154] Advantageously, the degree of parfocality can be adjusted as desired by changing the size and / or spacing of the artificial pinholes formed by the SLM 320b. For example, increasing the size of the pinhole by increasing the number of pixels in the pinhole and / or decreasing the pinhole spacing can reduce the degree of parfocality, thereby reducing the degree of depth selectivity for the desired focal plane. On the other hand, reducing the size of the pinhole by decreasing the number of pixels in the pinhole and / or increasing the pinhole spacing can increase the degree of parfocality, thereby increasing the degree of depth selectivity for the desired focal plane. In some embodiments, the depth selectivity can be proportional to the ratio of the number of "off" and "on" pixels of the SLM 320b. Therefore, the SLM 320b can advantageously allow, as desired, switching between wide field of view and confocal imaging by conveniently adjusting the pinhole size and / or spacing.

[0155] Additionally, the pinhole pattern formed by the pixels of SLM 320b advantageously allows confocal imaging of multiple regions on a sample simultaneously illuminated by excitation pattern 100. This can increase the speed and / or throughput of acquiring a hyperspectral imaging dataset on a sample at a desired focal plane compared to conventional confocal microscopy using sequential point-by-point scanning.

[0156] like Figure 8 As shown, in an embodiment of the system 300 using the SLM 320b, the dispersive element 340b can be located in the collimation space between the tube lenses 330d and 330e. Because the pinhole pattern on the SLM 320b matches the excitation pattern 100 on the sample, as described above, the fluorescence 412 reflected by the artificial pinhole of the SLM 320b can be dispersed by the dispersive element 340b, so that the fluorescence emission spectrum corresponding to the excitation point of the excitation pattern 100 can be acquired by the 2-D sensor of the imaging device 380.

[0157] Selective filtering of fluorescence emission spectra

[0158] In some applications, different fluorophores with spaced-apart fluorescence emission spectra, such as green and red fluorophores, may be used or present in the sample. This may result in lateral gaps along the emission wavelength axis in the fluorescence emission spectra acquired in the 2-D image 200, resulting in inefficient use of space on the 2-D sensor of the imaging device 380.

[0159] In other applications, combinations of different fluorophores may result in an overall broad fluorescence emission spectrum being obtained by the imaging device 380. In some cases, multiple spectral regions within the broad emission fluorescence spectrum may be more useful than others. Acquiring the full broad fluorescence emission spectrum may result in inefficient use of space on the 2-D sensor of the imaging device 380 and further reduce the throughput of acquiring a hyperspectral imaging dataset.

[0160] To improve the efficiency of the sensor space using the imaging device 380 and increase the throughput of the system 300, the spectral segmentation system 342 may be included in the detection system at the collimated space along the optical axis. Figure 8 As shown, the spectral segmentation system 342 can be located between the tube lenses 330d and 330e and can be placed before the dispersive element 340b in the detection system. The spectral segmentation system 342 can selectively pass one or more spectral bands having tunable bandwidths and / or center wavelengths, thereby allowing the acquisition of a hyperspectral imaging data set having a desired spectral band and / or a desired spectral resolution.

[0161] like Figure 8 As shown, the spectral segmentation system 342 may include a plurality of spectral segmentation modules 344. The fluorescence 408 emitted by the sample may enter the spectral segmentation system 342 after being collimated or re-collimated. Using one or more beam splitters and / or motorized flip mirrors, the spectral segmentation system 342 may split the input collimated fluorescence beam into one or more beams, each having a different spectral band, and guide them through the spectral segmentation modules 344, respectively. Each spectral segmentation module 344 may filter one of the beams to have a desired bandwidth and central wavelength. After filtering, the spectral segmentation system 342 may combine the filtered beams into an output beam using one or more beam splitters and / or motorized flip mirrors.

[0162] Spectral segmentation modules 344 can each work as a tunable bandpass filter with a tunable passband width and / or a tunable center wavelength. For example, spectral segmentation modules 344 can include a longpass filter and a shortpass filter along its optical axis. At least one of the longpass filter and the shortpass filter can be rotated relative to the optical axis. The rotating filter can adjust the incident angle of the light beam on the filter, thereby shifting the wavelength of their absorption or reflection edge. Therefore, rotating the longpass filter and / or the shortpass filter can tune the bandwidth and / or the center wavelength of the spectral passband formed by the longpass filter and the shortpass filter. Or, the spectral segmentation modules 344 can each include a tunable bandpass filter, whose passband can be tuned by rotating the filter, thereby tuning the incident angle of the light beam on the filter.

[0163] Spectral segmentation system 342 allows the measured fluorescence emission spectrum to be adjustably filtered to a desired spectral range useful for a particular application. By selecting the desired spectral range, the space on the 2-D sensor of imaging device 380 can be used more efficiently. For example, as described above, the degree of dispersion caused by dispersive element 340b is adjustable. The spectral resolution of a selected spectral range of the fluorescence emission spectrum can be increased by increasing the degree of spectral dispersion using dispersive element 340b, thereby providing more information about the fluorophores or fluorescent molecules in the sample.

[0164] Additionally, selecting a desired spectral range can allow for a reduction in the lateral spacing between fluorescence emission spectra along the emission wavelength axis in the 2-D image 200, thereby increasing the throughput of data set acquisition. For example, by reducing the period of the excitation pattern 100 in the horizontal direction and reducing the degree of spectral dispersion using the dispersive element 340b, the period of the array of fluorescence emission spectra in the horizontal direction in the 2-D image 200 can be reduced. This in turn can increase the number of fluorescence emission spectra that can be acquired in one exposure, thereby increasing the efficiency of using the sensor space of the imaging device 380.

[0165] Alternative configuration

[0166] In some applications, it may be desirable to have a more compact configuration of system 300. In such cases, system 300 may use a diffractive element instead of SLM 320a and / or SLM 320b. Figure 9-13 An embodiment of such a configuration of system 300 is described.

[0167] Fig. 9 is a schematic representation of an exemplary compact embodiment of system 300. Fig. 9 As shown, system 300 can advantageously use transmissive illumination to simplify its geometry. However, depending on the application, other methods such as Figure 3-Figure 8 The reflective illumination configuration shown. In the illumination system, the excitation light 402 from the light source 310 (such as a supercontinuum laser source provided by an optical fiber) is collimated by the lens 330a, transmitted through the first diffraction element 600a, and then illuminates the sample placed on the sample holder 370. The diffraction element 600a modulates the phase of the excitation light 402 transmitted through it, and structures the excitation light 402 to generate the excitation pattern 100. The phase modulation can present multiple wavelets of the transmitted excitation light 430 with different directions and / or phases, generating a diffraction pattern in the far field. The diffraction pattern focused on the sample is called the excitation pattern 100.

[0168] Fig.12 is a schematic representation of an exemplary diffractive element 600a. Fig.12 As shown, in some embodiments, the diffraction element 600a can be a 2-D array of diffraction lenses 610. For excitation light 402 having a single wavelength, the diffraction element 600a generates a 2-D array of excitation points, with each diffraction lens 610 generating one excitation point. For excitation light 402 having multiple discrete wavelengths or wavelength ranges, excitation light 402 of different wavelengths is diffracted by each diffraction lens 610 into a plurality of light beams propagating along different angular directions. Therefore, when focused on the sample, the excitation light 402 of different wavelengths can have focal points that are spatially displaced from each other in a first lateral direction (e.g., a vertical direction), thereby generating a 2-D array of excitation points. Figure 1 or Figure 2An excitation pattern 100 is shown.

[0169] In some embodiments, the diffractive lens 610 of the diffractive element 600a can be a zone plate with transparent and opaque spectral bands, a conventional grating made by, for example, binary lithography, grayscale lithography, or a molding process, or a sub-wavelength grating made by binary lithography. In other embodiments, the diffractive element 600a can be replaced by a 2-D lenslet array and a transmission diffraction grating having a phase modulation capability for generating the excitation pattern 100 as described above.

[0170] In the detection system, the fluorescence 408 emitted by the sample is collected and collimated by the objective lens 360, transmitted through the dispersive element 340b, and then focused by the lens 330b onto the imaging device 380. The dispersive element 340b can spectrally disperse the fluorescence 408 in the second lateral direction (e.g., horizontal direction) as described above. The dispersive element 340b can have the same features and functions as described above.

[0171] In some embodiments, system 300 may include a second linear polarizer 390c. Fluorescent light 408 may pass through polarizer 390c. When excitation light 402 is linearly polarized, polarizer 390c may be used to substantially reflect the polarized excitation light, thereby blocking it from reaching imaging device 380. In other embodiments, a set of notch filters or a single multi-notch filter (not shown) may be added to the detection system along the optical axis.

[0172] Because the diffractive element 600a does not have the same digital programmability as an SLM, the diffractive element 600a or the sample holder 370 can be translated in the spatial dimension to scan the excitation pattern 100 over the field of view or sample to obtain a complete 4-D hyperspectral imaging data set. The scanning scheme can be the same as described above with reference to Figure 1 and Figure 2 The same as described. Different areas in each scanning unit 110 can be illuminated by spatially shifting the excitation pattern 100 in the vertical and horizontal directions. At each spatial position of the excitation pattern 100, at least one 2-D image 200 of the fluorescence emission spectrum of the illuminated area can be acquired. Then, multiple 2-D images 200 of the fluorescence emission spectrum can be acquired corresponding to a series of excitation patterns 100 that are laterally shifted from each other and used to reconstruct a 4-D hyperspectral imaging dataset.

[0173] Fig.10 is a schematic diagram representation of another exemplary compact embodiment of system 300. Fig.10 The system 300 shown may allow for the acquisition of a 4-D hyperspectral imaging dataset by performing a scan in one lateral direction. For example, the system 300 may include a diffractive element 600a in the detection system and another diffractive element 600b in the illumination system. Fig.10 As shown, in the illumination system, the excitation light 402 from the light source 310 is collimated by the lens 330 a , transmitted through the diffraction element 600 b , and then illuminates the sample placed on the sample holder 370 .

[0174] Fig.13 is a schematic representation of an exemplary diffractive element 600b. The diffractive element 600b can modulate the phase of the excitation light 402 transmitted through it and present wavelets of the transmitted excitation light 430 with different directions and / or phases. In some embodiments, the diffractive element 600b may include a linear array of diffractive cylindrical lenslets 620. For excitation light 402 of a single wavelength, the diffractive element 600b generates a repeating pattern of monochromatic stripes, with each cylindrical lenslet 620 generating one monochromatic stripe. For excitation light 402 having multiple discrete wavelengths or wavelength ranges, excitation light 402 of different wavelengths is diffracted by each cylindrical lenslet 620 into several beams propagating along different angular directions. Therefore, when focused on the sample, the excitation light 402 of different wavelengths may have focal points that are spatially displaced from each other in a first lateral direction (e.g., a vertical direction), generating a series of shifted repeating patterns of stripes of different colors. Depending on the spectrum of the light source 310, the stripes of different colors may be connected or separated in the first lateral direction. The repeating pattern of the shifted stripes of different colors is then illuminated on the sample.

[0175] In the detection system, instead of using the dispersive element 340b, the diffractive element 600a can be added and placed in front of the imaging device 380. The fluorescent light 408 emitted by the sample is collected and collimated by the objective lens 360, transmitted through the polarizer 390c, and then imaged onto the diffractive element 600a through the lens 330b. The diffractive lens 610 of the diffractive element 600a can then spectrally disperse the fluorescent light in a second lateral direction (e.g., horizontal direction) and image the spectrally dispersed fluorescent light 410 onto the 2-D sensor of the imaging device 380.

[0176] In some embodiments, the focal length of lens 330b is selected so that the diffraction limited spot size of lens 330b at its focal plane can cover multiple pixels of the 2-D sensor of imaging device 380. This may affect the numerical aperture (NA), focal ratio (f-ratio), and / or magnification of lens 330b. For example, to increase the diffraction limited spot size of lens 330b, lens 330b can have a longer focal length, a smaller NA or a larger f-ratio, and / or a larger magnification.

[0177] Diffractive element 600a may be designed or selected so that the diameter of its diffractive lenses 610 is approximately the size of the diffraction limited spot of lens 330b. Different wavelengths of fluorescent light 410 deflected and focused by each diffractive lens 610 may have focal points that are spatially displaced from each other in the second lateral direction, generating a pattern such as Figure 1 or Figure 2 Arrays of fluorescence emission spectra are shown.

[0178] like Fig.10 The embodiment of the system 300 shown allows for the identification of areas or locations on a sample that are illuminated by a repeating pattern of a series of laterally shifted stripes of different colors. Figure 1 or Figure 2 The fluorescence emission spectrum is obtained in the 2-D image 200 shown. To obtain the excitation spectrum, the repeating pattern can be scanned along the first lateral direction so that the area or position on the sample that was previously illuminated by the color stripes of the repeating pattern is illuminated by stripes of different colors. The shifting of the repeating pattern in the first lateral direction and the subsequent acquisition of the corresponding 2-D image 200 can be performed multiple times. In this case, the fluorescence emission spectrum corresponding to the excitation wavelength of each area or position on the sample can be obtained and used to reconstruct the 4-D hyperspectral imaging dataset.

[0179] In such Fig.10 In the embodiment of the system 300 shown, because the repeating pattern of a series of laterally shifted stripes of different colors is continuous in the second lateral direction, it may only be necessary to scan the repeating pattern along the first lateral direction to obtain the excitation spectra of all regions or positions within the field of view. This can further improve the throughput and efficiency of the system 300 for acquiring 4-D hyperspectral imaging data sets.

[0180] Along the second lateral direction, each area illuminated by the continuous color stripes can be imaged to the diffractive lens 610, which then disperses the fluorescence and focuses it to the imaging device 380. In this case, the spatial resolution along the second lateral direction can depend on the size and focal length of the diffractive lens 610, the focal lengths of the lens 330b and the objective lens 360, and / or the size of the 2-D sensor of the imaging device 380. In some embodiments, increasing the focal length of the lens 330b can allow the use of a larger diffractive lens 610. The spectral resolution along the second lateral direction can depend on the width and / or focal length of the diffractive lens 610, and the off-axis focal shift generated by the diffractive lens 610 in the second lateral direction. For example, increasing the groove density of the diffractive lens 610 will increase the diffraction angle of the fluorescence, thereby increasing the off-axis focal shift, thereby increasing the spectral resolution in the second lateral direction.

[0181] Fig.11FIG. 4 is a schematic diagram of another exemplary compact embodiment of a system 300 that provides the capability to measure fluorescence polarization. Fig.11 As shown, the system 300 may include two polarizers 390a and 390c. The polarizer 390a may be located at an appropriate position along the optical axis in the illumination system to generate linearly polarized excitation light. The polarizer 390c may be located at an appropriate position along the optical axis in the detection system to transmit the emitted fluorescence 408 having a given vibration orientation. In order to perform fluorescence polarization measurement, the transmission axis of the polarizer 390c may be rotated between orientations parallel to and orthogonal to the vibration orientation of the linearly polarized excitation light. The imaging device 380 may acquire a 2-D image 200 of the fluorescence emission spectrum of the fluorescence 408 having a vibration orientation parallel to and orthogonal to the vibration orientation of the polarized excitation light, respectively. The acquired 2-D image 200 may then be used for fluorescence polarization (or anisotropy) measurement.

[0182] The system 300 described herein may be used in various hyperspectral imaging methods. Fig.14 is a flow chart of an exemplary method 700 for performing hyperspectral imaging or for acquiring a hyperspectral imaging dataset of a sample. The method 700 uses the system 300 and the above referenced Figure 3-Figure 13 Features of an embodiment of system 300 are described.

[0183] In step 702, a light source 310 having a discrete spectrum or a continuous spectrum is provided and configured to emit an excitation light 402 having one or more wavelengths. In step 704, the excitation light 402 is structured by the SLM 320a into a predetermined two-dimensional pattern at a conjugate plane of the focal plane in the sample. In step 706, the structured excitation light (e.g., the excitation light 404 reflected by the SLM 320a) is spectrally dispersed in a first lateral direction by the dispersive element 340a. In step 708, the spectrally dispersed excitation light 406 is directed to the sample and focused on the sample, illuminating the sample with the excitation pattern 100 having one or more wavelengths dispersed in the first lateral direction. In step 710, the fluorescence 408 collected from the sample is spectrally dispersed in a second lateral direction by the dispersive element 340b. In step 712, the spectrally dispersed fluorescence 410 is imaged to the 2-D sensor of the imaging device 380.

[0184] The method 700 may further include additional steps. For example, the method 700 may include calibrating the system 300 prior to acquiring the 2-D image 200. The various optical components in the system 300 may be properly calibrated and aligned so that a focused 2-D image 200 with reduced or minimal distortion may be acquired.

[0185] The method 700 may further include polarizing the excitation light 402 to be directed to the sample using a first polarizer, and substantially reflecting light collected from the sample having the same polarization as the polarized excitation light using a second polarizer or polarizing beam splitter (PBS).

[0186] The method 700 may further include sequentially illuminating the sample in a series of excitation patterns 100 that are laterally displaced from one another, obtaining a plurality of 2-D images 200 of spectrally dispersed emission light corresponding to the series of excitation patterns 100, and reconstructing the plurality of 2-D images 200 to provide a 4-D hyperspectral imaging dataset. As described above, each 2-D image 200 records an array of fluorescence emission spectra corresponding to each laterally displaced excitation pattern 100.

[0187] The method 700 may further include providing a programmable artificial optical pinhole at a plane conjugate to the focal plane through the SLM 320b, forming a series of pinhole patterns through the pixels of the SLM 320b, and matching the series of pinhole patterns with the series of excitation patterns 100. As described above, the light collected from the SLM 320b is imaged to the imaging device 380 using one or more lenses. A 2-D image 200 of spectrally dispersed emission light can be acquired after each lateral shift of the excitation pattern 100 and the formation of a pinhole pattern matching it. The method 700 may further include reconstructing the 2-D image 200 corresponding to the series of excitation patterns 100 to provide a 4-D hyperspectral imaging data set of a selected focal plane of the sample.

[0188] The foregoing description is presented for illustrative purposes. It is not exhaustive, nor is it limited to the precise forms or embodiments disclosed. Modifications and adaptations of the embodiments will be apparent from consideration of the specification and practice of the disclosed embodiments. For example, the described implementations include hardware and software, but systems and methods consistent with the present disclosure may be implemented as hardware alone. Furthermore, although certain components have been described as being coupled to each other, these components may be integrated with each other or distributed in any suitable manner.

[0189] In addition, although illustrative embodiments have been described herein, the scope includes any and all embodiments having equivalent elements, modifications, omissions, combinations (e.g., aspects of the various embodiments), modifications, and / or variations based on the present disclosure. The elements in the claims are to be interpreted broadly based on the language used in the claims and are not limited to the examples described in this specification or the examples described in the implementation of the application, which are to be interpreted as non-exclusive. In addition, the steps of the disclosed methods may be modified in any way, including reordering steps and / or inserting or deleting steps.

[0190] The instructions or operating steps stored by the computer readable medium can be in the form of a computer program, program module or code. As described herein, computer programs, program modules and codes (such as those used by the controller) based on the written description of this specification are easily within the purview of software developers. Various programming techniques can be used to create computer programs, program modules or codes. For example, Java, C, C++, assembly language or any such programming language can be used to design these computer programs, program modules or codes. One or more such programs, modules or codes can be integrated into a device system or existing communication software. Programs, modules or codes can also be implemented or copied as firmware or circuit logic.

[0191] According to the detailed description, the features and advantages of the present disclosure are obvious, and therefore, the appended claims are intended to cover all systems and methods that fall within the true spirit and scope of the present disclosure. As used herein, the indefinite articles "one" and "an" mean "one or more". Similarly, the use of plural terms does not necessarily mean plural unless it is clear in a given context. Unless otherwise specifically indicated, words such as "and" or "or" mean "and / or". In addition, since many modifications and changes will be prone to occur according to the study of the present disclosure, it is not desired to limit the present disclosure to the exact structure and operation shown and described, and accordingly, all suitable modifications and equivalents can be resorted to, and these modifications and equivalents all fall within the scope of the present disclosure.

[0192] Other embodiments will be apparent from consideration of the specification and practice of the embodiments disclosed herein. It is intended that the specification and embodiments be considered as exemplary only, with the true scope and spirit of the disclosed embodiments being indicated by the following claims.

Claims

1. A hyperspectral imaging system, comprising: a sample holder configured to hold a sample; a light source configured to emit excitation light having one or more wavelengths; Two-dimensional 2D imaging array; and Optical system, including: a first spatial light modulator SLM and a first dispersive element, or a first diffractive element; and a second dispersive element or a second diffractive element; The optical system is configured to (i) use the first SLM to construct the excitation light into a two-dimensional excitation pattern at a focal plane in the sample or a conjugate plane of the focal plane and use the first dispersive element to spectrally disperse the two-dimensional excitation pattern; (ii) illuminate the sample with an excitation pattern having one or more wavelengths dispersed along a first lateral direction in the focal plane, or illuminate the sample with an excitation pattern constructed from the excitation light by the first diffraction element; (iii) collect emission light from the sample; (iv) spectrally disperse the collected emission light using the second dispersive element or the second diffraction element; and (v) focus the collected emission light and provide it to the imaging array so that the collected emission light is spectrally dispersed along a second lateral direction, wherein the first lateral direction is different from the second lateral direction, Wherein, the first SLM or the first diffraction element is configured to modulate the phase of the excitation light.

2. The hyperspectral imaging system according to claim 1, wherein: The first diffractive element comprises a two-dimensional array of diffractive lenses.

3. The hyperspectral imaging system according to claim 2, wherein: Each of the diffractive lenses includes a zone sheet having transparent zones and opaque zones.

4. The hyperspectral imaging system according to claim 2, wherein: Each of the diffractive lenses includes a grating.

5. The hyperspectral imaging system according to claim 4, wherein: The grating is a sub-wavelength grating.

6. The hyperspectral imaging system according to claim 1, wherein: The first diffraction element includes a two-dimensional lenslet array and a transmission diffraction grating.

7. The hyperspectral imaging system according to claim 1, wherein: The first diffractive element comprises a linear array of diffractive cylindrical lenslets.

8. The hyperspectral imaging system according to claim 1, wherein: The second diffractive element comprises a two-dimensional array of diffractive lenses.

9. The hyperspectral imaging system according to claim 8, wherein: The diameter of each of the diffractive lenses is the diameter of the diffraction limited spot of the lens that images the emitted light onto the imaging array.

10. The hyperspectral imaging system according to claim 1, wherein: The diffraction limited spot of the lens that images the emitted light onto the imaging array covers a plurality of pixels of the imaging array.

11. The hyperspectral imaging system according to claim 1, wherein: The optical system further includes a first polarizer configured to polarize the excitation light.

12. The hyperspectral imaging system according to claim 11, wherein: The optical system further includes at least one of a second polarizer or a polarization beam splitter PBS configured to reflect light having the same polarization as the polarized excitation light.

13. The hyperspectral imaging system according to claim 11, wherein: The optical system also includes a second polarizer having a transmission axis rotatable relative to the vibrational orientation of the polarized excitation light.

14. A method for hyperspectral imaging, the method comprising: providing excitation light having one or more wavelengths from a light source; structuring the excitation light from the light source into a predetermined two-dimensional excitation pattern at a focal plane or a conjugate plane of the focal plane in the sample by a first spatial light modulator SLM; Spectrally dispersed excitation pattern; illuminating the sample in the focal plane with an excitation pattern having one or more wavelengths dispersed along a first lateral direction, or illuminating the sample with an excitation pattern constructed from the excitation light by a first diffractive element, wherein the first SLM or the first diffractive element is configured to modulate the phase of the excitation light; spectrally dispersing emission light collected from the sample; and The spectrally dispersed emission light is imaged using a two-dimensional (2D) imaging array, wherein the spectrally dispersed emission light is received by the imaging array in a focused manner such that the spectrally dispersed emission light is spectrally dispersed along a second lateral direction, wherein the first lateral direction is different from the second lateral direction.

15. The method according to claim 14, further comprising: polarizing the excitation light by a first polarizer; and Light having the same polarization as that of the polarized excitation light is reflected by at least one of the second polarizer or the polarization beam splitter PBS.

16. The method according to claim 14, wherein: The first diffraction element comprises: A two-dimensional array of diffractive lenses, 2D lenslet arrays and transmission diffraction gratings, or Linear array of diffractive cylindrical lenslets.

17. The method according to claim 14, further comprising: The phase of the excitation light is modulated by the first diffraction element.

18. The method according to claim 14, further comprising: sequentially illuminating the sample with a series of excitation patterns that are laterally displaced from one another in a focal plane within the sample; obtaining a plurality of two-dimensional images of the spectrally dispersed emission light using the imaging array, wherein each of the obtained two-dimensional images corresponds to a respective one of the series of excitation patterns; as well as The acquired multiple two-dimensional images are reconstructed to provide a four-dimensional hyperspectral imaging dataset.

19. The method according to claim 14, wherein: The emitted light is spectrally dispersed by a second diffractive element comprising a two-dimensional array of diffractive lenses.

Images