Photothermal infrared spectroscopy with autofluorescence enhancement

Abstract

The methods and systems described herein detect the autofluorescence of a sample. These methods and systems eliminate the need to add a fluorophore to the sample to create IR absorption. An IR source and an excitation source are used to generate an emission spectrum of a selected sample, which is then analyzed to determine various data about the sample.

Description

[0001] Cross-reference to related applications

[0002] This application was filed on September 28, 2023 as a PCT international patent application and claims priority and benefit to U.S. Patent Application No. 17 / 937,447, filed on October 1, 2022, which is incorporated herein by reference in its entirety. Technical Field

[0003] The embodiments disclosed herein generally relate to the use of optical systems, such as the study or analysis of materials using infrared, visible, or ultraviolet light. The embodiments described herein relate to imaging and spectroscopy, and more specifically, to enhancements to photothermal imaging and spectroscopy systems and techniques for acquiring spectral information (e.g., information related to infrared (IR) absorption spectra combined with simultaneously juxtaposed fluorescence imaging) indicating the optical properties and / or material or chemical composition of a sample. Background Technology

[0004] Infrared (IR) spectroscopy is a powerful technique for the chemical characterization and analysis of materials, including the mapping and identification of chemical species in complex environments such as biological materials. IR spectroscopy works by illuminating a sample with a beam of infrared radiation and then measuring the amount of light absorbed, transmitted, reflected, and / or scattered from the sample. The frequencies of infrared light, particularly mid-infrared light (wavelengths from 2.5 μm to 20 μm), correspond to the vibrational frequencies in molecular bonds. Therefore, when a sample is irradiated with mid-IR light, it will absorb light at IR radiation frequencies corresponding to the specific molecular vibrations of the chemical species in the sample. By measuring the change in the sample's absorption of IR light as a function of IR frequency (i.e., the IR absorption spectrum), the pattern of the absorption peaks provides a "fingerprint" that can be used to characterize and / or identify the chemical species in the sample.

[0005] Optical photothermal infrared (OPTIR) spectroscopy is an emerging field that uses infrared spectroscopy with spatial resolution ten times or more finer than conventional Fourier transform infrared (FTIR) spectroscopy to provide chemical analysis. OPTIR achieves higher spatial resolution than conventional IR spectroscopy by sensing photothermal distortions in the infrared absorption regions of a sample using a shorter wavelength "probe beam." Various OPTIR techniques are described, for example, in U.S. Patents 9,091,594, 9,841,324, 10,677,722, 11,002,665, 10,942,116, and 10,809,184, each of which is incorporated herein by reference in its entirety.

[0006] Confocal fluorescence microscopy is a laser-based technique in which radiation of one wavelength excites a fluorescence response in a sample detected at a second wavelength or wavelength range. Numerous libraries of fluorescent dyes have been developed for different functional and structural elements of biological materials (e.g., cells, tissues, and organisms). Fluorescence microscopy enables researchers and clinicians to create, visualize, and analyze micrographs of samples, where each color represents the distribution of a specific target structure within the biological material. Various fluorescence microscopy techniques are described, for example, in: Renz, “Fluorescence Microscopy—A historical and technical perspective,” Cytometry Part A, Vol. 83, pp. 767–779 (2013); and Sanderson et al., “Fluorescence Microscopy,” Cold Spring Harb Protoc., 2014(10): pdb.top071795.doi:10.1101 / pdb.top071795. Summary of the Invention

[0007] According to the embodiments described herein, a method for performing infrared imaging of a sample includes: illuminating a first region of the sample with an infrared source that generates a modulated infrared beam; illuminating the sample in a second region that at least partially overlaps with the first region with an excitation radiation beam to excite spontaneous fluorescence emission; detecting at least a portion of the spontaneous fluorescence emission from the second region as detected spontaneous fluorescence emission by at least one array-based detector; generating a first frame of spontaneous fluorescence data set of the second region using the detected spontaneous fluorescence emission of the sample at a first value of the infrared beam energy; generating a second frame of spontaneous fluorescence data set of the second region using the detected spontaneous fluorescence emission at a second value of the infrared beam energy; and constructing an output indicating infrared absorption of the sample using the first frame of spontaneous fluorescence data set and the second frame of spontaneous fluorescence data set.

[0008] For example, target categories may include any or all of NAD(P)H bound, NAD(P)H free, fatty acids, vitamins, flavins, proteins, porphyrins, and chlorophyll. Filtering the light may include substantially maximizing the transmission of light at the peak amplitude of the sample's autofluorescence emission spectrum, or substantially maximizing the transmission of light at the peak amplitude of the sample's autofluorescence emission spectrum, by using a bandpass filter substantially aligned with the center of the local autofluorescence emission peak of a substance within the sample. Filtering the light may also include making the light transmission at wavelengths where the autofluorescence emission from a first substance within the sample is significantly greater than the light transmission at wavelengths where the autofluorescence emission from a second substance within the sample is not a local peak amplitude of the autofluorescence emission spectrum of the first substance.

[0009] In any of these embodiments, the output can indicate the infrared absorption of the sample in the region that overlaps with the first and second regions.

[0010] The first and second datasets each include an image, an array, or another mapping of the location of absorption or autofluorescence emission onto the sample.

[0011] According to another embodiment, an apparatus for performing infrared imaging of a sample includes: an excitation radiation source configured to irradiate a region of the sample with an excitation radiation beam to excite autofluorescence emission in the sample; a modulated infrared source configured to irradiate a region of the sample with a modulated infrared beam to create an infrared-irradiated region of the sample; at least one array-based detector configured to: capture at least a portion of the autofluorescence emission from the infrared-irradiated region of the sample; generate a first frame autofluorescence image of the region of the sample with a first value of infrared beam energy; and generate a second frame autofluorescence image of the infrared-irradiated region of the sample with a second value of infrared beam energy; and a processor configured to construct a signal indicating infrared absorption of the region of the sample using the first frame autofluorescence image and the second frame autofluorescence image.

[0012] In some embodiments, the apparatus may also include a stage configured to hold a sample. The stage may be movable relative to at least one of the infrared illumination beam and the excitation radiation beam.

[0013] The modulated infrared source can be a pulsed infrared source configured to generate a modulated infrared beam. The modulated infrared source can also be a continuous wave source paired with a chopper to generate a modulated infrared beam. In some embodiments, the excitation radiation source can be modulated rather than pulsed.

[0014] The device may have at least one pulse generator operatively coupled to a modulated infrared source and a modulated radiation source, wherein the at least one pulse generator is configured to control the timing of the modulated infrared source and the excitation radiation source with an adjustable delay.

[0015] The processor can be configured to construct the infrared absorption signal of the indication region by controlling the detector to obtain a first frame autofluorescence image and a second frame autofluorescence image based on the controlled timing of the modulated infrared source and the modulated radiation source.

[0016] The device may also include an infrared objective lens configured to receive a modulated infrared beam and focus the modulated infrared beam onto a region of the sample.

[0017] The device may also include an excitation beam objective configured to: receive a modulated excitation radiation beam; and focus the modulated excitation radiation beam onto a region of the sample; and the excitation beam objective is further configured to: receive autofluorescence emission from the sample; and transmit the received autofluorescence emission toward a detector based on an imaging device.

[0018] The device may also include a dichroic mirror disposed between the excitation beam objective and the detector based on the camera device, wherein the dichroic mirror is selected to: reflect most of the light having the wavelength of excitation radiation; and transmit most of the light having the wavelength of autofluorescence emission.

[0019] The device may include a self-fluorescent filter disposed between the excitation beam objective and the detector based on the camera device.

[0020] Self-fluorescent filters can be long-pass filters.

[0021] The processor can be configured to construct an infrared absorption signal of an indicator region by detecting fluorescence at various wavelengths, each corresponding to a specific autofluorescence wavelength of the substance.

[0022] At least one of a first value or a second value of the infrared beam energy can be substantially zero, or alternatively, one of the first and second values ​​can be less than the other. The excitation source can be one or more light-emitting diodes, which can be pulsed. At least one array-based detector can be configured to detect the autofluorescence response of chlorophyll or another biological substance that converts light into energy.

[0023] The above description of the invention is not intended to describe every illustrated embodiment or implementation of the subject matter herein. The following figures and detailed descriptions illustrate various embodiments in more detail. Attached Figure Description

[0024] The aspects and advantages of the embodiments provided herein are described in detail below with reference to the accompanying drawings. Throughout the drawings, reference numerals may be used repeatedly to indicate the correspondence between referenced elements. The drawings are provided to illustrate exemplary embodiments described herein, and are not intended to limit the scope of this disclosure.

[0025] Figure 1A This is a simplified conceptual block diagram of a microscope system for detecting the fluorescence response to an applied excitation beam and an IR beam, according to an embodiment.

[0026] Figure 1B This is a simplified conceptual block diagram of a microscope system for fluorescence-enhanced photothermal infrared spectroscopy and simultaneous confocal autofluorescence imaging.

[0027] Figure 1C This is a simplified conceptual block diagram of a microscope system employing autofluorescence-enhanced photothermal infrared spectroscopy and simultaneous confocal imaging using a backpropagation geometry.

[0028] Figure 1D This is a simplified conceptual block diagram of a microscope system for using backpropagation geometry and transmission detection for autofluorescence-enhanced photothermal infrared spectroscopy and simultaneous confocal autofluorescence imaging.

[0029] Figure 1E This is a simplified conceptual block diagram of a microscope system employing autofluorescence-enhanced photothermal infrared spectroscopy and simultaneous confocal autofluorescence imaging with a backpropagation geometry featuring an inverted optical microscope configuration.

[0030] Figure 1F This is a simplified conceptual block diagram of a microscope system for autofluorescence-enhanced photothermal infrared spectroscopy and simultaneous confocal fluorescence imaging with multi-line laser excitation and multi-line autofluorescence detection.

[0031] Figure 2 An example of a selected wavelength for detecting autofluorescence according to an embodiment is shown.

[0032] Figure 3A This is a timing diagram showing the IR beam pulse, sample temperature, and excitation beam pulse along a shared time axis according to an embodiment.

[0033] Figure 3B An embodiment is shown. Figure 3A A timing diagram showing the duration of both hot frame measurements and cold frame measurements over a longer period.

[0034] Figure 4A and Figure 4B Depicting something similar to... Figure 1A -1 describes the effect of changing the wavelength of the IR beam in the system. Figure 4A It was shown at 1660cm -1 IR absorption at that location, while Figure 4B This demonstrates tuning to 900 cm using the same sample. -1 Absorption of the IR beam.

[0035] Figures 5A to 5D The autofluorescence of a biological sample at the time of detection is shown according to the embodiments disclosed herein.

[0036] Figures 6A to 6C The image shows AF-PTIR IR absorption images of the human stratum corneum at three different IR wavelengths.

[0037] Figure 7 A, Figure 7 B Figure 7 C and Figure 7 D shows AF-PTIR IR absorption images of algal samples at four different IR wavelengths.

[0038] Figure 8 The AF-PTIR absorption spectra are shown as sequence-extracted from AF-PTIRIR absorption images of four different regions on an algal sample.

[0039] While various embodiments are suitable for a variety of modifications and alternatives, their details have been shown by way of example in the accompanying drawings and will be described in detail. However, it should be understood that it is not intended to limit the claimed invention to the specific embodiments described. Rather, it is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the subject matter defined in the claims. Detailed Implementation

[0040] For the purposes of this specification, the following terms are specifically defined as follows:

[0041] "Analyzer / controller" refers to a system that facilitates data acquisition and control of an autofluorescence photothermal infrared (AF-PTIR) system. The analyzer / controller may be a single integrated electronic housing or may include multiple distributed components. Control elements can provide control for positioning and / or scanning of the fiber optic probe and / or sample. Control elements can also collect data on probe beam deflection, motion, or other responses, providing control over excitation and / or probe power, polarization, steering, focusing, and / or other functions. Control elements, etc., may include computer program methods or digital logic methods and can be implemented using any combination of various computing devices (computers, personal electronic devices), analog and / or digital discrete circuit components (transistors, resistors, capacitors, inductors, diodes, etc.), programmable logic, microprocessors, microcontrollers, application-specific integrated circuits, or other circuit elements. A memory configured to store the computer program may be implemented together with the discrete circuit components to perform one or more of the processes described herein.

[0042] A "beam combiner" refers to an optical element that can combine two beams onto the same optical path. A dichroic mirror or filter can be used as a beam combiner, for example, where the dichroic mirror can be configured to transmit one or more wavelength ranges and reflect one or more other wavelength ranges. For example, two beams arranged at right angles are incident on a dichroic mirror, and the two beams can then be combined onto the same path. In one configuration, the beam combiner can be a beam splitter used in opposite directions, i.e., a beam reflected from the beam splitter interface is combined with another beam transmitted through the beam splitter interface. For example, a beam splitter cube can be used as both a beam splitter and a beam combiner. Even optical elements sold as beam splitters that are not used to split light into two paths can be used as beam combiners. For example, a Mach-Zehnder interferometer uses a beam splitter to split the incident light into two paths and uses a second beam splitter to recombine the two beams. In this case, the second beam splitter is used as a beam combiner. In a Michelson interferometer, a single beam splitter is used both to split the incident light and then to recombine it. Therefore, a beam splitter in a Michelson interferometer serves as both a beam splitter and a beam combiner. A beam combiner can also be, for example, a fiber-based device that combines light from two input fibers into a single output fiber, such as a 1x2 fiber coupler. A single 1x2 fiber coupler can serve as both a beam splitter and a beam combiner.

[0043] A beam splitter is an optical element that can divide light into at least two paths. A beam splitter can include a plate, a cube, and / or a prism, or other shapes / configurations that can divide a beam. A beam splitter can include a thin film that partially reflects light at the wavelength of interest, such that a portion of the incident beam is reflected and another portion is transmitted. A beam splitter can be polarized, wherein it transmits light of essentially one polarization and reflects light of orthogonal polarization. For example, in the case of a Nomarski or Wollaston prism, a beam splitter can also divide light along two transmission paths based on polarization. A beam splitter can also be unpolarized, wherein light is split into two paths substantially independent of the polarization of the incident light. A beam splitter can also be a fiber-based device, such as a 1x2 fiber coupler, that separates light from one input fiber into at least two output fibers. A beam splitter can be a 50:50 beam splitter in which substantially equal portions of light are directed onto two different paths. Beam splitters can also be unbalanced, such as a 90:10 beam splitter that directs 90% of the light onto one path and 10% onto another, or a 70:30 beam splitter that directs 70% of the light onto one path and 30% onto another, or similar beam splitters.

[0044] "Bleaching" refers to photobleaching or the tendency of a sample to exhibit a decreased fluorescence response to optical stimuli with increasing exposure duration. Fluoresceins can be damaged by light exposure, particularly at the excitation wavelengths that cause autofluorescence responses.

[0045] "Collecting probe light" and "collecting probe radiation" refer to the collection of radiation from a probe beam that has already interacted with the sample. Probe light can be collected after reflection, scattering, transmission, evanescent wave coupling, and / or transmission through an aperture detector.

[0046] "Collimating optics" refers to any of the above optical elements arranged in a manner that is approximately collimated radiation. In some embodiments, the same optics can be used as both a focusing optics and a collimating optics, for example, focusing light in one direction of propagation and then recolliding the light in the opposite direction of propagation.

[0047] "Confocal microscopy" refers to optical microscopy in which light collected at the detector is confined to a small volume of light passing through the 3D focusing volume of an optical objective on the sample. Confocal microscopy is typically performed by placing a "confocal aperture" at a focal plane equivalent to the focal plane of the sample, thereby blocking stray light that does not pass through the focusing volume on the sample.

[0048] A “detector” is a device that generates a signal indicating the power, intensity, and / or energy of light / radiation incident on the surface of a detector. This signal is typically an electrical signal, such as voltage, current, and / or charge. A detector can be a photodiode, phototransistor, or charge-coupled device (CCD). In some cases, the detector can be a semiconductor detector, such as a silicon PIN photodiode. A detector can also be an avalanche photodiode, photomultiplier tube, or any other device that generates changes in current, voltage, charge, conductivity, etc., upon light incidence. A detector can include a single element, multiple detector elements, such as dual-cell or quad-cell, linear or two-dimensional arrays of detector elements, including detectors based on camera devices. “Array-based detector” means a detector that can include multiple photosensitive elements (e.g., a linear or 2D array of photosensitive elements, and / or a 1D or 2D array of photosensitive pixels). An array-based detector can be an optical detector comprising an array of multiple photosensitive elements, such as a linear or 2D array of photodiodes or a linear or 2D array of photosensitive pixels, as in detectors based on camera devices. Array-based detectors can be analog devices such as photodiode arrays and / or hybrid analog / digital devices such as camera-based detectors. "Camera device" or "camera-based detector" refers to a type of array-based photodetector comprising multiple photosensitive pixels. The camera device can include one or more technologies, including but not limited to CCD, EM-CCD, CMOS, s-CMOS, and / or other photosensitive array technologies. The camera device can support frame rates of several frames per second, hundreds of frames per second, or even thousands of frames per second or higher.

[0049] The "diffraction limit" of a light beam refers to the minimum distance between two light sources that can be distinguished by a detector. The Abbe diffraction limit d of a microscope with a numerical aperture NA and operating at wavelength λ is defined as d = λ / (2NA). The physical limitations of the numerical aperture of a microscope prevent very large numerical apertures; therefore, the diffraction limit of a microscope is strongly dependent on the operating wavelength used for detection, where larger wavelengths correspond to relatively poorer resolution, and higher wavelengths correspond to improved precision.

[0050] "Demodulation" or "demodulation" refers to the extraction of the information-carrying signal from the overall signal, typically but not necessarily at a specific frequency. For example, in this application, the collected probe light gathered at a photodetector represents the overall signal. The demodulation process selects out portions of infrared light interference absorbed by the sample. Demodulation can be accomplished using a lock-in amplifier, a fast Fourier transform (FFT), calculation of the discrete Fourier components at the desired frequency, a resonant amplifier, a narrowband bandpass filter, or any other technique that greatly enhances the signal of interest while suppressing background and noise signals that are out of sync with the modulation. The demodulator can generate any of a series of signals, including the amplitude and / or phase of the modulation of the information-carrying signal, and / or in-phase (X) or quadrature (Y) signals, or any other desired signal indicating changing properties of the information-carrying signal.

[0051] A demodulator is a device or system that performs demodulation.

[0052] A "quality factor" refers to any measure or index of the relative quality of a signal or measurement. For example, a quality factor can be measurement sensitivity, signal strength, noise level, signal-to-noise ratio, background level, signal-to-background ratio, any combination of these, or other measures that enable ranking of the relative quality of signals and / or measurements.

[0053] "Focusing optics" refers to one or more optical elements that have the ability to focus light. Focusing optics may include one or more refractive lenses, mirrors, diffractive optics, Fresnel lenses, volume holograms, metamaterials or any combination thereof, or any other device or component capable of focusing radiation.

[0054] "Fluorescence" refers to the emission of light from a sample at one wavelength due to excitation at one wavelength. The fluorescence excitation and emission process is a form of inelastic scattering of incident light and can be used to characterize a sample by providing information about the type of fluorescence emission (the number of emitted photons and the wavelength of the emitted photons) based on the specific intensity and spectrum of the incident light. "Autofluorescence" refers to the fluorescence that occurs naturally when a sample is exposed to an excitation source, while fluorescence can more broadly refer to either autofluorescence or exogenous fluorescence resulting from the application / integration of external fluorophores (such as fluorescent dyes, fluorescent proteins, and fluorescent nanoparticles, or other fluorescent treatments).

[0055] "Illuminate," "illuminating," and "illumination" refer to directing radiation toward an object, such as the surface of a sample, a probe tip, and / or the area of ​​probe-sample interaction. Illumination can include radiation in the infrared wavelength range, visible light, and other wavelengths from ultraviolet to millimeters or longer. Illumination can include any configuration of radiation sources, reflective elements, focusing elements, and any other beam control or beam adjustment elements.

[0056] "Infrared absorption spectrum" refers to a spectrum at an indicator wavelength that corresponds to a similar indication of a sample's infrared absorption coefficient, absorbance, or IR absorption properties. An example of an infrared absorption spectrum is an absorption measurement produced by a Fourier transform infrared spectrometer (FTIR), i.e., an FTIR absorption spectrum. Typically, infrared light will be absorbed (i.e., a portion of the infrared absorption spectrum), transmitted (i.e., a portion of the infrared transmission spectrum), or reflected. The reflected or transmitted spectrum of the collected probe light can have different intensities at that wavelength compared to the intensity at each wavelength in the probe source. Note that IR measurements are often plotted as a substitute for the amount of light absorbed, showing the amount of transmitted light. For the purposes of this definition, IR transmission spectra and IR absorption spectra are considered equivalent datasets because a simple relationship exists between the two measurements.

[0057] "Infrared source" and "infrared radiation source" refer to one or more light sources that generate or emit radiation in the infrared wavelength range, typically between 2 micrometers and 25 micrometers. The radiation source can be one of a large number of sources, including heat sources or silicon carbide rod light sources, supercontinuum laser sources, frequency combs, difference frequency generators, sum frequency generators, harmonic generators, optical parametric oscillators (OPO), optical parametric generators (OPG), quantum cascade lasers (QCL), inter-band cavity lasers (ICL), synchrotron infrared radiation sources, nanosecond, picosecond, femtosecond, and attosecond laser systems, CO2 lasers, microheaters, electrically or chemically generated sparks, laser-driven thermal sources, and / or any other source that generates infrared radiation. In a preferred embodiment, the source emits infrared radiation, but the source can also emit in other wavelength ranges (e.g., from ultraviolet to THz). The source can be narrowband (e.g., spectral width <10 cm⁻¹ or <1 cm⁻¹) or wideband (e.g., spectral width >10 cm⁻¹, >100 cm⁻¹, or greater than 500 cm⁻¹). Wideband sources can be made narrowband using filters, monochromators, and other devices. Infrared sources can also consist of one of discrete emission lines, such as those tuned to a specific absorption band of the target species. IR sources with narrower wavelengths can be coupled together to produce IR sources with a wider wavelength range, such as QCLs and OPOs, or multiple external cavity QCL modules, or even a large array of QCL emitters.

[0058] In the context of interacting with a sample, “interaction” means that light illuminating the sample is scattered, refracted, absorbed, distorted, redirected, diffracted, transmitted, and reflected by the sample, through the sample, and / or from the sample.

[0059] A "lock-in amplifier" is an example of a "demodulator" (as defined above) and is a device, system, and / or algorithm that demodulates the response of a system at one or more reference frequencies. A lock-in amplifier can be an electronic component comprising analog electronics, digital electronics, and combinations thereof. It can also be a computational algorithm implemented in digital electronic devices such as microprocessors, field-programmable gate arrays (FPGAs), digital signal processors, and personal computers. A lock-in amplifier can generate signals indicative of various measures of an oscillating system, including amplitude, phase, in-phase (X) components, and quadrature (Y) components, or any combination thereof. In this context, a lock-in amplifier can also generate such measurements at the reference frequency, higher harmonics of the reference frequency, and / or sideband frequencies of the reference frequency.

[0060] In the context of radiation incident on a sample, "modulation" or "modulation" refers to the periodic alteration of the intensity of an infrared laser beam at a given location. Modulation of beam intensity can be achieved through mechanical chopping of the beam, controlled laser pulses, and / or deflection of the laser beam, for example, by using a tilting mirror or a high-speed rotating mirror device that is electrostatically or electromagnetically driven using a piezoelectric actuator or other means to tilt or deform the mirror. Modulation can also be achieved using devices that provide time-varying transmission, such as acousto-optic modulators, electro-optic modulators, photoelastic modulators, pockelcells, etc., and more generally, by any electronically controllable device that can alter the amplitude and / or optical phase of the incident beam. Modulation can also be achieved using diffraction effects, for example, through a modulator based on diffraction MEMS or through a high-speed shutter, attenuator, or other mechanisms that alter the intensity, angle, and / or phase of the laser beam incident on the sample. Modulation can, for example, include any form of periodic waveform, such as a sine wave, square wave, pulse, triangular wave, chirp, etc. Modulation can be performed at a single modulation frequency or can be a synthesis of any number of desired frequency components. In some cases, modulation can also be aperiodic or consist of only a single pulse. A “modulated radiation source” is a radiation source that includes, in some way, the ability to modulate the optical power it transmits. A modulated radiation source can be a source with internal capabilities for pulsed generation and / or modulation of its power, a radiation source with external trigger / modulation control, and / or a CW radiation source coupled to an external modulator, such as a mechanical chopper, an acousto-optic modulator, or any of the other modulators described above, or any other suitable mechanism for varying the power transmitted by the radiation source. Note that in a modulated radiation source, the radiation source and modulator can be physically decoupled, and other optical elements, such as lenses, mirrors, filters, etc., can exist between the radiation source and the modulator.

[0061] "Near-infrared light" usually refers to the wavelength range of infrared (IR) light corresponding to 0.75 μm to 2 μm.

[0062] "Narrowband light source" refers to a light source with a narrow bandwidth or narrow linewidth, such as light with a linewidth of less than 8 cm⁻¹. However, "narrowband light source" can also be a light source with a linewidth so narrow that it does not cover the spectral range of interest of the sample.

[0063] "Optical properties" refers to the optical properties of a sample, including but not limited to refractive index, absorption coefficient, reflectivity, absorptivity, the real and / or imaginary part of the refractive index, the real and / or imaginary part of the sample's dielectric function, and / or any property that can be mathematically derived from one or more of these optical properties.

[0064] "Optical response" refers to the result of the interaction between radiation and a sample. Optical response is related to one or more optical properties defined above. Optical response can be radiation absorption, temperature rise, thermal expansion, photoinductive force, light reflection and / or scattering, or other responses of the material due to interaction with the irradiating radiation.

[0065] "Photothermal distortion" refers to changes in sample properties caused by the absorption of light energy, such as infrared radiation. Photothermal distortion can refer to changes in refractive index, changes in reflectivity, thermal expansion, surface distortion, or other effects that can be detected by a probe beam.

[0066] "Excitation source," "excitation source," or "excitation radiation source" refers to a light source that can be used to induce fluorescence in a sample. Such a source can be used to detect the sample's response to incident light from an infrared source. For example, the excitation source can include a gas laser, a laser diode, a diode-pumped solid-state laser, a supercontinuum laser, an optical parametric oscillator, a superluminescent diode (SLD), a near-infrared laser, or a UV and / or visible laser beam generated via sum-frequency or difference-frequency generation. In some embodiments, the detection source can operate at wavelengths outside the tuning or emission range of the infrared source, but the detection source can also be a fixed-wavelength source at a selected wavelength that actually overlaps with the tuning range of the infrared source. The "detection beam" or "sensing beam" is the beam initially emitted from the detection source. The "autofluorescence excitation source," "autofluorescence excitation source," or "autofluorescence excitation radiation source" is an excitation source configured to excite autofluorescence emission from a sample.

[0067] A "retarder" is an optical element that causes a relative optical phase delay in an optical path. Examples of retarders are waveplates, such as half-wave plates, quarter-wave plates, and eight-wave plates. One or more retarders / waveplates can be used to introduce an optical phase difference between two polarizations of light, for example, to introduce a phase difference between two paths in an orthogonal interferometer.

[0068] A “signal indicating…” refers to a signal that is mathematically related to the property of interest. The signal can be an analog signal, a digital signal, and / or one or more numbers stored in a computer or other digital electronic device. The signal can be voltage, current, or any other signal that can be easily converted and recorded. The signal can be mathematically identical to the property being measured, such as an explicit absolute phase signal or absorption coefficient. The signal can also be a signal that is mathematically related to one or more properties of interest (e.g., involving linear or other scaling, offsetting, inversion, or more complex mathematical operations).

[0069] "Spectrum" refers to the measurement of one or more properties of a sample as a function of wavelength or equivalently (and more commonly) as a function of wavenumber.

[0070] The terms “approximately” or “close to” are synonyms and are used to indicate that the value modified by the term has a known range associated with it, wherein the range may be ±20%, ±15%, ±10%, ±5%, or ±1%. The term “substantially” is used to indicate that a result (e.g., a measurement) is close to a target value, for example, where close may mean that the result is within 80%, 90%, 95%, or 99% of the value.

[0071] Fluorescence-enhanced IR spectroscopy and simultaneous confocal fluorescence imaging

[0072] The applicant’s previous work has demonstrated that fluorescence-enhanced IR spectroscopy and simultaneous confocal fluorescence imaging can be used to obtain high-sensitivity and high-spatial-resolution measurements of IR absorption using simultaneous confocal fluorescence imaging.

[0073] For example, the applicant's previously filed patent application related to fluorescence-enhanced photothermal (FE-PTIR) spectroscopy (published as WO 2022 / 020356) is incorporated herein by reference in its entirety. As described in that reference, fluorescent dyes and / or labels can be used to label samples to increase the measurable response to an excitation beam. As further described in that reference, although some portions of some samples may exhibit autofluorescence, the emission efficiency of temperature-dependent fluorescent dyes causes a significant change in the measured fluorescence intensity with IR heating. A dye discussed in that reference improves thermal sensitivity by approximately two orders of magnitude, exceeding what can be achieved using autofluorescence alone.

[0074] The FE-PTIR method described in WO 2022 / 020356 relates to a novel approach for detecting infrared absorption, which uses the temperature dependence of fluorescence emission efficiency as a reporter of sample infrared absorption. Specifically, the FE-PTIR method irradiates the sample with two beams: an excitation beam for exciting fluorescence emission from the sample and an infrared beam for exciting molecular vibrations within the sample. When the sample is irradiated with infrared light at wavelengths corresponding to molecular vibrations within the sample, the absorbed IR light causes localized heating that alters the fluorescence emission efficiency.

[0075] In this disclosure, a “part” of the sample is referred to as being illuminated. It should be understood that the part in question can refer to the extent of the imaged area or a portion of the light within the imaged area. In various embodiments, a part can be determined by changing the positioning of the sample or the positioning of the light source or detector used. Furthermore, it is not necessary to collect all light from the sample: for example, a sample illuminated across a square cross-section may only be imaged across a suitable circular area within the square; even if a “part” of the illuminated sample is slightly larger or has a different shape than the portion from which light is collected, this is still within the meaning of detecting light from a “part” of the sample. The reverse is also true: even if infrared light is directed to a smaller portion, excitation light may be directed to the entire sample.

[0076] Therefore, the IR absorption of a sample can be inferred by observing the response of changes in fluorescence emission from the sample to pulses of IR radiation. IR absorption spectra can be created by measuring changes in fluorescence emission at different wavelengths, and IR chemical maps can be created by measuring changes in fluorescence emission at one or more wavelengths as a function of localization on the sample. FE-PTIR achieves approximately 100 times better sensitivity compared to conventional O-PTIR measurements because the temperature dependence of approximately 1% / ℃ fluorescence emission efficiency is about 100 times higher than the inherent photothermal sensitivity of most materials.

[0077] In the previously filed application WO 2022 / 020356, the following general knowledge was noted: fluorescent dyes provide a significant improvement over the inherent photothermal sensitivity of most materials. Examples of this general knowledge are provided, for instance, in the following references: C. Paviolo et al., “Temperature measurement in the microscopic regime: a comparison between fluorescence lifetime-and intensity-based methods,” *Journal of Microscopy*, March 22, 2013; and Kuzkova et al., “Application of Temperature-Dependent Fluorescent Dyes to the Measurement of Millimeter Wave Absorption in Water Applied to Biomedical Experiments,” *International Journal of Biomedical Imaging*, November 2014. The applicant noted that the systems and methods described therein can improve the sensitivity of photothermal measurements by approximately a factor of 100-fold compared to photothermal measurements alone.

[0078] Because the FE-PTIR method offers co-localization fluorescence imaging and IR spectroscopy with up to 100 times greater sensitivity than previously available methods, it is expected to become extremely important commercially. However, the addition of fluorophores to provide this significantly enhanced sensitivity can be time-consuming, expensive, and in some cases, may alter the sample in undesirable ways, as described, for example, in the following literature: Jensen, “Use of Fluorescent Probes: Their Effect on Cell Biology and Limitations,” 295ARInsights12 (October 12, 2012).

[0079] Conventional fluorescent labeling is performed to highlight and localize specific components in biological cells and tissues. Conventional fluorescent labeling provides an additional benefit in FE-PTIR by acting as an IR absorption reporter that is more sensitive than the inherent photothermal sensitivity of most samples. However, a similar sensitivity enhancement is desired without the fluorescent labeling step, or where fluorescent labeling is impractical or undesirable.

[0080] Autofluorescence is a known property of many biological materials, but it is often considered a detrimental background for conventional fluorescence microscopy, and conventional fluorescent dyes / proteins are chosen to attempt to have emission efficiencies that are substantially higher than the autofluorescence background. Therefore, one might nominally expect autofluorescence emission efficiencies to be too low for efficient FE-PTIR operation. Subsequently and surprisingly, it has been found that the systems and methods described herein can achieve sufficient excitation of autofluorescence and sufficient modulation of it via IR absorption for practical fluorescence-enhanced photothermal infrared spectroscopy and imaging without relying on the addition of external fluorophores (such as fluorescent dyes or fluorescent proteins). Specifically, it has been found that high levels of signal-to-noise ratio can be achieved in autofluorescence-based photothermal analysis by using specific structures, wavelengths, pulse timing, and analytical methods as described herein, while eliminating the need for dyes or other added fluorophores. Furthermore, conventional photothermal analysis methods result in bleaching that is too rapid for many types of samples, and the effectiveness of bleaching using the methods described herein is reduced, both for applications using exogenous fluorophores such as fluorescent dyes and proteins, and for autofluorescence applications without externally added fluorophores.

[0081] Using autofluorescence-based FE-PTIR also allows for a significant reduction in coherent interference artifacts from the image, as described in more detail below. In contrast, laser-based methods used in conventional OPTIR can generate coherent interference artifacts that can lead to confusion regarding the distribution of different chemical species within the sample. Specifically, when a sample is irradiated with probe radiation from a narrowband laser source, the laser beam may be reflected or scattered from multiple surfaces within the sample and sample substrate. If the probe light from these multiple reflection or scattering events coherently recombines at the detector, this can result in constructive or destructive interference that can indicate variations in sample height (rather than different intensities of IR absorption). The FE-PTIR method essentially eliminates coherent artifacts because the light collected at the detector is at a different wavelength compared to the excitation light from the probe beam. Specifically, one or more filters are used in the FE-PTIR method to block the excitation probe light from reaching the detector.

[0082] Any probe excitation light that undergoes multiple reflection / scattering events does not contribute to the IR photothermal signal and therefore does not produce unwanted artifacts. It is desirable to minimize or eliminate these coherent artifacts for all samples, but many samples with external fluorophores are undesirable. However, it is still desirable to suppress coherent artifacts lacking external fluorophores.

[0083] Backpropagation microscope design

[0084] Figure 1AA simplified schematic diagram of a microscope 100 for autofluorescence-based FE-PTIR and simultaneous confocal fluorescence imaging of sample 102 is shown. In summary, the autofluorescence excitation radiation source 106 is used to excite autofluorescence emission from sample 102. The absorption of infrared radiation from IR source 104 by sample 102 results in modulation of the amount of autofluorescence emission from the sample. By recording the amount of autofluorescence emission from the sample in response to changes in IR radiation, a signal indicating the IR absorption characteristics of the sample can be generated. The key aspects are described below: (1) exciting autofluorescence emission from the sample; (2) irradiating the sample with infrared light to induce a temperature change in the sample; (3) collecting autofluorescence emission from the sample; (4) detecting the autofluorescence emission; (5) detecting the IR-induced temperature change via changes in autofluorescence emission; and (6) creating a signal indicating the IR absorption of the sample based on changes in autofluorescence emission.

[0085] (1) Excite the autofluorescence emission from the sample.

[0086] AF excitation radiation source 106 generates an AF excitation beam, such as Figure 1A As shown, the arrow originates from the AF excitation source 106, which passes through the excitation filter 112, is reflected by the dichroic mirror 114, focused by the objective lens 116, and focused onto the sample 102. Specifically, the AF excitation source 106 generates an AF excitation beam that illuminates the area of ​​the sample to be illuminated by the IR beam, as described below. Any number of additional guiding structures, focusing structures, filtering structures, or modulation structures can be arranged along the beam path between the AF excitation source 106 and the sample 102.

[0087] The AF excitation radiation source is preferably capable of emitting at a wavelength corresponding to the effective spontaneous fluorescence emission from the sample of interest. For example, as will be discussed later. Figure 2 As described in connection, many biological materials exhibit fluorescent autofluorescence emission when excited in the UV (e.g., in the range of 250 nm to 400 nm, or particularly at UV laser lines of about 266 nm and 366 nm).

[0088] In one embodiment, to achieve an improved SNR, the AF excitation source 106 should generate short, bright light pulses within the range of different fluorescence excitation lines of interest, and the AF excitation source 106 can be synchronized with the timing of the maximum temperature rise in the sample, as per [reference to...]. Figure 3A and Figure 3BAs described, suitable pulsed excitation sources include, but are not limited to: ultra-bright pulsed LEDs, pulsed monochromatic laser sources, pulsed laser diodes, pulsed superluminescent diodes, pulsed tunable lasers, pulsed supercontinuum broadband visible laser sources, and other suitable sources with an appropriate emission range for fluorescence / autofluorescence excitation. Conventional UV / white light excitation sources such as halogen lamps can also be used in conjunction with suitable shutter / wavelength filters.

[0089] A suitable AF excitation radiation source 106 can be constructed using an ultra-bright light-emitting diode (LED) incorporating a high-current pulsed drive circuit, such as that described in Carlen's U.S. Patent No. 9,603,210, "High speed, high current pulsed driver circuit". Suitable pulsed drive electronics for the ultra-bright LED using an enhancement-mode field-effect transistor half-bridge circuit, such as those from high-efficiency power conversion... Transistor. The AF excitation source 106 can be, for example, a tunable light source or a fixed-wavelength light source, typically (though not necessarily) at a wavelength shorter than that of the IR source 104. The AF excitation source 106 can be configured to correspond to wavelengths of specific molecular bonds to identify specific materials within the sample 102, as described below. Figure 2 A more detailed description.

[0090] Excitation filter 112 is an optional component that can be used to provide an AF excitation beam at the desired wavelength even when the AF excitation source is not tuned to a specific wavelength. Particularly in the case of a broadband LED excitation source, excitation filter 112 can be placed after the excitation source to select a specific wavelength range for effective autofluorescence excitation. Excitation filter 112 preferably has high transmittance around the optimal wavelength or autofluorescence excitation and preferably blocks wavelengths corresponding to possible autofluorescence emission from the sample. Excitation filter 112 may include one or more short-pass filters, band-pass filters, and / or multi-band filters. Excitation filter 112 can also be fixed, interchangeable, adjustable, or tunable. For example, an acousto-optic tunable filter (AOTF) can be used as a tunable excitation filter 112.

[0091] Dichroic mirror 114, such as excitation filter 112, is an optional component arranged between AF excitation source 106 and sample 102. Figure 1A In the illustrated embodiment, the AF excitation beam is incident on a dichroic mirror 114 that reflects the excitation beam toward the sample 102. The excitation beam is focused by the objective lens 116 toward a region of the sample. While the excitation radiation can be focused to a diffraction-limited point for point measurements, in a preferred embodiment, the excitation radiation is configured to illuminate a wide region of the sample 102. Figure 1AIn the wide field-of-view embodiment shown, the wide region is at least 10 μm wide, and preferably >50 μm wide, or even >100 μm wide. The excitation light is selected to have sufficient intensity and wavelength to generate a significant autofluorescence response in sample 102.

[0092] The autofluorescence emission from sample 102 was collected by objective lens 116. Figure 1A In the embodiment shown, the collecting objective is the same objective 116 used to illuminate the sample with excitation light, i.e., the so-called epifluorescence configuration. In this case, the autofluorescence emission from the sample is transmitted through the dichroic mirror 114, while the excitation beam originating from the AF excitation source 106 at the original wavelength (and optionally filtered by the excitation filter 112) is substantially reflected, wherein only a small fraction of the excitation light passes through the dichroic mirror 114 due to the practical optical density limitation of the dichroic mirror.

[0093] (2) Infrared excitation

[0094] The microscope 100 includes a second light source, an IR source 104, for detecting molecular vibrations in a sample for the purpose of performing infrared spectroscopy and infrared chemical imaging. The IR source 104 may be, for example, a tunable narrowband IR source and / or a broadband IR source. When the beam 105 emitted by the IR source 104 contains at least one wavelength corresponding to an absorption band within the sample 102, the IR absorption region of the sample 102 will (depending on the details of the IR source) heat up with each IR pulse or within each IR modulation cycle.

[0095] The light sources (104, 106) can be controlled by one or more pulse generators 108. The pulse generators 108 can trigger output pulses, control inputs, and / or modulate the output from each of the light sources (104, 106). The IR source 104 and the AF excitation source 106 can be directly modulated (i.e., by causing the sources 104 and 106 themselves to generate pulsed beams) or indirectly modulated (e.g., by operating a chopper or other structure that modulates a constant beam). Both the IR source 104 and the AF excitation source 106 can be directly modulated, both can be indirectly modulated, or one can be modulated while the other is indirectly modulated.

[0096] IR source 104 generates an IR beam, in Figure 1A The arrows originating from IR source 104 indicate that the IR beam is received at IR objective 110 and focused onto sample 102. In one embodiment, IR source 104 is a tunable mid-IR laser that can be tuned to different wavelengths. Figure 1AA mirror is shown arranged between the IR source 104 and the IR objective. In an embodiment, any number of optical structures can be arranged between the IR source 104 and the sample 102, providing modulation, direction, filtering, and focusing to reach the sample 102. Specifically, the IR source 104 generates an IR beam that is directed to a specific region of the sample 102, which is also illuminated by the AF excitation beam as described above.

[0097] Using an IR objective 110 as a focusing optics, light from an IR source 104 is optionally focused onto a sample 104 to create an IR-irradiated region or area on the sample that at least partially overlaps with the region irradiated by the AF excitation source 106. The IR source 104 generates an IR beam, which can be focused using a reflecting objective 110 (e.g., a Schwarzschild-designed reflecting objective, an off-axis parabolic mirror, or other reflecting optics, and / or a refractive objective constructed using an IR-permeable material). Alternatively, the raw beam from an IR laser can be used simply if it is small enough and / or sufficiently strong. The IR optics 110 is configured to irradiate a wide region of the sample, which is at least 10 μm wide and preferably >50 μm wide, or even >100 μm wide. In one embodiment, the IR excitation irradiation region and the AF excitation irradiation region can be arranged substantially similarly, such that a common wide region of the sample exists, which has both IR irradiation and AF-based IR absorption readings.

[0098] (3) Collect autofluorescence emission from the sample

[0099] After exciting autofluorescence emission from the sample, microscope 100 is used to detect modulation in the emission due to absorption of IR light by sample 102. The autofluorescence emission is collected by an objective lens (e.g., UV / vis objective lens 116). Alternatively / additionally, the autofluorescence emission may be collected by an IR objective lens 110 and / or by additional collecting optics. Figure 1A The diagram illustrates a backpropagation geometry where an IR beam is delivered to one side of the sample, and autofluorescence emission / collection is performed on the other side. However, autofluorescence photothermal IR measurements can also be performed in a copropagation configuration, for example, where an IR-compatible objective (e.g., a Schwarzschild objective) can be used to focus both IR and autofluorescence excitation, and the same objective is used to collect the autofluorescence emission.

[0100] continue Figure 1AIn the backpropagation configuration shown, the autofluorescence emission from sample 102 can pass through dichroic mirror 114. At dichroic mirror 114, the autofluorescence emission from sample 102 can be additionally filtered by an optional autofluorescence filter 118 to block any remnants of the original excitation wavelength from AF excitation source 106, while allowing the autofluorescence emission from sample 102 to pass through efficiently. Suitable emission filters are available, for example, from Thorlabs, Semrock, Chroma, and other suppliers. However, Figure 1A The autofluorescence filter 118 in the microscope 100 shown is preferably different from the filters conventionally used in fluorescence microscopy where fluorophores have been added. When using a specific fluorophore, the emission filter is selected to: isolate fluorescence emission from the specific fluorophore; and exclude emission from other fluorophores. In the microscope 100, it may be desirable to collect as much autofluorescence as possible to cover as much material as possible. Therefore, a long-pass filter can be used instead of the band-pass filter more commonly used for fluorescence microscopy applications. In one embodiment, the emission filter may be a long-pass filter with a cutoff frequency of approximately 425 nm, such as the Chroma model ET425lp or the Thorlabs 400 nm or 450 nm FELH0400 or FELH0450.

[0101] (4) Detection of autofluorescence emission

[0102] Once the autofluorescence emission optionally passes through the dichroic mirror 114 and the autofluorescence filter 118, the autofluorescence emission is collected at the detector 122. Optional optics 120 can be used to guide at least a portion of the collected autofluorescence emission onto the detector 122. The detector 122 can be a single-element detector for collecting light from a single region of the sample. However, in a preferred embodiment, the detector 122 can be an array-based detector, such as a camera-based detector, capable of collecting autofluorescence emissions from multiple locations on the sample. In cases of wide-field detection using an array / camera-based detector, the optional optics 120 can be, for example, a microscope tube lens 120.

[0103] In a wide field-of-view implementation, detector 122 can be any of a variety of structures capable of wide field-of-view detection. In the case of an array-based detector, detector 122 may include a detector having multiple photosensitive elements (e.g., a linear or 2D array of photosensitive elements, and / or a 1D or 2D array of photosensitive pixels). Using a camera-based detector in this case allows for simultaneous measurement of IR-induced changes in fluorescence emission at many locations on the sample. Such measurements can be performed using time-domain measurements (e.g., subtracting the measured fluorescence frame from the camera frame while the IR beam is on and off) and / or utilizing a locked camera device, as described below. Figure 3A and Figure 3B A more detailed description.

[0104] The description of the equipment in this article involves Figure 1A However, it should be understood that detection of autofluorescence can be performed using any of a variety of architectures and arrangements. As described in more detail below, autofluorescence excitation and collection can be performed, for example, using different optics / objectives on opposite sides of the sample. For example, autofluorescence excitation can be alternately provided from below the sample by coupling autofluorescence excitation source 106 to condenser / objective 110. Figure 1A In this implementation, the condenser / objective 110 is also used to deliver infrared radiation to the sample. In an alternative configuration, the IR beam from source 104 can be directed to the top side of the sample, for example, in the configuration of a conventional inverted optical microscope, where fluorescence excitation / detection can be performed from the bottom. In fact, any arrangement of IR from the top or bottom, autofluorescence excitation from the top or bottom, and autofluorescence detection from the top or bottom can be used to achieve a working implementation of AF-PTIR.

[0105] Figures 1B to 1F Some of these alternative configurations are shown. Figure 1B This illustrates a co-propagation configuration where the IR beam and the autofluorescence excitation beam are arranged substantially collinearly and then focused onto the sample by a common objective. The autofluorescence emission from the sample is then collected by the same objective and directed toward the detector. Figures 1B to 1F Some of the alternatives shown depict single-point detection systems that transmit collected light through a pinhole (e.g., Figures 1B to 1E ), while others have shown wide field-of-view detection systems (e.g., Figure 1F Such systems are typically interchangeable; that is, when demonstrating a single-point or grating detection system... Figures 1A to 1F In each of the figures, a wide field-of-view detection configuration can be used, and within the wide field-of-view detection configuration, a single-point or raster detection system can be used. Figures 1A to 1F Each of the figures in the diagram.

[0106] exist Figure 1B In one implementation, the collected light is optionally focused through a confocal pinhole before the detector. This arrangement may be ideal: using an autofluorescence excitation source (e.g., a laser source) that can be focused to a close point, the confocal pinhole then provides depth-resolved detection and suppression of autofluorescence from outside the focal plane of the objective lens. Figure 1B It also demonstrates the use of interchangeable emission filters to select a specific range of autofluorescence. Figure 1B The embodiments shown use different structures because these different structures include those with... Figure 1A The different versions of the blocking filter 118 depicted in the text.

[0107] Including the blocking filter 118 provides a significant improvement in spatial resolution for photothermal measurements because the blocking filter 118 will only allow wavelength-shifted fluorescence 119 to pass through. Therefore, in Figure 1B In the example, filter 118 will only allow fluorescence emitted from body 150 to pass through. However, the fluorescence emitted / collected from body 150 still contains the imprint of IR absorption from body 150. That is, the periodic IR absorption by body 150 will cause corresponding periodic changes in the intensity, angular distribution, and / or phase of the fluorescence emitted from body 150. Therefore, by analyzing the periodic changes in the collected fluorescence, IR absorption measurements can be performed over a length range even smaller than the size of the focused probe beam 116.

[0108] Figure 1C A backpropagation arrangement is illustrated, in which IR light is transmitted from below to the sample, and autofluorescence excitation and detection are performed from above the sample. This embodiment also illustrates the use of interchangeable emission filters and confocal detection, either of which can be used in… Figure 1A This is implemented in the above-mentioned implementation methods.

[0109] exist Figure 1CIn the illustrated embodiment, an IR beam 102 is emitted from an IR source 100 and optionally reflected from a mirror or dichroic mirror 200, at which point the IR beam 102 is then focused onto the sample 110 by a focusing optics 201. In this case, the focusing optics 201 can be optimized to focus IR light, while the focusing optics 106 can be optimized to focus / collect / image the wavelengths of visible light, and specifically the UV / VIS source 112, and the fluorescence emission from the sample 110 / sample 111. The focusing optics 201 can be, for example, a Schwarzschild or similarly designed reflecting objective, an off-axis parabolic mirror, a refracting objective made of an IR-permeable material, or any similar optics capable of focusing IR light. The focusing optics 204 are used to focus and collect UV / visible light, including the incident probe light 113, and any fluorescence emission from the sample. The focusing optics 204 can be a high-quality, high-numerical-aperture refractive objective, such as those commonly used in fluorescence microscopy. Specifically, the focusing optics 204 can be selected to have good performance at wavelengths corresponding to the fluorescence excitation and emission wavelengths, and therefore can include optical components that transmit UV light (in the case of a refractive objective) or reflect UV light (in the case of a reflective objective). In this configuration, if a sample substrate 111 is used, it may be desirable to select a material with high IR transmittance, although the substrate does not need to transmit UV / visible light because fluorescence emission is collected in a backscattering configuration. Optics 200 can be a simple fixed mirror, an adjustable mirror, or an electronically controllable mirror (such as a galvanometer or a fast-turning mirror) to provide relative adjustment of the overlap between the focal points of the visible and IR beams. Alternatively or additionally, optics 200 can be a dichroic mirror to allow visible light illumination from below the sample, i.e., reflecting IR light and transmitting at least a desired range of visible light.

[0110] Figure 1D The following embodiment is illustrated, in which, in an epifluorescence configuration similar to that of a conventional inverted optical microscope, IR excitation is arranged from above the sample, and AF excitation and detection are performed from below the sample. This arrangement has the particular advantage that the sample substrate (i.e., the surface on which the sample is mounted) does not need to be IR-permeable. For example, the sample substrate can be a coverslip or slide, where AF excitation and emission can easily penetrate the substrate, but the IR beam does not need to transmit through the substrate as long as the sample is on the top surface of the substrate. This configuration allows the use of low-cost glass coverslips or slides instead of IR-permeable substrates.

[0111] Figure 1D A simplified schematic diagram of an implementation using backpropagation geometry and transmission detection fluorescence-enhanced photothermal infrared spectroscopy is shown. Figure 1D based on Figures 1A to 1C And when using the same reference numerals, with Figures 1A to 1C The associated descriptions are appropriately applied. Except that the detection of fluorescence emission and fluorescence-enhanced IR absorption is performed after the probe light has passed through sample 110 / sample 111, Figure 1D and Figure 1A , Figure 1B and Figure 1C Similarly, this arrangement may be ideal when the sample or specimen has low reflectivity to visible light and / or when the current scattering path is more efficient for fluorescence collection. In the illustrated embodiment, the dichroic mirror 300 is selected to reflect IR wavelengths and transmit wavelengths corresponding to the fluorescence emission of the sample, and optionally also transmits a probe beam. The dichroic mirror 300 may also be selected to reflect or absorb the excitation wavelength of the probe beam source, thereby eliminating the need for the blocking filter 118. In this case, the focusing optics 204 can be optimized for focusing UV / visible light, and the focusing optics 106 is used both for focusing IR light and for collecting fluorescence emission from the sample 110.

[0112] Figure 1C and Figure 1D The embodiment shown has IR light from below the sample and UV / VIS probe light from above the sample, similar to the configuration of a conventional upright optical microscope in which the refracting microscope objective is placed above the sample. This configuration can be easily reversed so that the UV / VIS light comes from below the sample and the IR light comes from above the sample. Figure 1E Such a reverse configuration is shown, which can then be compatible with the settings of conventional reverse optical microscopes commonly used in cell biology and other biological research. Figure 1E based on Figures 1A to 1D And when using the same reference numerals, with Figures 1A to 1D The relevant descriptions are appropriately applied. Figure 1E In this embodiment, IR light 102 from IR source 100 is transmitted from above the sample to sample 110 via focusing optics 201, while UV / visible light 113 is transmitted from below the sample to sample via focusing optics 204.

[0113] This arrangement is advantageous for several reasons. First, the sample substrate 111 does not need to be IR-permeable and can in fact be a conventional slide, coverslip, or culture dish as commonly used in cell biology. However, the sample substrate 111 should transmit the wavelength of interest of the UV / VIS source 112 as well as any fluorescence emission wavelength of interest of the sample. Fluorescence-enhanced photothermal IR spectroscopy measurements can also be performed with the sample in a fluid (e.g., a buffered aqueous solution). This can be achieved by using a thin IR-permeable cover glass 400 on top of the sample 110 and sample substrate 111, with a thin layer of liquid 402 between them. Larger thicknesses of liquid can also be supported, for example by adhering the sample to the bottom of the cover glass 400, thereby eliminating the need for IR light transmission through the liquid and avoiding any loss associated with IR absorption. The sample / sample can also be mounted in a sealed fluid reservoir to prevent liquid evaporation, and / or mounted in a perfusion reservoir to allow for fluid / nutrient exchange, etc.

[0114] Figure 1E It shows about Figure 1D A variation thereof, wherein an IR-transmittable window or coverslip is placed on top of the sample and sample substrate to enable measurement of the sample immersed in a liquid (e.g., a buffered aqueous solution).

[0115] Figure 1F An implementation method employing simultaneous or sequential multi-line fluorescence and fluorescence-enhanced photothermal IR measurements is shown. Figure 1F based on Figure 1A and Figure 1B And when the same reference numerals are used, the description associated with Figure 1 applies appropriately. Figure 1FIn this embodiment, the UV / VIS source 112 includes multiple UV / VIS sources 500, 502, and 504, which emit at different center wavelengths 1i, 12, and 13, corresponding to different excitation wavelengths of the fluorescent tag of interest and / or inherent autofluorescence. Alternatively, the UV / VIS source may be a supercontinuum laser emitting light in a wavelength range and may be combined with one or more narrowband filters, including variable tunable filters. In the case of the discrete UV / VIS source shown, dichroic mirrors 506 and 508 can be used to combine the beams from sources 500, 502, and 504 onto the same optical path. (Element 510 may be a simple mirror.) Note that the discrete UV / VIS source can also be located remotely from the microscope system and fiber-coupled into the microscope, in which case dichroic mirrors 506 and 508 combine the beams from the output of the fiber rather than directly from a light-generating source such as a laser diode. Optionally, the combined UV / VIS beam can be directed to the beam adjustment / scanning unit 512, where the beam can be expanded, filtered, and / or attenuated as needed. Unit 512 may also include optical mechanisms to provide beam scanning, such as one or more galvanometer scanners, scanning lenses, etc. The UV / VIS source 112 may also include a shutter and / or a flip mirror to allow / block emissions from one or more of the individual UV sources 500, 502, 504 from leaving the integrated UV / VIS source 112.

[0116] After one or more UV / VIS beams leave the beam conditioning / scanning unit 512, in addition to being able to excite the sample at more than one wavelength at a time, the device can, as shown in... Figure 1A It operates in a similar manner to that described in [the text]. Furthermore, fluorescence emission can occur at multiple different wavelengths. To accommodate this, multiple detectors 514, 516, and 518 are provided, along with appropriate dichroic filters 520 and 522, to direct fluorescence within the desired wavelength range towards detectors 516 and 518; and an optional blocking filter 524 is provided to further filter any light outside the range of interest measured at detector 514 that passes through filters 520 and 522. If desired, dichroic filters 520 and 522, as well as blocking filter 524, can be interchangeable and / or tunable to accommodate fluorescence from different fluorescent dyes. Figure 1A and Figure 1BSimilarly, the outputs from detectors 514, 516, and 518 travel along two parallel paths. In one case, as described above, one path leads to the demodulator to extract the AC signal indicating IR absorption, and another path leads to an alternative path for fluorescence detection. This arrangement allows for simultaneous or sequential measurement of fluorescence and fluorescence-enhanced photothermal IR signals at multiple excitation and emission wavelengths. Note that although three emission wavelengths are shown at three excitation / emission wavelengths, the device can be adapted to accommodate multiple desired simultaneous excitation / emission wavelengths.

[0117] exist Figures 1A to 1F In each of these, the pulse source is coupled to two light sources for an excitation beam and a heating / infrared / pump beam. As described in more detail below with respect to Figure 3, the coordinated timing of these light sources can be used to generate a pulse train, which can be used to obtain autofluorescence data to characterize the sample.

[0118] (5) Detecting IR-induced temperature changes by detecting changes in autofluorescence emission.

[0119] The collected fluorescence detected at detector 122 is demodulated by demodulator 124 to generate an AC-modulated signal indicating the collected autofluorescence. In one embodiment, demodulator 124 may be, for example, a lock-in amplifier, and the AC signal may be an amplitude signal, a phase signal, an in-phase signal, or a quadrature signal, or any synchronous measurement of AC modulation of autofluorescence.

[0120] In the case where detector 122 is an array-based detector (such as a camera device), the temperature change caused by IR can be determined by comparing images obtained at different intensities of IR irradiation, for example, comparing an autofluorescence image frame obtained at a first value of IR irradiation with an autofluorescence image frame obtained at a second value of IR irradiation. In the simplest case, the first value of IR irradiation can be zero (i.e., with the IR beam off). The second value of IR irradiation can be selected to cause a detectable change in the autofluorescence emission from the sample. However, in some cases, one of the IR irradiation values ​​does not have to be zero. For example, the IR light can be modulated between 50% and 100% of the nominal beam power, and this will still produce modulation in the autofluorescence from the sample. In an embodiment, a sinusoidal modification of the amplitude of the IR light can be performed, and the photothermal effect on autofluorescence emission can be detected between periods of low IR amplitude and periods of high IR amplitude. In the case of a camera-based detector, the demodulator can be an image subtractor, for example, used to calculate the image difference between the first and second autofluorescence images at the first and second IR irradiation values.

[0121] Further accumulation of the image pair differences may be necessary, such as summing the differences of tens, hundreds, or even thousands of autofluorescence images at two different IR illuminance values. In the simplest case, the demodulator can, for example, sum the differences between so-called cold frames (IR off) and hot frames (IR on) in processor 126. Note that the image subtraction / accumulation using demodulator 124 and processor 126 shown in Figure 1 is for illustrative purposes and does not necessarily imply a separate physical structure for implementing image pair subtraction and accumulation. For example, the image subtraction and accumulation functions can be performed on the same processor (e.g., a field-programmable gate array (FPGA) or microprocessor and / or host computer). In some cases, all image subtraction and accumulation can be performed using onboard demodulator / processor functions integrated with the detector of the camera-based device. Alternatively, images can be rapidly transferred to an image grabber and / or host computer, and image subtraction / accumulation can be performed on either / both of these devices. Image subtraction and accumulation can be performed on the same processor, or these functions can be partitioned among different processors as needed.

[0122] Demodulators for camera-based detection can also be more complex than simple image subtraction / cumulation. For example, in the case of a sinusoidally modulated IR source, it may be desirable to scale different images based on the relative intensity of the IR power at the time of image capture. Noise reduction techniques can also be employed in image summation, such as using statistical analysis to maintain and enhance features common to differences across consecutive image pairs, while discarding those features that are not reproducible and are caused by noise. Camera-based detectors can also be used to perform “lock-in” detection. For example, using a reference signal from pulse generator 108 (as indicated by the dashed line), a highly sensitive measurement of AC modulation of the fluorescence signal can be performed in sync with the modulation of IR source 104 and AF excitation source 106. In all the above cases, the demodulator (e.g., 124) and / or processor (e.g., 126) generate signals indicating changes in the autofluorescence emission from the sample in response to the IR absorption of the sample.

[0123] (6) The IR absorption of the sample is indicated by the change in autofluorescence emission.

[0124] The signal generated by demodulator 124 and / or processor 126, indicating changes in autofluorescence emission from the sample, can be used to calculate IR absorption spectra and / or generate IR chemical images based on photothermal changes in autofluorescence emission. Specifically, if changes in autofluorescence emission from the sample are measured at multiple locations on the sample at one or more IR wavelengths, these measurements can produce graphs or images showing the intensity of IR absorption of the sample at selected wavelengths. More specifically, these graphs / images can represent the distribution of different chemical species based on the IR absorption characteristics of different chemical species. Furthermore, if changes in autofluorescence emission are measured at multiple IR wavelengths, IR absorption spectra can be generated, i.e., a representation of the IR absorption characteristics of the sample at one or more locations on the sample. In the case of wide-field detection using array-based detectors (e.g., imaging devices), IR absorption spectra at hundreds, thousands, or even millions of locations on the sample can be measured simultaneously.

[0125] In use, Figure 1A The microscope 100 depicted in the image can generate an image of sample 102, which (e.g., in the form of a hyperspectral array, i.e., an array of AF-PTIR images at different IR wavelengths) indicates the presence of various types of molecules, compounds, or chemical bonds representing the chemical structure within sample 102.

[0126] Figure 2 These are the autofluorescence emission spectra of various biomolecules. Figure 2 This image is adapted from Croce and Botteiroli's article "Autofluorescence spectroscopy and imaging: a tool for biomedical research and diagnosis" (European Journal of Histochemistry 2014; Vol. 58: 2461), which is incorporated herein by reference. The figure depicts the autofluorescence intensity of each of various biological compounds and structures when irradiated at 366 nm by an excitation source. Figure 2 As shown, the peak fluorescence response varies for each such material. Besides the broad emission of lipochromes, Figure 2 The spectrum in the image has been normalized to the maximum emission peak for presentation. Figure 2The most efficient excitation / emission wavelengths for various biomolecules of potential interest are shown. A key observation is that many of the listed biomolecules are efficiently excited in the UV range (e.g., 240 nm to 400 nm), and some exhibit efficient autofluorescence excitation in the 400 nm to 500 nm range. Therefore, for efficient autofluorescence excitation, an excitation source with emission in the UV range and optionally in a short wavelength range of visible light (e.g., 400 nm to 500 nm) is desirable.

[0127] Figure 2 The dashed lines below each of the solid lines depict the autofluorescence response of the corresponding material, which is reduced due to irradiation with an infrared beam. This reduction is reversible; that is, removing the IR light from the sample will cause the autofluorescence intensity to increase back to the amount shown in the solid lines. For clarity, in... Figure 2 In the diagram, the fluorescence intensity shown by the dashed line is exaggerated; depending on the compound or material, the decrease in autofluorescence may be less than the depicted decrease. For some substances, the decrease in autofluorescence emission during infrared irradiation may be between 0.05% and 2%.

[0128] To detect a specific substance, one approach is to filter all wavelength bands except those centered on the peak fluorescence intensity of the specific material, such as wavelength 202A used for detecting porphyrins.

[0129] In other implementations or for the detection of other substances, using a wavelength band corresponding to the peak fluorescence may not provide optimal differentiation between different chemical species. For example, wavelength 202B can provide a more reliable indication of the presence of flavins in a sample compared to positioning the wavelength 202B at the peak of the fluorescence response. Detecting fluorescence at the higher wavelength shoulder of the response only causes a slightly reduced detectable response. Meanwhile, other signals—such as those from vitamin A, fatty acids, and those without NAD(P)H—decline significantly at higher wavelengths.

[0130] To detect free NAD(P)H in samples that also include NAD(P)H binding and fatty acids, detecting fluorescence at wavelength 202C may be most effective. Wavelength 202C corresponds to the region where fluorescence signals from NAD(P)H binding and fatty acids are lowest, while those from free NAD(P)H are near their peak. Detecting fluorescence at higher wavelengths may elicit a significant signal from fatty acids, while detecting fluorescence at lower wavelengths corresponds to a reduced detection signal from free NAD(P)H itself and an increased signal from NAD(P)H binding. Depending on the sample (e.g., Figure 1A The expected composition of sample 102 can be selected, and the wavelength band used for detection can be chosen to highlight specific compounds.

[0131] To effectively excite autofluorescence emission as described above, the AF excitation source should preferably emit light at one or more wavelengths in the range of 250 nm to 400 nm, and in one embodiment, specifically, the AF excitation source emits light at approximately 365 nm. Suitable AF excitation sources can be light-emitting diodes, arrays of LEDs, lasers, halogen sources, tunable lasers, etc. Suitable excitation is ultraviolet LEDs, such as 365 nm LEDs, such as the Thorlabs M265L3 or M365LP1 or similar products. The 365 nm center wavelength of the Thorlabs source mentioned above is a good choice for exciting autofluorescence in a wide variety of biological materials. Figure 2 As shown, many different biological materials produce autofluorescence in the range of 400 nm to 700 nm upon excitation at 366 nm. Broadband LED irradiators, such as those available from Lumencor and Excelitas, are also suitable because they can be configured to supply UV excitation in addition to the excitation at other wavelengths required for conventional fluorescence microscopy. Alternatively or additionally, tunable broadband sources, such as supercontinuum spectral sources, from NKT Photonics can be used to provide continuously tunable excitation to optimize the autofluorescence (AF) excitation of molecules of interest.

[0132] Specifically, in the case of a broadband LED excitation source, an optional excitation filter can be placed after the excitation source to select a specific wavelength range for effective autofluorescence excitation. The excitation filter preferably has high transmittance around the optimal wavelength or autofluorescence excitation and preferably blocks wavelengths corresponding to possible autofluorescence emission from the sample. The excitation filter may include one or more short-pass filters, band-pass filters, and / or multi-band filters. These excitation filters can also be fixed, interchangeable, and / or adjustable / tunable. For example, an acousto-optic tunable filter (AOTF) can be used as a tunable excitation filter.

[0133] Pulsed fluorescence excitation

[0134] Figure 3A and Figure 3B Pulse trains for high signal-to-noise ratio, low-bleach FE-PTIR are shown, with each pulse train showing pulses along a common timeline.

[0135] Figure 3A This is a timing diagram of IR pulse 302, sample temperature 304 (and temperature change ΔT), and fluorescence excitation 306. The fluorescence excitation pulse 306 is substantially synchronized with the IR pulse 302, which has an adjustable delay, to overlap with the maximum sample temperature rise 304.

[0136] Temperature 304 increases in response to irradiation by IR pulse 302 (indicated as ΔT). The longer pulse 302 is on, the greater the increase in sample temperature 304 will be (i.e., the larger ΔT will be). Once IR pulse 302 is turned off, sample temperature 304 will decrease back to baseline level. This decrease is not immediate, therefore Figure 3A The peak associated with ΔT appears only at the end of the 302 pulse.

[0137] Figure 3B It shows Figure 3A The pattern is shown, but on a longer timeline. For each camera exposure, thousands of IR and fluorescence excitation pulses (302, 306) are accumulated. When the IR pulse is turned on, fluorescence emission 308 changes in the IR absorption region of the sample due to the temperature-dependent emission efficiency of the fluorophore. For example, for a typical fluorophore near room temperature, the fluorescence emission efficiency changes with temperature by approximately 1% / °C, meaning that for every degree Celsius increase in temperature due to IR absorption, fluorescence emission from the sample will decrease by approximately 1%. The camera frames alternate between IR-on (hot frame 310) and IR-off (cold frame 312), and the IR absorption pattern can be calculated based on the emission difference between the hot and cold frames. Any number of hot / cold frames can be added together to achieve the desired SNR level. Note that for the cold frames, it is not necessary to completely turn off the IR beam. Simply modulating / attenuating the IR beam to a lower power level than the hot frames is sufficient.

[0138] Benefits of pulsed fluorescence excitation

[0139] The use of pulsed IR sources, or otherwise modulated IR sources, was discussed above in the context of generating signals indicating changes in fluorescence emission from the sample. This section outlines techniques for using pulsed fluorescence excitation sources to achieve additional benefits, providing advantages for two types of photothermal IR measurements performed using both autofluorescence and the addition of external fluorophores. Pulsed fluorescence excitation sources are often ideal for two main reasons. First, the fluorescence excitation pulse can be timed to correspond to the time of maximum photothermal response in sample 102 or the time of highest sample temperature increase, as discussed above. Figure 3A and Figure 3B Described in more detail. This method maximizes photothermal signal detection and reduces the effects of noise by preventing photon binding at the imaging device 122 during periods when there is almost no photothermal signal or no photothermal signal at all. Secondly, the use of a pulsed fluorescence excitation source 106 may have a significant advantage in reducing photobleaching of the sample 102.

[0140] Photobleaching refers to the situation where fluorescence emission from a sample may decrease over time in response to extended excitation radiation due to photodamage to fluorophores (autofluorescence or added fluorophores) in the sample. This is of particular concern in the case of extended measurements (e.g., hyperspectral arrays comprising multiple measurements at different infrared wavelengths). For example, it may be desirable to obtain a series of wide-field photothermal IR images within a range of IR excitation frequencies (e.g., 900 cm⁻¹ to 1800 cm⁻¹, where the spectral resolution is 4 cm⁻¹). In this case, IR absorption images will be acquired at 226 different IR wavelengths, each requiring fluorescence excitation of the sample. In this situation, if fluorescence excitation is performed using continuous wave (CW) excitation, the fluorescence emission of the sample can be substantially bleached before the hyperspectral array is complete, or alternatively, measurements obtained at later wavelengths may suffer some significantly reduced AF-PTIR / FE-PTIR sensitivity. Therefore, it is desirable to use pulsed AF excitation or otherwise modulated AF excitation to generate a signal that optimizes the detection of the maximum photothermal response of the sample and avoids unnecessary bleaching of the sample when a minimum photothermal signal is present. Thus, it is desirable that the fluorescence excitation source 106 be pulsed to correspond to the timing of the peak photothermal signal, thereby corresponding to an increase in the peak temperature in the sample 102.

[0141] Therefore, the pulse generator 108 can be configured to drive both the IR source 104 and optionally the fluorescence excitation source 106. The pulse generator 108 typically provides a variable delay time between the pulses sent to the IR source and the pulses sent to the fluorescence excitation source to compensate for any delay differences between the trigger-to-light emission timing of the two radiation sources, and also times the fluorescence excitation source to emit at a time substantially corresponding to the maximum photothermal response from the sample, i.e., the maximum change in fluorescence emission due to absorption of IR radiation.

[0142] Compared to equivalent systems using a CW source, pulsed fluorescence excitation provides an improved signal-to-noise ratio (SNR). Note that among the benefits of pulsed fluorescence excitation are those from intrinsic, endogenous autofluorescence and those from PTIR, which enhances fluorescence using exogenous fluorophores (e.g., fluorescent dyes and fluorescent proteins). Table 1 shows the SNRσ for both CW and modulated fluorescence excitation. conve Formulas. In these formulas, γ is the photothermal emission sensitivity of the fluorophore, and ΔT ave The average temperature of the IR absorption region of the sample (e.g., 102) increases, τ r It is the thermal relaxation time of the sample absorption region, τ IR N is the time between IR pulses, and N is the number of hot / cold image pairs (see [reference]). Figure 3BC is the average number of photoelectrons per exposure performed by the camera device (e.g., 122).

[0143]

[0144] Pulsed fluorescence excitation can achieve SNR benefits from two sources: (1) at the peak sample temperature ΔT max Instead of at the average temperature increase ΔT ave (2) Measure fluorescence emission at the location; and (3) by avoiding τ r / τ IR To improve SNR, τ r / τ IR The term arises in the CW case due to the integral of photon shot noise during the period following thermal relaxation when no photothermal signal is present. FE-PTIR and AF-PTIR typically use an IR laser repetition rate of 100 kHz (with τ). IR =10μs corresponding) is executed, while the typical thermal decay time is about 1μs or less, therefore τ r / τ IR <0.1. Therefore, τ was eliminated by pulsed fluorescence excitation. r / τ IR This discount provides >10 times better SNR. In ΔT max Measurements at this location can provide a further improvement of approximately 2 times compared to CW. Photobleaching will also be reduced by 10 times because the sample is not exposed to fluorescence excitation during the dead time between IR pulses. Tests show that this will be sufficient to obtain samples with at least 4 cm⁻¹ fluorescence. -1 A high-spectral-resolution array that minimizes bleaching of many common fluorophores and fluorescent proteins as well as endogenous spontaneous fluorescence.

[0145] Figure 4A and Figure 4B An example AF-PTIR measurement of a sample of mouse brain tissue is shown. AF image pairs are accumulated with the IR beam alternately turned off and on (i.e., continuously alternating cold and hot frames), as shown regarding... Figure 3A and Figure 3B As described. For a total measurement integration time of 10 seconds, the exposure time per frame is 100 milliseconds, and 50 hot / cold image pairs are accumulated. The number of hot / cold image pairs can be as low as one or as high as required to achieve the desired signal-to-noise ratio. In practice, 10 to 200 image pairs are typically acquired, therefore, 2 to 40 seconds are typically required for each photothermal IR absorption image. Below is... Figure 4A and Figure 4B The image shown is 66.5 μm wide and has 512 x 512 pixels, thus creating an IR absorption image of 262,144 locations on the sample that are measured simultaneously in parallel.

[0146] In particular, Figure 4A The IR absorption at 1660 cm⁻¹ is shown, which corresponds to the amide I absorption band in the protein as a result of vibrational resonance. Figure 4B An IR absorption image of the same sample is shown, but in which the IR source is tuned to 900 cm⁻¹, where there is negligible absorption in biological tissue, thus producing a essentially blank image.

[0147] Figures 5A to 5D AF-PTIR imaging of mouse brain tissue is shown. Figure 5A The image shows an autofluorescence image of mouse brain tissue excited at approximately 482 nm. Figure 5B The same sample was shown, which was irradiated at 1656 cm⁻¹ against FE-PTIRIR absorption, and an image of the autofluorescence difference between IR-off and IR-on was constructed by accumulating the autofluorescence difference. Figure 5C It shows Figure 5A and Figure 5B The ratio of the autofluorescence difference images at 1656 cm⁻¹. The ratio is normalized to the variation in autofluorescence intensity to provide a plot of IR absorption only. Figure 5C The image depicted shows a brighter area in the center, associated with higher IR intensity at the image center. The ratio of acquiring AF-PTIR / AF images at two different IR wavelengths eliminates the effect of variable IR beam intensity, effectively providing flat-field correction. Figure 5D The ratio of the AF-PTIR / AF image (amide II band) at 1544 cm⁻¹ to the AF-PTIR / AF image (amide I band) at 1656 cm⁻¹ is shown. In this region of the sample, the amide II / amide II ratio is fairly constant, resulting in a mostly flat image.

[0148] Figures 6A to 6C Another sample is shown, namely the human stratum corneum at three different excitation wavelengths. Figure 6A It shows 1658cm -1 The sample at that location, Figure 6B It shows 1543cm -1 The samples at the location, and Figure 6C It shows 1238cm -1 The sample at that location.

[0149] Figure 7Another example of measurement is shown, depicting an AF-PTIR image of an algal sample under these conditions. Chlorophyll in algae produces a strong autofluorescence signal, which is highly temperature-dependent and therefore a strong indicator of IR absorption. Measurements of chlorophyll and the autofluorescence of chlorophyll-containing plants provide insights into the photosynthetic process and serve as indicators of plant health, as described in, for example, in the following literature: Sánchez-Moreiras et al., “Imaging of Chlorophyll a Fluorescence in Natural Compound-Induced Stress Detection” (Front. Plant Sci., December 21, 2020, Sec. Technical Advances in Plant Science, https: / / doi: / 10.3389 / fpls.2020.583590); and L. Donaldson, “Autofluorescence in Plants” (Molecules 2020, 25(10), 2393, https: / / doi.org / 10.3390 / molecules25102393), both of which are incorporated herein by reference. The AF-PTIR method and apparatus described in this article offer significant benefits for the study of plants, naturally photosynthetic materials, and other materials under investigation known as "artificial photosynthesis," which refers to biomimetic systems that artificially mimic aspects of natural photosynthesis for the purpose of light capture and energy storage / conversion. In each of these applications, the AF-PTIR instrument provides measurements of autofluorescence, offering the benefits outlined above, as well as simultaneous and co-located IR absorption measurements, providing additional supplementary information about the chemical structure of the sample.

[0150] Figure 7 A to Figure 7 D shows AF-PTIR IR absorption images of the algal sample at four example IR wavelengths. Note how the contrast decreases at 1800 cm⁻¹, where the IR source has minimum power and the sample has a low absorbance. For these measurements, autofluorescence was excited using a center wavelength of 628 ± 20 nm, and autofluorescence was detected using a center wavelength of 692 ± 20 nm. The photothermal IR images were obtained by accumulating the difference between thermal and cold images (i.e., “hot” frames where an IR pulse train was delivered to the sample and “cold” frames where no such IR pulse train was delivered to the sample). Figure 7The images shown were collected for each wavelength using 50 image pairs, with an integration time of 100 ms for each image. For strong autofluorescence, acceptable IR images can be achieved using as few as one hot / cold image pair. These acceptable IR images can be acquired in some embodiments using existing s-CMOS imaging devices and LED illuminators for fluorescence excitation with an IR image acquisition time as low as 200 ms. These images are 512 x 512 pixels and 65.6 μm wide, giving a pixel resolution of 0.13 μm. Generation is performed using a 50 x 0.8 NA refractive objective for fluorescence excitation / collection and a 25 mm focal length zinc selenide (ZnSe) lens from below for focusing the IR light. Figure 7 The image was measured. The ZnSe lens was operated several hundred micrometers away from the optimal focal point to provide more uniform IR illumination within a field of view of >50 micrometers.

[0151] Figure 8 This paper illustrates the extraction of IR absorption spectra from hyperspectral stacking of AF-PTIR images at multiple IR wavelengths. AF-PTIR IR absorption images were acquired at intervals of 4 cm⁻¹ at each wavenumber from 1000 cm⁻¹ to 1800 cm⁻¹. The intensity of IR absorption was then plotted for this wavenumber range against four different regions of interest on the algal sample. The size of the region of interest can be varied to achieve an acceptable trade-off between resolution and signal-to-noise ratio. The spectra were also corrected for any bleaching and / or excitation intensity variations in the sample by calculating the ratio of the spectrum to an independent measurement of DC autofluorescence at each pixel and wavenumber. For these measurements, hot / cold image pairs were used for each wavenumber, with an image integration time of 100 ms per wavenumber, i.e., a total acquisition time of 2 seconds for each IR absorption image.

[0152] Figure 4 to Figure 7 One important point to note is that there are no coherent interference artifacts. This is because spontaneous fluorescence emission occurs at a wavelength different from the excitation wavelength, therefore the emitted radiation cannot coherently interfere with the primary or secondary reflections of the excitation wavelength.

[0153] As can be understood from the applicant’s prior work (such as U.S. Patent No. 10,942,116B2, Figure 21) as described above and incorporated by reference, different excitation wavelengths can be used and layered to create hyperspectral images.

[0154] Furthermore, by constructing IR absorption spectra from regions of interest in AF-PTIR images, the detection of specific materials in samples, as described in this paper, can be targeted at specific regions.

[0155] Furthermore, improved image stacking can be performed by stacking images within the range of excitation wavelengths.

[0156] The following examples illustrate implementation methods within the scope of this invention.

[0157] Example

[0158] Example 1.

[0159] This document describes an apparatus for performing infrared imaging of a sample, comprising a tunable autofluorescence excitation source used in the manner described above. The infrared source described in Example 1 is modulated, which may mean that either a pulsed source (i.e., the source itself creates periodic pulses) is used or the beam from the source is modulated (i.e., the source is constant or emits pulses at a slower rate, and external devices such as a chopper are used to create the desired periodicity and wave shape).

[0160] The device in Example 1 includes:

[0161] A tunable excitation radiation source is configured to irradiate a region of a sample with an excitation radiation beam to excite spontaneous fluorescence emission in the sample.

[0162] The detector is configured to: capture at least a portion of the autofluorescence emission from the sample; and generate a first frame of autofluorescence image of a region of the sample;

[0163] A modulated infrared source is configured to irradiate a region of the sample with an infrared beam to create an infrared irradiated region of the sample.

[0164] A detector configured to: capture at least a portion of the autofluorescence emission from an infrared-illuminated region of the sample; and generate a second frame of autofluorescence image of the region of the sample; and

[0165] The processor is configured to: construct an infrared absorption signal indicating the region of the image using a first frame autofluorescence image of a region of the sample and a second frame autofluorescence image of a region of the sample.

[0166] In an implementation, the tunable excitation radiation source in Example 1 can be configured to operate in the range of about 400 nm to about 700 nm.

[0167] In an implementation, the tunable excitation source in Example 1 can be configured to generate light of multiple wavelengths within the range, and the device further includes a bandpass filter configured to selectively allow a narrower wavelength range within the range.

[0168] Example 2.

[0169] Example 2 describes a device for performing infrared imaging of a sample, where detection is performed by point rather than as a wide field-of-view detection method. For example, in Figures 1B to 1F The diagram illustrates a point-based system. Example 2's device includes:

[0170] The modulated radiation source is configured to irradiate a region of the sample with a pulsed excitation radiation beam to excite spontaneous fluorescence emission in the sample.

[0171] The detector is configured to: capture at least a portion of the autofluorescence emission from the sample; and generate a first frame of autofluorescence image of a region of the sample;

[0172] A modulated infrared source is configured to illuminate an infrared irradiation region of a sample using a pulsed infrared beam, the infrared irradiation region including a point region of the sample.

[0173] A detector configured to: capture at least a portion of the autofluorescence emission from a dotted region of the sample; and generate a second frame of autofluorescence image of the dotted region of the sample; and

[0174] The processor is configured to construct a signal indicating the infrared absorption of the sample by combining multiple sets of first and second frames at different point regions on the sample.

[0175] Although not listed and described separately in this article, those skilled in the art will recognize the various modifications to the system, including the example described above with respect to the wide field-of-view measurement device of Example 1.

[0176] Example 3.

[0177] Example 3 describes an apparatus for performing infrared imaging of a sample. Example 3 (like Example 2) relates to a point-by-point detection system, rather than a wide-field-of-view detection system. The apparatus of Example 3 includes a tunable excitation source, allowing the same tunable source to be used to detect the autofluorescence of various compounds or materials within a single sample. The apparatus of Example 3 includes:

[0178] A tunable excitation radiation source is configured to irradiate a point region of a sample with an excitation radiation beam to excite spontaneous fluorescence emission in the sample.

[0179] The detector is configured to: capture at least a portion of the autofluorescence emission from the sample; and generate a first frame of autofluorescence image of a dotted region of the sample;

[0180] A modulated infrared source is configured to irradiate an infrared irradiation region of the sample, the infrared irradiation region including a point region of the sample.

[0181] A detector configured to: capture at least a portion of the autofluorescence emission from an infrared dot region of the sample; and generate a second frame of autofluorescence image of the dot region of the sample; and

[0182] The processor is configured to construct a signal indicating the infrared absorption of the sample by combining multiple sets of first and second frames at different point regions on the sample.

[0183] Although not listed and described separately in this article, those skilled in the art will recognize the various modifications to the system, including the example described above with respect to the wide field-of-view measurement device of Example 1.

[0184] Example 4.

[0185] Example 4 describes a method for performing infrared imaging of a sample, illustrating what happens within a specific sample according to an embodiment of the invention. In Example 4, the method includes:

[0186] A radiation source with a modulated excitation beam is used to irradiate a region of the sample to excite spontaneous fluorescence emission in the sample; then

[0187] Without infrared heating, light indicating autofluorescence emission from the sample is collected as a cold frame sample;

[0188] Irradiate the sample area using an infrared source; then

[0189] After infrared heating, light indicating autofluorescence emission from the sample is collected as a thermal frame of the sample; and

[0190] Infrared absorption signals indicating regions of the sample are constructed by comparing cold-frame and hot-frame samples.

[0191] In implementation, the target category includes at least one material selected from the group consisting of: NAD(P)H bound, NAD(P)H free, fatty acids, vitamins, flavonoids, proteins, chlorophyll or other biological materials that convert light into energy, and porphyrins.

[0192] In one embodiment, using a radiation source to generate a modulated excitation radiation beam to excite spontaneous fluorescence emission in a sample includes tuning the tunable excitation radiation source to a wavelength that substantially maximizes the transmission of the peak amplitude of the emission spectrum of the sample.

[0193] Maximizing the transmission of the peak amplitude of the sample's emission spectrum can include using a bandpass filter that is substantially aligned with the center of the local emission peaks of the substance within the sample. Alternatively, such as... Figure 2As shown, using a tunable source (or using a bandpass filter) to select a wavelength can be used to measure a peak that deviates at least slightly from the maximum value of a particular material, in order to increase a portion or quality of the signal from the material of interest relative to other signals or noise of no interest.

[0194] The output measured in Example 4 can indicate the infrared absorption of the sample in the region that overlaps with the first and second regions.

[0195] The first and second datasets may each include an image, an array, or another mapping of the locations on the sample where absorption or autofluorescence emission occurs.

[0196] Example 5.

[0197] According to the fifth example, the method includes: ensuring that a region of the sample is not irradiated for a certain time period based on the thermal relaxation time of the sample between excitation radiation pulses; collecting autoemission data at a first infrared wavelength and repeating the method at a second wavelength; and calculating the ratio at each point in the image to remove artifacts caused by variations in IR beam intensity or autofluorescence emission.

[0198] It should be understood that this method is compatible with the other methods or devices described above and can be performed as a calibration step in some embodiments. This calibration can eliminate the effects of scattering or reflection due to sample parameters, which is particularly prevalent in biological materials that are not always flat and may always have various refractive indices.

[0199] Example 6.

[0200] According to Example 6, a method for performing infrared imaging of a sample is disclosed, the method comprising: calculating a ratio as described in Example 5 to remove artifacts caused by variations in IR beam intensity or autofluorescence emission. The method of Example 6 includes:

[0201] (a) Irradiate a region of the sample with a radiation source having an excitation radiation beam at a first wavelength to excite spontaneous fluorescence emission corresponding to a first expected material in the sample;

[0202] (b) Collect light from the sample indicating autofluorescence emission as a cold frame sample without infrared heating;

[0203] (c) Irradiate the area of ​​the sample using an infrared source; then

[0204] (d) After being heated in the infrared, light from the sample indicating autofluorescence emission is collected as a thermal frame sample;

[0205] (e) Compare the hot frame sample and the cold frame sample to identify the location of the first expected material in the region of the sample;

[0206] (f) Repeat (a) through (e) at the second wavelength to excite autofluorescence emission corresponding to the second expected material in the sample; and

[0207] (g) Calculate the ratio between the identification location of the first expected material and the identification location of the second expected material at each point in the image to remove artifacts caused by changes in IR beam intensity or autofluorescence emission.

[0208] Other changes and alternatives

[0209] Depending on the implementation, specific actions, events, or functions of any method step described herein may be performed in a different order, may be added, combined, or may be omitted entirely (e.g., not all described actions or events are necessary for the implementation of the algorithm). Furthermore, in some implementations, the actions or events described herein may be performed concurrently rather than sequentially.

[0210] The various illustrative logic blocks, optical elements, and control elements, as well as method steps described in conjunction with the embodiments disclosed herein, can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, various illustrative components, blocks, modules, and steps have been described above in general terms of their functionality. Whether such functionality is implemented as hardware or software depends on the design constraints imposed on the entire system and the specific application. The described functionality can be implemented in different ways for each specific application, but such implementation decisions should not be construed as departing from the scope of this disclosure.

[0211] The various illustrative logic blocks and modules described in conjunction with the embodiments disclosed herein can be implemented or executed by a machine, such as a processor configured with specific instructions, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The processor may be a microprocessor, but alternatively, it may be a controller, a microcontroller, or a state machine, a combination thereof, etc. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0212] Elements of the methods, processes, or algorithms described in conjunction with the embodiments disclosed herein may be directly contained in hardware, in a software module executed by a processor, or a combination of both. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of computer-readable storage medium known in the art. An exemplary storage medium may be coupled to a processor such that the processor can read information from and write information to the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The software module may include computer-executable instructions that cause the hardware processor to perform computer-executable instructions.

[0213] Unless otherwise specified, the conditional language used herein, such as “can,” “may,” “may,” “for example,” etc., or otherwise understood in the context in which they are used, is generally intended to convey that a particular implementation includes certain features, elements, and / or states, while other implementations do not include certain features, elements, and / or states. Therefore, such conditional language is not generally intended to imply that features, elements, and / or states are required in any way for one or more implementations, or that one or more implementations must include logic for determining, with or without author input or prompting, whether such features, elements, and / or states are included in any particular implementation or to be performed in any particular implementation. The terms “comprising,” “including,” “having,” “involving,” etc., are synonyms and are used inclusively in an open-ended manner, without excluding additional elements, features, behaviors, operations, etc. Furthermore, the term “or” is used in its inclusive meaning (rather than its exclusive meaning) such that, for example, when the term “or” is used to connect a list of elements, the term “or” means one, some, or all of the elements in the list.

[0214] Unless otherwise explicitly stated, disjunctive languages ​​such as the phrase “at least one of X, Y, or Z” are understood in the context to generally indicate that an item, term, etc., can be X, Y, or Z or any combination thereof (e.g., X, Y, and / or Z). Therefore, such disjunctive languages ​​are generally not intended and should not imply that a particular implementation requires the presence of at least one of X, at least one of Y, or at least one of Z.

[0215] Unless otherwise explicitly stated, articles such as “a” or “an” should generally be interpreted as including one or more described items. Therefore, phrases such as “a device configured to…” are intended to include one or more described devices. Such devices of one or more descriptions can also be collectively configured to perform the described descriptions. For example, “processors configured to perform descriptions A, B, and C” could include a first processor configured to perform description A working in conjunction with a second processor configured to perform descriptions B and C.

[0216] Any inclusion by reference to the foregoing documents is restricted so that it does not include subject matter contrary to the express disclosure herein. Any inclusion by reference to the foregoing documents is further restricted so that the claims included in those documents are not incorporated herein by reference. Any inclusion by reference to the foregoing documents is further restricted so that any definitions provided in those documents are not incorporated herein by reference unless expressly included herein.

[0217] For the purposes of interpreting the claims, unless the specific terms "means for..." or "steps for..." are recited in the claims, it is expressly intended that the provisions of Section 112, paragraph 6 of 35 U.S.SC should not be invoked. While the detailed description above has shown, described, and pointed out novel features applicable to the illustrative embodiments, it will be understood that various omissions, substitutions, and changes may be made to the form and details of the illustrated apparatus or method without departing from the spirit of this disclosure. As will be appreciated, certain embodiments described herein may be embodied in a form that does not provide all the features and benefits set forth herein, because some features can be used or practiced separately from other features. All changes falling within the equivalent meaning and scope of the claims should be included within their scope.

Claims

1. A method for performing infrared imaging of a sample, the method comprising: The first region of the sample is irradiated using an infrared source that generates a modulated infrared beam; The sample is irradiated with an excitation radiation beam in a second region that at least partially overlaps with the first region to excite spontaneous fluorescence emission; At least a portion of the spontaneous fluorescence emission from the second region is detected as the detected spontaneous fluorescence emission by at least one array-based detector; The first frame of autofluorescence dataset of the second region is generated using the detected autofluorescence emission of the sample at a first value of infrared beam energy; The detected spontaneous fluorescence emission is used to generate a second frame of spontaneous fluorescence dataset for the second region with a second value of infrared beam energy; as well as The first frame of autofluorescence dataset and the second frame of autofluorescence dataset are used to construct an output indicating the infrared absorption of the sample.

2. The method according to claim 1, wherein, The detected autofluorescence emission includes the detection of target categories selected from the group consisting of: NAD(P)H bound, NAD(P)H free, fatty acids, vitamins, flavonoids, proteins, porphyrins, and chlorophyll.

3. The method according to claim 1, wherein, The output configuration includes: substantially maximizing the transmission of the peak amplitude of the autofluorescence emission spectrum of the sample.

4. The method according to claim 3, wherein, Maximizing the transmission of the peak amplitude of the autofluorescence emission spectrum of the sample includes using a bandpass filter that is substantially aligned with the center of the local emission peak of the substance within the sample.

5. The method according to claim 1, wherein, Filtering the light includes: making the light transmitted at a wavelength substantially greater than the wavelength of the autofluorescence emission of a first substance in the sample than that of the autofluorescence emission of a second substance in the sample, wherein the wavelength of the transmitted light is not a local peak amplitude of the emission spectrum of the first substance.

6. The method according to claim 1, wherein, The output indicates the infrared absorption of the sample in the region that overlaps with the first and second regions.

7. The method according to claim 1, wherein, The first frame of autofluorescence dataset and the second frame of autofluorescence dataset each include an image, an array, or another mapping of the location on the sample where absorption or autofluorescence emission occurs.

8. An apparatus for performing infrared imaging of a sample, the apparatus comprising: An excitation radiation source is configured to irradiate a region of the sample with an excitation radiation beam to excite spontaneous fluorescence emission in the sample. A modulated infrared source is configured to irradiate a region of the sample with a modulated infrared beam to create an infrared irradiated region of the sample. At least one array-based detector is configured to: capture at least a portion of the autofluorescence emission from an infrared-illuminated region of the sample; generate a first frame autofluorescence image of the region of the sample with a first value of infrared beam energy; and generate a second frame autofluorescence image of the infrared-illuminated region of the sample with a second value of infrared beam energy. as well as The processor is configured to construct an infrared absorption signal indicating a region of the sample using the first frame of the autofluorescence image and the second frame of the autofluorescence image.

9. The apparatus of claim 8, further comprising a stage configured to hold the sample.

10. The device according to claim 9, wherein, The stage is movable relative to at least one of the modulated infrared beam and the excitation radiation beam.

11. The device according to claim 8, wherein, The modulated infrared source is a pulsed infrared source configured to generate the modulated infrared beam.

12. The device according to claim 8, wherein, The modulated infrared source is a continuous wave source paired with a chopper to generate the modulated infrared beam.

13. The device according to claim 8, wherein, The excitation radiation source is modulated.

14. The device of claim 8, further comprising at least one pulse generator operatively coupled to the modulated infrared source and the excitation radiation source, wherein, The at least one pulse generator is configured to control the timing of the modulated infrared source and the excitation radiation source with an adjustable delay.

15. The device according to claim 14, wherein, The processor is configured to construct a signal indicating infrared absorption in the region by controlling an array-based detector to obtain a first frame autofluorescence image and a second frame autofluorescence image based on the timing of the modulated infrared source and the excitation radiation source.

16. The apparatus of claim 8, further comprising an infrared objective lens configured to: receive the modulated infrared beam; and focus the modulated infrared beam onto a region of the sample.

17. The apparatus of claim 8, further comprising an excitation beam objective configured to: receive an excitation radiation beam; and focus the excitation radiation beam onto a region of the sample, and the excitation beam objective further configured to: receive autofluorescence emission from the sample; and transmit the received autofluorescence emission toward the array-based detector.

18. The apparatus of claim 17, further comprising a dichroic mirror disposed between the excitation beam objective and the array-based detector, wherein, The dichroic mirror is selected as follows: Most of the light reflecting the wavelength of the excitation radiation beam; and Most of the light transmitted has the wavelength of the spontaneous fluorescence emission.

19. The apparatus of claim 17, further comprising an autofluorescence filter disposed between the excitation beam objective and the array-based detector.

20. The device according to claim 19, wherein, The self-fluorescent filter is a long-pass filter.

21. The device according to claim 8, wherein, The processor is configured to construct an infrared absorption signal indicating the region by detecting fluorescence at various wavelengths, each wavelength corresponding to a specific autofluorescence wavelength of the substance.

22. The device according to claim 8, wherein, At least one of the first value or the second value of the infrared beam energy is substantially zero.

23. The device according to claim 8, wherein, The excitation radiation source includes at least one light-emitting diode.

24. The device according to claim 23, wherein, The at least one light-emitting diode emits excitation radiation of at least one wavelength in the range of 250 nm to 400 nm.

25. The device according to claim 24, wherein, The at least one light-emitting diode is pulsed.

26. The device according to claim 25, wherein, The at least one light-emitting diode emits a pulse with a pulse duration in the range of 100 nanoseconds to 100 microseconds.

27. The device according to claim 8, wherein, The at least one array-based detector is configured to detect the autofluorescence response of chlorophyll or other organic photosynthetic materials or artificial photosynthetic materials.

Images