Systems and methods for spectral phasor unmixing

Abstract

An optical detection system for fluorescence unmixing, including: a diffraction grating configured to spectrally disperse fluorescence light; a light filter configured to receive the spectrally dispersed light, the light filter configured to divide the spectrally dispersed light into a plurality of different portions of light; one or more detectors configured to detect the plurality of different portions of light from the light filter; and a controller configured to unmix the fluorescence light based on signals from the one or more detectors.

Description

MGH 2024-547-03Quarles 125141.04912SYSTEMS AND METHODS FOR SPECTRAL PHASOR UNMIXINGCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The present application is based on and claims priority from U.S. Patent Application Ser. No. 63 / 721,851, fded on November 18, 2024, the entire disclosure of which is incorporated herein by reference.STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

[0002] This invention was made with government support under grant number 5R01HL165453-03 (agreement 2021 A013694) awarded by the National Institutes of Health. The government has certain rights in the invention.BACKGROUND

[0003] Acquiring full emission spectra may be undesirable for point scan fluorescence applications including those in vivo (such as but not limited to esophageal capsule based, intravascular probe based, etc.). This is due to decreased photon counts and a lower resulting detection sensitivity. Array-based spectroscopic approaches involve spreading light into many channels and adequate signal levels may not be available for reliable unmixing, especially at high scan rates. Additionally, not all detector configurations exist in an array-based configuration or may exist but may be prohibitive (e.g., too expensive) for some applications. Filter-based spectral phasor unmixing is a simpler method that uses a single detector to collect fluorescence emission after it has been transmitted through cosine / sine / null optical filters. Prior filter-based spectral phasor unmixing configurations required switching filters using a mechanical filter wheel. However, this approach would be challenging with rapid, continuous beam scanning near-infrared fluorescence (N1RF) optical coherence tomography (OCT), as sequential spectral filter acquisition could slow data acquisition as well as result in spatial misregistration,1QB\99549432.1MGH 2024-547-03Quarles 125141.04912decreasing the accuracy of the unmixing results. Multi -detector configurations also exist but there are practical limits to the modulation frequency of the cosine / sine filters that can be obtained, and with filters, fabrication limitations mean that modulation depth is reduced, especially at higher modulation frequencies.SUMMARY

[0004] Accordingly, new optical detection systems, methods, and apparatus for fluorescence unmixing are desirable.

[0005] In one embodiment, an optical detection system for fluorescence unmixing includes a diffraction grating configured to spectrally disperse fluorescence light, creating a wavelengthdependent spatial distribution. A light filter receives the spectrally dispersed light and divides it into a plurality of different portions of light, enabling parallel processing of spectral information. One or more detectors detect these different portions of light, while a controller unmixes the fluorescence light based on signals from the detectors.

[0006] In one embodiment, the light filter divides the spectrally dispersed light into sine and cosine portions, which provide orthogonal basis functions for spectral decomposition. A first detector detects the sine portion and a second detector detects the cosine portion, enabling mathematical analysis based on these complementary signals. A third detector may be used to obtain normalization light which is used as part of the mathematical calculations.

[0007] The light filter may be implemented as a patterned mask including a plurality of reflective regions interspaced with a plurality of transmissive regions arranged in either a square wave pattern, or as a sine / cosine pattern with partially transmissive elements, or as a sine / cosine pattern made of binary elements having a discrete number of squares in a target wavelength range being reflective and a discrete number being reflective. That is, given that the each wavelength has been spread into a row, the light filter simply reflects and / or transmits a fraction of that row to give net sine modulation (if it was seen by a spectrometer). Each region2QB\99549432.1MGH 2024-547-03Quarles 125141.04912corresponds to a particular wavelength of the spectrally dispersed light. The first and second detectors detect light transmitted through the patterned mask, while a third detector may detect a normalization portion of light reflected from the patterned mask to provide reference calibration.

[0008] Alternatively, the light filter may include a digital micromirror device (DMD) that produces programmable patterns to divide the spectrally dispersed light. The DMD produces alternating patterns including sine patterns, cosine patterns, and normalization patterns, with bands corresponding to particular wavelengths. Detectors collect light from DMD elements in different states, with a first detector collecting light from elements in the on state and optionally a second detector collecting light from elements in the off state.

[0009] Alternatively, the DMD may have static patterns including a top sine pattern and a bottom cosine pattern with bands corresponding to particular wavelengths. In various embodiments, three or more detectors may collect light from DMD elements in different states, with a first detector collecting light from elements in the on sine state, a second detector collecting light from the on cosine state, and a third detector collecting all light relayed to the off state; optionally, the third detector may collect sine off state light and a fourth detector may collect cosine off state light.

[0010] The system may include a cylindrical mirror or cylindrical lens to direct and focus the spectrally dispersed light. In some embodiments, the light filter is positioned at the focal plane of these optical elements to ensure optimal light concentration and pattern definition. In other embodiments, the light filter may be placed out of focus to a certain degree so that square masks will modulate sinusoidally.

[0011] The corresponding method for fluorescence unmixing involves spectrally dispersing fluorescence light using a diffraction grating, receiving the spectrally dispersed light at a light filter, and dividing the light into multiple different portions. The method includes detecting these portions using one or more detectors and unmixing the fluorescence light using a controller based on the detected signals.3QB\99549432.1MGH 2024-547-03Quarles 125141.04912

[0012] The method may specifically involve dividing the light into sine and cosine portions, detecting these portions with separate detectors, and performing unmixing based on the mathematical relationship between these orthogonal components. When using a patterned mask, the method includes detecting transmitted and reflected light portions. When using a DMD, the method involves producing alternating sine, cosine, and normalization patterns and detecting light from different mirror states. Using either a mask or a DMD, detection can be performed using one or multiple detectors. In some embodiments, a DMD can be used in a manner similar to a mask (e.g., by using a fixed DMD pattern) with three or more detectors.BRIEF DESCRIPTION OF FIGURES

[0013] Various objects, features, and advantages of the disclosed subject matter can be more fully appreciated with reference to the following detailed description of the disclosed subject matter when considered in connection with the following drawings, in which like reference numerals identify like elements.

[0014] FIG. 1 illustrates an optical schematic of a configuration for a detection arm for a spectral phasor fluorescence unmixing system, according to aspects of the present disclosure. FIG. 1, panel (A) shows an optical schematic of an unmixing system; panel (B) shows a mask structure showing sine (top) and cosine (bottom) patterns; panel (C) shows a half mirror (HM) orientation relative to the mask, with the HM being aligned with the lower (cosine) portion of the mask such that the HM reflects the sine portion of the signal to the sine detector while the cosine portion bypasses the HM and is reflected to the cosine detector.

[0015] FIG. 2 shows schematic diagrams of configurations of detection arm optical systems for fluorescence unmixing which use DMD (digital micromirror device) arrays as light filters to4QB\99549432.1MGH 2024-547-03Quarles 125141.04912separate out different portions of light for detection. Panel (a) shows a configuration with a single detector and panel (b) shows a configuration with two detectors.

[0016] FIG. 3 shows a schematic diagram of another configuration of a detection arm optical system for fluorescence unmixing using a DMD array as a light filter to separate out different portions of light for detection.

[0017] FIG. 4 shows a schematic block diagram of a multimodality OCT-NIRAF catheter imaging system according to an exemplary embodiment of the present disclosure. Abbreviations: RJ - rotary junction, DCF - dual clad fiber, WDM - wavelength division multiplexer, DCFC -dual clad fiber coupler.

[0018] FIG. 5 shows an example of a system for fluorescence unmixing in accordance with some embodiments of the disclosed subject matter.

[0019] FIG. 6 shows an example of hardware that can be used to implement computing device and server in accordance with some embodiments of the disclosed subject matter.

[0020] FIG. 7 provides a comparison of fluorescence imaging results showing preincubation and post-incubation conditions for plaque samples, according to aspects of the disclosure. FIG. 7 shows images produced from an intraluminal pullback (using a 100 kHz A-line rate) showing plaques incubated with the LUMISIGHT agent. The left-hand panels show scans from two different samples prior to incubation with the agent and the right-hand panels show scans from the same samples after incubation with the agent. The top two panels show scans from a sample with a fibrocalcific plaque, and the bottom two panels show scans from a sample with a cathepsin rich lipid plaque. L - lipid rich region, C - visible bulk calcium.

[0021] FIG. 8 shows results of a benchtop validation of the disclosed procedures. The images in the top row show white light images while the images in the bottom row show fluorescence reflectance imaging of the respective samples, either before (Pre Incubation) or5QB\99549432.1MGH 2024-547-03Quarles 125141.04912after (4 Hours) incubation with Cy5-conjugated LUMISIGHT. Lipid rich (L) and fibrocalcific (FC) plaque cross sections are identified in each of the fluorescence reflectance imaging images.

[0022] FIG. 9 shows images from an in vivo scan of a rabbit aorta which have been processed using blind unmixing, which has separated out two distinct NIRAF components. The pink arrow in the left panel shows NIRAF dominated by the inner ring NIRAF component while the blue arrow in the right panel shows NIRAF dominated by the outer ring component.DETAILED DESCRIPTION

[0023] In accordance with some embodiments of the disclosed subject matter, optical detection mechanisms (which can include systems, methods, and apparatus) for fluorescence unmixing are provided.

[0024] Embodiments of the present disclosure relate to optical detection systems and methods for fluorescence unmixing that enable high-speed spectral analysis of fluorescence signals. In various embodiments, the technology addresses a need for rapid and accurate separation of overlapping fluorescence spectra, particularly in applications such as optical coherence tomography (OCT) fluorescence systems where high-speed data acquisition analysis is important.

[0025] Embodiments of the optical detection system include several components working together to collect data suitable for fluorescence unmixing calculations (FIGS. 1-3). One component of the system is a diffraction grating which receives incoming fluorescence light and spectrally disperses it according to wavelength. This spectral dispersion creates a spatial separation of different wavelengths which facilitates subsequent spectral data collection and analysis.6QB\99549432.1MGH 2024-547-03Quarles 125141.04912

[0026] The spectrally dispersed light is then directed to a light filter, which is configured to divide the spectrally dispersed light into multiple different portions, each containing distinct spectral information that can be processed using an unmixing process.

[0027] Embodiments of the system include two different types of light filters, patterned masks, spatial light modulators to include liquid crystal displays, and digital micromirror devices, although other light filters are also possible. In one embodiment, the light filter includes a patterned mask featuring a plurality of reflective regions interspaced with a plurality of transmissive regions. These regions may be arranged in square wave patterns, with each region corresponding to a particular wavelength of the spectrally dispersed light. This spatial correspondence ensures that specific wavelengths are either transmitted through or reflected from designated areas of the mask. The use of the patterned mask to reflect or transmit the spectrally dispersed light from the diffraction grating enables the creation of sine and cosine portions of the spectrally dispersed light which provides orthogonal basis functions for subsequent spectral unmixing / decomposition calculations.

[0028] In various embodiments, the system employs one or more detectors to capture different portions of light that are either transmitted through or reflected from the patterned mask. A first detector is positioned to detect the sine portion by collecting light transmitted through specific regions of the mask. A second detector similarly detects the cosine portion from complementary transmitted regions. Additionally, a third detector may be positioned to detect a normalization portion including light reflected from the mask, providing a reference signal for calibration and improved accuracy.

[0029] In another embodiment, the light filter utilizes a digital micromirror device (DMD), a type of spatial light modulator (SLM). Compared to the patterned mask which is fixed, the DMD offers advantages in terms of flexibility and programmability, as it can produce various patterns electronically without requiring physical modification of optical components.7QB\99549432.1MGH 2024-547-03Quarles 125141.04912

[0030] In various embodiments, the DMD is configured to produce alternating patterns (e.g., stripes) that include sine patterns, cosine patterns, and normalization patterns. These patterns are created by controlling individual micromirrors within the DMD, with each element of the pattern (e.g., each stripe) corresponding to a specific wavelength of the spectrally dispersed light.

[0031] In embodiments of the DMD implementation, detectors collect light from micromirror elements that are in different states. A first detector collects light from elements in the "on" state while a second detector collects light from elements in the "off state. This dualdetection approach maximizes the utilization of available light (e.g., compared to procedures only collecting light from elements that are in the “on” state) and thus can improve signal-to-noise ratios.

[0032] The system may incorporate additional optical elements to optimize light collection and focusing. In one embodiment, a cylindrical mirror may be positioned to receive spectrally dispersed light from the diffraction grating, providing focus in one dimension (the power axis) while maintaining spectral dispersion in the perpendicular dimension. In some embodiments, the light filter is disposed at the focal plane of the cylindrical mirror to ensure optimal light concentration and pattern definition. In some embodiments, the light filter may be defocused to a certain degree to change the square wave pattern (e.g., from a patterned mask) to a gaussian pattern.

[0033] In another embodiment, a cylindrical lens may be used to direct spectrally dispersed light from the diffraction grating onto the DMD. Similar to the cylindrical mirror configuration, the DMD may be positioned at the focal plane of the cylindrical lens to achieve proper focusing and pattern formation, although in certain embodiments it may alternatively be defocused to produce a gaussian pattern.

[0034] The detection system includes one or more detectors (e.g., photomultiplier tubes, PMTs, Indium Gallium Arsenide, InGaAs, Avalanche Photodiode (APD)) configured to capture spectral information from the respective portions of divided light. These detectors convert optical 8QB\99549432.1MGH 2024-547-03Quarles 125141.04912signals (e.g., a sum of all of the modulated light in the sine and / or cosine bands) into electrical signals that can be collected and processed by the controller to perform spectral unmixing. In various embodiments, the detectors are selected for high-speed operation to enable the other all optical system (e.g., an OCT-based optical system) to achieve acquisition rates at or above 100 kHz.

[0035] The controller processes signals from the detectors, utilizing the sine and cosine portions, and optionally the normalization portions, to perform spectral decomposition. The mathematical relationship between these portions enables the controller to separate overlapping fluorescence spectra and identify individual fluorophore contributions using spectral phasor analysis procedures for unmixing (e g., as disclosed in Fereidouni et al., “Spectral phasor analysis allows rapid and reliable unmixing of fluorescence microscopy spectral images,” Optics Express Vol. 20, Issue 12, pp. 12729-12741 (2012), incorporated by reference herein in its entirety).

[0036] In various embodiments, procedures for fluorescence unmixing start with spectral dispersion of incoming fluorescence light using a mechanism such as a diffraction grating. This creates a wavelength dependent spatial distribution of light emitted from the sample which facilitates subsequent processing steps. The spectrally dispersed light is received at the light filter, which divides it into multiple different portions (e.g., sine, cosine, and / or normalization) according to the implemented pattern (either patterned mask or DMD-based). This division process separates the light by wavelength as well as by sine and cosine, ensuring the portions of light contain distinct spectral information. The detection step involves capturing the different portions using one or more appropriately positioned detectors, which may occur simultaneously for all portions to enable real-time processing of spectral data.

[0037] As noted herein, the unmixing process utilizes the controller to analyze the detected signals and perform spectral decomposition. This involves mathematical operations that leverage9QB\99549432.1MGH 2024-547-03Quarles 125141.04912the orthogonal nature of the sine and cosine portions, along with normalization data, to separate individual fluorophore contributions from the composite fluorescence signal.

[0038] An advantage of the disclosed technology is its ability to operate at extremely high acquisition rates, exceeding 100 kHz. This high-speed capability is achieved through the parallel processing architecture enabled by the light filter design, that is, by the fact that the light is spectrally dispersed and subsequently filtered by a light filter that does not require mechanical switching (e.g., a patterned mask) or by a suitable high-speed mechanism (e.g., a DMD array) and spectrally detected (e.g., by a PMT). Thus, rather than sequentially analyzing different spectral components, the system simultaneously processes multiple portions of the spectrum which reduces the time required for complete spectral analysis. The high-speed operation makes the system particularly suitable for dynamic applications where fluorescence signals may change rapidly, such as in live cell imaging or real-time monitoring of biological processes.

[0039] In various embodiments, the system can be integrated with other imaging modalities, for example to obtain structural imaging information such as optical coherence tomography (OCT) fluorescence systems. In applications such as these, the fluorescence light may be obtained simultaneously with OCT measurements, providing both structural and functional information about biological samples.

[0040] The detectors are configured to capture comprehensive spectral information from their respective portions of divided light along with intensities at each wavelength. The ability of the optical detection system to facilitate collection of spectral information at high acquisition rates makes it suitable for various applications in biomedical imaging, fluorescence microscopy, and spectroscopic analysis where acquisition speed is important.

[0041] Probe based optical coherence tomography (OCT) with fluorescence enables the imaging of tissue morphological and molecular signatures. Emerging fluorescent dyes can image deeper in tissue using near-infrared light. However, emitted fluorescence can spectrally overlap10QB\99549432.1MGH 2024-547-03Quarles 125141.04912with endogenous near-infrared autofluorescence and thus unmixing of signals is important to be able to distinguish between signals from different sources.

[0042] Typically, a spectrum is acquired, and fluorophores are unmixed numerically after first acquiring a full spectrum. When combined with other modalities, such as in intravascular applications, an 80 mm portion of artery may be helically scanned but the artery can only be cleared of blood for a short amount of time for the procedure. This necessitates high OCT A-line rates (>100 kHz) and similarly high fluorescence point rates, which puts limits on the use of spectrometers. Further complicating matters, dyes that have fluorescence emission light which falls outside of the visible wavelength range can require specialized detectors (e.g. PMT, APD, InGaAs). While arrays and spectrometers do exist to cover these wavelengths, they are relatively expensive. Accordingly, systems are needed that allow for high-throughput unmixing irrespective of the detector used.

[0043] Thus, disclosed herein are one or more systems, methods, and apparatus which use an all-optical, detector-agnostic configuration suitable for real time unmixing. In some embodiments the configuration may be all-reflective. In certain embodiments, light filtration may be provided by various alternatives including in some embodiments a light filter that does not require mechanical filter switching such as a mask (e.g., aluminum deposited on glass) or in other embodiments a spatial light modulator (SLM, e.g., a digital micromirror device (DMD)) which in certain configurations can simultaneously detect cosine / sine filtered emission spectra and total transmission.

[0044] FIG. 1 illustrates an optical schematic of a spectral phasor fluorescence unmixing system, according to aspects of the present disclosure. FIG. 1A shows an optical schematic of an unmixing system; FIG. IB shows a mask structure showing Sin (top) and cosine (bottom) patterns; and FIG. 1C shows a half mirror orientation relative to the mask. In FIG. 1 A, a multimode fiber (MMF) from system light (e.g., light collected from a sample) is collimated by a11QB\99549432.1MGH 2024-547-03Quarles 125141.04912parabolic mirror (PM1) and directed to a grating (G). The grating diffracts out the collimated light and a cylindrical mirror (CM) focuses the light in one dimension (FIG. 1A shows a cross-sectional view through the power (focusing) axis of the CM) and a mask (M) is placed at the focal plane of the CM. Transmitted light is separated by a half mirror (HM), clipping sine / cosine masked light and guiding it to respective detectors (D) labeled “Sin” and “Cos,” respectively. That is, the HM is aligned with the lower (cosine) portion of the mask such that the HM reflects the cosine portion of the signal to the parabolic mirror PM3 which in turn directs the light to the cosine detector, while the sine portion bypasses the HM and is reflected via parabolic mirror PM4 to the sine detector. Finally, light reflected off the mask is collected via parabolic mirror PM2 and directed to a normalization detector (D), labeled “Norm.”

[0045] FIG. 2 shows schematic diagrams of configurations of detection arm optical systems for fluorescence unmixing which use DMD (digital micromirror device) arrays as light filters to separate out different portions of light for detection. A DMD array may include hundreds of thousands of individually-controllable mirrors which can be rapidly switched into two positions, referred to as the “on” state and the “off’ state. In various embodiments, the two DMD mirror positions differ from one another by a particular angle (e.g., 10-12 degrees) and so in certain embodiments, optical components and detectors can be configured to separately collect light from the mirrors in the on and off states by locating them in different positions relative to the DMD array, as noted herein.

[0046] FIG. 2a shows a single-detector configuration and FIG. 2b shows a dual-detector configuration. In both configurations, fluorescence emission or reflectance is delivered from a multimode input fiber. In each configuration the light is first collimated (e.g., using a collimating lens) and then is directed onto a transmission diffraction grating and a cylindrical lens, which focuses the dispersed wavelengths onto the DMD (lens power axis indicated in FIGS. 2a, 2b). Depending on the orientation of the micromirrors, light is directed either to a single detector in12QB\99549432.1MGH 2024-547-03Quarles 125141.04912the “on” state (single-detector configuration, FIG. 2a), or to separate detectors for the “on” and “off’ states (dual-detector configuration, FIG. 2b); the collection lenses serve only to relay light to the detectors. While the embodiments of FIGS. 2a and 2b use transmission optical components, reorienting the input fiber and collimating optics enables the use of a reflective grating, as in the configuration of FIG. 1, instead of a transmission grating.

[0047] When using a single detector, the pattern on the DMD array alternates between sine (“Sin” in right-hand inset), cosine (“Cos” in right-hand inset), and “all on” (“Normalization” in right-hand inset) to give the three sets of data needed for phasor analysis (light reflected from elements in the “off’ state may be absorbed by the beam dump). On the other hand, when using two detectors the data for normalization can be obtained by simply summing the on and off states to obtain the normalization data. In another embodiment, a portion of the input light may be relayed to a third detector using a 25% / 75% beamsplitter immediately after the collimating lens, where a portion (generally the 25% portion) is directed to a detector which collects normalization data while the remaining portion (e.g., 75%) is used to collect the sine and cosine portions. Fixed mask approaches also exist if >3 detectors are used (one for sine ‘on’, one for cosine ‘on’, and one for light from the ‘all off state).

[0048] FIGS. 3A and 3B provide a schematic diagram of another configuration of a detection arm optical system for fluorescence unmixing using a DMD array as a light filter to separate out different portions of light for detection. FIGS. 3A and 3B illustrate embodiments of a DMD-based spectral phasor detection scheme together with cosine / sine detectors, respectively. Note only one set of cosine / sine detectors are shown in the figures for simplicity. In practice, there may be two or more detector units, for example one for in-phase (DMD ‘on’ state) and another for 180° (DMD ‘off state) detection.

[0049] In the DMD-based detection embodiment of FIG. 3, light returning from the detection arm (i.e., emitted from “Fiber Bundle”) may be spectrally separated and relayed onto13QB\99549432.1MGH 2024-547-03Quarles 125141.04912the columns of the surface of the DMD via a diffraction grating and a cylindrical lens (FIG. 3A). Whereas in the previously-described DMD-based detection systems the DMD array alternated between different patterns (e.g., sin and cos), in this embodiment the DMD array is divided into two portions which provide the sin and cos patterns. In one embodiment, the top half of the DMD may be programmed to be in the sine state and the bottom half will be in the cosine state (note that the designations of top and bottom are arbitrary and are used for convenience). Light from ‘on’ mirrors will be relayed to two separate PMTs by a lens to a half mirror, as shown in FIG. 3B. A similar configuration may be used for the ‘off mirrors, detecting cosine and sine filtered spectra that are 180° out of phase. The use of both DMD phases enables detection of all emission light, reducing losses. Total transmission can also be measured by summing the signal from all four detectors. Another strength of this DMD approach is that it allows modulation frequency / phase tuning, enabling optimization of phasor space separation and unmixing sensitivity.

[0050] A single detector can be used in place of the two out-of-phase detectors to simply collect all “off state” light.

[0051] As disclosed herein, in place of a DMD a patterned mask (e.g., a Ronci ruling including mirror lines printed on a glass slide) can be used to mask the fluorescence and the geometry can be changed to transmission. Forward transmitted light can be collected as in phase, whereas out of phase would be collected in the same configuration as the DMD.

[0052] In various embodiments, the embodiments of detection arms of FIGS. 1-3 may be integrated into a multimodality OCT-fluorescence system such as that shown in FIG. 4. The multimodality OCT-fluorescence catheter imaging system of FIG. 4 integrates optical coherence tomography (OCT) with near-infrared fluorescence molecular imaging in a single catheter platform for simultaneous structural and molecular tissue characterization. In various embodiments, the OCT-fluorescence system of FIG. 4 can be used to collect data that is processed for spectral unmixing by one of the detection systems disclosed herein where, for 14QB\99549432.1MGH 2024-547-03Quarles 125141.04912example, the detection system of any one of FIGS. 1-3 could be integrated at the “unmixing system” block.

[0053] One embodiment of the system includes a microstructural imaging system that detects back-reflected light (e.g., 1310 nm) from a sample (e.g., an artery, esophagus, stomach, ear, colon, duodenum, nasal cavity, brain, skin) for OCT imaging, and a near-infrared laser (e.g., 730 nm) that provides excitation for fluorescence imaging. A single mode opticalfiber SMF connects the OCT system to a dual clad fiber coupled (DCFC) via a wavelength division multiplexer (WDM), which couples the light from the near-infrared source and the OCT source / system. The DCFC is coupled to a rotary junction (RJ) using a dual clad fiber (DCF). The RJ is coupled to a catheter probe using DCF (see inset in FIG. 4 showing cross section in rejection near the RJ, corresponding to the vertical dashed line). In some embodiments the probe includes provisions for using a guidewire (transparent outer sheath in cross-sectional view) to guide the catheter into the sample (e.g., into or through a vessel).

[0054] The RJ serves as an interface between stationary imaging systems and the rotating catheter, combining optical beams from both modalities while maintaining coupling during catheter movement. This enables simultaneous acquisition of OCT microstructural images and fluorescence information for comprehensive tissue assessment, particularly valuable for atherosclerotic plaque characterization where structural features can be correlated with inflammatory biomarkers.

[0055] A computer system 500 (FIG. 5) for fluorescence unmixing includes an integrated hardware and software platform designed to perform spectral unmixing and optionally to process data from other modalities such as interferometric data from optical coherence tomography measurements.

[0056] The computer system 500 includes a computing device 510 that receives light emission data (e.g., fluorescence and / or interferometric data) from an optical system 512, for example from one or more detectors that are part of the optical system 512. The computing 15QB\99549432.1MGH 2024-547-03Quarles 125141.04912device 510 executes a system for fluorescence unmixing 504 to identify one or more separate components of the optical signal based on the received light emission data. The system 504 may operate in a distributed configuration where computing device 510 communicates with a server 520 over a communication network 506 for enhanced processing capabilities.

[0057] The system incorporates one or more detectors 502 that may be local to computing device 510 or remotely connected via cable, direct wireless link, or communication network 506. The one or more detectors 502 provide signals corresponding to light emission which are used for spectral unmixing of fluorescence signals and / or for obtaining structural information from a sample.

[0058] The computing device 510 includes a processor 602 that may be any suitable hardware processor or combination of processors, including central processing units and graphics processing units (FIG. 6). A display 604 provides visual output for presenting data analysis results and other information to users such as researchers, operators, or clinicians.

[0059] Input devices 606 include keyboards, mice, touchscreens, and microphones for user interaction with the system. Communication systems 608 enable data exchange over communication network 506 and include transceivers, communication chips, and hardware / firmware / software for establishing Wi-Fi, Bluetooth, cellular, and Ethernet connections.

[0060] Memory 610 stores instructions, values, and data used by processor 602 for system operation. The memory includes volatile memory (RAM), non-volatile memory (ROM, EEPROM), and storage devices (flash drives, hard disks, solid state drives, optical drives). A computer program encoded in memory 610 controls computing device 510 operation, enabling content presentation, data communication with server 520, and execution of procedures for spectral unmixing.

[0061] Server 520 provides enhanced computational capabilities and includes processor 612, display 614, inputs 616, communications systems 618, and memory 620 with similar16QB\99549432.1MGH 2024-547-03Quarles 125141.04912specifications to the computing device components. The server executes server programs for processing received light emission data, performing spectral unmixing, and communicating results to computing devices 510.

[0062] Communication network 506 facilitates data exchange between system components and may include Wi-Fi networks, peer-to-peer networks (Bluetooth), cellular networks (4G, 5G, LTE, etc.), and wired networks. The network can be configured as local area networks, wide area networks, public networks (Internet), or private / semi-private networks (corporate or university intranets).

[0063] EXAMPLES

[0064] The following are non-limiting examples in accordance with one or more embodiments of the disclosure.

[0065] Method

[0066] Study. A human carotid artery was incubated with the Cy5 conjugated (-667 nm em.) LUMISIGHT agent that agent for 6 hours. Pre- and post-incubation, the artery was imaged intra-luminally with the unmixing system and a single PMT we evaluate the performance of the system unmixing the agent from the near-infrared autofluorescence (NIRAF) background. We then imaged a balloon-injured New Zealand White Rabbit with a catheter in vivo where we show multiple species of NIRAF can be unmixed.

[0067] Optical System. Light is guided from the fluorescence exit fiber (105 pm) of an in vivo fluorescence system (similar to that of Yoo et al.). The light is collimated and spread by a diffraction grating (FIG. 1 A). The light is then focused in 1 dimension onto a mask using a cylindrical lens, where each line on the mask represents a single wavelength. In various embodiments, the mask is patterned such that it includes a plurality of reflective regions interspaced with a plurality of transmissive regions. In some embodiments, the mask is chromed on glass, with the top being square waves which represent the sine pattern, and bottom phase17QB\99549432.1MGH 2024-547-03Quarles 125141.04912shifted by representing the cosine pattern (FIG. IB). The reflected light off the mask is captured by a parabolic mirror and relayed to a number of photo multiplier tubes (PMTs). The transmitted light is separated into two paths, sine and cosine. Each path is independently relayed via parabolic mirrors to the PMTs. In certain embodiments, the square wave pattern can be converted to sinusoidal modulation simply by defocusing the masks (e.g., by adjusting the location of the mask so that the mask is not at the exact focal plane of the cylindrical lens), which is equivalent to mathematically convolving the mask with a gaussian function.

[0068] Pre-Processing. The light signals used for the sine / cosine readout are direct detector read outs scaled by a transmission factor (calculated using the reflectance from an LED (700 + / -10 nm) with same incidence angle as cylindrical lens). The value used for normalization is the sum of the three detectors with the normalization detector scaled by the reflectivity and the sine / cosine scaled by the transmissibility.

[0069] Results

[0070] Intraluminal scans were acquired from a human carotid plaque which was incubated with the LUMISIGHT agent and scanned at 100 kHz (FIG. 7). Unmixing is performed immediately post-procedure with complete reconstructions taking <10 s. The unmixing was automatic as the cluster centers were known in advance. The NIRAF profile was extracted from the carotid pre-incubation using blind unmixing, and the Cy5 cluster location was extracted from separate scans of trypsinized LUMISIGHT agent.

[0071] Fluorescence reflectance imaging was used to validate the unmixed signals (FIG. 8). Pre-incubation images show bright NIRAF at the fibrocalcific regions but limited in the lipid rich regions. Post-incubation shows uptake in lipid rich regions and slight uptake in the calcified regions.18QB\99549432.1MGH 2024-547-03Quarles 125141.04912

[0072] The artery contained complex regions including two sources of NIRAF contrast: calcium, and oxidized lipids. The spatial distribution did change post incubation but the regions remained fluorescent. The change in distribution was also noticed ex -vivo and is believed to be related to interaction of the agent with NIRAF.

[0073] A rabbit was also imaged intravascularly without the use of a contrast agent (FIG. 9). For the unmixing step in this case, blind clustering was performed. Although the rabbit did not have a large plaque, different NIRAF profiles were seen for diseased tissue with different OCT scattering properties with one component dominating vs. another. The rabbit data was promising and indicates that unmixing does in fact work in vivo, although resolving the exact source of NIRAF contrast is beyond the scope of this paper.

[0074] In its current embodiment the modulation frequency is fixed. It is straightforward to change the masks however, but the fluorescence components must be known as a priori or a parameter sweep must first be done using a spectrometer scanned system.

[0075] Conclusion

[0076] This example demonstrates a fluorescence unmixing system that can easily be added onto point scanning devices that is capable of high-speed fluorescence unmixing in real time. Replacing the mask with a spatial light modulator (e.g., as shown in FIGS. 2-3) would allow for dynamic tuning of the phasor harmonics and allow unmixing of >3 components and dynamically optimizing the unmixing signal-to-noise ratio.

[0077] Supplementary Methods

[0078] Simulation of NIRAF and LUMICELL Unmixing. TheLUMISIGHT agent is conjugated to the Cy5 fluorophore. As a result, its spectra is known a-priori. A scan was taken of a complex human plaque from a discarded carotid specimen the mean spectra was taken as NIRAF19QB\99549432.1MGH 2024-547-03Quarles 125141.04912

[0079] Background Subtraction. Fiber based systems such as dual clad fiber, often have a high autofluorescence background. This can be either unmixed (as 1 of the 3 components) or it can be subtracted out at the detector level prior to imaging (keeping the 3 components). The background subtraction routine was implemented here.

[0080] Simulations. Rabbits injected with the Cy5 Conjugated agent (LUMISIGHT) were imaged using a point scan system to obtain a full spectrum such that the two dominant spatially localized NIRAF spectra were extracted. A parameter sweep of modulation frequencies was done until the greatest cluster separation between the three dominant FL components was observed (LUMISIGHT, NIRAF 1, NIRAF 2). Simulations indicated the optimal modulation period was -13.1 nm.

[0081] Animal Experiment. New Zealand White Rabbits were used which has been balloon injured allowing plaque to form. The near-infrared subcomponents of this plaque have distinct spectroscopic components. Rabbit plaques are often simpler than human plaques and predominantly include ceroid (oxidized lipids) and less intense residual components. In humans, NIRAF positive components are intraplaque hemorrhage, ceroid, and NIRAF also colocalizes to calcium sites.

[0082] Bibliography - Each of the following documents is incorporated by reference in its entirety:

[0083] 1. Carbary-Ganz, J.L., et al., In vivo molecular imaging of colorectal cancer using quantum dots targeted to vascular endothelial growth factor receptor 2 and optical coherence tomography / laser-induced fluorescence dual-modality imaging. J Biomed Opt, 2015. 20(9): p.096015.

[0084] 2 Hariri, L.P , et al., Endoscopic optical coherence tomography and laser-induced fluorescence spectroscopy in a murine colon cancer model. Lasers Surg Med, 2006. 38(4): p. 305-13.20QB\99549432.1MGH 2024-547-03Quarles 125141.04912

[0085] 3. Ughi, G.J., et al., Dual modality intravascular optical coherence tomography (OCT) and near-infrared fluorescence (NIRF) imaging: a fully automated algorithm for the distance-calibration of NIRF signal intensity for quantitative molecular imaging. The international journal of cardiovascular imaging, 2015. 31(2): p. 259-268.

[0086] 4. Wartak, A., et al., Dual-modality optical coherence tomography and fluorescence tethered capsule endomicroscopy. Biomed Opt Express, 2021. 12(7): p. 4308-4323.

[0087] 5. Winkler, A.M., et al., In vivo, dual-modality OCT / LIF imaging using a novel VEGF receptor-targeted NIR fluorescent probe in the AOM-treated mouse model. Molecular imaging and biology, 2011. 13: p. 1173-1182.

[0088] 6. Whitley, M.J., et al., A mouse-human phase 1 co-clinical trial of a protease-activated fluorescent probe for imaging cancer. Science translational medicine, 2016. 8(320): p.320ra4-320ra4.

[0089] 7 Gibbs, S.L., Near infrared fluorescence for image-guided surgery. Quantitative imaging in medicine and surgery, 2012. 2(3): p. 177.

[0090] 8. Calfon, M.A., et al., In vivo near infrared fluorescence (NIRF) intravascular molecular imaging of inflammatory plaque, a multimodal approach to imaging of atherosclerosis. Journal of visualized experiments: JoVE, 2011(54): p. 2257.

[0091] 9. Zhou, Q., et al. fluorescence-guided surgery with panitumumab-IRDye800 and cetuximab-IRDye800 in glioblastoma patients (Conference Presentation), in Molecular-Guided Surgery: Molecules, Devices, and Applications V. 2019. SPIE.

[0092] 10. Schmidt, F., et al. Accurate database for constrained linear unmixing, in European Planetary Science Congress 2012. 2012.

[0093] 1 l.Xu, H. and B.W. Rice, In-vivo fluorescence imaging with a multivariate curve resolution spectral unmixing technique. J Biomed Opt, 2009. 14(6): p. 064011.

[0094] 12.Guiffant, G., et al., Flushing of intravascular access devices (IVADs)-efficacy of pulsed and continuous infusions. The journal of vascular access, 2012. 13(1): p. 75-78.21QB\99549432.1MGH 2024-547-03Quarles 125141.04912

[0095] 13.Yoo, H., et al., Intra-arterial catheter for simultaneous microstructural and molecular imaging in vivo. Nature medicine, 2011. 17(12): p. 1680-1684.

[0096]

[0097] Features of any of the examples or embodiments outlined above may be combined to create additional examples or embodiments without losing the intended effect. It should be understood that the description of an embodiment or example provided above is by way of example only, and various modifications could be made by one skilled in the art. Furthermore, one skilled in the art will recognize that numerous further modifications and combinations of various aspects are possible. Accordingly, the described aspects are intended to encompass all such alterations, modifications, and variations that fall within the scope of the appended claims.22QB\99549432.1

Claims

MGH 2024-547-03Quarles 125141.04912CLAIMSWhat is claimed is:

1. An optical detection system for fluorescence unmixing, comprising:a diffraction grating configured to spectrally disperse fluorescence light;a light filter configured to receive the spectrally dispersed light, the light filter configured to divide the spectrally dispersed light into a plurality of different portions of light;one or more detectors configured to detect the plurality of different portions of light from the light filter; anda controller configured to unmix the fluorescence light based on signals from the one or more detectors.

2. The optical system of claim 1, wherein the light filter is configured to divide the spectrally dispersed light into the plurality of different portions of light, wherein the plurality of different portions of light comprises a sine portion and a cosine portion.

3. The optical system of claim 2, wherein the one or more detectors comprises a first detector configured to detect the sine portion and a second detector configured to detect the cosine portion.

4. The optical system of any one of claims 1 to 3, wherein the light filter comprises a patterned mask.23QB\99549432.1MGH 2024-547-03Quarles 125141.049125. The optical system of claim 4, wherein the patterned mask comprises a plurality of reflective regions interspaced with a plurality of transmissive regions.

6. The optical system of claim 5, wherein the plurality of reflective regions and the plurality of transmissive regions comprise square wave patterns.

7. The optical system of claim 5 or 6, wherein each of the plurality of reflective regions and the plurality of transmissive regions corresponds to a particular wavelength of the spectrally dispersed light.

8. The optical system of any one of claims 4 to 7, wherein the first detector and the second detector are configured to detect light transmitted through the patterned mask, and wherein the one or more detectors comprises a third detector configured to detect a normalization portion of light reflected from the patterned mask.

9. The optical system of claim 8, wherein the controller is further configured to unmix the fluorescence light based on the sine portion detected by the first detector, the cosine portion detected by the second detector, and the normalization portion detected by the third detector.

10. The optical system of any one of claims 1 to 9, further comprising a cylindrical mirror,24QB\99549432.1MGH 2024-547-03Quarles 125141.04912wherein the diffraction grating is configured to spectrally disperse the fluorescence light onto the cylindrical mirror.

11. The optical system of claim 10, wherein the light filter is disposed at a focal plane of the cylindrical mirror.

12. The optical system of claim 1, wherein the light filter comprises a digital micromirror device (DMD),wherein the DMD is configured to produce patterns which divide the spectrally dispersed light into the plurality of different portions of light.

13. The optical system of claim 12, wherein the DMD is further configured to produce alternating patterns comprising a sine pattern and a cosine pattern.

14. The optical system of claim 13, wherein the alternating patterns produced by the DMD further comprise a normalization pattern.

15. The optical system of claim 14, wherein the one or more detectors are configured to detect the plurality of different portions of light reflected from the DMD based on at least one of the sine pattern, the cosine pattern, or the normalization pattern.25QB\99549432.1MGH 2024-547-03Quarles 125141.0491216. The optical system of claim 15, wherein the one or more detectors comprises a first detector configured to collect the plurality of different portions of light from elements of the DMD that are in the on state.

17. The optical system of claim 16, wherein the controller is further configured to unmix the fluorescence light based on the plurality of different portions of light reflected from the DMD to the first detector based on the sine pattern, the cosine pattern, and the normalization pattern.

18. The optical system of claim 16, wherein the one or more detectors further comprises a second detector configured to collect the plurality of different portions of light from elements of the DMD that are in the off state.

19. The optical system of claim 18, wherein the controller is further configured to unmix the fluorescence light based on the plurality of different portions of light reflected from the DMD by the first detector and the second detector based on the sine pattern and the cosine pattern.

20. The optical system of any one of claims 12 to 19, wherein at least one of the sine pattern, the cosine pattern, or the normalization pattern of the DMD comprises bands corresponding to particular wavelengths of the spectrally dispersed light.26QB\99549432.1MGH 2024-547-03Quarles 125141.0491221. The optical system of claim 20, further comprising a cylindrical lens configured to direct the spectrally dispersed light from the diffraction grating onto the DMD.

22. The optical system of claim 21, wherein the DMD is disposed at a focal plane of the cylindrical lens.

23. The optical system of any one of claims 1 to 22, wherein the optical system is configured to operate at acquisition rates exceeding 100 kHz.

24. The optical system of any one of claims 1 to 23, wherein the fluorescence light is obtained from an optical coherence tomography (OCT) fluorescence optical system.

25. The optical system of any one of claims 1 to 24, wherein the one or more detectors are configured to detect spectral information from the respective plurality of different portions of light.

26. A method for fluorescence unmixing, comprising:spectrally dispersing fluorescence light using a diffraction grating;receiving the spectrally dispersed light at a light filter;dividing the spectrally dispersed light into a plurality of different portions of light using the light filter;detecting the plurality of different portions of light using one or more detectors; and27QB\99549432.1MGH 2024-547-03Quarles 125141.04912unmixing the fluorescence light based on signals from the one or more detectors using a controller.

27. The method of claim 26, wherein dividing the spectrally dispersed light comprises dividing the spectrally dispersed light into a sine portion and a cosine portion.

28. The method of claim 27, wherein detecting the plurality of different portions of light comprises detecting the sine portion using a first detector and detecting the cosine portion using a second detector.

29. The method of any one of claims 26 to 28, wherein the light filter comprises a patterned mask.

30. The method of claim 29, wherein the patterned mask comprises a plurality of reflective regions interspaced with a plurality of transmissive regions.

31. The method of claim 30, wherein the plurality of reflective regions and the plurality of transmissive regions comprise square wave patterns.

32. The method of claim 30 or 31, wherein each of the plurality of reflective regions and the plurality of transmissive regions corresponds to a particular wavelength of the spectrally dispersed light.28QB\99549432.1MGH 2024-547-03Quarles 125141.0491233. The method of any one of claims 29 to 32, wherein detecting the plurality of different portions of light comprises detecting light transmitted through the patterned mask using the first detector and the second detector, and detecting a normalization portion of light reflected from the patterned mask using a third detector.

34. The method of claim 33, wherein unmixing the fluorescence light comprises unmixing the fluorescence light based on the sine portion detected by the first detector, the cosine portion detected by the second detector, and the normalization portion detected by the third detector.

35. The method of any one of claims 26 to 34, further comprising directing the spectrally dispersed fluorescence light onto a cylindrical mirror using the diffraction grating.

36. The method of claim 35, wherein the light filter is disposed at a focal plane of the cylindrical mirror.

37. The method of claim 26, wherein the light filter comprises a digital micromirror device (DMD), and wherein dividing the spectrally dispersed light comprises producing patterns using the DMD to divide the spectrally dispersed light into the plurality of different portions of light.29QB\99549432.1MGH 2024-547-03Quarles 125141.0491238. The method of claim 37, wherein producing the patterns comprises producing alternating patterns comprising a sine pattern and a cosine pattern using the DMD.

39. The method of claim 38, wherein producing alternating patterns further comprises producing a normalization pattern using the DMD.

40. The method of claim 39, wherein detecting the plurality of different portions of light comprises detecting the plurality of different portions of light reflected from the DMD based on at least one of the sine pattern, the cosine pattern, or the normalization pattern.

41. The method of claim 40, wherein detecting the plurality of different portions of light comprises collecting the plurality of different portions of light from elements of the DMD that are in the on state using a first detector.

42. The method of claim 41, wherein unmixing the fluorescence light comprises unmixing the fluorescence light based on the plurality of different portions of light reflected from the DMD to the first detector based on the sine pattern, the cosine pattern, and the normalization pattern.

43. The method of claim 41, wherein detecting the plurality of different portions of light further comprises collecting the plurality of different portions of light from elements of the DMD that are in the off state using a second detector.30QB\99549432.1MGH 2024-547-03Quarles 125141.0491244. The method of claim 43, wherein unmixing the fluorescence light comprises unmixing the fluorescence light based on the plurality of different portions of light reflected from the DMD by the first detector and the second detector based on the sine pattern and the cosine pattern.

45. The method of any one of claims 37 to 44, wherein at least one of the sine pattern, the cosine pattern, or the normalization pattern of the DMD comprises bands corresponding to particular wavelengths of the spectrally dispersed light.

46. The method of claim 45, further comprising directing the spectrally dispersed light from the diffraction grating onto the DMD using a cylindrical lens.

47. The method of claim 46, wherein the DMD is disposed at a focal plane of the cylindrical lens.

48. The method of any one of claims 26 to 47, wherein the method operates at acquisition rates exceeding 100 kHz.

49. The method of any one of claims 26 to 48, wherein the fluorescence light is obtained from an optical coherence tomography (OCT) fluorescence optical system.31QB\99549432.1MGH 2024-547-03Quarles 125141.0491250. The method of any one of claims 26 to 49, wherein detecting the plurality of different portions of light comprises detecting spectral information from the respective plurality of different portions of light.32QB\99549432.1

Images