A multimodal super-resolution microscopy system and method based on multifocal structured light illumination

Abstract

This invention discloses a multimodal super-resolution microscopy imaging system and method based on multifocal structured light illumination, belonging to the field of optical microscopy imaging technology. The system includes: a laser beam generation module for generating and controlling the spatial coherence of the illumination beam; a multifocal structured light excitation module, including a beam modulation device, which modulates the illumination beam into a multifocal structured light array and projects it onto the sample plane to excite the sample to generate interference scattering signals and / or fluorescence signals; an interference scattering signal detection module for detecting interference scattering signals; a fluorescence signal detection module for acquiring fluorescence signals; and a control system module electrically connected to the beam modulation device, the interference scattering signal detection module, and the fluorescence signal detection module, respectively, for controlling the generation of multifocal illumination and its synchronization with signal acquisition. This invention solves the problem that existing interference scattering microscopy techniques cannot simultaneously achieve high speed, high signal-to-noise ratio, and high-resolution imaging.

Description

Technical Field

[0001] This invention relates to the field of optical microscopy imaging technology, and more specifically to a multimodal super-resolution microscopy imaging system and method based on multifocal structured light illumination. Background Technology

[0002] Interferometric scattering microscopy (iSCAT) is a highly sensitive optical detection technique based on the principle of interference between scattered and reference light. First proposed by V. Sandohdar's team in 2004, its core advantage lies in its ability to detect weak scattering signals from non-fluorescent samples. Compared to traditional fluorescence microscopy, iSCAT eliminates the need for exogenous fluorescent labeling of samples, thus avoiding photobleaching and phototoxicity issues. This gives it a unique advantage in long-term, high-frequency observations. Furthermore, iSCAT possesses extremely high spatiotemporal resolution, capable of locating individual nanoparticles with microsecond-level temporal resolution. Currently, this technology is widely used in cutting-edge life sciences and materials science, such as real-time tracking of viruses and single nanoparticles, monitoring of cytoskeleton dynamics, studies of single-molecule binding dynamics, and analysis of microscopic electrochemical reaction processes in batteries.

[0003] In terms of system architecture, iSCAT typically employs a common-path interference optical path design. Its working principle utilizes the interface where the sample is located, usually the coverslip-water interface, as a natural beam splitter: a small portion of the illumination beam is reflected by the interface as a reference beam, while the light passing through the interface and illuminating the sample is scattered and used as the signal beam. Since the reference beam and signal beam overlap for most of the optical path, their phase shifts caused by environmental vibrations and air disturbances are essentially the same, resulting in extremely high phase stability. Furthermore, the device is compact and easy to assemble.

[0004] Despite the advantages mentioned above, traditional iSCAT systems still face a technical bottleneck in practical applications, where achieving both high image quality and high imaging speed is difficult. Traditional iSCAT systems typically employ wide-field laser illumination to achieve high-speed, large-field-of-view real-time imaging. However, due to the extremely high spatial coherence of the laser source, even minor imperfections, dust, or internal refractive index inhomogeneities on the surfaces of optical components such as lenses and waveplates in the optical path can generate parasitic reflections and scattering. These stray lights interfere with the reference light, forming high-contrast background speckle and Newton's rings-like interference fringes in the image. This high-intensity fixed-mode noise often overwhelms small target scattering signals, severely affecting the system's detection sensitivity and image signal-to-noise ratio, limiting its ability to detect smaller particles, such as those smaller than 20 nm. To suppress background noise and improve image contrast, researchers have proposed various improvement schemes, but all involve significant performance trade-offs.

[0005] 1. Confocal Illumination iSCAT. This method incorporates the principles of confocal microscopy, using tiny pinholes at conjugate positions in the probe optical path for spatial filtering, effectively eliminating defocused background light and parasitic interference fringes in the optical path. While this method significantly improves the signal-to-noise ratio and contrast of the image and endows the system with axial tomography capabilities, its inherently point-scan imaging mode greatly limits the image acquisition speed. This makes confocal iSCAT difficult to capture high-speed dynamic processes such as rapid intracellular transport or Brownian motion, sacrificing iSCAT's original high-speed imaging advantage.

[0006] 2. Reducing coherence, such as with rotating diffusers. Another common strategy is to add rotating frosted glass or use fast-deflecting mirrors to the illumination path. This smooths the background speckle and reduces parasitic interference by disrupting the temporal or spatial coherence of the light source. While this method preserves the high-speed characteristics of wide-field imaging, reducing the coherence of the light source leads to decreased interference efficiency, thereby weakening the intensity of the effective signal. Simultaneously, the introduced diffuser often increases the divergence angle of the illumination beam, sacrificing the system's optical resolution, especially lateral resolution, resulting in blurred imaging details.

[0007] In summary, existing interferometric scattering microscopy techniques struggle to simultaneously meet the demands for high imaging speed, high signal-to-noise ratio, and high resolution. Therefore, there is an urgent need to develop a novel illumination and imaging mechanism that can effectively suppress coherent background noise while retaining the high-speed characteristics of wide-field imaging and the high-resolution characteristics of confocal illumination. Summary of the Invention

[0008] In view of this, the purpose of the present invention is to provide a multimodal super-resolution microscopy imaging system and method based on multifocal structured light illumination, which aims to solve the problem that existing interferometric scattering microscopy techniques cannot simultaneously achieve high speed, high signal-to-noise ratio and high resolution imaging.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, embodiments of the present invention provide a multimodal super-resolution microscopy system based on multifocal structured light illumination, comprising: A laser beam generation module is used to generate and modulate the spatial coherence of the illumination beam; A multi-focus structured light excitation module, optically connected to the laser beam generation module, includes a beam modulation device for modulating the illumination beam into a multi-focus structured light array and projecting it onto the sample plane to excite the sample to generate interference scattering signals and / or fluorescence signals. An interference scattering signal detection module is used to collect the interference scattering signal generated by the sample after it is excited by the multifocal structured light. A fluorescence signal detection module is used to collect the fluorescence signal generated by the sample after it is excited by the multifocal structured light. The control system module is electrically connected to the beam modulation device, the interference scattering signal detection module, and the fluorescence signal detection module, respectively, and is used to control the generation of multifocal illumination and its synchronization with signal acquisition.

[0010] In one embodiment, the laser beam generating module includes a first laser, a rotating frosted glass, and a beam expanding and collimating unit; the rotating frosted glass and the beam expanding and collimating unit are sequentially arranged in the output optical path of the first laser to reduce the coherence of the illumination beam and then expand and collimate the beam to suppress interference background noise.

[0011] In one embodiment, the beam modulation device is a digital micromirror device, a spatial light modulator, or an acousto-optic modulator.

[0012] In one embodiment, the multifocal structured light excitation module further includes a polarization control unit and a spatial filtering unit; The polarization control unit includes a half-wave plate, a first polarizing beam splitter prism, and a quarter-wave plate arranged sequentially along the optical path. The half-wave plate is used to adjust the polarization direction of the beam incident on the first polarizing beam splitter prism. The first polarizing beam splitter prism and the quarter-wave plate are combined to convert the linearly polarized excitation light into circularly polarized light and incident it on the sample, while separating the back-propagating interference scattering signal from the sample to the detection optical path. The spatial filtering unit includes a first filtering lens, a spatial filtering aperture, and a second filtering lens arranged along the optical path, used to filter out higher-order diffraction light generated by the beam modulation device.

[0013] In one embodiment, the interference scattering signal detection module and the multifocal structured light excitation module share a portion of the optical path, and the excitation optical path and the detection optical path are separated by the polarization control unit.

[0014] In one embodiment, the fluorescence signal detection module and the multifocal structured light excitation module share a portion of the optical path, including an objective lens, a mirror assembly for changing the direction of the optical path, and a first dichroic mirror for separating the fluorescence signal from the excitation light.

[0015] In one embodiment, it further includes an anti-drift module, which includes a reference light generating unit, a tilt illumination unit, a position detection unit, and a feedback control unit; The control system module is connected to the position detection unit and the feedback control unit, respectively. The reference light generated by the reference light generation unit is incident on the sample plane off-axis through the tilted illumination unit and is reflected; the control system module detects the lateral displacement of the reflected light by the position detection unit to calculate the axial drift, and controls the single-axis displacement stage to perform axial negative feedback compensation through the feedback control unit based on the drift.

[0016] In one embodiment, the tilted illumination unit of the anti-drift module shares the objective lens, quarter-wave plate, mirror assembly, and first dichroic mirror with the multifocal structured light excitation module, such that the anti-drift reference light and the imaging signal light partially overlap in the optical path before reaching the sample.

[0017] In a second aspect, embodiments of the present invention provide a multimodal super-resolution microscopy imaging method based on multifocal structured light illumination, applied to the system as described in any one of the first aspects, the method comprising: S1: Generates a coherently modulated illumination beam; S2: The illumination beam is modulated into a multifocal structured light array using a beam modulation device and projected onto the sample plane to excite interference scattering signals and / or fluorescence signals; S3: After adjusting the sample to a suitable focal plane using a three-dimensional displacement stage, the system achieves long-term focal plane stability by controlling the anti-drift module and the single-axis displacement stage through the control system module and performing negative feedback adjustment. S4: Synchronously acquire the interference scattering image sequence and / or fluorescence image sequence generated by the multifocal structured light excitation; S5: Reconstruct the acquired image sequence. The reconstruction process includes: calculating the spatial offset vector based on the image of each focal region, and performing sub-pixel precision repositioning and superposition to reconstruct the final super-resolution image.

[0018] In one embodiment, step S5 specifically includes: S51: Extract the micro-image region centered on each illumination focus from the multi-focus structured light illumination image sequence; S52: For the interference scattering image sequence, apply the radial variance transformation algorithm to each extracted micro-image region to eliminate coherent phase interference and generate the corresponding pseudo-fluorescence image; for the fluorescence image sequence, directly use the micro-image region; based on the pseudo-fluorescence image or the micro-image region, calculate the spatial offset vector of each off-axis illumination focus relative to the central reference focus; S53: Using the spatial offset vector, the micro-image region obtained in step S51 is repositioned and superimposed with sub-pixel precision to reconstruct the super-resolution initial image. S54: Perform background correction on the initial image to obtain the final high-contrast super-resolution image.

[0019] As can be seen from the above technical solution, compared with the prior art, the present invention has the following technical advantages: This invention modulates the light field using a digital micromirror array (DMD). The principle is to apply different voltages to each micromirror of the DMD, generating different deflections to illuminate beams of specific shapes. By loading a lattice pattern into the DMD, multifocal structured light illumination is achieved. The parallel scanning strategy overcomes the slow imaging speed of confocal scanning, while significantly improving the signal-to-noise ratio and contrast compared to wide-field illumination. Furthermore, this invention introduces a rotating diffuser in the excitation optical path to control the spatial coherence of the light source and suppress scattering and interference fringe backgrounds caused by stray light from the system. In addition, this invention can simultaneously detect interference scattering and fluorescence signals in the probe optical path, while simultaneously tracking specifically labeled molecules and unlabeled structural backgrounds. This invention combines the high signal-to-noise ratio of confocal imaging with the high speed of wide-field imaging, significantly improving image contrast and sharpness, and is suitable for high spatiotemporal resolution imaging of microscopic dynamic processes. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 This is a diagram of the optical path structure of a multimodal super-resolution microscopy system based on multifocal structured light illumination provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the optical path of the interference scattering signal detection module provided in this embodiment of the invention; Figure 3 This is a flowchart of a multimodal super-resolution microscopy method based on multifocal structured light illumination provided in an embodiment of the present invention; Figure 4 This is a flowchart of step S5, the reconstruction process, provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1: Please see Figure 1 This invention provides a multimodal super-resolution microscopy imaging system based on multifocal structured light illumination, comprising: a laser beam generation module, a multifocal structured light excitation module, an interference scattering signal detection module, a fluorescence signal detection module, an anti-drift module, and a control system module.

[0024] 1. Laser beam generation module; like Figure 1 As shown, the laser beam generation module is used to generate a laser beam of appropriate size and to control the spatial coherence of the illumination beam; it includes a first laser 1, a rotating frosted glass 2, and a beam expanding and collimating unit, which is composed of a first lens 3 and a second lens 4.

[0025] The first laser 1 is used to generate a continuous or pulsed laser beam, preferably a single-mode continuous laser. Continuous lasers, with their relatively long transverse and axial coherence, are prone to interference artifacts in the system due to defects in optical components. Using a rotating diffuser can control the beam coherence and reduce interference artifacts. If the continuous laser is replaced with a laser diode, which has relatively poor coherence, the same number of interference artifacts can be generated without using a rotating diffuser.

[0026] The rotating frosted glass 2 is positioned in the output optical path of the first laser 1. Its core function is to disrupt or reduce the temporal and spatial coherence of the illumination beam. Specifically, it can control the coherence of the laser, appropriately reduce the transverse and axial coherence of the beam generated by the continuous laser, and simultaneously make the beam more uniform. When the laser beam passes through the high-speed rotating frosted glass 2, its wavefront phase is randomly modulated, thereby effectively suppressing parasitic interference fringes and laser speckle noise caused by defects in optical components, dust, etc., in the subsequent optical path; ultimately reducing the speckle background and interference artifacts generated by optical components in the detection signal. The beam after passing through the rotating frosted glass 2 enters the beam expander and collimator unit. This unit typically consists of two lenses: a first lens 3 and a second lens 4, forming a telescope system used to expand the beam diameter to a size suitable for subsequent DMD modulation and output a well-collimated illumination beam.

[0027] 2. Multifocal structured light excitation module; The multifocal structured light excitation module is optically connected to the laser beam generation module. Its core component is a beam modulation device used to modulate the illumination beam into a multifocal structured light array and project it onto the sample plane to excite the sample to generate interference scattering signals and / or fluorescence signals. For example... Figure 1As shown, in this embodiment, the beam modulation device is preferably a digital micromirror device 7. In other modified embodiments, a spatial light modulator (SLM) or an acousto-optic modulator (AOD) can also be used to achieve similar functions. The spatial light modulator can perform phase modulation on the beam in the spectral plane, and obtain multifocal structured light illumination after Fourier transform by a lens; the acousto-optic modulator can spatially modulate the beam, and perform rapid single-focus scanning of the sample within a single camera exposure time to obtain a multifocal illumination image.

[0028] like Figure 1 As shown, its specific components include: a first reflecting mirror 5, a second reflecting mirror 6, a digital micro-reflector device 7, a half-wave plate 8, a spatial filtering unit consisting of a first filtering lens 9, a spatial filtering aperture 10, and a second filtering lens 11; a first polarizing beam splitter prism 12; a first tube mirror 13; a first dichroic mirror 14; a third reflecting mirror 15; a fourth reflecting mirror 16; a quarter-wave plate 17; a uniaxial displacement stage 18; an objective lens 19; a three-dimensional fine-tuning displacement stage 31; and a sample stage 20. The first polarizing beam splitter prism 12 is used to separate the excitation light and probe light in the interference scattering channel. The quarter-wave plate 17 is used to convert linearly polarized light into circularly polarized light. The combination of the first polarizing beam splitter prism 12 and the quarter-wave plate 17 forms a unidirectional optical path, used to separate two beams of light with opposite propagation directions. The objective lens 19 and the first tube mirror 13 are used to reduce the multifocal structured light array of the beam to a suitable magnification and project it onto the sample plane. The first dichroic mirror 14 is used to separate the fluorescence signal and the beam of the anti-drift module. The single-axis displacement stage 18 is used for anti-drift module calibration. The sample stage 20 is used to support the sample. The three-dimensional fine-tuning displacement stage 31 is used for three-dimensional adjustment of the sample's spatial position, enabling sample selection and focusing.

[0029] The specific excitation process of multifocal structured light is as follows: After beam expansion and collimation, the beam passes through the first reflector 5 and the second reflector 6, and is reflected at a 24° angle to the normal of the digital micromirror device 7 panel. The beam is then spatially encoded by a dot matrix mask loaded onto the digital micromirror device. Under the control of the control system module, the digital micromirror device 7 loads a pre-designed binary dot matrix mask. Each micromirror deflects according to the mask encoding, thereby spatially modulating the uniform illumination beam into a multifocal structured light array. This array consists of hundreds to thousands of discrete, on-demand arranged illumination spots. The light emitted from the digital microlens array is multifocal structured light. The polarization state of the emitted beam is controlled by a half-wave plate 8. The beam is then spatially filtered by a spatial filtering unit, retaining the middle diffraction order of the spectral plane and filtering out the excess diffraction order. After passing through the first polarizing beam splitter prism 12, the first tube mirror 13, the first dichroic mirror 14, the third reflecting mirror 15, the fourth reflecting mirror 16, the quarter-wave plate 17, and the objective lens 19, the beam is projected onto the sample stage 20 to excite the sample with multifocal structured light.

[0030] In the multifocal structured light excitation module, a half-wave plate 8 works in conjunction with a first polarizing beam splitter prism 12 to adjust the power of the excitation light. The first polarizing beam splitter prism 12 works in conjunction with a quarter-wave plate 17 to generate circularly polarized light to excite the sample. The incident circularly polarized light ensures that different polarization orientations cause specific scattering of the sample when probing anisotropic samples. At the same time, a unidirectional optical path is formed to separate the interference scattering detection signal of the excitation light and the sample.

[0031] Functionally, this module includes polarization control and spatial filtering, corresponding to the polarization control unit and the spatial filtering unit, respectively.

[0032] like Figure 1 As shown, the polarization control unit includes a half-wave plate 8, a first polarizing beam splitter prism 12, and a quarter-wave plate 17 along the optical path. The half-wave plate 8 is used to fine-tune the polarization direction of the beam after reflection by the digital micromirror device 7, making it linearly polarized light suitable for transmission by the first polarizing beam splitter prism 12. After passing through the first polarizing beam splitter prism 12, this linearly polarized light passes through the first tube mirror 13, the first dichroic mirror 14, the third mirror 15, and the fourth mirror 16, and then enters the quarter-wave plate 17. The quarter-wave plate 17 converts it into circularly polarized light, which is finally converged by the high numerical aperture objective lens 19 and projected onto the sample plane of the sample stage 20, realizing multifocal parallel excitation. Circularly polarized light can avoid signal differences caused by polarization direction when exciting anisotropic samples.

[0033] The spatial filtering unit is a 4f filtering system used to spatially filter the light beam, removing higher-order diffraction orders generated by the digital micromirror device 7. It includes a first filtering lens 9, a spatial filtering stop 10, and a second filtering lens 11 arranged sequentially along the optical path. The light field modulated by the digital micromirror device 7 forms multiple diffraction orders on the spectral plane, i.e., the back focal plane of the first filtering lens 9. The spatial filtering stop 10 is placed on this spectral plane, and its aperture only allows the required diffraction orders to pass through, while blocking the zero-order and other higher-order diffraction light generated by the DMD pixel structure, thereby purifying the illumination light field.

[0034] 3. Interference scattering signal detection module; The interference scattering signal detection module is used to collect the interference scattering signal generated after a sample is excited by a multifocal light spot. For example... Figure 1 and Figure 2 As shown, this module shares optical path components with the multifocal structured light excitation module, including objective lens 19, quarter-wave plate 17, mirror assembly (fourth mirror 16, third mirror 15), first dichroic mirror 14, first tube mirror 13, and first polarizing beam splitter prism 12. It also includes a first filter 29 and a first detector 24.

[0035] Its unique feature lies in the fact that the back-propagating interference scattering signal generated by the sample, after passing through the quarter-wave plate 17 again, has its polarization direction rotated by 90 degrees relative to the excitation light, that is, it changes from circular polarization to linear polarization orthogonal to it. When this signal returns to the first polarization beam splitter 12, because its polarization state is orthogonal to the excitation light, it will be reflected by the first polarization beam splitter 12, thereby achieving the separation of the excitation light path and the detection light path. The separated interference scattering signal then passes through the first filter 29 to filter out ambient stray light, and is then acquired by the first detector 24, such as an sCMOS camera, to obtain a series of multifocal illumination interference scattering raw images.

[0036] The specific detection process is as follows: The interference scattering signal generated by the sample passes sequentially through objective lens 19, quarter-wave plate 17, fourth reflecting mirror 16, third reflecting mirror 15, first dichroic mirror 14, first tube mirror 13, first polarizing beam splitter prism 12, and first filter 29, and is collected by first detector 24. Objective lens 19 and first tube mirror 13 form a 4f system, and the target surface of first detector 24 is located at the back focal plane of first tube mirror 13. Since the direction of interference scattering light is opposite to the direction of excitation light propagation, after passing through quarter-wave plate 17 and first polarizing beam splitter prism 12, the polarization states of the outgoing light and the incident light are orthogonal. Therefore, the outgoing light is reflected by first polarizing beam splitter prism 12 and enters first detector 24.

[0037] The complete process of generating and acquiring interference scattering signals is as follows: Figure 2 As shown, by loading a series of multifocal mask patterns t1, t2, ..., t3 into the digital micromirror device... N Projected onto the sample, Figure 2 The sample shown is an LCO (lithium cobalt oxide) sheet. The scattered light emitted from the sample interferes with the reference light reflected from the cover glass, generating interference scattering signals, namely interference scattering image 1, interference scattering image 2, ... interference scattering image N. These interference scattering signals are collected by an interference scattering signal detection module. During this process, the digital micromirror device and the first detector remain synchronized. Each time a multifocal mask is applied to the digital micromirror device, the camera acquires one frame of image, ultimately resulting in a series of interference scattering images.

[0038] The principle of interference scattering signal generation: The scattered light emitted by the sample interferes with the reflected light from the cover glass of the sample, specifically in the following form.

[0039] in, The reference light reflected by the coverslip. The scattered light from the sample, This represents the phase difference between the two. When they interfere, the coherence terms... The presence of [something] will enhance or weaken the scattering signal of the sample, resulting in a contrast enhancement effect in the sample image detected by the detector.

[0040] 4. Fluorescence signal detection module; The fluorescence signal detection module is used to acquire the fluorescence signal emitted by the sample. For example... Figure 1 As shown, this module also shares part of the optical path with the multifocal structured light excitation module, including optical path components such as objective lens 19, quarter-wave plate 17, fourth mirror 16, third mirror 15, and first dichroic mirror 14; in addition, the fluorescence signal detection module also includes: second tube mirror 21, second dichroic mirror 22, second filter 30, and second detector 23.

[0041] The first dichroic mirror 14 is used to separate the fluorescence signal from the excitation light. The first dichroic mirror 14 has the characteristics of high reflectivity to the excitation laser wavelength and high transmittance to the fluorescence wavelength emitted by the sample.

[0042] Therefore, the specific detection process is as follows: the fluorescence signal generated by the sample sequentially passes through objective lens 19, quarter-wave plate 17, fourth reflecting mirror 16, third reflecting mirror 15, and first dichroic mirror 14 into the optical path specifically designed for fluorescence detection. Subsequently, it passes through second tube mirror 21, second dichroic mirror 22 to further filter out residual excitation light, and second filter 30, before being acquired by second detector 23 (like another sCMOS camera) to obtain the desired multifocal array fluorescence signal. Objective lens 19 and second tube mirror 21 form a 4f system, with the target surface of second detector 23 located at the back focal plane of second tube mirror 21.

[0043] 5. Anti-drift module; The anti-drift module is used to monitor and compensate for sample axial (Z-direction) drift in real time during imaging, ensuring focal plane stability during long-term observation. For example... Figure 1 As shown, the module includes a reference light generation unit (second laser 27), a tilted illumination unit, a position detection unit (quadrant detector 28), and a feedback control unit (integrated with the control system module).

[0044] The tilted illumination unit shares the objective lens 19, quarter-wave plate 17, mirror assembly (fourth mirror 16, third mirror 15), and first dichroic mirror 14 with the aforementioned multifocal structured light excitation module. The tilted illumination unit shares the second dichroic mirror 22 and second tube mirror 21 with the aforementioned fluorescence signal detection module.

[0045] The specific anti-drift principle is as follows: The reference light generated by the second laser 27 passes through the fifth reflecting mirror 26 and enters the second polarizing beam splitter prism 25 with an appropriate offset from the optical axis of the microscope. It is emitted in the form of linearly polarized light and passes through the second dichroic mirror 22, the second tube mirror 21, the first dichroic mirror 14, the third reflecting mirror 15, the fourth reflecting mirror 16, the quarter-wave plate 17, and the objective lens 19 to reach the sample surface. The beam illuminates the sample at an angle. After being reflected by the sample, the emitted light also returns to the second polarizing beam splitter prism 25 in the same off-axis form. Since the direction of the emitted light is opposite to that of the incident light, after passing through the quarter-wave plate 17 and the second polarizing beam splitter prism 25, the polarization states of the emitted light and the incident light are orthogonal. Therefore, the emitted light will be reflected by the second polarizing beam splitter prism 25 and enter the four-quadrant detector 28. When the sample surface undergoes axial drift, the reference light reflected from the sample will also undergo lateral drift. The lateral offset of the reference light is read by the quadrant detector 28, and the control system module, i.e. the control terminal 32, reads the output signal of the quadrant detector 28 in real time, calculates the axial drift of the sample, and generates control commands to drive the single-axis displacement stage 18 (carrying objective lens 19 or sample stage 20) to perform reverse movement, thereby achieving closed-loop negative feedback compensation and locking the sample on the focal plane.

[0046] 6. Control system module; The core of the control system module is the control terminal 32, such as a computer, which is used to control the feedback adjustment of the anti-drift module and to control the synchronization of the digital micromirror array and the detector.

[0047] The control terminal is connected to both the single-axis displacement stage 18 and the four-quadrant detector 28. It controls the negative feedback adjustment of the anti-drift module to prevent axial drift of the sample throughout the detection process. It is also connected to the digital micromirror device 7, the first detector 24, and the second detector 23. This connection allows for the loading of a pre-designed multi-focal structured photomask onto the digital micromirror device 7 and for synchronizing the first detector 24 and the second detector 23 with the digital micromirror device 7 to acquire interference scattering and fluorescence signals. Specifically, it performs the following key synchronization and control functions: Illumination control: Send commands to the digital micromirror device 7 to load and switch different multifocal mask patterns.

[0048] Synchronous acquisition: The illumination mode of the digital micromirror device 7 is strictly synchronized with the exposure time of the first detector 24 and the second detector 23 to ensure that each multifocal pattern corresponds to one frame of image.

[0049] Drift compensation: Receive and process the signal from the four-quadrant detector 28, calculate the drift amount, and control the movement of the single-axis displacement stage 18.

[0050] Image reconstruction: Store and process the acquired raw image sequence, run the reconstruction algorithm, and finally generate a super-resolution image.

[0051] Example 2: Reference Figure 3 As shown, the present invention also provides a multimodal super-resolution microscopy imaging method based on multifocal structured light illumination, applied to the system as described in any one of Embodiments 1, the method comprising: S1: Generate a coherently modulated illumination beam; that is: activate the first laser 1, rotate the frosted glass 2, and generate a coherently modulated beam of expanded and collimated light.

[0052] S2: The illumination beam is modulated into a multifocal structured light array using a beam modulation device and projected onto the sample plane to excite interference scattering signals and / or fluorescence signals; S3: After adjusting the sample to a suitable focal plane using a three-dimensional displacement stage, the system achieves long-term focal plane stability by controlling the anti-drift module and the single-axis displacement stage through the control system module and performing negative feedback adjustment. S4: Synchronously acquire the interference scattering image sequence and / or fluorescence image sequence generated by the multifocal structured light excitation; for example, by controlling the synchronization of the digital micromirror device, the first detector, and the second detector through a control terminal, a series of interference scattering images and fluorescence images illuminated by multifocal structured light are acquired. It is worth noting that, due to the different optical paths of interference scattering and fluorescence, the two images may have translational or rotational deviations in the field of view when captured simultaneously. Typically, a registration algorithm based on image features is used for processing: using high-contrast biological structures such as microtubules and actin filaments in the fluorescence image as references, cross-correlation calculations or rigid body transformation matching are performed with the corresponding high-frequency contours in the interference scattering image to calculate and compensate for the translational and rotational deviations between the two, achieving high-precision multimodal image fusion.

[0053] S5: Reconstruct the acquired image sequence. This reconstruction process includes: calculating the spatial offset vector based on the images of each focal region, performing sub-pixel precision repositioning and superposition, and reconstructing the final super-resolution image. In this step, a series of interference scattering images and fluorescence images obtained from multifocal structured light illumination are processed as follows: Figure 4 The image reconstruction shown yields sample interference scattering and fluorescence images with extremely high signal-to-noise ratio and super-resolution. Here, super-resolution refers to breaking the traditional optical diffraction limit, causing the equivalent point spread function of the reconstructed image to shrink by approximately [percentage missing] compared to conventional wide-field or aperture confocal imaging. This achieves or approaches the theoretical lateral resolution limit of an ideal minimal closed-aperture confocal microscope system without compromising photon collection efficiency, i.e., without physically closing the pinhole.

[0054] Reference Figure 4 As shown, step S5 specifically includes: S51: Extract the micro-image region centered on each illumination focus from the multifocal structured light illumination image sequence; In practice, this step involves adding digital pinholes and performing phase cancellation based on radial variance transform (RVT). In a series of original multifocal structured light interference scattering images, digital pinholes are added around the center position of each focal point. The attributes of these digital pinholes are defined as follows: Size: In the experiment, a suitable digital pinhole needs to be selected according to the parameters of the objective lens. Its size should be about 1.5 times the diffraction limit of the objective lens.

[0055] Shape and Intensity Transmittance Model: The digital pinhole is set to a circular shape in space. Its intensity transmittance model is based on a spatial binary model, that is, the signal transmittance weight within the pixel photosensitive area is 1, and outside the area is 0; in order to further suppress background noise and sidelobe crosstalk, a two-dimensional Gaussian decay function can also be used as a virtual transmittance mask for the micro-image area.

[0056] Due to spatial phase modulation of coherent scattered light, the interferometric point spread function (iPSF) in the image often exhibits asymmetry or contrast reversal characteristics. To eliminate this phase interference, a radial variance transform (RVT) algorithm is applied to each extracted micro-image frame. The specific process is as follows: 1) Input parameter settings: Set the minimum and maximum evaluation radii of the concentric circles. The range of these radii must cover the effective size of the interference point spread function; 2) Local symmetry calculation: For each pixel in the micro-image, with that pixel as the center, calculate the variance of the pixel intensity values ​​on each concentric ring within the range from the minimum to the maximum evaluation radius. Based on the mean variance and mean-variance of the intensity variances of the concentric rings, evaluate the local radial symmetry of that pixel. 3) Feature output to generate pseudo-fluorescence maps: The radial symmetry value calculated for each pixel is used as the new pixel value for that point, thereby generating a series of distribution images that eliminate coherent phase dependence and only represent the spatial geometric center of the target. Because the scattering signal in this image is converted into a symmetrical light spot with a Gaussian distribution, it is morphologically equivalent to the fluorescence spot spread function of incoherent imaging, hence the name "pseudo-fluorescence map".

[0057] S52: For the interference scattering image sequence, apply the radial variance transformation algorithm to each extracted micro-image region to eliminate coherent phase interference and generate the corresponding pseudo-fluorescence image; for the fluorescence image sequence, directly use the micro-image region; based on the pseudo-fluorescence image or the micro-image region, calculate the spatial offset vector of each off-axis illumination focus relative to the central reference focus; S53: Using the spatial offset vector, the micro-image region obtained in step S51 is repositioned and superimposed with sub-pixel precision to reconstruct the super-resolution initial image. This step achieves subpixel-precision interpolation repositioning and superposition, as well as multi-frame redundancy removal: the subpixel-level spatial offset vector calculated in S52 is applied back to the image with added digital pinholes obtained in S51, and image repositioning and synthesis are performed. The specific implementation process includes: (1) Subpixel-level two-dimensional image interpolation: Since the calculated spatial offset vector usually contains non-integer subpixel displacement, in order to avoid quantization error, a high-precision two-dimensional image interpolation algorithm is used when repositioning and translating the off-axis original micro-image. Bicubic interpolation or spline interpolation is preferred to accurately map the translated off-axis interference signal into the preset high-resolution reconstruction grid space. (2) Linear superposition synthesis of signals: All off-axis micro-images that have been repositioned to the center position after sub-pixel repositioning are linearly superimposed with the corresponding pixels of the central micro-image. By accumulating rather than averaging, the coherent scattering signal energy collected by the entire array is maximized, thereby reconstructing the initial high signal-to-noise ratio, super-resolution interferometric scattering image; (3) Multi-frame image redundancy removal optimization: When processing multi-frame interferometric scattering image datasets containing three-dimensional depth or time series, in order to remove redundant calculations, an "offset vector reuse" strategy is adopted: the radial variance transformation and registration calculations of S51 and S52 are performed only on a single reference plane (such as the optimal focal plane) or reference time frame of the dataset to obtain a set of reference spatial offset vectors. Subsequently, this reference set is directly applied to all other frames in the dataset for sub-pixel relocation and superposition synthesis in this step, which greatly improves processing efficiency.

[0058] S54: Perform background correction on the initial image to obtain the final high-contrast super-resolution image.

[0059] In this step, a low-pass filter, preferably a Gaussian filter, is applied to the initial high-resolution interferometric scattering image obtained from the S53 reconstruction to extract a large-scale flat-field background image without structural features. The physical space size corresponding to the Sigma setting of the low-pass filter must be strictly larger than the point spread function (PSF) size of the single focal point in the system to ensure complete filtering out of the sample's high-frequency structural information. Subsequently, contrast is calculated using the initial image and the extracted flat-field background image. Specifically, the intensity values ​​of each pixel in the initial image are subtracted from the corresponding intensity values ​​of the background image, and then divided by the intensity value of the background image (i.e., ...). This eliminates macroscopic illumination inhomogeneity, resulting in a high-contrast, super-resolution interferometric scattering image.

[0060] It is important to note that when performing isomorphic reconstruction on multifocal fluorescence images, since fluorescence emission is an incoherent emission process and there is no phase-flipping interference from coherent light, the radial variance transform (RVT) algorithm in S51 is unnecessary. The remaining steps remain the same: the subpixel spatial offset vector calculation in S52 is directly performed using the original extracted fluorescence microimage, thereby completing the repositioning and overlay operation in S53. Furthermore, due to the extremely high molecular specificity and extremely low background of fluorescence itself, the flat-field contrast division step in S54 can be selectively skipped depending on the actual imaging requirements.

[0061] The imaging method provided by this invention is improved in two main aspects. In the excitation optical path, a beam modulation device, such as a digital micromirror, is used to control the light field, modulating the beam into multifocal structured light. High-resolution sample interference scattering and fluorescence images are obtained through image reconstruction. Multifocal structured light illumination possesses the high-resolution characteristics of a point-scanning system while offering faster imaging speed, which is significant for observing the nanoscale structure of samples. In the detection optical path, interference scattering and fluorescence signals of the sample under multifocal structured light illumination are simultaneously acquired, enabling the simultaneous tracking of specifically labeled molecules and unlabeled structural backgrounds, providing an important means for observing chemical changes during dynamic processes of the sample. Furthermore, the introduced anti-drift module meets the system's requirements for long-term observation of sample changes.

[0062] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0063] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multimodal super-resolution microscopy system based on multifocal structured light illumination, characterized in that, include: A laser beam generation module is used to generate and modulate the spatial coherence of the illumination beam; It includes a first laser, a rotating frosted glass, and a beam expanding and collimating unit; the rotating frosted glass and the beam expanding and collimating unit are sequentially arranged in the output optical path of the first laser to reduce the coherence of the illumination beam and then expand and collimate the beam to suppress interference background noise; A multi-focus structured light excitation module, optically connected to the laser beam generation module, includes a digital micromirror device for modulating the illumination beam into a multi-focus structured light array and projecting it onto the sample plane to excite the sample to generate interference scattering signals and / or fluorescence signals. An interference scattering signal detection module is used to collect the interference scattering signal generated after the sample is excited by the multifocal structured light, including a first detector; A fluorescence signal detection module is used to collect the fluorescence signal generated by the sample after it is excited by the multifocal structured light, including a second detector; The control system module is electrically connected to the digital micromirror device, the interference scattering signal detection module, and the fluorescence signal detection module, respectively. The control system module sends instructions to the digital micromirror device to load and switch different multifocal mask patterns. It also controls the synchronization of the digital micromirror device, the first detector, and the second detector to acquire a series of interference scattering images and fluorescence images of multifocal structured light illumination. The multi-focus structured light excitation module further includes a polarization control unit and a spatial filtering unit. The polarization control unit includes a half-wave plate, a first polarizing beam splitter prism, and a quarter-wave plate arranged sequentially along the optical path. The half-wave plate is used to adjust the polarization direction of the beam incident on the first polarizing beam splitter prism. The first polarizing beam splitter prism and the quarter-wave plate are combined to convert the linearly polarized excitation light into circularly polarized light and incident it on the sample. At the same time, the back-propagating interference scattering signal from the sample is reflected again by the first polarizing beam splitter prism to the interference scattering detection optical path after passing through the quarter-wave plate, thereby realizing the separation of the excitation optical path and the interference scattering detection optical path. The fluorescence signal detection module and the multifocal structured light excitation module share part of the optical path, including an objective lens, a mirror assembly for changing the direction of the optical path, and a first dichroic mirror for separating the fluorescence signal and the excitation light; The spatial filtering unit includes a first filtering lens, a spatial filtering aperture, and a second filtering lens arranged along the optical path, used to filter out the higher-order diffraction light generated by the digital micromirror device.

2. The system according to claim 1, characterized in that, The interference scattering signal detection module shares part of the optical path with the multi-focus structured light excitation module, and the excitation optical path and the detection optical path are separated by the polarization control unit.

3. The system according to claim 1, characterized in that, It also includes an anti-drift module, which comprises a reference light generation unit, a tilt illumination unit, a position detection unit, and a feedback control unit; The control system module is connected to the position detection unit and the feedback control unit, respectively. The reference light generated by the reference light generation unit is incident on the sample plane off-axis through the tilted illumination unit and is reflected; the control system module detects the lateral displacement of the reflected light by the position detection unit to calculate the axial drift, and controls the single-axis displacement stage to perform axial negative feedback compensation through the feedback control unit based on the drift.

4. The system according to claim 3, characterized in that, The tilted illumination unit of the anti-drift module shares the objective lens, quarter-wave plate, mirror assembly, and first dichroic mirror with the multifocal structured light excitation module, so that the anti-drift reference light and the imaging signal light partially overlap in the optical path when they reach the sample.

5. A multimodal super-resolution microscopy method based on multifocal structured light illumination, characterized in that, Applied to the system as described in any one of claims 1-4, the method comprises: S1: Generates a coherently modulated illumination beam; S2: The illumination beam is modulated into a multifocal structured light array using a beam modulation device and projected onto the sample plane to excite interference scattering signals and / or fluorescence signals; S3: After adjusting the sample to a suitable focal plane using a three-dimensional displacement stage, the system achieves long-term focal plane stability by controlling the anti-drift module and the single-axis displacement stage through the control system module and performing negative feedback adjustment. S4: Synchronously acquire the interference scattering image sequence and / or fluorescence image sequence generated by the multifocal structured light excitation; S5: Reconstruct the acquired image sequence. The reconstruction process includes: calculating the spatial offset vector based on the image of each focal region, and performing sub-pixel precision repositioning and superposition to reconstruct the final super-resolution image.

6. The method according to claim 5, characterized in that, Step S5 specifically includes: S51: Extract the micro-image region centered on each illumination focus from the multi-focus structured light illumination image sequence; S52: For the interference scattering image sequence, apply the radial variance transformation algorithm to each extracted micro-image region to eliminate coherent phase interference and generate the corresponding pseudo-fluorescence image; for the fluorescence image sequence, directly use the micro-image region; based on the pseudo-fluorescence image or the micro-image region, calculate the spatial offset vector of each off-axis illumination focus relative to the central reference focus; S53: Using the spatial offset vector, the micro-image region obtained in step S51 is repositioned and superimposed with sub-pixel precision to reconstruct the super-resolution initial image. S54: Perform background correction on the initial image to obtain the final high-contrast super-resolution image.

Images