Sparse scanning structured illumination super-resolution imaging system and method based on light field rotation

Abstract

The application discloses a kind of sparse scanning structure illumination super-resolution imaging system and method based on light field rotation, the system includes illumination unit, scanning rotation unit, microscopic imaging unit, control unit and super-resolution reconstruction unit, wherein illumination unit is used to configure light intensity and frequency under the control of control unit and emit parallel light beam;Scanning rotation unit is used to scan and rotate parallel light beam, form different directions, different phase shift amount sparse stripe structure light;Microscopic imaging unit is used to collect different directions, different phase shift amount sparse stripe structure light excited under original fluorescence image at different axial slice positions of sample;Control unit is used to control illumination unit, scanning rotation unit and microscopic imaging unit synchronously;Super-resolution reconstruction unit carries out super-resolution reconstruction to all original fluorescence images, obtains the three-dimensional super-resolution image of sample.The application can realize high-quality three-dimensional super-resolution imaging to thick sample.

Description

Technical Field

[0001] This invention belongs to the field of microscopic imaging technology, specifically relating to a sparse scanning structure illumination super-resolution imaging system and method based on light field rotation. Background Technology

[0002] Optical microscopes, with their advantages of being non-contact, low-damage, and capable of specific imaging, have become indispensable research tools in many fields such as biomedicine, life sciences, and materials science. However, limited by the diffraction limit, the lateral resolution of traditional optical microscopes is only about 200 nm, and the axial resolution is only about 500 nm, severely hindering further exploration of subcellular structures. Therefore, how to overcome the diffraction limit has become a major research topic in the field of optical microscopy.

[0003] With the continuous development of advanced lasers, highly sensitive detectors, and novel fluorescent probes, super-resolution optical microscopy, capable of breaking the diffraction limit, emerged in the early 21st century and has rapidly become an indispensable emerging technology in the life sciences. Currently, super-resolution optical microscopy is mainly divided into three types: stimulated emission depletion (STED) microscopy based on point spread function (PSF) compression; single-molecule localization microscopy (SMLM) that achieves time-division luminescence by controlling the "on" and "off" states of fluorescent molecules; and structured illumination microscopy (SIM) based on spatial spectrum spread. Among these three technologies, SIM technology has advantages such as fast imaging speed, low phototoxicity, no need for special fluorescent labeling, and strong tomographic capabilities, making it highly favored by biomedical researchers.

[0004] Simulation imaging (SIM) typically uses sinusoidal fringe structured light to illuminate the sample. By acquiring raw images with different directions and phase shifts, and combining them with post-processing algorithms, a super-resolution image of the sample can be reconstructed. However, the structured light generated by interference or projection in traditional SIM is essentially still wide-field illumination, and its penetration depth is easily affected by scattering from biological samples, generally limiting the imaging depth to within 20 μm, severely hindering the detailed interpretation of deeper information. Furthermore, the contrast of traditional SIM, which uses sinusoidal fringe structured light to illuminate the sample, decreases rapidly with increasing imaging depth, leading to a reduced signal-to-noise ratio of the original fluorescence image during deep imaging, significantly impacting the super-resolution reconstruction effect. Therefore, most SIM techniques aim to obtain super-resolution images of a thin layer of the sample surface, limiting their application in imaging thick biological samples.

[0005] While there are techniques that utilize focused scanning to enhance the penetration depth of SIM, the sinusoidal or near-sinusoidal striped structured light generated by these techniques still struggles to maintain contrast effectively when imaging thick samples. Furthermore, the combination of resonant scanning and light intensity modulation involved in these techniques makes it difficult to guarantee isotropy when generating striped light fields in different directions. Summary of the Invention

[0006] To address the aforementioned problems in the prior art, this invention provides a sparse scanning structure illumination super-resolution imaging system and method based on light field rotation. The technical problem to be solved by this invention is achieved through the following technical solution:

[0007] One aspect of the present invention provides a sparse scanning structure illumination super-resolution imaging system based on light field rotation, comprising an illumination unit, a scanning rotation unit, a microscopic imaging unit, a control unit, and a super-resolution reconstruction unit, wherein...

[0008] The lighting unit is configured to emit light intensity and frequency and to emit a parallel beam of light under the control of the control unit;

[0009] The scanning and rotating unit is used to scan and rotate the parallel beam generated by the illumination unit to form sparse stripe structured light with different directions and different phase shifts;

[0010] The microscopic imaging unit is used to acquire the original fluorescence images of the sparse stripe structure light excitation with different directions and different phase shifts at different axial slice positions of the sample;

[0011] The control unit is used to synchronously control the illumination unit, the scanning rotation unit, and the microscopic imaging unit;

[0012] The super-resolution reconstruction unit is used to perform super-resolution reconstruction on all the original fluorescence images acquired by the microscopic imaging unit to obtain a three-dimensional super-resolution image of the sample.

[0013] Another aspect of the present invention provides a sparse scanning structure illumination super-resolution imaging method based on light field rotation, comprising:

[0014] S1: Using a sparse scanning structured illumination super-resolution imaging system based on light field rotation, sparse stripe structured light with different directions and phase shifts is generated and used to excite fluorescence in the sample, so as to obtain the original fluorescence images of each axial slice of the sample under sparse stripe structured light illumination with different directions and phase shifts.

[0015] S2: Use the super-resolution reconstruction algorithm to solve for the high-order harmonics contained in the original fluorescence image at the current axial slice of the sample, discard the fundamental frequency component and fuse the high frequency component to obtain the two-dimensional super-resolution image at the current axial slice.

[0016] S3: Obtain two-dimensional super-resolution images of each axial slice of the sample, and then obtain three-dimensional super-resolution images of the sample.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0018] 1. This invention provides a sparse scanning structured illumination super-resolution imaging system based on optical field rotation. It generates sparse fringe structured light by rapidly scanning a focused spot, thereby exciting the sample for fluorescence. Compared to the wide-field sinusoidal fringe illumination used in traditional SIM technology, the focused spot can effectively overcome sample scattering and has a greater penetration depth. Simultaneously, the sparse fringe structured light better maintains fringe contrast with increasing imaging depth, thus obtaining a high signal-to-noise ratio original fluorescence image when imaging deep layers of thick samples. This ensures the quality of super-resolution reconstruction and enables large imaging depth and super-resolution imaging, with the imaging depth being more than an order of magnitude higher than that of traditional SIM technology.

[0019] 2. This invention achieves the generation of sparse fringe structured light with different directions and phase shifts by synchronously controlling a resonant scanning galvanometer, a linear scanning galvanometer, and a Dove prism. Resonant scanning boasts an ultra-high scanning speed of 12 kHz / line, which is 1-2 orders of magnitude faster than traditional linear scanning. Therefore, using resonant scanning instead of linear scanning effectively compensates for the inherent slowness of single-point scanning, thus significantly improving image acquisition speed. The Dove prism allows the generated sparse fringe structured light to be rotated in any direction, overcoming the limitation of two-dimensional galvanometer systems that can only quickly generate fringe structured light in one direction. This enables isotropic super-resolution reconstruction results through multi-directional, multi-step phase shifts.

[0020] 3. This invention develops a corresponding sparse SIM super-resolution image reconstruction algorithm for sparse fringe structured light illumination, which can improve spatial resolution by approximately 2 times, thereby achieving super-resolution imaging. Unlike the traditional SIM technology that uses equally spaced sinusoidal fringe illumination, this invention utilizes sparse fringe structured light with non-sinusoidal distribution characteristics to achieve fluorescence excitation. The original fluorescence image contains more higher-order harmonic components. Referring to the theoretical framework of nonlinear SIM reconstruction, this invention effectively demodulates the zero-frequency component and higher-order harmonic components in the frequency domain through multi-step phase shifting. By discarding the zero-frequency component, background noise can be filtered out, achieving the effect of light slice imaging. By superimposing higher-order harmonic components, effective spectral expansion of the system's optical transfer function can be achieved, thereby reconstructing a super-resolution image. Combining deconvolution techniques can further improve spatial resolution, ultimately achieving a 2-fold resolution improvement.

[0021] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a sparse scanning structure illumination super-resolution imaging system based on light field rotation provided in an embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of the overall structure of a sparse scanning structure illumination super-resolution imaging system based on light field rotation, provided in an embodiment of the present invention.

[0024] Figure 3 This is a timing diagram of various control signals for a sparse scanning structure illumination super-resolution imaging system based on light field rotation, provided in an embodiment of the present invention.

[0025] Figure 4 This is a schematic diagram of sparse stripe structured light with different directions and phase shifts acquired by a sparse scanning structured illumination super-resolution imaging system based on light field rotation provided in an embodiment of the present invention.

[0026] Figure 5 This is a schematic diagram of the imaging results of a monolayer fluorescent microsphere sample achieved using scanning wide-field imaging and sparse SIM super-resolution imaging.

[0027] Figure 6 This is a schematic diagram of the imaging results of a three-dimensional fluorescent microsphere sample achieved using scanning wide-field imaging and sparse SIM super-resolution imaging.

[0028] Explanation of reference numerals in the attached figures:

[0029] 101-Illumination unit; 102-Scanning rotation unit; 103-Microscopic imaging unit; 104-Control unit; 105-Super-resolution reconstruction unit; 1-Laser; 2-Fiber optic; 3-Beam expander collimating lens; 4-Resonant scanning galvanometer; 5-Linear scanning galvanometer; 6-Scanning lens; 7-First sleeve lens; 8-Dowell prism; 9-Dichroic mirror; 10-Microscopic objective lens; 11-Sample stage; 12-Filter; 13-Second sleeve lens; 14-Camera; 15-Computer; 16-Data acquisition and control card. Detailed Implementation

[0030] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following describes in detail, with reference to the accompanying drawings and specific embodiments, a sparse scanning structure illumination super-resolution imaging system and method based on light field rotation proposed in accordance with the present invention.

[0031] The foregoing and other technical contents, features, and effects of the present invention will be clearly presented in the following detailed description of specific embodiments in conjunction with the accompanying drawings. Through the description of the specific embodiments, a more in-depth and concrete understanding can be gained of the technical means and effects adopted by the present invention to achieve its intended purpose. However, the accompanying drawings are for reference and illustration only and are not intended to limit the technical solutions of the present invention.

[0032] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that an article or apparatus comprising a list of elements includes not only those elements but also other elements not expressly listed. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the article or apparatus that includes said element.

[0033] Example 1

[0034] Please see Figure 1 , Figure 1This is a schematic diagram of a sparse scanning structure illumination super-resolution imaging system based on light field rotation according to an embodiment of the present invention. The imaging system includes an illumination unit 101, a scanning rotation unit 102, a microscopic imaging unit 103, a control unit 104, and a super-resolution reconstruction unit 105. The illumination unit 101, under the control of the control unit 104, configures the light intensity and frequency and emits a parallel beam. The scanning rotation unit 102 scans and rotates the parallel beam generated by the illumination unit 101 to form sparse fringe structured light with different directions and phase shifts. The microscopic imaging unit 103 acquires raw fluorescence images excited by sparse fringe structured light with different directions and phase shifts at different axial slice positions of the sample. The control unit 104 synchronously controls the illumination unit 101, the scanning rotation unit 102, and the microscopic imaging unit 103. The super-resolution reconstruction unit 105 performs super-resolution reconstruction on all raw fluorescence images acquired by the microscopic imaging unit 103 to obtain a three-dimensional super-resolution image of the sample.

[0035] Please see Figure 2 , Figure 2 This is a schematic diagram of the overall structure of a sparse scanning structure illumination super-resolution imaging system based on light field rotation provided in an embodiment of the present invention. The illumination unit 101 of this embodiment includes a laser 1, an optical fiber 2, and a beam expander and collimator lens 3. The laser 1 is used to configure the light intensity and frequency and emit a laser beam under the control of the control unit 104. The optical fiber 2 is used to guide the laser beam to the beam expander and collimator lens 3. The beam expander and collimator lens 3 is used to collimate and expand the laser beam to form a parallel beam of fixed size.

[0036] Furthermore, the scanning rotation unit 102 in this embodiment includes a resonant scanning galvanometer 4, a linear scanning galvanometer 5, a scanning lens 6, a first sleeve lens 7, and a Dowell prism 8 arranged sequentially along the optical path direction. The spot diameter of the parallel beam from the beam expanding collimating lens 3 is comparable to the size of the scanning galvanometer. The resonant scanning galvanometer 4 and the linear scanning galvanometer 5 form a two-dimensional scanning galvanometer system for two-dimensional scanning of the parallel light generated by the illumination unit 101. The scanning lens 6 and the first sleeve lens 7 form an optical conjugate system, which directs the two-dimensionally scanned beam onto the Dowell prism 8. The Dowell prism 8 can rotate parallel to the optical axis to rotate the two-dimensionally scanned beam, forming sparse stripe structured light with different directions and different phase shifts.

[0037] Specifically, the resonant scanning mirror 4 and the linear scanning mirror 5 respectively focus on the parallel beam generated by the illumination unit 101 at mutually perpendicular angles. x direction and y The direction is scanned to form a two-dimensional scanned beam. x direction and yThe directions are mutually perpendicular and all perpendicular to the optical axis (z-direction). In this embodiment, the Dowell prism 8 is mounted on an electrically driven rotating device to control the Dowell prism 8 to rotate parallel to the optical axis, thereby realizing the rotation of the beam after two-dimensional scanning.

[0038] The microscopic imaging unit 103 of this embodiment includes a dichroic mirror 9, a microscope objective 10, a sample stage 11, a filter 12, a second sleeve lens 13, and a camera 14. The dichroic mirror 9 reflects the sparse fringe structured light from the Dove prism 8 to the entrance pupil of the microscope objective 10. The microscope objective 10 focuses the sparse fringe structured light onto the sample placed on the sample stage 11 to form fluorescence excitation, collects the fluorescence signal emitted by the sample, and transmits the fluorescence signal through the dichroic mirror 9 to the filter 12. The filter 12 filters out stray light other than the fluorescence signal. The second sleeve lens 13 and the microscope objective 10 form a conjugate imaging system, conjugately imaging the fluorescence signal onto the target surface of the camera 14. The camera 14 acquires the fluorescence signal to form the original fluorescence image of the sample. In this embodiment, the camera 14 is an sCMOS camera.

[0039] In this embodiment, the dichroic mirror 9 is used to separate the incident laser beam from the emitted fluorescence signal. Specifically, it reflects the two-dimensionally scanned laser beam (laser beam) emitted from the scanning rotation unit 102 to the entrance pupil of the microscope objective 10 and transmits the fluorescence signal collected by the microscope objective 10 to the subsequent imaging optical path. Specifically, the sparse fringe structured light from the Dowell prism 8 is reflected by the dichroic mirror 9 and enters the microscope objective 10, where it is focused onto the sample on the sample stage 11 and excites fluorescence. The fluorescence signal emitted by the sample is collected by the microscope objective 10, transmitted through the dichroic mirror 9, and further filtered by the filter 12 before being focused by the second sleeve lens 13 onto the target surface of the camera 14. Finally, the original fluorescence image is output from the camera 14.

[0040] Furthermore, the control unit 104 in this embodiment includes a computer 15 and a data acquisition control card 16. The computer 15 controls the data acquisition control card 16 to generate control signals. The data acquisition control card 16 uses these control signals to synchronously control the intensity modulation of the laser 1, the scanning process of the resonant scanning mirror 4 and the linear scanning mirror 5, the rotation of the Dove prism 8, the image acquisition of the camera 14, and the axial displacement of the sample stage 11. In this embodiment, the computer 15 uses software to control the data acquisition control card 16 to generate various analog signals to synchronously control the various devices in the imaging system.

[0041] In this embodiment, the data acquisition control card 16 can generate a first analog voltage signal under the control of the computer 15, which is used to control the resonant scanning mirror 4 along... x The direction is used to scan the parallel beam from the illumination unit 101, and the scanning displacement function of the resonant scanning mirror 4 is: ,in, A For amplitude, f The resonant scanning frequency is used in this embodiment. f It is always equal to 12kHz. t For time; based on the scanning period of the resonant scanning galvanometer 4, the data acquisition control card 16 can also generate a second analog voltage signal to control the linear scanning galvanometer 5 along the time. y The direction is used to scan the parallel beam, and the scanning displacement function of the linear scanning galvanometer 5 is: ,in, B The amplitude determines the scanning step size of the linear scanning galvanometer 5; by setting the timing of the analog voltage signal output by the data acquisition control card 16, the two-dimensional scanning galvanometer system can complete the scanning step size. x direction and y Synchronous scanning in the direction enables the generation and phase shift of sparse fringe structured light. Specifically, the resonant scanning galvanometer 4 starts scanning under the trigger of the first analog voltage signal and outputs a TTL signal synchronously based on the scanning trajectory; the second analog voltage signal is a step signal. At each rising edge of the TTL signal, the second analog voltage signal is increased by a preset voltage value to form a step signal. The number of levels of the step signal is equal to the number of stripes of the sparse fringe structured light.

[0042] The data acquisition and control card 16 is also used to generate a light intensity modulation signal for the laser 1 based on the TTL signal (i.e., according to the scanning period of the resonant scanning mirror 4), and synchronously control the laser 1 to turn on during the outward scanning process of the resonant scanning mirror 4 and turn off during the return scanning process of the resonant scanning mirror 4, so as to eliminate laser grazing between the generated sparse fringe structured light. The data acquisition and control card 16 is also used to control the Dowell prism 8 to rotate after a set of phase shifts in one direction of the sparse fringe structured light, thereby starting the phase shift of the sparse fringe structured light in the next direction. The data acquisition and control card 16 is also used to control the camera 14 to perform image acquisition according to the scanning period of the resonant scanning mirror 4 and the number of fringe of the sparse fringe structured light, so that the exposure time of a single frame image of the camera 14 is consistent with the generation time of a single sparse fringe structured light.

[0043] In addition, the data acquisition and control card 16 can also generate a step signal to synchronously control the sample stage 11 to step along the optical axis after acquiring a set of original fluorescence images of a certain axial slice of the sample, thereby realizing the acquisition of original fluorescence images of the sample at different axial slice positions, that is, realizing the acquisition of three-dimensional original fluorescence images of the sample.

[0044] Please see Figure 3 , Figure 3 This is a timing diagram of the control signals for a sparse scanning structure illumination super-resolution imaging system based on light field rotation, provided in an embodiment of the present invention. (Refer to...) Figure 3 By synchronizing the timing of the analog voltage signal output by the data acquisition control card 16, a sparse fringe structured light field can be generated. Specifically, by uniformly adding a phase-shift voltage to the level amplitude of the step signal controlling the linear scanning galvanometer 5 through the data acquisition control card 16, the sparse fringe structured light can be translated in the direction perpendicular to the fringe, and the translation amount is determined by the phase-shift voltage value. Typically, to ensure the quality of super-resolution image reconstruction, the sparse fringe structured light needs to illuminate the entire imaging field of view through equally spaced phase shifts. Therefore, the phase shift step size of the fringe should be ≤ the full width at half maximum (FWHM) of the optical system, and the translation distance should ideally be an integer number of pixels.

[0045] Furthermore, after the sparse fringe structured light completes multiple phase shifts in one direction, the data acquisition and control card 16 controls the Dowell prism 8 mounted on the electrically driven rotating device to rotate to the next direction by generating pulse signals of a specific frequency and number. Then, the above steps are repeated to begin multiple phase shifts of the fringes in the next direction. Typically, to ensure the isotropic resolution of the super-resolution reconstructed image, a two-dimensional plane of the sample needs to be excited sequentially using sparse fringe structured light in three equally spaced directions (e.g., 0°, 60°, and 120°). Correspondingly, a series of original fluorescence images of the sample excited by the sparse fringe structured light on that plane are recorded by the camera 14.

[0046] Furthermore, the data acquisition control card 16 synchronously controls the sample stage 11 to move at equal intervals along the optical axis by generating a stepped signal. Specifically, after acquiring a set of raw fluorescence images of a single-layer axial slice of the sample, the sample stage 11 moves a small distance away from the microscope objective 10, and then repeats the above-mentioned multi-step phase-shifting data acquisition process in three directions, thereby finally realizing the acquisition of three-dimensional raw fluorescence images of the sample.

[0047] In this embodiment, by synchronously controlling the laser 1, resonant scanning mirror 4, linear scanning mirror 5, Dowell prism 8, and camera 14 using the data acquisition and control card 16 according to the above method, sparse fringe structured light can be generated through scanning. Figure 2 The super-resolution imaging system shown places a plane mirror on the sample stage 11, which can experimentally obtain sparse fringe structured light, as shown in the results. Figure 4 As shown. In this embodiment, the fringe period is 2.4 μm and the fringe movement step size is 0.2 μm. Twelve phase shifts are performed in three directions: 0°, 60° and 120°, so that the sparse fringe structured light sweeps across the entire imaging field of view. Therefore, a total of 36 original fluorescence images need to be acquired for a single layer of the sample (a single axial slice).

[0048] See also Figure 3 For example, in this embodiment, the resonant scanning mirror 4 along... xThe direction is scanned at a speed of 12kHz / line (reciprocating oscillation). A high-level input triggers the oscillation, and the voltage magnitude determines the angle of the reciprocating oscillation, which is reflected in the optical field as the length range of the light spot's reciprocating motion, i.e., the length of the generated stripes. The resonant scanning galvanometer 4 simultaneously outputs a TTL signal while oscillating, and uses this TTL signal to set the synchronization control signal for other devices.

[0049] Linear scanning galvanometer 5 along y Directional scanning: By inputting different voltages, the linear scanning galvanometer 5 can be rotated through different angles, which is reflected in the optical field as the light spot... y Different positions in the direction. In order to generate sparse striped structured light, the linear scanning mirror 5 needs to rotate to the next angle after the resonant scanning mirror 4 completes one cycle (83.3 μs) of round trip. The corresponding control signal is a step signal (i.e., the second analog voltage signal), which is set to increase by a voltage value according to the rising edge of the TTL signal synchronously output by the resonant scanning mirror 4.

[0050] To avoid laser beam residue during the round trip, the laser is shut off at both ends of the outward journey and during the return journey of the resonant scanning mirror 4. Specifically, a beam intensity modulation signal for laser 1 is set based on the TTL signal synchronously output by the resonant scanning mirror 4. Using the rising edge of this TTL signal as a reference, a high level is set to turn on the laser after a small delay of 5μs, and a low level is set to turn off the laser after 70μs. The laser 1 is still controlled by a TTL signal with a period of 83.3μs and a high-level duration of 70μs. (See [link to relevant documentation]). Figure 3 The light intensity modulation signal in the middle.

[0051] Furthermore, during the scanning of a frame of original fluorescence image, the camera 14 is always in the exposure state, and the rising edge of its trigger signal coincides with the first rising edge of the TTL signal synchronously output by the resonant scanning galvanometer 4. After that, it remains in a high level state until the scanning of the last stripe in the field of view ends, that is, the falling edge of the last TTL signal high level coincides with the falling edge of the trigger signal of the camera 14.

[0052] In this embodiment, by synchronously controlling the laser 1, resonant scanning mirror 4, linear scanning mirror 5, Dowell prism 8 and camera 14, sparse stripe structured light can be generated and made to undergo 12-step phase shifts in each of the three directions of 0°, 60° and 120°, with each 12-step phase shift corresponding to a 12-step stripe movement.

[0053] The rotation in three directions requires the use of the Dowell prism 8. Specifically, the Dowell prism 8 is initially positioned at 0°, and after acquiring 12 frames of raw fluorescence images in the 0° direction, the Dowell prism 8 is rotated to the second direction (30°), and then 12 frames of raw fluorescence images are acquired in the 60° direction. Finally, the Dowell prism 8 is rotated to the third direction (60°), and then 12 frames of raw fluorescence images are acquired in the 120° direction. At this point, the acquisition of raw fluorescence images for one axial slice of the sample is complete.

[0054] If the sample to be tested is thick and requires three-dimensional imaging, then the aforementioned 12×3 frames of raw fluorescence images need to be acquired at each axial slice of the sample. A stepped signal is used to synchronously control the sample's movement. z The microscope moves in the direction (optical axis direction) by 0.5 μm each time, and for each movement, 12×3 frames of images of the sample are acquired on the focal plane of the microscope objective 10.

[0055] This invention provides a sparse scanning structured illumination super-resolution imaging system based on optical field rotation. It generates sparse fringe structured light by rapidly scanning a focused spot, thereby exciting the sample for fluorescence. Compared to the wide-field sinusoidal fringe illumination used in traditional SIM technology, the focused spot effectively overcomes sample scattering and has a greater penetration depth. Simultaneously, the sparse fringe structured light better maintains fringe contrast with increasing imaging depth, thus obtaining a high signal-to-noise ratio original fluorescence image when imaging deep layers of thick samples. This ensures high-resolution reconstruction quality and enables large imaging depth and super-resolution imaging, with the imaging depth being more than an order of magnitude higher than that of traditional SIM technology.

[0056] This invention generates sparse fringe structured light with different directions and phase shifts by synchronously controlling a resonant scanning galvanometer, a linear scanning galvanometer, and a Dove prism. Resonant scanning boasts an ultra-high scanning speed of 12 kHz / line, which is 1-2 orders of magnitude faster than traditional linear scanning. Therefore, using resonant scanning instead of linear scanning effectively compensates for the inherent slowness of single-point scanning, thus significantly improving image acquisition speed. The Dove prism allows the generated sparse fringe structured light to be rotated in any direction, overcoming the limitation of two-dimensional galvanometer systems that can only quickly generate fringe structured light in one direction. This enables isotropic super-resolution reconstruction results through multi-directional, multi-step phase shifts.

[0057] Example 2

[0058] Based on Embodiment 1, this embodiment provides a sparse scanning structure illumination super-resolution imaging method based on light field rotation, the method comprising:

[0059] S1: Using the sparse scanning structure illumination super-resolution imaging system based on light field rotation, sparse stripe structured light with different directions and different phase shifts is generated and the sample is fluorescence excited to obtain the original fluorescence image of each axial slice of the sample under sparse stripe structured light illumination with different directions and different phase shifts.

[0060] First, turn on the imaging system and place a plane mirror on the sample stage 11. Set the resonant scanning mirror 4 and the linear scanning mirror 5 to scan synchronously. Fix the Dove prism 8 in one direction. Adjust the position of the sample stage 11 so that the camera 14 can clearly image the sparse fringe structured light. The sparse fringe structured light on the sample surface has non-sinusoidal distribution characteristics. Taking the one-dimensional case as an example, it can be expressed by the following formula:

[0061] , ,

[0062] in, The sparse stripe structured light on the sample surface Indicates the fringe frequency. p The fringe period represents the fringe period of sparse fringe structured light. r Represents the spatial coordinates perpendicular to the stripe direction. m Indicates harmonic order, n Indicates the first n Step phase shift, M This indicates the highest order of harmonics contained in sparse fringe structured light. N Indicates the number of phase shift steps. Indicates the amplitude of different harmonics. express m The initial phase of the subharmonic, Indicates the first n The phase shift amount of the step phase shift.

[0063] Remove the plane mirror, place the sample on the sample stage 11, and readjust the position of the sample stage 11 so that the camera 14 can clearly image the fluorescence signal excited by the sparse stripe structured light. Here, the camera 14 collects the original fluorescence image of the sample. It can be represented as:

[0064] ,

[0065] in, Indicates the spatial distribution of the sample structure. Indicates background noise. Let be the point spread function of the imaging system. This indicates a convolution operation.

[0066] Subsequently, by synchronously controlling laser 1, resonant scanning galvanometer 4, linear scanning galvanometer 5, Dove prism 8, sample stage 11, and camera 14, the original fluorescence images of each axial slice of the sample under sparse stripe structured light illumination in different directions and with different phase shifts were recorded. Specifically, the sample stage 11 was manually moved laterally and axially to determine the position and thickness of the target region of the sample; the scanning range and scanning step size of the sample stage 11 were set, and the laser 1, resonant scanning galvanometer 4, linear scanning galvanometer 5, Dove prism 8, sample stage 11, and camera 14 were synchronously controlled to record the original fluorescence images of all axial slices of the target region of the sample under sparse stripe structured light illumination in different directions and with different phase shifts.

[0067] S2: Use a super-resolution reconstruction algorithm to solve for the high-order harmonics contained in the original fluorescence image at the current axial slice of the sample, discard the fundamental frequency component and fuse the high-frequency component to obtain a two-dimensional super-resolution image at the current axial slice.

[0068] In this step, a super-resolution reconstruction algorithm is used to solve for the high-order harmonics contained in the original fluorescence image at each axial slice of the sample. By discarding the fundamental frequency component and fusing the high-frequency component, background noise is filtered out and the super-resolution image is reconstructed to obtain the spatial domain super-resolution image of the current axial slice.

[0069] Specifically, the original fluorescence image at the current axial slice is transformed to the frequency domain for data processing. Specifically, a Fourier transform is performed on the original fluorescence image to obtain its expression in the frequency domain:

[0070] ,

[0071] in, This represents the spectrum of the original fluorescence image in the frequency domain. k The symbol "~" indicates that a Fourier transform is performed on the physical quantity below it, representing spatial frequency. The spectrum representing the light field of sparse structure illumination. The spectrum representing the sample structure, The spectrum representing background noise; The optical transfer function of the imaging system is denoted as . The Fourier transform form of .

[0072] In order to solve the corresponding The harmonic components of the second order need to be established. M +1 of the above equations, by setting from arrive of N The phase shift can meet the requirements here. N ≥2 M +1. Simplify and combine the constant terms to CAfterwards, order = By dividing by 2, we can establish the following system of equations:

[0073] ,

[0074] in, = / 2, for M The amplitude of the subharmonic. = / 2, The amplitude of the first harmonic. - They represent 1- M The initial phase of the subharmonic, N Indicates the number of phase shift steps. - They represent 1- N The phase shift amount of the step phase shift; This represents the different frequency components of the sample.

[0075] The different frequency components of the sample are obtained by solving the above system of equations, including: ,in, Includes the sample's defocus background and zero-frequency information, the values ​​of which do not change with the phase shift of the fringes; discard those that do not change with the phase shift. This achieves the filtering out of the out-of-focus background and removes other higher harmonics. After being moved to the correct position and superimposed, the frequency domain spectrum is expanded to obtain the expanded spectrum; then, an inverse Fourier transform is performed on the expanded spectrum to reconstruct a two-dimensional super-resolution image of the current axial slice of the sample.

[0076] S3: Repeat step S2 for all axial slices of the sample to obtain a two-dimensional super-resolution image at each axial slice, and then obtain a three-dimensional super-resolution image of the sample based on this.

[0077] Specifically, step S2 is repeated on the original fluorescence images of all axial slices of the sample to obtain two-dimensional super-resolution images of all axial slices; the two-dimensional super-resolution images of all axial slices of the sample are stacked along the axial direction to form a three-dimensional stereo image of the sample; the three-dimensional stereo image is subjected to an overall three-dimensional deconvolution operation to obtain a three-dimensional super-resolution image of the sample.

[0078] This invention develops a corresponding sparse SIM super-resolution image reconstruction algorithm for sparse fringe structured light illumination, which can improve spatial resolution by approximately 2 times, thereby achieving super-resolution imaging. Unlike the traditional SIM technology that uses equally spaced sinusoidal fringe illumination, this invention utilizes sparse fringe structured light with non-sinusoidal distribution characteristics to achieve fluorescence excitation. The original fluorescence image contains more higher-order harmonic components. Referring to the theoretical framework of nonlinear SIM reconstruction, this invention effectively demodulates the zero-frequency component and higher-order harmonic components in the frequency domain through multi-step phase shifting. By discarding the zero-frequency component, background noise can be filtered out, achieving the effect of light slice imaging. By superimposing higher-order harmonic components, effective spectral expansion of the system's optical transfer function can be achieved, thereby reconstructing a super-resolution image. Combining this with deconvolution techniques further improves spatial resolution, ultimately achieving a 2-fold resolution improvement.

[0079] Example 3

[0080] Based on the above embodiments, this embodiment provides a sparse scanning structure illumination super-resolution imaging method based on light field rotation, the method comprising:

[0081] Step 1: Place the fluorescent sample to be tested on the sample stage 11, manually move the sample stage 11 laterally and axially to determine the target area for imaging, and make it clearly imaged in the plane of z=0μm.

[0082] Step 2: Move the sample axially to determine the axial imaging thickness (taking 210μm as an example), and then set the axial scanning range from z=0μm to z=210μm. Set the axial scanning step size to 0.5μm, which corresponds to 420 scanning layers, that is, 420 axial slices need to be scanned.

[0083] Step 3: Sequentially acquire the original fluorescence images of all layers of the sample under sparse stripe structure light excitation in different directions and with different phase shifts.

[0084] Step 4: Using a super-resolution reconstruction algorithm, the original fluorescence images under different sparse stripe structured light illumination are processed to reconstruct a two-dimensional super-resolution image of the sample.

[0085] In this embodiment, for super-resolution imaging of a single-layer sample or a specific layer of a thick sample, after adjusting the displacement stage until the imaging area is clear, the axial scanning range is set from z=0μm to z=0μm (i.e., the axial position remains unchanged), and the lateral scanning field of view is 57.6×57.6 μm. 2The sample was sequentially excited using 36 sparse fringe structured light fields (fringe period of 2.4 μm, with 12 phase shifts in each of the 0°, 60°, and 120° directions), and 36 raw fluorescence images were simultaneously recorded by an sCMOS camera, with a total time of approximately 0.8 seconds. The different frequency components contained in the raw fluorescence images were then determined using the super-resolution reconstruction method described above. , , , , By discarding information containing the sample's out-of-focus background and zero-frequency information To filter out background noise, the other eight frequency components are shifted and superimposed to achieve spectral expansion. A Fourier transform is then performed on the expanded spectrum to improve resolution. Based on this, a further deconvolution operation is performed on the Fourier-transformed image to finally obtain the super-resolution image of the sample under test.

[0086] In this embodiment, the system PSF is used. FWHM Using the full width at half maximum (FWHM) as the diffraction limit, the corresponding system cutoff frequency can be expressed as: ,in, This represents the full width and height of the system PSF. This indicates the numerical aperture of microscope objective 10. The wavelength representing the fluorescence signal from the sample, the fringe frequency can be expressed as... , This represents the stripe period. The maximum frequency value obtained by the above algorithm is... This means that the sample's spectral range is from Expand to The corresponding resolution improvement factor in the spatial domain is Based on experimental parameters and empirical estimates, the system uses a 10mm microscope objective. NA The value is 1.2, the wavelength of the fluorescence signal is 0.68 μm, and the stripe period is... p The highest order of the solution is 2.4 μm. M If the resolution is 4, the resolution improvement factor is approximately 1.47. After deconvolution, a final resolution improvement of 2 times can be achieved.

[0087] To determine the resolution of the sparse scanning structure illumination super-resolution imaging system based on light field rotation in this embodiment, scanning wide-field imaging and sparse SIM super-resolution imaging were performed on a monolayer fluorescent microsphere sample with a diameter of 100 nm, an excitation wavelength of 635 nm, and an emission wavelength of 680 nm. The scanning wide-field image (hereinafter referred to as wide-field) was generated by superimposing the aforementioned 36 original fluorescence images, and the sparse SIM super-resolution image was generated by processing the original fluorescence images using the aforementioned super-resolution reconstruction algorithm. Experimental results can be found in [link to experimental results]. Figure 5 . Figure 5 A comparison was made between wide-field images and sparse SIM super-resolution images, among which, Figure 5 Image (a) in the image is a wide-field image of a monolayer fluorescent microsphere. Figure 5 (b) in the image is the corresponding sparse SIM super-resolution image. The comparison between the two can intuitively show the resolution improvement effect. Figure 5 (c) shows the fluorescence intensity curves of the wide-field image and the sparse SIM super-resolution image of the same fluorescent microsphere sample. The results show that the full width at half maximum (FWHM) of the wide-field image and the full width at half maximum (FWHM) of the sparse SIM super-resolution image of the fluorescent microsphere sample are 340 nm and 167 nm, respectively, achieving a 2-fold resolution improvement, which is consistent with the theoretical analysis. Figure 5 (d) shows the intensity curves of two closely spaced fluorescent microsphere samples. The results show that two fluorescent microsphere samples that are indistinguishable in the wide-field image are accurately identified in the sparse SIM super-resolution image, and the experimentally measured distance between them is 267 nm.

[0088] In this embodiment, for three-dimensional super-resolution imaging of thick samples, after adjusting the displacement stage until the imaging area is clear, the lateral scanning field of view is set to 57.6 × 57.6 μm. 2 The axial scanning range is from z = 0 μm to z = 210 μm, with an axial scanning step size of 0.5 μm, corresponding to 420 scanning layers. At each axial slice of the sample, a 36-sparse fringe structured light field is used to excite the sample layer by layer, and an sCMOS camera simultaneously records 36 × 420 = 15120 original fluorescence images, with a total time of approximately 6 minutes. The super-resolution image of each layer (i.e., each axial slice) is reconstructed using the super-resolution reconstruction algorithm described above. Then, all layers are stacked in three dimensions. Based on this, a global three-dimensional deconvolution operation is performed on the stacked three-dimensional image to finally obtain the three-dimensional super-resolution image of the sample.

[0089] Furthermore, to determine the three-dimensional imaging capability of the sparse structure illumination super-resolution imaging system based on light field rotation in this embodiment for thick samples, wide-field scanning and sparse SIM imaging were performed on a three-dimensional fluorescent microsphere sample embedded in agarose gel. The three-dimensional fluorescent microsphere sample had a diameter of 100 nm, an excitation wavelength of 635 nm, and an emission wavelength of 680 nm. The experimental results can be found in [link to experimental results]. Figure 6 , Figure 6 The comparison shows the wide-field 3D imaging (a) and sparse SIM 3D imaging (b) effects of the same 3D fluorescent microsphere sample. The results indicate that sparse SIM 3D imaging significantly improves both image resolution and signal-to-noise ratio compared to wide-field 3D imaging. Further quantitative analysis shows that, compared to wide-field 3D imaging, sparse SIM 3D imaging in this embodiment improves both lateral and axial resolution by approximately 2 times and the signal-to-noise ratio by more than 20 times within the scanned 210 μm axial range.

[0090] In summary, this invention proposes a sparse scanning structured illumination super-resolution imaging system and method based on optical field rotation, enabling three-dimensional super-resolution microscopic imaging of thick samples. First, resonant scanning is used to rapidly generate sparse fringe structured light with high penetration and high fringe contrast, achieving high signal-to-noise ratio excitation for thick samples, with a penetration depth more than an order of magnitude higher than traditional SIM technology. Second, a Dove prism is used to rotate the sparse fringe structured light to any direction, overcoming the limitation of two-dimensional galvanometer systems that can only rapidly generate unidirectional fringes, thus facilitating isotropic super-resolution reconstruction. Then, a super-resolution reconstruction algorithm is developed based on nonlinear SIM reconstruction theory, effectively demodulating high-order harmonic components from multi-step phase shifts, while simultaneously filtering background noise and improving spatial resolution. Finally, axial scanning and three-dimensional deconvolution techniques are combined to achieve high-quality three-dimensional super-resolution imaging of thick samples. Compared to scanning wide-field images, the sparse SIM super-resolution images obtained using the system and method of this invention achieve approximately a 2-fold increase in resolution and a more than 20-fold increase in signal-to-noise ratio. This system and method can be applied to the biomedical field, and have a positive theoretical and technological impact on the development and application of three-dimensional super-resolution optical microscopy for in vivo imaging.

[0091] In the several embodiments provided by this invention, it should be understood that the systems and methods disclosed in this invention can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0092] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated module can be implemented in hardware or in the form of hardware plus software functional modules.

[0093] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A sparse scanning structure illumination super-resolution imaging method based on light field rotation, characterized in that, include: S1: Using a sparse scanning structure illumination super-resolution imaging system based on light field rotation, sparse stripe structure light with different directions and phase shifts is generated and used to excite fluorescence in the sample, so as to obtain the original fluorescence image of each axial slice of the sample under sparse stripe structure light illumination with different directions and phase shifts. S2: Use the super-resolution reconstruction algorithm to solve for the high-order harmonics contained in the original fluorescence image at the current axial slice of the sample, discard the fundamental frequency component and fuse the high frequency component to obtain the two-dimensional super-resolution image at the current axial slice. S3: Obtain two-dimensional super-resolution images of each axial slice of the sample, and then obtain three-dimensional super-resolution images of the sample; S2 includes: The original fluorescence image at the current axial slice is converted to the frequency domain, yielding the frequency domain expression: ， in, This represents the spectrum of the original fluorescence image in the frequency domain. k The symbol "~" indicates the spatial frequency, and "~" indicates that a Fourier transform is performed on the physical quantity below it. The spectrum representing the light field of sparse structure illumination. The spectrum representing the sample structure, The spectrum representing background noise. The optical transfer function of the imaging system is denoted as . The Fourier transform form, Let be the point spread function of the imaging system; Using the spectrograms of multiple raw fluorescence images recorded at the current axial slice, a system of equations is constructed: ， in, M This indicates that the sparse striped structured light contains the highest order of harmonics. = / 2, for M The amplitude of the subharmonic. = / 2, The amplitude of the first harmonic. - They represent 1- M The initial phase of the subharmonic, N Indicates the number of phase shift steps. - They represent 1- N The phase shift amount of the step phase shift; Represents the different frequency components of the sample. Indicates the fringe frequency. p The fringe period represents the sparse fringe structured light; The different frequency components of the sample were solved based on the equations. ,in, It contains the defocus background and zero-frequency information of the sample, and its value does not change with the phase shift of the fringes; Discard those that do not change with phase shift. Other higher harmonics After being moved to the correct position, the superimposed spectra are obtained; The expanded spectrum is subjected to inverse Fourier transform to reconstruct a two-dimensional super-resolution image of the current axial slice of the sample.

2. The sparse scanning structure illumination super-resolution imaging method based on light field rotation according to claim 1, characterized in that, S3 includes: Repeat step S2 on the original fluorescence images of all axial slices of the sample to obtain two-dimensional super-resolution images of all axial slices; Two-dimensional super-resolution images of all axial slices of the sample are stacked along the axial direction to form a three-dimensional image of the sample. A global 3D deconvolution operation is performed on the 3D image to obtain a 3D super-resolution image of the sample.

3. The sparse scanning structure illumination super-resolution imaging method based on light field rotation according to claim 1, characterized in that, The sparse scanning structure illumination super-resolution imaging system based on light field rotation includes an illumination unit (101), a scanning rotation unit (102), a microscopic imaging unit (103), a control unit (104), and a super-resolution reconstruction unit (105), wherein, The lighting unit (101) is configured to emit light intensity and frequency and to emit a parallel beam of light under the control of the control unit (104); The scanning and rotating unit (102) is used to scan and rotate the parallel beam generated by the illumination unit (101) to form sparse stripe structured light with different directions and different phase shifts; The microscopic imaging unit (103) is used to acquire the original fluorescence images of the sparse stripe structure light excitation with different directions and different phase shifts at different axial slice positions of the sample; The control unit (104) is used to synchronously control the illumination unit (101), the scanning rotation unit (102), and the microscopic imaging unit (103); The super-resolution reconstruction unit (105) is used to perform super-resolution reconstruction on all the original fluorescence images acquired by the microscopic imaging unit (103) to obtain a three-dimensional super-resolution image of the sample.

4. The sparse scanning structure illumination super-resolution imaging method based on light field rotation according to claim 3, characterized in that, The illumination unit (101) includes a laser (1), an optical fiber (2), and a beam-expanding collimating lens (3). The scanning rotation unit (102) includes a resonant scanning mirror (4), a linear scanning mirror (5), a scanning lens (6), a first sleeve lens (7), and a Dove prism (8) arranged sequentially along the optical path. The resonant scanning mirror (4) and the linear scanning mirror (5) constitute a two-dimensional scanning mirror system, which are used to scan the parallel light beam generated by the illumination unit (101) along... x direction and y The direction is scanned to form a two-dimensional scanned beam. x direction and y The directions are mutually perpendicular and all perpendicular to the optical axis; The scanning lens (6) and the first sleeve lens (7) constitute an optical conjugate system, which directs the two-dimensional scanned light beam onto the Dowell prism (8); The Dowell prism (8) can rotate parallel to the optical axis to rotate the two-dimensional scanned beam and form sparse stripe structured light with different directions and different phase shifts.

5. The sparse scanning structure illumination super-resolution imaging method based on light field rotation according to claim 4, characterized in that, The microscopic imaging unit (103) includes a dichroic mirror (9), a microscope objective (10), a sample stage (11), a filter (12), a second sleeve lens (13), and a camera (14), wherein, The dichroic mirror (9) is used to reflect the sparse fringe structured light from the Dowell prism (8) to the entrance pupil of the microscope objective (10); The microscope objective (10) is used to focus the sparse stripe structured light onto the sample set on the sample stage (11) to form fluorescence excitation, collect the fluorescence signal emitted by the sample and transmit the fluorescence signal through the dichroic mirror (9) to the filter (12). The filter (12) is used to filter stray light other than the fluorescence signal; the second sleeve lens (13) and the microscope objective (10) form a conjugate imaging system to conjugate the fluorescence signal onto the target surface of the camera (14); the camera (14) is used to acquire the fluorescence signal and form the original fluorescence image of the sample.

6. The sparse scanning structure illumination super-resolution imaging method based on light field rotation according to claim 5, characterized in that, The control unit (104) includes a computer (15) and a data acquisition control card (16), wherein, The computer (15) is used to control the data acquisition control card (16) to generate control signals; The data acquisition control card (16) is used to synchronously control the intensity modulation of the laser (1), the scanning process of the resonant scanning mirror (4) and the linear scanning mirror (5), the rotation of the Dove prism (8), the image acquisition of the camera (14), and the axial displacement of the sample stage (11) using the control signal.

7. The sparse scanning structure illumination super-resolution imaging method based on light field rotation according to claim 6, characterized in that, The data acquisition control card (16) can generate a first analog voltage signal under the control of the computer (15) to control the resonant scanning mirror (4) along... x The direction is scanned across the parallel beam; The data acquisition control card (16) can also generate a second analog voltage signal to control the linear scanning mirror (5) along... y The direction is scanned across the parallel beam; The resonant scanning mirror (4) performs scanning under the trigger of the first analog voltage signal and outputs a TTL signal synchronously based on the scanning trajectory. The second analog voltage signal is a step signal. At each rising edge of the TTL signal, the second analog voltage signal is increased by a preset voltage value.

8. The sparse scanning structure illumination super-resolution imaging method based on light field rotation according to claim 7, characterized in that, The data acquisition control card (16) is also used to generate a light intensity modulation signal for the laser (1) based on the TTL signal, and synchronously control the laser (1) to turn on the laser during the outward scanning process of the resonant scanning mirror (4) and turn off the laser during the return scanning process of the resonant scanning mirror (4) to eliminate the laser grazing between the generated sparse stripe structured light. The data acquisition control card (16) is also used to control the rotation of the sparse stripe structured light in one direction after a set of phase shifts, so as to start the phase shift of the sparse stripe structured light in the next direction.

9. The sparse scanning structure illumination super-resolution imaging method based on light field rotation according to claim 8, characterized in that, The data acquisition control card (16) is also used to control the camera (14) to acquire images according to the scanning period of the resonant scanning galvanometer (4) and the number of stripes of the sparse stripe structured light, so that the exposure time of a single frame image of the camera (14) is consistent with the generation time of a single sparse stripe structured light.

Images