Square lattice structure light illumination super-resolution microscopy system and method

Abstract

This invention discloses a square lattice structured light-illuminated super-resolution microscopy system and method. The system integrates a square blazed grating hologram generation module, a digital lens hologram generation module, and a composite hologram generation module on its phase modulator. Wavefront phase modulation of the incident beam is achieved by loading the composite hologram onto the incident beam. The acquired raw image is pre-filtered, multiplied by a pre-calculated weighting function, and superimposed to obtain an intermediate image. Finally, deconvolution is performed on the intermediate image to obtain the final super-resolution image. This invention concentrates light energy into effective diffraction orders through the loaded square blazed grating hologram, improving the system's light energy utilization efficiency. Combining the digital lens hologram and adjusting the lens focal length allows control of the focusing position of the diffraction orders. Furthermore, adjusting optical elements enables switching between different magnification objectives, meeting the needs of multi-scale sample observation.

Description

Technical Field

[0001] This invention belongs to the field of optical technology and relates to a structured illumination micro-imaging system, which can be widely used in fields such as biology, medicine, microelectronics and materials science. Background Technology

[0002] Optical microscopy plays a crucial role in basic scientific research and clinical diagnosis. However, traditional optical microscopes are limited by the diffraction limit, with spatial resolution typically restricted to 200 nm, making it difficult to resolve fine structures at the subcellular level. With the continuous development of novel fluorescent probes, optoelectronic devices, and imaging principles, a series of super-resolution optical microscopy techniques have emerged, successfully overcoming the diffraction limit and becoming important tools in modern biomedical research. Among these super-resolution techniques, stimulated emission depletion microscopy (STED), photoactivated localization microscopy (PALM), stochastic optical reconstruction microscopy (STORM), and structured illumination microscopy (SIM) are the most representative. SIM technology, in particular, boasts advantages such as fast imaging speed, low phototoxicity, and compatibility with various fluorescent labels, making it ideal for live-cell imaging. It provides a key means for resolving cellular structural composition and the interactions between its components, greatly advancing the development of cell biology.

[0003] Traditional striped structured illumination microscopy (SIM) is a typical representative of SIM technology and is widely used in scientific research and commercial applications. However, striped SIM typically requires nine original images to reconstruct a single super-resolution image. When observing biological samples for extended periods, this multi-frame acquisition mode results in a significantly higher photon dose to the sample compared to traditional wide-field microscopy, leading to strong phototoxicity and photobleaching. To reduce phototoxicity, the most direct solution in existing technologies is to reduce the number of original images to decrease the photon dose applied to the sample. However, this undersampling method often results in pathological problems in image reconstruction, easily leading to numerous artifacts that affect image resolution. Another approach is to shorten the single-frame exposure time, but excessively short exposure times cause a significant drop in the image signal-to-noise ratio, affecting the image reconstruction quality. How to improve and reduce the system's phototoxicity without compromising image reconstruction quality remains a key scientific problem to be solved in the field of structured illumination microscopy. Furthermore, traditional striped SIM systems use relatively simple objectives with fixed magnification, making them unsuitable for imaging samples of different sizes. Summary of the Invention

[0004] In view of the defects or deficiencies of the prior art, the present invention provides a square lattice structure light illumination super-resolution microscopy system.

[0005] Therefore, the square lattice structure light illumination super-resolution microscopy system provided by the present invention includes a laser source, a beam expander and collimating lens group, a polarizing beam splitter, a programmable phase modulator, a spatial filter, a polarizing optical element, a confocal lens group, a dichroic mirror, an objective lens, a lens, a filter, and a camera.

[0006] The emitted light from the laser source is incident on a polarizing beam splitter via a beam expander and collimating lens group. After passing through the polarizing beam splitter, the incident light is polarized. The polarized light is then incident on a programmable phase modulator via a reflecting prism. After phase modulation by the programmable phase modulator, multi-order diffracted light is generated. The multi-order diffracted light is then incident on a spatial filter via a reflecting prism. After the spatial filter blocks the light of the 0th order and other diffraction orders, four first-order diffracted lights are retained to pass through. The polarization state of the four first-order diffracted lights is adjusted by polarizing optical elements. After being scaled by a confocal lens group, the beams are then passed through a dichroic mirror and enter the objective lens. The four diffracted lights interfere at the focal plane of the objective lens to form a square lattice beam. A square lattice illumination beam irradiates the sample to produce fluorescence. The fluorescence is collected by the objective lens and then passes through a dichroic mirror, a lens, and a filter to form an image. Finally, the camera acquires the raw image. The programmable phase modulator includes a phase modulator, and the programmable phase modulator also integrates a square blazed grating hologram generation module, a digital lens hologram generation module, and a composite hologram generation module; The square blazed grating hologram generation model is used to generate square blazed grating holograms, and the phase distribution function of the square blazed grating hologram is described. for: (1) In formula (1): , This represents the total number of pixels (columns) in the horizontal direction (x-direction) of the programmable phase modulator. , This represents the total number of pixels (rows) in the vertical direction (y-direction) of the programmable phase modulator. ; This represents the maximum absolute value of coordinates m and n; The grating period; its value ranges from 4 to 16. This represents the remainder operation with respect to P; Let be the phase shift along the y-direction of the lattice optical field; The phase shift in the x-direction of the lattice light field is represented by the value [0, 2pi / 3, 4pi / 3]; i and j represent the phase shift steps in the two orthogonal directions x and y, respectively, i=1, 2, 3, j=1, 2, 3; The digital lens hologram generation module is used to generate digital lens holograms, the phase distribution function of which is... for: (2) In formula (2): The wavelength of the light emitted by the laser source; The focal length of the digital lens in the digital lens hologram generation model, in mm; The center-to-center distance between two adjacent pixels on the programmable phase modulator, in micrometers (µm). This indicates a remainder operation, and the result is then restricted to a certain range. Within the range; The composite hologram generation module is used to... and The sums are used to obtain the initial hologram, and then the phase of the initial hologram is constrained to... Within the range, a composite hologram is obtained; The phase modulator modulates the wavefront phase of the incident beam by loading a composite hologram onto the incident beam.

[0007] Alternatively, the programmable phase modulator can be a silicon-based liquid crystal spatial light modulator (SLM).

[0008] An alternative approach is to select an objective lens with the appropriate magnification based on the focal length of the digital lens and the focal length of the confocal lens group in the digital lens hologram generation module.

[0009] Alternatively, the polarizing optical element may be selected from a quarter-wave plate, a half-wave plate, or a partitioned polarizer.

[0010] The alternative is, The value can be [0, 2pi / 3, 4pi / 3]; The value is [0, 2pi / 3, 4pi / 3].

[0011] An optional solution is that the system also includes a displacement stage (15), on which a sample stage is provided. By controlling the movement of the displacement stage on the X and Y axes, the camera can acquire two-dimensional images of the region of interest of the sample on the sample stage. By moving the displacement stage on the Z axis, two-dimensional images of different layers of the sample can be acquired.

[0012] This invention also provides a method for super-resolution microscopy of a related square lattice structure under light illumination, the method comprising the following steps: Step 1: Use the above system to acquire 9 raw fluorescence images with different phase shifts and tetragonal lattice light fields: (3) The image shows the original fluorescence images with phase shift steps i and j; i and j represent the phase shift steps in the two orthogonal directions x and y, respectively, i=1, 2, 3, j=1, 2, 3; x and y are the two-dimensional plane coordinates of the system. This represents the displacement in the x and y directions; This represents any coordinate point of the sample in a two-dimensional plane. fluorescence intensity at that location This represents the intensity distribution of the lattice light field relative to the coordinate points. Displacement occurs. Represents coordinate points The degree of blurriness of the sample; Step 2, calculate the original fluorescence image using equation (4). Weighted image : (4) In formula (2): (5) (6) In equations (5) and (6): These represent the wave vectors of the lattice light fields in the x and y directions, respectively; These represent the initial phases of the lattice light fields in the x and y directions, respectively; Indicates the phase shift step size; These represent the modulation intensities of the lattice optical field in the x and y directions, respectively; Step 3: Calculate the pre-filtered original image using equation (7). : (7) In equation (7), The complex conjugate of the system's optical transfer function; The inverse Fourier transform operator; k is the AND operator. Corresponding frequency domain coordinates; Original fluorescence image Fourier transform; (8) In equation (8): This is the attenuation amplitude parameter; These are adjustable empirical parameters; Step 4: Use equation (9) to process the nine pre-filtered original images. With corresponding weighted images Perform dot products on each of the nine dot products and then sum the results to obtain the image before deconvolution. : (9) Step 5: Apply a deconvolution algorithm to the image obtained in Step 4. After deblurring and other optimization processes, the final super-resolution image is obtained.

[0013] The alternative is, The value range is 0 to 1; The value range is 0.5 to 2.5.

[0014] Alternatively, the deconvolution algorithm can be Wiener deconvolution, RL deconvolution, or TV deconvolution.

[0015] This invention also relates to related storage media and software products, wherein the storage media stores a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0016] The method of this invention uses four-beam interference to generate a square lattice illumination field without sacrificing any image quality such as signal-to-noise ratio. This reduces the photon dose applied to the sample by half compared to stripe illumination, significantly reduces system phototoxicity, and improves temporal resolution.

[0017] In addition, the present invention concentrates light energy into effective diffraction orders by loading a square blazed grating hologram, thereby improving the light energy utilization efficiency of the system; by combining a digital lens hologram and adjusting the lens focal length, the focusing position of the diffraction orders can be controlled, and by adjusting the optical elements, the function of switching between different magnification objectives can be realized, thus meeting the needs of multi-scale sample observation.

[0018] Furthermore, this invention uses digital lenses instead of traditional lenses, making the adjustment of the optical path simple and controllable, while shortening the optical path length and making the system more compact and stable. Attached Figure Description

[0019] Figure 1 The diagram shows the optical path of a four-beam interferometric square lattice SIM system based on spatial light modulator modulation and laser illumination used in an embodiment of the present invention. The reference numerals are as follows: 1-Laser illumination source, 2-Lens, 3-Lens, 4-Polarizing beam splitter, 5-Mirror, 6-Reflecting prism, 7-Programmable phase modulator, 8-Spatial filter, 9-Polarizing optical element, 10-Lens, 11-Lens, 12-Dichroic mirror, 13-Mirror, 14-Microscope objective, 15-Stage, 16-Bullet lens, 17-Filter, 18-Camera.

[0020] Figure 2 shows examples of a square blazed grating (a), a digital lens hologram (b), and a superimposed hologram (c) in the embodiments; Figure 3 The results are for HeLa cell microtubules; (a) shows nine original fluorescence images with lattice patterns; (b) shows the wide-field imaging results; (c) shows the super-resolution results using the reconstruction algorithm.

[0021] Figure 4 shows the experimental results comparing the microtubule structure of EGFP-labeled HeLa live cells under the lattice illumination of the present invention (a) and the existing stripe illumination method (b). Detailed Implementation

[0022] Unless otherwise specified, the scientific and technical terms used in this article are intended for understanding by those skilled in the art.

[0023] The following examples will use... Figure 1 Taking the four-beam interferometric square lattice illumination SIM system based on spatial light modulator modulation and laser illumination as an example, the specific implementation of this method will be described in detail.

[0024] Unless otherwise specified, the scientific and technical terms used herein are for the understanding of one of ordinary skill in the art. The following are specific embodiments provided by the inventors to further explain the solutions of the present invention.

[0025] Example: This embodiment adopts Figure 1 The four-beam interferometric square lattice illumination SIM super-resolution microscopy system based on spatial light modulator modulation and laser illumination was used to image microtubules in HeLa cells. The acquired original images were then reconstructed to obtain super-resolution images. The specific implementation scheme is as follows: Figure 1 The system shown includes a four-wavelength laser illumination source 1, a lens group 2 and 3 disposed behind the light source for beam expansion and collimation, a polarizing beam splitter 4 disposed behind lens 3, a reflector 5 disposed behind the polarizing beam splitter 4 for adjusting the beam direction, a reflecting prism 6 disposed behind the reflector 5, a programmable phase modulator 7, a spatial filter 8 disposed behind the programmable phase modulator 7 and the reflecting prism 6, a quarter-wave plate 9 disposed behind the programmable phase modulator 7 and the reflecting prism 6, a confocal lens group composed of lenses 10 and 11 disposed behind the quarter-wave plate 9, a dichroic mirror 12 disposed behind the confocal lens group, a reflector 13 disposed on the reflected light path of the dichroic mirror 12 for modulating the beam direction, a microscope objective 14 and an XYZ axis displacement stage 15, a lens 16 disposed on the transmitted light path of the dichroic mirror 12, an emission filter 17, and an SCMOS camera 18; in this embodiment, the XYZ axis displacement stage 15 is located above the objective, and the sample stage disposed on the displacement stage is located within the field of view of the objective. In this embodiment, the programmable phase modulator 7 is a silicon-based liquid crystal spatial light modulator (with 1024 pixels). 1024) It includes a phase modulator, and the programmable phase modulator integrates a square blazed grating hologram generation module, a digital lens hologram generation module, and a composite hologram generation module; this embodiment This is the grating period, specifically 14; , The values ​​are [0, 2pi / 3, 4pi / 3]; the center-to-center spacing of SLM pixels is The value is 17um; The value is 550 mm. The specific work process includes the following steps: Original image acquisition: Laser wavelength in this embodiment =561nm, after passing through lenses 2 and 3, the beam is expanded and collimated, and then horizontally linearly polarized light is generated by polarization beam splitter 4. The beam then passes through mirror 5 and mirror 6 and is incident on programmable phase modulator 7. A composite hologram generated based on the method of this invention is loaded on programmable phase modulator 7. The incident beam is diffracted and focused into a multi-level diffraction field with a specific spatial distribution. The spatial filter 8 blocks the light of other diffraction orders and retains 4 first-order diffraction beams to pass through. The beam's polarization state is adjusted to circular polarization state by quarter-wave plate 9. Then, the beam enters the microscope objective 14 through the confocal system composed of lens 10 and lens 11, as well as the dichroic mirror 12 and mirror 13. In this embodiment, the HeLa cell sample is placed on the stage 15 and adjusted to the focal plane of the objective lens. Four inclined circularly polarized beams interfere at the focal plane and form a square lattice illumination field, which excites the HeLa cell microtubules to produce fluorescence. The fluorescence signal of the HeLa cell microtubules is collected by the objective lens and returns along the original path through the mirror 13 and the dichroic mirror 12, and then through the lens 16 and the filter 17 to form an image. Finally, the image is acquired by the SCOMS camera 18 to obtain the original fluorescence image.

[0026] In other schemes, the lattice light field can be phase-shifted by replacing the hologram loaded on the programmable phase modulator, thereby acquiring nine different original images for reconstruction. Adjusting the displacement stage 15 moves the sample along the XYZ axes, allowing the camera to acquire two-dimensional images of the region of interest. Performing a Z-axis tomographic scan of the sample yields two-dimensional images of different layers, which are then superimposed to obtain three-dimensional image information of the sample. In addition, by replacing the digital lens hologram module and replacing lens 10 and lens 11, as well as the corresponding magnification objective lens, different magnification objective lenses can be replaced without changing the optical path, thus meeting the needs of multi-scale sample observation. The method of this invention is used to reconstruct nine original fluorescence images with different phase shifts and tetragonal lattice light fields on the same plane; wherein The values ​​are [-74.99, -70.07] and [-69.68, 75.19]; The values ​​are -2.1913 and -1.5632; The value is 2pi / 3; The value is 1.

[0027] In this embodiment, the system mainly uses an objective lens with a magnification of 100, and the focal length of lens 10 is... The focal length of lens 11 is 150 mm. The value is set to 200 mm. (To achieve switching to other magnification objectives, calculate the magnification relationship between the lenses to ensure the back pupil of the four first-order diffraction beams fills the objective lens, and then change the square blazed grating period P and the digital lens focal length.) And the focal lengths of lens 10 and lens 11, to enable switching between different magnification objectives.

[0028] This embodiment The value is 0.99; =1; This embodiment uses RL deconvolution on the image obtained in step 4. After optimization, the final super-resolution image is shown in Figure 4(a). Compared with the existing striped illumination result Figure 4(b) (which was obtained using the technology related to "high-speed reconstruction of micro-images with obvious super-resolution structured illumination" published by Wang et al. in Advanced Photonics in 2022), the cell microtubules under lattice illumination of the present invention can still maintain a high signal intensity and structural integrity after more than ten minutes, while the microtubules under striped illumination show obvious fluorescence decay and signal loss.

Claims

1. A super-resolution microscopy system with a square lattice structure illuminated by light, characterized in that, It includes a laser source (1), a beam expander collimating lens group, a polarizing beam splitter (4), a programmable phase modulator (7), a spatial filter (8), a polarizing optical element (9), a confocal lens group, a dichroic mirror (12), an objective lens (14), a lens (16), a filter (17), and a camera (18). The emitted light from the laser source is incident on the polarization beam splitter through the beam expanding and collimating lens group. The incident light is polarized after passing through the polarization beam splitter. The polarized light is incident on the programmable phase modulator through the reflecting prism (6). After phase modulation by the programmable phase modulator, multi-order diffraction light is generated. The multi-order diffraction light is incident on the spatial filter through the reflecting prism (6). After the spatial filter blocks the light of the 0th order and other diffraction orders, four first-order diffraction lights are retained to pass through. The polarization state of the four first-order diffraction lights is adjusted by the polarization optical element. After the beam is scaled by the confocal lens group, it enters the objective lens through the dichroic mirror. The four diffraction lights interfere at the focal plane of the objective lens to form a square lattice beam. A square lattice illumination beam irradiates the sample to produce fluorescence. The fluorescence is collected by the objective lens and then passes through a dichroic mirror, a lens, and a filter to form an image. Finally, the camera acquires the raw image. The programmable phase modulator includes a phase modulator, and the programmable phase modulator also integrates a square blazed grating hologram generation module, a digital lens hologram generation module, and a composite hologram generation module; The square blazed grating hologram generation model is used to generate square blazed grating holograms, and the phase distribution function of the square blazed grating hologram is described. for: (1) In formula (1): , This represents the total number of pixels in the horizontal direction for the programmable phase modulator. , This represents the total number of pixels in the vertical direction for the programmable phase modulator. ; This represents the maximum absolute value of coordinates m and n; The grating period; its value ranges from 4 to 16. This represents the remainder operation with respect to P; This represents the phase shift along the y-direction of the lattice optical field; The phase shift in the x-direction of the lattice light field is represented by the value [0, 2pi / 3, 4pi / 3]; i and j represent the phase shift steps in the two orthogonal directions x and y, respectively, i=1, 2, 3, j=1, 2, 3; The digital lens hologram generation module is used to generate digital lens holograms, the phase distribution function of which is... for: (2) In formula (2): The wavelength of the light emitted by the laser source; The focal length of the digital lens in the digital lens hologram generation model, in mm; The center-to-center distance between two adjacent pixels on the programmable phase modulator, in micrometers (µm). This indicates a remainder operation, and the result is then restricted to a certain range. Within the range; The composite hologram generation module is used to... and The sums are used to obtain the initial hologram, and then the phase of the initial hologram is constrained to... Within the range, a composite hologram is obtained; The phase modulator modulates the wavefront phase of the incident beam by loading a composite hologram onto the incident beam.

2. The square lattice structure light-illuminated super-resolution microscopy system according to claim 1, characterized in that, The programmable phase modulator is a silicon-based liquid crystal spatial light modulator.

3. The square lattice structure light-illuminated super-resolution microscopy system according to claim 1, characterized in that, Select the objective lens with the appropriate magnification based on the focal length of the digital lens and the focal length of the confocal lens group in the digital lens hologram generation module.

4. The square lattice structure light-illuminated super-resolution microscopy system according to claim 1, characterized in that, The polarizing optical element is selected from a quarter-wave plate, a half-wave plate, or a partitioned polarizer.

5. The square lattice structure light-illuminated super-resolution microscopy system according to claim 1, characterized in that, The value can be [0, 2pi / 3, 4pi / 3]; The value is [0, 2pi / 3, 4pi / 3].

6. The square lattice structure light-illuminated super-resolution microscopy system according to claim 1, characterized in that, The system also includes a displacement stage (15), on which a sample stage is provided. By controlling the movement of the displacement stage on the X and Y axes, the camera can acquire two-dimensional images of the region of interest of the sample on the sample stage. By moving the displacement stage on the Z axis, two-dimensional images of different layers of the sample can be acquired.

7. A super-resolution microscopy method for illumination of a square lattice structure, characterized in that, The method includes the following steps: Step 1: Acquire nine raw fluorescence images with different phase shifts and tetragonal lattice light fields using the system described in claim 1: (3) The image shows the original fluorescence images with phase shift steps i and j; i and j represent the phase shift steps in the two orthogonal directions x and y, respectively, i=1, 2, 3, j=1, 2, 3; x and y are the two-dimensional plane coordinates of the system. This represents the displacement in the x and y directions; This represents any coordinate point of the sample in a two-dimensional plane. fluorescence intensity at that location This represents the intensity distribution of the lattice light field relative to the coordinate points. Displacement occurs. Represents coordinate points The degree of blurriness of the sample; Step 2, calculate the original fluorescence image using equation (4). Weighted image : (4) In equation (4): (5) (6) In equations (5) and (6), These represent the wave vectors of the lattice light fields in the x and y directions, respectively; These represent the initial phases of the lattice light fields in the x and y directions, respectively; Indicates the phase shift step size; These represent the modulation intensities of the lattice optical field in the x and y directions, respectively; Step 3: Calculate the pre-filtered original image using equation (7). : (7) In equation (7), The complex conjugate of the system's optical transfer function; This is the inverse Fourier transform operator; k is the AND operator. Corresponding frequency domain coordinates; Original fluorescence image Fourier transform; (8) In equation (8): This is the attenuation amplitude parameter; These are adjustable empirical parameters; Step 4: Use equation (9) to process the nine pre-filtered original images. With corresponding weighted images Perform dot products on each of the nine dot products and then sum the results to obtain the image before deconvolution. : (9) Step 5: Apply a deconvolution algorithm to the image obtained in Step 4. After deblurring and other optimization processes, the final super-resolution image is obtained.

8. The method for super-resolution microscopy with light illumination of a square lattice structure according to claim 7, characterized in that, The value range is 0 to 1; The value range is 0.5 to 2.

5.

9. The method for super-resolution microscopy with light illumination of a square lattice structure according to claim 7, characterized in that, The deconvolution algorithm is Wiener deconvolution, RL deconvolution, or TV deconvolution.

10. A storage medium, characterized in that, It stores a computer program / instruction thereon, characterized in that when the computer program / instruction is executed by a processor, it implements the steps of the method described in any one of claims 7 to 9.

11. A software product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method described in any one of claims 7 to 9.

Images