Imaging apparatus and method for a structured light illumination microscope based on two-objective interferometry

Abstract

The application provides an imaging device and method of a structured light illumination microscope based on double objective lens interference. The laser coupling module of the device is used for emitting a first laser beam; the fast variable grating module is used for modulating the first laser beam into a spatial structured light beam; the polarization modulation module is used for compensating the non-equivalent phase delay between s-polarized light and p-polarized light of the spatial structured light beam; the six-beam alignment module is used for realizing the alignment and interference of the spatial structured light beam in the lateral and axial directions at the focal plane via two symmetrical microscopes; the optical path difference adjustment module is used for adjusting the optical path difference to be less than the coherence length of fluorescence; and the focus locking system is used for ensuring that the spatial structured light beam always keeps accurate alignment in long-term imaging. The embodiments of the application ensure that the illumination pattern of six-beam interference and the fluorescence interference keep stable for a long time, ensure the accurate alignment of the two objectives, accurately estimate the optical path difference information from two complementary images, and finally reconstruct an image with the least artifacts.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of optical super-resolution microscopy imaging, in particular to an imaging device and method of structured illumination microscopy based on double objective lens interference. BACKGROUND

[0002] Structured illumination microscopy (SIM) breaks the diffraction limit of optical microscopy by structured illumination in the frequency domain to load the high-frequency information of the object into the detection passband of the optical system in a spatial mixing manner. SIM has become the most widely used technology in the field of super-resolution optical microscopy of living cells due to its low excitation light intensity, non-specific requirement for fluorescent dyes, and fast wide-field imaging advantage. However, the axial resolution of the three-dimensional structured illumination system is lower than the lateral resolution, and the blurred axial details hinder the observation of fine subcellular structures in living cells. Therefore, it is necessary to provide an imaging device and method of structured illumination microscopy based on double objective lens interference to solve the above problems. SUMMARY

[0003] The present application is proposed in view of at least one of the above technical problems in the prior art. According to an aspect of the present application, an imaging device of structured illumination microscopy based on double objective lens interference is provided, which comprises a structured illumination system, an optical path difference adjusting module and a focus locking system; wherein,

[0004] The structured illumination system comprises a laser coupling module, a fast variable grating module, a polarization modulation module and a six-beam light alignment module; the laser coupling module is used for emitting a first laser beam; the fast variable grating module is arranged on the exit light path of the laser coupling module and is used for receiving the first laser beam and emitting the first laser beam to the polarization modulation module after modulating the first laser beam into a spatial structured light beam; the polarization modulation module is used for compensating the unequal phase delay between s-polarized light and p-polarized light of the spatial structured light beam; the six-beam light alignment module divides the processed spatial structured light beam into two symmetric spatial structured light beams, which are converged on a focal plane via two symmetric microscopic objectives, interference occurs at the focal plane, and a three-dimensional structured illumination pattern is formed; the fluorescence signal generated by the illumination pattern exciting the sample is collected by the microscopic objectives and the collection, and after convergence, fluorescence interference occurs, and finally the image is collected by a second camera, and fluorescence interference occurs;

[0005] The optical path difference adjusting module is arranged on the light path of the six-beam light alignment module, and the optical path difference adjusting module is used for adjusting the optical path difference of the imaging device to be less than the coherence length of the fluorescence;

[0006] The focus locking system comprises a second laser emitter; the second laser emitter is used to emit a second laser beam; the second laser beam is focused to the first camera through a lens group, a second dichroic mirror, a third dichroic mirror and a fourth dichroic mirror to monitor the position and shape of the processed spatial structured light beam on the first camera, so that the system control module adjusts the relative position of the microscope objective according to the position and the shape to ensure that the sample is always on the focal point.

[0007] In some embodiments, the optical path difference adjustment module comprises an optical path difference coarse adjustment module and an optical path difference fine adjustment module; wherein,

[0008] The optical path difference coarse adjustment module comprises a long-stroke linear translation stage and a piezoelectric linear actuator, and is used to adjust the optical path difference to a first preset range during initial calibration of the imaging device;

[0009] The optical path difference fine adjustment module comprises a linear piezoelectric platform, and is used to adjust the optical path difference to a second preset range;

[0010] The first preset range is a micrometer order fluorescence interference range, and the second preset range is a position where the optical path difference is zero in nanometer order.

[0011] In some embodiments, the laser coupling module comprises a first laser emitter, an acousto-optic tunable filter, a mechanical oscillator, a multimode optical fiber, a collimating lens and a lens group; the first laser emitter is used to generate the first laser beam; the first laser beam is coupled into a multimode optical fiber after being filtered by the acousto-optic tunable filter, and the multimode optical fiber is wound on the mechanical oscillator, wherein the mechanical oscillator is used to fill the speckle pattern of the multimode optical fiber to obtain uniform partially coherent light illumination; the first laser beam via the multimode optical fiber reaches the collimating lens for collimation, and the first laser beam collimated via the collimating lens is emitted to the fast variable grating module through the lens group.

[0012] In some embodiments, the fast variable grating module comprises a ferroelectric liquid crystal fast spatial light modulator, a polarization beam splitter prism and a first half-wave plate; the polarization beam splitter prism receives the first laser beam and shoots the first laser beam into the ferroelectric liquid crystal fast spatial light modulator via the first half-wave plate, wherein the ferroelectric liquid crystal fast spatial light modulator is used to modulate the first laser beam into the spatial structured light beam; the spatial structured light beam enters the polarization modulation module after being split by the polarization beam splitter prism.

[0013] In some embodiments, the polarization modulation module comprises a spatial filter, a second half-wave plate, a liquid crystal phase retarder, two quarter-wave plates and a depolarization beam splitter prism.

[0014] The spatial structure light beam passes through the spatial filter and enters the second half-wave plate, wherein the second half-wave plate is used to rotate the spatial structure light by 45° relative to s polarization; the spatial structure light emitted by the second half-wave plate enters the liquid crystal phase retarder, wherein the fast axis of the liquid crystal phase retarder is aligned with s polarization of the microscope objective; the liquid crystal phase retarder adjusts the phase of the spatial structure light beam;

[0015] The spatial structure light beam emitted via the liquid crystal phase retarder is reflected by the depolarization beam splitter prism and is divided into two spatial structure light beams with consistent phase retardation, and the two spatial structure light beams form a symmetric optical interference cavity;

[0016] The depolarization beam splitter prism is used to keep the polarization states of reflected light and transmitted light the same;

[0017] The two quarter-wave plates are arranged before the microscope objective and after all the mirrors respectively, and the fast axes of the two quarter-wave plates are arranged at 45° relative to s polarization.

[0018] In some embodiments, the six-beam alignment module includes the microscope objective, and the microscope objective includes an upper microscope objective and a lower microscope objective, wherein the lower microscope objective is mounted on an XY piezoelectric translation stage, and the upper microscope objective is mounted on a Z-axis piezoelectric displacement stage; when an interference image is acquired, fluorescent beads in a sample are taken as fiducials to align the lateral direction of the upper microscope objective and the axial direction of the lower microscope objective.

[0019] In some embodiments, the device further includes a wide-field illumination module;

[0020] The wide-field illumination module includes a third laser emitter; the third laser emitter is used to emit a third laser beam; the third laser beam is aligned with the first laser beam in the optical path and is focused on the focal plane; a dichroic mirror is arranged between the third laser beam and the first laser beam to provide one-sided wide-field illumination for the focal plane.

[0021] In some embodiments, the device further includes a dual-color detection module; wherein,

[0022] The double-color detection module comprises a fifth dichroic mirror; the fifth dichroic mirror is arranged on a light path emitted by the depolarization beam splitter, wherein the light path comprises an upper light path passing through a lens, a mirror and another lens; the fifth dichroic mirror separates the upper light path into different spectral fluorescent detection light paths; the double-color detection module further comprises a filter set and a second camera, different color filters are arranged on the two fluorescent detection light paths to filter first color fluorescent light and second color fluorescent light; the second camera collects the first color fluorescent light through a first channel, and the second camera collects the second color fluorescent light through a second channel.

[0023] In some embodiments, the device further comprises a sample stage module; wherein,

[0024] The sample stage module comprises a sample stage and a three-dimensional displacement stage for driving the sample stage, wherein the sample stage is used to carry a sample sealed in glue between two high-precision wave plates; the three-dimensional displacement stage comprises an XY piezoelectric translation stage and a Z-axis piezoelectric displacement stage.

[0025] In some embodiments, the device further comprises a system control module, and the system control module comprises a controller; wherein,

[0026] The controller is configured to adjust the voltage of the liquid crystal phase retarder at a preset time interval, generate a camera exposure signal and an adjustable optical filter illumination modulation signal to collect an interference image of the sample at a current focal plane, and move the Z-axis piezoelectric displacement stage to different depths of the sample to the focal plane at a preset step, collect interference images according to the above steps, until the entire sample is imaged.

[0027] The controller is further configured to trigger the structured light illumination module and the first camera.

[0028] In some embodiments, the system control module further comprises an image reconstruction module; wherein,

[0029] The image reconstruction module is configured to reconstruct the original image based on optical transfer function information to obtain a reconstructed super-resolution three-dimensional image by using a Wiener filter reconstruction algorithm according to structured light parameters and an optical path difference.

[0030] In some embodiments, the image reconstruction module comprises an original data preprocessing module, a structured light parameter estimation module, an optical path difference estimation module and an imaging module; wherein,

[0031] The original data preprocessing module is configured to construct the optical transfer function module and the imaging module to perform background removal and edge softening, Fourier transform on the original data; and the optical path difference estimation module is configured to perform registration operation on the first original image and the second original image to eliminate fluorescent interference and obtain spectral information of six-beam light interference.

[0032] The optical transfer function module is configured to obtain different orders of spectrum according to the double objective lens interference structured light illumination point spread function and the image interference point spread function, perform inverse convolution operation on the different orders of spectrum and the interference optical transfer function to obtain each order of illumination spectrum, perform phase modulation on the interference optical transfer function and the each order of illumination spectrum respectively, and then perform convolution operation on the modulated interference optical transfer function and the each order of illumination spectrum to generate an optical transfer function library corresponding to a plurality of optical path differences.

[0033] The structured light parameter estimation module is configured to calculate wave vector, modulation depth and initial phase based on the spectral information of six-beam light interference by using a preset algorithm.

[0034] The optical path difference estimation module is configured to determine the optical path difference according to the first original image and the second original image.

[0035] The imaging module is configured to construct a Wiener filter based on system optical transfer function, structured light parameters and optical path difference information by using a Wiener filter reconstruction algorithm to obtain a reconstructed image.

[0036] In some embodiments, a plurality of lasers of different wavelengths are used for six-beam structured light illumination imaging; one additional laser is used for image interference microscopy imaging; another additional laser is used for monitoring and correcting the drift of the objective lens; and the sample includes fluorescent beads, fixed cells or living cells.

[0037] Another aspect of the embodiments of the present application provides an imaging method of a structured light illumination microscope based on double objective lens interference, which is applied to the imaging device of the structured light illumination microscope based on double objective lens interference as described above, and the method comprises:

[0038] Acquiring a first original image and a second original image;

[0039] Preprocessing the first original image and the second original image to obtain spectral information of six-beam light interference;

[0040] Constructing a phase separation matrix according to the first original image which has completed the background removal and edge softening operation;

[0041] Obtaining a phase-separated frequency band according to the phase separation matrix;

[0042] estimating parameters from the spectrum information of the six-beam light interference;

[0043] estimating parameters from the spectrum information of the six-beam light interference, and generating a Wiener filter according to the matched optical transfer function of the dual-microscope interference structured light illumination microscope;

[0044] moving the Wiener filter;

[0045] multiplying each frequency band with the corresponding Wiener filter according to the phase-separated frequency bands and the Wiener filter;

[0046] moving and recombining the filtered spectrum;

[0047] reconstructing an image according to the recombined filtered spectrum.

[0048] In some embodiments, the method further comprises:

[0049] collecting a dual-microscope interference structured light illumination point spread function and an image interference point spread function;

[0050] generating an interference optical transfer function according to the interference point spread function;

[0051] spectrum separating the dual-microscope interference structured light illumination point spread function to obtain different order spectrums; and deconvolving the dual-microscope interference structured light illumination point spread function and the interference point spread function;

[0052] respectively phase modulating the interference optical transfer function and the illumination spectrums of each order, and then convolving the modulated interference optical transfer function and the illumination spectrums of each order to generate an optical transfer function corresponding to a specific optical path difference;

[0053] convolving the spectrum signals of different orders with the interference optical transfer function to obtain the spectrums of each order;

[0054] selecting a corresponding optical transfer function in the optical transfer function library according to the estimated optical path difference in the image reconstruction process, to be used for generating the Wiener filter.

[0055] In some embodiments, the first original image and the second original image are preprocessed, including:

[0056] mirroring and registering the second original image to obtain a third original image;

[0057] adding the first original image and the third original image after registration, and then performing Fourier transform to obtain the spectrum information of the six-beam light interference.

[0058] In another aspect, the application provides a storage medium having a computer program stored thereon, which, when executed by a processor, causes the processor to perform the imaging method of the structured light illumination microscope based on double-objective interference as described above.

[0059] The imaging device of the structured light illumination microscope based on double-objective interference according to the application can achieve the same interference contrast with half of the excitation power, because it uses two objectives for interference illumination and detection. In addition, the long-term stability of the illumination pattern of six beams of light is ensured by the hardware design, and the precise alignment of the two objectives is ensured by the focus locking module. The reconstruction algorithm can accurately estimate the optical path difference information from two complementary images and incorporate it into the reconstruction process, and finally generate an image with minimal artifacts, so that the three-dimensional isotropic super-resolution image of fine subcellular structures in living cells can be achieved. BRIEF DESCRIPTION OF DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0061] Figure 1 FIG. 1 shows a schematic diagram of the imaging device of the structured light illumination microscope based on double-objective interference according to the application;

[0062] Figure 2 FIG. 2 shows a schematic diagram of the structured light illumination optical path according to the application;

[0063] FIG. 3(a) shows a schematic diagram of the direction of the outgoing polarized light rotated by adjusting the liquid crystal retardation in the SIM system of the prior art;

[0064] FIG. 3(b) shows a schematic diagram of the polarization modulation module according to the application adjusting the polarization of the upper and lower illumination light to the required polarization state;

[0065] FIG. 4(a) shows a schematic diagram of the optical interference cavity according to the application;

[0066] FIG. 4(b) shows a record of the fluorescence intensity fluctuation of the optical path difference coarse adjustment according to the application;

[0067] FIG. 4(c) shows a record of the fluorescence intensity fluctuation of the optical path difference fine adjustment according to the application;

[0068] FIG. 5(a) shows a schematic diagram of the optical path of the focus locking module according to the application;

[0069] Figure 5(b) shows a schematic diagram of focus locking with a mobile objective lens according to an embodiment of the present application;

[0070] Figure 6(a) shows a schematic diagram of dichroic mirror DM5 separating two color signals, and filters F2 and F3 for further filtering fluorescent signals according to an embodiment of the present application;

[0071] Figure 6(b) shows a schematic diagram of excitation and emission spectra of fluorescent proteins (mStayGold and dCyOFP2s) for two-color imaging according to an embodiment of the present application;

[0072] Figure 7 Figure 7 shows a schematic diagram of a sample stage module according to an embodiment of the present application;

[0073] Figure 8 Figure 8 shows a schematic diagram of signal timing according to an embodiment of the present application;

[0074] Figures 9(a) to 9(j) Figure 9 shows a schematic diagram of the principle of optical path length difference (OPLD) estimation for 4Pi-SIM according to an embodiment of the present application;

[0075] Figure 10(a) shows a schematic diagram of the projection of fluorescent beads in the xy section according to an embodiment of the present application;

[0076] Figure 10(b) shows a schematic diagram of the reconstruction result with OPLD = 0 nm and the reconstruction result with OPLD = -160 nm according to an embodiment of the present application;

[0077] Figure 10(c) shows a schematic diagram of the comparison of intensity curves of a single fluorescent bead according to an embodiment of the present application;

[0078] Figure 10(d) shows a schematic diagram of the projection of fixed cells labeled with endoplasmic reticulum in the xy section according to an embodiment of the present application;

[0079] Figure 10(e) shows a schematic diagram of the reconstruction result with OPLD = 0 nm and the reconstruction result with OPLD = -180 nm according to an embodiment of the present application;

[0080] Figure 10(f) shows a schematic diagram of the comparison of intensity curves of a single layer of endoplasmic reticulum signals according to an embodiment of the present application;

[0081] Figure 11 Figure 11 shows a schematic flow chart of an imaging method of a structured light illumination microscope based on double objective lens interference according to an embodiment of the present application;

[0082] Figure 12 Figure 12 shows a schematic flow chart of generating an optical transfer function library of an imaging method of a structured light illumination microscope based on double objective lens interference according to an embodiment of the present application;

[0083] Figure 13 Fig. 6 shows a schematic flow chart of the pre-processing of the first and second raw images of the imaging method of the structured light illumination microscope based on dual objective lens interference according to an embodiment of the present application;

[0084] Figure 14 Fig. 7 shows a schematic flow chart of the imaging method of the structured light illumination microscope based on dual objective lens interference according to another embodiment of the present application. DETAILED DESCRIPTION

[0085] In order to make the skilled in the art better understand the technical solutions of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the skilled in the art without creative labor are within the scope of protection of the present application.

[0086] Based on the foregoing at least one technical problem, the application provides an imaging method of a structured light illumination microscope based on double-objective lens interference, the device comprising: a structured light illumination system, an optical path difference adjusting module and a focus locking system; wherein the structured light illumination system comprises a laser coupling module, a fast variable grating module, a polarization modulation module and a six-beam light alignment module; the laser coupling module is used for emitting a first laser beam; the fast variable grating module is arranged on an exit light path of the laser coupling module, used for receiving the first laser beam, and emitting the first laser beam to the polarization modulation module after modulating the first laser beam into a spatial structured light beam; the polarization modulation module is used for compensating for the unequal phase delay between s-polarized light and p-polarized light of the spatial structured light beam; the six-beam light alignment module divides the processed spatial structured light beam into two symmetric spatial structured light beams, converges the two symmetric spatial structured light beams on a focal plane via two symmetric microscopes, and forms a three-dimensional structured light illumination pattern at the focal plane through interference; the illumination pattern excites a fluorescence signal generated by a sample, and the fluorescence signal is collected by the microscopes and converges to generate fluorescence interference, and finally an image is collected by a second camera to generate fluorescence interference; the optical path difference adjusting module is arranged on a light path between the polarization modulation module and the six-beam light alignment module, and is used for adjusting the optical path difference of the imaging device to be less than the coherence length of fluorescence; the focus locking system comprises a second laser emitter; the second laser emitter is used for emitting a second laser beam; the second laser beam focuses on the first camera via a lens group and a dichroic mirror group to monitor the position and shape of the processed spatial structured light beam on the first camera, so that a system control module adjusts the relative position of the microscopes according to the position and the shape to ensure that the sample is always on the focal point. In addition, the embodiment of the application ensures that the illumination pattern of the six-beam light interference is kept stable for a long time in the design of hardware, and the focus locking module ensures the accurate alignment of the two microscopes; in the software design, the reconstruction algorithm can accurately estimate the optical path difference information from two complementary images and cover it into the reconstruction process, and finally generate an image with the smallest artifact.

[0087] Figure 1 A schematic diagram of an imaging device of a structured light illumination microscope based on double-objective lens interference according to an embodiment of the application is shown; as Figure 1 shown, the imaging device 100 of the structured light illumination microscope based on double-objective lens interference according to the embodiment of the application can comprise a structured light illumination system, an optical path difference adjusting module and a focus locking system.

[0088] The structure light illumination system comprises a laser coupling module, a fast variable grating module, a polarization modulation module and a six-beam light alignment module. The laser coupling module is used for emitting a first laser beam. The fast variable grating module is arranged on an exit light path of the laser coupling module, used for receiving the first laser beam, modulating the first laser beam into a spatial structure light beam and then emitting the spatial structure light beam to the polarization modulation module. The polarization modulation module is used for compensating for non-equivalent phase delay between s-polarized light and p-polarized light of the spatial structure light beam. The six-beam light alignment module divides the processed spatial structure light beam into two symmetric spatial structure light beams, converges the two symmetric spatial structure light beams on a focal plane through two symmetric microscopic objectives, and forms a three-dimensional structure light illumination pattern at the focal plane through interference. The illumination pattern excites a fluorescence signal generated by a sample, and the fluorescence signal is collected by the microscopic objectives and converges to generate fluorescence interference. Finally, an image is collected by a second camera, and fluorescence interference occurs. The sample comprises fluorescent beads, fixed cells or living cells.

[0089] The structure and function of each module of the structure light illumination system will be introduced below.

[0090] In the first embodiment, as shown in Figure 1 and Figure 2 , the laser coupling module comprises a first laser emitter, an acousto-optic tunable filter AOTF1, a mechanical oscillator Shaker, a multi-mode optical fiber MMF, a collimating lens CL and a lens group. The lens group comprises a first lens L1 and a second lens L2.

[0091] The first laser emitter is used for generating the first laser beam (Laser1), and a 488nm laser can be used as a light source. After the first laser beam (Laser1) is filtered by the acousto-optic tunable filter AOTF1, the first laser beam (Laser1) is emitted to the mechanical oscillator Shaker. The mechanical oscillator is used for filling the speckle pattern of the multi-mode optical fiber MMF to obtain uniform partially coherent light illumination. The first laser beam (Laser1) emitted by the multi-mode optical fiber MMF reaches the collimating lens CL for collimation. The collimated first laser beam (Laser1) is emitted to the fast variable grating module through the lens group L1 and L2.

[0092] Specifically, the optical power is modulated by the acousto-optic tunable filter AOTF1. The laser beam is coupled to the multi-mode optical fiber through a five-axis lens. The mechanical oscillator Shaker is fixed on the multi-mode optical fiber MMF wound into a bundle, and the speckle pattern of the multi-mode optical fiber MMF is filled by mechanical vibration to obtain uniform partially coherent light illumination. The divergent beam at the output port of the multi-mode optical fiber MMF is collimated by the collimating lens CL, and then passes through a pair of relay lenses L1 and L2 to ensure that the fiber end face is projected onto the back focal plane of the objective lens.

[0093] In the second embodiment, continuing to combine Figure 1 The fast variable grating module comprises a ferroelectric liquid crystal fast spatial light modulator SLM, a polarization beam splitter prism PBS, and a first half wave plate HWP1. The polarization beam splitter prism PBS receives the first laser beam and injects the first laser beam (Laser1) into the ferroelectric liquid crystal fast spatial light modulator SLM via the first half wave plate HWP1. The ferroelectric liquid crystal fast spatial light modulator SLM is configured to modulate the first laser beam (Laser1) into the spatial structured light beam. The spatial structured light beam enters the polarization modulation module after being split by the polarization beam splitter prism PBS.

[0094] Specifically, the variable grating module is composed of a ferroelectric liquid crystal fast spatial light modulator, a polarization beam splitter prism, and a first half wave plate HWP1. By loading different diffraction patterns on the SLM, the variable grating module can generate a diffraction beam within milliseconds (ms), and then form the required structured light fringe on the sample surface. In the structured light illumination system (4Pi-SIM) of double objective interference, it is necessary to generate three-direction, five-phase binary grating patterns and arrange them in a certain time sequence.

[0095] In addition, the embodiments of the present application also provide a spatial mask as a spatial filter to filter out the additional diffraction orders generated by the binary grating.

[0096] In the third embodiment, combining Figure 1 , FIG. 3(a) and FIG. 3(b), the polarization modulation module comprises a spatial filter Mask, a second half wave plate HWP2, a liquid crystal phase retarder HS-LCVR, two quarter wave plates QWP1 and QWP2, a depolarization beam splitter prism NPBS, a lens group L3, L4, L5, L6, L7, L8, L9, and a mirror group M1, M2, M3, M4, M5, M6, M7, M8, M9, M14, M15, and a first dichroic mirror DM1.

[0097] The spatial structure light beam passes through the spatial filter Mask and enters the second half-wave plate HWP2, wherein the second half-wave plate HWP2 is used to rotate the spatial structure light by 45° relative to s polarization; the spatial structure light emitted by the second half-wave plate HWP2 enters the liquid crystal phase retarder HS-LCVR, wherein the fast axis of the liquid crystal phase retarder HS-LCVR is aligned with the s polarization of the objective lens OBJ1 and OBJ2; the liquid crystal phase retarder HS-LCVR adjusts the phase of the spatial structure light beam. The spatial structure light beam emitted by the liquid crystal phase retarder HS-LCVR is transmitted and reflected by the depolarization beam splitter prism NPBS, and is divided into two spatial structure light beams with consistent phase retardation, which form a symmetric optical interference cavity.

[0098] The two quarter-wave plates QWP1 and QWP2 are respectively arranged before the objective lens OBJ1 and OBJ2 and after all the mirrors, and the fast axes of the two quarter-wave plates QWP1 and QWP2 are arranged at 45° relative to s polarization.

[0099] Specifically, in order to maximize the contrast of the illumination mode in the SIM, thereby optimizing the spectral expansion capability, it is necessary to set a polarization modulation module. The polarization modulation module can ensure that the polarization state of the six illumination lights is tangent to the back pupil surface of the objective lens on the back pupil surface, thereby achieving the highest interference contrast.

[0100] In the SIM system of the conventional technology, as shown in FIG. 3(a), a polarization direction rotator composed of a liquid crystal phase retarder HS-LCVR and quarter-wave plates QWP1 and QWP2 is usually used, and the direction of the outgoing polarized light can be rotated by adjusting the liquid crystal retardation. Such an optical design is not applicable in the 4Pi-SIM system, because many reflective optical elements are inevitably placed between the polarization modulation module and the sample. The reflection of these elements can introduce unequal phase retardation between s polarization and p polarization, resulting in elliptical polarized illumination light in the sample plane, thereby reducing the modulation contrast.

[0101] To solve this problem, the polarization modulation module of the embodiments of the present application can compensate for the non-equivalent phase delay between s-polarization and p-polarization. As shown in FIG. 3(b), the system uses a quarter-wave plate HWP2 to rotate the polarization to 45° relative to s-polarization before the liquid crystal phase retarder HS-LCVR, and the fast axis of the liquid crystal phase retarder HS-LCVR is set to 0°. Two quarter-wave plates QWP1 and QWP2 are placed before the objective lens and after all the mirrors at an angle of 45° to the s-polarization of the incident light. On the one hand, a symmetric design of the optical interference cavity is adopted to ensure that the phase delay experienced by the upper and lower light is consistent; on the other hand, a custom depolarization beam splitter prism NPBS is used to keep the polarization states of the reflected light and the transmitted light the same. Therefore, by adjusting the voltage of the liquid crystal phase retarder HS-LCVR, the polarization of the upper and lower illumination light can be adjusted to the desired polarization state.

[0102] In a fourth embodiment, as shown in FIG. 4, the six-beam light alignment module includes the microscope objective, which includes an upper microscope objective and a lower microscope objective, wherein the lower microscope objective is mounted on an XY piezoelectric translation stage, and the upper microscope objective is mounted on a Z-axis piezoelectric displacement stage; when acquiring an interference image, the fluorescent beads in the sample are used as fiducials to align the lateral direction of the upper microscope objective and the axial direction of the lower microscope objective. Figure 1

[0103] Specifically, in order to ensure that the six-beam light interference adjustment has sufficient degrees of freedom, the upper microscope objective is mounted on a one-dimensional linear piezoelectric translation stage, having a Z-direction degree of freedom; and the lower microscope objective is mounted on a two-dimensional piezoelectric translation stage, having XY-direction degrees of freedom.

[0104] In one example, multiple lasers of different wavelengths can be used for six-beam structured light illumination imaging; an additional laser is used for image interference microscopic imaging; and another additional laser is used for monitoring and correcting the drift of the objective lens.

[0105] The optical path difference adjustment module is arranged in the optical path of the six-beam light alignment module, and is used to adjust the optical path difference of the imaging device to be less than the coherence length of the fluorescent light.

[0106] ​In an embodiment of the present application, the optical path difference adjustment module comprises an optical path difference coarse adjustment module (CTS) and an optical path difference fine adjustment module (FTS), and mirrors M10, M11, M12 and M13. The optical path difference coarse adjustment module comprises a long-stroke linear translation stage and a piezoelectric linear actuator, and is used to adjust the optical path difference to a first preset range during initial calibration of the imaging device. The optical path difference fine adjustment module comprises a linear piezoelectric platform, and is used to adjust the optical path difference to a second preset range. The first preset range is a fluorescence interference range (micrometer level), and the second preset range is a position where the optical path difference is zero (nanometer level).

[0107] Specifically, in combination with Figure 1 and Figures 4(a) to 4(c) , the axial fringe contrast is particularly important for the performance of 4Pi-SIM. In order to improve the axial fringe contrast as much as possible, it is necessary to accurately adjust the optical path difference (Optical Path Length Difference, OPLD) between the upper and lower arms of the optical interference cavity to 0.

[0108] To solve this problem, an optical path difference (Optical Path Length Difference, OPLD) adjustment module is provided in an embodiment of the present application, which is composed of an optical path difference fine adjustment module FTS and an optical path difference coarse adjustment module CTS located in the upper and lower arms respectively. The two modules have similar designs in hardware: a compact optical delay line composed of fixed knife-edge right-angle silver mirrors PM1 and PM2 and a pair of silver mirrors installed on a translation stage. The difference between the optical path difference fine adjustment module FTS and the optical path difference coarse adjustment module CTS is that the optical path difference coarse adjustment module CTS is composed of a long-stroke linear translation stage and a piezoelectric linear actuator. The optical path difference coarse adjustment module CTS is used to coarsely adjust the optical path difference (Optical Path Length Difference, OPLD) to within 1 μm.

[0109] The optical path difference fine adjustment module FTS adopts a linear piezoelectric platform, which has a sub-nanometer resolution. The optical path difference fine adjustment module FTS can accurately adjust the optical path difference (Optical Path Length Difference, OPLD) to 0.

[0110] As shown in FIG. 4(a), it is a schematic diagram of the optical interference cavity of the embodiment of the present application. As shown in FIG. 4(b), it is a fluorescence intensity fluctuation record of the optical path difference coarse adjustment of the embodiment of the present application. As shown in FIG. 4(c), it is a fluorescence intensity fluctuation record of the optical path difference fine adjustment of the embodiment of the present application.

[0111] The focus locking system comprises a second laser emitter for emitting a second laser beam (Laser2). The second laser beam (Laser2) is focused on the first camera after passing through a lens group, a second dichroic mirror, a third dichroic mirror and a fourth dichroic mirror, so as to monitor the position and shape of the processed spatial structured light beam on the first camera. The system control module adjusts the relative position of the microscope objective according to the position and shape, so as to ensure that the sample is always on the focal point.

[0112] Specifically, the second laser emitter can generate near-infrared 785nm laser as a focus locking light source, as shown in FIGS. 5(a) and (b). Figure 1 The light beam is emitted from a single-mode optical fiber SMF2, collimated by a lens L18, reflected by a dichroic mirror DM2, and then reflected by another dichroic mirror DM3 after passing through two microscope objectives OBJ1 and OBJ2. The light beam is focused on the camera CAM1 by a lens L17, a dichroic mirror DM4 and a cylindrical lens CY1. By monitoring the position and shape of the laser beam on the first camera CAM1, the system can track and adjust the relative position between the two microscope objectives OBJ1 and OBJ2 in real time to keep the sample always on the focal point.

[0113] As shown in FIG. 5(b), if the objective lens is shifted in the lateral direction, the position of the light spot will change. If the objective lens is shifted up and down in the axial direction, the shape of the light spot will change due to the introduction of astigmatism. This variable quantitatively reflects the numerical value of the objective lens shift. By feeding back the distance value to the displacement stage module of the upper and lower microscope objectives OBJ1 and OBJ2, real-time focus locking can be achieved.

[0114] The embodiments of the present application ensure that the sample maintains the best focal position during a long imaging process, so as to obtain a high-quality super-resolution image through initial alignment, laser beam adjustment, focus monitoring and real-time adjustment.

[0115] In an embodiment of the present application, the imaging device further comprises a wide-field illumination module. The wide-field illumination module comprises a third laser emitter for emitting a third laser beam. The third laser beam is aligned with the first laser beam in the optical path and focused on the focal plane. A dichroic mirror is arranged between the third laser beam (Laser3) and the first laser beam (Laser1) to provide single-sided wide-field illumination for the focal plane.

[0116] The third laser emitter of the embodiment of the present application can adopt a 473 nm laser emitter. The 473 nm laser emitter emits a 473 nm laser beam, which is modulated by an acousto-optic tunable filter AOTF2, and then passes through a single-mode optical fiber SMF1 and a lens L16, and is introduced by reflection of a dichroic mirror DM2. The center light path of the 473 nm laser beam is aligned with that of the 488 nm laser beam, and the two beams are separated by the dichroic mirror to provide unilateral wide-field WF illumination. The resulting image interferometry (I 2 M) contains only fluorescent interference, which can be used to locate the target region of the sample in a large field of view and collect the optical transfer function (Optical Transfer Function, OTF) of the imaging device.

[0117] In an embodiment of the present application, the imaging device 100 further comprises a dual-color detection module. Figure 1 The dual-color detection module comprises a fifth dichroic mirror DM5, a lens group L10, L11, L12, L13, L14, L15, a blade right-angle silver mirror PM3, a mirror group M16, M17, M18, M19, a filter F1, and a second camera CAM2. The fifth dichroic mirror DM5 is arranged on the light path emitted by the depolarization beam splitter prism NPBS. The light path emitted by the depolarization beam splitter prism NPBS includes an upper light path passing through the lens L10, the mirror M16, and the lens L12, and a lower light path passing through the dichroic mirror DM1, the lens L11, the mirror M17, and the lens L13. The fifth dichroic mirror DM5 separates the upper light path into different spectral fluorescent detection light paths. The dual-color detection module further comprises a filter group and a second camera CAM2, and different colored filters F2 and F3 are arranged on the two fluorescent detection light paths to filter first color fluorescent light (e.g., green light) and second color fluorescent light (e.g., red light). The second camera CAM2 collects the first color fluorescent light through a first channel, and the second camera collects the second color fluorescent light through a second channel.

[0118] Specifically, in combination with Figure 1 and Figures 6(a) to 6(b) Figure 6(a) shows a schematic diagram of the fifth dichroic mirror DM5 separating two color signals, and filters F2 and F3 used to further filter fluorescent signals. Figure 6(b) shows a schematic diagram of the excitation spectrum and emission spectrum of fluorescent proteins (mStayGold and dCyOFP2s) for dual-color imaging.

[0119] The embodiments of the present application provide two ways of dual-color imaging: first, a single-wavelength laser simultaneously excites a conventional fluorescent protein (e.g., oxStaygold) and a special protein with a large Stokes shift (e.g., dCyOFP2s) to simultaneously collect signals of two channels. Second, a dual-wavelength laser sequentially excites a green fluorescent protein and a red fluorescent protein to sequentially collect fluorescent signals of two channels. The first way can simultaneously capture fluorescent signals of two colors in the same exposure time, and can simultaneously optimize axial illumination contrast of two detection channels.

[0120] On the imaging light path, a customized fifth dichroic mirror DM5 is introduced after the lens L12 and the lens L13 to separate fluorescent light of different spectra. The parameters of the fifth dichroic mirror DM5 include a flatness of not less than λ / 8, and a double-side anti-reflection coating to reduce multiple reflections of unconventional incident surfaces. The upper light path of the detection light path is installed with a green filter F2 to collect green fluorescent light; and the lower light path of the detection light path is installed with a red filter F3 to collect red fluorescent light.

[0121] In an embodiment of the present application, the imaging device further comprises a sample stage module. The sample stage module is used to carry a sample sealed by glue between two high-precision wave plates; and the three-dimensional displacement stage comprises an XY piezoelectric translation stage and a Z-axis piezoelectric displacement stage.

[0122] Specifically, in combination with Figure 1 and Figure 7 , the three-dimensional displacement stage is composed of a two-dimensional XY piezoelectric translation stage and a one-dimensional Z piezoelectric translation stage, so as to facilitate positioning and movement in the X, Y and Z axial directions. A sample groove for placing a sample is arranged on the three-dimensional displacement stage. The sample groove adopts a “sandwich” shape design, and the sample is fixed in a customized circular sample groove by two high-precision glass plates sealed by glue.

[0123] In an embodiment of the present application, the imaging device further comprises a system control module, and the system control module comprises a controller and an imaging module.

[0124] The controller is configured to adjust the voltage of the liquid crystal phase retarder according to a preset time interval, generate a camera exposure signal and an adjustable optical filter illumination modulation signal to collect an interference image of a current focal plane, collect an interference image of the sample at the current focal plane, and move the Z-axis piezoelectric displacement stage to different depths of the sample to the focal plane according to a preset step, and collect the interference image according to the above steps until the entire sample is imaged.

[0125] As Figure 8As shown, the system utilizes the Transistor-Transistor Logic (TTL) output signal of the ferroelectric liquid crystal spatial light modulator SLM's Light Emitting Diode Enable (LED_EN) transistor as the main trigger: the ferroelectric liquid crystal spatial light modulator SLM is programmed to set a series of binary pixel patterns and the display time of each pattern, thus its on-board microcontroller can ensure the switching and timing of pattern display and send out the Transistor-Transistor Logic (TTL) signal of Light Emitting Diode Enable (LED_EN) to trigger the camera exposure and Acousto-Optic Tunable Filter (AOTF1) modulation illumination. After detecting 5 falling edges of the Transistor-Transistor Logic (TTL) output of the ferroelectric liquid crystal spatial light modulator SLM, the input voltage of the liquid crystal phase delay device HS-LCVR starts to change, which indicates that the collection of one illumination direction has been completed. When the falling edge count reaches 15, it marks the completion of the collection on the focal plane, and the sample Z displacement stage will move to the next focal plane by a pre-set step size. Once all the focal planes are imaged, a reset signal will be sent to the Z displacement stage to return it to the initial position.

[0126] For three-dimensional time-lapse imaging, the embodiment of the present application sets a custom time interval between the collection of each three-dimensional image. During this time interval, the camera, laser, liquid crystal phase delay device HS-LCVR and sample stage are not triggered. For two-color imaging, since only one illumination wavelength is used and both color channels are detected simultaneously, the collection procedure is the same as that of monochrome imaging, which will not be described here.

[0127] The image reconstruction module is configured to reconstruct the original image based on the optical path difference containing the optical transfer function information to obtain a reconstructed super-resolution three-dimensional image according to the structured light parameter and using a Wiener filter reconstruction algorithm.

[0128] The image reconstruction module includes a structured light parameter estimation module, an optical path difference estimation module, an imaging module, a construction optical transfer function module and an original data preprocessing module.

[0129] The original data preprocessing module is configured to perform preprocessing for the construction optical transfer function module and the imaging module, including removing background and edge softening, Fourier transform according to the original data; and is configured to perform preprocessing for the optical path difference estimation module, including registration operation of the first original image and the second original image to eliminate fluorescence interference and obtain spectral information of six-beam light interference.

[0130] In order to perform parameter estimation and reconstruction, the original data needs to be pre-processed. First, the target area (A and B) of the interference complementary two sets of images is intercepted from the original data, and one of the images is mirror flipped and registered (B'). The two 4Pi-SIM images are de-backgrounded and edge softened, and then Fourier transformed to obtain their respective spectral information; the two 4Pi-SIM images are added after registration, and then Fourier transformed, which can eliminate fluorescence interference, thereby obtaining the spectral information of six-beam light interference. The above spectrum is separated by using a separation matrix constructed by phase intervals, and a five-order image is obtained.

[0131] The optical transfer function construction module is configured to obtain different order spectrums according to the double objective interference structured light illumination point spread function and the image interference point spread function, perform deconvolution operation on the different order spectrum signals and the interference optical transfer function to obtain each order illumination spectrum, perform phase modulation on the interference optical transfer function and the each order illumination spectrum respectively, and then perform convolution operation on the modulated interference optical transfer function and the each order illumination spectrum to generate an optical transfer function library corresponding to a plurality of optical path differences.

[0132] In order to compensate for the estimated optical path difference (Optical Path Length Difference, OPLD) and optimize image reconstruction, a 4Pi-SIM optical transfer function (OTF) matched with the optical path difference (Optical Path Length Difference, OPLD) in the imaging data needs to be generated. In the embodiment of the present application, the system image interference microscopy optical transfer function (I 2 MOTF) and the six-beam light illumination spectrum information can be collected to obtain the 4Pi-SIM optical transfer function (OTF) by convolving the image interference microscopy optical transfer function (I 2 MOTF) and the 4Pi-SIM illumination optical transfer function (OTF).

[0133] By using the structured light illumination module and the wide-field illumination module in the embodiment of the present application, the point spread function (Point Spread Function, PSF) of 4Pi-SIM and the image interference microscopy point spread function (I 2 M PSF) can be collected. The spectral separation of the 4Pi-SIM data can obtain different order spectrums. The deconvolution of the different order spectrum signals and the optical transfer function (OTF) of the image interference microscopy (I 2 M) can obtain each order illumination spectrum.

[0134] In order to generate optical transfer functions (OTF) with different optical path length differences (OPLD), we phase modulate the image interferometric microscopy optical transfer function (I 2 M OTF) and the illumination spectrum, and then perform convolution operation on the two. In this way, a series of optical transfer functions (OTF) with specific optical path length differences (OPLD) are generated for image reconstruction.

[0135] The estimated structured light parameter module is configured to calculate a wave vector k0, a modulation depth m and an initial phase φ0 based on the spectrum information of the six-beam interference by using a preset algorithm.

[0136] For the wave vector k0, the integer pixel position thereof can be obtained by cross-correlation of the 0-order spectrum and the 2-order spectrum. For the sub-pixel estimation, the gradient descent method can be used to locate the position where the above two-order spectrum overlap region is the largest. Once the direction and size of k0 are determined, the 2-order spectrum can be moved to the corresponding position, and the pixel value of the overlap region of the two-order spectrum is counted. The pixel value of the overlap region is a complex number, the modulus of which represents the modulation depth, and the amplitude angle thereof represents the initial phase φ0. In the embodiments of the present application, the modulus is statistically averaged as the modulation depth m, and the amplitude angle is subjected to empirical mode decomposition (EMD) fitting, and after the low-order components of three items are discarded, the signal is recombined, and the maximum value of the filter curve is taken as the initial phase φ0.

[0137] The estimated optical path difference module is configured to determine the optical path difference according to the first original image and the second original image.

[0138] Due to factors such as air disturbance and thermal drift, it is very difficult to keep the optical path difference of the optical interference cavity to be zero during imaging. If the image is collected under the condition that the optical path difference is not zero, but it is assumed that the optical path difference is zero when the image is reconstructed, the problem of mismatching of the spectrum phase will be caused, which is manifested as axial ringing artifacts in the reconstructed image. In order to solve this problem, as shown in Figures 9(a) to 9(j) Fig. 9 shows the principle of optical path difference (OPLD) estimation of 4Pi-SIM. Fig. 9(a) shows a schematic diagram of the excitation pupil function. Fig. 9(b) shows a schematic diagram of the detection pupil function. Fig. 9(c) shows a schematic diagram of the optical transfer function (OTF) of the excitation light system. Fig. 9(d) shows a schematic diagram of the optical transfer function (OTF) of the detection light system. Fig. 9(e) shows the optical transfer function (OTF) of the 4Pi-SIM system. Figures 9(f) to 9(h) Fig. 9(f) shows the 0-order, 1-order and 2-order separated spectra. Figures 9(i) to 9(j) Fig. 9(g) shows the phase distribution of the six-beam interference spectrum and the overlap part. Fig. 9(h) shows the spectrum without sample information (such as an ideal point light source located at the center), and Fig. 9(i) shows the spectrum containing sample distribution. Fig. 9(g) shows the phase distribution of the six-beam interference spectrum and the overlap part. Fig. 9(h) shows the spectrum without sample information (such as an ideal point light source located at the center), and Fig. 9(i) shows the spectrum containing sample distribution.

[0139] The embodiment of the present application provides a method for estimating and compensating the influence of optical path difference even in a complex biological sample. Figures 9(a) to 9(j) In the method, a and b respectively represent unknown phase values generated by excitation and detection optical path difference (OPLD). Since the transmission and reflection are different by half-wave loss π at the depolarization beam splitter prism NPBS, the two sets of interference fringes are complementary (energy conservation). If the complementary patterns are added and subtracted, an image in which the influence of fluorescence interference is eliminated, i.e., an image in which only six beams interfere, is obtained. In the frequency domain, this is expressed as I 2 The side lobe of M is eliminated, and only the central lobe representing the wide field is left, which greatly simplifies the spectral information after Fourier transform, so that the optical path difference parameter can be extracted.

[0140] It is worth noting that, in the method, Figure 1 In the imaging device of the structured light illumination microscope based on double-objective interference shown in the embodiment of the present application, when performing double-color imaging, the fifth dichroic mirror DM5 and the filters F2 and F3 need to be arranged to obtain images of two different fluorescence channels corresponding to the first color fluorescence and the second color fluorescence; in the double-color imaging mode, the optical path difference information of the system cannot be estimated. In single-color imaging, the fifth dichroic mirror DM5 does not need to be arranged, so that the two images formed are complementary, and the phases of the two images are φ and φ+π respectively, and the optical path difference of the system can be estimated according to the phases of the two images.

[0141] In the simplified spectrum, the overlapping parts (region ③ and region ⑥) of the 0th order and the 1st order contain the optical path difference (OPLD) related information a. However, since these information is mixed with the phase of the sample (i.e., the phase s) and the initial phase of the illumination fringe (i.e., the phase φ0), it cannot be extracted. However, once the spectrum is moved to the position of k0, the overlapping parts will share the same sample phase. As long as φ0 estimated from the overlapping regions ④ and ⑤ is removed, the remaining phase represents the phase difference information of the excitation light. The fluorescence phase b can be obtained through the wavelength relationship b=aλ ex / λ em Calculation, wherein λ ex and λ em respectively represent the excitation wavelength and the emission wavelength.

[0142] The imaging module is configured to adopt a Wiener filter reconstruction algorithm, construct a Wiener filter based on a system optical transfer function, a structured light parameter and optical path difference information, and obtain a reconstructed image. The imaging formula is as follows:

[0143]

[0144] wherein S(K) represents a reconstructed spectrum, H d,nLet D represent the complex optical transfer function (OTF) of the system, D represent the separation spectrum, m represent the modulation depth, and p represent the modulation depth. n Let w represent the wave vector of each order, w represent the Wiener parameter related to the image signal-to-noise ratio, d represent the direction, and n represent the order.

[0145] like Figures 10(a) to 10(f) Figure 10(a) shows a schematic diagram of the reconstructed image according to an embodiment of this application. In this embodiment, incorporating optical path difference (OPLD) information into the reconstruction process can eliminate ringing artifacts. Figure 10(a) is a schematic diagram of the projection of fluorescent beads on the xy section. The upper and lower rows of dots in Figure 10(b) respectively show the reconstruction results when OPLD = 0 nm and the reconstruction results when OPLD = -160 nm. Figure 10(c) shows a schematic diagram comparing the intensity curves of a single fluorescent bead. Figure 10(d) shows a schematic diagram of the projection of fixed cells labeled with endoplasmic reticulum on the xy section. The upper and lower rows of dots in Figure 10(e) respectively show the reconstruction results when OPLD = 0 nm and the reconstruction results when OPLD = -180 nm. Figure 10(f) shows a schematic diagram comparing the intensity curves of a single layer of endoplasmic reticulum signal.

[0146] Furthermore, the hardware design of this application ensures the long-term stability of the illumination pattern of the six-beam interference, while the focus-locking module ensures the precise alignment of the two objectives. In terms of software design, the reconstruction algorithm can accurately estimate the optical path difference information from two complementary images and incorporate it into the reconstruction process, ultimately generating an image with minimal artifacts. This enables the creation of three-dimensional isotropic super-resolution images of fine subcellular structures in living cells.

[0147] like Figure 11 The diagram shown is a schematic flowchart of an imaging method for a structured light illumination microscope based on dual-objective interference according to another embodiment of this application. The imaging method 1100 for a structured light illumination microscope based on dual-objective interference according to an embodiment of this application may include the following steps: S1101, S1102, S1103, S1104, S1105, S1106, S1107, S1108, S1109, and S1110.

[0148] In step S1101, the first original image and the second original image are acquired;

[0149] In step S1102, the first original image and the second original image are preprocessed to obtain the spectral information of the six-beam interference.

[0150] In step S1103, a phase separation matrix is ​​constructed based on the first original image for which background removal and edge softening operations have been completed;

[0151] At step S1104, a phase-separated frequency band is obtained according to the phase separation matrix;

[0152] At step S1105, a structured parameter is estimated according to the spectrum information of the six-beam light interference;

[0153] At step S1106, a Wiener filter is generated according to the spectrum information of the six-beam light interference and the optical transfer function of the matched dual-objective interference structured light illumination microscope;

[0154] At step S1107, the Wiener filter is shifted;

[0155] At step S1108, each frequency band is multiplied with a corresponding Wiener filter according to the phase-separated frequency band and the Wiener filter;

[0156] At step S1109, the filtered bands are shifted and recombined;

[0157] At step S1110, an image is reconstructed according to the recombined filtered spectrum.

[0158] In an embodiment of the present application, as shown in Figure 12 the method further comprises steps S1201, S1202, S1203, S1204, S1205 and S1206:

[0159] At step S1201, a dual-objective interference structured light illumination point spread function and an image interference point spread function are collected;

[0160] At step S1202, an interference optical transfer function is generated according to the interference point spread function;

[0161] At step S1203, the dual-objective interference structured light illumination point spread function is spectrally separated to obtain different order spectrums; the dual-objective interference structured light illumination point spread function and the interference point spread function are deconvoluted;

[0162] At step S1204, the interference optical transfer function and the illumination spectrums of different orders are respectively phase-modulated, and then the modulated interference optical transfer function and the illumination spectrums of different orders are convoluted to generate an optical transfer function library corresponding to multiple optical path differences;

[0163] At step S1205, the spectrum signals of different orders are convoluted with the interference optical transfer function to obtain spectrums of different orders;

[0164] At step S1206, the optical transfer function corresponding to the estimated optical path difference according to the image reconstruction procedure is selected from the optical transfer function library for generating the Wiener filter.

[0165] In one embodiment of the present application, as shown in FIG. 13, the pre-processing of the first original image and the second original image includes steps S1301 and S1302: Figure 13

[0166] At step S1301, the second original image is mirror flipped and registered to obtain a third original image;

[0167] At step S1302, the first original image and the third original image are registered and added, and then Fourier transformed to obtain the spectral information of six-beam light interference.

[0168] In another embodiment of the present application, as shown in FIG. 14, the image reconstruction procedure 1400 can include: Figure 14

[0169] At step S1401, an original image A is acquired;

[0170] At step S1402, an original image B is acquired;

[0171] At step S1403, the original image B is mirror flipped and registered to obtain an original image B';

[0172] At step S1404, the original images A and B' are Fourier transformed to obtain the spectral information of six-beam light interference;

[0173] At step S1405, the original image A is subjected to background removal and edge softening operation;

[0174] At step S1406, a phase separation matrix is constructed according to the original image A after the background removal and edge softening operation;

[0175] At step S1407, separate bands are obtained according to the phase separation matrix; then step S1411 is performed;

[0176] At step S1408, a structured parameter is estimated according to the spectral information of six-beam light interference;

[0177] At step S1409, a Wiener filter is generated according to the spectral information of six-beam light interference, the estimated parameter, and a matched dual-objective interference structured light illumination optical transfer function (OTF);

[0178] ​​At step S1410, the Wiener filter is shifted according to the wave vector k0; then step S1411 is performed;

[0179] At step S1411, each band is multiplied by the corresponding Wiener filter according to the phase-separated bands and the Wiener filter;

[0180] At step S1412, the filtered bands are shifted and recombined;

[0181] At step S1413, an image is reconstructed according to the recombined filtered bands;

[0182] At step S1414, a dual-microscope interferometric structured illumination point spread function is acquired;

[0183] At step S1415, an interferometric point spread function is acquired;

[0184] At step S1416, the dual-microscope interferometric structured illumination point spread function and the interferometric point spread function are subjected to a Deconvolution operation;

[0185] At step S1417, an interferometric optical transfer function is generated according to the interferometric point spread function;

[0186] At step S1418, the dual-microscope interferometric structured illumination point spread function is subjected to a spectral separation to obtain different orders of spectrum;

[0187] At step S1419, the different orders of spectrum are subjected to a Convolution operation with the interferometric optical transfer function to obtain illumination spectrum of each order;

[0188] At step S1420, an optical path length difference (OPLD) required for an image reconstruction process is acquired;

[0189] At step S1421, the interferometric optical transfer function and the illumination spectrum of each order are respectively subjected to a phase modulation, and then the modulated interferometric optical transfer function and the illumination spectrum of each order are subjected to a Convolution operation to generate an optical transfer function library (OTF Library) corresponding to a plurality of optical path length differences.

[0190] At step S1422, an optical transfer function corresponding to the optical path length difference required for the image reconstruction process is selected from the optical transfer function library to be used for generating the Wiener filter.

[0191] In addition, according to an embodiment of the present application, a storage medium is also provided, and program instructions are stored in the storage medium, and when the program instructions are run by a computer or a processor, the corresponding steps of the imaging method of the two-lens interference based structured light illumination microscope are executed. The storage medium may, for example, include a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media.

[0192] Although the example embodiments have been described herein with reference to the accompanying drawings, it is to be understood that the example embodiments are only exemplary and are not intended to limit the scope of the present application. Those of ordinary skill in the art can make various changes and modifications without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of the present application as claimed in the appended claims.

[0193] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in connection with the embodiments disclosed herein can be realized by electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those of ordinary skill in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0194] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another device, or some features can be omitted or not executed.

[0195] In the specification provided herein, a large number of specific details are described. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some examples, well-known methods, structures and techniques are not described in detail in order not to obscure the understanding of the specification.

[0196] Similarly, it is to be understood that the features of the present application that are of a proprietary nature are set forth in the appended set of claims. Those skilled in the art will appreciate that not all features of the application are necessarily patentable, and that the specifics recited in the specification merely provide a convenient methodology for achieving the aspects of the application that are deemed to be the most important, novel and non-obvious characteristics of the application. Therefore, the claims are what define the scope of the application, and the specification is not to be used in determining the scope of the application.

[0197] Those skilled in the art will appreciate that all features described herein (including all companion claims, abstract, and drawings) can be combined in any combination, except where such combinations are mutually exclusive. Each feature disclosed in this specification (including any companion claims, abstract, and drawings) can be replaced by alternative features supporting the same, equivalent or similar purpose, unless stated otherwise.

[0198] Furthermore, those skilled in the art will recognize that references in the specification to "one embodiment", "an embodiment", "an example embodiment", mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily referring to a single, "one embodiment".

[0199] Various component embodiments of the application can be implemented in hardware, or as software modules running in one or more processors, or in combinations thereof. Skilled persons will appreciate that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functionality of some of the modules according to embodiments of the application. The application can also be implemented as a program (for example, a computer program and computer program product) for performing part or all of the methods described herein. Such program(s) of the present application can be stored on computer readable media, or can be transmitted over a network using any communication protocol.

[0200] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps other than those listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unitary claim, several devices or sub-claims can be joined by means of the expression "and / or". The use of the term "at least" followed by a list of one or more items should be interpreted as including at least one of the items but it does not exclude the presence of others not listed. The use of the term "one" followed by a list of one or more items should be interpreted as including at least one of the items but it does not exclude the presence of others not listed. It is emphasized that the terms "comprises / comprising" when used in this specification are taken to specify the presence of stated features, integers, steps or components but do not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.

[0201] The above description is only specific embodiments of the present application or specific explanations of specific embodiments, and the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, and all of them should be covered in the protection scope of the present application. The protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An imaging apparatus of a structured light illumination microscope based on two- objective interferometry, characterized in that, The device comprises a structured light illumination system, an optical path difference adjustment module and a focus locking system; wherein The structured light illumination system comprises a laser coupling module, a fast variable grating module, a polarization modulation module and a six-beam light alignment module; the laser coupling module is used for emitting a first laser beam; the fast variable grating module is arranged on the exit light path of the laser coupling module, used for receiving the first laser beam and emitting the first laser beam to the polarization modulation module after modulating the first laser beam into a spatial structured light beam; the polarization modulation module is used for compensating the non-equivalent phase delay between s-polarized light and p-polarized light of the spatial structured light beam; the six-beam light alignment module divides the processed spatial structured light beam into two symmetric spatial structured light beams, converges the two symmetric spatial structured light beams on a focal plane via two symmetric microscopic objectives, and forms a three-dimensional structured light illumination pattern at the focal plane through interference; the illumination pattern excites the fluorescence signal generated by the sample, which is collected by the microscopic objectives and converges to occur fluorescence interference, and finally the image is collected by a second camera to occur fluorescence interference; The optical path difference adjustment module is arranged on the light path of the six-beam light alignment module, and the optical path difference adjustment module is used for adjusting the optical path difference of the imaging device to be less than the coherence length of fluorescence; The focus locking system comprises a second laser emitter; the second laser emitter is used for emitting a second laser beam; the second laser beam focuses on a first camera via a lens group, a second dichroic mirror, a third dichroic mirror and a fourth dichroic mirror, to monitor the position and shape of the processed spatial structured light beam on the first camera, so that a system control module adjusts the relative position of the microscopic objective according to the position and the shape, to ensure that the sample is always on the focal point.

2. The apparatus of claim 1, wherein, The optical path difference adjustment module comprises an optical path difference coarse adjustment module and an optical path difference fine adjustment module; wherein The optical path difference coarse adjustment module comprises a long-stroke linear translation stage and a piezoelectric linear actuator, and is used for adjusting the optical path difference to a first preset range during initial calibration of the imaging device; The optical path difference fine adjustment module comprises a linear piezoelectric platform, and is used for adjusting the optical path difference to a second preset range; The first preset range is a micrometer order fluorescence interference range, and the second preset range is a nanometer order position where the optical path difference is zero.

3. The apparatus of claim 1, wherein, The laser coupling module comprises a first laser emitter, an acousto-optic tunable filter, a mechanical oscillator, a multimode optical fiber, a collimating lens and a lens group; the first laser emitter is used for generating the first laser beam; the first laser beam is filtered by the acousto-optic tunable filter, coupled into the multimode optical fiber, and the multimode optical fiber is wound on the mechanical oscillator, wherein the mechanical oscillator is used for filling the speckle pattern of the multimode optical fiber to obtain uniform partially coherent light illumination; the first laser beam passing through the multimode optical fiber reaches the collimating lens for collimation, and the first laser beam collimated by the collimating lens is emitted to the fast variable grating module via the lens group.

4. The apparatus of claim 3, wherein, The fast variable grating module comprises a ferroelectric liquid crystal fast spatial light modulator, a polarization beam splitter prism and a first half-wave plate; the polarization beam splitter prism receives the first laser beam and shoots the first laser beam into the ferroelectric liquid crystal fast spatial light modulator via the first half-wave plate, wherein the ferroelectric liquid crystal fast spatial light modulator is used to modulate the first laser beam into the spatial structured light beam; the spatial structured light beam enters the polarization modulation module after being split by the polarization beam splitter prism.

5. The apparatus of claim 4, wherein, The polarization modulation module comprises a spatial filter, a second half-wave plate, a liquid crystal phase retarder, two quarter-wave plates and a depolarization beam splitter prism; The spatial structured light beam is shot into the second half-wave plate via the spatial filter, wherein the second half-wave plate is used to rotate the spatial structure light by 45° relative to s polarization; the spatial structure light shot out of the second half-wave plate is shot into the liquid crystal phase retarder, wherein the fast axis of the liquid crystal phase retarder is aligned with the s polarization of the microscope objective; the liquid crystal phase retarder adjusts the phase of the spatial structured light beam; The spatial structured light beam shot out of the liquid crystal phase retarder is reflected by the depolarization beam splitter prism and is divided into two spatial structured light beams with consistent phase retardation, and the two spatial structured light beams form a symmetrical optical interference cavity; The depolarization beam splitter prism is used to keep the polarization states of the reflected light and the transmitted light the same. The two quarter-wave plates are respectively arranged before the microscope objective and after all the mirrors, and the fast axes of the two quarter-wave plates are arranged at 45° relative to s polarization.

6. The apparatus of claim 5, wherein, The six-beam alignment module comprises the microscope objective, which comprises an upper microscope objective and a lower microscope objective, wherein the lower microscope objective is installed on an XY piezoelectric translation stage, and the upper microscope objective is installed on a Z-axis piezoelectric displacement stage; when an interference image is acquired, fluorescent beads in a sample are taken as fiducials to align the lateral direction of the upper microscope objective and the axial direction of the lower microscope objective.

7. The apparatus of claim 1, wherein, The device further comprises a wide-field illumination module; The wide-field illumination module comprises a third laser emitter; the third laser emitter is used to emit a third laser beam; the third laser beam is aligned with the first laser beam in the optical path and is focused on the focal plane, and a dichroic mirror is arranged between the third laser beam and the first laser beam to provide one-side wide-field illumination for the focal plane.

8. The apparatus of claim 1, wherein, The device further comprises a two-color detection module; wherein The two-color detection module comprises a fifth dichroic mirror; the fifth dichroic mirror is arranged on the light path shot out of the depolarization beam splitter prism, wherein the light path comprises an upper light path passing through a lens, a mirror and another lens; the fifth dichroic mirror separates different spectral fluorescent detection light paths from the upper light path; the two-color detection module further comprises a filter set and a second camera, and different colored filters are arranged on the two fluorescent detection light paths to filter first color fluorescent light and second color fluorescent light; the second camera collects the first color fluorescent light through a first channel, and the second camera collects the second color fluorescent light through a second channel.

9. The apparatus of claim 1, wherein, The device further comprises a sample stage module; wherein The sample stage module, a sample stage and a three-dimensional displacement stage for driving the sample stage, wherein the sample stage is used to carry a sample sealed in glue between two high-precision wave plates; the three-dimensional displacement stage comprises an XY piezoelectric translation stage and a Z-axis piezoelectric displacement stage.

10. The apparatus of claim 1, wherein, The device further comprises a system control module, and the system control module comprises a controller; wherein The controller is configured to adjust the voltage of the liquid crystal phase retarder at a preset time interval, generate a camera exposure signal and an adjustable optical filter illumination modulation signal to collect an interference image of the sample at a current focal plane, and move the Z-axis piezoelectric displacement stage to different depths of the sample to the focal plane at a preset step, collect interference images according to the above steps, and until the whole sample is imaged. The controller is further configured to trigger the structured light illumination module and the first camera.

11. The apparatus of claim 10, wherein, The system control module further comprises an image reconstruction module; wherein The image reconstruction module is configured to reconstruct the original image based on the optical path difference containing the optical transfer function information to obtain a reconstructed super-resolution three-dimensional image by using a Wiener filter reconstruction algorithm according to the structured light parameters.

12. The apparatus of claim 11, wherein, The image reconstruction module comprises an original data preprocessing module, a constructed optical transfer function module, a structured light parameter estimation module, an estimated optical path difference module and an imaging module; wherein The original data preprocessing module is configured to preprocess the original data to remove background and edge softening, Fourier transform for the constructed optical transfer function module and the imaging module; and is configured to perform registration operation on the first original image and the second original image for the estimated optical path difference module to eliminate fluorescence interference and obtain spectral information of six-beam light interference; The constructed optical transfer function module is configured to obtain different orders of frequency spectrum according to the double-object lens interference structured light illumination point spread function and the image interference point spread function, perform deconvolution operation on the different orders of frequency spectrum signals and the interference optical transfer function to obtain each order of illumination spectrum, respectively modulate the phase of the interference optical transfer function and the each order of illumination spectrum, and then perform convolution operation on the modulated interference optical transfer function and the each order of illumination spectrum to generate an optical transfer function library corresponding to a plurality of optical path differences; The estimated structured light parameter module is configured to calculate wave vector, modulation depth and initial phase based on the spectral information of the six-beam light interference by using a preset algorithm; The estimated optical path difference module is configured to determine the optical path difference according to the first original image and the second original image; The imaging module is configured to construct a Wiener filter based on the system optical transfer function, the structured light parameters and the optical path difference information by using a Wiener filter reconstruction algorithm to obtain a reconstructed image.

13. The device according to any one of claims 1 to 12, wherein A plurality of lasers of different wavelengths are used for six-beam structured light illumination imaging; an additional laser is used for image interference microscopic imaging; another additional laser is used for monitoring and correcting the drift of the objective lens; and the sample comprises fluorescent beads and fixed cells.

14. The device according to any one of claims 1 to 12, wherein Six-beam structured light illumination imaging using multiple different wavelength lasers; image interferometric microscopy using one additional laser; monitoring and correcting for objective drift using another additional laser; the sample further comprising live cells.

15. An imaging method of a structured light illumination microscope based on dual objective lens interference, the method being applied to the imaging device of the structured light illumination microscope based on dual objective lens interference according to claims 1 to 14, characterized in that, The method comprises: acquiring a first raw image and a second raw image; pre-processing the first raw image and the second raw image to obtain six-beam interferometric spectral information; constructing a phase separation matrix from the first raw image having completed background removal and edge softening operations; obtaining phase-separated frequency bands from the phase separation matrix; estimating parameters from the six-beam interferometric spectral information; generating a Wiener filter from the six-beam interferometric spectral information and the optical transfer function of the matched dual-objective interferometric structured light illumination microscope; shifting the Wiener filter; multiplying each frequency band with the corresponding Wiener filter from the phase-separated frequency bands and the Wiener filter; shifting and recombining the filtered spectrum; reconstructing an image from the recombined filtered spectrum.

16. The method of claim 15, wherein, The method further comprises: acquiring a dual-objective interferometric structured light illumination point spread function and an image interferometric point spread function; generating an interferometric optical transfer function from the interferometric point spread function; spectrally separating the dual-objective interferometric structured light illumination point spread function to obtain different order spectra; deconvolving the dual-objective interferometric structured light illumination point spread function and the interferometric point spread function; phase modulating the interferometric optical transfer function and each order illumination spectrum respectively, and then convolving the modulated interferometric optical transfer function and each order illumination spectrum to generate an optical transfer function library corresponding to a specific optical path difference; convolving the different order spectral signals with the interferometric optical transfer function to obtain each order spectrum; selecting the corresponding optical transfer function in the optical transfer function library according to the estimated optical path difference in the image reconstruction process to generate the Wiener filter.

17. The method of claim 15, wherein, The pre-processing of the first raw image and the second raw image comprises: mirroring and registering the second raw image to obtain a third raw image; adding the first raw image and the third raw image after registration, and then performing Fourier transform to obtain six-beam interferometric spectral information.

18. A storage medium, characterized by The storage medium has a computer program stored thereon, which, when executed by a processor, causes the processor to perform the imaging method of the dual-objective interferometric structured light illumination microscope according to any one of claims 15 to 17.

Images