Imaging device and method of structured light illumination microscope based on double objective lens interference

Abstract

The invention provides an imaging device and method of a structured light illumination microscope based on double objective lens interference. A laser coupling module of the device is used for emitting a first laser beam; the fast variable grating module is used for modulating the first laser beam into a spatial structure beam; the polarization modulation module is used for compensating non-equivalent phase delay between s polarized light and p polarized light of the spatial structure light beam; the six-beam light alignment module is used for realizing transverse and axial alignment and interference of spatial structure light beams at a focal plane through two symmetrical microobjective lenses; the optical path difference adjusting module is used for adjusting the optical path difference to be smaller than the coherence length of fluorescence; and the focus locking system is used for ensuring that the spatial structure light beams are always kept in accurate alignment in long-time-history imaging. According to the embodiment of the invention, the illumination pattern of six-beam interference and fluorescence interference are kept stable for a long time, the accurate alignment of the two objective lenses is ensured, the optical path difference information is accurately estimated from the two complementary images, and finally the image with the minimum artifact is reconstructed.

Description

Technical Field

[0001] The present application relates to the technical field of optical super-resolution microscopic imaging, and in particular to an imaging device and method of a structured light illumination microscope based on dual objective lens interference. Background Art

[0002] Structured illumination microscopy (SIM) uses structured illumination to load the high-frequency information of an object into the detection passband of the optical system in the frequency domain in a spatial mixing manner to achieve super-resolution optical microscopy that breaks the diffraction limit. Structured illumination microscopy has become the most widely used technology in super-resolution optical microscopy of living cells due to its lower excitation light intensity, non-specific requirements for fluorescent dyes, and fast wide-field imaging advantages. However, the axial resolution of the three-dimensional structured light illumination system is lower than the lateral resolution, and the blurred axial details hinder the observation of fine subcellular structures in living cells. Therefore, it is necessary to provide an imaging device and method for a structured light illumination microscope based on dual objective interference in order to solve the above problems. Summary of the invention

[0003] In view of at least one of the above technical problems existing in the prior art, this application is proposed. According to one aspect of this application, an imaging device of a structured light illumination microscope based on dual objective lens interference is provided, the device comprising a structured light illumination system, an optical path difference adjustment module and a focus locking system; wherein,

[0004] The structured light illumination system comprises a laser coupling module, a fast variable grating module, a polarization modulation module and a six-beam alignment module; the laser coupling module is used to emit a first laser beam; the fast variable grating module is arranged on the outgoing light path of the laser coupling module, and is used to receive the first laser beam, and modulate the first laser beam into a spatial structure beam and then emit it to the polarization modulation module; the polarization modulation module is used to compensate for the unequal phase delay between the s-polarized light and the p-polarized light of the spatial structure beam; the six-beam alignment module divides the processed spatial structure beam into two symmetrical spatial structure beams, which converge on a focal plane through two symmetrical microscope objective lenses, and interfere at the focal plane to form a three-dimensional structured light illumination pattern; the fluorescence signal generated by the sample excited by the illumination pattern is collected by the microscope objective lens, and fluorescence interference occurs after convergence, and the final image is collected by the second camera, and fluorescence interference occurs;

[0005] The optical path difference adjustment module is arranged on the optical path of the six-beam light alignment module, and is used to adjust the optical path difference of the imaging device to be smaller than the coherence length of the fluorescence;

[0006] The focus locking system includes a second laser emitter; the second laser emitter is used to emit a second laser beam; the second laser beam is focused to the first camera after passing through a lens group, a second dichroic mirror, a third dichroic mirror and a fourth dichroic mirror, so as to monitor the position and shape of the processed spatial structure beam on the first camera, so that the system control module adjusts the relative position of the microscope objective lens according to the position and the shape to ensure that the sample is always in focus.

[0007] In some embodiments, the optical path difference adjustment module includes an optical path difference coarse adjustment module and an optical path difference fine adjustment module; wherein,

[0008] The optical path difference coarse adjustment module comprises a long-stroke linear translation stage and a piezoelectric linear actuator, and the optical path difference coarse adjustment module is used to adjust the optical path difference to a first preset range when initially calibrating the imaging device;

[0009] The optical path difference fine adjustment module comprises a linear piezoelectric platform for adjusting the optical path difference to a second preset range;

[0010] The first preset range is a fluorescence interference range of the micrometer order, and the second preset range is a position where the optical path difference of the nanometer order is zero.

[0011] In some embodiments, the laser coupling module includes a first laser emitter, an acousto-optic tunable filter, a mechanical oscillator, a multimode optical fiber, a collimating lens and a lens group; the first laser emitter is used to generate the first laser beam; the first laser beam is filtered by the acousto-optic tunable filter and then coupled into the multimode optical fiber, and the multimode optical fiber is wound on the mechanical oscillator, wherein the mechanical oscillator is used to fill the speckle pattern of the multimode optical fiber to obtain uniform partially coherent light illumination; the first laser beam via the multimode optical fiber reaches the collimating lens for collimation, and the first laser beam collimated by the collimating lens is emitted to the fast variable grating module through the lens group.

[0012] In some embodiments, the fast variable grating module includes a ferroelectric liquid crystal fast spatial light modulator, a polarization beam splitter prism and a first half-wave plate; the polarization beam splitter prism receives the first laser beam and injects the first laser beam into the ferroelectric liquid crystal fast spatial light modulator via the first half-wave plate, wherein the ferroelectric liquid crystal fast spatial light modulator is used to modulate the first laser beam into the spatial structure beam; the spatial structure beam enters the polarization modulation module after being split by the polarization beam splitter prism.

[0013] In some embodiments, the polarization modulation module includes a spatial filter, a second half-wave plate, a liquid crystal phase retarder, two quarter-wave plates, and a depolarizing beam splitter prism;

[0014] The spatial structure light beam passes through the spatial filter and is incident on the second half-wave plate, wherein the second half-wave plate is used to rotate the spatial structure light by 45° relative to the s-polarization; the spatial structure light emitted by the second half-wave plate is incident on the liquid crystal phase retarder, wherein the fast axis of the liquid crystal phase retarder is aligned with the s-polarization of the microscope objective lens; the liquid crystal phase retarder performs phase adjustment on the spatial structure light beam;

[0015] The spatial structure light beam emitted through the liquid crystal phase retarder is divided into two upper and lower spatial structure light beams with the same phase delay after being reflected by the depolarization beam splitter prism, and the upper and lower two spatial structure light beams form a symmetrical optical interference cavity;

[0016] Wherein, the depolarizing beam splitter prism is used to keep the polarization states of the reflected light and the transmitted light the same;

[0017] The two quarter wave plates are respectively arranged before the microscope objective lens and after all the reflectors, and the fast axes of the two quarter wave plates are set to 45° relative to the s polarization.

[0018] In some embodiments, the six-beam alignment module includes the microscope objective lens, which includes an upper microscope objective lens and a lower microscope objective lens, wherein the lower microscope objective lens is mounted on an XY piezoelectric translation stage, and the upper microscope objective lens is mounted on a Z-axis piezoelectric translation stage; when acquiring an interference image, the fluorescent beads in the sample are used as a target to align the lateral direction of the upper microscope objective lens and the axial direction of the lower microscope objective lens.

[0019] In some embodiments, the apparatus further comprises a wide field illumination module;

[0020] The wide-field illumination module comprises a third laser emitter, which is used to emit a third laser beam. The third laser beam is aligned with the first laser beam in optical path and is focused on a focal plane, and a dichroic mirror is arranged between the third laser beam and the first laser beam to provide single-sided wide-field illumination for the focal plane.

[0021] In some embodiments, the device further comprises a dual-color detection module; wherein,

[0022] The dual-color detection module includes a fifth dichroic mirror; the fifth dichroic mirror is arranged on the light path emitted by the depolarizing beam splitter prism, wherein the light path includes an upper light path passing through a lens, a reflector and another lens; the fifth dichroic mirror separates the upper light path into fluorescence detection light paths of different spectra respectively; the dual-color detection module also includes a filter group and a second camera, and filters of different colors are respectively arranged on the two fluorescence detection light paths to filter the first color fluorescence and the second color fluorescence; the second camera collects the first color fluorescence through the first channel, and the second camera collects the second color fluorescence through the second channel.

[0023] In some embodiments, the device further comprises a sample stage module; wherein,

[0024] The sample stage module, the sample stage and the three-dimensional displacement stage for driving the sample stage, wherein the sample stage is used to carry the sample sealed between two high-precision wave plates; the three-dimensional displacement stage includes an XY piezoelectric displacement stage and a Z-axis piezoelectric displacement stage.

[0025] In some embodiments, the device further comprises a system control module, wherein the system control module comprises a controller; wherein,

[0026] The controller is used to adjust the voltage of the liquid crystal phase retarder according to a preset time interval, generate a trigger camera exposure signal and an adjustable filter illumination modulation signal, so as to collect an interference image of the sample at a current focal plane; and move the Z-axis piezoelectric translation stage to different depths of the sample to the focal plane according to a preset step length, and collect interference images according to the above steps until the entire sample is imaged;

[0027] The controller is also used to trigger the structured light illumination module and the first camera.

[0028] In some embodiments, the system control module further includes an image reconstruction module; wherein,

[0029] The image reconstruction module is used to reconstruct the original image based on the optical path difference containing optical transfer function information according to the structured light parameters and the Wiener filter reconstruction algorithm to obtain a reconstructed super-resolution three-dimensional image.

[0030] In some embodiments, the image reconstruction module includes a raw data preprocessing module, an optical transfer function building module, a structured light parameter estimation module, an optical path difference estimation module and an imaging module; wherein,

[0031] The raw data preprocessing module is used to construct the optical transfer function module and the imaging module, and the preprocessing includes removing the background and softening the edge, and Fourier transforming according to the raw data; the preprocessing for estimating the optical path difference module includes the registration operation of the first raw image and the second raw image to eliminate fluorescence interference and obtain the spectrum information of the six-beam interference;

[0032] The optical transfer function construction module is used to obtain frequency spectra of different orders according to the dual-objective interference structured light illumination point spread function and the image interference point spread function, perform a deconvolution operation on the frequency spectrum signals of different orders and the interference optical transfer function to obtain illumination spectra of each order, respectively perform phase modulation on the interference optical transfer function and the illumination spectra of each order, and then perform a convolution operation on the modulated interference optical transfer function and the illumination spectra of each order to generate a plurality of optical transfer function libraries corresponding to optical path differences;

[0033] The module for estimating structured light parameters is used to calculate the wave vector, modulation depth and initial phase using a preset algorithm based on the spectrum information of the six-beam interference;

[0034] The optical path difference estimation module is used to determine the optical path difference according to the first original image and the second original image;

[0035] The imaging module is used to adopt the Wiener filter reconstruction algorithm to construct a Wiener filter based on the system optical transfer function, structured light parameters and optical path difference information to obtain a reconstructed image.

[0036] In some embodiments, multiple lasers of different wavelengths are used to perform six-beam structured light illumination imaging; an additional laser is used to perform image interference microscopy imaging; another additional laser is used to monitor and correct the drift of the objective lens; and the sample includes fluorescent beads, fixed cells, or living cells.

[0037] Another aspect of the present application provides an imaging method of a structured light illumination microscope based on dual objective lens interference, the method being applied to the imaging device of the structured light illumination microscope based on dual objective lens interference as described above, the method comprising:

[0038] Acquire a first original image and a second original image;

[0039] Preprocessing the first original image and the second original image to obtain spectrum information of six-beam interference;

[0040] constructing a phase separation matrix according to the first original image after background removal and edge softening operations;

[0041] Obtaining a phase-separated frequency band according to the phase-separation matrix;

[0042] Estimating parameters according to the spectrum information of the six-beam interference;

[0043] Estimating parameters according to the spectrum information of the six-beam interference and the optical transfer function of the matching dual-objective interference structured light illumination microscope to generate a Wiener filter;

[0044] moving the Wiener filter;

[0045] According to the separated frequency bands and the Wiener filter, multiplying each frequency band with a corresponding Wiener filter;

[0046] Shift and reassemble the filtered spectrum;

[0047] An image is reconstructed based on the recombined filtered spectrum.

[0048] In some embodiments, the method further comprises:

[0049] Collect dual-objective interference structured light illumination point spread function and image interference point spread function;

[0050] generating an interferometric optical transfer function according to the interferometric point spread function;

[0051] Performing spectrum separation on the dual-objective lens interference structured light illumination point spread function to obtain spectrums of different orders; performing a deconvolution operation on the dual-objective lens interference structured light illumination point spread function and the interference light point spread function;

[0052] Phase modulating the interference optical transfer function and the illumination spectrum of each order respectively, and then performing a convolution operation on the modulated interference optical transfer function and the illumination spectrum of each order to generate an optical transfer function corresponding to a specific optical path difference;

[0053] Convolution operation is performed on the spectrum signals of different orders and the interference optical transfer function to obtain spectrums of various orders;

[0054] A corresponding optical transfer function is selected from the optical transfer function library according to the optical path difference estimated in the image reconstruction process to generate the Wiener filter.

[0055] In some embodiments, preprocessing the first original image and the second original image includes:

[0056] Performing mirror flipping and registration on the second original image to obtain a third original image;

[0057] The first original image and the third original image are registered and added, and then Fourier transformed to obtain spectrum information of six-beam interference.

[0058] On another aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor executes the imaging method of the structured light illumination microscope based on dual objective lens interference as described above.

[0059] The imaging device of the structured light illumination microscope based on dual objective lens interference of the embodiment of the present application uses two objective lenses for interference illumination and detection, so only half the excitation power is needed to achieve the same interference contrast. In addition, the hardware design of the embodiment of the present application ensures that the illumination pattern of the six-beam interference remains stable for a long time, and the focus lock module ensures the precise alignment of the two objective lenses; in terms of software design, the reconstruction algorithm can accurately estimate the optical path difference information from the two complementary images and include it in the reconstruction process, and finally generate an image with minimal artifacts, which can realize three-dimensional isotropic super-resolution images of fine sub-cells in living cells. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0061] Figure 1 A schematic diagram showing an imaging device of a structured light illumination microscope based on dual objective lens interference according to an embodiment of the present application;

[0062] Figure 2 A schematic diagram showing a structured light illumination optical path according to an embodiment of the present application;

[0063] FIG. 3( a ) is a schematic diagram showing that the direction of the outgoing polarized light can be rotated by adjusting the liquid crystal delay in a conventional SIM system;

[0064] FIG3( b ) is a schematic diagram showing a polarization modulation module according to an embodiment of the present application that adjusts the polarization of upper and lower illumination lights to a desired polarization state;

[0065] FIG4( a) shows a schematic diagram of an optical interference cavity according to an embodiment of the present application;

[0066] FIG4( b ) shows a record of fluorescence intensity fluctuations of a coarse adjustment of optical path difference according to an embodiment of the present application;

[0067] FIG4( c ) shows a record of fluorescence intensity fluctuations with fine adjustment of the optical path difference according to an embodiment of the present application;

[0068] FIG5( a) is a schematic diagram showing an optical path of a focus locking module according to an embodiment of the present application;

[0069] FIG5( b ) is a schematic diagram showing a method of moving an objective lens to lock focus according to an embodiment of the present application;

[0070] FIG6( a ) is a schematic diagram showing a dichroic mirror DM5 that separates two color signals, and filters F2 and F3 that are used to further filter the fluorescence signal according to an embodiment of the present application;

[0071] FIG6( b ) is a schematic diagram showing the excitation spectrum and emission spectrum of fluorescent proteins (mStayGol d and dCyOFP2s) used for dual-color imaging according to an embodiment of the present application;

[0072] Figure 7 A schematic diagram showing a sample stage module according to an embodiment of the present application;

[0073] Figure 8 shows a signal timing diagram according to an embodiment of the present application;

[0074] Figure 9(a) to Figure 9(j) A schematic diagram showing the optical path difference (OPLD) estimation principle of 4Pi-SIM according to an embodiment of the present application is shown;

[0075] FIG10( a ) is a schematic diagram showing a projection of a fluorescent bead in an xy cross section according to an embodiment of the present application;

[0076] FIG. 10( b ) is a schematic diagram showing the reconstruction result when OPLD=0 nm and the reconstruction result when OPLD=−160 nm according to an embodiment of the present application;

[0077] FIG10( c ) is a schematic diagram showing a comparison of intensity curves of a single fluorescent bead according to an embodiment of the present application;

[0078] FIG10( d ) shows a schematic projection diagram of a fixed cell labeled with endoplasmic reticulum in an xy cross section according to an embodiment of the present application;

[0079] FIG. 10( e ) is a schematic diagram showing a reconstruction result of OPLD=0 nm and a reconstruction result of OPLD=−180 nm according to an embodiment of the present application;

[0080] FIG10( f ) is a schematic diagram showing a comparison of intensity curves of single-layer endoplasmic reticulum signals according to an embodiment of the present application;

[0081] Fig.11 A schematic flow chart showing an imaging method of a structured light illumination microscope based on dual objective lens interference according to an embodiment of the present application;

[0082] Fig.12 A schematic flow chart showing the generation of an optical transfer function library according to an imaging method of a structured light illumination microscope based on dual objective lens interference according to an embodiment of the present application;

[0083] Fig.13 A schematic flow chart showing preprocessing of a first original image and a second original image according to an imaging method of a structured light illumination microscope based on dual objective lens interference according to an embodiment of the present application;

[0084] Fig.14 A schematic flow chart showing an imaging method of a structured light illumination microscope based on dual objective lens interference according to another embodiment of the present application. DETAILED DESCRIPTION

[0085] In order to enable those skilled in the art to better understand the technical solutions of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0086] Based on at least one of the aforementioned technical problems, the present application provides an imaging method of a structured light illumination microscope based on dual objective lens interference, the device comprising: a structured light illumination system, an optical path difference adjustment module and a focus locking system; wherein the structured light illumination system comprises a laser coupling module, a fast variable grating module, a polarization modulation module and a six-beam alignment module; the laser coupling module is used to emit a first laser beam; the fast variable grating module is arranged on the outgoing light path of the laser coupling module, for receiving the first laser beam, and modulating the first laser beam into a spatial structure beam and then emitting it to the polarization modulation module; the polarization modulation module is used to compensate for the unequal phase delay between the s-polarized light and the p-polarized light of the spatial structure beam; the six-beam alignment module divides the processed spatial structure beam into two symmetrical spatial structure beams, which are converged through two symmetrical microscope objective lenses. The six-beam light is focused on the focal plane, and interference occurs at the focal plane to form a three-dimensional structured light illumination pattern; the fluorescence signal generated by the sample excited by the illumination pattern is collected by the microscope objective lens, and fluorescence interference occurs after convergence, and the final image is collected by the second camera, and fluorescence interference occurs; the optical path difference adjustment module is arranged on the optical path between the polarization modulation module and the six-beam light alignment module, and the optical path difference adjustment module is used to adjust the optical path difference of the imaging device to be less than the coherence length of the fluorescence; the focus locking system includes a second laser emitter; the second laser emitter is used to emit a second laser beam; the second laser beam is focused to the first camera after passing through a lens group and a color separation mirror group, so as to monitor the position and shape of the processed spatial structure light beam on the first camera, so that the system control module adjusts the relative position of the microscope objective lens according to the position and the shape to ensure that the sample is always in focus. In addition, the hardware design of the embodiment of the present application ensures that the illumination pattern of the six-beam interference remains stable for a long time, and the focus lock module ensures the precise alignment of the two objective lenses; in terms of software design, the reconstruction algorithm can accurately estimate the optical path difference information from the two complementary images and incorporate it into the reconstruction process, ultimately generating an image with minimal artifacts.

[0087] Figure 1 A schematic diagram of an imaging device of a structured light illumination microscope based on dual objective lens interference according to an embodiment of the present application is shown; Figure 1 As shown, the imaging device 100 of the structured light illumination microscope based on dual objective lens interference according to an embodiment of the present application may include a structured light illumination system, an optical path difference adjustment module and a focus locking system.

[0088] The structured light illumination system includes a laser coupling module, a fast variable grating module, a polarization modulation module and a six-beam alignment module; the laser coupling module is used to emit a first laser beam; the fast variable grating module is arranged on the output light path of the laser coupling module, and is used to receive the first laser beam, and modulate the first laser beam into a spatial structure beam and then emit it to the polarization modulation module; the polarization modulation module is used to compensate for the unequal phase delay between the s-polarized light and the p-polarized light of the spatial structure beam; the six-beam alignment module divides the processed spatial structure beam into two symmetrical spatial structure beams, which converge at the focal plane through two symmetrical microscope objective lenses, and interfere at the focal plane to form a three-dimensional structured light illumination pattern; the fluorescence signal generated by the sample excited by the illumination pattern is collected by the microscope objective lens, and fluorescence interference occurs after convergence, and the final image is collected by the second camera, and fluorescence interference occurs. The sample includes fluorescent beads, fixed cells or living cells.

[0089] The structure and function of each module of the structured light illumination system are introduced below.

[0090] In the first embodiment, if Figure 1 and Figure 2 As shown, the laser coupling module includes a first laser emitter, an acousto-optic tunable filter AOTF1, a mechanical oscillator Shaker, a multimode optical fiber MMF, a collimating lens CL and a lens group, wherein the lens group includes a first lens L1 and a second lens L2.

[0091] The first laser emitter is used to generate the first laser beam (Laser1), and a 488nm laser can be used as the light source. The first laser beam (Laser1) is filtered by the acousto-optic tunable filter AOTF1 and then emitted to the mechanical oscillator Shaker. The mechanical oscillator is used to fill the speckle pattern of the multimode optical fiber MMF to obtain uniform partially coherent light illumination. The first laser beam (Laser1) emitted from the multimode optical fiber MMF reaches the collimating lens CL for collimation, and the collimated first laser beam (Laser1) is emitted to the fast variable grating module through the lens groups L1 and L2.

[0092] Specifically, the optical power is modulated by an acousto-optic tunable filter AOTF1. The laser beam is coupled to a multimode fiber through a five-axis lens. The mechanical oscillator Shaker is fixed on the multimode fiber MMF wound into a bundle, and the speckle pattern of the multimode fiber MMF is filled by mechanical vibration to obtain uniform partially coherent light illumination. The divergent light beam at the output port of the multimode fiber MMF is collimated by a collimating lens CL, and then passes through a pair of relay lenses L1 and L2 to ensure that the fiber end face is projected onto the back focal plane of the objective lens.

[0093] In the second embodiment, continue to combine Figure 1 The fast variable grating module includes a ferroelectric liquid crystal fast spatial light modulator SLM, a polarization beam splitter prism PBS, and a first half-wave plate HWP1; the polarization beam splitter prism PBS receives the first laser beam, and injects the first laser beam (Laser1) into the ferroelectric liquid crystal fast spatial light modulator SLM through the first half-wave plate HWP1, wherein the ferroelectric liquid crystal fast spatial light modulator SLM is used to modulate the first laser beam (Laser1) into the spatial structure beam; the spatial structure beam enters the polarization modulation module after being split by the polarization beam splitter prism PBS.

[0094] Specifically, the variable grating module is composed of a ferroelectric liquid crystal fast spatial light modulator, a polarization beam splitter prism and the first half glass HWP1. By loading different diffraction patterns on the SLM, the variable grating module can generate diffracted light beams within milliseconds (ms), and then interfere on the sample surface to form the required structured light stripes. In the dual-objective interference structured light illumination system (4Pi-SIM), it is necessary to generate binary grating patterns in three directions and five phases and arrange them in a certain time sequence.

[0095] In addition, the embodiment of the present application further provides a spatial mask as a spatial filter to filter out additional diffraction orders generated by the binary grating.

[0096] In a third embodiment, combining Figure 1 3(a) and 3(b), the polarization modulation module includes a spatial filter Mask, a second half-wave plate HWP2, a liquid crystal phase retarder HS-LCVR, two quarter-wave plates QWP1 and QWP2, a depolarizing beam splitter prism NPBS, a lens group L3, L4, L5, L6, L7, L8, L9, and a reflector group M1, M2, M3, M4, M5, M6, M7, M8, M9, M14, M15, and a first dichroic mirror DM1.

[0097] The spatial structured light beam passes through the spatial filter Mask and is incident on the second half-wave plate HWP2, wherein the second half-wave plate HWP2 is used to rotate the spatial structured light by 45° relative to the s-polarization; the spatial structured light emitted by the second half-wave plate HWP2 is incident on the liquid crystal phase retarder HS-LCVR, wherein the fast axis of the liquid crystal phase retarder HS-LCVR is aligned with the s-polarization of the microscope objective lenses OBJ1 and OBJ2; the liquid crystal phase retarder HS-LCVR performs phase adjustment on the spatial structured light beam. The spatial structured light beam emitted through the liquid crystal phase retarder HS-LCVR is transmitted and reflected by the depolarization beam splitter NPBS and is divided into two upper and lower spatial structured light beams with the same phase delay, and the upper and lower two spatial structured light beams form a symmetrical optical interference cavity.

[0098] The depolarizing beam splitter NPBS is used to keep the polarization states of the reflected light and the transmitted light the same. The two quarter wave plates QWP1 and QWP2 are respectively arranged before the microscope objective lenses OBJ1 and OBJ2 and after all the reflectors, and the fast axes of the two quarter wave plates QWP1 and QWP2 are set to 45° relative to the s polarization.

[0099] Specifically, in order to maximize the contrast of the illumination mode in SIM and thus optimize its spectrum expansion capability, a polarization modulation module needs to be provided, which can ensure that the polarization states of the six illumination lights are tangent to the rear pupil plane of the objective lens, thereby achieving the highest interference contrast.

[0100] In the SIM system of the conventional technology, as shown in Figure 3(a), a polarization direction rotator composed of a liquid crystal phase retarder HS-LCVR and quarter-wave plates QWP1 and QWP2 is usually used. The direction of the outgoing polarized light can be rotated by adjusting the liquid crystal delay. Such an optical design is not applicable in the 4Pi-SIM system because many reflective optical elements are inevitably placed between the polarization modulation module and the sample. The reflection of these elements may introduce unequal phase delays between s-polarization and p-polarization, resulting in elliptically polarized illumination light appearing in the sample plane, thereby reducing the modulation contrast.

[0101] In order to solve this problem, the polarization modulation module of the embodiment of the present application can compensate for the unequal phase delay between s-polarization and p-polarization. As shown in Figure 3(b), the system uses a quarter-wave plate HWP2 to rotate the polarization to 45° relative to the s-polarization before the liquid crystal phase retarder HS-LCVR, and the fast axis of the liquid crystal phase retarder HS-LCVR is set to 0°. The two quarter-wave plates QWP1 and QWP2 are at a 45° angle to the s-polarization of the incident light and are placed before the objective lens and after all reflectors. On the one hand, the symmetrical design of the optical interference cavity is adopted to ensure that the phase delay experienced by the upper and lower lights is consistent; on the other hand, a customized depolarizing beam splitter NPBS is used to keep the polarization state between the reflected light and the transmitted light the same. Therefore, it is only necessary to adjust the voltage of the liquid crystal phase retarder HS-LCVR to adjust the polarization of the upper and lower illumination lights to the desired polarization state.

[0102] In the fourth embodiment, Figure 1 As shown, the six-beam alignment module includes the microscope objective lens, which includes an upper microscope objective lens and a lower microscope objective lens, wherein the lower microscope objective lens is mounted on an XY piezoelectric translation stage, and the upper microscope objective lens is mounted on a Z-axis piezoelectric translation stage; when acquiring an interference image, the fluorescent beads in the sample are used as target marks to align the lateral direction of the upper microscope objective lens and the axial direction of the lower microscope objective lens.

[0103] Specifically, in order to ensure that the six-beam interference adjustment has sufficient degrees of freedom, the upper microscope objective is mounted on a one-dimensional linear piezoelectric translation stage with freedom in the Z direction; the lower objective is mounted on a two-dimensional piezoelectric translation stage with freedom in the XY direction.

[0104] In one example, multiple lasers of different wavelengths can be used for six-beam structured light illumination imaging; an additional laser can be used for image interference microscopy imaging; and another additional laser can be used to monitor and correct objective lens drift.

[0105] Wherein, the optical path difference adjustment module is arranged on the optical path of the six-beam light alignment module, and the optical path difference adjustment module is used to adjust the optical path difference of the imaging device to be smaller than the coherence length of the fluorescence.

[0106] In one embodiment of the present application, the optical path difference adjustment module includes an optical path difference coarse adjustment module (CTS) and an optical path difference fine adjustment module (FTS), as well as reflector groups M10, M11, M12, and M13. The optical path difference coarse adjustment module includes a long-stroke linear translation stage and a piezoelectric linear actuator, and the optical path difference coarse adjustment module is used to adjust the optical path difference to a first preset range when the imaging device is initially calibrated. The optical path difference fine adjustment module includes a linear piezoelectric platform for adjusting the optical path difference to a second preset range. The first preset range is the fluorescence interference range (micrometer level), and the second preset range is the position where the optical path difference is zero (nanometer level).

[0107] Specifically, combined Figure 1 and Figure 4(a) to Figure 4(c) The axial fringe contrast is particularly important to the performance of 4Pi-SIM. In order to maximize the axial fringe contrast, the optical path length difference (OPLD) between the upper and lower arms of the optical interferometer cavity needs to be accurately adjusted to 0.

[0108] In order to solve this problem, an embodiment of the present application provides an optical path difference (OPLD) adjustment module, which is composed of an optical path difference fine adjustment module FTS and an optical path difference coarse adjustment module CTS, which are respectively located in the upper and lower arms. The two modules have similar designs in hardware: a compact optical delay line composed of fixed knife-edge right-angle silver mirrors PM1 and PM2 and a pair of silver mirrors mounted on a translation stage. The difference between the optical path difference fine adjustment module FTS and the optical path difference coarse adjustment module CTS is that the optical path difference coarse adjustment module CTS is composed of a long-stroke linear translation stage and a piezoelectric linear actuator. The optical path difference coarse adjustment module CTS is used to coarsely adjust the optical path difference (OPLD) to within 1μm.

[0109] The optical path difference fine adjustment module FTS uses a linear piezoelectric platform with sub-nanometer resolution. The optical path difference fine adjustment module FTS can accurately adjust the optical path difference (OPLD) to 0.

[0110] As shown in FIG4(a), it is a schematic diagram of an optical interference cavity according to an embodiment of the present application. As shown in FIG4(b), it is a record of fluorescence intensity fluctuations of coarse adjustment of optical path difference according to an embodiment of the present application. As shown in FIG4(c), it is a record of fluorescence intensity fluctuations of fine adjustment of optical path difference according to an embodiment of the present application.

[0111] Wherein, the focus locking system comprises a second laser emitter, and the second laser emitter is used to emit a second laser beam (Laser2). The second laser beam (Laser2) is focused to the first camera after passing through a lens group, a second dichroic mirror, a third dichroic mirror and a fourth dichroic mirror, so as to monitor the position and shape of the processed spatial structure beam on the first camera, and enable the system control module to adjust the relative position of the microscope objective lens according to the position and the shape, so as to ensure that the sample is always in focus.

[0112] Specifically, continue to combine Figure 1 5(a) to (b), the second laser emitter can generate a near-infrared 785nm laser as a focus-locking light source. The light beam is emitted from the single-mode fiber SMF2, collimated by the lens L18, and then reflected by the dichroic mirror DM2. After passing through two microscope lenses OBJ1 and OBJ2, it is reflected by another dichroic mirror DM3 and focused onto the camera CAM1 by the lens L17, the dichroic mirror DM4 and the cylindrical lens CY1. By monitoring the position and shape of the laser beam on the first camera CAM1, the system can track and adjust the relative position between the two microscope lenses OBJ1 and OBJ2 in real time to keep the sample always in focus.

[0113] As can be seen from Figure 5(b), if the objective lens is offset in the lateral direction, the spot position will change; if the objective lens is offset up and down in the axial direction, the spot shape will change due to the introduction of astigmatism. This variable quantitatively reflects the value of the objective lens offset, and by feeding this distance value back to the translation stage module of the upper and lower microscope objective lenses OBJ1 and OBJ2, real-time focus locking can be achieved.

[0114] The embodiments of the present application ensure that the sample maintains the optimal focal position during long-term imaging through initial alignment, laser beam adjustment, focus monitoring and real-time adjustment, thereby obtaining high-quality super-resolution images.

[0115] In one embodiment of the present application, the imaging device further includes a wide-field illumination module. The wide-field illumination module includes a third laser emitter; the third laser emitter is used to emit a third laser beam; the third laser beam is aligned with the first laser beam in optical path and focused on a focal plane, and a dichroic mirror is provided between the third laser beam (Laser3) and the first laser beam (Laser1) to provide single-sided wide-field illumination for the focal plane.

[0116] The third laser transmitter of the embodiment of the present application can be a 473nm laser transmitter. The 473nm laser transmitter emits a 473nm laser beam, which is modulated by the acousto-optic tunable filter AOTF2, and then passes through the single-mode fiber SMF1 and lens L16, and is introduced into the 473 laser through the reflection of the dichroic mirror DM2. The 473nm laser beam is aligned with the central optical path of the 488nm laser beam and separated by the dichroic mirror to provide single-sided wide-field WF illumination. The resulting image interference microscopy (I 2 M) The image contains only fluorescence interference and can be used to locate the target area of ​​the sample in a large field of view and to collect the optical transfer function (OTF) of the imaging device.

[0117] In one embodiment of the present application, continue to combine Figure 1 , the imaging device 100 also includes a two-color detection module. The two-color detection module includes a fifth dichroic mirror DM5, a lens group L10, L11, L12, L13, L14, L15, a knife-edge right-angle silver mirror PM3, a mirror group M16, M17, M18, M19, a filter F1, and a second camera CAM2; the fifth dichroic mirror DM5 is arranged on the light path emitted by the depolarizing beam splitter prism NPBS. The light path emitted by the depolarizing beam splitter prism NPBS includes an upper light path passing through the lens L10, the reflector M16 and the lens L12, and a lower light path passing through the dichroic mirror DM1, the lens L11, the reflector M17, and the lens L13. The fifth dichroic mirror DM5 separates the upper light path into fluorescence detection light paths of different spectra. The dual-color detection module also includes a filter set and a second camera CAM2. Filters F2 and F3 of different colors are respectively arranged on the two fluorescence detection light paths to filter the first color fluorescence (e.g., green light) and the second color fluorescence (e.g., red light). The second camera CAM2 collects the first color fluorescence through the first channel, and the second camera collects the second color fluorescence through the second channel.

[0118] Specifically, combined Figure 1 and Figure 6(a) to Figure 6(b) , Figure 6(a) shows a schematic diagram of the fifth dichroic mirror DM5 separating the two color signals, and filters F2 and F3 are used to further filter the fluorescence signal. Figure 6(b) shows a schematic diagram of the excitation spectrum and emission spectrum of the fluorescent proteins (mStayGold and dCyOFP2s) used for dual-color imaging.

[0119] The embodiments of the present application provide two methods of dual-color imaging: first, a single-wavelength laser simultaneously excites a conventional fluorescent protein (e.g., oxStaygold) and a special protein with a large Stokes shift (e.g., dCyOFP2s) to simultaneously collect signals from two channels. Second, a dual-wavelength laser sequentially excites green fluorescent protein and red fluorescent protein to sequentially collect fluorescence signals from the two channels. The first method can capture two colors of fluorescence simultaneously within the same exposure time, and can simultaneously optimize the axial illumination contrast of the two detection channels.

[0120] In the imaging optical path, a customized fifth dichroic mirror DM5 is introduced after lens L12 and lens L13 to separate fluorescence of different spectra. The parameters of the fifth dichroic mirror DM5 include a flatness not less than λ / 8 and double-sided anti-reflection coating to reduce multiple reflections from unconventional incident surfaces. A green filter F2 is installed on the upper optical path of the detection optical path to collect green fluorescence; a red filter F3 is installed on the lower optical path of the detection optical path to collect red fluorescence.

[0121] In one embodiment of the present application, the imaging device further comprises a sample stage module, wherein the sample stage module is used to carry a sample sealed between two high-precision wave plates; and the three-dimensional displacement stage comprises an XY piezoelectric displacement stage and a Z-axis piezoelectric displacement stage.

[0122] Specifically, combined Figure 1 and Figure 7 The three-dimensional translation stage is composed of a two-dimensional XY piezoelectric translation stage and a one-dimensional Z piezoelectric translation stage, so as to facilitate positioning and movement in the X, Y, and Z axis directions. A sample slot for placing samples is set on the three-dimensional translation stage. The sample slot adopts a "sandwich" shape design, and the sample is fixed in the customized circular sample slot by two high-precision glass slides sealed with glue.

[0123] In one embodiment of the present application, the imaging device further includes a system control module, and the system control module includes a controller and an imaging module.

[0124] The controller is used to adjust the voltage of the liquid crystal phase retarder according to a preset time interval, generate a trigger camera exposure signal and an adjustable filter illumination modulation signal to collect the interference image of the current focal plane; to collect the interference image of the sample in the current focal plane; and to move the Z-axis piezoelectric translation stage to different depths of the sample to the focal plane according to a preset step size, and collect the interference image according to the above steps until the entire sample is imaged.

[0125] like Figure 8As shown in the figure, the system uses the LED_EN transistor-transistor logic (TTL) output signal of the ferroelectric liquid crystal spatial light modulator SLM as the main trigger: the ferroelectric liquid crystal spatial light modulator SLM is programmed to set a series of binary pixel patterns and the display time of each pattern, so its on-board microcontroller ensures the switching and timing of the pattern display and sends out the LED_EN transistor-transistor logic (TTL) signal to trigger the camera exposure and the acousto-optic tunable filter AOTF1 to modulate the illumination. After detecting 5 falling edges of the transistor-transistor logic (TTL) output of the ferroelectric liquid crystal spatial light modulator SLM, the input voltage of the liquid crystal phase retarder HS-LCVR begins to change, indicating that the acquisition of one illumination direction has been completed. When the falling edge count reaches 15, it indicates that the acquisition on this focal plane is completed, and the sample Z translation stage will move to the next focal plane according to the preset step size. Once all focal planes have been imaged, a reset signal is sent to the Z stage to return it to its initial position.

[0126] For 3D time-lapse imaging, the embodiment of the present application sets a custom time interval between the acquisition of each 3D image. During this time interval, the camera, laser, liquid crystal phase retarder HS-LCVR and sample stage will not be triggered. For two-color imaging, since only one illumination wavelength is used and the two-color channels are detected simultaneously, the acquisition procedure is the same as that of monochrome imaging and will not be repeated here.

[0127] The image reconstruction module is used to reconstruct the original image based on the optical path difference containing optical transfer function information according to the structured light parameters and the Wiener filter reconstruction algorithm to obtain a reconstructed super-resolution three-dimensional image.

[0128] The image reconstruction module includes an optical transfer function construction module, a raw data preprocessing module, a structured light parameter estimation module, an optical path difference estimation module and an imaging module.

[0129] Among them, the raw data preprocessing module is used to construct the optical transfer function module and the imaging module, and the preprocessing includes background removal and edge softening and Fourier transform according to the raw data; the preprocessing for estimating the optical path difference module includes the alignment operation of the first raw image and the second raw image to eliminate fluorescence interference and obtain the spectrum information of the six-beam interference.

[0130] In order to perform parameter estimation and reconstruction, the raw data needs to be preprocessed. First, the target area (A and B) of the two sets of interference complementary images is intercepted from the raw data, and one of the images is mirrored and registered (B'). The two 4Pi-SIM images are respectively background-removed and edge-softened, and then Fourier transformed to obtain their respective spectral information; the two 4Pi-SIM images are added after registration and then Fourier transformed to eliminate fluorescence interference, thereby obtaining the spectral information of the six-beam interference. The above spectrum is separated using the separation matrix constructed using the phase interval to obtain a fifth-order image.

[0131] Among them, the optical transfer function construction module is used to obtain frequency spectra of different orders according to the dual-objective interference structured light illumination point spread function and the image interference point spread function, deconvolve the frequency spectrum signals of different orders with the interference optical transfer function to obtain illumination spectra of each order, phase modulate the interference optical transfer function and the illumination spectra of each order respectively, and then convolve the modulated interference optical transfer function and the illumination spectra of each order to generate an optical transfer function library corresponding to multiple optical path differences.

[0132] In order to compensate for the estimated optical path length difference (OPLD) and optimize image reconstruction, it is necessary to generate a 4Pi-SIM optical transfer function (OTF) that matches the optical path length difference (OPLD) in the imaging data. In the embodiment of the present application, the system image interference microscopy optical transfer function (OTF) can be collected. 2 M OTF) and six-beam illumination spectrum information to be obtained by image interferometry optical transfer function (I 2 The 4Pi-SIM optical transfer function (OTF) is obtained by convolving the 4Pi-SIM illumination optical transfer function (MOTF) with the 4Pi-SIM illumination optical transfer function (OTF).

[0133] By using the structured light illumination module and the wide field illumination module of the embodiment of the present application, the point spread function (PSF) of 4Pi-SIM and the point spread function (PSF) of image interference microscopy can be collected. 2 MPSF). Spectra of different orders can be obtained by spectrum separation of 4Pi-SIM data. Spectral signals of different orders are then combined with image interference microscopy (I 2 By deconvolving the optical transfer function (OTF) of M), the illumination spectrum of each order can be obtained.

[0134] In order to generate the optical transfer function (OTF) of different optical path length differences (OPLD), we respectively 2 The optical transfer function (OTF) is phase modulated with the illumination spectrum and then convolved with the illumination spectrum. This generates a series of optical transfer functions (OTFs) with specific optical path differences (OPLDs) for image reconstruction.

[0135] The module for estimating structured light parameters is used to calculate the wave vector k0, the modulation depth m and the initial phase φ0 using a preset algorithm based on the spectrum information of the six-beam interference.

[0136] For the wave vector k0, its integer pixel position can be obtained by cross-correlating the 0-level spectrum and the 2-level spectrum. For sub-pixel estimation, the gradient descent method can be used to locate the position where the overlapping area of ​​the above two-level spectrum is the largest. Once the direction and size of k0 are determined, the 2-level spectrum can be moved to the corresponding position, and the pixel value statistics of the overlapping area of ​​the two-level spectrum are performed. The pixel value of the overlapping area is a complex number, whose modulus value represents the modulation depth, and its amplitude angle represents the initial phase φ0. In an embodiment of the present application, the modulus value is statistically averaged as the modulation depth m, and the amplitude angle is fitted by empirical mode decomposition (EMD), and the signal is recombined after discarding the three low-order components, and the maximum value of the filtering curve is taken as the initial phase φ0.

[0137] The optical path difference estimation module is used to determine the optical path difference according to the first original image and the second original image.

[0138] Due to factors such as air disturbance and thermal drift, it is very difficult to keep the optical path difference of the optical interferometer cavity zero during imaging. If the image is acquired under the condition that the optical path difference is not zero, but it is assumed to be zero when reconstructing the image, this will cause the problem of spectral phase mismatch, which appears as axial ringing artifacts in the reconstructed image. In order to solve this problem, such as Figure 9(a) to Figure 9(j) As shown in the figure, it is a schematic diagram of the optical path difference (OPLD) estimation principle of 4Pi-SIM. Figure 9(a) shows a schematic diagram of the excitation pupil function. Figure 9(b) shows a schematic diagram of the detection pupil function. Figure 9(c) shows a schematic diagram of the optical transfer function (OTF) of the excitation light system. Figure 9(d) shows a schematic diagram of the optical transfer function (OTF) of the detection light system. Figure 9(e) shows the optical transfer function (OTF) of the 4Pi-SIM system. Figure 9(f) to Figure 9(h) The 0th, 1st and 2nd order separated spectra are shown. Figure 9(i) to Figure 9(j) The six-beam interference spectrum and the phase distribution of the overlapping part are shown. Figure 9 (i) shows the spectrum without sample information (such as an ideal point light source located in the center), and Figure 9 (j) shows the spectrum with sample distribution.

[0139] The embodiment of the present application provides an optical path difference estimation method that can estimate and compensate for the influence of optical path difference even in complex biological samples. Figure 9(a) to Figure 9(j) In the figure, a and b represent the unknown phase values ​​generated by the excitation and detection optical path differences (OPLD), respectively. Since the transmission and reflection differ by half-wave loss π at the depolarizing beam splitter NPBS, the two sets of interference fringes are complementary (energy conservation). If the complementary patterns are added together, an image with the effect of fluorescence interference eliminated will be obtained, that is, an image with only six beams of interference. In the frequency domain, this is manifested as I generated by fluorescence interference. 2 The side lobes of M are eliminated, leaving only the central lobe representing the wide field, which greatly simplifies the spectrum information after Fourier transformation and enables the optical path difference parameters to be extracted.

[0140] It is worth noting that in Figure 1 In the imaging device of the structured light illumination microscope based on dual objective interference shown in the figure, when performing dual-color imaging, it is necessary to set the fifth dichroic mirror DM5 and filters F2 and F3 to obtain images of two different fluorescence channels corresponding to the first color fluorescence and the second color fluorescence; in the dual-color imaging mode, the optical path difference information of the system cannot be estimated. In monochrome imaging, it is not necessary to set the fifth dichroic mirror DM5, so that the two images formed are complementary, and the phases of the two images are φ and φ+π respectively, and the optical path difference of the system can be estimated based on the phases of the two images.

[0141] In the simplified spectrum, the overlapping parts of the 0th and 1st order (areas ③ and ⑥) contain information a related to the optical path difference (OPLD). However, since this information is mixed with the phase of the sample (i.e., phase s) and the initial phase of the illumination stripe (i.e., phase φ0), it cannot be extracted. However, once the spectrum moves to the position k0, the overlapping parts will share the same sample phase. Simply remove φ0 estimated from the overlapping areas ④ and ⑤, and the remaining phase represents the phase difference information of the excitation light. The fluorescence phase b can be obtained through the wavelength relationship b = aλ ex / λ em Calculate, where λ ex and λ em represent the excitation wavelength and emission wavelength, respectively.

[0142] The imaging module is used to adopt the Wiener filter reconstruction algorithm to construct a Wiener filter based on the system optical transfer function, structured light parameters and optical path difference information to obtain a reconstructed image. The imaging formula is as follows:

[0143]

[0144] Among them, S(K) represents the reconstructed spectrum, H d,nrepresents the complex optical transfer function (OTF) of the system, D represents the separation spectrum, m represents the modulation depth, and p n Represents wave vectors of various orders, w represents the Wiener parameter related to the image signal-to-noise ratio, d represents the direction, and n represents the order.

[0145] like Figure 10(a) to Figure 10(f) As shown, it is a schematic diagram of the reconstructed image according to the embodiment of the present application. In the embodiment of the present application, adding the optical path difference (OPLD) information to the reconstruction process can eliminate the ringing artifact. Figure 10 (a) is a schematic diagram of the projection of the fluorescent beads in the xy section. The upper and lower rows of dots in Figure 10 (b) respectively show the reconstruction results when OPLD = 0nm and the reconstruction results when OPLD = -160nm. Figure 10 (c) shows a schematic diagram of the intensity (Intensity) curve comparison of a single fluorescent bead. Figure 10 (d) shows a schematic diagram of the projection of fixed cells labeled with endoplasmic reticulum in the xy section. The upper and lower rows of dots in Figure 10 (e) respectively show the reconstruction results when OPLD = 0nm and the reconstruction results when OPLD = -180nm. Figure 10 (f) shows a schematic diagram of the intensity curve comparison of a single layer of endoplasmic reticulum signal.

[0146] In addition, the hardware design of the embodiment of the present application ensures that the illumination pattern of the six-beam interference remains stable for a long time, and the focus lock module ensures the precise alignment of the two objective lenses; in terms of software design, the reconstruction algorithm can accurately estimate the optical path difference information from the two complementary images and incorporate it into the reconstruction process, ultimately generating an image with minimal artifacts, which can achieve three-dimensional isotropic super-resolution images of fine subcellular structures in living cells.

[0147] like Fig.11 , which is a schematic flow chart of an imaging method of a structured light illumination microscope based on dual objective lens interference according to another embodiment of the present application. The imaging method 1100 of a structured light illumination microscope based on dual objective lens interference according to the method of the embodiment of the present application may include the following steps S1101, S1102, S1103, S1104, S1105, S1106, S1107, S1108, S1109 and S1110;

[0148] In step S1101, a first original image and a second original image are acquired;

[0149] In step S1102, preprocessing is performed on the first original image and the second original image to obtain spectrum information of six-beam interference;

[0150] In step S1103, a phase separation matrix is ​​constructed according to the first original image after background removal and edge softening operations have been completed;

[0151] In step S1104, a phase-separated frequency band is obtained according to the phase-separation matrix;

[0152] In step S1105, a structuring parameter is estimated according to the spectrum information of the six-beam interference;

[0153] In step S1106, a Wiener filter is generated based on the estimated parameters of the spectrum information of the six-beam interference and the optical transfer function of the matching dual-objective interference structured light illumination microscope;

[0154] In step S1107, shift the Wiener filter;

[0155] In step S1108, according to the separated frequency bands and the Wiener filter, each frequency band is multiplied by a corresponding Wiener filter;

[0156] In step S1109, the filter bands are moved and reassembled;

[0157] In step S1110, an image is reconstructed according to the recombined filtered spectrum.

[0158] In one embodiment of the present application, Fig.12 As shown, the method further includes step S1201, step S1202, step S1203, step S1204, step S1205 and step S1206:

[0159] In step S1201, a dual-objective lens interference structured light illumination point spread function and an image interference point spread function are collected;

[0160] In step S1202, an interference optical transfer function is generated according to the interference point spread function;

[0161] In step S1203, the dual-objective lens interference structured light illumination point spread function is subjected to spectrum separation to obtain spectrums of different orders; the dual-objective lens interference structured light illumination point spread function and the interference light point spread function are subjected to a deconvolution operation;

[0162] In step S1204, phase modulation is performed on the interference optical transfer function and the illumination spectrum of each order respectively, and then a convolution operation is performed on the modulated interference optical transfer function and the illumination spectrum of each order to generate a plurality of optical transfer function libraries corresponding to optical path differences;

[0163] In step S1205, the spectrum signals of different orders are convolved with the interference optical transfer function to obtain spectrums of various orders;

[0164] In step S1206, a corresponding optical transfer function is selected from the optical transfer function library according to the optical path difference estimated in the image reconstruction process, so as to generate the Wiener filter.

[0165] In one embodiment of the present application, Fig.13 As shown, preprocessing the first original image and the second original image includes step S1301 and step S1302:

[0166] In step S1301, the second original image is mirror-flipped and registered to obtain a third original image;

[0167] In step S1302, the first original image and the third original image are registered and added, and then Fourier transformed to obtain spectrum information of six-beam interference.

[0168] In yet another embodiment of the present application, Fig.14 As shown, the image reconstruction process 1400 may include:

[0169] In step S1401, an original image A is acquired;

[0170] In step S1402, an original image B is acquired;

[0171] In step S1403, mirror flipping and registration are performed on B to obtain the original image B';

[0172] In step S1404, Fourier transform is performed based on A and B' to obtain spectrum information of six-beam interference;

[0173] In step S1405, background removal and edge softening operations are performed on A;

[0174] In step S1406, a phase separation matrix is ​​constructed according to A which has completed background removal and edge softening operations;

[0175] In step S1407, phase separation frequency bands are obtained according to the phase separation matrix; then step S1411 is performed;

[0176] In step S1408, a structuring parameter is estimated according to the spectrum information of the six-beam interference;

[0177] In step S1409, a Wiener filter is generated based on the estimated parameters of the spectrum information of the six-beam interference and the matched double-objective interference structured light illumination optical transfer function (OTF);

[0178] In step S1410, the Wiener filter is moved according to the wave vector k0; then step S1411 is executed;

[0179] In step S1411, according to the separated frequency bands and the Wiener filter, each frequency band is multiplied by a corresponding Wiener filter;

[0180] In step S1412, the filter bands are transferred and reassembled;

[0181] In step S1413, an image is reconstructed according to the recombined filter bands;

[0182] In step S1414, collecting the dual objective lens interference structured light illumination point spread function;

[0183] In step S1415, collecting the interference point spread function;

[0184] In step S1416, a deconvolution operation is performed on the dual-objective lens interference structured light illumination point spread function and the interference light point spread function;

[0185] In step S1417, an interference optical transfer function is generated according to the interference point spread function;

[0186] In step S1418, spectrum separation is performed on the dual-objective lens interference structured light illumination point spread function to obtain spectrums of different orders;

[0187] In step S1419, the spectrum signals of different orders are convolved with the interference optical transfer function to obtain illumination spectra of each order;

[0188] In step S1420, an optical path difference (OPLD) required for the image reconstruction process is obtained;

[0189] In step S1421, the interference optical transfer function and the illumination spectra of each order are phase modulated respectively, and then the modulated interference optical transfer function and the illumination spectra of each order are convolved to generate an optical transfer function library (OTF Library) corresponding to multiple optical path differences.

[0190] In step S1422, a corresponding optical transfer function is selected from the optical transfer function library according to the optical path difference required in the image reconstruction process, so as to generate the Wiener filter.

[0191] In addition, according to an embodiment of the present application, a storage medium is also provided, on which program instructions are stored, and when the program instructions are run by a computer or a processor, the corresponding steps of the imaging method of the structured light illumination microscope based on dual objective lens interference of the embodiment of the present application are executed. The storage medium may include, for example, a memory card of a smart phone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media.

[0192] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application to this. Those of ordinary skill in the art may make various changes and modifications therein without departing from the scope and spirit of the present application. All these changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0193] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0194] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0195] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0196] Similarly, it should be understood that in order to streamline the present application and help understand one or more of the various inventive aspects, in the description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the method of the present application should not be interpreted as reflecting the following intention: the claimed application requires more features than the features clearly stated in each claim. More specifically, as reflected in the corresponding claims, the inventive point is that the corresponding technical problem can be solved with features less than all the features of a single disclosed embodiment. Therefore, the claims following the specific embodiment are hereby explicitly incorporated into the specific embodiment, wherein each claim itself serves as a separate embodiment of the present application.

[0197] It will be understood by those skilled in the art that, except for mutually exclusive features, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this specification may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0198] In addition, those skilled in the art will appreciate that, although some embodiments described herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0199] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some modules according to the embodiments of the present application. The application can also be implemented as a device program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0200] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0201] The above is only a specific implementation or description of a specific implementation of the present application, and the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. An imaging device of a structured light illumination microscope based on dual objective lens interference, characterized in that: The device comprises a structured light illumination system, an optical path difference adjustment module and a focus locking system; wherein, The structured light illumination system comprises a laser coupling module, a fast variable grating module, a polarization modulation module and a six-beam alignment module; the laser coupling module is used to emit a first laser beam; the fast variable grating module is arranged on the outgoing light path of the laser coupling module, and is used to receive the first laser beam, and modulate the first laser beam into a spatial structure beam and then emit it to the polarization modulation module; the polarization modulation module is used to compensate for the unequal phase delay between the s-polarized light and the p-polarized light of the spatial structure beam; the six-beam alignment module divides the processed spatial structure beam into two symmetrical spatial structure beams, which converge on a focal plane through two symmetrical microscope objective lenses, and interfere at the focal plane to form a three-dimensional structured light illumination pattern; the fluorescence signal generated by the sample excited by the illumination pattern is collected by the microscope objective lens, and fluorescence interference occurs after convergence, and the final image is collected by the second camera, and fluorescence interference occurs; The optical path difference adjustment module is arranged on the optical path of the six-beam light alignment module, and is used to adjust the optical path difference of the imaging device to be smaller than the coherence length of the fluorescence; The focus locking system includes a second laser emitter; the second laser emitter is used to emit a second laser beam; the second laser beam is focused to the first camera after passing through a lens group, a second dichroic mirror, a third dichroic mirror and a fourth dichroic mirror, so as to monitor the position and shape of the processed spatial structure beam on the first camera, so that the system control module adjusts the relative position of the microscope objective lens according to the position and the shape to ensure that the sample is always in focus.

2. The device according to claim 1, characterized in that The optical path difference adjustment module includes an optical path difference coarse adjustment module and an optical path difference fine adjustment module; wherein, The optical path difference coarse adjustment module comprises a long-stroke linear translation stage and a piezoelectric linear actuator, and the optical path difference coarse adjustment module is used to adjust the optical path difference to a first preset range when initially calibrating the imaging device; The optical path difference fine adjustment module comprises a linear piezoelectric platform for adjusting the optical path difference to a second preset range; The first preset range is a fluorescence interference range of the micrometer order, and the second preset range is a position where the optical path difference of the nanometer order is zero.

3. The device according to claim 1, characterized in that The laser coupling module includes a first laser emitter, an acousto-optic tunable filter, a mechanical oscillator, a multimode optical fiber, a collimating lens and a lens group; the first laser emitter is used to generate the first laser beam; after being filtered by the acousto-optic tunable filter, the first laser beam is coupled into the multimode optical fiber, and the multimode optical fiber is wound on the mechanical oscillator, wherein the mechanical oscillator is used to fill the speckle pattern of the multimode optical fiber to obtain uniform partially coherent light illumination; the first laser beam via the multimode optical fiber reaches the collimating lens for collimation, and the first laser beam collimated by the collimating lens is emitted to the fast variable grating module through the lens group.

4. The device according to claim 3, characterized in that The fast variable grating module includes a ferroelectric liquid crystal fast spatial light modulator, a polarization beam splitter prism and a first half-wave plate; the polarization beam splitter prism receives the first laser beam and injects the first laser beam into the ferroelectric liquid crystal fast spatial light modulator through the first half-wave plate, wherein the ferroelectric liquid crystal fast spatial light modulator is used to modulate the first laser beam into the spatial structure beam; the spatial structure beam enters the polarization modulation module after being split by the polarization beam splitter prism.

5. The device according to claim 4, characterized in that The polarization modulation module includes a spatial filter, a second half-wave plate, a liquid crystal phase retarder, two quarter-wave plates and a depolarizing beam splitter prism; The spatial structure light beam passes through the spatial filter and is incident on the second half-wave plate, wherein the second half-wave plate is used to rotate the spatial structure light by 45° relative to the s-polarization; the spatial structure light emitted by the second half-wave plate is incident on the liquid crystal phase retarder, wherein the fast axis of the liquid crystal phase retarder is aligned with the s-polarization of the microscope objective lens; the liquid crystal phase retarder performs phase adjustment on the spatial structure light beam; The spatial structure light beam emitted through the liquid crystal phase retarder is divided into two upper and lower spatial structure light beams with the same phase delay after being reflected by the depolarization beam splitter prism, and the upper and lower two spatial structure light beams form a symmetrical optical interference cavity; Wherein, the depolarizing beam splitter prism is used to keep the polarization states of the reflected light and the transmitted light the same; The two quarter wave plates are respectively arranged before the microscope objective lens and after all the reflectors, and the fast axes of the two quarter wave plates are set to 45° relative to the s polarization.

6. The device according to claim 5, characterized in that The six-beam alignment module includes the microscope objective lens, which includes an upper microscope objective lens and a lower microscope objective lens, wherein the lower microscope objective lens is installed on an XY piezoelectric translation stage, and the upper microscope objective lens is installed on a Z-axis piezoelectric translation stage; when acquiring an interference image, fluorescent beads in the sample are used as target marks to align the lateral direction of the upper microscope objective lens and the axial direction of the lower microscope objective lens.

7. The device according to claim 1, characterized in that The device also includes a wide field lighting module; The wide-field illumination module comprises a third laser emitter, which is used to emit a third laser beam. The third laser beam is aligned with the first laser beam in optical path and is focused on a focal plane, and a dichroic mirror is arranged between the third laser beam and the first laser beam to provide single-sided wide-field illumination for the focal plane.

8. The device according to claim 1, characterized in that The device also includes a dual-color detection module; wherein, The dual-color detection module includes a fifth dichroic mirror; the fifth dichroic mirror is arranged on the light path emitted by the depolarizing beam splitter prism, wherein the light path includes an upper light path passing through a lens, a reflector and another lens; the fifth dichroic mirror separates the upper light path into fluorescence detection light paths of different spectra respectively; the dual-color detection module also includes a filter group and a second camera, and filters of different colors are respectively arranged on the two fluorescence detection light paths to filter the first color fluorescence and the second color fluorescence; the second camera collects the first color fluorescence through the first channel, and the second camera collects the second color fluorescence through the second channel.

9. The device according to claim 1, characterized in that The device also includes a sample stage module; wherein, The sample stage module, the sample stage and the three-dimensional displacement stage for driving the sample stage, wherein the sample stage is used to carry the sample sealed between two high-precision wave plates; the three-dimensional displacement stage includes an XY piezoelectric displacement stage and a Z-axis piezoelectric displacement stage.

10. The device according to claim 1, characterized in that The device further comprises a system control module, and the system control module comprises a controller; wherein, The controller is used to adjust the voltage of the liquid crystal phase retarder according to a preset time interval, generate a trigger camera exposure signal and an adjustable filter illumination modulation signal, so as to collect an interference image of the sample at a current focal plane; and move the Z-axis piezoelectric translation stage to different depths of the sample to the focal plane according to a preset step length, and collect interference images according to the above steps until the entire sample is imaged; The controller is also used to trigger the structured light illumination module and the first camera.

11. The device according to claim 10, characterized in that The system control module also includes an image reconstruction module; wherein, The image reconstruction module is used to reconstruct the original image based on the optical path difference containing optical transfer function information according to the structured light parameters and the Wiener filter reconstruction algorithm to obtain a reconstructed super-resolution three-dimensional image.

12. The device according to claim 11, characterized in that The image reconstruction module includes a raw data preprocessing module, an optical transfer function building module, a structured light parameter estimation module, an optical path difference estimation module and an imaging module; wherein, The raw data preprocessing module is used to construct the optical transfer function module and the imaging module, and the preprocessing includes removing the background and softening the edge, and Fourier transforming according to the raw data; the preprocessing for estimating the optical path difference module includes the registration operation of the first raw image and the second raw image to eliminate fluorescence interference and obtain the spectrum information of the six-beam interference; The optical transfer function construction module is used to obtain frequency spectra of different orders according to the dual-objective interference structured light illumination point spread function and the image interference point spread function, perform a deconvolution operation on the frequency spectrum signals of different orders and the interference optical transfer function to obtain illumination spectra of each order, respectively perform phase modulation on the interference optical transfer function and the illumination spectra of each order, and then perform a convolution operation on the modulated interference optical transfer function and the illumination spectra of each order to generate a plurality of optical transfer function libraries corresponding to optical path differences; The module for estimating structured light parameters is used to calculate the wave vector, modulation depth and initial phase using a preset algorithm based on the spectrum information of the six-beam interference; The optical path difference estimation module is used to determine the optical path difference according to the first original image and the second original image; The imaging module is used to adopt a Wiener filter reconstruction algorithm to construct a Wiener filter based on the system optical transfer function, structured light parameters and optical path difference information to obtain a reconstructed image.

13. The device according to any one of claims 1 to 12, characterized in that Six-beam structured light illumination imaging is performed using multiple lasers of different wavelengths; an additional laser is used for image interference microscopy imaging; another additional laser is used to monitor and correct the drift of the objective lens; and the samples include fluorescent beads, fixed cells or living cells.

14. An imaging method of a structured light illumination microscope based on dual objective lens interference, the method being applied to the imaging device of a structured light illumination microscope based on dual objective lens interference as claimed in claims 1 to 13, characterized in that: The method comprises: Acquire a first original image and a second original image; Preprocessing the first original image and the second original image to obtain spectrum information of six-beam interference; constructing a phase separation matrix according to the first original image after background removal and edge softening operations; Obtaining a phase-separated frequency band according to the phase-separation matrix; Estimating parameters according to the spectrum information of the six-beam interference; Estimating parameters according to the spectrum information of the six-beam interference and the optical transfer function of the matching dual-objective interference structured light illumination microscope to generate a Wiener filter; moving the Wiener filter; According to the separated frequency bands and the Wiener filter, multiplying each frequency band with a corresponding Wiener filter; shifting and recombining the filtered spectrum; An image is reconstructed based on the recombined filtered spectrum.

15. The method according to claim 14, characterized in that The method further comprises: Collect dual-objective interference structured light illumination point spread function and image interference point spread function; generating an interferometric optical transfer function according to the interferometric point spread function; Performing spectrum separation on the dual-objective lens interference structured light illumination point spread function to obtain spectrums of different orders; performing a deconvolution operation on the dual-objective lens interference structured light illumination point spread function and the interference light point spread function; Phase modulating the interference optical transfer function and the illumination spectrum of each order respectively, and then performing a convolution operation on the modulated interference optical transfer function and the illumination spectrum of each order to generate an optical transfer function library corresponding to a specific optical path difference; Convolution operation is performed on the spectrum signals of different orders and the interference optical transfer function to obtain spectrums of various orders; A corresponding optical transfer function is selected from the optical transfer function library according to the optical path difference estimated in the image reconstruction process to generate the Wiener filter.

16. The method according to claim 14, characterized in that Preprocessing the first original image and the second original image includes: Performing mirror flipping and registration on the second original image to obtain a third original image; The first original image and the third original image are registered and added, and then Fourier transformed to obtain spectrum information of six-beam interference.

17. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, enables the processor to execute the imaging method of a structured light illumination microscope based on dual objective lens interference as described in any one of claims 14 to 16.

Images