A turntable-based dual confocal image scanning microscope and imaging method

Abstract

The present invention discloses a dual confocal image scanning microscope based on a turntable and an imaging method, which relates to the fields of optical elements, systems and instruments, and imaging technologies. The present invention includes: a signal control component, and an imaging light source component, an optical fiber transmission component, a light source collimation component, a structured light generation component, a turntable component, a microscopic magnification component and a camera detection component arranged along the optical path; the structured light generation module realizes structured light illumination of the sample surface by loading a specific mask sequence to achieve super-resolution imaging. The pinholes on the turntable component are arranged equidistantly according to the Archimedean spiral, and the turntable component rotates at high speed to achieve multi-point parallel scanning and de-scanning; the detection module uses a area array detector for detection to achieve a resolution improvement of twice the diffraction limit; the dynamic pinhole array-pixel reassignment method performs super-resolution reconstruction based on the principle of the image scanning microscope system, and can effectively correct Stokes shift, optical aberration and other non-ideal conditions.

Description

Technical Field

[0001] The present invention relates to the fields of optical elements, systems, instruments and imaging technologies, and particularly relates to a dual confocal image scanning microscope and an imaging method based on a turntable. Background Art

[0002] Currently, optical microscopes have become indispensable research tools in the biomedical field. Among them, super-resolution optical microscopy technology can break through the diffraction limit and achieve clear imaging of biological structures at the subcellular scale, and has been widely used. However, due to the limited penetration depth of light waves in biological tissues and the interference of out-of-focus signals, scattering signals, etc. during the imaging process, there are problems such as poor resolution and low fidelity in deep tissue super-resolution imaging. The annular loss beam of the stimulated emission depletion microscope is prone to distortion when penetrating tissues, resulting in the inability to effectively erase the periphery of the point spread function, leading to a serious degradation of resolution; the fringe pattern of the structured illumination microscope is easily submerged by background signals, resulting in inaccurate estimation of algorithm parameters and obvious artifacts in the finally reconstructed image; the single molecule localization microscope is also affected by background fluorescence and light scattering in tissue samples, reducing the positioning accuracy of excitation points and thus distorting the image structure.

[0003] Confocal microscopes have become the preferred technology for biologists in fluorescence imaging due to their excellent optical sectioning ability and versatility. However, traditional single-point scanning confocal microscopes have certain limitations in terms of imaging speed, photobleaching and phototoxicity during live cell imaging. Currently, there are also multi-point scanning confocal microscopy configurations, such as the spinning disk confocal microscopy technology. However, as a traditional microscopy imaging technology, its resolution is limited by the diffraction limit and cannot meet the high-resolution observation requirements of fine structures. Summary of the Invention

[0004] To solve the above problems existing in the prior art, the present invention provides a multi-point confocal image scanning microscope and an imaging method.

[0005] To achieve the above object, the present invention provides the following solutions.

[0006] A dual confocal image scanning microscope based on a turntable, comprising: a signal control component, and an imaging light source component, an optical fiber transmission component, a light source collimation component, a structured light generation component, a turntable component, a microscopic magnification component and a camera detection component arranged along the optical path;

[0007] The imaging light source assembly is used to output laser light; the optical fiber transmission assembly is used to homogenize the laser light and input the homogenized laser light into the light source collimation assembly; the light source collimation assembly is used to collimate and output the laser light output by the optical fiber transmission assembly; the structured light generation assembly is used to generate and scan structured light based on the laser light output by the light source collimation assembly; a pinhole array distributed along multiple clusters of concentric Archimedean spirals is provided on the turntable assembly; multiple pinholes in the pinhole array are arranged at equal intervals along multiple clusters of concentric Archimedean spirals; the rotation speed of the turntable assembly is adjusted by a three-phase brushless DC motor controller in the signal control assembly; the microscopic magnification assembly is used to achieve the magnification nominal of the microscopic objective and the clamping of the sample, reflect the structured light as the excitation light to the rotating turntable assembly, irradiate the sample with the excitation light passing through the pinholes to generate fluorescent emission light, and transmit the fluorescent emission light passing through the pinholes to the camera detection assembly; the camera detection assembly is used to detect and output the fluorescent signal;

[0008] The signal control assembly is used to control the imaging light source assembly, the structured light generation assembly, the turntable assembly and the camera detection assembly to work synchronously; the signal control assembly is also used to obtain the detection result and use the detection result as the original image; based on the original image, imaging reconstruction is realized by using the dynamic pinhole array-pixel reassignment method;

[0009] The dynamic pinhole array-pixel reassignment method includes: performing Fourier transform on each pixel of the three-dimensional image stack along the scanning direction, and performing DC suppression preprocessing on the original image; performing Fourier transform on the DC-suppressed preprocessed original image and superimposing the spectra, and using the spectral grid basis vector formula to determine the basis vectors in the spectral grid; generating a lattice based on the basis vectors in the spectral grid; using the spatial local maximum points of the light spots as a reference, calculating the offset vector between the lattice point spacing and the excitation light axis array point spacing; positioning the excitation light axis based on the offset vector; setting a virtual detection array centered on the excitation light axis, and synchronously moving the virtual detection array with the excitation dot matrix scanning to complete the virtual detection array sampling, obtaining a plurality of confocal sub-images; performing apodization processing on the plurality of confocal sub-images by multiplying by a Hanning window, and convolving the plurality of confocal sub-images with a Gaussian kernel to remove the noise of the plurality of confocal sub-images, obtaining a plurality of confocal denoised sub-images; aligning and superimposing the plurality of confocal denoised sub-images with the central confocal denoised sub-image to obtain an initial reconstructed image; performing upsampling processing on the initial reconstructed image according to the size of the initial image to obtain dual confocal image scanning microscopy imaging.

[0010] Optionally, the imaging light source assembly includes a high-power single-mode laser, a high-power multimode laser or a high-power LED;

[0011] In multi-color imaging, the imaging light source group is combined by using a multi-fiber or combined by a dichroic mirror.

[0012] Optionally, the light source collimation component is an aspherical lens, a 90° off-axis parabolic mirror, an aspherical mirror with adjustable focal length, an achromatic doublet lens, or an air-spaced doublet lens.

[0013] Optionally, the structured light generation component includes: a one-dimensional galvanometer, a digital micromirror device, a first lens, and a second lens;

[0014] The one-dimensional galvanometer is used to control the angle at which the laser output by the light source collimation component is incident on the target surface of the digital micromirror device;

[0015] The digital micromirror device is used to generate and scan structured light based on the light beam output by the one-dimensional galvanometer, and output the one-dimensional galvanometer to the first lens;

[0016] The second lens is used to transmit the structured light output by the first lens to the microscopic magnification component.

[0017] Optionally, the microscopic magnification component includes: a dichroic mirror, a tube lens, a microscopic objective lens, and a sample clamping component arranged along the optical path;

[0018] The sample clamping component is used to clamp the sample;

[0019] The dichroic mirror is used to reflect the structured light as excitation light to the rotating turntable component;

[0020] The tube lens is used to receive the excitation light passing through the pinhole and output the excitation light passing through the pinhole to the microscopic objective lens;

[0021] The microscopic objective lens is used to receive the excitation light passing through the pinhole output by the tube lens, output the excitation light passing through the pinhole to the sample, and receive the fluorescence emission light formed by the sample irradiated by the excitation light passing through the pinhole, and output the fluorescence emission light to the tube lens;

[0022] The tube lens is also used to receive the fluorescence emission light output by the tube lens and output the fluorescence emission light to the rotating turntable component;

[0023] The dichroic mirror is also used to transmit the fluorescence emission light passing through the pinhole to the camera detection component.

[0024] Optionally, the camera detection component includes: a relay lens and a camera arranged along the optical path;

[0025] The relay lens is used to achieve conjugate imaging;

[0026] The camera is used to detect and output the fluorescence signal.

[0027] Optionally, the relay lens can be replaced with a single-lens reflex lens; the single-lens reflex lens is used to achieve equal-proportion conjugate imaging with a large field of view.

[0028] An imaging method using a dual confocal image scanning microscope based on a turntable, comprising:

[0029] Obtaining a detection result using the described dual confocal image scanning microscope based on a turntable, and using the detection result as the original image;

[0030] Based on the original image, implementing imaging reconstruction using the dynamic pinhole array - pixel reassignment method.

[0031] Optionally, the implementing imaging reconstruction based on the original image using the dynamic pinhole array - pixel reassignment method includes:

[0032] Performing Fourier transform on each pixel of the three-dimensional image stack along the scanning direction, and performing direct-current suppression preprocessing on the original image;

[0033] Performing Fourier transform on the original image after direct-current suppression preprocessing and superimposing the spectra, and determining the basis vectors in the spectral grid using the spectral grid basis vector formula;

[0034] Generating a lattice based on the basis vectors in the spectral grid;

[0035] Using the spatial local maximum point of the light spot as a reference, calculating the offset vector between the lattice point spacing and the excitation light axis array point spacing;

[0036] Locating the excitation light axis based on the offset vector;

[0037] Setting a virtual detection array centered on the excitation light axis, and synchronously moving the virtual detection array with the excitation dot scanning to complete virtual detection array sampling, obtaining a plurality of confocal sub-images;

[0038] Performing apodization processing on the plurality of confocal sub-images by multiplying by a Hann window, and convolving the plurality of confocal sub-images with a Gaussian kernel to remove the noise of the plurality of confocal sub-images, obtaining a plurality of confocal denoised sub-images;

[0039] Aligning and superimposing the plurality of confocal denoised sub-images with the central confocal denoised sub-image, obtaining an initial reconstructed image;

[0040] Performing upsampling processing on the initial reconstructed image according to the size of the initial image, obtaining dual confocal image scanning microscopy imaging.

[0041] Optionally, the imaging method using the dual confocal image scanning microscope based on a turntable further includes:

[0042] Load a stripe mask for projection SIM imaging to achieve seamless switching between multiple imaging modalities; perform phase shifting on the stripes during projection SIM imaging to achieve uniform illumination of the field of view;

[0043] Reconstruct the SIM image based on Fourier domain operations;

[0044] Notch the high-order points that cause artifacts in the reconstructed spectrum to obtain the SIM reconstruction result.

[0045] In the present invention, a signal control component is provided, as well as an imaging light source component, an optical fiber transmission component, a light source collimation component, a structured light generation component, a turntable component, a microscopic magnification component, and a camera detection component arranged along the optical path; the pinhole on the high-speed rotating turntable serves as both the excitation pinhole during the scanning process and the detection pinhole during the de-scanning process, which constitutes the first confocal effect; in addition, a square hole array mask is loaded on the Digital Micromirror Devices (DMD) to achieve multi-point excitation of the sample, and at the same time, a area array detector is used instead of single-point detection for detection. Each pixel of the area array detector can be regarded as a point detector with an infinitely small pinhole, which constitutes the second confocal effect. This dual confocal configuration can effectively reduce the defocus background interference and improve the system resolution at the same time. Compared with the background removal algorithm, the turntable removes defocus by physical means, which not only retains the original intensity distribution but also retains the linear relationship of the focus, not only providing clear confocal images but also providing high-quality original data for subsequent super-resolution reconstruction.

[0046] The dynamic pinhole array-pixel reassignment (DPA-PR) method disclosed in the present invention performs super-resolution reconstruction based on the principle of an image scanning microscopy (ISM). Compared with traditional pixel reassignment algorithms, it can effectively correct Stokes shift, optical aberration, and other non-ideal conditions. Description of the Drawings

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0048] Figure 1 It is a schematic diagram of the relationship between multiple modalities and the DMD mask in an embodiment of the present application.

[0049] Figure 2Optical path diagram of a dual confocal image scanning microscope based on a turntable in an embodiment of the present application.

[0050] Figure 3 Schematic diagram of the relationship between the turntable rotation speed and the number of spiral clusters N in an embodiment of the present application.

[0051] Figure 4 Schematic diagram of the geometric relationship of the digital micromirror device and the quantitative relationship of key physical quantities in an embodiment of the present application.

[0052] Figure 5 Physical de - focusing effect display diagram of the turntable in an embodiment of the present application.

[0053] Figure 6 Influence diagram of the introduction of the turntable on multi - focus excitation in an embodiment of the present application.

[0054] Figure 7 ISM super - resolution reconstruction result and resolution improvement comparison diagram in an embodiment of the present application.

[0055] Figure 8 Algorithm fidelity comparison result diagram in an embodiment of the present application.

[0056] Figure 9 Influence of the introduction of the turntable on fringe excitation and SIM reconstruction result diagram in an embodiment of the present application.

[0057] Figure 10 Comparison diagram of confocal and ISM super - resolution multi - color imaging results in an embodiment of the present application.

[0058] Symbol description:

[0059] Imaging light source assembly - 10; optical fiber transmission assembly - 20; light source collimation assembly - 30; structured light generation assembly - 40; turntable assembly - 50; microscopic magnification assembly - 60; camera detection assembly - 70; signal control assembly - 80; one - dimensional galvanometer - 41; digital micromirror device - 42; first lens - 43; second lens - 44; dichroic mirror - 61; tube lens - 62; microscopic objective - 63; sample - 64; relay lens - 71; camera - 72. Detailed implementation manners

[0060] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts belong to the scope of protection of the present invention.

[0061] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] As Figure 2 shown, a dual confocal image scanning microscope based on a turntable is provided, including: a signal control component 80, and an imaging light source component 10, an optical fiber transmission component 20, a light source collimation component 30, a structured light generation component 40, a turntable component 50, a microscopic magnification component, and a camera detection component 70 arranged along the optical path.

[0063] The imaging light source component 10 is used to excite a fluorescent sample (output laser).

[0064] The optical fiber transmission component 20 is used to transmit the laser to an optical imaging device and achieve homogenization (perform homogenization processing on the laser and input the homogenized laser into the light source collimation component).

[0065] The light source collimation component 30 is used to collimate and output the laser transmitted by the optical fiber to achieve large-spot uniform illumination;

[0066] The structured light generation component 40 is used to generate and scan structured light based on the laser output by the light source collimation component 30, and includes a one-dimensional galvanometer 41, a digital micromirror device 42, a first lens 43, a second lens 44. The single collimated spot output by the light source collimation component 30 adjusts the incident angle through the one-dimensional galvanometer 41, and structured light illumination is generated through the digital micromirror device 42. The one-dimensional galvanometer 41 is used to control the angle of the laser output by the light source collimation component 30 incident on the target surface of the digital micromirror device; the digital micromirror device is used to generate and scan structured light based on the beam output by the one-dimensional galvanometer and output the one-dimensional galvanometer to the first lens; the second lens is used to transmit the structured light output by the first lens to the microscopic magnification component.

[0067] The turntable component 50 is used to rotate the pinholes arranged in an Archimedean spiral at a high speed and stably. A pinhole array distributed along multiple clusters of concentric Archimedean spirals is provided on the turntable component; multiple pinholes in the pinhole array are arranged at equal intervals along multiple clusters of concentric Archimedean spirals; the rotation speed of the turntable component is adjusted by a three-phase brushless DC motor controller in the signal control component. The equally spaced pinholes on the turntable are distributed along multiple clusters of Archimedean spirals. In addition, it can also be designed in a stripe pattern, which can also form uniform illumination for the field of view. Compared with the pinholes, its sectioning effect is reduced, but the light transmittance is increased.

[0068] The microscopic magnification component 60 is used to achieve the magnification specified by the microscopic objective and the clamping of the sample, reflect the structured light as the excitation light to the rotating turntable component, irradiate the sample with the excitation light passing through the pinhole to generate fluorescent emission light, and transmit the fluorescent emission light passing through the pinhole to the camera detection component; the camera detection component is used to detect and output the fluorescent signal. The microscopic magnification component 60 includes a dichroic mirror 61, a tube lens 62, a microscopic objective 63 and a sample 64, and is used to achieve the magnification specified by the microscopic objective and the clamping of the sample.

[0069] The camera detection component 70 includes a relay lens 71 and a camera 72, and is used to detect and output the fluorescent signal. The fluorescent signal generated by the above excitation enters the camera detection component 70 after passing through the turntable component 50, the microscopic objective 63, the tube lens 62 and the dichroic mirror 61 in the microscopic magnification component 60.

[0070] The signal control component 80 is used to synchronize the imaging light source component 10, the structured light generation component 40, the turntable component 50 and the camera detection component 70; the signal control component is also used to obtain the detection result and use the detection result as the original image; based on the original image, use the dynamic pinhole array - pixel reassignment method to achieve imaging reconstruction; the dynamic pinhole array - pixel reassignment method includes: performing Fourier transform on each pixel of the three-dimensional image stack along the scanning direction, and performing DC suppression preprocessing on the original image; performing Fourier transform on the DC-suppressed preprocessed original image and superimposing the spectra, and using the spectral grid basis vector formula to determine the basis vectors in the spectral grid; generating a lattice based on the basis vectors in the spectral grid; using the spatial local maximum points of the light spot as a reference, calculating the offset vector between the lattice point spacing and the excitation light axis array point spacing; positioning the excitation light axis based on the offset vector; setting a virtual detection array centered on the excitation light axis, and synchronously moving the virtual detection array with the excitation dot scan to complete virtual detection array sampling to obtain multiple confocal sub-images; performing apodization processing on the multiple confocal sub-images by multiplying by a Hanning window, and performing convolution on the multiple confocal sub-images with a Gaussian kernel to remove the noise of the multiple confocal sub-images to obtain multiple confocal denoised sub-images; aligning and superimposing the multiple confocal denoised sub-images with the central confocal denoised sub-image to obtain an initial reconstruction image; performing upsampling processing on the initial reconstruction image according to the size of the initial image to obtain a dual confocal image scanning microscopy.

[0071] Further, the imaging light source component 10 may include a high-power single-mode laser, a high-power multi-mode laser, a high-power LED, to generate a high-power illumination light source. In multi-color imaging, beam combination is performed using a multi-in-one optical fiber or through a dichroic mirror.

[0072] Furthermore, the optical fiber transmission component 20 generally uses multimode optical fibers or liquid optical waveguides for transmission, and single-mode optical fibers are also tried. The core diameter of multimode optical fibers generally ranges from 50 μm to 400 μm, and larger or smaller core diameters are also possible. The liquid optical waveguide generally uses a 3-mm core for output. The liquid optical waveguide and multimode optical fiber have advantages in terms of transmission efficiency, and the coupling efficiency can reach 90%.

[0073] Furthermore, the light source collimation component 30 uses an aspherical lens, and the light output from the multimode optical fiber or liquid optical waveguide is collimated by the aspherical lens. It can also be collimated by a 90° off-axis parabolic mirror, an aspherical mirror with adjustable focal length, an achromatic doublet lens, an air-spaced doublet lens, etc. The light output from single-mode and multimode optical fibers can also be collimated in the above manner.

[0074] Furthermore, the microscopic magnification component includes: a dichroic mirror, a tube lens, a microscopic objective lens, and a sample clamping component arranged along the optical path; the sample clamping component is used to clamp the sample; the dichroic mirror is used to reflect the structured light as the excitation light to the rotating turntable component; the tube lens is used to receive the excitation light passing through the pinhole and output the excitation light passing through the pinhole to the microscopic objective lens; the microscopic objective lens is used to receive the excitation light passing through the pinhole output by the tube lens, output the excitation light passing through the pinhole to the sample, and receive the fluorescence emission light formed by the sample irradiated by the excitation light passing through the pinhole, and output the fluorescence emission light to the tube lens; the tube lens is also used to receive the fluorescence emission light output by the tube lens and output the fluorescence emission light to the rotating turntable component; the dichroic mirror is also used to transmit the fluorescence emission light passing through the pinhole to the camera detection component. In the structured light generation component 40, a one-dimensional galvanometer 41 is used to control the angle of the light beam incident on the target surface of the digital micromirror device 42 to ensure that the direction of the main diffraction maximum of a specific order remains unchanged under different wavelength imaging, and an analog signal generated by the signal control component controls the movement of the galvanometer.

[0075] Furthermore, the digital micromirror device 42 is used to generate and scan the structured light. Compared with generating and scanning multiple foci using a pinhole array and a galvanometer, the digital micromirror device provides a more stable step. The mask for the digital micromirror device used to generate multi-focal illumination is designed as a periodic lattice, and each lattice element consists of a square aperture formed by 4×4 on-state digital micromirror device pixels. The size of each digital micromirror device pixel is 5.4 μm, and under a 100× objective lens, an excitation spot of approximately 220 nm can be formed on the sample plane. The multi-focal illumination pattern moves two DMD pixels each time, which is equivalent to a 108-nm step length on the sample plane, achieving a sampling rate twice that of the diffraction limit. The interval between the lattice elements is controlled to be 12 pixels, that is, the side length of the square and the distance between each other are 4:12, further reducing the number of original frames required for image scanning microscopic reconstruction.

[0076] Furthermore, the turntable assembly 50 achieves rapid physical defocus removal, improving the imaging contrast. The spinning disk confocal microscope generates an array of excitation point sources through a rotating Archimedean spiral pinhole array placed in the conjugate plane of the microscope sample, and at the same time acts as its conjugate pinhole to remove the defocus signal in the sample. Each pinhole in the array scans to produce an illumination arc (curve) on the sample, and the fluorescence generated by these scans is collected and naturally superimposed during the camera exposure time. When the turntable motor is driven at a high enough rate to repeatedly scan the sample, a real-time confocal image is generated. The pinhole array disk is etched with pinholes arranged in an Archimedean spiral on chrome-plated fused silica glass, and a circular hole is laser-cut in the center of the disk, and a motor with a very high and stable rotation speed is used to drive the rotation of the pinhole disk.

[0077] Furthermore, the rotation speed of the turntable assembly 50 is adjusted by a three-phase brushless DC motor controller.

[0078] Furthermore, the dichroic mirror 61 is used to separate the excitation light and the emission light, reflect the aforementioned excitation light into the tube lens 62, and transmit the aforementioned fluorescence emission light into the camera detection assembly 70.

[0079] Furthermore, the camera detection assembly includes: a relay lens 71 and a camera 72 arranged along the optical path; the relay lens is used to achieve conjugate imaging; the camera is used to detect and output the fluorescence signal. The relay lens 71 in the camera detection assembly 70 uses a single-lens reflex lens to collect the returned fluorescence, and the single-lens reflex lens can achieve large-field-of-view equal-proportion conjugate imaging. A scientific-grade camera with a large target surface can detect the returned fluorescence with high sensitivity. Other relay lens groups can also be used to achieve conjugate imaging, and ordinary cameras can also be used to achieve fluorescence detection.

[0080] Furthermore, the signal control component 80 can output 4 analog signals and 2 digital signals. One analog signal controls the galvanometer for one-dimensional scanning, and the other three analog signals respectively control the DMD mask switching, the camera in the camera detection assembly, and the nano-positioning piezoelectric sample scanning stage in the microscope main body assembly. The DMD operates in the "Pattern On-The-Fly" mode, the exposure time of the DMD is synchronized with the exposure time of the camera, and the dark time of the DMD matches the readout time of the camera. The two digital signals are used as address codes to achieve arbitrary switching of the light source through a demultiplexer, and cooperate with the DMD exposure time to achieve the synchronization of the light source. Multimodal and digital micromirror device masks are as Figure 1 shown.

[0081] Furthermore, the present invention also provides an imaging method using a dual confocal image scanning microscope based on a turntable, including: step 100, step 101, and step 102.

[0082] Among them, step 100: Use a dual confocal image scanning microscope based on a turntable to obtain a detection result, and use the detection result as the original image. Before using the dual confocal image scanning microscope based on the turntable to obtain the detection result, the design of the turntable, the self-synchronization with the camera exposure, and the calibration of the DMD need to be carried out; among them, the calibration of the DMD includes: the calibration of the DMD spatial coordinates and the calibration of the DMD diffraction effect.

[0083] 1. Design of the turntable and self-synchronization with the camera exposure.

[0084] The pinhole on the high-speed rotating turntable acts as both the excitation pinhole during the scanning process and the detection pinhole during the de-scanning process. The high-speed rotation realizes multi-point parallel scanning and de-scanning, achieving the first confocal effect. The present invention uses the following method for design to achieve uniform scanning illumination of the field of view and is more convenient to synchronize with the camera exposure time.

[0085] The pinhole array on the turntable surface is arranged along a concentric Archimedean spiral. Taking the center of the turntable as the pole to establish a coordinate system, the Archimedean spiral equation is as follows:

[0086] .

[0087] In the formula, is the radial position and angular position of a point in the polar coordinate system, is the concentric Archimedean spiral cluster number; is the number of concentric Archimedean spiral clusters (an integer greater than or equal to 1), represents the starting position of the turntable, controls the distance of the radial position step when the spiral rotates one circle. In order to ensure that the turntable rotation realizes uniform illumination of the field of view, it is necessary to ensure that the arc length between adjacent pinholes is equal and equal to . Taking a certain one of the N concentric Archimedean spirals as the research object, its equation is:

[0088] .

[0089] In the formula, , . Let the angular coordinates of the kth point on this spiral be , and its arc length from the starting point of the spiral (the point at ) is , and its calculation formula is as follows:

[0090] .

[0091] According to the adjacent pinhole spacing equal to it can be obtained that , and then The specific value of to obtain the corresponding radial position accordingly.

[0092] The spiral of the turntable consists of N sub-spirals, and each sub-spiral is exactly the same, only with a rotation between their spatial positions. When the turntable rotates, each sub-spiral is responsible for the of the field of view, that is, the field of view can be evenly illuminated times per rotation ( Figure 3 ). The camera exposure event needs to be synchronized with the rotation speed of the turntable, that is, the successive exposure events need to be equal to or an integer multiple of the time required for the turntable to evenly cover the field of view once. Specifically, for the rotation speed n (unit: round per minute, rpm) of the turntable, the exposure time t (unit: s), and N, the following relationship needs to be satisfied:

[0093] .

[0094] mod represents the modulo operation. It can be found that with a constant rotation speed n, increasing N can make the shortest exposure time t shrink, meaning it is easier to synchronize the rotation speed of the turntable and the camera exposure time. It can also be found that with the same exposure time t, the number of times the field of view is evenly illuminated can increase by N times. Then even if the exposure time is , then due to the increase in the number of times the light field is swept during the uniform exposure time stage, the fluorescence intensity increases, and the extra part that forms the stripe artifacts during the exposure time will also be less obvious, and the longer the exposure time, the less obvious the stripes.

[0095] 2. Calibration of DMD.

[0096] 2.1 Calibration of DMD spatial coordinates.

[0097] In this step, first, the DMD is adjusted to be non-tilted to avoid introducing additional systematic errors into the imaging. In addition, a mapping relationship between the DMD coordinate system and the camera coordinate system is established for convenient subsequent calculation and processing. Therefore, this operation is also called "coordinate system registration".

[0098] Coordinate system registration first generates a rectangular binary mask with a length and width of and pixels respectively. It should be noted that the rectangle should not be located at the center of the mask, otherwise it is impossible to determine whether the coordinate system has rotated 180°. Subsequently, an image is taken with the fluorescence plate as a sample, and the is binarized using the Otsu method, and then edge detection is performed to obtain the width and height of the rectangle in as and Pixels, so the scaling factor is:

[0099] .

[0100] Subsequently, according to the orientation of the rectangle, the rotation angle from the camera coordinate system plane to the DMD coordinate system plane is obtained. The rotation angle is caused by the mirrors in the optical path and is generally an integer multiple of, which is easy to judge. Therefore, the present invention obtains the rotation and scaling matrix from the camera coordinate system to the DMD coordinate system : :

[0101] .

[0102] In the formula, , are the central coordinates of rotation and can be set as the center of the image captured by the camera. The second stage is the fine registration stage. After the coarse registration image is obtained through the action of the rotation and scaling matrix , subsequently, feature point detection is performed on and and by the target detection algorithm, then the feature points are matched by the Brute-Force matcher. Finally, a more refined transformation matrix can be calculated through these matched feature points. This matrix includes the translation and affine transformation of the image, which can not only achieve the alignment of the plane, but also further correct the scaling factor and rotation angle obtained in the first stage. is a matrix. The change from the coordinate in the camera coordinate system to the coordinate in the DMD coordinate system is as follows:

[0103] .

[0104] 2.2 Calibration of the DMD diffraction effect.

[0105] At the same time, the DMD is a periodic structure formed by the arrangement of a micro-mirror array and will exhibit a diffraction effect similar to a two-dimensional grating. There are different diffraction efficiencies and exit angles for the excitation light with different incident angles and wavelengths. Therefore, the present invention uses a one-dimensional galvanometer to adjust the angle of the light beam incident on the DMD target surface when excited by different wavelengths, so as to achieve the same high diffraction efficiency and the smallest field-of-view shift at different wavelengths. The specific principle is as follows.

[0106] The DMD is used as a projection device. Lights of different wavelengths are incident at the same angle, but the diffracted lights will have angular offsets and diffraction efficiency offsets, which will affect the performance of the system's multi-color imaging. The rotation axis of each micromirror is along the diagonal. In the present invention, the DMD is arranged so that the rotation axis is perpendicular to the horizontal plane, and the light beam is incident on the DMD surface parallel to the horizontal plane. As Figure 4 shown, when in the flat state, the DMD is located in the ABC plane, the rotation axis is BC, and in the on state, the micromirror rotates around BC, and point A rotates to point A'. The incident light ray and the outgoing light ray are both located in the plane OAA', where the angle , and are the incident angle, the outgoing angle, and the micromirror rotation angle respectively. The Ox axis and the Oy axis are parallel to both sides of the micromirror, and the Oz and Oz' axes are the normal lines of the plane before and after the micromirror rotates. In order to establish the quantitative relationship between the incident angle and the outgoing angle, project this three-dimensional structure onto the plane xOz, and the angles , and are projected onto the angles , and respectively. According to spatial geometry, it is easy to know that:

[0107] .

[0108] .

[0109] The outgoing angle needs to be the direction of the main diffraction maximum and needs to satisfy the grating equation:

[0110] .

[0111] .

[0112] where, d represents the period of the micromirror array, m represents the diffraction order, is the excitation light wavelength. The corresponding diffraction efficiency is equal to:

[0113] .

[0114] where, a is the interval between two adjacent micromirrors, and its value is equal to , and s is the fill factor of the DMD. Figure 4 Shows the relationship between the diffraction efficiency, the outgoing angle, the incident light wavelength, and the incident angle. In practice, taking three-color imaging as an example, assume that the three-color wavelengths are 561 nm, 488 nm, and 405 nm respectively, and the outgoing angle is along the direction of the DMD target surface, that is, is equal to 0. Given that the period d of the micromirror array is 5.4 μm, substituting it into the formula for calculation, the incident angle should satisfy:

[0115] 。

[0116] By adjusting the order m to make the incident angles close to each other, it is more conducive to the adjustment of the galvanometer angle.

[0117] Step 101: Based on the original image, use the dynamic pinhole array-pixel reassignment method to achieve imaging reconstruction, that is, based on DPA-PR to achieve ISM reconstruction, including: performing Fourier transform on each pixel of the three-dimensional image stack along the scanning direction, and performing DC suppression preprocessing on the original image; performing Fourier transform on the DC-suppressed preprocessed original image and superimposing the spectra, and using the spectral grid basis vector formula to determine the basis vectors in the spectral grid; generating a lattice based on the basis vectors in the spectral grid; using the spatial local maximum points of the light spots as a reference to calculate the offset vector between the lattice point spacing and the excitation optical axis array point spacing; positioning the excitation optical axis based on the offset vector; setting a virtual detection array centered on the excitation optical axis, and synchronously moving the virtual detection array with the excitation dot matrix scanning to complete the sampling of the virtual detection array to obtain a plurality of confocal sub-images; performing apodization processing on the plurality of confocal sub-images by multiplying by a Hanning window, and convolving the plurality of confocal sub-images with a Gaussian kernel to remove the noise of the plurality of confocal sub-images to obtain a plurality of confocal denoised sub-images; aligning and superimposing the plurality of confocal denoised sub-images with the central confocal denoised sub-image to obtain an initial reconstructed image; performing upsampling processing on the initial reconstructed image according to the size of the initial image to obtain a dual confocal image scanning microscopy.

[0118] The ISM mode requires multi-focus excitation mode illumination and scanning of the sample. Compared with using a pinhole array and a galvanometer to generate and scan multiple foci, the DMD provides a more stable step. The DMD mask for generating multi-focus illumination is designed as a periodic lattice, and each lattice element consists of a square aperture formed by 4×4 open DMD pixels. Considering that the size of each DMD pixel is 5.4 microns, under a 100× objective lens, the size of the excitation spot on the sample plane is about 220 nanometers. The multi-focus illumination pattern moves two DMD pixels each time, which is equivalent to a step size of 108 nanometers on the sample plane, achieving a sampling rate twice that of the diffraction limit, and performing multi-focus scanning on the sample. Since a turntable was previously used to remove the out-of-focus light signal, in the multi-focus illumination here in the present invention, the distance between the foci can be closer without worrying about crosstalk. Therefore, the present invention controls the interval between the lattice elements to be 12 pixels, that is, the side length of the square and the distance between each other are 4:12, which can further reduce the number of original frames required for image scanning microscopic reconstruction.

[0119] The reconstruction of traditional ISM is generally achieved by computational or optical means, either by shrinking the light spot to 1 / 2 or by expanding the spacing between light spots to twice the original. This process of shifting the signals recorded by the detector back to their most likely generation positions is called Pixel Reassignment (PR). However, the scaling factor of 1 / 2 here is based on the ideal situation where the excitation point spread function and the detection point spread function of the system are exactly the same, Stokes uniqueness is ignored, and there is no distortion. The Adaptive Pixel Reassignment (APR) algorithm uses a 5×5 single-photon avalanche diode detector array to record the fluorescence signals emitted when a single focus scans the sample. When the sampling condition is met, that is, the scanning step size is equal to the single-point detector interval, each SPAD will record a sub-image of the same sample area. By performing cross-correlation registration to align the sub-images in the spatial domain, adaptive scaling can be achieved instead of a constant 2-fold scaling. Finally, the aligned sub-images are superimposed to obtain an ideal ISM image, and a 2-fold resolution improvement can be achieved through further deconvolution. However, compared with systems based on scanning and de-scanning configurations, when the multi-focus excitation mode scans in this system, the light spot moves on the camera target surface while the sample remains stationary. Therefore, the present invention specifically proposes the DPA-PR algorithm to achieve super-resolution reconstruction of ISM.

[0120] With the scanning step size as the detector interval, a 5×5 Virtual Detector Array (VDA) is constructed. Since multi-focus excitation is used, this means that a VDA should be assumed to exist at each focus. The gray value detected by each VDA element is interpolated from the actual gray value detected by the camera. By separately extracting the values of different VDA elements and arranging them according to the light spot position and scanning order, 25 sub-images can be obtained. Theoretically, the object information contained in different sub-images is the same, except for the overall spatial position shift and signal-to-noise ratio difference. The specific reconstruction process is divided into three steps: 3.1 Excitation optical axis positioning; 3.2 VDA sampling; 3.3 Cross-correlation registration and superposition. Finally, a two-fold resolution improvement can be achieved through further deconvolution.

[0121] 3.1 Excitation optical axis positioning.

[0122] The position of the excitation optical axis is the center of the illumination light spot. However, due to the modulation effect of the sample structure on the excitation light spot, the position of the maximum gray value in the obtained illumination light spot image may not be consistent with the actual excitation optical axis. Based on the prior knowledge that the excitation optical spot follows a square lattice distribution, the present invention can obtain the basis vectors of the lattice in the spatial domain and (ideally, ),(representing the horizontal and vertical directions respectively, the distance between the two ringing light spots. By using these vectors to generate lattice points and calculating the average deviation between the generated lattice points and the brightest illumination point, the offset vector can be obtained.) Vector , and can be used to determine the exact position of the excitation light axis array. The specific process is as follows:

[0123] (1) DC suppression preprocessing: Perform Fourier transform on each pixel of the three-dimensional image stack along the scanning direction. Since the background stray signal is relatively uniform on the macroscopic scale and does not change with the movement of the excitation light spot, it exhibits DC characteristics. By eliminating the zero-frequency component, the background of the acquired image can be further removed.)

[0124] (2) Determine and : Since the Fourier spectrum of the dot matrix image in the spatial domain is still a dot matrix, and the translation of the dot matrix in the spatial domain does not change the corresponding position in the frequency domain, the present invention performs Fourier transform on the original image and superimposes their spectra. Assuming that the basis vectors in the spectral grid are and , the coordinates of the spectral order points are:

[0125] .

[0126] In the formula represents the coordinates in the spectrum, represents the order. By locating the local maximum, the present invention determines the coordinates of a series of lattice points, as well as their corresponding orders . Then, by the least squares method, and are obtained:

[0127] .

[0128] (3) Determine : Use and to generate a lattice . Ideally, the point spacing of this lattice should be equal to the point spacing of the excitation light axis array, but there is a deviation in their spatial positions, represented by . The present invention takes the spatially local maximum point of the light spot as a reference, calculates the offset vector from the nearest lattice point, and takes the average value to eliminate the influence of sample modulation on the excitation light spot, obtaining . Each frame has its own . The present invention uses to represent the of the k-th frame.)

[0129] 3.2 VDA sampling.

[0130] It is assumed in the present invention that there is a 5×5 VDA, the center of which is located on the excitation optical axis and moves synchronously with the excitation dot matrix scanning. The spacing between adjacent VDA pixels is equal to the scanning step of the excitation lattice. The image is resampled by the VDA to obtain the pixel values detected at the scanning positions of 25 sub-images. The key steps are as follows.

[0131] (1) Determine the spacing of the VDA: Taking an 8×8 scan as an example, the pattern loaded into the DMD is scanned line by line. The calculation of the spacing value is based on . First, similar to phase unwrapping, there may be a difference in the basic vectors between vectors, so adjustment is required . Then, the rows and columns of the 8×8 cells are averaged, and only the x-component and y-component are taken respectively to obtain and . and The increments of are the spacings of the VDA elements and , which can be obtained by linear regression.

[0132] .

[0133] Among them, and are the spacings of the elements in the row direction and column direction of the virtual detection array respectively; k is the first-order coefficient of linear regression, and its least-squares estimated value is the spacing of the virtual detection array; b is the constant term coefficient of linear regression; is a constant vector; and are the average values of the cells of the deviation vector between the lattice point spacing and the point spacing of the excitation optical axis array along the row direction and column direction.

[0134] (2) Determine the gray value of VDA sampling: The spacing of VDA pixels is not equal to the size of camera pixels. It is assumed that the image gray value distribution follows a discrete function , where represents the discrete camera pixel coordinates, and the present invention constructs the VDA pixel coordinates relative to the VDA pixel coordinates. By resampling and calculating each VDA position, the gray value of VDA sampling can be obtained.

[0135] 3.3 Cross-correlation registration and superposition.

[0136] After 5×5 VDA sampling, 25 confocal sub-images can be obtained. Then, the DPA-PR reconstruction result is obtained by aligning and superimposing the sub-images.

[0137] (a) Registration: First, the image is apodized by multiplying with a Hann window and convolved with a Gaussian kernel to remove noise. Then, phase cross-correlation is performed between the k-th edge sub-image and the central sub-image (i.e., k = 13 when using 5×5 VDA) to obtain the k-th shift value :

[0138] 。

[0139] where, and represent the Fourier transform and the inverse Fourier transform respectively. denotes the Hadamard product, and the superscript asterisk * represents the complex conjugate.

[0140] (b) Superposition and post-processing: After moving the sub-image to align it with the central sub-image, the reconstruction result is obtained by directly superimposing the sub-images, and its pixel size is equal to the scanning step size. Then, the reconstruction result is adjusted to the original image size by upsampling. Due to reasons such as computational precision error, tilted DMD position, and aberration, and may have slight differences. In addition, there may also be a difference between the cumulative displacement after moving 8 steps or (i.e., scanning a whole section of distance along the fast axis direction or the slow axis direction) and or (the actual excitation lattice grid spacing). These deviations are corrected by scale-invariant feature transform.

[0141] To achieve high-fidelity reconstruction of ISM, the present invention has developed the DPA-PR algorithm. However, the data collected by the present invention can also be reconstructed based on the algorithms of PR and multi-image deconvolution, but the linear correlation, structural similarity, and other indicators of the reconstruction result with respect to the original image will become worse, that is, the image fidelity is reduced. During the DPA-PR reconstruction process, the size of the VDA is not necessarily 5×5, and it can also be 3×3 or 7×7. However, 3×3 will cause signal loss at the edges of some light spots, and for a 7×7 VDA, the peripheral pixels can hardly detect the optical signal, and it will also lead to a larger computational cost and a slower reconstruction speed.

[0142] Step 102: Implement high-depth super-resolution projection SIM imaging; including: loading a stripe mask for projection SIM imaging to achieve seamless switching between multiple imaging modalities; performing phase shifting on the stripes during projection SIM imaging to achieve uniform illumination of the field of view; reconstructing the SIM image based on Fourier domain operations; notching the high-order points causing artifacts in the reconstructed spectrum to obtain the SIM reconstruction result.

[0143] Benefiting from the fact that the DMD can flexibly control the wavefront, this system is also compatible with other super-resolution modes. By loading a stripe mask, projection SIM imaging can be performed to achieve seamless switching between multiple imaging modalities. The period of the stripe mask is 8 DMD pixels, and the corresponding stripe period on the sample surface under a 100× objective is 432 nanometers. Under 640-nanometer excitation light, a 1.6-fold spread spectrum can be achieved. Compared with 2D-SIM that forms excitation stripes based on interference, which can achieve a spread spectrum close to 2 times, projection SIM is limited by the attenuation of the optical system MTF for high-frequency signals. If the stripe period is too small, the visibility will rapidly decrease, and its improvement in resolution is limited. However, at the same time, due to the stripes formed by projection being within the sample volume and only visible near the focal plane and attenuating as it moves away from the focal plane, it can provide better sectioning ability.

[0144] SIM imaging requires and phase shifting of the stripes to achieve uniform illumination of the field of view. However, this requires that the DMD period be at least 6 pixels or 12 pixels. However, the spatial frequency of the stripes formed by a 6-pixel period is 0.8 times the cut-off frequency, and the visibility decreases to an unacceptable level after passing through the optical system. A 12-pixel period can only provide a 1.3-fold spread spectrum, and the resolution improvement is limited. Therefore, in the present invention, the phase of the stripes is shifted by changing 2 DMD pixels each time, corresponding to and phase shifts. The present invention reconstructs the SIM image based on Fourier domain operations:

[0145] .

[0146] In the formula, is the spectrum of the acquired image, is the spectrum of the sample distribution, is the optical transfer function of the system, is the frequency coordinate. Let take , and respectively. represents the spatial frequency of the illumination mode, and its subscript represents the direction of the stripe spatial frequency. is the sample along First-order frequency broadening in the direction. Subsequently, notch filtering is performed on the high-order points causing artifacts in the reconstructed spectrum to obtain the final SIM reconstruction result.

[0147] Furthermore, the imaging method based on a turntable dual confocal image scanning microscope provided in this embodiment further includes: achieving adaptive illumination of the sample.

[0148] In the traditional imaging process, the sample area within the imaging field of view is equally illuminated. By using the DMD to modulate the excitation light, adaptive illumination can be performed according to the sample structure, which not only reduces the overall illumination dose, reduces phototoxicity and photobleaching, and extends the sample observation time, but also finely adjusts the light dose of each pixel to make the acquired image reach the specified signal-to-noise ratio.

[0149] The process of coordinate registration is as shown in 2.1, and the spatial transformation relationship between the DMD coordinate system and the camera coordinate system can be obtained. Subsequently, all pixels of the DMD are turned on, and the power of the illumination light source is lowered for rapid shooting to obtain a snapshot image , and it is transformed to obtain the image in the DMD coordinate system :

[0150] .

[0151] Subsequently, based on calculate the gray mask . According to different requirements of adaptive illumination, it can be divided into three modes.

[0152] Mode 1: Only illuminate the area with the sample, and do not illuminate the background area. At this time, the gray mask is actually a binary mask, and the present invention needs to set a binary hard threshold , the part with a gray value higher than is the sample area, and the part with a gray value lower than is the background area:

[0153] .

[0154] Mode 2: Provide high-intensity illumination for areas with high fluorescence signals and low-intensity illumination for areas with low fluorescence signals to further improve the imaging contrast. Here, the present invention uses a linear mask, that is and only have a linear change in gray value from a 16-bit image to an 8-bit image.

[0155] .

[0156] Among them, represents element-wise division, Indicates rounding to the nearest integer.

[0157] Mode 3: Provide low-intensity illumination for areas with high fluorescence signals and high-intensity illumination for areas with low fluorescence signals. The signal-to-noise ratio (SNR) of the entire image is more uniform, sacrificing the SNR of the bright areas of the image in exchange for the possibility of longer imaging of the sample. Here, the present invention uses a non-linear mask, and has an inverse proportional relationship with the gray value. However the illumination intensity of the background area cannot be infinitely large, so a lower limit needs to be set , and pixels with gray values lower than are considered as the background and directly set to 0.

[0158] .

[0159] To prevent the pixel values from exceeding the upper limit of an 8-bit image, it is required that .

[0160] Benefiting from the fast response of the DMD and the efficient and fast gray-scale mask generation algorithm, the illumination mode and intensity can be adjusted in real time according to the acquired image information.

[0161] The use of high-power LEDs or multimode lasers in the imaging light source assembly 10 can effectively remove speckle artifacts and provide a large illumination field of view. It can shorten the exposure time when the power is sufficient.

[0162] The introduction of the turntable assembly 50 effectively removes defocus signals. Compared with the wide-field imaging results without the turntable, the confocal results after introducing the turntable are clearer ( Figure 5 ).

[0163] The structured light generation assembly 40 uses a digital micromirror device to generate a multi-focus illumination mode. Compared with using a pinhole array and a galvanometer to generate and scan multi-foci, the DMD provides a more stable step. And the DMD is low-cost and easy to operate,

[0164] The combination of the turntable assembly 50 and the digital micromirror device 42 enables the distance between foci to be closer in multi-focus illumination without worrying about crosstalk. The interval between lattice elements can be controlled to be 12 pixels, that is, the side length of the square and the distance between each other are 4:12. This can further reduce the number of original frames required for image scanning microscopy reconstruction, and only 36 frames are needed for ISM reconstruction.

[0165] The introduction of the turntable assembly 50 ensures that in the ISM imaging process, the multi-focus illumination is clearly visible and will not be masked by the background, and the corresponding spectral feature points are also more obvious, which is beneficial for the DPA-PR algorithm to determine the basic vectors when determining the excitation optical axis (Figure 6 ).

[0166] Based on the principle of ISM, using the DPA-PR algorithm to perform super-resolution reconstruction on images can effectively improve the resolution. On tissue samples, compared with wide-field and confocal microscopy, the multi-confocal image scanning microscope significantly removes the interference of out-of-focus signals, and the resolved structure is clearer. The lateral resolution reaches 150 nanometers respectively, achieving a two-fold improvement in three-dimensional resolution ( Figure 7 ).

[0167] Compared with the traditional PR algorithm, the DPA-PR algorithm can achieve better reconstruction result fidelity. Its results in terms of structural similarity map and peak signal-to-noise ratio are superior to those of PR reconstruction, and the coefficient of determination is as high as 92%, proving that the gray level is relatively well maintained during the reconstruction process, enabling quantitative imaging ( Figure 8 ).

[0168] The introduction of the turntable assembly 50 ensures that during the SIM imaging process, the striped structured light can penetrate deep into the tissue sample, achieving a depth imaging of 50 micrometers, exceeding that of traditional SIM. Without the turntable, the stripes are invisible and a clear spectrum cannot be seen in the frequency domain. After the turntable is introduced, the stripes are visible and the first-order frequency of the stripes is obvious. There is also a significant resolution improvement after SIM reconstruction compared with confocal microscopy ( Figure 9 ).

[0169] The one-dimensional galvanometer 41 can effectively correct the diffraction effect of the digital micromirror device 42, enabling efficient imaging at different wavelengths and making the system compatible with multi-color imaging ( Figure 10 ).

[0170] The present invention designs a dual confocal image scanning microscopy method based on a turntable for three-dimensional thick sample super-resolution and high-fidelity microscopy imaging. The system consists of a uniform illumination module composed of a multimode laser, a multimode homogenizing optical fiber, and an achromatic doublet lens, a structured light generation module composed of a one-dimensional galvanometer and a Digital Micromirror Devices (DMD), a scanning module composed of a high-speed rotating turntable, and a detection module. The uniform illumination module uses a multimode laser in cooperation with a multimode homogenizing optical fiber to decohere the light source and provide high-energy, speckle-free, and uniform excitation light; the structured light generation module uses the DMD as a projection device to achieve structured light illumination of the sample surface by loading a specific mask sequence to achieve super-resolution imaging. At the same time, the DMD is a periodic structure formed by an array of micromirrors and will exhibit a diffraction effect similar to a two-dimensional grating. There are different diffraction efficiencies and exit angles for excitation light with different incident angles and wavelengths. Therefore, a one-dimensional galvanometer is used to control the angle of incidence of excitation light with different wavelengths on the DMD target surface to achieve high-efficiency multi-color imaging. The high-speed rotating turntable is placed behind the dichroic mirror, and the pinholes on it are arranged equidistantly according to the Archimedean spiral, and the high-speed rotation realizes multi-point parallel scanning and desweeping; the detection module uses a area array detector for detection. The system can achieve an imaging depth greater than 180 microns, a resolution improvement of twice the diffraction limit, and an image linearity of 92%, with good quantitative characteristics, and is suitable for high-fidelity super-resolution microscopy imaging in application scenarios requiring high depth and high resolution. In addition, the illumination of the system is realized through a programmable DMD. Therefore, in addition to the ISM mode, the system can also realize Structured Illumination Microscope (SIM) and adaptive illumination modes ( Figure 1 ). For the imaging of the projection SIM implemented by the system, the depth extension can reach 50 microns, which greatly exceeds the traditional SIM based on interference to form fringes; while adaptive illumination can selectively illuminate the region of interest without introducing additional light dose, with low phototoxicity and improving the ability of long-term imaging of living cells. These functions make the present system a multi-functional tool, promoting cell imaging and tissue-scale exploration and meeting the needs of modern biological imaging.

[0171] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same and similar parts among the various embodiments, reference can be made to each other.

[0172] In this article, specific examples are used to elaborate on the principles and implementation modes of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation modes and application scopes. In summary, the content of this specification should not be construed as a limitation on the present invention.

Claims

1. A dual confocal image scanning microscope based on a turntable, characterized in that, Including: A signal control component, and an imaging light source component, an optical fiber transmission component, a light source collimation component, a structured light generation component, a turntable component, a microscopic magnification component, and a camera detection component arranged along the optical path; The imaging light source component is used to output laser light; the optical fiber transmission component is used to homogenize the laser light and input the homogenized laser light into the light source collimation component; the light source collimation component is used to collimate and output the laser light output by the optical fiber transmission component; the structured light generation component is used to generate and scan structured light based on the laser light output by the light source collimation component; a pinhole array distributed along multiple clusters of concentric Archimedean spirals is arranged on the turntable component; multiple pinholes in the pinhole array are arranged at equal intervals along multiple clusters of concentric Archimedean spirals; the rotation speed of the turntable component is adjusted by a three-phase brushless DC motor controller in the signal control component; the microscopic magnification component is used to achieve the magnification factor specified by the microscopic objective and hold the sample, reflect the structured light as excitation light to the rotating turntable component, irradiate the sample with the excitation light passing through the pinhole to generate fluorescence emission light, and transmit the fluorescence emission light passing through the pinhole to the camera detection component; the camera detection component is used to detect and output the fluorescence signal; The signal control component is used to control the imaging light source component, the structured light generation component, the turntable component, and the camera detection component to work synchronously; The signal control component is further used to obtain the detection result and use the detection result as the original image; Based on the original image, imaging reconstruction is realized by using the dynamic pinhole array-pixel reassignment method; The dynamic pinhole array-pixel reassignment method includes: performing Fourier transform on each pixel of the three-dimensional image stack along the scanning direction, and performing DC suppression preprocessing on the original image; performing Fourier transform on the DC-suppressed preprocessed original image and superimposing the spectra, and using the spectral grid basis vector formula to determine the basis vectors in the spectral grid; generating a lattice based on the basis vectors in the spectral grid; using the spatial local maximum points of the light spot as a reference, calculating the offset vector between the lattice point spacing and the excitation light axis array point spacing; positioning the excitation light axis based on the offset vector; setting a virtual detection array centered on the excitation light axis, and synchronously moving the virtual detection array with the excitation dot scanning to complete virtual detection array sampling to obtain multiple confocal sub-images; performing apodization processing on the multiple confocal sub-images by multiplying a Hanning window, and convolving the multiple confocal sub-images with a Gaussian kernel to remove the noise of the multiple confocal sub-images to obtain multiple confocal denoised sub-images; aligning and superimposing the multiple confocal denoised sub-images with the central confocal denoised sub-image to obtain an initial reconstructed image; performing upsampling processing on the initial reconstructed image according to the size of the initial image to obtain dual confocal image scanning microscopy imaging.

2. The dual confocal image scanning microscope based on a turntable according to claim 1, characterized in that The imaging light source component includes a high-power single-mode laser, a high-power multimode laser, or a high-power LED; In multi-color imaging, the imaging light source group is combined by using a multi-in-one optical fiber or combined by a dichroic mirror.

3. The dual confocal image scanning microscope based on a turntable according to claim 1, wherein, The light source collimation component is an aspherical lens, a 90° off-axis parabolic mirror, an aspherical mirror with adjustable focal length, an achromatic doublet lens or an air-spaced doublet lens.

4. The dual confocal image scanning microscope based on a turntable according to claim 1, characterized in that, The structured light generation component includes: a one-dimensional galvanometer, a digital micromirror device, a first lens and a second lens; The one-dimensional galvanometer is used to control the angle of the laser output by the light source collimation component incident on the target surface of the digital micromirror device; The digital micromirror device is used to generate and scan structured light based on the light beam output by the one-dimensional galvanometer, and output the one-dimensional galvanometer to the first lens; The second lens is used to transmit the structured light output by the first lens to the microscopic magnification component.

5. The dual confocal image scanning microscope based on a turntable according to claim 1, characterized in that, The microscopic magnification component includes: a dichroic mirror, a tube lens, a microscopic objective lens and a sample clamping component arranged along the optical path; The sample clamping component is used to clamp the sample; The dichroic mirror is used to reflect the structured light as excitation light to the rotating turret component; The tube lens is used to receive the excitation light passing through the pinhole and output the excitation light passing through the pinhole to the microscopic objective lens; The microscopic objective lens is used to receive the excitation light passing through the pinhole output by the tube lens, output the excitation light passing through the pinhole to the sample, and receive the fluorescence emission light formed by the sample irradiated by the excitation light passing through the pinhole, and output the fluorescence emission light to the tube lens; The tube lens is also used to receive the fluorescence emission light output by the tube lens and output the fluorescence emission light to the rotating turret component; The dichroic mirror is also used to transmit the fluorescence emission light passing through the pinhole to the camera detection component.

6. The dual confocal image scanning microscope based on a turntable according to claim 1, wherein The camera detection component includes: a relay lens and a camera arranged along the optical path; The relay lens is used to achieve conjugate imaging; The camera is used to detect and output the fluorescence signal.

7. The dual confocal image scanning microscope based on a turntable according to claim 6, characterized in that, The relay lens can be replaced by a single-lens reflex lens; the single-lens reflex lens is used to achieve large-field-of-view equal-proportion conjugate imaging.

8. An imaging method using a dual confocal image scanning microscope based on a turntable, characterized in that, including: Obtain a detection result by using the turret-based dual confocal image scanning microscope according to any one of claims 1-7, and use the detection result as the original image; Based on the original image, use the dynamic pinhole array-pixel reassignment method to achieve imaging reconstruction.

9. The imaging method using a dual confocal image scanning microscope based on a turntable according to claim 8, characterized in that, The using the dynamic pinhole array-pixel reassignment method to achieve imaging reconstruction based on the original image includes: Perform Fourier transform on each pixel of the three-dimensional image stack along the scanning direction, and perform DC suppression preprocessing on the original image; Perform Fourier transform on the original image after DC suppression preprocessing and superimpose the spectra, and use the spectral grid basis vector formula to determine the basis vectors in the spectral grid; Generate a lattice based on the basis vectors in the spectral grid; Taking the spatial local maximum point of the light spot as a reference, calculate the offset vector between the lattice point spacing and the excitation light axis array point spacing; Locate the excitation light axis based on the offset vector; Set a virtual detection array centered on the excitation light axis, and synchronously move the virtual detection array with the excitation dot matrix scanning to complete the virtual detection array sampling, and obtain a plurality of confocal sub-images; Perform apodization processing on the plurality of confocal sub-images by multiplying by a Hann window, and perform convolution on the plurality of confocal sub-images with a Gaussian kernel to remove the noise of the plurality of confocal sub-images, and obtain a plurality of confocal denoised sub-images; Align and superimpose multiple confocal denoised sub-images with the central confocal denoised sub-image to obtain an initial reconstructed image; Perform upsampling processing on the initial reconstructed image according to the size of the initial image to obtain a dual confocal image scanning microscopy.

10. The imaging method using the turntable-based dual confocal image scanning microscope according to claim 9, wherein, The method for imaging using a spinning disk-based dual confocal image scanning microscope further includes: Loading a stripe mask for projection SIM imaging to achieve seamless switching between multiple imaging modalities; performing phase shift on the stripes during projection SIM imaging to achieve uniform illumination of the field of view; Reconstruct the SIM image based on Fourier domain operations; Notch the high-order points causing artifacts in the reconstructed spectrum to obtain the SIM reconstruction result.

Images