Dual confocal image scanning microscope based on turntable and imaging method

Abstract

The invention discloses a turntable-based dual confocal image scanning microscope and an imaging method, and relates to the technical field of optical elements, systems, instruments and imaging. The imaging light source assembly, the optical fiber transmission assembly, the light source collimation assembly, the structured light generation assembly, the turntable assembly, the microscopic amplification assembly and the camera detection assembly are arranged along a light path; the structured light generation module realizes structured light illumination on a sample surface by loading a specific mask sequence, super-resolution imaging is realized, pinholes in the turntable assembly are equidistantly arranged according to an Archimedes spiral, and the turntable assembly rotates at a high speed to realize multi-point parallel scanning and de-scanning; the detection module performs detection by using an area array detector, so that the resolution improvement of double diffraction limit is realized; according to the dynamic pinhole array-pixel redistribution method, super-resolution reconstruction is carried out based on an image scanning microscopic system principle, and Stokes shift, optical aberration and other non-ideal conditions can be effectively corrected.

Description

Technical Field

[0001] The present invention relates to the fields of optical elements, systems and instruments, and imaging technologies, and particularly relates to a dual confocal image scanning microscope and an imaging method based on a turntable. Background Art

[0002] At present, optical microscopes have become indispensable research tools in the biomedical field. Among them, super-resolution optical microscopy technology can break through the diffraction limit and achieve clear imaging of biological structures at the subcellular scale, and has been widely used. However, due to the limited penetration depth of light waves in biological tissues, and the existence of interference such as out-of-focus signals and scattering signals during the imaging process, there are problems such as poor resolution and low fidelity in deep tissue super-resolution imaging. The annular loss beam of the stimulated emission depletion microscope is prone to distortion when penetrating tissues, resulting in the inability to effectively erase the periphery of the point spread function, leading to a serious degradation of resolution; the fringe pattern of the structured illumination microscope is easily submerged by background signals, resulting in inaccurate estimation of algorithm parameters, and finally obvious artifacts appear in the reconstructed image; single molecule localization microscopy is also affected by background fluorescence and light scattering in tissue samples, reducing the localization accuracy of excitation points, thus distorting the image structure.

[0003] Confocal microscopes have become the preferred technology for biologists in fluorescence imaging due to their excellent optical sectioning ability and versatility. However, traditional single-point scanning confocal microscopes have certain limitations in terms of imaging speed, photobleaching and phototoxicity during live cell imaging. Currently, there are also multi-point scanning confocal microscopy configurations, such as the spinning disk confocal microscopy technology. However, as a traditional microscopy imaging technology, its resolution is limited by the diffraction limit and cannot meet the high-resolution observation requirements of fine structures. Summary of the Invention

[0004] To solve the above problems existing in the prior art, the present invention provides a multi-point confocal image scanning microscope and an imaging method.

[0005] To achieve the above object, the present invention provides the following solutions.

[0006] A dual confocal image scanning microscope based on a turntable, comprising: a signal control component, and an imaging light source component, an optical fiber transmission component, a light source collimation component, a structured light generation component, a turntable component, a microscopic magnification component and a camera detection component arranged along the optical path; The imaging light source assembly is used to output laser light; the optical fiber transmission assembly is used to homogenize the laser light and input the homogenized laser light into the light source collimation assembly; the light source collimation assembly is used to collimate and output the laser light output by the optical fiber transmission assembly; the structured light generation assembly is used to generate and scan structured light based on the laser light output by the light source collimation assembly; a pinhole array distributed along multiple clusters of concentric Archimedean spirals is provided on the turntable assembly; multiple pinholes in the pinhole array are arranged at equal intervals along multiple clusters of concentric Archimedean spirals; the rotation speed of the turntable assembly is adjusted by a three-phase brushless DC motor controller in the signal control assembly; the microscopic magnification assembly is used to achieve the magnification specified by the microscopic objective and hold the sample, reflect the structured light as excitation light to the rotating turntable assembly, irradiate the sample with the excitation light passing through the pinholes to generate fluorescent emission light, and transmit the fluorescent emission light passing through the pinholes to the camera detection assembly; the camera detection assembly is used to detect and output the fluorescence signal; The signal control assembly is used to control the imaging light source assembly, the structured light generation assembly, the turntable assembly and the camera detection assembly to work synchronously; the signal control assembly is also used to obtain the detection result and use the detection result as the original image; based on the original image, imaging reconstruction is realized by using the dynamic pinhole array-pixel reassignment method; The dynamic pinhole array-pixel reassignment method includes: performing Fourier transform on each pixel of the three-dimensional image stack along the scanning direction, and performing DC suppression preprocessing on the original image; performing Fourier transform on the DC-suppressed preprocessed original image and superimposing the spectra, and using the spectral grid basis vector formula to determine the basis vectors in the spectral grid; generating a lattice based on the basis vectors in the spectral grid; using the spatial local maximum point of the light spot as a reference, calculating the offset vector between the lattice point spacing and the excitation light axis array point spacing; positioning the excitation light axis based on the offset vector; setting a virtual detection array centered on the excitation light axis, and synchronously moving the virtual detection array with the excitation dot scanning to complete the virtual detection array sampling, obtaining multiple confocal sub-images; performing apodization processing on the multiple confocal sub-images by multiplying by a Hann window, and performing convolution on the multiple confocal sub-images with a Gaussian kernel to remove the noise of the multiple confocal sub-images, obtaining multiple confocal denoised sub-images; aligning and superimposing the multiple confocal denoised sub-images with the central confocal denoised sub-image to obtain an initial reconstructed image; performing upsampling processing on the initial reconstructed image according to the size of the initial image to obtain dual confocal image scanning microscopy imaging.

[0007] Optionally, the imaging light source assembly includes a high-power single-mode laser, a high-power multimode laser or a high-power LED; In multi-color imaging, the imaging light source group is combined by using a multi-in-one optical fiber or combined by a dichroic mirror.

[0008] Optionally, the light source collimation assembly is an aspherical lens, a 90° off-axis parabolic mirror, an aspherical mirror with adjustable focal length, an achromatic doublet lens, or an air-spaced doublet lens.

[0009] Optionally, the structured light generation assembly includes: a one-dimensional galvanometer, a digital micromirror device, a first lens, and a second lens; The one-dimensional galvanometer is used to control the angle at which the laser output by the light source collimation assembly is incident on the target surface of the digital micromirror device; The digital micromirror device is used to generate and scan structured light based on the light beam output by the one-dimensional galvanometer, and output the one-dimensional galvanometer to the first lens; The second lens is used to transmit the structured light output by the first lens to the microscopic magnification assembly.

[0010] Optionally, the microscopic magnification assembly includes: a dichroic mirror, a tube lens, a microscopic objective lens, and a sample clamping assembly arranged along the optical path; The sample clamping assembly is used to clamp the sample; The dichroic mirror is used to reflect the structured light as excitation light to the rotating turret assembly; The tube lens is used to receive the excitation light passing through the pinhole and output the excitation light passing through the pinhole to the microscopic objective lens; The microscopic objective lens is used to receive the excitation light passing through the pinhole output by the tube lens, output the excitation light passing through the pinhole to the sample, and receive the fluorescence emission light formed by the sample irradiated by the excitation light passing through the pinhole, and output the fluorescence emission light to the tube lens; The tube lens is also used to receive the fluorescence emission light output by the tube lens and output the fluorescence emission light to the rotating turret assembly; The dichroic mirror is also used to transmit the fluorescence emission light passing through the pinhole to the camera detection assembly.

[0011] Optionally, the camera detection assembly includes: a relay lens and a camera arranged along the optical path; The relay lens is used to achieve conjugate imaging; The camera is used to detect and output the fluorescence signal.

[0012] Optionally, the relay lens can be replaced by a single-lens reflex lens; the single-lens reflex lens is used to achieve large-field-of-view equal-proportion conjugate imaging.

[0013] An imaging method using a rotary disk-based dual confocal image scanning microscope includes: Obtaining a detection result using the rotary disk-based dual confocal image scanning microscope described above, and using the detection result as the original image; Based on the original image, using the dynamic pinhole array-pixel reassignment method to achieve imaging reconstruction.

[0014] Optionally, based on the original image, imaging reconstruction is achieved by using a dynamic pinhole array - pixel reassignment method, including: Perform Fourier transform on each pixel of the three - dimensional image stack along the scanning direction, and perform DC suppression pre - processing on the original image; Perform Fourier transform on the original image after DC suppression pre - processing and superimpose the spectra, and determine the basis vectors in the spectral grid using the spectral grid basis vector formula; Generate a lattice based on the basis vectors in the spectral grid; Taking the spatial local maximum point of the light spot as a reference, calculate the offset vector between the lattice point spacing and the excitation light axis array point spacing; Locate the excitation light axis based on the offset vector; Set a virtual detection array centered on the excitation light axis, and synchronously move the virtual detection array with the excitation dot scanning to complete virtual detection array sampling, obtaining multiple confocal sub - images; Perform apodization on the multiple confocal sub - images by multiplying with a Hann window, and perform convolution on the multiple confocal sub - images with a Gaussian kernel to remove the noise of the multiple confocal sub - images, obtaining multiple confocal denoised sub - images; Align and superimpose the multiple confocal denoised sub - images with the central confocal denoised sub - image to obtain an initial reconstructed image; According to the size of the initial image, perform up - sampling processing on the initial reconstructed image to obtain dual - confocal image scanning microscopy.

[0015] Optionally, the imaging method of applying a dual - confocal image scanning microscope based on a turntable further includes: Load a fringe mask for projection - type SIM imaging to achieve seamless switching of multiple imaging modalities; perform phase - shifting on the fringes during projection - type SIM imaging to achieve uniform illumination of the field of view; Reconstruct the SIM image based on operations in the Fourier domain; Notch the high - order points causing artifacts in the reconstructed spectrum to obtain the SIM reconstruction result.

[0016] In the present invention, a signal control component is provided, as well as an imaging light source component, an optical fiber transmission component, a light source collimation component, a structured light generation component, a turntable component, a microscopic magnification component, and a camera detection component arranged along the optical path; the pinhole on the high-speed rotating turntable serves as both the excitation pinhole during the scanning process and the detection pinhole during the non-scanning process, which constitutes the first confocal effect; in addition, a square hole array mask is loaded on a Digital Micromirror Device (DMD) to achieve multi-point excitation of the sample, and at the same time, a area array detector is used instead of single-point detection for detection. Each pixel of the area array detector can be regarded as a point detector with an infinitely small pinhole, which constitutes the second confocal effect. This dual confocal configuration can effectively reduce the defocus background interference and improve the system resolution at the same time. Compared with the background removal algorithm, the turntable removes defocus by physical means, which not only retains the original intensity distribution but also retains the linear relationship of the focus, providing not only clear confocal images but also high-quality original data for subsequent super-resolution reconstruction.

[0017] The Dynamic Pinhole Array-Pixel Reassigment (DPA-PR) method disclosed in the present invention performs super-resolution reconstruction based on the principle of an Image Scanning Microscopy (ISM). Compared with traditional pixel reassignment algorithms, it can effectively correct Stokes shift, optical aberration, and other non-ideal conditions. Brief Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a schematic diagram showing the relationship between multiple modalities and the mask of the Digital Micromirror Device in an embodiment of the present application.

[0020] Figure 2 It is an optical path diagram of a dual confocal image scanning microscope based on a turntable in an embodiment of the present application.

[0021] Figure 3 It is a schematic diagram showing the relationship between the rotation speed of the turntable and the number of spiral clusters N in an embodiment of the present application.

[0022] Figure 4 It is a schematic diagram of the geometric relationship of the Digital Micromirror Device and a schematic diagram of the quantitative relationship of key physical quantities in an embodiment of the present application.

[0023] Figure 5 This is a diagram showing the physical defocusing effect of the turntable in an embodiment of the present application.

[0024] Figure 6 This is a diagram showing the influence of the introduction of the turntable on multi - focus excitation in an embodiment of the present application.

[0025] Figure 7 This is a diagram comparing the ISM super - resolution reconstruction results and the resolution improvement in an embodiment of the present application.

[0026] Figure 8 This is a diagram showing the comparison result of algorithm fidelity in an embodiment of the present application.

[0027] Figure 9 This is a diagram showing the influence of the introduction of the turntable on fringe excitation and the SIM reconstruction result in an embodiment of the present application.

[0028] Figure 10 This is a diagram comparing the confocal and ISM super - resolution multi - color imaging results in an embodiment of the present application.

[0029] Symbol description: Imaging light source assembly - 10; optical fiber transmission assembly - 20; light source collimation assembly - 30; structured light generation assembly - 40; turntable assembly - 50; microscopic magnification assembly - 60; camera detection assembly - 70; signal control assembly - 80; one - dimensional galvanometer - 41; digital micromirror device - 42; first lens - 43; second lens - 44; dichroic mirror - 61; tube lens - 62; microscopic objective - 63; sample - 64; relay lens - 71; camera - 72. Detailed implementation manners

[0030] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0031] To make the above - mentioned objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0032] As Figure 2 shown, a dual - confocal image scanning microscope based on a turntable is provided, including: a signal control assembly 80, and an imaging light source assembly 10, an optical fiber transmission assembly 20, a light source collimation assembly 30, a structured light generation assembly 40, a turntable assembly 50, a microscopic magnification assembly, and a camera detection assembly 70 arranged along the optical path.

[0033] The imaging light source assembly 10 is used to excite a fluorescent sample (output laser).

[0034] The optical fiber transmission assembly 20 is used to transmit the laser to the optical imaging device and homogenize it (homogenize the laser and input the homogenized laser into the light source collimation assembly).

[0035] The light source collimation assembly 30 is used to collimate and output the laser transmitted by the optical fiber to achieve large-spot uniform illumination; The structured light generation assembly 40 is used to generate and scan structured light based on the laser output by the light source collimation assembly 30, and includes a one-dimensional galvanometer 41, a digital micromirror device 42, a first lens 43, a second lens 44. The single collimated spot output by the light source collimation assembly 30 adjusts the incident angle through the one-dimensional galvanometer 41, and the structured light illumination is generated through the digital micromirror device 42. The one-dimensional galvanometer 41 is used to control the angle of the laser output by the light source collimation assembly 30 incident on the target surface of the digital micromirror device; the digital micromirror device is used to generate and scan structured light based on the beam output by the one-dimensional galvanometer and output the one-dimensional galvanometer to the first lens; the second lens is used to transmit the structured light output by the first lens to the microscopic magnification assembly.

[0036] The turntable assembly 50 is used to rotate the pinholes arranged in an Archimedean spiral at a high speed and stably. The turntable assembly is provided with a pinhole array distributed along multiple clusters of concentric Archimedean spirals; multiple pinholes in the pinhole array are arranged at equal intervals along multiple clusters of concentric Archimedean spirals; the rotation speed of the turntable assembly is adjusted by a three-phase brushless DC motor controller in the signal control assembly. The equally spaced pinholes on the turntable are distributed along multiple clusters of Archimedean spirals. In addition, it can also be designed in a stripe pattern, which can also form uniform illumination of the field of view. Compared with the pinholes, its sectioning effect is reduced, but the light transmittance is increased.

[0037] The microscopic magnification assembly 60 is used to achieve the magnification specified by the microscopic objective and hold the sample, reflect the structured light as the excitation light to the rotating turntable assembly, irradiate the sample through the excitation light of the pinhole to generate fluorescent emission light, and transmit the fluorescent emission light passing through the pinhole to the camera detection assembly; the camera detection assembly is used to detect and output the fluorescent signal. The microscopic magnification assembly 60 includes a dichroic mirror 61, a tube lens 62, a microscopic objective 63 and a sample 64, and is used to achieve the magnification specified by the microscopic objective and hold the sample.

[0038] The camera detection assembly 70 includes a relay lens 71 and a camera 72, and is used to detect and output the fluorescent signal. The aforementioned fluorescent signal generated by excitation enters the camera detection assembly 70 after passing through the turntable assembly 50, the microscopic objective 63, the tube lens 62 and the dichroic mirror 61 in the microscopic magnification assembly 60.

[0039] A signal control component 80 is used to synchronize the imaging light source component 10, the structured light generation component 40, the turntable component 50, and the camera detection component 70; the signal control component is also used to obtain a detection result and use the detection result as an original image; based on the original image, imaging reconstruction is achieved by using a dynamic pinhole array - pixel reassignment method; the dynamic pinhole array - pixel reassignment method includes: performing Fourier transform on each pixel of a three - dimensional image stack along the scanning direction, and performing direct - current suppression pre - processing on the original image; performing Fourier transform on the original image after direct - current suppression pre - processing and superimposing the spectra, and using the spectral grid basis vector formula to determine the basis vectors in the spectral grid; generating a lattice based on the basis vectors in the spectral grid; using the spatial local maximum points of the light spots as a reference, calculating the offset vector between the lattice point spacing and the excitation light axis array point spacing; positioning the excitation light axis based on the offset vector; setting a virtual detection array centered on the excitation light axis, and synchronously moving the virtual detection array with the excitation dot array scanning to complete virtual detection array sampling, obtaining multiple confocal sub - images; performing apodization processing on the multiple confocal sub - images by multiplying by a Hanning window, and performing convolution on the multiple confocal sub - images with a Gaussian kernel to remove the noise of the multiple confocal sub - images, obtaining multiple confocal denoised sub - images; aligning and superimposing the multiple confocal denoised sub - images with the central confocal denoised sub - image to obtain an initial reconstructed image; performing upsampling processing on the initial reconstructed image according to the size of the initial image to obtain dual - confocal image scanning microscopy imaging.

[0040] Further, the imaging light source component 10 may include a high - power single - mode laser, a high - power multi - mode laser, a high - power LED, to generate a high - power illumination light source. In multi - color imaging, beam combination is performed using a multi - in - one optical fiber or through a dichroic mirror.

[0041] Further, the optical fiber transmission component 20 generally uses a multi - mode optical fiber or a liquid light waveguide for transmission, and single - mode optical fibers are also tried. The core diameter of the multi - mode optical fiber generally ranges from 50um to 400um, and larger or smaller core diameters are also possible. The liquid light waveguide generally uses a 3mm core output. The liquid light waveguide and the multi - mode optical fiber have advantages in terms of transmission efficiency, and the coupling efficiency can reach 90%.

[0042] Further, the light source collimation component 30 uses an aspherical lens, and the light output from the multi - mode optical fiber or the liquid light waveguide is collimated by the aspherical lens. It can also be collimated by a 90° off - axis paraboloidal reflector, an adjustable - focal - length aspherical mirror, an achromatic doublet lens, an air - spaced doublet lens, etc. The light output from the single - mode optical fiber and the multi - mode optical fiber can also be collimated in the above - mentioned manner.

[0043] Further, the microscopic magnification component includes: a dichroic mirror, a tube lens, a microscopic objective lens, and a sample clamping component arranged along the optical path; the sample clamping component is used to clamp the sample; the dichroic mirror is used to reflect the structured light as the excitation light to the rotating turntable component; the tube lens is used to receive the excitation light passing through the pinhole and output the excitation light passing through the pinhole to the microscopic objective lens; the microscopic objective lens is used to receive the excitation light passing through the pinhole output by the tube lens, output the excitation light passing through the pinhole to the sample, and receive the fluorescence emission light formed by the sample irradiated by the excitation light passing through the pinhole, and output the fluorescence emission light to the tube lens; the tube lens is further used to receive the fluorescence emission light output by the tube lens and output the fluorescence emission light to the rotating turntable component; the dichroic mirror is further used to transmit the fluorescence emission light passing through the pinhole to the camera detection component. In the structured light generation component 40, a one-dimensional galvanometer 41 is used to control the angle of the light beam incident on the target surface of the digital micromirror device 42, ensuring that the direction of the diffraction main maximum of a specific order remains unchanged under different wavelength imaging, and an analog signal generated by the signal control component controls the movement of the galvanometer.

[0044] Further, the digital micromirror device 42 is used to generate and scan the structured light. Compared with generating and scanning multi-foci using a pinhole array and a galvanometer, the digital micromirror device provides a more stable step. The mask of the digital micromirror device for generating multi-focal illumination is designed as a periodic lattice, and each lattice element consists of a square aperture formed by 4×4 open-state digital micromirror device pixels. The size of each digital micromirror device pixel is 5.4 micrometers, and under a 100× objective lens, an excitation spot of approximately 220 nanometers can be formed on the sample plane. The multi-focal illumination pattern moves two DMD pixels each time, corresponding to a step size of 108 nanometers on the sample plane, achieving a sampling rate twice that of the diffraction limit. The interval between the lattice elements is controlled to be 12 pixels, that is, the side length of the square and the distance between each other are 4:12, further reducing the number of original frames required for image scanning microscopic reconstruction.

[0045] Further, the turntable component 50 realizes rapid physical defocusing and improves the imaging contrast. The spinning disk confocal microscope generates an array of excitation point sources through a rotating Archimedean spiral pinhole array placed in the conjugate plane of the microscope sample, and at the same time acts as its conjugate pinhole to remove the defocus signal in the sample. Each pinhole in the array scans to generate an illumination arc (curve) on the sample, and the fluorescence generated by these scans is collected and naturally superimposed during the camera exposure time. When the turntable motor is driven at a high enough rate to repeatedly scan the sample, a real-time confocal image is generated. The pinhole array disk is etched with pinholes arranged in an Archimedean spiral on chrome-plated fused quartz glass, and a circular hole is laser-cut at the center of the disk, and a motor with a very high and stable rotation speed is used to drive the rotation of the pinhole disk.

[0046] Further, the rotation speed of the turntable component 50 is adjusted by a three-phase brushless DC motor controller.

[0047] Furthermore, the dichroic mirror 61 is used to separate the excitation light and the emission light, reflect the aforementioned excitation light into the tube lens 62, and transmit the aforementioned fluorescence emission light into the camera detection assembly 70.

[0048] Furthermore, the camera detection assembly includes: a relay lens 71 and a camera 72 arranged along the optical path; the relay lens is used to achieve conjugate imaging; the camera is used to detect and output the fluorescence signal. The relay lens 71 in the camera detection assembly 70 uses a single-lens reflex lens to collect the returned fluorescence, and the single-lens reflex lens can achieve equal-proportion conjugate imaging with a large field of view. A scientific-grade camera with a large target surface can detect the returned fluorescence with high sensitivity. Other relay lens groups can also be used to achieve conjugate imaging, and ordinary cameras can also be used to achieve fluorescence detection.

[0049] Furthermore, the signal control assembly 80 can output 4 analog signals and 2 digital signals. One analog signal controls the galvanometer mirror for one-dimensional scanning, and the other three analog signals respectively control the DMD mask switching, the camera in the camera detection assembly, and the nano-positioning piezoelectric sample scanning stage in the microscope main body assembly. The DMD operates in the "Pattern On-The-Fly" mode, the exposure time of the DMD is synchronized with the exposure time of the camera, and the dark time of the DMD matches the readout time of the camera. The two digital signals serve as address codes to achieve arbitrary switching of the light source through a demultiplexer, and cooperate with the DMD exposure time to achieve synchronization of the light source. Multimodal and digital micromirror device masks are as Figure 1 shown.

[0050] Furthermore, the present invention also provides an imaging method using a turntable-based dual confocal image scanning microscope, including: step 100, step 101, and step 102.

[0051] Among them, step 100: Obtain a detection result using a turntable-based dual confocal image scanning microscope, and use the detection result as the original image. Before obtaining the detection result using the turntable-based dual confocal image scanning microscope, the design of the turntable and the self-synchronization with the camera exposure and the calibration of the DMD need to be carried out; the calibration of the DMD includes: the calibration of the DMD spatial coordinates and the calibration of the DMD diffraction effect.

[0052] 1. Design of the turntable and self-synchronization with the camera exposure.

[0053] The pinhole on the high-speed rotating turntable serves as both the excitation pinhole during the scanning process and the detection pinhole during the de-scanning process. The high-speed rotation realizes multi-point parallel scanning and de-scanning, achieving the first confocal effect. The present invention adopts the following method for design to achieve uniform scanning illumination of the field of view and is more convenient for synchronization with the camera exposure time.

[0054] The pinhole array on the turntable disk is arranged along a concentric Archimedean spiral. Taking the center of the turntable as the pole to establish a coordinate system, the equation of the Archimedean spiral is as follows: .

[0055] In the formula, are the radial position and angular position of a point in the polar coordinate system, is the serial number of the concentric Archimedean spiral cluster; is the number of concentric Archimedean spiral clusters (an integer greater than or equal to 1), represents the starting position of the turntable, controls the distance of the radial position stepping for each rotation of the spiral. In order to ensure uniform illumination of the field of view when the turntable rotates, it is necessary to ensure that the arc length between adjacent pinholes is equal and equal to . Taking a certain one of the N concentric Archimedean spirals as the research object, its equation is: .

[0056] In the formula, , . Let the angular coordinates of the kth point on this spiral be , and its arc length from the starting point of the spiral (the point at ) is , and its calculation formula is as follows: .

[0057] According to the adjacent pinhole spacing being equal to , it can be obtained that , and then the specific value of can be solved, so as to obtain the corresponding radial position , and the specific positions of all points on the turntable are determined through this position.

[0058] And the spiral of the turntable is composed of N sub-spirals, each sub-spiral is exactly the same, and only there is a rotation between their spatial positions. When the turntable rotates, each sub-spiral is responsible for of the field of view, that is, it can uniformly illuminate the field of view times ( Figure 3 ) for one rotation. The camera exposure event needs to be synchronized with the rotation speed of the turntable, that is, the successive exposure events need to be equal to or an integer multiple of the time required for the turntable to uniformly cover the field of view for one week. Specifically, the rotation speed n (unit: round per minute, rpm) of the turntable, the exposure time t (unit: s) and N need to satisfy the following relationship: .

[0059] The "mod" represents the modulo operation. It can be found that with a constant rotational speed n, increasing N can reduce the shortest exposure time t, meaning it is easier to synchronize the rotational speed of the turntable and the exposure time of the camera. It can also be found that at the same exposure time t, the number of times of uniform illumination of the field of view can be increased by N times. Then, even at this time when the exposure time under , then due to the increase in the number of times the light field is swept through during the uniform exposure time stage, the fluorescence intensity increases, and the extra part that forms the fringe artifacts during the exposure time will also be less obvious, and the longer the exposure time, the less obvious the fringes.

[0060] 2. Calibration of the DMD.

[0061] 2.1 Calibration of the DMD spatial coordinates.

[0062] In this step, first, the DMD is adjusted to be non-tilted to avoid introducing additional systematic errors into the imaging. In addition, a mapping relationship between the DMD coordinate system and the camera coordinate system is established for convenient subsequent calculation and processing. Therefore, this operation is also called "coordinate system registration".

[0063] Coordinate system registration first generates a rectangular binary mask , with the length and width being and pixels respectively. It should be noted that the rectangle should not be located at the center of the mask, otherwise it is impossible to determine whether the coordinate system has rotated 180°. Subsequently, an image is taken with the fluorescence plate as a sample, and the is binarized using the Otsu method, and then edge detection is performed to obtain the width and height of the rectangle in as and pixels respectively. Therefore, the scaling factor is: .

[0064] Subsequently, according to the orientation of the rectangle, the rotation angle from the camera coordinate system plane to the DMD coordinate system plane is obtained. The rotation angle is caused by the mirrors in the optical path and is generally an integer multiple of , which is easy to judge. Therefore, the present invention obtains the rotation and scaling matrix from the camera coordinate system to the DMD coordinate system: .

[0065] In the formula, , are the central coordinates of the rotation and can be set as the center of the image captured by the camera. The second stage is the fine registration stage. After the action of the rotation and scaling matrix the coarsely registered image is obtained , and then through the object detection algorithm, feature point detection is performed on and , and then the feature points are matched through the Brute-Force matcher. Finally, a more refined transformation matrix can be further calculated through these matched feature points. This matrix includes the translation and affine transformation of the image, and can not only achieve the alignment of the planes, but also further correct the scaling factor and rotation angle obtained in the first stage It is a matrix. The change from the coordinates in the camera coordinate system to the coordinates in the DMD coordinate system is as follows: .

[0066] 2.2 Calibration of the DMD diffraction effect

[0067] At the same time, the DMD is a periodic structure formed by the arrangement of a micro-mirror array, and will exhibit a diffraction effect similar to a two-dimensional grating. There are different diffraction efficiencies and exit angles for the excitation light with different incident angles and wavelengths. Therefore, the present invention uses a one-dimensional galvanometer to adjust the angle of the light beam incident on the DMD target surface when excited by different wavelengths, so as to achieve the same high diffraction efficiency and the smallest field-of-view shift at different wavelengths. The specific principle is as follows

[0068] The DMD is used as a projection device. Lights of different wavelengths are incident at the same angle, but the diffracted light will have angular shift and diffraction efficiency shift, and these shifts will affect the performance of the system's multi-color imaging. The rotation axis of each micro-mirror is along the diagonal. The present invention arranges the DMD so that the rotation axis is perpendicular to the horizontal plane, and the light beam is incident on the DMD surface parallel to the horizontal plane. As Figure 4 shown, when in the flat state, the DMD is located in the ABC plane, the rotation axis is BC, and in the open state, the micro-mirror rotates around BC, and point A rotates to point A'. The incident light ray and the exit light ray are both located in the plane OAA', where the angle , and are the incident angle, the exit angle, and the micro-mirror rotation angle respectively. The Ox axis and the Oy axis are parallel to the two sides of the micro-mirror, and the Oz and Oz' axes are the normal vectors of the plane before and after the micro-mirror rotates. In order to establish the quantitative relationship between the incident angle and the exit angle, project this three-dimensional structure onto the plane xOz, and the angles , and are projected onto the angles , and , it can be easily known from spatial geometry that: .

[0069] .

[0070] The exit angle needs to be the direction of the main diffraction maximum and needs to satisfy the grating equation: .

[0071] .

[0072] where d represents the period of the micromirror array, m represents the diffraction order, is the excitation light wavelength. The corresponding diffraction efficiency is equal to: .

[0073] where a is the spacing between two adjacent micromirrors, and its value is equal to , and s is the fill factor of the DMD. Figure 4 shows the relationship between the diffraction efficiency, the exit angle, the incident light wavelength, and the incident angle. In practice, taking three-color imaging as an example, assuming that the three-color wavelengths are 561 nm, 488 nm, and 405 nm respectively, and the exit angle is along the DMD target plane direction, that is is equal to 0. Given that the period d of the micromirror array is 5.4 μm, substituting it into the formula for calculation, the incident angle should satisfy: .

[0074] By adjusting the order m to make the incident angles close to each other, it is more conducive to the galvanometer to adjust the angle.

[0075] Step 101: Based on the original image, use the dynamic pinhole array - pixel reassignment method to achieve imaging reconstruction, that is, based on DPA - PR to achieve ISM reconstruction, including: performing Fourier transform on each pixel of the three - dimensional image stack along the scanning direction, and performing DC suppression pre - processing on the original image; performing Fourier transform on the DC - suppressed pre - processed original image and superimposing the spectra, and using the spectral grid basis vector formula to determine the basis vectors in the spectral grid; generating a lattice based on the basis vectors in the spectral grid; using the spatial local maximum points of the light spots as a reference, calculating the offset vector between the lattice point spacing and the excitation optical axis array point spacing; positioning the excitation optical axis based on the offset vector; setting a virtual detection array centered on the excitation optical axis, and synchronously moving the virtual detection array with the excitation dot array scanning to complete the virtual detection array sampling, obtaining multiple confocal sub - images; performing apodization processing on the multiple confocal sub - images by multiplying by a Hanning window, and performing convolution on the multiple confocal sub - images with a Gaussian kernel to remove the noise of the multiple confocal sub - images, obtaining multiple confocal denoised sub - images; aligning and superimposing the multiple confocal denoised sub - images with the central confocal denoised sub - image to obtain an initial reconstructed image; performing up - sampling processing on the initial reconstructed image according to the size of the initial image to obtain a dual - confocal image scanning microscopy image.

[0076] The ISM mode requires multi - focus excitation mode illumination and scanning of the sample. Compared with using a pinhole array and galvanometer to generate and scan multi - foci, DMD provides a more stable step. The DMD mask used to generate multi - focus illumination is designed as a periodic lattice, and each lattice element consists of a square aperture formed by 4×4 open - state DMD pixels. Considering that the size of each DMD pixel is 5.4 microns, under a 100 - fold objective lens, the size of the excitation light spot on the sample plane is approximately 220 nanometers. The multi - focus illumination pattern moves two DMD pixels each time, which is equivalent to a 108 - nanometer step on the sample plane, achieving a sampling rate twice that of the diffraction limit and performing multi - focus scanning on the sample. Since a turntable was previously used to remove out - of - focus light signals, in the multi - focus illumination here in the present invention, the distance between the foci can be closer without worrying about crosstalk. Therefore, the present invention controls the interval between the lattice elements to be 12 pixels, that is, the side length of the square and the distance between each other is 4:12, which can further reduce the number of original frames required for image scanning microscopic reconstruction.

[0077] The reconstruction of traditional ISM is generally achieved by computational or optical means, either by shrinking the light spot to 1 / 2 or by expanding the spacing between light spots to twice the original. This process of shifting the signals recorded by the detector back to their most likely generation positions is called Pixel Reassignment (PR). However, the scaling factor of 1 / 2 here is based on the ideal situation where the excitation point spread function and the detection point spread function of the system are exactly the same, Stokes uniqueness is ignored, and there is no distortion. The Adaptive Pixel Reassignment (APR) algorithm uses a 5×5 single-photon avalanche diode detector array to record the fluorescence signals emitted when a single focus scans the sample. When the sampling condition is met, that is, the scanning step size is equal to the single-point detector interval, each SPAD will record a sub-image of the same sample area. By performing cross-correlation registration to align the sub-images in the spatial domain, adaptive scaling can be achieved instead of a constant 2-fold scaling. Finally, by superimposing the aligned sub-images, an ideal ISM image can be obtained, and a 2-fold resolution improvement can be achieved through further deconvolution. However, compared with systems based on scanning and de-scanning configurations, when the multi-focus excitation mode scans in this system, the light spot moves on the camera target surface while the sample remains stationary. Therefore, the present invention specifically proposes the DPA-PR algorithm to achieve super-resolution reconstruction of ISM.

[0078] With the scanning step size as the detector interval, a 5×5 Virtual Detector Array (VDA) is constructed. Since multi-focus excitation is used, this means that a VDA should be assumed to exist at each focus. The gray value detected by each VDA element is interpolated from the actual gray value detected by the camera. By separately extracting the values of different VDA elements and arranging them according to the light spot position and scanning order, 25 sub-images can be obtained. Theoretically, the object information contained in different sub-images is the same, except for the overall spatial position shift and signal-to-noise ratio difference. The specific reconstruction process is divided into three steps: 3.1 Excitation optical axis positioning; 3.2 VDA sampling; 3.3 Cross-correlation registration and superposition. Finally, a two-fold resolution improvement can be achieved through further deconvolution.

[0079] 3.1 Excitation optical axis positioning.

[0080] The position of the excitation optical axis is the center of the illumination light spot. However, due to the modulation effect of the sample structure on the excitation light spot, the position of the maximum gray value in the obtained illumination light spot image may not be consistent with the actual excitation optical axis. Based on the prior knowledge that the excitation optical spot follows a square lattice distribution, the present invention can obtain the basis vectors of the lattice in the spatial domain and (ideally, ),(indicating the horizontal and vertical directions respectively, the distance between the two ringing light spots. By using these vectors to generate lattice points and calculating the average deviation between the generated lattice points and the brightest illumination point, the offset vector can be obtained.) Vector , and can be used to determine the exact position of the excitation light axis array. The specific process is as follows: (1) DC suppression preprocessing: Perform Fourier transform on each pixel of the three-dimensional image stack along the scanning direction. Since the background stray signal is relatively uniform on the macroscopic scale and does not change with the movement of the excitation light spot, it exhibits DC characteristics. By eliminating the zero-frequency component, the background of the acquired image can be further removed.)

[0081] (2) Determine and : Since the Fourier spectrum of the dot matrix image in the spatial domain is still a dot matrix, and the translation of the dot matrix in the spatial domain does not change the corresponding position in the frequency domain, the present invention performs Fourier transform on the original image and superimposes their spectra. Assuming that the basis vectors in the spectral grid are and , the coordinates of the spectral order points are: .

[0082] In the formula represents the coordinates in the spectrum, represents the order. By locating the local maximum value, the present invention determines the coordinates of a series of lattice points, and their corresponding orders . Then, by the least squares method, and are obtained: .

[0083] (3) Determine : Use and to generate a lattice . Ideally, the point spacing of this lattice should be equal to the point spacing of the excitation light axis array, but there is a deviation in their spatial positions, which is represented by . The present invention takes the spatial local maximum point of the light spot as a reference, calculates the offset vector from the nearest lattice point, and takes the average value to eliminate the influence of sample modulation on the excitation light spot, obtaining . Each frame has its own . The present invention uses to represent the of the k-th frame.)

[0084] 3.2 VDA sampling.)

[0085] It is assumed in the present invention that there is a 5×5 VDA, the center of which is located on the excitation optical axis and moves synchronously with the excitation dot matrix scanning. The spacing between adjacent VDA pixels is equal to the scanning step of the excitation lattice. The image is resampled by the VDA to obtain the pixel values detected at the scanning positions of 25 sub-images. The key steps are as follows.

[0086] (1) Determine the spacing of the VDA: Taking an 8×8 scan as an example, the pattern loaded into the DMD is scanned line by line. The calculation of the spacing value is based on . First, similar to phase unwrapping, there may be a difference in the basic vectors between vectors, so adjustment is required . Then, the rows and columns of the 8×8 cells are averaged, and only the x-component and y-component are taken respectively to obtain and . and The increments of are the spacings of the VDA elements and , which can be obtained by linear regression.

[0087] .

[0088] Among them, and are the spacings of the elements in the row direction and column direction of the virtual detection array respectively; k is the first-order coefficient of linear regression, and its least-squares estimated value is the spacing of the virtual detection array; b is the constant-term coefficient of linear regression; is a constant vector; and are the average values of the cells of the deviation vector between the lattice point spacing and the point spacing of the excitation optical axis array along the row direction and column direction.

[0089] (2) Determine the gray value sampled by the VDA: The spacing of the VDA pixels is not equal to the size of the camera pixels. It is assumed that the image gray value distribution follows a discrete function , where represents the discrete camera pixel coordinates. The present invention constructs the VDA pixel coordinates relative to the VDA pixel coordinates. By resampling and calculating each VDA position, the gray value sampled by the VDA can be obtained.

[0090] 3.3 Cross-correlation registration and superposition.

[0091] After 5×5 VDA sampling, 25 confocal sub-images can be obtained. Then, the DPA-PR reconstruction result is obtained by aligning and superimposing the sub-images.

[0092] (a) Registration: First, the image is apodized by multiplying with a Hann window and convolved with a Gaussian kernel to remove noise. Then, phase cross-correlation is performed between the k-th edge sub-image and the central sub-image (i.e., k = 13 when using a 5×5 VDA) to obtain the k-th shift value : .

[0093] Wherein, and represent the Fourier transform and the inverse Fourier transform respectively. denotes the Hadamard product, and the superscript asterisk * denotes the complex conjugate.

[0094] (b) Superposition and post-processing: After moving the sub-image to align it with the central sub-image, the reconstruction result is obtained by directly superposing the sub-images, and its pixel size is equal to the scanning step. Then, the reconstruction result is adjusted to the original image size by upsampling. Due to reasons such as computational precision error, tilted DMD position, and aberration, and there may be slight differences between them. In addition, there are also differences between the cumulative displacement after moving 8 steps or (i.e., scanning a full distance along the fast axis direction or the slow axis direction) and or (the actual excitation lattice grid spacing). These deviations are corrected by scale-invariant feature transform.

[0095] To achieve high-fidelity reconstruction of ISM, the present invention has developed the DPA-PR algorithm. However, the data collected by the present invention can also be reconstructed based on the algorithms of PR and multi-image deconvolution, but the linear correlation, structural similarity and other indicators of the reconstruction result with respect to the original image will become worse, that is, the image fidelity is reduced. During the DPA-PR reconstruction process, the size of the VDA is not necessarily 5×5, and it can also be 3×3 or 7×7. However, 3×3 will cause signal loss at the edges of some light spots, and for a 7×7 VDA, the peripheral pixels can hardly detect the optical signal, and it will also cause an increase in computational overhead and a slowdown in the reconstruction speed.

[0096] Step 102: Implement high-depth super-resolution projection SIM imaging; including: loading a stripe mask for projection SIM imaging to achieve seamless switching of multiple imaging modalities; performing phase shift on the stripes during projection SIM imaging to achieve uniform illumination of the field of view; reconstructing the SIM image based on operations in the Fourier domain; notching the high-order points that cause artifacts in the reconstructed spectrum to obtain the SIM reconstruction result.

[0097] Benefiting from the flexible wavefront modulation of the DMD, this system is also compatible with other super-resolution modes. By loading a stripe mask, projection SIM imaging can be performed to achieve seamless switching between multiple imaging modalities. The period of the stripe mask is 8 DMD pixels, and the corresponding stripe period on the sample surface under a 100× objective is 432 nm. Under 640 nm excitation light, a 1.6-fold spread spectrum can be achieved. Compared with 2D-SIM that forms excitation stripes by interference, which can achieve a spread spectrum of nearly 2 times, projection SIM is limited by the attenuation of the optical system MTF for high-frequency signals. When the stripe period is too small, the visibility drops rapidly, and its improvement in resolution is limited. However, at the same time, due to the stripes formed by projection being within the sample volume and only visible near the focal plane and attenuating as it moves away from the focal plane, it can provide better sectioning ability.

[0098] SIM imaging requires and phase shifts of the stripe to achieve uniform illumination of the field of view. However, this requires that the DMD period is at least 6 pixels or 12 pixels. However, the spatial frequency of the stripes formed by a 6-pixel period is 0.8 times the cut-off frequency, and the visibility is reduced to an unacceptable level after passing through the optical system. A 12-pixel period can only provide a 1.3-fold spread spectrum, with limited resolution improvement. Therefore, in the present invention, the phase shift of the stripe is achieved by changing 2 DMD pixels each time, corresponding to and phase shifts. The present invention reconstructs the SIM image based on Fourier domain operations: .

[0099] In the formula, is the spectrum of the acquired image, is the spectrum of the sample distribution, is the optical transfer function of the system, is the frequency coordinate. Let take , and respectively. represents the spatial frequency of the illumination mode, and its subscript represents the direction of the stripe spatial frequency. is the first-order frequency multiplication of the sample along the direction. Subsequently, in the reconstructed spectrum, notch filtering is performed on the high-order points that cause artifacts to obtain the final SIM reconstruction result.

[0100] Furthermore, the imaging method of the dual confocal image scanning microscope based on a turntable provided in this embodiment further includes: achieving adaptive illumination of the sample.

[0101] In the traditional imaging process, the sample area within the imaging field of view is illuminated equally. By modulating the excitation light using a DMD, adaptive illumination can be performed according to the sample structure, which not only reduces the overall illumination dose, decreases phototoxicity and photobleaching, and extends the sample observation time, but also finely adjusts the light dose of each pixel to enable the acquired image to reach the specified signal-to-noise ratio.

[0102] The process of coordinate registration is as shown in 2.1, and the spatial transformation relationship between the DMD coordinate system and the camera coordinate system can be obtained. Subsequently, all pixels of the DMD are turned on, and the power of the illumination light source is lowered for rapid shooting to obtain a snapshot image. Perform transformation to obtain the image in the DMD coordinate system. : .

[0103] Subsequently, based on Calculate the gray mask. . According to different requirements of adaptive illumination, it can be divided into three modes.

[0104] Mode 1: Only illuminate the area with samples, and do not illuminate the background area. At this time, the gray mask is actually a binary mask, and the present invention needs to set a binary hard threshold according to the imaging situation . The part with a gray value higher than is the sample area, and the part with a gray value lower than is the background area: .

[0105] Mode 2: Provide high-intensity illumination for areas with high fluorescence signals and low-intensity illumination for areas with low fluorescence signals to further improve the imaging contrast. Here, the present invention uses a linear mask, that is, and only have a linear change in gray value from a 16-bit image to an 8-bit image.

[0106] .

[0107] Among them, represents element-wise division, and represents rounding to the nearest integer.

[0108] Mode 3: Provide low-intensity illumination for areas with high fluorescence signals and high-intensity illumination for areas with low fluorescence signals, so that the signal-to-noise ratio (SNR) of the entire image is more uniform, and the possibility of imaging the sample for a longer time is obtained by sacrificing the SNR of the bright area of the image. Here, the present invention uses a non-linear mask, and have an inverse proportional relationship in gray value. However The intensity of the background area illumination light cannot be infinitely large, so a lower limit needs to be set , and pixels with gray values lower than are considered as the background and are directly set to 0.

[0109] .

[0110] To prevent the pixel values from being higher than the upper limit of an 8-bit image, it is required that .

[0111] Thanks to the fast response of the DMD and the efficient and fast gray-scale mask generation algorithm, the illumination mode and intensity can be adjusted in real time according to the acquired image information.

[0112] The use of high-power LEDs or multimode lasers in the imaging light source assembly 10 can effectively remove speckle artifacts and provide a large illumination field of view. It can shorten the exposure time when the power is sufficient.

[0113] The introduction of the turntable assembly 50 effectively removes defocus signals. Compared with the wide-field imaging results without a turntable, the confocal results after introducing the turntable are clearer ( Figure 5 ).

[0114] The structured light generation assembly 40 uses a digital micromirror device to generate a multi-focus illumination mode. Compared with using a pinhole array and a galvanometer to generate and scan multiple foci, the DMD provides a more stable step. And the DMD is low-cost and easy to operate. The combination of the turntable assembly 50 and the digital micromirror device 42 enables the distance between foci to be closer in multi-focus illumination without worrying about crosstalk. The interval between lattice elements can be controlled to be 12 pixels, that is, the side length of the square and the distance between each other is 4:12. This can further reduce the number of original frames required for image scanning microscopic reconstruction, and only 36 frames are needed for ISM reconstruction.

[0115] The introduction of the turntable assembly 50 ensures that in the ISM imaging process, the multi-focus illumination is clearly visible and will not be masked by the background, and the corresponding spectral feature points are also more obvious, which is beneficial to the determination of the basic vector when the DPA-PR algorithm determines the excitation optical axis ( Figure 6 ).

[0116] Based on the principle of ISM, using the DPA-PR algorithm to perform super-resolution reconstruction on the image can effectively improve the resolution. On tissue samples, compared with wide-field and confocal, the multi-confocal image scanning microscope significantly removes defocus signal interference, and the resolved structure is clearer. The lateral resolution reaches 150 nanometers respectively, achieving a two-fold improvement in three-dimensional resolution ( Figure 7 ).

[0117] Compared with the traditional PR algorithm, the DPA-PR algorithm can achieve better reconstruction result fidelity. Its reconstructed similarity graph and peak signal-to-noise ratio are better than those of the PR reconstruction result, and the coefficient of determination is as high as 92%, which proves that the gray level is relatively well maintained during the reconstruction process and quantitative imaging can be performed ( Figure 8 ).

[0118] The introduction of the turntable assembly 50 ensures that during the SIM imaging process, the striped structured light can penetrate deep into the tissue sample, achieving a depth imaging of 50 microns, exceeding that of traditional SIM. When there is no turntable, the stripes are invisible and no clear spectrum can be seen in the frequency domain. After the turntable is introduced, the stripes are visible and the first-order frequency of the stripes is obvious. There is also an obvious resolution improvement in the SIM reconstruction compared with confocal microscopy ( Figure 9 ).

[0119] The one-dimensional galvanometer 41 can effectively correct the diffraction effect of the digital micromirror device 42, achieve efficient imaging at different wavelengths, and make the system compatible with multi-color imaging ( Figure 10 ).

[0120] The present invention designs a dual confocal image scanning microscopy method based on a turntable for three-dimensional thick sample super-resolution and high-fidelity microscopy imaging. The system consists of a uniform illumination module composed of a multimode laser, a multimode homogenizing optical fiber, and an achromatic doublet lens, a structured light generation module composed of a one-dimensional galvanometer and a digital micromirror device (DMD), a scanning module composed of a high-speed rotating turntable, and a detection module. The uniform illumination module uses a multimode laser in cooperation with a multimode homogenizing optical fiber to decohere the light source and provide high-energy, speckle-free, and uniform excitation light; the structured light generation module uses the DMD as a projection device to achieve structured light illumination of the sample surface by loading a specific mask sequence to achieve super-resolution imaging. At the same time, the DMD is a periodic structure formed by an array of micromirrors and will exhibit a diffraction effect similar to a two-dimensional grating. There are different diffraction efficiencies and exit angles for excitation light with different incident angles and wavelengths. Therefore, a one-dimensional galvanometer is used to control the angle of incidence of excitation light with different wavelengths on the DMD target surface to achieve high-efficiency multi-color imaging. The high-speed rotating turntable is placed behind the dichroic mirror, and the pinholes on it are arranged equidistantly according to the Archimedean spiral, and the high-speed rotation realizes multi-point parallel scanning and descanning; the detection module uses a area array detector for detection. The system can achieve an imaging depth greater than 180 microns, a resolution improvement of twice the diffraction limit, and an image linearity of 92%, with good quantitative characteristics, and is suitable for high-fidelity super-resolution microscopy imaging in application scenarios that require high depth and high resolution. In addition, the illumination of the system is realized through a programmable DMD. Therefore, in addition to the ISM mode, the system can also realize structured illumination microscopy (SIM) and adaptive illumination mode ( Figure 1 ). For the imaging of the projection SIM implemented by this system, the depth extension can reach 50 microns, which greatly exceeds the traditional SIM based on interference to form fringes; while adaptive illumination can selectively illuminate the region of interest without introducing additional light dose, with low phototoxicity and improving the ability of long-term imaging of living cells. These functions make the present system a multi-functional tool, promoting cell imaging and tissue-scale exploration and meeting the needs of modern biological imaging.

[0121] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same and similar parts among the various embodiments, reference can be made to each other.

[0122] In this article, specific examples are used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and application scope. In summary, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A dual confocal image scanning microscope based on a rotating disk, characterized in that: include: A signal control component, and an imaging light source component, an optical fiber transmission component, a light source collimation component, a structured light generation component, a turntable component, a micro-magnification component, and a camera detection component arranged along the optical path; The imaging light source assembly is used to output laser; the optical fiber transmission assembly is used to homogenize the laser and input the homogenized laser into the light source collimation assembly; the light source collimation assembly is used to collimate the laser output by the optical fiber transmission assembly; the structured light generation assembly is used to generate and scan structured light based on the laser output by the light source collimation assembly; the turntable assembly is provided with a pinhole array distributed along multiple clusters of concentric Archimedean spirals; a plurality of pinholes in the pinhole array are arranged at equal intervals along multiple clusters of concentric Archimedean spirals; the rotation speed of the turntable assembly is adjusted by a three-phase brushless DC motor controller in a signal control assembly; the microscope magnification assembly is used to realize the nominal magnification of the microscope objective and the clamping of the sample, reflect the structured light as the excitation light to the rotating turntable assembly, irradiate the sample with the excitation light of the pinhole to generate fluorescent emission light, and transmit the fluorescent emission light through the pinhole to the camera detection assembly; the camera detection assembly is used to realize the detection and output of the fluorescent signal; The signal control component is used to control the imaging light source component, the structured light generation component, the turntable component and the camera detection component to work synchronously; The signal control component is also used to obtain the detection result and use the detection result as the original image; Based on the original image, imaging reconstruction is achieved using a dynamic pinhole array-pixel reallocation method; The dynamic pinhole array-pixel reallocation method comprises: performing Fourier transform on a three-dimensional image stack pixel by pixel along a scanning direction, and performing DC suppression preprocessing on the original image; performing Fourier transform on the original image after DC suppression preprocessing and superimposing a spectrum, and determining a basis vector in the spectrum grid using a spectrum grid basis vector formula; generating a lattice based on the basis vectors in the spectrum grid; calculating an offset vector between a lattice point spacing and an excitation light axis array point spacing with a spatial local maximum point of a light spot as a reference; locating the excitation light axis based on the offset vector; setting a virtual detection array with the excitation light axis as the center, so that the virtual detection array moves synchronously with the excitation point array scanning to complete the virtual detection array sampling, and obtain multiple confocal sub-images; performing toe-cutting processing on the multiple confocal sub-images by multiplying them by a Haining window, and performing convolution on the multiple confocal sub-images with a Gaussian kernel to remove the noise of the multiple confocal sub-images, and obtain multiple confocal denoised sub-images; aligning and superimposing the multiple confocal denoised sub-images with the central confocal denoised sub-image to obtain an initial reconstructed image; performing upsampling processing on the initial reconstructed image according to the size of the initial image, and obtaining a dual confocal image scanning microscopic imaging.

2. The dual confocal image scanning microscope based on a spinning disk according to claim 1, characterized in that: The imaging light source assembly includes a high-power single-mode laser, a high-power multi-mode laser or a high-power LED; In multi-color imaging, the imaging light source group combines the light beams using an all-in-one optical fiber or by using a dichroic mirror.

3. The dual confocal image scanning microscope based on a spinning disk according to claim 1, characterized in that: The light source collimating component is an aspheric lens, a 90° off-axis parabolic reflector, an aspheric mirror with adjustable focal length, an achromatic doublet lens or an air-spaced doublet lens.

4. The dual confocal image scanning microscope based on a spinning disk according to claim 1, characterized in that: The structured light generating assembly comprises: a one-dimensional galvanometer, a digital micromirror device, a first lens and a second lens; The one-dimensional galvanometer is used to control the angle at which the laser output by the light source collimation assembly is incident on the target surface of the digital micromirror device; The digital micromirror device is used to generate and scan structured light based on the light beam output by the one-dimensional galvanometer, and output the one-dimensional galvanometer to the first lens; The second lens is used to transmit the structured light output by the first lens to the microscope magnification component.

5. The dual confocal image scanning microscope based on a spinning disk according to claim 1, characterized in that: The microscope magnification assembly includes: a dichroic mirror, a tube lens, a microscope objective lens and a sample clamping assembly arranged along the optical path; The sample clamping assembly is used to clamp the sample; The dichroic mirror is used to reflect the structured light as excitation light to the rotating turntable assembly; The tube lens is used to receive the excitation light passing through the pinhole and output the excitation light passing through the pinhole to the microscope objective lens; The microscope objective is used to receive the excitation light output by the tube lens through the pinhole, output the excitation light through the pinhole to the sample, and receive the fluorescent emission light formed by the sample being irradiated by the excitation light through the pinhole, and output the fluorescent emission light to the tube lens; The tube lens is also used to receive the fluorescent emission light output by the tube lens, and output the fluorescent emission light to the rotating turntable assembly; The dichroic mirror is also used to transmit the fluorescent emission light through the pinhole to the camera detection assembly.

6. The dual confocal image scanning microscope based on a spinning disk according to claim 1, characterized in that: The camera detection assembly includes: a relay lens and a camera arranged along the optical path; The relay lens is used to achieve conjugate imaging; The camera is used to detect and output the fluorescence signal.

7. The dual confocal image scanning microscope based on a spinning disk according to claim 6, characterized in that: The relay lens can be replaced by a SLR lens; the SLR lens is used to achieve large-field proportional conjugate imaging.

8. An imaging method using a dual confocal image scanning microscope based on a spinning disk, characterized in that: include: Acquire a detection result using a rotating disk-based dual confocal image scanning microscope as described in any one of claims 1 to 7, and use the detection result as an original image; Based on the original image, imaging reconstruction is achieved using a dynamic pinhole array-pixel reallocation method.

9. The imaging method using a spinning disk-based dual confocal image scanning microscope according to claim 8, characterized in that: The method of realizing imaging reconstruction based on the original image by using a dynamic pinhole array-pixel redistribution method comprises: Fourier transform is performed pixel by pixel on the three-dimensional image stack along the scanning direction, and DC suppression preprocessing is performed on the original image; Perform Fourier transform on the original image after DC suppression preprocessing and superimpose the spectrum, and use the spectrum grid basis vector formula to determine the basis vector in the spectrum grid; Generate a lattice based on basis vectors in the spectral grid; Taking the local maximum point of the light spot as a reference, calculate the offset vector between the lattice point spacing and the excitation light axis array point spacing; Positioning the excitation light axis based on the offset vector; A virtual detection array is set with the excitation optical axis as the center, and the virtual detection array is moved synchronously with the excitation dot matrix scanning to complete the virtual detection array sampling and obtain multiple confocal sub-images; A plurality of confocal sub-images are apodized by multiplying them by a Hening window, and the noise of the plurality of confocal sub-images is removed by convolving the plurality of confocal sub-images with a Gaussian kernel to obtain a plurality of confocal denoised sub-images; Aligning and superimposing the multiple confocal denoised sub-images with the central confocal denoised sub-image to obtain an initial reconstructed image; According to the size of the initial image, the initial reconstructed image is upsampled to obtain a dual confocal image scanning microscopy imaging.

10. The imaging method using a spinning disk-based dual confocal image scanning microscope according to claim 9, characterized in that: The imaging method using a spinning disk-based dual confocal image scanning microscope also includes: Loading stripe masks for projection SIM imaging enables seamless switching between multiple imaging modes; when performing projection SIM imaging, the stripes are phase-shifted to achieve uniform illumination of the field of view; Reconstruct the SIM image based on Fourier domain operations; The high-order points that cause artifacts in the reconstructed spectrum are notched to obtain the SIM reconstruction result.

Images