Microscopic imaging device and super-resolution microscopic imaging method

Abstract

The invention discloses a microscopic imaging device and a super-resolution microscopic imaging method, and the device comprises a sample stage which is used for containing a sample; the laser is used for emitting laser; the first digital micromirror is used for receiving the laser and generating diffracted light; the light condensing unit is used for condensing the plurality of diffracted light to form structured light so as to excite fluorescence carrying sample information; the second digital micromirror is used for receiving the fluorescence and converting the fluorescence into a random binary coding image; the image acquisition unit is used for receiving images and transmitting the images to the processing unit; the processing unit is used for reconstructing a microscopic super-resolution image based on the plurality of random binary coding images; the first digital micromirror, the second digital micromirror and the image acquisition unit are in communication connection, and the second digital micromirror controls the grating pattern conversion frequency of the first digital micromirror and controls the image acquisition frequency of the image acquisition unit. The biological cell sample can be protected from being damaged by phototoxicity and photobleaching performance, and super-resolution imaging in the structured illumination obvious microsystem is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of optical imaging equipment, and in particular to a microscopic imaging device and a super-resolution microscopic imaging method. Background Art

[0002] In the field of modern biomedicine, the research scale of nanoscale biological samples such as animal and plant organelles, neurons, and bacteria is advancing step by step. Optical microscopes with super-resolution capabilities, as an important observation tool, play a decisive role in related research work. Currently, common super-resolution microscopy techniques include structured illumination microscopy super-resolution, stimulated emission depletion microscopy, random optical reconstruction microscopy, and single-molecule localization microscopy.

[0003] The above technology requires active illumination light to illuminate the biological cell samples, which will cause obvious photobleaching and phototoxicity to living cells, so that during the observation of active biological cells, the cells will be killed quickly, and the life activities of biological cells cannot be observed for a long time. If the illumination light intensity is reduced, the light information carrying the biological cells will be too weak to capture images with a costly dark light camera; or an ordinary camera can be used for long exposure acquisition. However, due to the influence of dark current noise of ordinary cameras, long exposure acquisition will result in the signal-to-noise ratio of the captured image being too low, thus losing the significance of the experiment. Summary of the invention

[0004] In order to solve the deficiencies of the prior art, this application adopts the following technical solutions: In a first aspect, the present application provides a microscopic imaging device, the microscopic imaging device comprising: A sample table, used to hold samples; Laser, used for emitting laser light; A first digital micromirror is configured to receive the laser light and form a grating image to modulate the laser light to generate diffracted light of multiple diffraction orders; A focusing unit is configured to converge the plurality of diffracted lights generated by the first digital micromirror, so that the diffracted lights interfere with each other to form structured light, and focus the structured light on the sample to excite fluorescence carrying sample information; a second digital micromirror, receiving the fluorescence and converting the fluorescence into a random binary coded image; A cylindrical lens, the cylindrical lens being disposed in the optical path between the second digital micromirror and the image acquisition unit, and being used for converging the random binary coded image into a light beam distributed along a preset direction; An image acquisition unit, comprising a line array camera, wherein the line array camera is disposed on a convergence surface of the cylindrical lens, and the line array camera is configured to receive the random binary coded image converged by the cylindrical lens, and transmit the random binary coded image to a processing unit; A processing unit, wherein the processing unit is configured to reconstruct a microscopic super-resolution image based on a plurality of random binary coded images; The first digital micromirror, the second digital micromirror and the image acquisition unit are communicatively connected, and the second digital micromirror sends control instructions to the first digital micromirror and the image acquisition unit respectively to control the grating pattern conversion frequency of the first digital micromirror and the image acquisition frequency of the image acquisition unit.

[0005] In summary, the present application provides a microscopic imaging device, which modulates laser into multi-order diffraction light through a digital micromirror and a focusing unit, and makes the diffraction light interfere with each other to form structured light. The structured light converges on the sample to excite a fluorescent signal carrying sample information, and the fluorescence is converted and encoded to obtain a random binary coded image. Finally, by reconstructing the random binary coded image, the super-resolution imaging effect in the structured light illumination microscopy system is achieved; and under low light intensity conditions, the loss of light energy in the optical system is reduced, protecting biological cell samples from phototoxicity and photobleaching.

[0006] Furthermore, the first digital micromirror is configured to sequentially play 9 grating patterns to generate different diffracted lights, wherein the 9 grating patterns include three grating directions of 0°, 60°, and 120°, and each grating direction includes 0, , Three phases.

[0007] Further, the first digital micromirror is configured to modulate the laser to generate three diffraction orders of 0, +1, and -1; The microscopic imaging device further includes a filter configured to filter out 0th order diffraction light and allow +1st order diffraction light and -1st order diffraction light to pass.

[0008] Further, the microscopic imaging device further comprises a dichroic mirror, and the dichroic mirror is configured to reflect the +1-order diffraction light and the -1-order diffraction light to the focusing unit; The focusing unit converges the +1st order diffraction light and the -1st order diffraction light to generate interference, thereby forming the structured light.

[0009] Further, the refresh frequency of the random binary coded image of the second digital micromirror is defined as N, where N is a natural number greater than 1. Then, the refresh frequency of the random binary coded image of the second digital micromirror is N times the grating pattern transformation frequency of the first digital micromirror, and the refresh frequency of the random binary coded image of the second digital micromirror is the same as the image acquisition frequency of the image acquisition unit.

[0010] Further, the second digital micromirror is configured to: in response to refreshing the random binary coding pattern once, send a control instruction to the image acquisition unit to trigger the image acquisition unit to perform image acquisition; in response to refreshing the random binary coding pattern N+1 times, reset the refresh count calculation of the current round of the random binary coding pattern, and send a control instruction to the first digital micromirror to trigger the first digital micromirror to perform grating pattern transformation.

[0011] Furthermore, the processing unit is configured to: use a deep learning image reconstruction algorithm to reconstruct a plurality of the random binary encoded images to obtain an image with a structured light illumination pattern, and perform microscopic super-resolution reconstruction on a plurality of the images with the structured light illumination pattern to obtain the microscopic super-resolution image.

[0012] Furthermore, the deep learning image reconstruction algorithm is used to reconstruct the plurality of random binary coded images, including: grouping the continuously acquired measurement data according to the target frame rate, and constructing an imaging model, wherein the imaging model is represented by the following formula: ; In the formula, is the scene frame associated with the desired frame rate, , Q is a constant greater than 1, T represents time, represents the measurement noise, represents the measurement matrix, Represents multiple reconstructed large-size image information containing structured light illumination information.

[0013] Further, the 3M images with structured light illumination patterns are Fourier transformed to obtain the frequency spectrum of each image, where M is 1, 2, or 3; The original image in the spectrum of each image is obtained using a preset window function, the spectrum of each original image is extracted, and the spectrum of the extracted original image is translated to the exact center of the entire spectrum, a new spectrum is synthesized, and an inverse Fourier transform is performed on it to obtain the microscopic super-resolution image.

[0014] In a second aspect, the present application further provides a super-resolution microscopic imaging method, the method using the above-mentioned microscopic imaging device, the method comprising: A deep learning image reconstruction algorithm is used to reconstruct a plurality of random binary coded images to obtain an image with a structured light illumination pattern, and a plurality of images with the structured light illumination pattern are microscopically super-resolved to obtain the microscopic super-resolved image. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic diagram of the structure of a microscopic imaging device provided in one embodiment of the present application; Figure 2 A schematic diagram of the optical path of a microscopic imaging device provided in one embodiment of the present application; Figure 3 A schematic diagram of signal frequencies of a first digital micromirror, a second digital micromirror, and an image acquisition unit of a microscopic imaging device provided in one embodiment of the present application; Figure 4 A flowchart of the steps of a super-resolution microscopy method provided in one embodiment of the present application. DETAILED DESCRIPTION

[0016] The present application will be described in detail below in conjunction with the specific implementation modes shown in the accompanying drawings, but these implementation modes do not limit the present application. Structural, methodological, or functional changes made by ordinary technicians in the field based on these implementation modes are included in the protection scope of the present application.

[0017] In order to solve the shortcomings of the prior art, firstly, Figure 1 As shown, the present application provides a microscopic imaging device 100, which includes: a sample stage 11, a laser 12, a first digital micromirror 13, a focusing unit 14, a second digital micromirror 15, a cylindrical lens 26, an image acquisition unit 16 and a processing unit 17.

[0018] The sample stage 11 is used to hold the sample, and the laser 12 is used to emit laser light; the first digital micromirror 13 is configured to receive the laser light and form a grating image to modulate the laser light to generate diffraction light of multiple diffraction orders; the focusing unit 14 is configured to converge the multiple diffraction lights generated by the first digital micromirror 13, so that the diffraction lights interfere with each other to form structured light, and converge the structured light on the sample to excite the fluorescence carrying the sample information; the second digital micromirror 15 receives the fluorescence and converts the fluorescence into a random binary coded image; the cylindrical lens 26 is arranged in the optical path between the second digital micromirror 15 and the image acquisition unit 16, and the cylindrical lens 26 is used to converge the random binary coded image into light distributed along a preset direction; the image acquisition unit 16 includes a linear array camera, which is arranged on the convergence surface of the cylindrical lens 26, and the image acquisition unit 16 is configured to receive the random binary coded image and transmit the random binary coded image to the processing unit 17; the processing unit 17 is configured to reconstruct a microscopic super-resolution image based on a plurality of random binary coded images.

[0019] Among them, the first digital micromirror 13, the second digital micromirror 15 and the image acquisition unit 16 are communicatively connected, and the second digital micromirror 15 sends control instructions to the first digital micromirror 13 and the image acquisition unit 16 respectively to control the grating pattern conversion frequency of the first digital micromirror 13 and the image acquisition frequency of the image acquisition unit 16.

[0020] Specifically, Figure 2 As shown, the sample stage 11 is used to fix the biological sample to ensure the stability of the sample during imaging and to avoid the sample displacement resulting in blurred imaging. The protein in the biological sample is stained and marked with fluorescence.

[0021] The laser 12 serves as a light source to emit a laser of a specific wavelength. The laser emitted by the laser 12 is received by the first digital micromirror 13. The laser is used to optically excite the fluorescently dyed protein in the biological sample, thereby stimulating the fluorescent signal. Furthermore, the laser emitted by the laser 12 can be controlled within a low light range to reduce the phototoxicity to the living sample and reduce the photobleaching effect. Optionally, a beam expander 18 and a first reflector 19 are provided between the laser 12 and the first digital micromirror 13. The laser emitted by the laser 12 is expanded and collimated by the beam expander 18, and then reflected by the first reflector 19 to reach the first digital micromirror 13.

[0022] Through the refraction of the laser by the first reflector 19, the first digital micromirror 13 receives the laser after expansion and collimation, and the first digital micromirror 13 spatially modulates the laser by dynamically loading a grating pattern. The first digital micromirror 13 can decompose a single laser beam into diffraction lights of multiple diffraction orders by quickly switching grating patterns of different directions and phases. Subsequently, the diffraction light generated by the first digital micromirror 13 is reflected to the focusing unit 14 through the second reflector 21.

[0023] Through the refraction of the diffracted light by the second reflector 21, the focusing unit 14 receives the diffracted light generated by the first digital micromirror 13, and the focusing unit 14 converges the multiple diffracted lights so that the multiple diffracted lights interfere with each other, thereby forming a variety of structured lights with different directions and phases. The focusing unit 14 is located above the sample stage 11, and the structured light converges on the biological sample on the sample stage 11. The structured light excites the protein marked with fluorescent dye in the biological sample, thereby generating a fluorescent signal carrying the biological sample information. Optionally, a lens 22 can be arranged between the second reflector 21 and the focusing unit 14. The focusing unit 14 receives the diffracted light refracted by the second reflector 21 through the lens 22. The lens 22 can converge the divergent or parallel diffracted light beams to a specific focal position, avoid light energy dispersion, and improve the intensity and contrast of the subsequent interference structured light. In the embodiment of the present application, the focusing unit 14 is configured as a microscope objective.

[0024] The second digital micromirror 15 is configured to receive a fluorescent signal carrying biological sample information, and the second digital micromirror 15 is loaded with a random binary coding pattern, and the fluorescent signal carrying the biological sample information is modulated by quickly refreshing the coding pattern, and the fluorescent signal is converted into a random binary coding pattern, providing raw material for the subsequent processing unit 17 to reconstruct a high-resolution image.

[0025] After the fluorescent signal carrying the biological sample information is modulated into a random binary coding pattern by the second digital micromirror 15, the cylindrical lens can converge the coded light signal into a linearly distributed optical fiber along a preset direction. In the embodiment of the present application, the coded random binary coded image converges into a horizontally distributed light after passing through the cylindrical lens 26, and is collected by a linear array camera placed on the convergence surface of the cylindrical lens 26. The linear array camera can collect signals with low noise and high frame rate. The coordinated application of the cylindrical lens 26 and the linear array camera overcomes the signal-to-noise ratio challenge under low light conditions and provides a high-quality raw data foundation for super-resolution imaging.

[0026] The processing unit 17 uses a reconstruction algorithm to reconstruct the image of the plurality of random binary coded patterns. By fusing the plurality of random binary coded patterns, it breaks through the limitation of traditional optical resolution and reconstructs an image with microscopic super-resolution, thereby achieving the super-resolution imaging effect of the microscopic imaging device 100.

[0027] Furthermore, the first digital micromirror 13, the second digital micromirror 15 and the image acquisition unit 16 are communicatively connected, and the second digital micromirror 15 sends a control signal to the first digital micromirror 13 and the image acquisition unit 16 at a certain control frequency, so as to respectively control the switching frequency of the grating pattern of the first digital micromirror 13 and the image acquisition frequency of the image acquisition unit 16, so as to meet different imaging requirements and enhance the flexibility of imaging.

[0028] According to the above description, a microscopic imaging device 100 provided in the present application modulates laser to form structured light through a digital micromirror and a focusing unit 14. The structured light is focused on the sample to excite a fluorescent signal carrying sample information, and the fluorescence is converted and encoded to obtain a random binary coded image. Finally, the random binary coded image is reconstructed, thereby achieving a super-resolution imaging effect in a structured light illumination microscopic system; and under low light intensity conditions, the loss of light energy in the optical system is reduced, protecting biological cell samples from phototoxicity and photobleaching.

[0029] As an implementation method, the first digital micromirror 13 is configured to play 9 grating patterns in time sequence to generate different diffracted lights, wherein the 9 grating patterns include three grating directions of 0°, 60°, and 120°, and each grating direction includes 0, , Three phases.

[0030] Specifically, the 9 grating patterns played by the first digital micromirror 13 include three grating directions of 0°, 60°, and 120° and three phases of 0, , The three grating directions are evenly distributed at intervals of 60°, thus covering a wider spatial frequency range in the frequency domain. Different grating directions form interference patterns of specific angles on the sample surface. By superimposing different directions, the response information of the sample at different spatial frequencies is collected, thereby synthesizing a super-resolution image during image reconstruction.

[0031] Furthermore, each grating direction includes three different phases. The phase change can cause the grating stripes to shift laterally in the sample plane, thereby capturing the response differences of the sample under different lighting conditions, obtaining the high-frequency information of the sample, providing more information for subsequent algorithm reconstruction, and improving the contrast and resolution of the reconstructed image.

[0032] As an implementation method, the first digital micromirror 13 is configured to modulate the laser to generate three diffraction orders of 0, +1, and -1; the first digital micromirror 13 forms a dynamic grating pattern by rapid flipping. When the laser irradiates the surface of the first digital micromirror 13, the periodic structure of the grating pattern spatially modulates the laser. According to the Fraunhofer diffraction theory, the periodic grating modulates the phase or amplitude of the incident light wave to form light beams of 0, +1, and -1 diffraction orders. Among them, the propagation direction of the 0th order diffracted light is consistent with the incident light, and the 0th order diffracted light does not carry the spatial frequency information of the grating modulation; the +1st and -1st order diffracted lights are generated by the periodic modulation of the grating, and are deflected to both sides at symmetrical angles, respectively, and the +1st and -1st order diffracted lights carry the spatial frequency information of the grating modulation.

[0033] The microscopic imaging device 100 further includes a filter 23, which is configured to filter out the 0th order diffraction light and allow the +1st order diffraction light and the -1st order diffraction light to pass through. Since the 0th order diffraction light does not carry the spatial frequency information of the grating modulation, the illumination of the 0th order diffraction light on the sample surface will cover the interference effect of the structured light. The filter 23 filters out the 0th order diffraction light and allows the +1st order diffraction light and the -1st order diffraction light to pass through, thereby avoiding the introduction of the 0th order diffraction light to affect the imaging contrast and make it impossible to achieve the reconstruction of the super-resolution image.

[0034] As an implementation method, the microscopic imaging device 100 also includes a dichroic mirror 24, which is configured to reflect the +1-order diffraction light and the -1-order diffraction light to the focusing unit 14; the focusing unit 14 converges the +1-order diffraction light and the -1-order diffraction light to generate interference and form structured light.

[0035] Specifically, the diffracted light modulated by the first digital micromirror 13 is refracted by the second reflector 21 and then passes through the lens 22 and the filter 23. The filter 23 filters out the 0-order diffracted light, and the remaining +1-order diffracted light and -1-order diffracted light are irradiated to the dichroic mirror 24. The dichroic mirror 24 is a reflector with a special coating. The +1-order diffracted light and -1-order diffracted light are refracted by the dichroic mirror 24, and the +1-order diffracted light and -1-order diffracted light are refracted to the focusing unit 14. The dichroic mirror 24 is tilted at a specific angle to ensure that the propagation paths of the refracted +1-order diffracted light and -1-order diffracted light are aligned with the optical axis of the focusing unit 14 to avoid light energy loss or light direction deviation.

[0036] The focusing unit 14 converges the +1st order diffraction light and the -1st order diffraction light to interfere with each other, forming 9 kinds of structured lights with different directions and phases. The structured light excites the fluorescent dye-labeled proteins in the biological sample. The fluorescent signal carrying the biological sample information is received by the second digital micromirror 15 through the dichroic mirror 24, and the second digital micromirror 15 converts the fluorescent signal carrying the biological sample information into a random binary coded image.

[0037] Furthermore, the microscopic imaging device 100 also includes a tube lens 25, which is arranged between the dichroic mirror 24 and the second digital micromirror 15. The fluorescent signal carrying the biological sample information passes through the dichroic mirror 24 and the tube lens 25 successively and is received by the second digital micromirror 15. The tube lens 25 can further avoid light energy loss or light direction deviation.

[0038] As an implementation method, the refresh frequency of the random binary coded image of the second digital micromirror 15 is defined as N, where N is z greater than 1. Then, the refresh frequency of the random binary coded image of the second digital micromirror 15 is N times the grating pattern transformation frequency of the first digital micromirror 13, and the refresh frequency of the random binary coded image of the second digital micromirror 15 is the same as the image acquisition frequency of the image acquisition unit 16.

[0039] Specifically, the refresh frequency of the second digital micromirror 15 is N times that of the first digital micromirror 13, that is, within the same grating period, the second digital micromirror 15 generates multiple different random binary coding patterns to provide redundant information for subsequent algorithm reconstruction. The first digital micromirror 13 is responsible for generating the grating pattern required for structured light illumination. The refresh frequency of the first digital micromirror 13 determines the update rate of the structured light. The first digital micromirror 13 can switch the grating direction or phase at a low frequency to ensure that the structured light is sufficiently stable on the sample surface and forms effective interference.

[0040] The image acquisition frequency of the image acquisition unit 16 is consistent with the refresh frequency of the random binary coded image of the second digital micromirror 15. Every time the coded pattern is refreshed, the second digital micromirror 15 triggers the image acquisition unit 16 to perform an image acquisition, ensuring that each frame of the coded image strictly matches the current grating lighting conditions, and the image acquisition unit 16 collects multiple frames of coded data under the same lighting conditions, providing redundant information for subsequent algorithm reconstruction, thereby suppressing noise and improving image resolution.

[0041] As an implementation method, the second digital micromirror 15 is configured to: in response to refreshing the random binary coding pattern once, send a control instruction to the image acquisition unit 16 to trigger the image acquisition unit 16 to perform image acquisition; in response to refreshing the random binary coding pattern N+1 times, reset the refresh count calculation of this round of random binary coding pattern, and send a control instruction to the first digital micromirror 13 to trigger the first digital micromirror 13 to perform grating pattern transformation.

[0042] Exemplarily, in the embodiment of the present application, the random binary coded image refresh frequency of the second digital micromirror 15 is set to 20 kHz, then, each time the second digital micromirror 15 refreshes the pattern, it sends a signal to the image acquisition unit 16, controls the image acquisition unit 16 to expose once and acquire an image, that is, the second digital micromirror 15 loads 20k random binary coded images per second, and sends 20k signals to the image acquisition unit 16, and the image acquisition unit 16 exposes 20k times to acquire 20k images. In addition, the second digital micromirror 15 resets the refresh count calculation of the current round of random binary coded pattern at 20k+1 times, and sends a signal to the first digital micromirror 13, triggering the first digital micromirror 13 to load the next grating pattern.

[0043] Combined with the above description, if Figure 3 As shown, the refresh frequency of the random binary coded image of the second digital micromirror 15 is N times the grating pattern transformation frequency of the first digital micromirror 13, the refresh frequency of the random binary coded image of the second digital micromirror 15 is the same as the image acquisition frequency of the image acquisition unit 16, the second digital micromirror 15 controls the image acquisition unit 16 to have an acquisition frequency of 20 kHz, the grating pattern transformation frequency of the first digital micromirror 13 is 1 Hz, and the image acquisition frequency of the image acquisition unit 16 is 20 kHz.

[0044] As an implementation method, the processing unit 17 is configured to: use a deep learning image reconstruction algorithm to reconstruct a plurality of random binary coded images to obtain an image with a structured light illumination pattern, and perform microscopic super-resolution reconstruction on a plurality of images with a structured light illumination pattern to obtain a microscopic super-resolution image.

[0045] Specifically, the image acquisition unit 16 transmits the acquired image to the processing unit 17, and the processing unit 17 reconstructs the plurality of random binary coded images using a deep learning graphic reconstruction algorithm. Exemplarily, the resolution of the linear array camera is set to 2000×2, and 20k continuous images acquired during the time when the first digital micromirror 13 plays one grating pattern are delivered to the deep learning reconstruction algorithm, and 10 images (or more than 10 images, which is determined by the compression rate set in the reconstruction algorithm in the actual application) containing a single structured light illumination pattern are reconstructed, and then the continuous images acquired during the time when the first digital micromirror 13 plays nine grating patterns are delivered to the deep learning reconstruction algorithm, and 10×9 images containing structured light illumination patterns can be reconstructed.

[0046] Furthermore, the above-mentioned 10×9 reconstructed images containing structured light illumination patterns are transmitted to the structured light illumination super-resolution reconstruction algorithm, and every 9 structured light illumination images containing 9 different phases and illumination directions can be synthesized into one super-resolution image. 10×9 images can be used to reconstruct 10 super-resolution images, thereby obtaining a microscopic super-resolution image.

[0047] As an implementation method, the processing unit 17 uses 3M images with structured light illumination patterns to perform microscopic super-resolution reconstruction to obtain a microscopic super-resolution image, where M is 1, 2, or 3. That is, the microscopic imaging device 100 provided in the present application can keep the optical path unchanged while adjusting the signal sending and receiving frequency in the optical path and the image acquisition time interval, and use 3 or 6 structured light illumination images to synthesize a single super-resolution image. As the number of images used for reconstruction increases, the resolution of the synthesized microscopic super-resolution image is higher.

[0048] According to the above description, a microscopic imaging device 100 provided in the present application modulates laser into multi-order diffraction light through a digital micromirror and a focusing unit 14, and makes the diffraction light interfere with each other to form structured light. The structured light converges on the sample to excite a fluorescent signal carrying sample information, and the fluorescence is converted and encoded to obtain a random binary coded image. Finally, by reconstructing the random binary coded image, a super-resolution imaging effect in a structured light illumination microscopic system is achieved; and under low light intensity conditions, the loss of light energy in the optical system is reduced, protecting biological cell samples from phototoxicity and photobleaching.

[0049] In a second aspect, the present application also provides a super-resolution microscopic imaging method, which uses the microscopic imaging device 100 described above, such as Figure 4 As shown, the method includes: Step S11, using a deep learning image reconstruction algorithm to reconstruct a plurality of random binary coded images to obtain an image with a structured light illumination pattern.

[0050] Specifically, step S11 includes: for the dynamic scene of the obtained sample The measured value , we set a target frame rate, for example 10 frames per second, and group the continuously collected measurement data according to the target frame rate. Assume that, is the scene frame related to the required frame rate, then the constructed imaging model can be expressed by the following formula: ; In the formula, is the scene frame associated with the desired frame rate, , Q is a constant greater than 1, T represents time, represents the measurement noise, represents the measurement matrix, Represents multiple reconstructed large-size image information containing structured light illumination information.

[0051] Through the above imaging model, multiple large-size image information containing structured light illumination information can be restored.

[0052] Step S12, performing microscopic super-resolution reconstruction on a plurality of images with structured light illumination patterns to obtain microscopic super-resolution images.

[0053] Step S12 includes: performing Fourier transform on 3M images with structured light illumination patterns to obtain frequency spectra of each image, where M is 1, 2, or 3; The original image in the spectrum of each image is obtained using a preset window function, the spectrum of each original image is extracted, and the spectrum of the extracted original image is translated to the exact center of the entire spectrum, a new spectrum is synthesized, and an inverse Fourier transform is performed on it to obtain the microscopic super-resolution image.

[0054] Specifically, the multiple large-size images containing structured light illumination information obtained in the first step are input into the structured light super-resolution reconstruction algorithm. For example, 9 images containing structured light illumination information (i=1,2,3...,9) is input into the structured light super-resolution reconstruction algorithm. First, Perform Fourier transform and get , where u and v are the spatial frequencies in the x and y directions respectively, and x and y represent the horizontal and vertical coordinates in the two-dimensional coordinate system. Use the preset window function to obtain the original image in the spectrum of each image, extract the spectrum of each original image and translate the extracted spectrum of the original image to the exact center of the entire spectrum, synthesize a new spectrum, and perform an inverse Fourier transform on it. The resulting image is , which is the super-resolution image of the sample.

[0055] In this way, by inputting 9n images into the structured light super-resolution reconstruction algorithm, super-resolution images of n samples can be obtained.

[0056] This super-resolution microscopy method can protect biological cell samples from phototoxicity and photobleaching, and achieve super-resolution imaging in structured light illumination microscopy systems.

[0057] In summary, the microscopic imaging device provided by the present application has at least the following advantages: the observation environment targeted by the present solution is more extreme (low lighting conditions), the device can adapt to the super-resolution reconstruction of images under low lighting conditions, can image the structure of the sample at a resolution level of hundreds of nanometers, and can achieve long-time exposure acquisition and reconstruction. Secondly, the image processing process of the present application is implemented by two steps, and they are interconnected. Changing the order or using alternative solutions cannot achieve the final result. Thirdly, the present application also has the advantage of low cost. Since the microscopic solutions under ordinary low lighting conditions all use extremely expensive low-light cameras such as sCMOS, they cannot be widely popularized. The acquisition equipment used in the present invention is an ordinary linear array camera, which greatly improves the utilization rate of light while being low-cost.

[0058] It will be appreciated that the word "exemplary" as used herein means "serving as an example, instance, or illustration". Any embodiment described as "exemplary" is not necessarily preferred or superior to other embodiments and / or does not exclude the combination of features of other embodiments. It will be appreciated that certain features of the present application described in the context of separate embodiments for the sake of clarity may also be provided in a single embodiment by combination. Conversely, various features of the present application described in the context of a single embodiment for the sake of clarity may also be provided individually or in any suitable combination or as any other described embodiment of the present application.

[0059] The above disclosure is only the preferred embodiment of the present application, but it is not intended to limit the scope of rights of the present application. A person of ordinary skill in the art can understand that without departing from the spirit and scope of the present application and the appended claims, changes, modifications, substitutions, combinations, and simplifications should all be equivalent replacement methods and still fall within the scope of the invention.

Claims

1. A microscopic imaging device, characterized in that: The microscopic imaging device comprises: A sample table, used to hold samples; Laser, used for emitting laser light; A first digital micromirror is configured to receive the laser light and form a grating image to modulate the laser light to generate diffracted light of multiple diffraction orders; A focusing unit is configured to converge the plurality of diffracted lights generated by the first digital micromirror, so that the diffracted lights interfere with each other to form structured light, and focus the structured light on the sample to excite fluorescence carrying sample information; a second digital micromirror, receiving the fluorescence and converting the fluorescence into a random binary coded image; A cylindrical lens, the cylindrical lens being disposed in the optical path between the second digital micromirror and the image acquisition unit, and being used for converging the random binary coded image into a light beam distributed along a preset direction; An image acquisition unit, comprising a line array camera, wherein the line array camera is disposed on a convergence surface of the cylindrical lens, and the line array camera is configured to receive the random binary coded image converged by the cylindrical lens, and transmit the random binary coded image to a processing unit; A processing unit, wherein the processing unit is configured to reconstruct a microscopic super-resolution image based on a plurality of random binary coded images; The first digital micromirror, the second digital micromirror and the image acquisition unit are communicatively connected, and the second digital micromirror sends control instructions to the first digital micromirror and the image acquisition unit respectively to control the grating pattern conversion frequency of the first digital micromirror and the image acquisition frequency of the image acquisition unit.

2. The microscopic imaging device according to claim 1, characterized in that: The first digital micromirror is configured to play 9 grating patterns in time sequence to generate different diffracted lights, wherein the 9 grating patterns include three grating directions of 0°, 60°, and 120°, and each grating direction includes 0, , Three phases.

3. The microscopic imaging device according to claim 2, characterized in that: The first digital micromirror is configured to modulate the laser to generate three diffraction orders of 0, +1, and -1; The microscopic imaging device further includes a filter configured to filter out 0th order diffraction light and allow +1st order diffraction light and -1st order diffraction light to pass.

4. The microscopic imaging device according to claim 3, characterized in that: The microscopic imaging device further comprises a dichroic mirror, wherein the dichroic mirror is configured to reflect the +1-order diffracted light and the -1-order diffracted light to the light focusing unit; The focusing unit converges the +1st order diffraction light and the -1st order diffraction light to generate interference, thereby forming the structured light.

5. The microscopic imaging device according to claim 1, characterized in that: Define the refresh frequency of the random binary coded image of the second digital micromirror as N, where N is a natural number greater than 1. Then, the refresh frequency of the random binary coded image of the second digital micromirror is N times the grating pattern transformation frequency of the first digital micromirror, and the refresh frequency of the random binary coded image of the second digital micromirror is the same as the image acquisition frequency of the image acquisition unit.

6. The microscopic imaging device according to claim 5, characterized in that: The second digital micromirror is configured to: in response to refreshing the random binary coding pattern once, send a control instruction to the image acquisition unit to trigger the image acquisition unit to perform image acquisition; in response to refreshing the random binary coding pattern N+1 times, reset the calculation of the number of refresh times of the random binary coding pattern in this round, and send a control instruction to the first digital micromirror to trigger the first digital micromirror to perform grating pattern transformation.

7. The microscopic imaging device according to claim 1, characterized in that: The processing unit is configured to: use a deep learning image reconstruction algorithm to reconstruct a plurality of the random binary coded images to obtain an image with a structured light illumination pattern, and perform microscopic super-resolution reconstruction on the plurality of images with the structured light illumination pattern to obtain the microscopic super-resolution image.

8. The microscopic imaging device according to claim 7, characterized in that: The deep learning image reconstruction algorithm is used to reconstruct the plurality of random binary coded images, including: grouping the continuously acquired measurement data according to the target frame rate, and constructing an imaging model, wherein the imaging model is represented by the following formula: ； In the formula, is the scene frame associated with the desired frame rate, , Q is a constant greater than 1, T represents time, represents the measurement noise, represents the measurement matrix, Represents multiple reconstructed large-size image information containing structured light illumination information.

9. The microscopic imaging device according to claim 8, characterized in that: The performing microscopic super-resolution reconstruction on the plurality of images with the structured light illumination patterns to obtain the microscopic super-resolution image comprises: Perform Fourier transform on 3M images with structured light illumination patterns to obtain the frequency spectrum of each image, where M is 1, 2, or 3; The original image in the spectrum of each image is obtained using a preset window function, the spectrum of each original image is extracted, and the spectrum of the extracted original image is translated to the exact center of the entire spectrum, a new spectrum is synthesized, and an inverse Fourier transform is performed on it to obtain the microscopic super-resolution image.

10. A super-resolution microscopic imaging method, characterized in that: The method uses the microscopic imaging device according to any one of claims 1 to 7, and the method comprises: A deep learning image reconstruction algorithm is used to reconstruct a plurality of random binary coded images to obtain an image with a structured light illumination pattern, and a plurality of images with the structured light illumination pattern are microscopically super-resolved to obtain the microscopic super-resolved image.

Images