Microscopic imaging device and super-resolution microscopic imaging method

Abstract

The application discloses a kind of microscopic imaging device and super-resolution microscopic imaging method, comprising: sample table, for containing sample;Laser, for emitting laser;First digital micro mirror, for receiving laser, produce diffraction light;Light collection unit, for converging multiple diffraction light, form structured light, to excite the fluorescence carrying sample information;Second digital micro mirror, receive fluorescence and convert fluorescence into random binary encoding image;Image acquisition unit, for receiving image and transmission to processing unit;Processing unit, based on complex random binary encoding image reconstruction microscopic super-resolution image;First digital micro mirror, second digital micro mirror and image acquisition unit communication connection, second digital micro mirror controls the grating pattern conversion frequency of first digital micro mirror, and controls the image acquisition frequency of image acquisition unit.The application can protect biological cell sample from the invasion of phototoxicity and photobleaching, realize super-resolution imaging in structured light illumination microscopic system.

Description

Technical Field

[0001] The present application relates to the technical field of optical imaging equipment, and in particular to a microscopic imaging device and a super-resolution microscopic imaging method. Background Art

[0002] In modern biomedicine, research on nanoscale biological samples, such as animal and plant organelles, neurons, and bacteria, is steadily advancing. Optical microscopes with super-resolution capabilities, as a crucial observation tool, play a decisive role in related research. Common super-resolution microscopy techniques include structured illumination microscopy, stimulated emission depletion microscopy, stochastic optical reconstruction microscopy, and single-molecule localization microscopy.

[0003] The above-mentioned technology requires active illumination light to illuminate the biological cell samples, which will cause significant photobleaching and phototoxicity to living cells, resulting in rapid cell death during the observation of active biological cells, making it impossible to observe the life processes of biological cells for a long time. If the illumination light intensity is reduced, the optical information carrying the biological cells will be too weak due to the signal intensity, requiring the use of expensive low-light cameras to capture images; or using ordinary cameras for long exposure acquisition. However, due to the influence of dark current noise and other factors of ordinary cameras, long exposure acquisition will result in a very low signal-to-noise ratio of the captured images, thus defeating the purpose of the experiment. Summary of the Invention

[0004] In order to solve the deficiencies of the prior art, this application adopts the following technical solutions:

[0005] In a first aspect, the present application provides a microscopic imaging device, comprising:

[0006] Sample table, used to hold samples;

[0007] Laser, for emitting laser light;

[0008] a first digital micromirror configured to receive the laser light and form a grating image to modulate the laser light to generate diffracted light of multiple diffraction orders;

[0009] a light focusing unit configured to converge the plurality of diffracted lights generated by the first digital micromirror, causing the diffracted lights to interfere with each other to form structured light, and to focus the structured light on the sample to excite fluorescence carrying sample information;

[0010] a second digital micromirror, receiving the fluorescence and converting the fluorescence into a random binary coded image;

[0011] a cylindrical lens, the cylindrical lens being disposed in the optical path between the second digital micromirror and the image acquisition unit, and being configured to converge the random binary coded image into light rays distributed along a preset direction;

[0012] an image acquisition unit, comprising a line array camera, the line array camera being disposed on a converging surface of the cylindrical lens, the line array camera being configured to receive the random binary coded image converged by the cylindrical lens, and transmit the random binary coded image to a processing unit;

[0013] a processing unit configured to reconstruct a microscopic super-resolution image based on a plurality of random binary coded images;

[0014] The first digital micromirror, the second digital micromirror and the image acquisition unit are communicatively connected, and the second digital micromirror sends control instructions to the first digital micromirror and the image acquisition unit respectively to control the grating pattern conversion frequency of the first digital micromirror and the image acquisition frequency of the image acquisition unit.

[0015] In summary, the present application provides a microscopic imaging device that modulates laser light into multi-order diffraction light through a digital micromirror and a focusing unit, and makes the diffraction light interfere with each other to form structured light. The structured light converges on the sample to excite a fluorescence signal carrying sample information, and the fluorescence is converted and encoded to obtain a random binary coded image. Finally, by reconstructing the random binary coded image, the super-resolution imaging effect in the structured light illumination microscopy system is achieved; and under low light conditions, the loss of light energy in the optical system is reduced, protecting biological cell samples from phototoxicity and photobleaching.

[0016] Furthermore, the first digital micromirror is configured to play 9 grating patterns in a time sequence to generate different diffracted lights, wherein the 9 grating patterns include three grating directions of 0°, 60°, and 120°, and each grating direction includes 0, , Three phases.

[0017] Furthermore, the first digital micromirror is configured to modulate the laser to generate three diffraction orders of 0, +1, and -1;

[0018] The microscopic imaging device further includes a filter configured to filter out the 0th order diffraction light and allow the +1st order diffraction light and the -1st order diffraction light to pass.

[0019] Furthermore, the microscopic imaging device further includes a dichroic mirror, wherein the dichroic mirror is configured to reflect the +1-order diffraction light and the -1-order diffraction light to the focusing unit;

[0020] The condensing unit converges the +1 order diffracted light and the -1 order diffracted light to generate interference, forming the structured light.

[0021] Further, the refresh frequency of the random binary coded image of the second digital micromirror is defined as N, N is a natural number greater than 1, then the refresh frequency of the random binary coded image of the second digital micromirror is N times of the grating pattern transformation frequency of the first digital micromirror, and the refresh frequency of the random binary coded image of the second digital micromirror is the same as the image acquisition frequency of the image acquisition unit.

[0022] Further, the second digital micromirror is configured to: in response to refreshing the random binary coded pattern once, send a control instruction to the image acquisition unit to trigger the image acquisition unit to perform image acquisition; and in response to refreshing the random binary coded pattern N+1 times, reset the refresh count calculation of the random binary coded pattern in this round, and send a control instruction to the first digital micromirror to trigger the first digital micromirror to perform grating pattern transformation.

[0023] Further, the processing unit is configured to: use a deep learning image reconstruction algorithm to reconstruct a plurality of the random binary coded images to obtain images with structured light illumination patterns, and perform microscopic super-resolution reconstruction on the plurality of images with structured light illumination patterns to obtain the microscopic super-resolution image.

[0024] Further, the use of a deep learning image reconstruction algorithm to reconstruct a plurality of the random binary coded images includes: grouping the continuously acquired measurement data according to a target frame rate to construct an imaging model, the imaging model being represented by the following formula:

[0025] ;

[0026] In the formula, is the scene frame related to the required frame rate, , Q is a constant greater than 1, and T represents time, represents measurement noise, represents a measurement matrix, represents a plurality of large-size image information containing structured light illumination information obtained by reconstruction.

[0027] Further, Fourier transform is performed on the 3M images with structured light illumination patterns to obtain the frequency spectrum of each image, and M is 1 or 2 or 3.

[0028] The original image in the spectrum of each image is obtained using a preset window function, the spectrum of each original image is extracted, and the extracted spectrum of the original image is shifted to the positive center position of the entire spectrum, a new spectrum is synthesized, and inverse Fourier transform is performed on the new spectrum to obtain the microscopic super-resolution image.

[0029] In a second aspect, the present application further provides a super-resolution microscopic imaging method, wherein the method uses the microscopic imaging device described above, and the method comprises:

[0030] A deep learning image reconstruction algorithm is used to reconstruct a plurality of random binary coded images to obtain images with structured light illumination patterns, and a plurality of microscopic super-resolution reconstruction is performed on the images with structured light illumination patterns to obtain the microscopic super-resolution image. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 A schematic diagram of the composition structure of the microscopic imaging device according to an embodiment of the present application is provided.

[0032] Figure 2 A schematic diagram of the optical path of the microscopic imaging device according to an embodiment of the present application is provided.

[0033] Figure 3 A schematic diagram of the signal frequency of the first digital micromirror, the second digital micromirror, and the image acquisition unit of the microscopic imaging device according to an embodiment of the present application is provided.

[0034] Figure 4 A flowchart of the steps of the super-resolution microscopic imaging method according to an embodiment of the present application is provided. DETAILED DESCRIPTION

[0035] The present application will be described in detail below with reference to the specific embodiments shown in the accompanying drawings, but these embodiments do not limit the present application, and any changes in structure, method, or function made by a person of ordinary skill in the art based on these embodiments are included within the scope of protection of the present application.

[0036] To solve the problems of the prior art, in a first aspect, as shown in the accompanying drawings, Figure 1 The present application provides a microscopic imaging device 100, which comprises a sample stage 11, a laser 12, a first digital micromirror 13, a light condensing unit 14, a second digital micromirror 15, a cylindrical lens 26, an image acquisition unit 16, and a processing unit 17.

[0037] The sample stage 11 is used to hold the sample, and the laser 12 is used to emit laser light. The first digital micromirror 13 is configured to receive the laser light and form a grating image to modulate the laser light, generating diffracted light of multiple diffraction orders. The focusing unit 14 is configured to converge the multiple diffracted light generated by the first digital micromirror 13, causing the diffracted light to interfere with each other to form structured light, and then converge the structured light on the sample to excite fluorescence carrying sample information. The second digital micromirror 15 receives the fluorescence and converts it into a random binary coded image. The cylindrical lens 26 is arranged in the optical path between the second digital micromirror 15 and the image acquisition unit 16. The cylindrical lens 26 is used to converge the random binary coded image into light distributed along a preset direction. The image acquisition unit 16 includes a line array camera, which is arranged on the converging surface of the cylindrical lens 26. The image acquisition unit 16 is configured to receive the random binary coded image and transmit the random binary coded image to the processing unit 17. The processing unit 17 is configured to reconstruct a microscopic super-resolution image based on the multiple random binary coded images.

[0038] The first digital micromirror 13, the second digital micromirror 15 and the image acquisition unit 16 are communicatively connected, and the second digital micromirror 15 sends control instructions to the first digital micromirror 13 and the image acquisition unit 16, respectively, to control the grating pattern conversion frequency of the first digital micromirror 13 and the image acquisition frequency of the image acquisition unit 16.

[0039] Specifically, such as Figure 2 As shown, the sample stage 11 is used to fix the biological sample to ensure the stability of the sample during imaging and avoid the sample displacement resulting in blurred imaging. The protein in the biological sample is stained and marked with fluorescence.

[0040] Laser 12, acting as a light source, emits laser light of a specific wavelength. The laser light emitted by laser 12 is received by first digital micromirror 13 and is used to photoexcite fluorescently stained proteins within the biological sample, thereby stimulating a fluorescent signal. Furthermore, the laser light emitted by laser 12 can be controlled within a low-light range to reduce phototoxicity to living samples and mitigate photobleaching effects. Optionally, a beam expander 18 and a first reflector 19 are provided between laser 12 and first digital micromirror 13. The laser light emitted by laser 12 is expanded and collimated by beam expander 18 before being reflected by first reflector 19 and reaching first digital micromirror 13.

[0041] Through the refraction of the laser by the first reflector 19, the first digital micromirror 13 receives the laser after beam expansion and collimation, and the first digital micromirror 13 spatially modulates the laser by dynamically loading a grating pattern. The first digital micromirror 13 can decompose a single laser beam into diffraction lights of multiple diffraction orders by quickly switching grating patterns of different directions and phases. Subsequently, the diffraction light generated by the first digital micromirror 13 is reflected to the focusing unit 14 through the second reflector 21.

[0042] Through the refraction of the diffracted light by the second reflector 21, the focusing unit 14 receives the diffracted light generated by the first digital micromirror 13, and the focusing unit 14 converges the multiple diffracted lights so that the multiple diffracted lights interfere with each other, thereby forming structured lights of multiple directions and phases. The focusing unit 14 is located above the sample stage 11, and the structured light converges on the biological sample on the sample stage 11. The structured light excites the fluorescent-stained proteins in the biological sample, thereby generating a fluorescent signal carrying information about the biological sample. Optionally, a lens 22 can be provided between the second reflector 21 and the focusing unit 14. The focusing unit 14 receives the diffracted light refracted by the second reflector 21 through the lens 22. The lens 22 can converge the divergent or parallel diffracted light beams to a specific focal position, thereby avoiding light energy dispersion and improving the intensity and contrast of the subsequent interference structured light. In an embodiment of the present application, the focusing unit 14 is configured as a microscope objective.

[0043] The second digital micromirror 15 is configured to receive a fluorescent signal carrying biological sample information, and the second digital micromirror 15 is loaded with a random binary coding pattern. The fluorescent signal carrying biological sample information is modulated by rapidly refreshing the coding pattern, and the fluorescent signal is converted into a random binary coding pattern, providing raw material for the subsequent processing unit 17 to reconstruct a high-resolution image.

[0044] After the fluorescent signal carrying biological sample information is modulated into a random binary code pattern by the second digital micromirror 15, the cylindrical lens can converge the encoded optical signal into a linearly distributed optical fiber along a preset direction. In the embodiment of the present application, the encoded random binary code image is converged into a horizontally distributed light after passing through the cylindrical lens 26 and is captured by a line array camera placed on the converging surface of the cylindrical lens 26. The line array camera can acquire signals with low noise and high frame rate. The coordinated application of the cylindrical lens 26 and the line array camera overcomes the signal-to-noise ratio challenges in low-light conditions and provides a high-quality raw data foundation for super-resolution imaging.

[0045] The processing unit 17 uses a reconstruction algorithm to reconstruct the image of the multiple random binary coding patterns. By fusing the multiple random binary coding patterns, it breaks through the traditional optical resolution limitation and reconstructs an image with microscopic super-resolution, thereby achieving the super-resolution imaging effect of the microscopic imaging device 100.

[0046] Furthermore, the first digital micromirror 13, the second digital micromirror 15 and the image acquisition unit 16 are communicatively connected, and the second digital micromirror 15 sends a control signal to the first digital micromirror 13 and the image acquisition unit 16 at a certain control frequency, respectively controlling the frequency of switching the grating pattern of the first digital micromirror 13 and the image acquisition frequency of the image acquisition unit 16, so as to meet different imaging requirements and enhance the flexibility of imaging.

[0047] According to the above description, the present application provides a microscopic imaging device 100, which modulates laser light to form structured light through a digital micromirror and a focusing unit 14. The structured light is focused on the sample to excite a fluorescent signal carrying sample information, and the fluorescence is converted and encoded to obtain a random binary coded image. Finally, the random binary coded image is reconstructed, thereby achieving a super-resolution imaging effect in a structured light illumination microscopic system; and under low light conditions, the loss of light energy in the optical system is reduced, thereby protecting biological cell samples from phototoxicity and photobleaching.

[0048] As an implementation method, the first digital micromirror 13 is configured to play 9 grating patterns in time sequence to generate different diffracted lights, wherein the 9 grating patterns include three grating directions of 0°, 60°, and 120°, and each grating direction includes 0, , Three phases.

[0049] Specifically, the nine grating patterns played by the first digital micromirror 13 include three grating directions of 0°, 60°, and 120° and three phases of 0, , The three grating directions are evenly distributed at 60° intervals, thus covering a wider spatial frequency range in the frequency domain. Different grating directions form interference patterns at specific angles on the sample surface. By superimposing different directions, the response information of the sample at different spatial frequencies is collected, thereby synthesizing a super-resolution image during image reconstruction.

[0050] Furthermore, each grating direction includes three different phases. The phase change can cause the grating stripes to shift laterally in the sample plane, thereby capturing the response differences of the sample under different lighting conditions, obtaining the high-frequency information of the sample, providing more information for subsequent algorithm reconstruction, and improving the contrast and resolution of the reconstructed image.

[0051] As an implementation method, the first digital micromirror 13 is configured to modulate the laser to produce three diffraction orders: 0, +1, and -1. The first digital micromirror 13 forms a dynamic grating pattern by rapidly flipping. When the laser irradiates the surface of the first digital micromirror 13, the periodic structure of the grating pattern spatially modulates the laser. According to Fraunhofer diffraction theory, the periodic grating modulates the phase or amplitude of the incident light wave to form beams of three diffraction orders: 0, +1, and -1. The propagation direction of the 0th-order diffracted light is consistent with that of the incident light, and the 0th-order diffracted light does not carry the spatial frequency information of the grating modulation. The +1st-order and -1st-order diffracted light, generated by the periodic modulation of the grating, are deflected to the sides at symmetrical angles, respectively, and carry the spatial frequency information of the grating modulation.

[0052] Microscopic imaging apparatus 100 further includes a filter 23 configured to filter out 0th-order diffracted light and pass +1st-order and -1st-order diffracted light. Because 0th-order diffracted light does not carry the spatial frequency information of the grating modulation, its illumination of the sample surface will overwhelm the interference effect of the structured light. Filter 23 filters out 0th-order diffracted light and passes +1st-order and -1st-order diffracted light, preventing the introduction of 0th-order diffracted light from affecting imaging contrast and preventing reconstruction of a super-resolution image.

[0053] As an implementation method, the microscopic imaging device 100 also includes a dichroic mirror 24, which is configured to reflect the +1-order diffraction light and the -1-order diffraction light to the focusing unit 14; the focusing unit 14 converges the +1-order diffraction light and the -1-order diffraction light to produce interference and form structured light.

[0054] Specifically, the diffracted light modulated by the first digital micromirror 13 is refracted by the second reflector 21 and then passes through the lens 22 and the filter 23. The filter 23 filters out the 0th-order diffracted light, and the remaining +1st-order diffracted light and -1st-order diffracted light are irradiated by the dichroic mirror 24. The dichroic mirror 24 is a reflector with a special coating. The +1st-order diffracted light and -1st-order diffracted light are refracted by the dichroic mirror 24 and refracted to the focusing unit 14. The dichroic mirror 24 is tilted at a specific angle to ensure that the propagation paths of the refracted +1st-order diffracted light and -1st-order diffracted light are aligned with the optical axis of the focusing unit 14, thereby preventing light energy loss or light direction deviation.

[0055] The focusing unit 14 converges the +1st order diffraction light and the -1st order diffraction light to interfere with each other, forming nine types of structured light with different directions and phases. The structured light excites the fluorescent-stained proteins in the biological sample. The fluorescent signal carrying the biological sample information is received by the second digital micromirror 15 through the dichroic mirror 24, and the second digital micromirror 15 converts the fluorescent signal carrying the biological sample information into a random binary coded image.

[0056] Furthermore, the microscopic imaging device 100 also includes a tube lens 25, which is arranged between the dichroic mirror 24 and the second digital micromirror 15. The fluorescent signal carrying the biological sample information passes through the dichroic mirror 24 and the tube lens 25 in succession and is received by the second digital micromirror 15. The tube lens 25 can further avoid light energy loss or light direction deviation.

[0057] As an implementation method, the refresh frequency of the random binary coded image of the second digital micromirror 15 is defined as N, where N is a natural number greater than 1. Then, the refresh frequency of the random binary coded image of the second digital micromirror 15 is N times the grating pattern transformation frequency of the first digital micromirror 13, and the refresh frequency of the random binary coded image of the second digital micromirror 15 is the same as the image acquisition frequency of the image acquisition unit 16.

[0058] Specifically, the refresh rate of the second digital micromirror 15 is N times that of the first digital micromirror 13. That is, within the same grating period, the second digital micromirror 15 generates multiple different random binary coded patterns, providing redundant information for subsequent algorithm reconstruction. The first digital micromirror 13 is responsible for generating the grating pattern required for structured light illumination. The refresh rate of the first digital micromirror 13 determines the update rate of the structured light. The first digital micromirror 13 can switch the grating direction or phase at a low frequency to ensure that the structured light is sufficiently stable on the sample surface and forms effective interference.

[0059] The image acquisition frequency of the image acquisition unit 16 is consistent with the refresh frequency of the random binary coded image of the second digital micromirror 15. Every time the coded pattern is refreshed, the second digital micromirror 15 triggers the image acquisition unit 16 to perform an image acquisition, ensuring that each frame of coded image strictly matches the current grating lighting conditions. In addition, the image acquisition unit 16 acquires multiple frames of coded data under the same lighting conditions, providing redundant information for subsequent algorithm reconstruction, thereby suppressing noise and improving image resolution.

[0060] As an implementation method, the second digital micromirror 15 is configured to: in response to refreshing the random binary coding pattern once, send a control instruction to the image acquisition unit 16 to trigger the image acquisition unit 16 to perform image acquisition; in response to refreshing the random binary coding pattern N+1 times, reset the refresh count calculation of the current round of random binary coding pattern, and send a control instruction to the first digital micromirror 13 to trigger the first digital micromirror 13 to perform grating pattern transformation.

[0061] For example, in an embodiment of the present application, the refresh frequency of the random binary coded image of the second digital micromirror 15 is set to 20 kHz. Then, each time the second digital micromirror 15 refreshes the pattern, it sends a signal to the image acquisition unit 16, controlling the image acquisition unit 16 to expose and capture an image. That is, per second, the second digital micromirror 15 loads 20k random binary coded images and sends 20k signals to the image acquisition unit 16, causing the image acquisition unit 16 to expose 20k images and capture 20k images. Furthermore, at 20k+1 times, the second digital micromirror 15 resets the refresh count for the current round of random binary coded patterns and sends a signal to the first digital micromirror 13, triggering the first digital micromirror 13 to load the next grating pattern.

[0062] Combined with the above description, if Figure 3As shown, the refresh frequency of the random binary coded image of the second digital micromirror 15 is N times the grating pattern conversion frequency of the first digital micromirror 13. The refresh frequency of the random binary coded image of the second digital micromirror 15 is the same as the image acquisition frequency of the image acquisition unit 16. The second digital micromirror 15 controls the image acquisition unit 16 to acquire an acquisition frequency of 20 kHz, the grating pattern conversion frequency of the first digital micromirror 13 is 1 Hz, and the image acquisition frequency of the image acquisition unit 16 is 20 kHz.

[0063] As an implementation method, the processing unit 17 is configured to: use a deep learning image reconstruction algorithm to reconstruct a plurality of random binary coded images to obtain an image with a structured light illumination pattern, and perform microscopic super-resolution reconstruction on a plurality of images with a structured light illumination pattern to obtain a microscopic super-resolution image.

[0064] Specifically, the image acquisition unit 16 transmits the captured images to the processing unit 17, which uses a deep learning image reconstruction algorithm to reconstruct the plurality of random binary-coded images. For example, if the line scan camera resolution is set to 2000×2, the 20k consecutive images captured during the time it takes the first digital micromirror 13 to play one grating pattern are fed into the deep learning reconstruction algorithm to reconstruct 10 images containing a single structured light illumination pattern (or more than 10 images, depending on the compression rate set in the reconstruction algorithm in actual applications). Furthermore, the consecutive images captured during the time it takes the first digital micromirror 13 to play nine grating patterns are fed into the deep learning reconstruction algorithm to reconstruct 10×9 images containing the structured light illumination pattern.

[0065] Furthermore, the above-mentioned 10×9 reconstructed images containing structured light illumination patterns are transmitted to the structured light illumination super-resolution reconstruction algorithm. Every 9 structured light illumination images containing 9 different phases and illumination directions can be synthesized into 1 super-resolution image. 10×9 images can be used to reconstruct 10 super-resolution images, thereby obtaining a microscopic super-resolution image.

[0066] As an implementation method, the processing unit 17 uses 3M images with structured light illumination patterns to perform microscopic super-resolution reconstruction to obtain a microscopic super-resolution image, where M is 1, 2, or 3. That is, the microscopic imaging device 100 provided in the present application can maintain the optical path unchanged while adjusting the signal transmission and reception frequency in the optical path and the image acquisition time interval to use 3 or 6 structured light illumination images to synthesize a single super-resolution image. As the number of images used for reconstruction increases, the resolution of the synthesized microscopic super-resolution image increases.

[0067] According to the above description, the present application provides a microscopic imaging device 100, which modulates laser light into multi-order diffraction light through a digital micromirror and a focusing unit 14, and makes the diffraction light interfere with each other to form structured light. The structured light converges on the sample to excite a fluorescence signal carrying sample information, and the fluorescence is converted and encoded to obtain a random binary coded image. Finally, by reconstructing the random binary coded image, a super-resolution imaging effect in a structured light illumination microscopic system is achieved; and under low light conditions, the loss of light energy in the optical system is reduced, protecting biological cell samples from phototoxicity and photobleaching.

[0068] In a second aspect, the present application also provides a super-resolution microscopic imaging method, which uses the microscopic imaging device 100 described above, such as Figure 4 As shown, the method includes:

[0069] Step S11: Reconstruct a plurality of random binary coded images using a deep learning image reconstruction algorithm to obtain an image with a structured light illumination pattern.

[0070] Specifically, step S11 includes: for the dynamic scene of the obtained sample The measured value , we set a target frame rate, for example 10 frames per second, and group the continuously collected measurement data according to the target frame rate. Assume that, is the scene frame related to the required frame rate, then the constructed imaging model can be expressed by the following formula:

[0071] ;

[0072] Where, The scene frames associated with the desired frame rate, , Q is a constant greater than 1, T represents time, represents the measurement noise, represents the measurement matrix, Represents multiple reconstructed large-size image information containing structured light illumination information.

[0073] Through the above imaging model, multiple large-size image information containing structured light illumination information can be restored.

[0074] Step S12 , performing microscopic super-resolution reconstruction on the plurality of images with structured light illumination patterns to obtain microscopic super-resolution images.

[0075] Step S12 includes: performing Fourier transform on the 3M images with the structured light illumination pattern to obtain a frequency spectrum of each image, where M is 1, 2, or 3;

[0076] A pre-set window function is used to obtain the original image in the spectrum of each image, the spectrum of each original image is extracted, and the spectrum of the extracted original image is translated to the center of the entire spectrum, a new spectrum is synthesized, and an inverse Fourier transform is performed on it to obtain the microscopic super-resolution image.

[0077] Specifically, the multiple large-size images containing structured light illumination information obtained in the first step are input into the structured light super-resolution reconstruction algorithm. For example, 9 images containing structured light illumination information (i=1,2,3...,9) is input into the structured light super-resolution reconstruction algorithm. First, Perform Fourier transform and get , where u and v are the spatial frequencies in the x and y directions, respectively, and x and y represent the horizontal and vertical coordinates in the two-dimensional coordinate system. The original image in the spectrum of each image is obtained using a pre-set window function. The spectrum of each original image is extracted and the spectrum of the extracted original image is shifted to the center of the entire spectrum. A new spectrum is synthesized and inverse Fourier transformed. The resulting image is , which is the super-resolution image of the sample.

[0078] In this way, by inputting 9n images into the structured light super-resolution reconstruction algorithm, super-resolution images of n samples can be obtained.

[0079] This super-resolution microscopy method can protect biological cell samples from phototoxicity and photobleaching, and achieve super-resolution imaging in structured light illumination microscopy systems.

[0080] In summary, the microscopic imaging device provided by the present application has at least the following advantages: the observation environment targeted by this solution is more extreme (low lighting conditions), and the device can adapt to super-resolution reconstruction of images under low lighting conditions, can image the structure of the sample at a resolution level of hundreds of nanometers, and can achieve long-time exposure acquisition and reconstruction. Secondly, the image processing process of the present application is implemented in two steps, and they are interconnected. Changing the order or using alternative solutions cannot achieve the final result. Thirdly, the present application also has the advantage of low cost. Since microscopic solutions under ordinary low lighting conditions all use extremely expensive low-light cameras such as sCMOS, they cannot be widely popularized. The acquisition equipment used in the present invention is an ordinary linear array camera, which greatly improves the utilization rate of light while being low-cost.

[0081] It will be understood that the word "exemplary" as used herein means "serving as an example, instance, or illustration." Any embodiment described as "exemplary" is not necessarily preferred or advantageous over other embodiments and / or does not exclude the ability to combine features of other embodiments. It will be understood that certain features of the present application, which are described in the context of separate embodiments for the sake of clarity, may also be provided in combination in a single embodiment. Conversely, various features of the present application, which are described in the context of a single embodiment for the sake of clarity, may also be provided separately or in any suitable combination or as any other described embodiment of the present application.

[0082] The above disclosure is only a preferred embodiment of the present application, but it is not intended to limit the scope of rights of the present application. A person skilled in the art can understand that without departing from the spirit and scope of the present application and the appended claims, changes, modifications, substitutions, combinations, and simplifications should all be equivalent replacement methods and still fall within the scope of the invention.

Claims

1. A microscopic imaging device, characterized in that: The microscopic imaging device comprises: Sample table, used to hold samples; Laser, for emitting laser light; a first digital micromirror configured to receive the laser light and form a grating image to modulate the laser light to generate diffracted light of multiple diffraction orders; a light focusing unit configured to converge the plurality of diffracted lights generated by the first digital micromirror, causing the diffracted lights to interfere with each other to form structured light, and to focus the structured light on the sample to excite fluorescence carrying sample information; a second digital micromirror, receiving the fluorescence and converting the fluorescence into a random binary coded image; a cylindrical lens, the cylindrical lens being disposed in the optical path between the second digital micromirror and the image acquisition unit, and being configured to converge the random binary coded image into light rays distributed along a preset direction; an image acquisition unit, comprising a line array camera, the line array camera being disposed on a converging surface of the cylindrical lens, the line array camera being configured to receive the random binary coded image converged by the cylindrical lens, and transmit the random binary coded image to a processing unit; a processing unit configured to reconstruct a microscopic super-resolution image based on a plurality of random binary coded images; The first digital micromirror, the second digital micromirror, and the image acquisition unit are communicatively connected, and the second digital micromirror sends control instructions to the first digital micromirror and the image acquisition unit, respectively, to control the grating pattern conversion frequency of the first digital micromirror and the image acquisition frequency of the image acquisition unit; The refresh frequency of the random binary coded image of the second digital micromirror is defined as N, where N is a natural number greater than 1. Then, the refresh frequency of the random binary coded image of the second digital micromirror is N times the grating pattern transformation frequency of the first digital micromirror, and the refresh frequency of the random binary coded image of the second digital micromirror is the same as the image acquisition frequency of the image acquisition unit.

2. The microscopic imaging device according to claim 1, characterized in that The first digital micromirror is configured to play 9 grating patterns in time sequence to generate different diffracted lights, wherein the 9 grating patterns include three grating directions of 0°, 60°, and 120°, and each grating direction includes 0, , Three phases.

3. The microscopic imaging device according to claim 2, characterized in that The first digital micromirror is configured to modulate the laser to generate three diffraction orders of 0, +1, and -1; The microscopic imaging device further includes a filter configured to filter out the 0th order diffraction light and allow the +1st order diffraction light and the -1st order diffraction light to pass.

4. The microscopic imaging device according to claim 3, characterized in that The microscopic imaging device further includes a dichroic mirror configured to reflect the +1st order diffraction light and the -1st order diffraction light to the focusing unit; The focusing unit converges the +1st order diffraction light and the -1st order diffraction light to generate interference, thereby forming the structured light.

5. The microscopic imaging device according to claim 1, characterized in that The second digital micromirror is configured to: in response to refreshing the random binary coded image once, send a control instruction to the image acquisition unit to trigger the image acquisition unit to perform image acquisition; in response to refreshing the random binary coded image N+1 times, reset the refresh count calculation of the random binary coded image in this round, and send a control instruction to the first digital micromirror to trigger the first digital micromirror to perform raster pattern transformation.

6. The microscopic imaging device according to claim 1, characterized in that The processing unit is configured to: use a deep learning image reconstruction algorithm to reconstruct a plurality of the random binary coded images to obtain an image with a structured light illumination pattern, and perform microscopic super-resolution reconstruction on the plurality of images with the structured light illumination pattern to obtain the microscopic super-resolution image.

7. The microscopic imaging device according to claim 6, characterized in that The reconstructing of the plurality of random binary coded images using a deep learning image reconstruction algorithm includes grouping the continuously acquired measurement data according to a target frame rate and constructing an imaging model. The imaging model is represented by the following formula: ； Where, The scene frames associated with the desired frame rate, , Q is a constant greater than 1, T represents time, represents the measurement noise, represents the measurement matrix, Represents multiple reconstructed large-size image information containing structured light illumination information.

8. The microscopic imaging device according to claim 7, characterized in that: The performing microscopic super-resolution reconstruction on the plurality of images with the structured light illumination patterns to obtain the microscopic super-resolution image comprises: Perform Fourier transform on 3M images with structured light illumination patterns to obtain the spectrum of each image, where M is 1, 2, or 3; A pre-set window function is used to obtain the original image in the spectrum of each image, the spectrum of each original image is extracted, and the spectrum of the extracted original image is translated to the center of the entire spectrum, a new spectrum is synthesized, and an inverse Fourier transform is performed on it to obtain the microscopic super-resolution image.

9. A super-resolution microscopy method, characterized in that: The method uses the microscopic imaging device according to any one of claims 1 to 6, and the method comprises: A deep learning image reconstruction algorithm is used to reconstruct a plurality of random binary coded images to obtain an image with a structured light illumination pattern, and microscopic super-resolution reconstruction is performed on the plurality of images with the structured light illumination pattern to obtain the microscopic super-resolution image.

Images