Multi-mode head-mounted small microscope chromatography method and device

Abstract

The invention discloses a multi-mode head-mounted small microscope chromatography method and device, and relates to the technical field of image reconstruction, and the method comprises the specific steps: generating a uniform illumination image and a time sequence fringe image in a target region through a time sequence uniform illumination light beam and a fringe illumination light beam; collecting the time sequence fringe image frame by frame according to a time sequence, and performing fusion processing on the time sequence fringe image and the uniform illumination image to generate a fused image; and performing calculation reconstruction based on the fused image to generate a tomographic image. Structured light illumination is combined with fringe projection coding, the focal plane signal intensity is selectively enhanced through optical modulation, out-of-focus background fluorescence interference is effectively inhibited, and the imaging contrast in complex biological tissue is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of image reconstruction technology, and more particularly to a multimodal head-mounted miniature microscope tomography method and device. Background Art

[0002] Currently, head-mounted miniaturized fluorescence microscopy has become an essential tool for studying neural activity in freely behaving animals. Single-photon widefield microscopy, through area array imaging, achieves millimeter-scale fields of view and millisecond-scale dynamic capture, while two-photon microscopy, through point scanning, surpasses the optical tomography capabilities of deep-tissue imaging. These two systems have achieved significant breakthroughs in the spatiotemporal resolution and imaging depth of in vivo neural signal acquisition, respectively, providing key experimental tools for analyzing the dynamics of neural circuits.

[0003] However, single-photon widefield systems lack axial resolution and face inherent limitations due to out-of-focus background fluorescence interference in complex biological tissues, resulting in a significant decrease in effective signal contrast. While two-photon systems possess excellent optical tomography properties, their point-scanning imaging mode is constrained by the spatiotemporal bandwidth product, making it difficult to simultaneously achieve a wide field of view and high frame rate while maintaining micron-level resolution. This technical bottleneck makes existing solutions incompatible with the quadruple technical indicators of whole-brain neural cluster observation, single-cell precision analysis, millisecond-level dynamic capture, and three-dimensional tomography.

[0004] Therefore, how to break through the physical limitations of existing microscopic imaging technology and develop a tomographic microscopic system with large field of view and high-speed area array imaging capabilities is an urgent problem that technicians in this field need to solve. Summary of the Invention

[0005] In view of this, the present invention provides a multimodal head-mounted small microscope tomography method and device to overcome the above-mentioned defects.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A multimodal head-mounted miniature microscope tomography method, comprising the following steps:

[0008] Generate a uniform illumination image and a time-sequential stripe image in a target area using a time-sequential uniform illumination beam and a stripe illumination beam;

[0009] Acquire the time-sequential fringe image frame by frame in a time sequence, and fuse the time-sequential fringe image with the uniform illumination image to generate a fused image;

[0010] Computational reconstruction is performed based on the fused image to generate a tomographic image.

[0011] Furthermore, the expression of the tomographic image is:

[0012] I P =[(I1-I2) 2 +(I1-I3) 2 +(I2-I3) 2 ] 1 / 2 ;

[0013] Where I1, I2, and I3 represent the image intensity values obtained at the same pixel under different phases or different lighting modes.

[0014] Furthermore, during the tomographic image reconstruction process, frequency domain filtering is introduced, and the specific steps are as follows:

[0015] Perform Fourier transform on the initial tomographic image to obtain a frequency domain image;

[0016] Using a Gaussian filter function to filter the frequency domain image;

[0017] The filtered frequency domain image is converted back to the spatial domain using inverse Fourier transform to obtain a tomographic image with fringe interference removed.

[0018] Furthermore, the expression of frequency domain filtering is:

[0019]

[0020] Where, represents the image after filtering; F represents Fourier transform; F -1 represents the inverse Fourier transform; exp represents the exponential function; x, y represent the spatial or frequency domain coordinates; ɑ, β represent the parameters of the filter; δ 2 Represents the parameter that controls the bandwidth of the filter.

[0021] A multimodal head-mounted miniature microscope tomography device, comprising:

[0022] Synchronous control device, used to coordinate the operation timing synchronization of various devices;

[0023] An imaging device for collecting optical information of a target area and converting it into digital image data;

[0024] A structured light illumination device, comprising multiple illumination sources and a stripe grating, for projecting multi-directional structured light stripes onto a target area to provide illumination support for imaging;

[0025] The reconstruction calculation device is used to perform tomographic algorithm processing on the digital image data to generate a tomographic image.

[0026] Furthermore, it also includes a carrying device having a magnetic interface for integrating and fixing the synchronization control device, the imaging device and the structured light illumination device.

[0027] Furthermore, the imaging device integrates a CMOS and a primary mirror, wherein the CMOS is used to capture optical signals and convert them into digital image data; and the primary mirror is used to focus and amplify optical information of the target object.

[0028] Furthermore, the reconstruction calculation device further includes a filtering unit, which includes:

[0029] A frequency domain conversion subunit, configured to perform Fourier transform on the initial tomographic image to obtain a frequency domain image;

[0030] An image filtering subunit, configured to filter the frequency domain image using a Gaussian filter function;

[0031] The space and conversion subunit is used to convert the filtered frequency domain image back to the space domain by using the inverse Fourier transform to obtain a tomographic image with stripe interference removed.

[0032] As can be seen from the above technical solutions, the present invention provides a multimodal head-mounted miniature microscope tomography method and device, which has the following beneficial effects compared with the prior art:

[0033] 1. Structured light illumination combined with fringe projection coding selectively enhances focal plane signal intensity through optical modulation, effectively suppresses out-of-focus background fluorescence interference, and significantly improves imaging contrast in complex biological tissues.

[0034] 2. Using structured light spatial frequency domain coding, we achieve millimeter-level field of view and high frame rate synchronous imaging, breaking through the limitations of space-time bandwidth product and meeting the needs of millisecond-level dynamic capture of neural activity.

[0035] 3. Based on the phase information of multi-angle fringe projection and deep learning reconstruction algorithm, the three-dimensional spatial distribution of the sample can be reconstructed with only a single exposure, supporting three-dimensional tomographic imaging of brain tissue in freely behaving animals. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0037] Figure 1 A schematic flow chart of the method provided by the present invention;

[0038] Figure 2 A schematic diagram of the internal connections of the device provided by the present invention;

[0039] Figure 3A schematic diagram of the structure of the device provided by the present invention;

[0040] Figure 4 A schematic diagram of the principle of improving tomographic imaging effects using stripe structured illumination provided by the present invention;

[0041] In the figure, 301 is the PCB board; 302 is the CMOS; 303 is the main mirror; 304 is LED-1; 305 is LED-2; 306 is LED-3; 307 is LED-4; 308 is the stripe grating-1; 309 is the stripe grating-2; 310 is the stripe grating-3; 311 is the magnetic hole. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] On the one hand, the embodiment of the present invention discloses a multimodal head-mounted small microscope tomography method, such as Figure 1 As shown, the specific steps are:

[0044] Generate a uniform illumination image and a time-sequential stripe image in a target area using a time-sequential uniform illumination beam and a stripe illumination beam;

[0045] Collecting time-series fringe images frame by frame in time sequence, and fusing the time-series fringe images with the uniform illumination image to generate a fused image;

[0046] Computational reconstruction is performed based on the fused images to generate tomographic images.

[0047] Furthermore, first, an excitation light beam is generated, specifically: it is used to generate a light beam that meets the requirements of time-sequential uniform illumination or stripe illumination; wherein, uniform illumination is intended to ensure that the light intensity is evenly distributed on the target surface, while stripe illumination is used to form a stripe pattern with a specific period on the target surface; then imaging is performed, which is mainly used to collect time-sequential uniform illumination images or stripe images; finally, computational reconstruction is performed, which is a key link in generating high signal-to-noise ratio tomographic images. By processing and analyzing the time-sequential stripe images, the rich information contained therein is fully utilized, and the structured illumination reconstruction method is used to accurately reconstruct the target area, thereby obtaining a clear and high signal-to-noise ratio tomographic image.

[0048] In one embodiment, the multi-frame image processing technology of structured illumination microscopy is used to evaluate the image contrast by the following formula (1). Wherein, I1, I2, and I3 represent the image intensity values obtained at the same pixel under different phases or different illumination modes. The tomographic image I can be obtained by the following formula p .

[0049] I P =[(I1-I2) 2 +(I1-I3) 2 +(I2-I3) 2 ] 1 / 2 , (1).

[0050] In one embodiment, frequency domain filtering is introduced during the tomographic image reconstruction process, and the specific steps are as follows:

[0051] Perform Fourier transform on the initial tomographic image to obtain a frequency domain image;

[0052] The frequency domain image is filtered using a Gaussian filter function;

[0053] The filtered frequency domain image is converted back to the spatial domain using inverse Fourier transform to obtain a tomographic image with fringe interference removed.

[0054] Furthermore, considering that in actual applications, the edges of the stripes I1, I2, and I3 may overlap in space due to tolerance and installation, which may lead to the reconstructed tomographic image I P There is stripe interference in the image. Therefore, a method for removing stripes based on frequency domain filtering is proposed on the basis of tomographic image reconstruction. Its expression is:

[0055]

[0056] This technology can effectively improve the imaging quality, highlight the image details and enhance the image quality. represents the processed image (usually an optical section image obtained by filtering or an image with fringe interference removed); F represents Fourier transform, which is used to convert the image from the spatial domain to the frequency domain; F -1 stands for inverse Fourier transform, which converts the frequency domain image back to the spatial domain; exp exponential function represents the Gaussian filter function; x, y represent the spatial domain or frequency domain coordinates; ɑ, β represent the parameters of the filter (ɑ, β∈[0,1]); δ 2 Represents the parameter that controls the bandwidth of the filter, which is usually related to the filtering strength and range.

[0057] This embodiment utilizes a combination of three-sided stripe illumination and one-sided uniform illumination. Stripe illumination enhances the structural information of tomographic imaging, improving overall imaging quality. Stripe illumination has a time-sequential nature, capturing stripe images frame by frame in chronological order. The captured time-sequential stripe images and uniform illumination images are fused to extract tomographic feature information of the target region. Computational reconstruction is performed based on the fused data to generate a tomographic image with a high signal-to-background ratio. This method, through the combination of stripe illumination and uniform illumination, fuses the time-sequential stripe images with the uniform illumination images, extracts tomographic feature information of the target region, and computationally reconstructs a tomographic image with a high signal-to-noise ratio, effectively improving overall imaging quality and the detailed representation of structural information.

[0058] On the other hand, this embodiment further discloses a multimodal head-mounted small microscope tomography device, comprising:

[0059] Synchronous control device, used to coordinate the operation timing synchronization of various devices;

[0060] An imaging device for collecting optical information of a target area and converting it into digital image data;

[0061] A structured light illumination device, comprising multiple illumination sources and a stripe grating, for projecting multi-directional structured light stripes onto a target area to provide illumination support for imaging;

[0062] The reconstruction computing device is used to perform tomographic algorithm processing on the digital image data to generate a tomographic image.

[0063] In one embodiment, a carrying device is further included, which has a magnetic hole 311 and is used to integrate and fix the synchronization control device, the imaging device and the structured light illumination device.

[0064] In one embodiment, the imaging device integrates a CMOS 302 and a primary mirror 303 . The CMOS 302 is used to capture optical signals and convert them into digital image data; the primary mirror 303 is used to focus and magnify optical information of a target object.

[0065] In one embodiment, the reconstruction calculation device further includes a filtering unit, which includes:

[0066] A frequency domain conversion subunit, configured to perform Fourier transform on the initial tomographic image to obtain a frequency domain image;

[0067] An image filtering subunit, configured to filter the frequency domain image using a Gaussian filter function;

[0068] The space and conversion subunit is used to convert the filtered frequency domain image back to the spatial domain using the inverse Fourier transform to obtain a tomographic image with stripe interference removed.

[0069] Furthermore, its structure is as follows Figure 2 and Figure 3 As shown, the system comprises: a synchronization control device, including a PCB board 301, which coordinates the actions and operating sequence of each device or module to ensure that the system operates precisely and synchronously according to the predetermined process; an imaging device, including a COMS 302 and a primary mirror 303, which uses optical and electronic technologies to collect optical information of the target object and convert it into a visual image for observation, recording, and analysis; a structured light illumination device, including LED-1 304, LED-2 305, LED-3 306, LED-4 307, and stripe grating 1 308, stripe grating 2 309, and stripe grating 3 310, which enhances the surface feature information of the target object by projecting a specific structure of light pattern stripes, providing illumination support for high-precision imaging; a reconstruction calculation device, which processes and performs algorithmic operations on the collected data to restore the target information into a tomographic image with a high signal-to-background ratio; and a supporting device, which provides a stable support and mounting platform for fixing and integrating system components, ensuring stable operation and convenient operation of the equipment. The above modules work together to achieve high precision and high efficiency in tomographic imaging using a multimodal microscope.

[0070] The synchronous control device, imaging device and structured light illumination device together constitute the body of the small head-mounted microscope. Figure 3 As shown, the body of the small head-mounted microscope includes: PCB board 301, CMOS 302, main mirror 303, LED304-307, and stripe grating 308-310. The PCB board 301 is used to carry and integrate the electronic components and circuits of the microscope to realize the signal processing, data transmission and power supply management functions of the device. CMOS 302 serves as the image sensor of the microscope, which is used to capture optical signals and convert them into digital image data for subsequent processing and display. The main mirror 303 is the core optical component of the head-mounted microscope, which is used to focus and amplify the optical information of the target object to ensure imaging quality and resolution. LED serves as an illumination light source to provide high-brightness and stable lighting conditions for the target object, supporting multimodal imaging requirements. The stripe grating is used to generate structured light stripes, which enhances the surface feature information of the target object by modulating the illumination pattern, providing support for tomographic imaging, such as Figure 4 Magnetic holes 311 are used to install and secure modular components, enabling quick assembly and disassembly through magnetic connection, thus improving the operational convenience and flexibility of the device.

[0071] To achieve a lightweight design for a small head-mounted microscope, this embodiment strictly controls the weight of the device to less than 3 grams, effectively improving the wearing comfort and normal movement flexibility of experimental mice in dynamic scenes. The weight distribution of each component has been carefully optimized, specifically: PCB board 3010.9 grams, CMOS 302 approximately 1 gram, primary mirror 303 approximately 0.8 grams, LED approximately 0.1 grams, and magnetic iron approximately 0.2 grams. In addition, the body of the integrated small head-mounted microscope system is coated with a lightweight, high-strength resin material to further reduce weight and improve structural strength. In order to simultaneously achieve lightweight, miniaturization, and portability of the device, this embodiment also optimizes the volume of the microscope to approximately 2 cubic centimeters.

[0072] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0073] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multimodal head-mounted miniature microscope tomography method, characterized in that: The specific steps are: Generate a uniform illumination image and a time-sequential stripe image in a target area using a time-sequential uniform illumination beam and a stripe illumination beam; Acquire the time-sequential fringe image frame by frame in a time sequence, and fuse the time-sequential fringe image with the uniform illumination image to generate a fused image; Computational reconstruction is performed based on the fused image to generate a tomographic image.

2. A multimodal head-mounted miniature microscope tomography method according to claim 1, characterized in that: The expression of the tomographic image is: I P =[(I1-I2) 2 +(I1-I3) 2 +(I2-I3) 2 ] 1 / 2 ; Where I1, I2, and I3 represent the image intensity values obtained at the same pixel under different phases or different lighting modes.

3. A multimodal head-mounted miniature microscope tomography method according to claim 1, characterized in that: During the tomographic image reconstruction process, frequency domain filtering is introduced, and the specific steps are as follows: Perform Fourier transform on the initial tomographic image to obtain a frequency domain image; Using a Gaussian filter function to filter the frequency domain image; The filtered frequency domain image is converted back to the spatial domain using inverse Fourier transform to obtain a tomographic image with fringe interference removed.

4. A multimodal head-mounted miniature microscope tomography method according to claim 3, characterized in that: The expression of frequency domain filtering is: Where, represents the image after filtering; F represents Fourier transform; F -1 represents the inverse Fourier transform; exp represents the exponential function; x, y represent the spatial or frequency domain coordinates; ɑ, β represent the parameters of the filter; δ 2 Represents the parameter that controls the bandwidth of the filter.

5. A multimodal head-mounted miniature microscope tomography device, characterized in that: include: Synchronous control device, used to coordinate the operation timing synchronization of various devices; An imaging device for collecting optical information of a target area and converting it into digital image data; A structured light illumination device, comprising multiple illumination sources and a stripe grating, for projecting multi-directional structured light stripes onto a target area to provide illumination support for imaging; The reconstruction calculation device is used to perform tomographic algorithm processing on the digital image data to generate a tomographic image.

6. The multimodal head-mounted miniature microscope tomography device according to claim 5, characterized in that: It also includes a carrying device having a magnetic hole (311) for integrating and fixing the synchronous control device, the imaging device and the structured light illumination device.

7. The multimodal head-mounted miniature microscope tomography device according to claim 5, characterized in that: The imaging device is integrated with a CMOS (302) and a primary mirror (303), wherein the CMOS (302) is used to capture optical signals and convert them into digital image data; and the primary mirror (303) is used to focus and amplify optical information of a target object.

8. The multimodal head-mounted miniature microscope tomography device according to claim 5, characterized in that: The reconstruction calculation device further includes a filtering unit, which includes: A frequency domain conversion subunit, configured to perform Fourier transformation on the initial tomographic image to obtain a frequency domain image; An image filtering subunit, configured to filter the frequency domain image using a Gaussian filter function; The space and conversion subunit is used to convert the filtered frequency domain image back to the space domain by using the inverse Fourier transform to obtain a tomographic image with stripe interference removed.

Images