High-temporal-spatial-resolution large-volume three-dimensional microscopic imaging device and method

Abstract

The invention relates to a high-temporal-spatial-resolution large-volume three-dimensional microscopic imaging device and method, and the device comprises an exciting light generation module which is used for generating strip-shaped illumination exciting light; the scanning module is used for determining a fluorescence signal emitted by the sample and performing de-scanning on the fluorescence signal to generate a de-scanned fluorescence signal; the imaging module is used for carrying out optical spatial multiplexing imaging on the spatial frequency spectrum information of the fluorescence signals after de-scanning, traversing all the strip-shaped sub-view fields according to corresponding multi-view angle imaging results, and generating imaging results of all the strip-shaped sub-view fields under different imaging view angles; and the reconstruction module is used for reconstructing imaging results of all the strip-shaped sub-view fields under different imaging view angles so as to determine three-dimensional reconstruction images of all the strip-shaped sub-view fields, and splicing the three-dimensional reconstruction images of all the strip-shaped sub-view fields so as to generate a three-dimensional microscopic image of the target large view field. Therefore, the problem that large-volume and high-temporal-spatial-resolution three-dimensional microscopic imaging and the like are difficult to realize in related technologies is solved.

Description

Technical Field

[0001] The present application relates to the field of optical microscopy technology, and particularly relates to a large-volume three-dimensional microscopic imaging device and method with high spatio-temporal resolution. Background Art

[0002] Biological dynamics is a basic attribute of life, which is usually rapidly coordinated by various functional tissues and cells distributed in three-dimensional space. Therefore, biological research cannot be carried out without observing biological dynamics, and there is an urgent need to develop three-dimensional microscopic imaging technology with large volume and high spatio-temporal resolution.

[0003] Traditional three-dimensional microscopic imaging is usually point-by-point scanning imaging (such as two-photon microscopes, confocal microscopes, etc.) or plane-by-plane scanning imaging (such as light sheet microscopes, etc.). Limited by the mechanical inertia of the scanning device, especially the mechanical inertia of the axial scanning device, its three-dimensional imaging speed is difficult to meet the application requirements of biological dynamics observation. In recent years, Fourier light field microscopy has obtained fast three-dimensional imaging capabilities by recording two-dimensional images from different perspectives and then combining three-dimensional reconstruction. However, since it needs to simultaneously collect information of spatial-spatial frequencies, the imaging field of view in a single exposure is limited by the number of camera pixels, and it is difficult to record all the information within the large field of view supported by the objective lens at a high sampling rate, and thus it is difficult to achieve large-volume and high-spatial-resolution microscopic imaging.

[0004] By adopting the method of sub-field scanning, Fourier light field microscopy can effectively achieve large-volume and high-spatial-resolution imaging. However, limited by the imaging frame rate of the camera, the imaging speed is usually limited within 100Hz, and it is still difficult to meet the requirements of high-speed dynamic imaging. The data readout method of commercial CMOS cameras is row readout. The read time of a single image is related to the number of rows of the image and has nothing to do with the number of columns of the image. Therefore, the imaging frame rate of the camera can be achieved by compressing the number of rows of the image. However, in a general Fourier light field microscope, reducing the number of rows used by the camera for imaging will lose the imaging results of some perspectives, which is difficult to meet the reconstruction requirements, and thus the spatial resolution or the imaging field of view will be lost, and there is an urgent need for improvement. Summary of the Invention

[0005] The present application provides a large-volume three-dimensional microscopic imaging device and method with high spatio-temporal resolution to solve problems such as the difficulty in achieving large-volume and high-spatio-temporal-resolution three-dimensional microscopic imaging in related technologies.

[0006] An embodiment of the first aspect of the present application provides a large-volume three-dimensional microscopic imaging device with high spatio-temporal resolution, including: an excitation light generation module for generating bar-shaped illumination excitation light; a scanning module for deflecting the excitation light to a bar-shaped sub-field of view, exciting a sample within the bar-shaped sub-field of view to determine a fluorescence signal emitted by the sample, and performing de-scanning on the fluorescence signal to generate a de-scanned fluorescence signal; an imaging module for performing optical spatial multiplexing imaging on spatial frequency spectrum information of the de-scanned fluorescence signal to generate a multi-view imaging result corresponding to the bar-shaped sub-field of view, traversing all bar-shaped sub-fields of view according to the corresponding multi-view imaging results until multi-view imaging results of all bar-shaped sub-fields of view are obtained, and segmenting the multi-view imaging results according to different imaging perspectives to generate imaging results of all bar-shaped sub-fields of view under different imaging perspectives; a reconstruction module for reconstructing the imaging results of all bar-shaped sub-fields of view under the different imaging perspectives to determine three-dimensional reconstruction images of all bar-shaped sub-fields of view, and stitching the three-dimensional reconstruction images of all bar-shaped sub-fields of view to generate a three-dimensional microscopic image of a target large field of view.

[0007] Optionally, in an embodiment of the present application, the excitation light generation module includes: a laser light source for emitting the excitation light; an excitation end filter for filtering the excitation light to generate filtered excitation light; a first lens and a second lens for expanding the filtered excitation light to generate expanded excitation light; a first aperture for restricting an illumination range of the expanded excitation light to generate bar-shaped illumination excitation light; a third lens for performing an optical Fourier transform on the bar-shaped illumination excitation light to generate target excitation light.

[0008] Optionally, in an embodiment of the present application, the scanning module includes: a dichroic mirror for reflecting the excitation light to generate reflected excitation light; a one-dimensional scanning galvanometer for deflecting the reflected excitation light at a preset angle to generate deflected excitation light; a fourth lens and a fifth lens for relaying the deflected excitation light at a preset magnification to generate relayed excitation light; an objective lens for collecting the fluorescence signal of the sample and sequentially passing the fluorescence signal back through the fifth lens, the fourth lens, and the one-dimensional scanning galvanometer to generate the de-scanned fluorescence signal.

[0009] Optionally, in an embodiment of the present application, the imaging module includes: a sixth lens for focusing the fluorescence signal to determine the imaging on the rear focal plane of the sixth lens; a second aperture located on the rear focal plane of the sixth lens for restricting the field of view of the imaging on the rear focal plane of the sixth lens within a target range; a seventh lens for performing an optical Fourier transform on the imaging on the rear focal plane of the sixth lens within the target range to generate spatial spectrum information after the optical Fourier transform; a collecting end filter located on the rear focal plane of the seventh lens for filtering out the residual excitation light in the imaging light; a microlens array composed of a plurality of microlenses and located on the rear focal plane of the seventh lens for performing spatial multiplexing imaging on the spatial spectrum information after the optical Fourier transform to generate the multi-view imaging result; and a camera located on the rear focal plane of the microlens array for recording the multi-view imaging result.

[0010] Optionally, in an embodiment of the present application, the reconstruction module includes: a view angle segmentation unit for calibrating the imaging position coordinates of each sub-view angle on the fluorescent microsphere at the imaging depth center depth based on the imaging result at the imaging depth center depth, and performing view angle segmentation on the original imaging result of the strip-shaped sub-field of view according to the imaging position coordinates to obtain the imaging results of the strip-shaped sub-field of view at different imaging view angles; a per-pixel adaptive correction unit for obtaining an image calibration value that satisfies a preset low aberration condition based on the imaging results of the strip-shaped sub-field of view at different imaging view angles with the imaging result at the central view angle as a reference, and performing per-pixel alignment on the imaging results according to the image calibration value that satisfies the preset low aberration condition to generate a corrected multi-view imaging result; a three-dimensional reconstruction unit for three-dimensionally reconstructing the data processed by view angle segmentation and adaptive correction by using the three-dimensional point spread function of the strip-shaped sub-field of view based on the corrected multi-view imaging result to determine a three-dimensional imaging result that satisfies a preset condition; and a three-dimensional stitching unit for calculating the intensity ratio of the overlapping area between adjacent strip-shaped sub-fields of view based on the three-dimensional imaging result that satisfies the preset condition, linearly compensating the adjacent strip-shaped sub-fields of view according to the intensity ratio to generate a linear compensation result, and stitching the three-dimensional reconstruction images of all the strip-shaped sub-fields of view into a three-dimensional microscopic image of the target field of view based on the linear compensation result.

[0011] The second aspect of the present application provides a method for large-volume three-dimensional microscopic imaging with high spatio-temporal resolution, including: generating an excitation light with bar-shaped illumination; deflecting the excitation light to a bar-shaped sub-field of view, exciting a sample within the bar-shaped sub-field of view to determine a fluorescence signal emitted by the sample, and performing de-scanning on the fluorescence signal to generate a de-scanned fluorescence signal; performing optical spatial multiplexing imaging on the spatial frequency spectrum information of the de-scanned fluorescence signal to generate a multi-view imaging result corresponding to the bar-shaped sub-field of view, traversing all bar-shaped sub-fields of view according to the corresponding multi-view imaging results until the multi-view imaging results of all bar-shaped sub-fields of view are obtained, and splitting the multi-view imaging results according to different imaging perspectives to generate imaging results of all bar-shaped sub-fields of view under different imaging perspectives; reconstructing the imaging results of all bar-shaped sub-fields of view under the different imaging perspectives to determine three-dimensional reconstruction images of all bar-shaped sub-fields of view, and stitching the three-dimensional reconstruction images of all bar-shaped sub-fields of view to generate a three-dimensional microscopic image of a target large field of view.

[0012] Optionally, in an embodiment of the present application, the generating an excitation light with bar-shaped illumination includes: emitting the excitation light; performing filtering processing on the excitation light to generate a filtered excitation light; expanding the filtered excitation light to generate an expanded excitation light; restricting the illumination range of the expanded excitation light to generate the excitation light with bar-shaped illumination; performing an optical Fourier transform on the excitation light with bar-shaped illumination to generate a target excitation light.

[0013] Optionally, in an embodiment of the present application, the deflecting the excitation light to a bar-shaped sub-field of view, exciting a sample within the bar-shaped sub-field of view to determine a fluorescence signal emitted by the sample, and performing de-scanning on the fluorescence signal to generate a de-scanned fluorescence signal includes: reflecting the excitation light to generate a reflected excitation light; deflecting the reflected excitation light at a preset angle to generate a deflected excitation light; relaying the deflected excitation light at a preset magnification to generate a relayed excitation light; collecting the fluorescence signal of the sample and generating a de-scanned fluorescence signal.

[0014] Optionally, in an embodiment of the present application, the generating imaging results of all bar-shaped sub-fields of view under different imaging perspectives includes: focusing the fluorescence signal; generating spatial frequency spectrum information after an optical Fourier transform; filtering out residual excitation light in the imaging light; performing spatial multiplexing imaging on the spatial frequency spectrum information after the optical Fourier transform to generate the multi-view imaging result; recording the multi-view imaging result.

[0015] Optionally, in an embodiment of the present application, reconstructing the imaging results of all the strip sub - fields of view at different imaging perspectives to determine the three - dimensional reconstruction images of all the strip sub - fields of view, and stitching the three - dimensional reconstruction images of all the strip sub - fields of view to generate a three - dimensional microscopic image of a target large field of view includes: Based on the imaging results at the depth of the imaging depth - of - field center, calibrating the imaging position coordinates of the fluorescence microspheres at each sub - perspective at the depth of the imaging depth - of - field center, and performing perspective segmentation on the original imaging results of the strip sub - field of view according to the imaging position coordinates to obtain the imaging results of the strip sub - field of view at different imaging perspectives; Based on the imaging results of the strip sub - field of view at different imaging perspectives, taking the imaging result at the central perspective as a reference, obtaining an image calibration value that meets the preset low - aberration condition, and performing pixel - by - pixel alignment on the imaging results according to the image calibration value that meets the preset low - aberration condition to generate a corrected multi - perspective imaging result; Based on the corrected multi - perspective imaging result, using the three - dimensional point spread function of the strip sub - field of view to three - dimensionally reconstruct the data that has undergone perspective segmentation and adaptive correction processing to determine a three - dimensional imaging result that meets the preset conditions; Based on the three - dimensional imaging result that meets the preset conditions, calculating the intensity ratio of the overlapping regions between adjacent strip sub - fields of view, linearly compensating the adjacent strip sub - fields of view according to the intensity ratio to generate a linear compensation result, and based on the linear compensation result, stitching the three - dimensional reconstruction images of all the strip sub - fields of view into the three - dimensional microscopic image of the target field of view.

[0016] In the embodiment of the present application, by optimizing the design of the microlens array, the imaging of the strip sub - field of view is realized. While ensuring the imaging field of view, the requirement of the Fourier light - field microscope for the number of rows in the camera target surface is reduced, the read - out time of the image is reduced, the imaging speed is accelerated, and fast three - dimensional microscopic imaging with large volume and high spatio - temporal resolution is realized; By introducing a scanning module, the imaging area can be flexibly adjusted according to actual needs, full - field scanning imaging can be realized, and scanning imaging between any fields of view can also be realized, avoiding data redundancy and further accelerating the imaging speed; By using a single - axis galvanometer for scanning between strip sub - fields of view, the problem that it is difficult for multi - axis galvanometers to be optically conjugated to each other is avoided, so that the relative position between the spatial frequency spectrum plane of the imaging light and the microlens array remains stable, ensuring the consistency of the point spread function within the entire field of view. Thus, the problems in the ordinary Fourier light - field microscope are solved, such as when reducing the number of rows used by the camera for imaging, the imaging results of some perspectives will be lost, it is difficult to meet the reconstruction requirements, and thus the spatial resolution or the imaging field of view will be lost.

[0017] Additional aspects and advantages of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description of embodiments in conjunction with the accompanying drawings, where:

[0019] Figure 1 FIG. 4 is a schematic structural diagram of a large-volume three-dimensional microscopic imaging device with high spatio-temporal resolution provided according to an embodiment of the present application;

[0020] Figure 2 FIG. 8 is a schematic diagram of a large-volume three-dimensional microscopic imaging device with high spatio-temporal resolution according to an embodiment of the present application;

[0021] Figure 3 FIG. 12 is a schematic diagram of the mapping relationship between a microlens array and the exposure area of a camera according to an embodiment of the present application;

[0022] Figure 4 FIG. 16 is a schematic diagram of a method for all-field scanning imaging according to an embodiment of the present application;

[0023] Figure 5 FIG. 20 is a schematic diagram of a method for scanning and imaging a target observation area according to an embodiment of the present application;

[0024] Figure 6 FIG. 24 is a schematic flow diagram of a method for scanning and imaging a target observation area according to an embodiment of the present application;

[0025] Figure 7 FIG. 28 is a schematic diagram of a computational reconstruction method for high spatio-temporal resolution large-volume three-dimensional microscopic imaging according to an embodiment of the present application;

[0026] Figure 8 FIG. 32 is a schematic flow diagram of a method for high spatio-temporal resolution large-volume three-dimensional microscopic imaging provided according to an embodiment of the present application; Detailed Embodiments

[0027] The embodiments of the present application will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to explain the present application, and should not be construed as limiting the present application.

[0028] Before introducing the high spatio-temporal resolution large-volume three-dimensional microscopic imaging device and its imaging method according to the embodiments of the present invention, the working principle of Fourier light field microscopy will be introduced first.

[0029] The Fourier light field microscope needs to record images at multiple viewpoints (quantity: v) simultaneously for three-dimensional reconstruction. Therefore, for the object space with the same field of view, the Fourier light field microscope needs to record v times the information of a conventional wide-field microscope in a single exposure. If each viewpoint image contains x×y pixels (usually x≈y), the images of v viewpoints are distributed in different regions of the camera. Taking uniform distribution as an example, at least about pixels are required.

[0030] If the imaging speed is increased by restricting the number of rows in the camera imaging area, the imaging results of a large number of viewpoints will be lost, thereby reducing the reconstruction accuracy and the size of the imaging field of view. Therefore, the multi-viewpoint images of the Fourier light field microscope usually need to use all the pixels of the camera for imaging and read out all the rows, and the three-dimensional imaging speed is limited by the camera readout speed. Even if large-volume and high-spatial-resolution three-dimensional microscopy imaging is achieved through the sub-field scanning imaging method, high temporal resolution still cannot be obtained.

[0031] It can be seen that the current Fourier light field microscope is still difficult to achieve large-volume, high-spatiotemporal-resolution three-dimensional microscopy imaging.

[0032] To solve the above problems, the embodiments of the present application provide a large-volume three-dimensional microscopy imaging device and method with high spatiotemporal resolution. Aiming at the problem that the related technologies mentioned in the above background technology are difficult to record all the information within the large field of view supported by the objective lens at a high sampling rate, and thus it is difficult to achieve large-volume and high-spatial-resolution microscopy imaging, and in a conventional Fourier light field microscope, reducing the number of rows used by the camera for imaging will result in the loss of imaging results of certain viewpoints, making it difficult to meet the reconstruction requirements, thereby losing spatial resolution or the imaging field of view, the present application provides a large-volume three-dimensional microscopy imaging device with high spatiotemporal resolution. In this device, an excitation light is emitted by a laser generation module, and the excitation light is filtered and focused. The filtered and focused excitation light is expanded and shaped into a strip-shaped excitation light, and the above excitation light is deflected by a one-dimensional scanning galvanometer of a scanning module. The deflected excitation light is focused by an objective lens on a target strip-shaped sub-field, exciting the sample in the target strip-shaped sub-field, and the fluorescence signal emitted by the sample is collected by the objective lens and introduced into an imaging system. The spatial spectrum information of the fluorescence signal is optically spatially multiplexed and imaged by a microlens array, and the imaging result of the optical spatial multiplexing is recorded by a camera. By imaging the strip-shaped sub-field, the requirement for the number of rows of the camera is reduced, and at the same time, through the expansion in the column direction of the camera, simultaneous imaging of a large volume is maintained during a single exposure. Thereby, the problem that the related technologies are difficult to achieve large-volume, high-spatiotemporal-resolution three-dimensional microscopy imaging is solved.

[0033] Specifically, Figure 1 is a schematic structural diagram of a large-volume three-dimensional microscopy imaging device with high spatiotemporal resolution provided by the embodiments of the present application.

[0034] As Figure 1 shown, the large-volume three-dimensional microscopic imaging device with high spatio-temporal resolution includes: an excitation light generation module 100, a scanning module 200, an imaging module 300, and a reconstruction module 400.

[0035] Specifically, the excitation light generation module 100 is used to generate excitation light for bar illumination.

[0036] It can be understood that the excitation light generation module 100 in the embodiments of the present application corresponds to the Figure 2 illumination module in

[0037] In the actual execution process, the excitation light generation module 100 in the embodiments of the present application is used to generate excitation light for bar illumination, so as to improve the imaging speed, and further realize large-volume and high-spatial-resolution imaging.

[0038] Optionally, in an embodiment of the present application, the laser generation module 100 includes: a laser light source for emitting excitation light; an excitation-end filter for filtering the excitation light to generate filtered excitation light; a first lens and a second lens for expanding the filtered excitation light to generate expanded excitation light; a first aperture for restricting the illumination range of the expanded excitation light to generate excitation light for bar illumination; and a third lens for performing optical Fourier transform on the excitation light for bar illumination to generate target excitation light.

[0039] It can be understood that, as Figure 2 shown, the laser generation module 100 (illumination module) in the embodiments of the present application includes a laser light source 1, an excitation-end filter 2, a first lens 3, a second lens 4, a first aperture 5, and a third lens 6 arranged in sequence.

[0040] In the actual execution process, the laser light source 1 in the embodiments of the present application is used to emit excitation light, the excitation-end filter 2 is used to filter the excitation light to generate filtered excitation light, the first lens 3 and the second lens 4 are used to expand the filtered excitation light to generate expanded excitation light, the first aperture 5 is used to restrict the illumination range of the expanded excitation light to generate excitation light for bar illumination, and the third lens 6 is used to perform optical Fourier transform on the excitation light for bar illumination to generate target excitation light.

[0041] Among them, the shape of the first aperture 5 in the embodiments of the present application is bar-shaped, and its aspect ratio is the same as that of the target bar-shaped sub-field of view.

[0042] The scanning module 200 is used to deflect the excitation light to a bar-shaped sub-field of view, excite the sample in the bar-shaped sub-field of view to determine the fluorescence signal emitted by the sample, and perform de-scanning on the fluorescence signal to generate a de-scanned fluorescence signal.

[0043] Among them, the scanning module 200 in the embodiments of the present application is used to deflect the excitation light to a strip-shaped sub-field of view, and excite the sample in the strip-shaped sub-field of view to determine the fluorescence signal emitted by the sample, and perform de-scanning on the fluorescence signal to generate a de-scanned fluorescence signal. By introducing the scanning module 200 in the embodiments of the present application, the imaging area can be flexibly adjusted according to actual needs, and full-field scanning imaging can be realized, or scanning imaging between any fields of view can be realized, avoiding data redundancy and further accelerating the imaging speed.

[0044] Optionally, in an embodiment of the present application, the scanning module 200 includes: a dichroic mirror for reflecting the excitation light to generate the reflected excitation light; a one-dimensional scanning galvanometer for deflecting the reflected excitation light at a preset angle to generate the deflected excitation light; a fourth lens and a fifth lens for relaying the deflected excitation light at a preset magnification to generate the relayed excitation light; and an objective lens for collecting the fluorescence signal of the sample and passing the fluorescence signal reversely through the fifth lens, the fourth lens, and the one-dimensional scanning galvanometer in sequence to generate a de-scanned fluorescence signal.

[0045] It can be understood that, as Figure 2 shown, the scanning module 200 in the embodiments of the present application includes a dichroic mirror 7, a one-dimensional scanning galvanometer 8, a fourth lens 9, a fifth lens 10, and an objective lens 11 arranged in sequence; the fluorescence signal in the embodiments of the present application can be a green fluorescence signal.

[0046] In the actual execution process, the dichroic mirror 7 in the embodiments of the present application is used to reflect the above excitation light to generate the reflected excitation light, the one-dimensional scanning galvanometer 8 is used to deflect the reflected excitation light at a specific angle to generate the deflected excitation light, the fourth lens 9 and the fifth lens 10 are used to relay the deflected excitation light at a preset magnification to generate the relayed excitation light, the objective lens 11 is used to collect the green fluorescence signal emitted by the sample 12, and pass the green fluorescence signal reversely through the fifth lens, the fourth lens, and the one-dimensional scanning galvanometer in sequence to generate a de-scanned fluorescence signal. The de-scanned green fluorescence signal enters the imaging system after passing through the dichroic mirror 7.

[0047] Among them, the first voltage signal is the voltage signal corresponding to the target strip-shaped sub-field of view generated by the digital board.

[0048] Specifically, the dichroic mirror 7 can be a long-pass dichroic mirror, and reflect the excitation light relayed by the laser generation module 100 (illumination module) to the one-dimensional scanning galvanometer 8. After receiving the voltage signal given by the digital board, the one-dimensional scanning galvanometer 8 generates a mechanical deflection angle and deflects the propagation direction of the above excitation light at an angle .

[0049] The fourth lens 9 and the fifth lens 10 form a group of 4f relay systems, which relay the excitation light on the central plane of the one-dimensional scanning galvanometer 8 to the rear pupil plane of the objective lens 11. The objective lens 11 focuses the bar-shaped excitation light with the deflected propagation direction on the target bar-shaped sub-field located at the rear focal plane of the objective lens to excite the sample in the bar-shaped sub-field. Let the focal length of the fourth lens 9 be |f9|, the focal length of the fifth lens 10 be |f 10 |, and the focal length of the objective lens 11 be |f 11 |. Then the distance between the center position of the bar-shaped sub-field and the optical axis is

[0050] In the embodiment of the present application, the two-dimensional scanning galvanometer is converted into a one-dimensional scanning galvanometer 8, which avoids the problem of spectral plane drift caused by non-conjugation between galvanometers. By using a single-axis galvanometer to scan between bar-shaped sub-fields, the problem that it is difficult for multi-axis galvanometers to be optically conjugated to each other is avoided, so that the relative position between the spatial spectral plane of the imaging light and the following microlens array 17 is kept stable, ensuring the consistency of the point spread function in the entire field of view.

[0051] Specifically, in the two-dimensional scanning galvanometer, the mechanical deflection angle of the two-dimensional scanning galvanometer is The distance between the centers of the two galvanometers constituting the two-dimensional scanning galvanometer is d galvo , then the spectral plane will generate drift. However, the single-axis galvanometer adopted in the embodiment of the present application will not generate drift at any mechanical deflection angle.

[0052] It should be noted that the preset angle and the preset magnification can be set by those skilled in the art according to the actual situation, and no specific limitation is made here.

[0053] The imaging module 300 is used to perform optical spatial multiplexing imaging on the spatial spectral information of the fluorescence signal after de-scanning to generate a multi-view imaging result corresponding to the bar-shaped sub-field, and traverse all bar-shaped sub-fields according to the corresponding multi-view imaging result until the multi-view imaging results of all bar-shaped sub-fields are obtained, and the multi-view imaging results are segmented according to different imaging views to generate the imaging results of all bar-shaped sub-fields at different imaging views.

[0054] Among them, the imaging module 300 in the embodiments of the present application is used to perform optical spatial multiplexing imaging on the spatial spectral information of the fluorescence signal after backward scanning, so as to generate multi-view imaging results corresponding to the strip sub-fields of view, and sequentially traverse all the strip sub-fields of view according to the scanning order encoded by the corresponding multi-view imaging results until the multi-view imaging results of all the strip sub-fields of view (the number is n, the same as the number of sub-fields of view) are obtained, and the multi-view imaging results are segmented according to different imaging perspectives to obtain the imaging results of all the strip sub-fields of view under different imaging perspectives (the number is n×v), which helps to more accurately restore the true three-dimensional structure of the sample in the subsequent computational reconstruction process, improve the reconstruction quality of the three-dimensional image, and further provide support for constructing a large field of view and high-resolution three-dimensional microscopic image.

[0055] Optionally, in an embodiment of the present application, the imaging module 300 includes: a sixth lens for focusing the fluorescence signal to determine the imaging on the rear focal plane of the sixth lens; a second aperture located on the rear focal plane of the sixth lens for restricting the field of view of the imaging on the rear focal plane of the sixth lens within a target range; a seventh lens for performing an optical Fourier transform on the imaging on the rear focal plane of the sixth lens within the target range to generate the spatial spectral information after the optical Fourier transform; a collection end filter located on the rear focal plane of the seventh lens for filtering out the residual excitation light in the imaging light; a microlens array composed of a plurality of microlenses and located on the rear focal plane of the seventh lens for performing spatial multiplexing imaging on the spatial spectral information after the optical Fourier transform to generate multi-view imaging results; and a camera located on the rear focal plane of the microlens array for recording the multi-view imaging results.

[0056] It can be understood that, as Figure 2 shown, the imaging module 300 in the embodiments of the present application includes a sixth lens 13, a second aperture 14, a seventh lens 15, a collection end filter 16, a microlens array 17, and a camera 18 arranged in sequence.

[0057] In the actual implementation process, the sixth lens 13 in the embodiment of the present application is used to focus the green fluorescence signal to determine the imaging on the rear focal plane of the sixth lens 13. The second aperture 14 is located on the rear focal plane of the sixth lens 13 and is used to limit the field of view of the above imaging. The seventh lens 15 is used to perform an optical Fourier transform on the imaging on the rear focal plane of the sixth lens 13 within the target range to generate spatial spectrum information after the optical Fourier transform. The collection end filter 16 is located on the rear focal plane of the seventh lens 15 and is used to filter out the residual excitation light in the imaging light. The microlens array 17 is composed of multiple microlenses, and the microlens array 17 is located on the rear focal plane of the seventh lens 15 and is used to perform spatial multiplexing imaging on the spatial spectrum information after the optical Fourier transform to generate a multi-view imaging result. The camera 18 includes M×N pixels, and the size of each pixel is d pixel , and the camera 18 is located on the rear focal plane of the microlens array 17 and is used to record the multi-view imaging result composed of the microlens array 17.

[0058] Among them, the aspect ratio of the second aperture 14 is the same as the aspect ratio of the first aperture 5 in the laser generation module 100 (illumination module).

[0059] Further, the microlens array 17 is composed of multiple (v≥2) microlenses, and the microlenses are arranged in a row along the column direction of the camera.

[0060] Further, the number of perspectives (v≥2) of the multi-view images recorded by the camera is the same as the number of microlenses (v≥2) in the microlens array 17.

[0061] Further, during a single imaging of the camera 18, multi-view imaging is only performed on a specific strip sub-field of view. The camera 18 has sufficient pixels to provide a high sampling rate for the samples within the strip sub-field of view, and thus a high spatial resolution can be obtained.

[0062] Further, during the imaging process of the camera, only the pixels (the number of pixels m = v×d ML / d pixel , where d ML is the size of the microlens, and m < M) within the range covered by the microlens array 17 are exposed for imaging in the row direction of the camera 18, and all the pixels (the number of pixels = N) in the column direction of the camera 18 are exposed for imaging. The image size obtained by a single exposure of the camera 18 is m×N, and the readout time of this image is m / M times the readout time of the full-frame image of the camera, and thus a high time resolution can be obtained.

[0063] Such as Figure 3As shown in the figure, in the embodiments of the present application, the design of the microlens array is optimized to reduce the system's requirement for the number of rows, thereby improving the time resolution. Among them, the microlens array 17 is composed of three microlenses arranged along the column direction of the camera 18, and images of a certain bar-shaped sub-field of view at three different sub-angles of view are collected on the camera 18, having a high spatial resolution. The images of the sub-angles of view collected use all N pixels in the column direction of the camera 18 for exposure, but only a small number of m pixels are used for exposure in the row direction of the camera 18. At this time, the image readout time of the camera 18 is reduced to m / M times of the full-frame image readout time of the camera 18, and thus a high time resolution can be obtained. It should be noted that although the present application reduces the requirement for the number of rows of the camera, through the expansion in the column direction, a relatively large imaging volume can still be recorded during single imaging.

[0064] In the embodiments of the present application, by optimizing the design of the microlens array 17, imaging of a bar-shaped sub-field of view is realized. While ensuring the imaging field of view, the requirement of the Fourier light field microscope for the number of rows in the camera target surface is reduced, and the image readout time is reduced. The embodiments of the present application optimize the design of the microlens and the utilization of the camera, accelerate the imaging speed, improve the time resolution, and provide support for realizing fast three-dimensional microscopic imaging with large volume, high temporal and spatial resolution.

[0065] The reconstruction module 400 is used to reconstruct the imaging results of all bar-shaped sub-fields of view at different imaging angles of view to determine the three-dimensional reconstruction images of all bar-shaped sub-fields of view, and splice the three-dimensional reconstruction images of all bar-shaped sub-fields of view to generate a three-dimensional microscopic image of the target large field of view.

[0066] Among them, in the embodiments of the present application, based on the pre-collected three-dimensional point spread function and the deconvolution reconstruction algorithm, the reconstruction module 400 reconstructs the imaging results of all bar-shaped sub-fields of view at different imaging angles of view to obtain the three-dimensional reconstruction images of all bar-shaped sub-fields of view, and splices the three-dimensional reconstruction images of all bar-shaped sub-fields of view to obtain a three-dimensional microscopic image of the target large field of view, and thus a three-dimensional imaging result with a large volume can be obtained.

[0067] As Figure 4 shown, the entire field of view supported by the objective lens 11 can be divided into n bar-shaped sub-fields of view. In single-exposure imaging, multi-angle imaging can be performed on the sample of any bar-shaped sub-field of view, and the three-dimensional reconstruction image of the bar-shaped sub-field of view can be obtained through computational reconstruction. Subsequently, through the one-dimensional galvanometer 8, sequential scanning is performed based on the analog voltage signal corresponding to the position of each bar-shaped sub-field of view. By repeating the above steps, the three-dimensional reconstruction images of all bar-shaped sub-fields of view are obtained, and through three-dimensional splicing, the three-dimensional imaging result of the entire field of view supported by the objective lens 11 is obtained, and thus a three-dimensional imaging result with a large volume can be obtained.

[0068] Further, the target observation area can be determined, and the target observation area is divided into i strip-shaped sub-fields of view. The one-dimensional scanning galvanometer 8 of the scanning module 200 is used to randomly scan and image the i strip-shaped sub-fields of view within the target area until multi-view imaging results of all the strip-shaped sub-fields of view within the target area are obtained. Then, the multi-view imaging results of all the strip-shaped sub-fields of view are segmented according to different imaging perspectives to obtain the imaging results of all the strip-shaped sub-fields of view at different imaging perspectives. By using the pre-acquired three-dimensional point spread function and the deconvolution reconstruction algorithm, the imaging results of all the strip-shaped sub-fields of view at different imaging perspectives are reconstructed to obtain the three-dimensional reconstruction images of all the strip-shaped sub-fields of view, and the three-dimensional reconstruction images of all the strip-shaped sub-fields of view are stitched together to obtain the three-dimensional microscopic image of the target observation area.

[0069] As Figure 5 shown, the target observation area can be determined within the entire field of view supported by the objective lens, and the target observation area is divided into i strip-shaped sub-fields of view. In single-exposure imaging, multi-view imaging can be performed on the samples of the strip-shaped sub-fields of view of any target observation area, and the three-dimensional reconstruction image of the strip-shaped sub-field of view can be obtained through computational reconstruction. Subsequently, the one-dimensional galvanometer 8 performs random scanning based on the analog voltage signal corresponding to the position of each strip-shaped sub-field of view, and the above steps are repeated to obtain the three-dimensional reconstruction images of all the strip-shaped sub-fields of view within the target observation area, and the three-dimensional imaging result of the target area is obtained through three-dimensional stitching.

[0070] Further, if the target observation area determined within the entire field of view supported by the objective lens is discontinuous, it can be divided into multiple discrete strip-shaped sub-fields of view, and random scanning imaging is performed on the above-mentioned multiple discrete strip-shaped sub-fields of view until multi-view imaging results of all the discrete strip-shaped sub-fields of view within the target area are obtained. Then, the multi-view imaging results of all the discrete strip-shaped sub-fields of view are segmented according to different imaging perspectives to obtain the imaging results of all the discrete strip-shaped sub-fields of view at different imaging perspectives. By using the pre-acquired three-dimensional point spread function and the deconvolution reconstruction algorithm, the imaging results of all the discrete strip-shaped sub-fields of view at different imaging perspectives are reconstructed to obtain the three-dimensional reconstruction images of all the discrete strip-shaped sub-fields of view. According to actual requirements, the three-dimensional reconstruction images of each strip-shaped sub-field of view are stitched or not stitched, and the three-dimensional reconstruction image of the target observation area can be obtained.

[0071] Among them, Figure 6 is a schematic flowchart of the method for three-dimensional microscopic imaging with large volume and high spatio-temporal resolution according to an embodiment of the present invention. As Figure 6 shown, the method for three-dimensional microscopic imaging with large volume and high spatio-temporal resolution includes the following steps:

[0072] Step S601: Based on a preset division strategy, divide the target large field of view into multiple strip-shaped sub-fields of view, determine the position and scanning imaging order of each strip-shaped sub-field of view, calculate the analog voltage signal corresponding to the position of each strip-shaped sub-field of view, and pre-collect the three-dimensional point spread function of each strip-shaped sub-field of view along the direction perpendicular to the strip-shaped sub-field of view.

[0073] Among them, the preset division strategy is to determine the magnification of the system according to the size of the field of view supported by the objective lens 11 in the camera row direction. If the size of the camera 18 in the row direction is L and the size of the field of view supported by the objective lens 11 is l, then the system magnification M1 = L / l. The size of a single strip-shaped sub-field of view in the row direction is l, and the size in the column direction is d ML / M1, where d ML is the diameter of a single microlens.

[0074] Specifically, in the embodiment of the present application, the input voltage of the one-dimensional scanning galvanometer 8 control component can be modulated to determine the positions of each strip-shaped sub-field of view (the number is n) scanned and imaged in the large field of view. Subsequently, the scanning order of each sub-field of view is encoded. For example, sub-field 1, sub-field 2... sub-field n. Within the field of view supported by the objective lens 11, by deflecting the excitation light in different angles in the direction of the excitation light through the one-dimensional scanning galvanometer 8, the excitation light is focused on the determined different strip-shaped sub-fields of view at the front focal plane of the objective lens. Within each determined strip-shaped sub-field of view, move the fluorescent beads along the direction perpendicular to the field of view with a preset step size (for example, 1 μm). After each movement, collect an image at this depth until the entire depth of field range (for example, 200 μm) is traversed, and complete the acquisition of the three-dimensional point spread function of each view angle in this sub-field of view.

[0075] It should be noted that before passing through the one-dimensional scanning galvanometer 8, the center of the excitation light propagates along the optical axis direction of the system, that is, the central propagation angle is 0. After being deflected by the scanning galvanometer, the central propagation angle of the excitation light is twice the mechanical rotation angle of the scanning galvanometer , that is where the mechanical rotation angle generated by the one-dimensional scanning galvanometer 8 is proportional to the value of the voltage applied to the one-dimensional scanning galvanometer 8.

[0076] The equivalent focal length of the optical system that the excitation light experiences before reaching the strip-shaped sub-field of view is f o , and the vector distance between the central position of the strip-shaped sub-field of view where the excitation light with the central propagation angle is focused behind the objective lens 11 and the central position of the field of view of the objective lens 11 is

[0077] The positions and quantities of the pre-coded strip sub-fields in the large field of view can be adjusted according to needs, and can include full field of view scanning or scanning between any fields of view. Full field of view scanning means traversing and imaging all the fields of view supported by the scanning objective lens 11 along a specific scanning trajectory, and scanning between any fields of view means arbitrarily selecting one or more strip sub-fields within the field of view supported by the objective lens 11 for scanning and imaging.

[0078] When collecting the three-dimensional point spread functions of each viewing angle in each strip sub-field, the size of the selected fluorescent microspheres should be smaller than the lateral resolution R of the system x , and the preset step size for moving along the direction perpendicular to the field of view should be smaller than the axial resolution R of the system z .

[0079] Step S602: Generate a voltage signal corresponding to any strip sub-field, and apply the voltage signal to the one-dimensional scanning galvanometer, so that the one-dimensional scanning galvanometer deflects the excitation light to the corresponding strip sub-field and excites the sample within the corresponding strip sub-field.

[0080] Specifically, while generating the excitation light, the control component of the one-dimensional scanning galvanometer 8 receives a voltage signal that enables it to scan to any strip sub-field and generates a deflection at a corresponding angle. After the one-dimensional scanning galvanometer 8 deflects the excitation light, it is projected onto the strip sub-field through the objective lens 11 to excite the sample within the strip sub-field, and at the same time, a trigger signal is given to the camera 18.

[0081] It should be noted that the mechanical deflection of the one-dimensional scanning galvanometer 8 and the exposure of the camera 18 are synchronized through analog / digital signals. After the one-dimensional scanning galvanometer 8 undergoes a mechanical deflection, the camera 18 simultaneously receives a trigger signal to start exposure, and the exposure time of the camera 18 is less than the time interval between adjacent trigger signals. Before the one-dimensional scanning galvanometer 8 undergoes the next mechanical deflection, the camera 18 completes this exposure and data transmission.

[0082] Step S603: Collect the fluorescence emitted by the sample through the objective lens, and relay the spatial frequency spectrum information of the fluorescence to the microlens array at a preset magnification. The microlens array performs optical spatial multiplexing imaging on the spatial frequency spectrum information to obtain a multi-view imaging result of the corresponding strip sub-field, and records the multi-view imaging result through the camera.

[0083] Specifically, the number of perspectives (v≥2) of the multi-perspective images recorded by the camera 18 is the same as the number of microlenses in the microlens array 17 (v≥2). The objective lens 11 collects the fluorescence emitted by the samples in the sub-field of view in step S602. This fluorescence passes backward through the one-dimensional scanning galvanometer 8 to cancel the deflection angle generated by the scanning and propagates along the optical axis of the system. Subsequently, the spatial frequency spectrum information of this fluorescence is relayed to the microlens array 17 at a specific magnification. The microlens array 17 performs optical spatial multiplexing imaging on the above spatial frequency spectrum information. The camera placed at the back focal plane of the microlens array 17 starts to expose after receiving the trigger signal in step S602, records the multi-perspective (the number of perspectives is v) imaging result of this sub-field of view, stops exposing after reaching the set exposure time, and transmits the image to the computer.

[0084] Step S604: Traverse all strip-shaped sub-fields of view until the multi-perspective imaging results of all strip-shaped sub-fields of view are obtained, and segment the multi-perspective imaging results of all sub-fields of view according to different imaging perspectives to obtain the imaging results of all sub-fields of view at different imaging perspectives.

[0085] Specifically, the embodiments of the present application can sequentially traverse all sub-fields of view in the scanning order encoded in step S601, and repeat steps S602 to S603. After the traversal is completed, one cycle of acquisition is completed, and the images of each sub-field of view (the number is n, the same as the number of sub-fields of view) are obtained. Subsequently, the imaging results of each sub-field of view are segmented according to different imaging perspectives to obtain the imaging results of each perspective of each sub-field of view (the number is n×v).

[0086] Step S605: Calculate and reconstruct the imaging results of all sub-fields of view at different imaging perspectives and splice them to obtain a three-dimensional microscopic image of the target large field of view.

[0087] Optionally, in an embodiment of the present application, the reconstruction module 400 includes: a perspective segmentation unit, configured to calibrate the imaging position coordinates of each sub-perspective on the fluorescent beads at the imaging depth of field center depth based on the imaging result at the imaging depth of field center depth, and perform perspective segmentation on the original imaging result of the strip sub-field of view according to the imaging position coordinates to obtain the imaging results of the strip sub-field of view at different imaging perspectives; a pixel-by-pixel adaptive correction unit, configured to obtain an image calibration value that meets a preset low aberration condition based on the imaging results of the strip sub-field of view at different imaging perspectives, with the imaging result at the central perspective as a reference, and perform pixel-by-pixel alignment on the imaging results according to the image calibration value that meets the preset low aberration condition to generate a corrected multi-perspective imaging result; a three-dimensional reconstruction unit, configured to three-dimensionally reconstruct the data processed by perspective segmentation and adaptive correction by using the three-dimensional point spread function of the strip sub-field of view based on the corrected multi-perspective imaging result to determine a three-dimensional imaging result that meets the preset conditions; a three-dimensional stitching unit, configured to calculate the intensity ratio of the overlapping regions between adjacent strip sub-fields of view based on the three-dimensional imaging result that meets the preset conditions, linearly compensate the adjacent strip sub-fields of view according to the intensity ratio to generate a linear compensation result, and based on the linear compensation result, stitch the three-dimensional reconstruction images of all strip sub-fields of view into a three-dimensional microscopic image of the target field of view.

[0088] It can be understood that Figure 7 is a schematic diagram of a computational reconstruction method for high spatio-temporal resolution large-volume three-dimensional microscopic imaging according to an embodiment of the present invention; the image calibration value that meets the preset low aberration condition in the embodiment of the present application may be a low aberration image calibration value; the three-dimensional imaging result that meets the preset conditions in the embodiment of the present application may be the three-dimensional imaging result provided by the three-dimensional reconstruction unit.

[0089] In the actual execution process, the perspective segmentation unit in the embodiment of the present application includes a position coordinate calibration method and a perspective segmentation method. The position coordinate calibration method calibrates the imaging position coordinates (number v≥2) of each sub-perspective (number v≥2) on the fluorescent beads at this depth based on the imaging result at the imaging depth of field center depth. The perspective segmentation method performs perspective segmentation on the original imaging result (number 1) of each strip sub-field of view based on the above-calibrated position coordinates (number v≥2) to obtain the multi-perspective imaging results (number v≥2) of the strip sub-field of view.

[0090] Among them, when performing perspective segmentation on the original imaging result (number 1) of each strip sub-field of view, the number of pixels m in the row direction of each strip sub-field of view = d ML / d pixel , where d ML is the micro-lens size, d pixel is the pixel size), and the number of pixels n in the column direction = N, where N is the number of pixels of the camera in the column direction.

[0091] Further, the pixel-by-pixel adaptive correction unit includes a low aberration image calibration method and a pixel-by-pixel alignment method. The low aberration image calibration method first images a single-layer fluorescent bead at the central depth of the imaging depth of field, and then segments the imaging result through the above-mentioned perspective segmentation module. The spatial frequency of the imaging light collected by the central perspective is the lowest and can be used as the low aberration image for calibration. Subsequently, based on the imaging result under the central perspective, the coordinates (x0, y0) of each fluorescent bead in the field of view are calibrated, that is, the low aberration image calibration value is obtained. The pixel-by-pixel alignment method first finds the image corresponding to each fluorescent bead calibrated under the central perspective in the imaging results of other perspectives except the central perspective based on the obtained low aberration image calibration value, and calibrates its coordinates (x i , y i ), and calculates the position offset (Δx i = x i - x0, Δy i = y i - y0). Subsequently, surface fitting is performed through the coordinates of each fluorescent bead and its position offset, and interpolation correction is performed for each different coordinate, repeating until the imaging results of all perspectives are traversed.

[0092] Among them, the interval between each fluorescent bead in the single-layer fluorescent bead sample should be higher than the position offset (Δx i , Δy i ) to avoid confusion of the imaging results of fluorescent beads between different perspectives during the calibration process.

[0093] Further, the 3D reconstruction unit includes a multi-perspective iterative deconvolution method, which is used to 3D reconstruct the data processed by perspective segmentation and adaptive correction based on the corrected multi-perspective imaging results using the 3D point spread function of the bar-shaped sub-field of view to determine the 3D imaging result that meets the preset conditions, that is, after the original imaging data is processed by the above-mentioned perspective segmentation unit and the pixel-by-pixel adaptive correction unit, deconvolution 3D reconstruction is performed through the 3D point spread function of each bar-shaped sub-field of view pre-acquired in the above step S601, repeating until all bar-shaped sub-fields of view are traversed.

[0094] Among them, the mathematical form of the multi-perspective iterative deconvolution method is:

[0095]

[0096] Among them, Obj k (x, y, z) is the 3D imaging result under the k-th iteration, I i (x, y) is the 2D imaging result of perspective i, PSF i (x, y, z n) is the three-dimensional point spread function of each strip sub-field of view pre-acquired in the above step S601. is a convolution operation. The number of iterations k should be determined according to the signal-to-noise ratio of the imaging and the number of viewing angles v. When the image signal-to-noise ratio is low, the number of iterations k should be minimized to avoid reconstruction artifacts introduced by image noise. The fewer the number of image viewing angles v, the more the number of iterations k.

[0097] Furthermore, the three-dimensional stitching unit includes a three-dimensional stitching method. After obtaining the three-dimensional imaging result provided by the above three-dimensional reconstruction module, calculate the intensity ratio of the reconstruction results of adjacent strip sub-fields of view based on the overlapping area between adjacent strip sub-fields of view, and perform linear compensation. Subsequently, use the three-dimensional point spread function of this sub-field of view acquired in the above step S601 and the Fourier light field reconstruction algorithm based on the multi-view iterative deconvolution method to reconstruct the imaging results of each view (the number is v) into a three-dimensional image (the number is 1). After completing the three-dimensional reconstruction of all sub-fields of view (the number is n). Subsequently, stitch the three-dimensional reconstruction results of all obtained sub-fields of view according to their three-dimensional positions to obtain a large field of view and high-resolution three-dimensional microscopic image (the number is 1).

[0098] It should be noted that if the position and number of sub-fields of view are adjusted according to actual needs in the above step S601, such as scanning and imaging between any discontinuous sub-fields of view, then there is no need to stitch between the imaging sub-fields of view, and the stitching step can be skipped according to requirements.

[0099] The selection of the low-aberration reference image should refer to the imaging result under the viewing angle with the lowest spatial frequency of the imaging light collected in the imaging optical path. Other preset conditions can be set by those skilled in the art according to the actual situation, and no specific limitations are made here.

[0100] A large-volume three-dimensional microscopic imaging device with high spatio-temporal resolution proposed according to an embodiment of the present application realizes imaging of strip-shaped sub-fields of view by optimizing the design of the microlens array, reduces the requirement of the Fourier light field microscope for the number of rows in the camera target surface while ensuring the imaging field of view, reduces the readout time of the image, speeds up the imaging speed, and realizes fast three-dimensional microscopic imaging with large volume and high spatio-temporal resolution; by introducing a scanning module, the imaging area can be flexibly adjusted according to actual needs, full-field scanning imaging can be realized, and scanning imaging between any fields of view can also be realized, avoiding data redundancy and further speeding up the imaging speed; by using a single-axis galvanometer for scanning between strip-shaped sub-fields of view, the problem that it is difficult for multi-axis galvanometers to be optically conjugated to each other is avoided, so that the relative position between the spatial frequency spectrum plane of the imaging light and the microlens array remains stable, ensuring the consistency of the point spread function within the entire field of view. Thus, the problem in the related art that it is difficult to record all information within a large field of view supported by the objective lens at a high sampling rate, and thus it is difficult to realize large-volume and high-spatial-resolution microscopic imaging, and in a common Fourier light field microscope, reducing the number of rows used by the camera for imaging will result in the loss of imaging results from certain perspectives, making it difficult to meet the reconstruction requirements, and thus losing spatial resolution or the imaging field of view is solved.

[0101] Next, a large-volume three-dimensional microscopic imaging method with high spatio-temporal resolution proposed according to an embodiment of the present application is described with reference to the accompanying drawings.

[0102] Figure 8 It is a schematic flowchart of a large-volume three-dimensional microscopic imaging method with high spatio-temporal resolution according to an embodiment of the present application.

[0103] As Figure 8 shown, the large-volume three-dimensional microscopic imaging method with high spatio-temporal resolution includes the following steps:

[0104] In step S801, an excitation light with strip-shaped illumination is generated.

[0105] In step S802, the excitation light is deflected to a strip-shaped sub-field of view, and the sample within the strip-shaped sub-field of view is excited to determine the fluorescence signal emitted by the sample, and the fluorescence signal is de-scanned to generate a de-scanned fluorescence signal.

[0106] In step S803, the spatial frequency spectrum information of the de-scanned fluorescence signal is subjected to optical spatial multiplexing imaging to generate a multi-view imaging result corresponding to the strip-shaped sub-field of view, and all strip-shaped sub-fields of view are traversed according to the corresponding multi-view imaging result until the multi-view imaging results of all strip-shaped sub-fields of view are obtained, and the multi-view imaging results are segmented according to different imaging views to generate imaging results of all strip-shaped sub-fields of view at different imaging views.

[0107] In step S804, reconstruct the imaging results of all strip sub-fields of view at different imaging perspectives to determine the three-dimensional reconstruction images of all strip sub-fields of view, and stitch the three-dimensional reconstruction images of all strip sub-fields of view to generate the three-dimensional microscopic image of the target large field of view.

[0108] Optionally, in an embodiment of the present application, generating the excitation light for strip illumination includes: emitting the excitation light; performing filtering processing on the excitation light to generate the filtered excitation light; expanding the filtered excitation light to generate the expanded excitation light; restricting the illumination range of the expanded excitation light to generate the excitation light for strip illumination; performing an optical Fourier transform on the excitation light for strip illumination to generate the target excitation light.

[0109] Optionally, in an embodiment of the present application, deflecting the excitation light to the strip sub-field of view and exciting the sample in the strip sub-field of view to determine the fluorescence signal emitted by the sample, and performing de-scanning on the fluorescence signal to generate the de-scanned fluorescence signal includes: reflecting the excitation light to generate the reflected excitation light; deflecting the reflected excitation light at a preset angle to generate the deflected excitation light; relaying the deflected excitation light at a preset magnification to generate the relayed excitation light; collecting the fluorescence signal of the sample and generating the de-scanned fluorescence signal.

[0110] Optionally, in an embodiment of the present application, generating the imaging results of all strip sub-fields of view at different imaging perspectives includes: focusing the fluorescence signal; generating the spatial spectrum information after the optical Fourier transform; filtering out the residual excitation light in the imaging light; performing spatial multiplexing imaging on the spatial spectrum information after the optical Fourier transform to generate the multi-perspective imaging results; recording the multi-perspective imaging results.

[0111] Optionally, in an embodiment of the present application, the imaging results of all strip sub-fields of view at different imaging perspectives are reconstructed to determine the three-dimensional reconstruction images of all strip sub-fields of view, and the three-dimensional reconstruction images of all strip sub-fields of view are stitched together to generate a three-dimensional microscopic image of the target large field of view, including: based on the imaging results at the depth of the center of the imaging depth of field, calibrating the imaging position coordinates of the fluorescence microspheres at each sub-perspective at the depth of the center of the imaging depth of field, and performing perspective segmentation on the original imaging results of the strip sub-field of view according to the imaging position coordinates to obtain the imaging results of the strip sub-field of view at different imaging perspectives; based on the imaging results of the strip sub-field of view at different imaging perspectives, taking the imaging results at the central perspective as a reference, obtaining an image calibration value that meets the preset low aberration condition, and performing pixel-by-pixel alignment on the imaging results according to the image calibration value that meets the preset low aberration condition to generate a corrected multi-perspective imaging result; based on the corrected multi-perspective imaging result, using the three-dimensional point spread function of the strip sub-field of view to three-dimensionally reconstruct the data that has undergone perspective segmentation and adaptive correction processing to determine a three-dimensional imaging result that meets the preset conditions; based on the three-dimensional imaging result that meets the preset conditions, calculating the intensity ratio of the overlapping regions between adjacent strip sub-fields of view, linearly compensating the adjacent strip sub-fields of view according to the intensity ratio to generate a linear compensation result, and based on the linear compensation result, stitching the three-dimensional reconstruction images of all strip sub-fields of view into a three-dimensional microscopic image of the target field of view.

[0112] It should be noted that the foregoing explanation of the embodiments of a high spatio-temporal resolution large-volume three-dimensional microscopic imaging device also applies to a high spatio-temporal resolution large-volume three-dimensional microscopic imaging method of this embodiment, and will not be elaborated here.

[0113] A high spatio-temporal resolution large-volume three-dimensional microscopic imaging method proposed according to an embodiment of the present application realizes the imaging of strip sub-fields by optimizing the design of the microlens array, reduces the requirement of the Fourier light field microscope for the number of rows in the camera target surface while ensuring the imaging field of view, reduces the readout time of the image, speeds up the imaging speed, and realizes fast three-dimensional microscopic imaging with large volume and high spatio-temporal resolution; by introducing a scanning module, the imaging area can be flexibly adjusted according to actual needs, full-field scanning imaging can be realized, and scanning imaging between any fields of view can also be realized, avoiding data redundancy and further speeding up the imaging speed; by using a single-axis galvanometer for scanning between strip sub-fields, the problem that it is difficult for multi-axis galvanometers to be optically conjugated to each other is avoided, so that the relative position between the spatial frequency spectrum plane of the imaging light and the microlens array remains stable, ensuring the consistency of the point spread function within the entire field of view. Thus, the problem in the related art that it is difficult to record all the information within the large field of view supported by the objective lens at a high sampling rate, and thus it is difficult to realize large-volume and high-spatial-resolution microscopic imaging, and in a common Fourier light field microscope, reducing the number of rows used by the camera for imaging will result in the loss of imaging results from certain perspectives, making it difficult to meet the reconstruction requirements, and thus losing spatial resolution or the imaging field of view is solved.

[0114] In the description of this specification, the description referring to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or N embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0115] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0116] Any process or method description represented in a flowchart or otherwise described herein can be understood to represent a module, segment, or portion of code including one or N executable instructions for implementing a customized logical function or process. The scope of the preferred embodiments of the present application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in the reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the art to which the embodiments of the present application pertain.

[0117] The logic and / or steps represented in a flowchart or otherwise described herein, for example, can be considered a sequenced list of executable instructions for implementing a logical function, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in conjunction with these instruction execution systems, apparatuses, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection portion with one or N wirings (electronic device), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically by optically scanning the paper or other media, followed by editing, interpretation, or otherwise processing as appropriate, and then stored in a computer memory.

[0118] It should be understood that the various parts of the present application can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the N steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0119] Those of ordinary skill in the art can understand that all or part of the steps carried out in the methods of the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0120] In addition, in each of the embodiments of the present application, the functional units can be integrated into a processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. When the above integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0121] The above-mentioned storage medium can be a read-only memory, a magnetic disk, an optical disc, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A large volume three-dimensional microscopic imaging device with high temporal and spatial resolution, characterized in that: include: An excitation light generating module, used for generating excitation light for strip lighting; A scanning module, used for deflecting the excitation light to the strip sub-field of view, and exciting the sample in the strip sub-field of view to determine the fluorescence signal emitted by the sample, and descanning the fluorescence signal to generate a descanned fluorescence signal; An imaging module, used for performing optical spatial multiplexing imaging on the spatial spectrum information of the descanned fluorescence signal to generate a multi-viewing angle imaging result corresponding to the strip sub-field of view, and traversing all the strip sub-fields of view according to the corresponding multi-viewing angle imaging result until the multi-viewing angle imaging results of all the strip sub-fields of view are obtained, and dividing the multi-viewing angle imaging results according to different imaging viewing angles to generate imaging results of all the strip sub-fields of view at different imaging viewing angles; A reconstruction module is used to reconstruct the imaging results of all the strip sub-fields of view at the different imaging angles to determine the three-dimensional reconstructed images of all the strip sub-fields of view, and to splice the three-dimensional reconstructed images of all the strip sub-fields of view to generate a three-dimensional microscopic image of the target large field of view.

2. A high temporal and spatial resolution large volume three-dimensional microscopic imaging device according to claim 1, characterized in that: The excitation light generating module comprises: A laser light source, used to emit the excitation light; An excitation end filter, used for filtering the excitation light to generate filtered excitation light; A first lens and a second lens are used to expand the filtered excitation light to generate expanded excitation light; A first aperture, used to limit the illumination range of the expanded excitation light to generate the excitation light for strip illumination; The third lens is used to perform optical Fourier transform on the excitation light of the strip illumination to generate target excitation light.

3. The high temporal and spatial resolution large volume three-dimensional microscopic imaging device according to claim 1, characterized in that: The scanning module comprises: a dichroic mirror, used to reflect the excitation light to generate reflected excitation light; A one-dimensional scanning galvanometer, used for deflecting the reflected excitation light at a preset angle to generate deflected excitation light; a fourth lens and a fifth lens, used to relay the deflected excitation light at a preset magnification to generate relayed excitation light; The objective lens is used to collect the fluorescence signal of the sample, and sequentially reversely pass the fluorescence signal through the fifth lens, the fourth lens and the one-dimensional scanning galvanometer to generate the descanned fluorescence signal.

4. A high temporal and spatial resolution large volume three-dimensional microscopic imaging device according to claim 3, characterized in that: The imaging module comprises: A sixth lens, used for focusing the fluorescent signal to determine the back focal plane imaging of the sixth lens; A second aperture, the second aperture being located on the back focal plane of the sixth lens and being used to limit the field of view of the back focal plane imaging of the sixth lens within a target range; a seventh lens, used for performing optical Fourier transform on the back focal plane imaging of the sixth lens within the target range to generate spatial spectrum information after optical Fourier transform; A collection end filter, the collection end filter is located on the rear focal plane of the seventh lens and is used to filter out residual excitation light in the imaging light; A microlens array, the microlens array is composed of a plurality of microlenses, and the microlens array is located on the back focal plane of the seventh lens, and is used to perform spatial multiplexing imaging on the spatial spectrum information after the optical Fourier transform, so as to generate the multi-view imaging result; A camera is located on the back focal plane of the microlens array and is used to record the multi-view imaging results.

5. The high temporal and spatial resolution large volume three-dimensional microscopic imaging device according to claim 1, characterized in that: The reconstruction module comprises: A viewing angle segmentation unit is used to calibrate the imaging position coordinates of the fluorescent spheres at the central depth of the imaging depth of field at each sub-viewing angle based on the imaging result at the central depth of the imaging depth of field, and perform viewing angle segmentation on the original imaging result of the strip sub-field of view according to the imaging position coordinates, so as to obtain the imaging results of the strip sub-field of view at the different imaging viewing angles; A pixel-by-pixel adaptive correction unit, configured to obtain an image calibration value satisfying a preset low-aberration condition based on the imaging results of the strip sub-field of view at the different imaging viewing angles and taking the imaging result at the central viewing angle as a reference, and to perform pixel-by-pixel alignment on the imaging results according to the image calibration value satisfying the preset low-aberration condition, so as to generate a corrected multi-view imaging result; A three-dimensional reconstruction unit, for three-dimensionally reconstructing the data processed by the perspective segmentation and adaptive correction by using the three-dimensional point spread function of the strip sub-field of view based on the corrected multi-view imaging result, so as to determine a three-dimensional imaging result that meets a preset condition; A three-dimensional stitching unit is used to calculate the intensity ratio of the overlapping area between adjacent strip sub-fields of view based on the three-dimensional imaging result that meets the preset conditions, and linearly compensate the adjacent strip sub-fields of view according to the intensity ratio to generate a linear compensation result, and based on the linear compensation result, stitch the three-dimensional reconstructed images of all the strip sub-fields of view into a three-dimensional microscopic image of the target field of view.

6. A method for large volume three-dimensional microscopic imaging with high temporal and spatial resolution, characterized in that: A method for large volume three-dimensional microscopic imaging with high temporal and spatial resolution as claimed in any one of claims 1 to 5 is adopted, comprising the following steps: Producing excitation light for strip lighting; Deflecting the excitation light to a strip-shaped sub-field of view, and exciting a sample in the strip-shaped sub-field of view to determine a fluorescence signal emitted by the sample, and descanning the fluorescence signal to generate a descanned fluorescence signal; Performing optical spatial multiplexing imaging on the spatial spectrum information of the descanned fluorescence signal to generate a multi-viewing angle imaging result corresponding to the strip sub-field of view, and traversing all the strip sub-fields of view according to the corresponding multi-viewing angle imaging result until the multi-viewing angle imaging results of all the strip sub-fields of view are obtained, and dividing the multi-viewing angle imaging results according to different imaging viewing angles to generate imaging results of all the strip sub-fields of view at different imaging viewing angles; The imaging results of all the strip sub-fields of view at the different imaging viewing angles are reconstructed to determine the three-dimensional reconstructed images of all the strip sub-fields of view, and the three-dimensional reconstructed images of all the strip sub-fields of view are spliced ​​to generate a three-dimensional microscopic image of the target large field of view.

7. A method for large volume three-dimensional microscopic imaging with high temporal and spatial resolution according to claim 6, characterized in that: The excitation light for generating strip lighting comprises: emitting the excitation light; Filtering the excitation light to generate filtered excitation light; Expanding the filtered excitation light to generate expanded excitation light; Limiting the illumination range of the expanded excitation light to determine the excitation light that meets the preset strip illumination conditions; The excitation light satisfying the preset strip illumination condition is subjected to optical Fourier transform to generate target excitation light.

8. A large volume three-dimensional microscopic imaging method with high temporal and spatial resolution according to claim 6, characterized in that: The step of deflecting the excitation light to the strip sub-field of view and exciting the sample in the strip sub-field of view to determine the fluorescence signal emitted by the sample, and descanning the fluorescence signal to generate a descanned fluorescence signal includes: reflecting the excitation light to generate reflected excitation light; deflecting the reflected excitation light at a preset angle to generate deflected excitation light; Relaying the deflected excitation light at a preset magnification to generate relayed excitation light; The fluorescence signal of the sample is collected, and a descanned fluorescence signal is generated.

9. A large volume three-dimensional microscopic imaging method with high temporal and spatial resolution according to claim 8, characterized in that: The generating of imaging results of all the strip-shaped sub-fields of view at different imaging viewing angles includes: focusing the fluorescent signal; Generate spatial spectrum information after optical Fourier transformation; Filter out the residual excitation light in the imaging light; Performing spatial multiplexing imaging on the spatial spectrum information after the optical Fourier transform to generate the multi-view imaging result; The multi-view imaging results are recorded.

10. A method for large volume three-dimensional microscopic imaging with high temporal and spatial resolution according to claim 6, characterized in that: The reconstructing the imaging results of all the strip sub-fields of view at the different imaging viewing angles to determine the three-dimensional reconstructed images of all the strip sub-fields of view, and splicing the three-dimensional reconstructed images of all the strip sub-fields of view to generate a three-dimensional microscopic image of the target large field of view, includes: Based on the imaging result at the central depth of the imaging depth of field, calibrate the imaging position coordinates of the fluorescent spheres at each sub-viewing angle at the central depth of the imaging depth of field, and perform perspective segmentation on the original imaging result of the strip sub-field of view according to the imaging position coordinates to obtain the imaging results of the strip sub-field of view at the different imaging perspectives; Based on the imaging results of the strip sub-field of view at the different imaging viewing angles, the imaging results at the central viewing angle are used as a reference to obtain image calibration values ​​that meet a preset low-aberration condition, and the imaging results are aligned pixel by pixel according to the image calibration values ​​that meet the preset low-aberration condition to generate a corrected multi-view imaging result; Based on the corrected multi-view imaging result, the data processed by the view segmentation and adaptive correction are three-dimensionally reconstructed using the three-dimensional point spread function of the strip sub-field of view to determine a three-dimensional imaging result that meets a preset condition; Based on the three-dimensional imaging result that meets the preset conditions, the intensity ratio of the overlapping area between adjacent strip sub-fields of view is calculated, and the adjacent strip sub-fields of view are linearly compensated according to the intensity ratio to generate a linear compensation result, and based on the linear compensation result, the three-dimensional reconstructed images of all the strip sub-fields of view are spliced ​​into a three-dimensional microscopic image of the target field of view.

Images