3D fluorescence imaging method and system for selective illumination and detection of arbitrary depth range

Abstract

The present invention belongs to the field of microscopic imaging technology, and specifically relates to a 3D fluorescence imaging method and system for selective illumination and detection of any depth range, including using a beam shaping device to shape an incident parallel light beam into a light beam with the same shape as the imaged area, selectively illuminating only the imaged area to excite the fluorescence at the desired depth position of the sample; by introducing an adjustable aperture diaphragm and a phase modulation device that are both conjugated with the rear focal plane of the imaging objective lens between the imaging objective lens and the fluorescence camera, respectively changing the effective numerical aperture of the imaging objective lens and the phase distribution of the rear focal plane of the objective lens, thereby changing the point spread function of the imaging objective lens; performing three-dimensional reconstruction based on the specific point spread function modulated and the superimposed image of the fluorescence signal at the desired depth position collected on the camera, and obtaining the original three-dimensional structure of the sample. The present invention can selectively illuminate and detect only the fluorescence in the depth range of interest, effectively improving the signal-to-noise ratio and contrast of three-dimensional image reconstruction.

Description

Technical Field

[0001] The present invention belongs to the field of microscopic imaging technology, and more specifically, relates to a 3D fluorescence imaging method and system for selective illumination and detection of arbitrary depth range. Background Art

[0002] Being able to quickly and accurately obtain the 3D structure of a sample is very important for many application scenarios. In the current field of 3D fluorescence microscopy, there are often two ways to obtain the 3D structure of a sample. First, a certain layer of the sample is illuminated by a traditional wide-field fluorescence microscope or a light-sheet illumination fluorescence microscope. The imaging objective has the characteristic of a Gaussian-distributed point spread function, which can effectively collect only the fluorescence signal located above the focal plane of the imaging objective. By scanning the sample axially layer by layer, images of the sample at different depths can be obtained. After subsequent axial stitching, the original 3D structure of the sample can be obtained. An obvious disadvantage of this imaging method is that the sample needs to be scanned layer by layer to obtain the 3D structure, so that the time to obtain the 3D structure of a sample is inversely proportional to the number of layers that need to be scanned. The time resolution is low and cannot meet some application scenarios that require rapid capture of 3D structures. Second: using light-field fluorescence microscopy, the shape of the imaging system's point spread function is designed to be related to the different depths of the sample. Then, using the known shape of the point spread function and the two-dimensional overlay pattern of fluorescence signals from different depths of the sample on the fluorescence camera, and using a 3D reconstruction algorithm (such as a 3D deconvolution algorithm), the original 3D structure of the sample can be reconstructed from a single two-dimensional image, thereby avoiding the problem of wide-field fluorescence microscopy needing to scan the sample layer by layer, greatly improving the imaging speed.

[0003] However, there are obviously two problems with traditional light field fluorescence microscopy: First, traditional light field fluorescence microscopy often uses epi-illumination from top to bottom, that is, the entire depth of the sample will be illuminated, which will cause a lot of spontaneous fluorescence and non-interested sample areas to be illuminated to produce fluorescence. After these redundant fluorescence signals are collected by the camera, they greatly affect the later 3D reconstruction effect. Second, traditional light field fluorescence microscopy often requires the use of a processed microlens array to modulate the shape of the point spread function in the detection light path, but this method is obviously very inflexible. With different sparseness of different samples and different depth ranges of interest (for example, sample 1 needs to be imaged at a depth of 500 microns, while sample 2 only needs to be imaged at a depth of 200 microns), the shape of the point spread function required for detecting different depth ranges of the sample should also be different. For traditional light field fluorescence microscopy, the only way is to reprocess a new microlens array, or sacrifice the effect of 3D reconstruction and use the same set of microlens arrays for all application scenarios. Summary of the invention

[0004] In view of the defects and improvement needs of the prior art, the present invention provides a 3D fluorescence imaging method and system for selective illumination and detection of any depth range, which aims to solve the problem that the prior art cannot flexibly detect fluorescence signals in the required depth range of the sample.

[0005] To achieve the above object, according to one aspect of the present invention, a 3D fluorescence imaging method for selective illumination and detection of an arbitrary depth range is provided, comprising:

[0006] A beam shaping device is used to shape the incident parallel light beam into a beam with the same shape as the area to be imaged, selectively illuminating only the area to be imaged to stimulate the fluorescence at the desired depth position of the sample;

[0007] By introducing an adjustable aperture diaphragm and a phase modulation device, both of which are conjugated with the rear focal plane of the imaging objective lens, between the imaging objective lens and the fluorescent camera, the effective numerical aperture of the imaging objective lens and the phase distribution of the rear focal plane of the objective lens are changed respectively, thereby changing the point spread function of the imaging objective lens; wherein the adjustable aperture diaphragm is located behind the imaging objective lens and is conjugated with the rear focal plane of the imaging objective lens, and is used to change the effective numerical aperture of the imaging objective lens, thereby changing the axial continuous coverage range of the point spread function according to the required detection position; the phase modulation device is located behind the adjustable aperture diaphragm, and is conjugated with the rear focal plane of the imaging objective lens and the adjustable aperture diaphragm at the same time, and is used to change the phase distribution of the rear focal plane of the imaging objective lens, thereby strengthening or weakening the partial axial coverage range of the point spread function according to the required detection position; the imaging objective lens is located above the sample, and is used to collect the fluorescence signal at the required depth position based on the specific point spread function modulated;

[0008] Based on the specific point spread function that is modulated and the superimposed image of the fluorescence signal at the required depth position collected by the camera, three-dimensional reconstruction is performed to obtain the original three-dimensional structure of the sample.

[0009] Furthermore, the selective illumination of only the area to be imaged is achieved by:

[0010] The light spot size of the light source required for exciting fluorescence is enlarged to evenly illuminate the beam shaping device; the beam shaping device generates a beam that selectively illuminates the sample at the required depth; the beam generated by the beam shaping device is reduced and conjugated to the sample cavity to excite sample fluorescence.

[0011] Furthermore, the beam shaping device is a digital micro-mirror array, which generates a light beam that selectively illuminates a sample at a desired depth position by modulating light intensity distribution.

[0012] Furthermore, the phase modulation device is specifically a spatial light modulator or a deformable mirror.

[0013] Furthermore, the phase distribution of the rear focal plane of the imaging objective lens is changed, and the specific implementation method is as follows:

[0014] According to the sample depth range to be detected, a required point spread function with multiple sub-foci is obtained, wherein the multiple sub-foci are evenly distributed within the depth range; the characteristic corresponding to each sub-foci i in the point spread function with multiple sub-foci is:

[0015] PSF i =W x-倾斜 PSF x-倾斜 +W y-倾斜 PSF y-倾斜 +W z-离焦 PSF z-离焦 ;

[0016] Among them, W x-倾斜 and W y-倾斜 The PSF is the degree of inclination of the control sub-focus in the x and y axis directions respectively. x-倾斜 、PSF y-倾斜 The weight of z-离焦 To control the defocus degree of the sub-focus in the z-axis direction PSF z-离焦 weight; change the weight W x-倾斜 and W y-倾斜 , to change the sub-focus tilt degree and capture the sample angular space information; change the weight W z-离焦 , to configure the sub-focus position according to the sample depth range required for detection;

[0017] By using Fourier optics theory, different weights are assigned to each sub-aperture region on the phase modulation device to perform regional sub-aperture phase synthesis, and finally the required phase distribution on the phase modulation device is generated to realize the required point spread function with multiple sub-foci, wherein the phase of each sub-aperture region is expressed as in, and They are respectively used to control the two-dimensional tilt terms in each sub-aperture area. and and out-of-focus items The weight of the phase of and Zernike polynomials The expressions when n and m take different values;

[0018] According to the required phase distribution, the voltage loaded on each pixel on the spatial light modulator is changed, thereby changing the phase delay of the pixel to the incident light, thereby modulating the phase distribution of the rear focal plane of the imaging objective lens.

[0019] The present invention also provides a 3D fluorescence imaging system for selective illumination and detection of any depth range, comprising: a selective area illumination device, a selective area imaging device, and an imaging result processing device;

[0020] The selective area lighting device is used to use a beam shaping device to shape the incident parallel light beam into a light beam with the same shape as the area to be imaged, selectively illuminating only the area to be imaged to stimulate the fluorescence at the desired depth position of the sample;

[0021] The selected area imaging device includes an imaging objective, an adjustable aperture diaphragm and a phase modulation device; the adjustable aperture diaphragm is located behind the imaging objective and is conjugated with the rear focal plane of the imaging objective, and is used to change the effective numerical aperture of the imaging objective, thereby changing the axial continuous coverage range of the point spread function according to the required detection position; the phase modulation device is located behind the adjustable aperture diaphragm and is conjugated with the rear focal plane of the imaging objective and the adjustable aperture diaphragm at the same time, and is used to change the phase distribution of the rear focal plane of the imaging objective, thereby strengthening or weakening the partial axial coverage range of the point spread function according to the required detection position; the imaging objective is located above the sample, and is used to collect the fluorescence signal at the required depth position based on the specific point spread function modulated;

[0022] The imaging result processing device is used to perform three-dimensional reconstruction according to the modulated specific point spread function and the superimposed image of the fluorescence signal at the required depth position collected by the camera to obtain the original three-dimensional structure of the sample.

[0023] Further, the selective area lighting device comprises: a laser, a laser beam expansion unit, a beam modulation unit, and a beam relay unit;

[0024] The laser is used to provide the light source required to excite fluorescence;

[0025] The laser beam expansion unit is used to expand the spot size of the light source and evenly illuminate the beam modulation unit;

[0026] The beam modulation unit is used to generate a beam that selectively illuminates a sample at a desired depth position by modulating the light intensity distribution;

[0027] The beam relay unit is used to reduce the beam generated by the beam modulation unit and used to selectively illuminate the sample at the desired depth position, and conjugate the beam into the sample cavity, so as to selectively activate the sample at the desired depth position.

[0028] Further, the selected area imaging device comprises: an imaging objective lens, a relay lens group 1, an adjustable aperture diaphragm, a polarizer, a relay lens group 2, a phase modulation device, and a camera;

[0029] The imaging objective lens is used to collect fluorescence signals from different depths when the sample is excited;

[0030] The relay lens group 1 is used to conjugate the rear focal plane of the imaging objective lens to the surface of the adjustable aperture stop and provide a magnification so that the size of the rear focal plane of the objective lens is within the effective range of the adjustable aperture stop;

[0031] The polarizer is used to convert fluorescence without a polarization state into polarized light with a single polarization state, and make its polarization direction parallel to the optical axis of the phase modulation device, so that it can be phase modulated by the phase modulation device;

[0032] The relay lens group 2 is used to further conjugate the plane of the adjustable aperture diaphragm conjugated with the rear focal plane of the objective lens to the surface of the phase modulation device, so that the three planes are conjugated with each other; by changing the voltage loaded on each pixel of the phase modulation device, the rear focal plane of the imaging objective lens is phase modulated, thereby changing the shape of the point spread function of the imaging objective lens;

[0033] The camera is used to receive the fluorescence signals finally coming from different depths of the sample and convolved with the point spread functions corresponding to each depth.

[0034] Further, the selected area imaging device includes a point spread function calculation device;

[0035] It is used to obtain the required point spread function with multiple sub-foci according to the sample depth range to be detected, wherein the multiple sub-foci are evenly distributed within the sample depth range; the characteristic corresponding to each sub-focus i in the point spread function with multiple sub-foci is: PSF i =W x-倾斜 PSF x-倾斜 +W y-倾斜 PSF y-倾斜 +W z-离焦 PSF z-离焦 ;

[0036] Among them, W x-倾斜 and W y-倾斜 are the weights for controlling the inclination of the sub-focus in the x and y axis directions respectively; W z-离焦 To control the weight of the sub-focus in the z-axis direction; change W x-倾斜 and W y-倾斜 The weight of W is used to change the tilt degree of the sub-focus to capture the sample angular space information; z-离焦 weights to configure the sub-focus position for the sample depth range required to be detected;

[0037] By using Fourier optics theory, different weights are assigned to each sub-aperture region on the phase modulation device to perform regional sub-aperture phase synthesis, and finally the required phase distribution on the phase modulation device is generated to realize the required point spread function with multiple sub-foci, wherein the phase of each sub-aperture region is expressed as in, and They are respectively used to control the two-dimensional tilt terms in each sub-aperture area. and and out-of-focus items The weight of the phase of and Zernike polynomials The expression when n and m take different values.

[0038] Furthermore, the phase modulation device is specifically a spatial light modulator or a deformable mirror.

[0039] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0040] (1) The present invention proposes selective illumination at the illumination end, and uses a beam shaping device to shape the incident parallel circular light beam into a beam of a specific shape according to the area to be imaged, selectively illuminating only the area to be imaged to stimulate fluorescence at the desired depth position of the sample.

[0041] (2) The present invention proposes to realize programmable point spread function at the imaging end, introduce an adjustable aperture diaphragm that can flexibly change the effective numerical aperture of the imaging system and a spatial light modulator that performs phase modulation on the rear focal plane of the objective lens into the imaging system, and further combine a new regional sub-aperture phase synthesis algorithm to generate the required phase distribution on the spatial light modulator, thereby programmably changing the point spread function of the imaging system. The generated specific point spread function is convolved with the structure of the sample at different depths and collected by the fluorescence camera. Using the three-dimensional reconstruction algorithm, the superimposed two-dimensional images are reconstructed into the three-dimensional structure of the sample.

[0042] In general, the present invention improves the problem of low signal-to-noise ratio and contrast in obtaining three-dimensional structure of samples by traditional light-field fluorescence microscopes from two aspects: first, the illumination end can selectively illuminate only the position of the sample of interest, reducing optical damage to non-interested areas and avoiding interference of fluorescence excited in non-interested areas with the subsequent three-dimensional reconstruction effect. Second, the imaging end can selectively detect only the position of the sample of interest, greatly improving the collection efficiency of fluorescence signals in the area of ​​interest. Combining the two together allows it to cope with more complex 3D fluorescence imaging scenes on the one hand, and on the other hand, for the same imaging scene, the three-dimensional reconstruction effect can have a higher signal-to-noise ratio and contrast than traditional light-field fluorescence microscopes. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A schematic diagram of a 3D fluorescence imaging system capable of selective illumination and detection of any depth range provided by an embodiment of the present invention;

[0044] Figure 2 A flow chart of a 3D fluorescence imaging system capable of selective illumination and detection of any depth range provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of a point spread function of a specific shape obtained by changing the aperture of the aperture stop and the pattern loaded onto the spatial light modulator provided in an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of a regional sub-aperture phase synthesis algorithm provided by an embodiment of the present invention;

[0047] Figure 5 A schematic diagram showing the influence of keeping the phase on the spatial light modulator unchanged and changing the aperture of the aperture stop on the shape of the point spread function and the three-dimensional reconstruction effect provided by an embodiment of the present invention;

[0048] Figure 6 A schematic diagram showing the influence of maintaining the clear aperture of the aperture stop and changing the phase on the spatial light modulator on the shape of the point spread function and the three-dimensional reconstruction effect provided by an embodiment of the present invention;

[0049] Figure 7 A schematic diagram of an example of selective illumination and detection of two depth regions of interest provided by an embodiment of the present invention;

[0050] Figure 8 A schematic diagram of a 10-micron fluorescent ball provided in an embodiment of the present invention using selective illumination and non-selective illumination;

[0051] Fig. 9A schematic diagram comparing the effects of three-dimensional reconstruction of 10-micron fluorescent spheres provided in an embodiment of the present invention using selective illumination and non-selective illumination.

[0052] Throughout the drawings, the same reference numerals are used to denote the same elements or structures, wherein:

[0053] 1 is the laser light source, 2 is the laser beam expansion unit, 3 is the laser shaping unit, 4 is the relay illumination lens, 5 is the illumination objective lens, 6 is the sample, 7 is the imaging objective lens, 8 is the first relay lens group, 9 is the reflector, 10 is the adjustable aperture diaphragm, 11 is the polarizer, 12 is the second relay lens group, 13 is the spatial light modulator, 14 is the lens, and 15 is the fluorescence camera. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0055] Embodiment 1

[0056] A 3D fluorescence imaging method for selective illumination and detection of arbitrary depth ranges, comprising:

[0057] A beam shaping device is used to shape the incident parallel light beam into a beam with the same shape as the area to be imaged, selectively illuminating only the area to be imaged to stimulate the fluorescence at the desired depth position of the sample;

[0058] By introducing an adjustable aperture diaphragm and a phase modulation device, both of which are conjugated with the rear focal plane of the imaging objective lens, between the imaging objective lens and the fluorescence camera, the effective numerical aperture of the imaging objective lens and the phase distribution of the rear focal plane of the objective lens are changed respectively, thereby changing the point spread function of the imaging objective lens; wherein the adjustable aperture diaphragm is located behind the imaging objective lens and is conjugated with the rear focal plane of the imaging objective lens, and is used to change the effective numerical aperture of the imaging objective lens, thereby changing the axial continuous coverage range of the point spread function according to the required detection position; the phase modulation device is located behind the adjustable aperture diaphragm and is conjugated with the rear focal plane of the imaging objective lens and the adjustable aperture diaphragm at the same time, and is used to change the phase distribution of the rear focal plane of the imaging objective lens, thereby strengthening or weakening the partial axial coverage range of the point spread function according to the required detection position; the imaging objective lens is located above the sample, and is used to collect the fluorescence signal at the required depth position based on the specific point spread function modulated;

[0059] Based on the specific point spread function that is modulated and the superimposed image of the fluorescence signal at the required depth position collected by the camera, three-dimensional reconstruction is performed to obtain the original three-dimensional structure of the sample.

[0060] This embodiment addresses two problems existing in the prior art: 1. It is not possible to flexibly generate point spread functions of different shapes according to different samples. 2. It uses indifferent illumination of all depths, so that the fluorescence of non-interested areas interferes with the subsequent three-dimensional reconstruction. This embodiment proposes a point spread function of selective illumination + selective detection.

[0061] The idea of ​​programmable point spread function (PSF) is mainly to solve the first of the above shortcomings. According to the physical model of the generation of point spread function (wave optics theory in imaging optical path), if you want to modulate the shape of point spread function, you can use two methods. First, change the effective numerical aperture (NA) of the imaging system; second, change the phase distribution of the focal plane behind the imaging objective (Fourier plane of the sample). Therefore, according to these two theories, in the traditional fluorescence imaging optical path, an adjustable aperture diaphragm and a spatial light modulator are introduced on the conjugate plane of the focal plane behind the imaging objective. The size of the adjustable aperture diaphragm can be changed to change the effective numerical aperture of the imaging system. By loading different voltages on the spatial light modulator, the phase distribution of the focal plane behind the objective can be changed, thereby affecting the shape of the point spread function of the imaging system. Combining Fourier optical theory and regional sub-aperture synthesis method, the phase distribution loaded on the spatial light modulator can be generated according to the sample depth required for imaging. Thus, point spread functions with focal points located in different depth ranges can be generated programmably according to the needs of the experimenter.

[0062] In addition, the embodiment proposes selective illumination. First, it is possible to selectively illuminate only the sample at a depth of interest, which can fundamentally avoid the generation of fluorescent signals at non-interested positions. The core point of selective illumination is that a light beam that only illuminates the depth of interest can be programmably generated. This embodiment can adopt a side lighting method and introduce a light intensity modulation device into the illumination light path (here is a digital micro-mirror array, but this is only an example used in the present invention, and the actual situation should not be limited to this device). By selectively turning on several micro-mirrors of the digital micro-mirror array, a light beam shape that only illuminates a specific depth of interest can be generated, thereby selectively illuminating a specific depth of the sample.

[0063] In summary, the combination of selective illumination and selective detection point spread functions enables selective illumination and detection of 3D fluorescence imaging in any depth range, improving the signal-to-noise ratio and contrast of subsequent three-dimensional reconstruction.

[0064] That is, because only the imaging end can programmatically generate a multi-focal point spread function, the interference of fluorescence signals in non-essential areas can be reduced, but it cannot be completely avoided (because non-essential areas will still be fully illuminated and produce fluorescence). Only by adding a device that can selectively illuminate the depth of interest at the illumination end and not illuminate non-essential areas can the interference of fluorescence signals in non-essential areas on later 3D reconstruction be completely avoided.

[0065] Preferably, the way to change the phase distribution of the rear focal plane of the imaging objective lens may be:

[0066] According to the sample depth range to be detected, the required point spread function with multiple sub-foci is obtained, wherein the multiple sub-foci are evenly distributed within the depth range; the characteristic corresponding to each sub-foci i in the point spread function with multiple sub-foci is:

[0067] PSF i =W x-倾斜 PSF x-倾斜 +W y-倾斜 PSF y-倾斜 +W z-离焦 PSF z-离焦 ;

[0068] Among them, W x-倾斜 and W y-倾斜 The PSF is the degree of inclination of the control sub-focus in the x and y axis directions respectively. x-倾斜 、PSF y-倾斜 The weight of z-离焦 To control the defocus degree of the sub-focus in the z-axis direction PSF z-离焦 weight; change the weight W x-倾斜 and W y-倾斜 , to change the sub-focus tilt degree and capture the sample angular space information; change the weight W z-离焦 , in order to flexibly configure the sub-focus position according to the sample depth range required for detection and optimize the 3D reconstruction effect;

[0069] By using Fourier optics theory, different weights are assigned to each sub-aperture region on the phase modulation device to perform regional sub-aperture phase synthesis, and finally the required phase distribution on the phase modulation device is generated to realize the required point spread function with multiple sub-foci, wherein the phase of each sub-aperture region is expressed as in, and They are respectively used to control the two-dimensional tilt terms in each sub-aperture area. and and out-of-focus items The weight of the phase of and Zernike polynomials The expressions when n and m take different values;

[0070] According to the required phase distribution, the voltage loaded on each pixel on the spatial light modulator is changed, thereby changing the phase delay of the pixel to the incident light, thereby modulating the phase distribution of the rear focal plane of the imaging objective lens.

[0071] The required point spread function with multiple sub-foci is explained as follows:

[0072] The three-dimensional distribution of the point spread function of a microscope system can be expressed as PSF(x,y,z). According to Fourier optics theory, the process of the fluorescence emitted from the sample space and finally propagating to the camera surface can be expressed as a discrete convolution process:

[0073]

[0074] Among them, Img camera It is the superposition image of samples at different depths (Z) taken by the camera; Sample z is the sample two-dimensional plane at depth Z; PSF z is the two-dimensional function of the point spread function at depth Z, and N is the number of two-dimensional planes within the required shooting depth range; is the convolution operator. According to the properties of Fourier transform, formula (1) can be written in the form of the following product:

[0075]

[0076] That is, the Fourier transform of the image captured by the camera is the Fourier transform of the sample at depth Z and the Fourier transform of the point spread function at depth Z Therefore, the process of three-dimensional microscopic imaging is equivalent to the original information of each depth Z of the sample. Passed through a low pass filter with different bandwidths Finally propagates to the camera. Therefore, when the depth is the low-pass filter at Z The larger the cutoff frequency, the larger the high-frequency signal that the image captured by the camera can carry (the larger the high-frequency signal, the higher the system imaging resolution). Therefore, in order to ensure the imaging resolution level of the target depth Z, keep the depth low-pass filter It is important to have a high cutoff frequency. Again, according to Fourier transform theory, if the PSF at a certain depth z With a very concentrated energy distribution (ie, the depth has a sub-focus and the sub-focus size is small), the cutoff frequency at this depth will be very large.

[0077] Therefore, the principle of generating a multi-focus point spread function according to the depth range of interest is:

[0078] (1) It is necessary to ensure that there are several sufficiently small sub-focuses within the depth range of interest to provide a higher cutoff frequency in the depth range and ensure imaging resolution;

[0079] (2) According to experiments, the more sub-focuses there are in the depth range of interest, the better the 3D reconstruction effect will be within that depth range.

[0080] (3) Each sub-focus in the multi-focus point spread function proposed in this embodiment has a certain inclination relative to the traditional Gaussian point spread function, and the inclination angles of different sub-focuses are different. The inclination of the sub-focus will cause different projection positions of samples at different depths on the camera surface, which is equivalent to the camera capturing information from different angles of the sample. The acquisition of this angular space information is more conducive to the three-dimensional reconstruction of the sample in the later stage.

[0081] For example: Assuming that the three-dimensional sample of interest to the researcher is in the range of -200 to -100 microns and +100 to +150 microns, a point spread function with 3×3=9 sub-foci can be generated. Among them, 6 sub-foci are set for the range of -200 to -100 microns, and the axial length of each sub-foci should be greater than The six focal points should be evenly distributed in the range of -200 to -100 microns; three sub-foci should be set for the range of +100 to +150 microns, and the axial length of each sub-focal point should be greater than And the three sub-foci should be evenly distributed in the range of +100 to +150 microns; according to the requirements of the multi-focal point spread function, the regional sub-aperture phase synthesis algorithm proposed in the present invention can be used to calculate the phase distribution required to be loaded on the spatial light modulator, so as to actually obtain the target point spread function.

[0082] The regional sub-aperture phase synthesis algorithm is described as follows:

[0083] Zernike polynomials are a complete set of infinitely many mutually orthogonal It plays an important role in the analysis of optical aberrations. and Control the tilt of the PSF in the x and y axes, The term controls the defocus size of the PSF. Therefore, by linearly superimposing a series of different Zernike polynomials, the shape and distribution of the PSF can be flexibly changed. For example, in this embodiment, in order to generate multi-focal PSFs with different depths, the SLM surface that can modulate the back pupil phase of the objective lens is first divided into N×N square sub-areas. Assume that the phase of one of the sub-areas (jth row and kth column) can be expressed as: in and are the weights of the tilt term and the defocus term in each sub-region. and When both are 0, the sub-region phase is not modulated, so the PSF has only one focus. and When the appropriate non-zero weight is and Weight and The PSF of the sub-region is tilted (the larger the weight, the greater the tilt), and the original single focus becomes N×N multi-focus. However, at this time, due to the is still 0 (controls the degree of defocus of PSF), so all multi-foci are located at the same depth. Assuming that the focus of the jth row and kth column needs to move along the axis, the sub-region should be assigned Suitable non-zero value (the larger the weight, the greater the defocus positive degree, The positive or negative value of affects the direction in which the focus moves up or down). Therefore, depending on the target depth range of the sample to be imaged, the The formula can accurately control the tilt and defocus of each sub-focus, thereby affecting the fluorescence depth range detected by the detection light path. It is worth noting that in this embodiment, since we are more concerned about the different depth ranges of the sample, a non-zero tilt term weight of an appropriate size (empirical value) is generally fixed for each SLM sub-area in this embodiment. and Only change , thereby changing the position of the sub-focus in the axial direction.

[0084] In addition, there are two main factors that affect the PSF shape of the imaging system: the effective numerical aperture (NA) of the objective lens and the effective ) and the phase distribution of the objective rear focal plane.

[0085] After a commercial objective lens is manufactured, its maximum numerical aperture (NA max) has been determined (generally marked on the surface of the objective lens barrel). However, by introducing a circular diaphragm with adjustable diameter on the conjugate plane of the rear focal plane of the objective lens, and using the diaphragm to block part of the light on the rear focal plane of the objective lens, the effective numerical aperture of the objective lens can be reduced (the smaller the diaphragm diameter, the more light is blocked, and the smaller the effective numerical aperture of the objective lens). Reducing the effective numerical aperture of the objective lens will cause the PSF of the imaging system to be axially elongated. Therefore, when it is necessary to simultaneously detect a large depth range of the sample, the effective numerical aperture of the objective lens can be reduced and the PSF can be axially elongated, so that the camera can simultaneously detect fluorescence in a larger depth range and increase the reconstructed depth range. Therefore, the effective numerical aperture (NA) of the objective lens can be changed. effective ) Change the depth range of the detected sample. If the sample depth range of interest is small, the aperture diameter should be increased to increase the effective numerical aperture and reduce the axial coverage of the PSF; if the sample depth range of interest is large, the aperture diameter should be reduced to reduce the effective numerical aperture and increase the axial coverage of the PSF.

[0086] However, if the effective numerical aperture of the objective lens is only reduced, the PSF can only be continuously stretched along the axial direction. On the one hand, it is impossible to generate a multi-focal PSF, and on the other hand, it is impossible to change the detection depth discontinuously. For example: jump detection of different depths of the sample, assuming that the observer only wants to image the two depth ranges of -200 to -100 microns and +100 to +200 microns in the sample, and does not care about the sample of -100 to +100 microns. In this case, reducing the effective numerical aperture of the objective lens is powerless. Therefore, the technology proposed in the present invention to change the phase distribution of the rear focal plane of the objective lens (regional sub-aperture phase synthesis algorithm) can solve this problem. The algorithm has two main advantages: first, it can generate a PSF with N×N multi-focal points (otherwise there will be only a single focus). Each sub-focus in the multi-focal point spread function used in the present invention has a certain inclination relative to the traditional Gaussian point spread function, and the inclination angles of different sub-focuses are different. The inclination of the sub-focus will cause the projection positions of samples of different depths on the camera surface to be different, which is equivalent to the camera capturing information from different angles of the sample. The acquisition of this angular spatial information is more conducive to the subsequent three-dimensional reconstruction of the sample; secondly, changing the phase of the regional sub-aperture The weight allows the focus generated by the sub-region to be moved to the range of interest according to the depth range of the sample of interest, avoiding the collection of fluorescence at depths that are not of interest, thereby avoiding the interference of non-interest signals on image reconstruction.

[0087] Therefore, both of the above two methods of changing the multifocal PSF shape can be implemented using this embodiment. By properly combining the two, the optimal multifocal PSF shape can be provided for different depth ranges of the sample of interest, thereby improving the quality of three-dimensional reconstruction.

[0088] In summary, this embodiment introduces an adjustable aperture diaphragm that can flexibly change the effective numerical aperture of the imaging system and a spatial light modulator that performs phase modulation on the rear focal plane of the objective lens in the imaging system, and further combines the phase synthesis algorithm of the regional sub-aperture proposed in this embodiment to generate the required phase distribution on the spatial light modulator, thereby programmably changing the point spread function of the imaging system. The generated specific point spread function is convolved with the structure of the sample at different depths, which is collected by the fluorescence camera, and the three-dimensional structure of the sample is reconstructed by the superimposed two-dimensional images using a three-dimensional reconstruction algorithm.

[0089] Further, it should be noted that programmable changes in the point spread function of the imaging system can cope with samples of different sparsity levels. Specifically, programmable changes in the point spread function of the imaging system give the system great flexibility for different three-dimensional imaging scenarios. For example: when the sample that the researcher needs to image is a dense sample, if the point spread function is directly axially stretched to cover the entire sample for imaging and three-dimensional reconstruction, the images of the fluorescent signals from different depths of the sample projected on the camera will be seriously superimposed on each other, and the three-dimensional reconstruction algorithm with sparse constraints (such as the three-dimensional deconvolution algorithm used in the present invention) is difficult to converge, which often makes the three-dimensional reconstruction effect very poor. At this time, in order to obtain a clearer three-dimensional reconstruction effect, the dense sample can be divided into several depth ranges, and the corresponding multi-focus point spread function can be designed for the depth range, and imaging is performed in sequence. For example: the researcher needs to image a densely labeled sample in the range of -200 to +200 microns. Assuming that the entire sample is imaged at the same time, the reconstruction effect is very poor. By using the present invention, the above volume can be divided into four sub-areas of -200 to -100 microns, -100 to 0 microns, 0 to +100 microns and +100 to +200 microns, and the corresponding multi-focus point spread functions are generated in sequence. By using this method, the density of projection on the camera surface can be artificially reduced to improve the three-dimensional reconstruction effect. It should be noted that this is only an application example of the present invention, and the specific number of sub-areas can be set by the experimenter according to the required three-dimensional imaging effect.

[0090] Embodiment 2

[0091] A 3D fluorescence imaging system for selective illumination and detection of any depth range, comprising: a selective area illumination device, a selective area imaging device, and an imaging result processing device;

[0092] The selective area illumination device is used to use a beam shaping device to shape the incident parallel light beam into a light beam with the same shape as the area to be imaged, selectively illuminate only the area to be imaged, so as to excite the fluorescence at the desired depth position of the sample; the selective area imaging device includes an imaging objective lens, an adjustable aperture diaphragm and a phase modulation device; the adjustable aperture diaphragm is located behind the imaging objective lens and is conjugated with the rear focal plane of the imaging objective lens, and is used to change the effective numerical aperture of the imaging objective lens, thereby changing the axial continuous coverage range of the point spread function according to the desired detection position; the phase modulation device is located behind the adjustable aperture diaphragm and is conjugated with the rear focal plane of the imaging objective lens and the adjustable aperture diaphragm at the same time, and is used to change the phase distribution of the rear focal plane of the imaging objective lens, thereby strengthening or weakening the partial axial coverage range of the point spread function according to the desired detection position; the imaging objective lens is located above the sample, and is used to collect the fluorescence signal at the desired depth position based on the specific point spread function modulated; the imaging result processing device is used to perform three-dimensional reconstruction based on the specific point spread function modulated and the superimposed image of the fluorescence signal at the desired depth position collected on the camera, so as to obtain the original three-dimensional structure of the sample.

[0093] Preferably, the selective area lighting device includes: a laser, a laser beam expansion unit, a beam modulation unit, and a beam relay unit; the laser is used to provide the light source required to excite fluorescence; the laser beam expansion unit is used to expand the spot size of the light source and evenly illuminate the beam modulation unit; the beam modulation unit is used to generate a light beam that selectively illuminates the sample at the desired depth position by modulating the light intensity distribution; the beam relay unit is used to shrink the light beam generated by the beam modulation unit that selectively illuminates the sample at the desired depth position and conjugate it into the sample cavity to selectively activate the sample at the desired depth position.

[0094] In addition, as another embodiment, the selective area lighting device still has other specific implementations. For example, (1) the selective area lighting device is not limited to Figure 1 The side illumination method shown in the figure. By inserting a folding mirror (Flipmirror) after the beam modulation unit, the modulated light beam is reflected to the relay unit 2 composed of another illumination lens tube lens and the imaging objective lens in the selective area imaging system by the folding mirror, and the modulated light beam can be vertically coupled into the imaging objective lens, and the position of interest can be selectively illuminated from top to bottom; (2) Further, the light intensity modulation device (such as a digital micro-mirror array) in the beam modulation unit is replaced with a phase modulation device (such as a spatial light modulator or a deformable mirror), and combined with the computational holography theory in coherent optics, an illumination beam with the same three-dimensional distribution as the target illumination area can be generated according to the three-dimensional spatial distribution of the sample of interest, which can solve the disadvantage of the current selective area illumination device that is not selective in the direction of beam propagation and realize selective illumination in three dimensions.

[0095] Preferably, the selective area imaging device comprises: an imaging objective lens, a relay lens group 1, an adjustable aperture diaphragm, a polarizer, a relay lens group 2, a phase modulation device, and a camera; the imaging objective lens is used to collect fluorescence signals from different depths when the sample is excited; the relay lens group 1 is used to conjugate the rear focal plane of the imaging objective lens to the surface of the adjustable aperture diaphragm and provide a magnification so that the size of the rear focal plane of the objective lens is within the effective range of the adjustable aperture diaphragm; the polarizer is used to convert fluorescence without a polarization state into polarized light with a single polarization state, and make its polarization The vibration direction is parallel to the optical axis of the phase modulation device, so that it can be phase modulated by the phase modulation device; the relay lens group 2 is used to further conjugate the plane of the adjustable aperture diaphragm conjugated with the rear focal plane of the objective lens to the surface of the phase modulation device, so that the three planes are conjugated with each other; by changing the voltage loaded on each pixel of the phase modulation device, the rear focal plane of the imaging objective lens is phase modulated, thereby changing the shape of the point spread function of the imaging objective lens; the camera is used to receive the final fluorescence signal from different depths of the sample and convolved with the point spread function corresponding to each depth.

[0096] Preferably, the selected area imaging device includes a point spread function calculation device;

[0097] It is used to obtain the required point spread function with multiple sub-foci according to the sample depth range to be detected, wherein the multiple sub-foci are evenly distributed within the sample depth range; the characteristic corresponding to each sub-focus i in the point spread function with multiple sub-foci is: PSF i =W x-倾斜 PSF x-倾斜 +W y-倾斜 PSF y-倾斜 +W z-离焦 PSF z-离焦 ;

[0098] Among them, W x-倾斜 and W y-倾斜 are the weights for controlling the inclination of the sub-focus in the x and y axis directions respectively; W z-离焦 To control the weight of the sub-focus in the z-axis direction; change W x-倾斜 and W y-倾斜 The weight of W is used to change the tilt degree of the sub-focus to capture the sample angular space information; z-离焦 weights to configure the sub-focus position for the sample depth range required to be detected;

[0099] By using Fourier optics theory, different weights are assigned to each sub-aperture region on the phase modulation device to perform regional sub-aperture phase synthesis, and finally the required phase distribution on the phase modulation device is generated to realize the required point spread function with multiple sub-foci, wherein the phase of each sub-aperture region is expressed as in, and They are respectively used to control the two-dimensional tilt terms in each sub-aperture area. and and out-of-focus items The weight of the phase of and Zernike polynomials The expression when n and m take different values.

[0100] Preferably, the phase modulation device is specifically a spatial light modulator or a deformable mirror.

[0101] like Figure 1 As shown, it mainly includes three parts: a selective area lighting device, a selective area imaging device, and an imaging result processing device. The selective area lighting device is: a laser light source 1 is used as an excitation light source for illuminating the sample, and a laser beam expansion unit 2 is used to expand the original laser beam to a size that just covers the surface of the laser shaping unit 3. The laser shaping unit 3 can generate a beam of selective area illumination, and then the beam relay unit composed of a relay illumination lens 4 and an illumination objective lens 5 reduces the above-mentioned selective area illumination beam by a suitable multiple and keeps the conjugate transmission to the sample cavity to excite the sample 6 to produce fluorescence. Selective area imaging device: After the imaging objective lens 7 collects the fluorescence excited in the sample, the first relay lens group 8 and the reflector 9 conjugate the back focal plane of the objective lens (the Fourier plane of the sample) to the adjustable aperture diaphragm 10. Adjusting the size of the adjustable aperture diaphragm 10 can change the effective numerical aperture of the imaging system to affect the shape of the point expansion function. The polarizer 11 modulates the non-polarized fluorescence into a polarization direction parallel to the direction of the optical axis of the spatial light modulator 13 to ensure that it can be modulated by the spatial light modulator. The fluorescence after passing through the polarizer 11 is conjugated to the surface of the spatial light modulator 13 by the second relay lens group 12. After the phase-modulated fluorescence passes through the lens 14, the two-dimensional image finally superimposed by the fluorescence at different depths of the sample is transmitted to the fluorescence camera 15. Imaging result processing device: The original three-dimensional structure of the sample can be reconstructed by using the shape of the experimentally measured point spread function, the image of the sample projected at different depths on the fluorescence camera and the three-dimensional reconstruction algorithm.

[0102] Figure 2 Flow chart of a 3D fluorescence imaging system that can selectively illuminate and detect any depth range. The main process includes:

[0103] S101: The light from the digital micro-mirror array is output as a single sheet beam to illuminate the sample, and the imaging light path is loaded with a plane phase on the spatial light modulator, so that the point spread function of the imaging light path is a Gaussian function, and the focus of the imaging light path is kept coincident with the illumination sheet beam. The system at this time is equivalent to a light sheet illumination fluorescence microscope. The sample is moved axially layer by layer, so as to obtain the original three-dimensional structure of the sample in a high-resolution horizontal state. This three-dimensional structure is used as the basis for selecting the imaging area of ​​interest later.

[0104] S102: After obtaining the imaging area of ​​interest through the above steps, a pattern for selective illumination is generated by using the correspondence between the sample and the micro-mirrors on the digital micro-mirror array and loaded onto the digital micro-mirror array to selectively illuminate the position of interest; Fourier optical theory and regional sub-aperture synthesis algorithm are used to obtain a phase distribution loaded onto the spatial light modulator to generate a point spread function with multiple foci, and the finest position of the multiple foci is only distributed at the position of interest, thereby achieving selective depth imaging.

[0105] S103: The sample is illuminated by the selective illumination beam above, and the point spread function of the detection light path only collects the fluorescence at the depth of interest. At this time, the fluorescence collected by the fluorescence camera is all from the two-dimensional projection of the position of interest of the sample (avoiding the interference of fluorescence signals from other non-interested areas). Using the three-dimensional deconvolution algorithm, the original structure of the sample area of ​​interest is calculated, which can not only obtain the three-dimensional distribution of the area of ​​interest from a single image, but also maximize the signal-to-noise ratio and contrast of the three-dimensional imaging.

[0106] Specifically, the implementation process of each step can refer to the introduction in the aforementioned method embodiment, which will not be repeated here.

[0107] like Figure 3 As shown in FIG. 1 , by changing the aperture size of the adjustable aperture diaphragm and using the phase synthesis algorithm of the regional sub-aperture (such as Figure 4 The regional sub-aperture phase synthesis algorithm shown in the figure can generate a phase map with a specific distribution for loading on the spatial light modulator. After the modulated fluorescence passes through the spatial light modulator, a point spread function with nine focal points mainly distributed at the middle depth is generated. This special point spread function, when convolved with the different depths of the sample, is converged to the camera by the imaging tube lens, and the original two-dimensional projection of multiple samples at different depths is obtained. This two-dimensional projection and the point spread function with nine focal points will be used for subsequent three-dimensional structure reconstruction.

[0108] like Figure 5As shown in the figure, by keeping the phase loaded on the spatial light modulator unchanged and only changing the aperture of the adjustable aperture diaphragm (the first row), it will affect the shape of the imaging system point spread function (the second row), the two-dimensional projection of the original sample obtained on the camera (the third row), the maximum projection of the original sample obtained by three-dimensional reconstruction on the xy plane (the fourth row) and the maximum projection on the zy plane (the fifth row). Figure 5 It can be seen that in this case, the smaller the size of the aperture stop (the first row), the wider the span of the point spread function in the axial direction (the second row), and the wider the axial range that can be reconstructed (the fifth row). This figure can prove that changing the size of the aperture can effectively affect the shape of the point spread function of the imaging system, thereby affecting the effect of the subsequent three-dimensional reconstruction. The sample used in this figure is 40-degree low-melting point agarose to embed 10um fluorescent beads, but in actual operation it is not limited to fluorescent beads, and can also be any fluorescent biological sample with a three-dimensional structure.

[0109] like Figure 6 As shown in the figure, keeping the aperture of the adjustable aperture diaphragm unchanged and only changing the phase distribution loaded on the spatial light modulator (first row) can change the position of the axial distribution of the point spread function of the imaging system (second row). When the point spread function with axially changed distribution is used to image the same fluorescent ball with a diameter of 10 microns, the two-dimensional projection obtained on the fluorescent camera (third row) will also be different. Using the known three-dimensional spatial distribution of the point spread function, the two-dimensional projection on the camera and the three-dimensional reconstruction algorithm, three different maximum projections of the original sample on the xy plane (fourth row) and the maximum projection on the xz plane (fifth row) can be obtained. It can be seen from the reconstructed maximum projection on the xz plane (fifth row) that changing the axial distribution of the multi-focus point spread function will change the clearest position of the reconstructed sample. If all the multi-focus point spread functions are distributed in the middle of the axial range, the reconstructed sample will also be clearest in the middle of the entire axial range; if some axial multi-foci are distributed above the axial range and others are distributed below, with no distribution in the middle (second row), the clearest parts of the reconstructed original sample will also be distributed at the top and bottom, and the middle will not be clear (fifth row). Figure 6 It has been fully proved that changing the phase loaded on the spatial light modulator can effectively change the shape of the point spread function of the imaging system, thereby affecting the clarity of samples at different depths reconstructed in three dimensions. For example: Assuming that you only care about the middle depth part of a three-dimensional sample, then use Figure 6 The point spread function in the first column can achieve the best 3D reconstruction effect; assuming that we only care about the depth range of the upper and lower halves of a 3D sample, but not the middle part, then use Figure 6The point spread function in the third column has the best three-dimensional reconstruction effect. Similarly, the sample used in this figure is 40-degree low-melting-point agarose to embed 10um fluorescent beads, but in actual operation it is not limited to fluorescent beads, and can also be any fluorescent biological sample with a three-dimensional structure.

[0110] Combination Figure 3-Figure 6 Based on the above imaging endpoint diffusion function programmable scheme, further Figure 7 As shown, assuming that we are only interested in two non-overlapping sub-regions above and below a three-dimensional space filled with 10-micron fluorescent balls, we should selectively illuminate the upper and lower sub-regions in the volume (left side), and the point spread function of the imaging light path should only focus on these two sub-regions (right side) to ensure that only the fluorescent signal at the position of interest is collected.

[0111] like Figure 8 As shown, there is a schematic diagram of non-selective illumination and detection of all samples (left side) and a schematic diagram of selective illumination and detection of part of the sample of interest (right side). Assuming that the region of interest is a letter of a HUST, the digital micro-mirror array can be used to selectively illuminate only the HUST letter area (this is an extreme case, and this operation is to effectively prove that the present invention can selectively illuminate three-dimensional volumes of various shapes almost arbitrarily). Secondly, the sample used in this figure is 40-degree low-melting point agarose to embed 10um fluorescent beads, but in actual operation it is not limited to fluorescent beads, and can also be any fluorescent biological sample with a three-dimensional structure.

[0112] like Fig. 9 As shown, in Figure 8 Compare the reconstruction effects of fluorescent beads at the same position under two different illumination and imaging modes on the left and right. Regardless of the maximum projection of the xy plane or the maximum projection of the xz plane, for the fluorescent beads in the region of interest, the signal-to-noise ratio and contrast of the three-dimensional reconstructed structure are significantly higher than those of all positions when the method of selective illumination and detection in the HUST letter area of ​​the present invention is used. Similarly, the sample used in this figure is 40-degree low-melting-point agarose to embed 10um fluorescent beads, but in actual operation it is not limited to fluorescent beads, and can also be any fluorescent biological sample with a three-dimensional structure.

[0113] In general, the present invention is a 3D fluorescence imaging method that can selectively illuminate and image only the depth range of interest in a three-dimensional sample. It includes expanding the output beam of the laser light source and irradiating it to an intensity modulation shaper to generate a beam that illuminates only the depth range of interest. The beam is projected into the sample cavity to excite only the fluorescence signal of the depth of interest, so as to reduce the optical damage of the laser to the sample in the imaging area of ​​non-interest and avoid the camera collecting the fluorescence excited in the non-interest area to interfere with the subsequent three-dimensional image reconstruction. In the detection light path, the variable aperture diaphragm and the spatial light modulator are placed on the conjugate focal plane of the detection objective lens, and the point spread function of the detection light path system can be changed by changing the diameter of the aperture diaphragm and the phase loaded on the spatial light modulator. Here, combined with Fourier optics, an algorithm is also proposed through the phase synthesis method of regional sub-apertures to generate a multi-focus point spread function that mainly collects fluorescence in a specified depth range. Combined with a device for selectively illuminating an arbitrary depth range and a device for selectively imaging an arbitrary depth range, a 3D fluorescence imaging method and system that can selectively illuminate and detect an arbitrary depth range is obtained. Compared with traditional light-field fluorescence microscopes, the present invention can selectively illuminate and detect fluorescence within a depth range of interest, thereby improving the ability of the fluorescence camera to collect useful information and effectively improving the signal-to-noise ratio and contrast of three-dimensional image reconstruction.

[0114] Of course, researchers can use this system to achieve traditional illumination and detection of all depths. The present invention provides users with more flexible application scenarios for different sample distributions. That is, since the invention itself is a technology with programmable point spread functions, it is not limited to generating three-dimensional multi-focus point spread functions. It can also generate a traditional Gaussian point spread function, turning it into a wide-field fluorescence microscope with one click; it can also generate an axially elongated point spread function (such as BesselPSF), turning it into a fluorescence microscope that can simultaneously obtain projections of samples at different depths with one click. The effective resolution of the detection light path can be changed by changing the size of the adjustable aperture diaphragm, thereby changing the effective numerical aperture of the imaging light path, thereby changing the thickness of the Gaussian point spread function. In addition, the present invention proposes a 3D fluorescence imaging method and system that can selectively illuminate and detect any depth range, and has the characteristics that both the illumination end and the imaging end are programmable. It can also realize the following application scenarios, such as: the illumination end illuminates the entire depth of the sample, and the imaging end is programmable; the illumination end generates a single sheet illumination beam, and the imaging end generates a Gaussian point spread function, and the focus of the position Gaussian point spread function coincides with the sheet illumination beam. Then the invention can be transformed into a light sheet illumination fluorescence microscope with one click, which is used to scan the sample layer by layer, and obtain the three-dimensional structure of the sample with higher spatial resolution. This scenario is more suitable for scenarios that do not require high three-dimensional imaging speed but have high imaging quality requirements; similarly, the imaging end generates an elongated point spread function (such as sphericali-aberration-assistedPSF), and at the same time combines multi-layer sheet illumination to realize a light sheet illumination microscopic imaging system with simultaneous multi-layer imaging. In short, due to the characteristics of programmable changes in both the imaging end and the illumination end, the present invention has a very flexible 3D fluorescence imaging scene, which is mainly used to solve the problem that the traditional light field fluorescence microscope cannot produce selective illumination and detection of a specific depth range for the depth range of interest of different samples, but its applicable fields are far from limited to this.

[0115] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for selective illumination and detection of 3D fluorescence imaging at any depth range, characterized in that: include: A beam shaping device is used to shape the incident parallel light beam into a beam with the same shape as the area to be imaged, selectively illuminating only the area to be imaged to stimulate the fluorescence at the desired depth position of the sample; By introducing an adjustable aperture diaphragm and a phase modulation device, both of which are conjugated with the rear focal plane of the imaging objective lens, between the imaging objective lens and the fluorescent camera, the effective numerical aperture of the imaging objective lens and the phase distribution of the rear focal plane of the objective lens are changed respectively, thereby changing the point spread function of the imaging objective lens; wherein the adjustable aperture diaphragm is located behind the imaging objective lens and is conjugated with the rear focal plane of the imaging objective lens, and is used to change the effective numerical aperture of the imaging objective lens, thereby changing the axial continuous coverage range of the point spread function according to the required detection position; the phase modulation device is located behind the adjustable aperture diaphragm, and is conjugated with the rear focal plane of the imaging objective lens and the adjustable aperture diaphragm at the same time, and is used to change the phase distribution of the rear focal plane of the imaging objective lens, thereby strengthening or weakening the partial axial coverage range of the point spread function according to the required detection position; the imaging objective lens is located above the sample, and is used to collect the fluorescence signal at the required depth position based on the specific point spread function modulated; Based on the specific point spread function that is modulated and the superimposed image of the fluorescence signal at the required depth position collected by the camera, three-dimensional reconstruction is performed to obtain the original three-dimensional structure of the sample.

2. A 3D fluorescence imaging method for selective illumination and detection of any depth range according to claim 1, characterized in that: The selective illumination of only the area to be imaged is achieved by: The light spot size of the light source required for exciting fluorescence is enlarged to evenly illuminate the beam shaping device; the beam shaping device generates a beam that selectively illuminates the sample at the required depth; the beam generated by the beam shaping device is reduced and conjugated to the sample cavity to excite sample fluorescence.

3. A 3D fluorescence imaging method for selective illumination and detection of arbitrary depth range according to claim 2, characterized in that: The beam shaping device is a digital micro-mirror array, which generates a light beam that selectively illuminates a sample at a desired depth by modulating light intensity distribution.

4. A 3D fluorescence imaging method for selective illumination and detection of any depth range according to any one of claims 1 to 3, characterized in that: The phase modulation device is specifically a spatial light modulator or a deformable mirror.

5. A 3D fluorescence imaging method for selective illumination and detection of any depth range according to any one of claims 1 to 3, characterized in that: The phase distribution of the rear focal plane of the imaging objective lens is changed, and the specific implementation method is as follows: According to the sample depth range to be detected, a required point spread function with multiple sub-foci is obtained, wherein the multiple sub-foci are evenly distributed within the depth range; the characteristic corresponding to each sub-foci i in the point spread function with multiple sub-foci is: PSF i =W x-倾斜 PSF x-倾斜 +W y-倾斜 PSF y-倾斜 +W z-离焦 PSF z-离焦 ; Among them, W x-倾斜 and W y-倾斜 The PSF is the degree of inclination of the control sub-focus in the x and y axis directions respectively. x-倾斜 、PSF y-倾斜 The weight of z-离焦 To control the defocus degree of the sub-focus in the z-axis direction PSF z-离焦 weight; change the weight W x-倾斜 and W y-倾斜 , to change the sub-focus tilt degree and capture the sample angular space information; change the weight W z-离焦 , to configure the sub-focus position according to the sample depth range required for detection; By using Fourier optics theory, different weights are assigned to each sub-aperture region on the phase modulation device to perform regional sub-aperture phase synthesis, and finally the required phase distribution on the phase modulation device is generated to realize the required point spread function with multiple sub-foci, wherein the phase of each sub-aperture region is expressed as in, and They are respectively used to control the two-dimensional tilt terms in each sub-aperture area. and and out-of-focus items The weight of the phase, and Zernike polynomials The expressions when n and m take different values; According to the required phase distribution, the voltage loaded on each pixel on the spatial light modulator is changed, thereby changing the phase delay of the pixel to the incident light, thereby modulating the phase distribution of the rear focal plane of the imaging objective lens.

6. A 3D fluorescence imaging system for selective illumination and detection of arbitrary depth ranges, characterized in that: include: Selecting a regional lighting device, selecting a regional imaging device, and an imaging result processing device; The selective area lighting device is used to use a beam shaping device to shape the incident parallel light beam into a light beam with the same shape as the area to be imaged, selectively illuminating only the area to be imaged to stimulate the fluorescence at the desired depth position of the sample; The selected area imaging device includes an imaging objective lens, an adjustable aperture diaphragm and a phase modulation device; The adjustable aperture diaphragm is located behind the imaging objective lens and is conjugated with the rear focal plane of the imaging objective lens, and is used to change the effective numerical aperture of the imaging objective lens, thereby changing the axial continuous coverage range of the point spread function according to the desired detection position; the phase modulation device is located behind the adjustable aperture diaphragm and is conjugated with the rear focal plane of the imaging objective lens and the adjustable aperture diaphragm at the same time, and is used to change the phase distribution of the rear focal plane of the imaging objective lens, thereby strengthening or weakening the partial axial coverage range of the point spread function according to the desired detection position; the imaging objective lens is located above the sample, and is used to collect the fluorescence signal at the desired depth position based on the specific point spread function modulated; The imaging result processing device is used to perform three-dimensional reconstruction according to the modulated specific point spread function and the superimposed image of the fluorescence signal at the required depth position collected by the camera to obtain the original three-dimensional structure of the sample.

7. A 3D fluorescence imaging system for selective illumination and detection of any depth range according to claim 6, characterized in that: The selective area lighting device comprises: a laser, a laser beam expansion unit, a beam modulation unit, and a beam relay unit; The laser is used to provide the light source required to excite fluorescence; The laser beam expansion unit is used to expand the spot size of the light source and evenly illuminate the beam modulation unit; The beam modulation unit is used to generate a beam that selectively illuminates a sample at a desired depth position by modulating the light intensity distribution; The beam relay unit is used to reduce the beam generated by the beam modulation unit and used to selectively illuminate the sample at the desired depth position, and conjugate the beam into the sample cavity, so as to selectively activate the sample at the desired depth position.

8. A 3D fluorescence imaging system for selective illumination and detection of any depth range according to claim 6 or 7, characterized in that: The selective area imaging device comprises: an imaging objective lens, a relay lens group 1, an adjustable aperture diaphragm, a polarizer, a relay lens group 2, a phase modulation device, and a camera; The imaging objective lens is used to collect fluorescence signals from different depths when the sample is excited; The relay lens group 1 is used to conjugate the rear focal plane of the imaging objective lens to the surface of the adjustable aperture stop and provide a magnification so that the size of the rear focal plane of the objective lens is within the effective range of the adjustable aperture stop; The polarizer is used to convert fluorescence without a polarization state into polarized light with a single polarization state, and make its polarization direction parallel to the optical axis of the phase modulation device, so that it can be phase modulated by the phase modulation device; The relay lens group 2 is used to further conjugate the plane of the adjustable aperture diaphragm conjugated with the rear focal plane of the objective lens to the surface of the phase modulation device, so that the three planes are conjugated with each other; by changing the voltage loaded on each pixel of the phase modulation device, the rear focal plane of the imaging objective lens is phase modulated, thereby changing the shape of the point spread function of the imaging objective lens; The camera is used to receive the fluorescence signals finally coming from different depths of the sample and convolved with the point spread functions corresponding to each depth.

9. A 3D fluorescence imaging system for selective illumination and detection of any depth range according to claim 6 or 7, characterized in that: The selected area imaging device includes a point spread function calculation device; It is used to obtain the required point spread function with multiple sub-foci according to the sample depth range to be detected, wherein the multiple sub-foci are evenly distributed within the sample depth range; the characteristic corresponding to each sub-focus i in the point spread function with multiple sub-foci is: PSF i =W x-倾斜 PSF x-倾斜 +W y-倾斜 PSF y-倾斜 +W z-离焦 PSF z-离焦 ; Among them, W x-倾斜 and W y-倾斜 are the weights for controlling the inclination of the sub-focus in the x and y axis directions respectively; W z-离焦 To control the weight of the sub-focus in the z-axis direction; change W x-倾斜 and W y-倾斜 The weight of W is used to change the tilt degree of the sub-focus to capture the sample angular space information; z-离焦 weights to configure the sub-focus position for the sample depth range required to be detected; By using Fourier optics theory, different weights are assigned to each sub-aperture region on the phase modulation device to perform regional sub-aperture phase synthesis, and finally the required phase distribution on the phase modulation device is generated to realize the required point spread function with multiple sub-foci, wherein the phase of each sub-aperture region is expressed as in, and They are respectively used to control the two-dimensional tilt terms in each sub-aperture area. and and out-of-focus items The weight of the phase of and Zernike polynomials The expression when n and m take different values.

10. A 3D fluorescence imaging system for selective illumination and detection of any depth range according to claim 6 or 7, characterized in that: The phase modulation device is specifically a spatial light modulator or a deformable mirror.

Images