Two-photon synthetic aperture microscopic imaging method, system, equipment and medium

Abstract

The invention provides a two-photon synthetic aperture microscopic imaging method, system and device and a medium, and the method comprises the steps: collecting two-photon excitation projection images of a plurality of preset sampling visual angles through a two-photon synthetic aperture microscope, and carrying out the up-sampling of the projection images, and obtaining a multi-pixel image; performing phase space reconstruction on the multi-pixel image by adopting a spark-fRL three-dimensional reconstruction algorithm, performing up-sampling processing on the reconstructed image to obtain a high-resolution three-dimensional two-photon imaging result, performing up-sampling processing on the multi-pixel image, and then performing reconstruction error estimation to suppress reconstruction artifacts, thereby obtaining a high-resolution three-dimensional two-photon imaging result. The problem of spatial resolution loss caused by sparse sampling is solved.

Description

Technical Field

[0001] The present invention relates to the field of optical imaging technology, and in particular to a two-photon synthetic aperture microscopy imaging method, system, equipment and medium. Background Art

[0002] In imaging frameworks based on point scanning imaging, such as confocal microscopy and two-photon microscopy, sampling needs to satisfy the Nyquist sampling theorem to achieve high spatial resolution imaging within the optical diffraction limit. For example, if the diffraction limit of an optical imaging system is 500nm, then the distance between two adjacent pixels during sampling should be less than or equal to 250nm to achieve the best imaging resolution for the microscopic image.

[0003] However, in this case, high spatial resolution imaging requires very dense sampling, which severely limits the imaging speed of the scanning system. Sparse sampling is an effective means to improve imaging speed, but it will bring serious loss of resolution.

[0004] Therefore, there is an urgent need to propose a two-photon synthetic aperture microscopy (2pSAM) imaging method, system, device and medium to solve the technical problem of loss of spatial resolution caused by sparse sampling. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the present disclosure provides a two-photon synthetic aperture microscopy imaging method and system to solve the technical problem of loss of spatial resolution caused by sparse sampling in the related art.

[0006] One or more embodiments of this specification provide a two-photon synthetic aperture microscopy imaging method, comprising the following steps:

[0007] Collecting two-photon excitation projection images of preset multiple sampling angles through a two-photon synthetic aperture microscope, and up-sampling the two-photon excitation projection images to obtain multi-pixel images;

[0008] A sparse-fRL (sparse-3D Richard-Lucy, sparse sampling three-dimensional deconvolution algorithm) three-dimensional reconstruction algorithm is used to reconstruct the multi-pixel image in phase space, and the reconstructed image is upsampled to obtain a high-resolution three-dimensional two-photon imaging result, wherein the multi-pixel image is upsampled and then the reconstruction error is estimated to suppress reconstruction artifacts.

[0009] Preferably, fixed frequency scanning is performed by a resonant galvanometer in a two-photon synthetic aperture microscope, and sparse sampling of a two-photon excitation projection image is performed by a linear galvanometer in the two-photon synthetic aperture microscope.

[0010] Preferably, the sparse sampling of the two-photon excitation projection image by the linear galvanometer in the two-photon synthetic aperture microscope further includes the following steps:

[0011] Dividing the physical distance between two adjacent pixels after downsampling equally according to the number of the preset multiple sampling angles;

[0012] The two-photon excitation projection images of each equally divided gap position are respectively collected through the preset multiple sampling angles, wherein each of the two corresponding adjacent pixels in different sampling angles is misaligned.

[0013] Preferably, the phase space reconstruction of the multi-pixel image using the sparse-fRL three-dimensional reconstruction algorithm further includes the following steps:

[0014] The data of the imaging system point spread function have the misalignment that is consistent with the projection image.

[0015] One or more embodiments of this specification provide a two-photon synthetic aperture microscopy imaging system, including an acquisition module and a reconstruction module;

[0016] The acquisition module is used to acquire two-photon excitation projection images of preset multiple sampling angles through a two-photon synthetic aperture microscope, and to upsample the two-photon excitation projection images to obtain multi-pixel images;

[0017] The reconstruction module is used to use a sparse-fRL three-dimensional reconstruction algorithm to perform phase space reconstruction on the multi-pixel image, and upsample the reconstructed image to obtain a high-resolution three-dimensional two-photon imaging result, wherein the reconstruction error is estimated after the multi-pixel image is upsampled to suppress reconstruction artifacts.

[0018] Preferably, fixed frequency scanning is performed by a resonant galvanometer in a two-photon synthetic aperture microscope, and sparse sampling of a two-photon excitation projection image is performed by a linear galvanometer in the two-photon synthetic aperture microscope.

[0019] Preferably, the acquisition module is further configured to equally divide the physical distance between two adjacent pixels after downsampling according to the number of the preset multiple sampling angles;

[0020] The two-photon excitation projection images of each equally divided gap position are respectively collected through the preset multiple sampling angles, wherein each of the two corresponding adjacent pixels in different sampling angles is misaligned.

[0021] Preferably, the reconstruction module is further configured so that the data of the imaging system point spread function have the misalignment that is consistent with the projection image.

[0022] One or more embodiments of the present specification provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned two-photon synthetic aperture microscopy imaging method when executing the computer program.

[0023] One or more embodiments of the present specification provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-mentioned two-photon synthetic aperture microscopy method.

[0024] The present invention provides a two-photon synthetic aperture microscopy imaging method, system, device and medium, which have the advantages of collecting two-photon excitation projection images of preset multiple sampling angles through a two-photon synthetic aperture microscope, up-sampling the two-photon excitation projection images to obtain a multi-pixel image, and encoding the high-frequency three-dimensional information of the sample in a manner of spatial angle information coupled acquisition during down-sampling imaging, thereby improving the down-sampling imaging performance; using a sparse-fRL three-dimensional reconstruction algorithm to perform phase space reconstruction on the multi-pixel image, and up-sampling the reconstructed image to obtain a high-resolution three-dimensional two-photon imaging result, wherein, after the multi-pixel image is up-sampled, reconstruction error estimation is performed to suppress reconstruction artifacts, thereby greatly improving the imaging speed of the two-photon synthetic aperture microscope without losing spatial resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 A schematic flow chart of a two-photon synthetic aperture microscopy imaging method provided for one or more embodiments of this specification;

[0027] Figure 2 A schematic diagram of an implementation process of a two-photon synthetic aperture microscopy imaging method provided in one or more embodiments of this specification;

[0028] Figure 3 A comparison diagram of sparse sampling high-resolution reconstruction results of a two-photon synthetic aperture microscope provided in one or more embodiments of this specification, Figure 3 (a) is a comparison of the sparse sampling high-resolution reconstruction results of the two-photon synthetic aperture microscope. Figure 3 (b) is a schematic diagram of the structural similarity index results comparing the statistical simulation results with the true value image;

[0029] Figure 4 A schematic diagram of the structure of a two-photon synthetic aperture microscopy imaging system provided for one or more embodiments of this specification;

[0030] Figure 5 A schematic diagram of the structure of a computer device provided for one or more embodiments of this specification. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the protection scope of the present invention.

[0032] The present invention is described in detail below in conjunction with specific implementation methods and the accompanying drawings.

[0033] Method Embodiment

[0034] According to an embodiment of the present invention, a two-photon synthetic aperture microscopy imaging method is provided, such as Figure 1 FIG. 1 is a flow chart of a two-photon synthetic aperture microscopy imaging method provided in this embodiment. A two-photon synthetic aperture microscopy imaging method according to an embodiment of the present invention comprises the following steps:

[0035] S110. Collect two-photon excitation projection images of preset multiple sampling perspectives through 2pSAM, and upsample the two-photon excitation projection images to obtain multi-pixel images. 2pSAM adopts multi-perspective image technology, usually 13 perspectives, to realize three-dimensional two-photon imaging, and uses "needle" scanning instead of the "point" scanning of traditional two-photon microscopes. "Needle" excitation in different directions can collect two-photon excitation projection images under preset multiple sampling perspectives.

[0036] S120, using a sparse-fRL three-dimensional reconstruction algorithm to perform phase space reconstruction on the multi-pixel image, and upsampling the reconstructed image to obtain a high-resolution three-dimensional two-photon imaging result, wherein the multi-pixel image is upsampled and then reconstruction error estimation is performed to suppress reconstruction artifacts. Figure 2As shown in the figure, X is the image coordinate when sparse sampling occurs, u is the projection angle, and the original projection image M(X) is directly used when calculating the reconstruction error Error. u ,u) divided by the forward projection P(X u ), and then restore the downsampled data to the upsampled state. The image coordinate that meets the Nyquist sampling rate is x, and the final reconstruction result is a high-resolution g(x,z) three-dimensional image.

[0037] The method provided in this embodiment collects two-photon excitation projection images of preset multiple sampling angles through a two-photon synthetic aperture microscope, and upsamples the two-photon excitation projection images to obtain a multi-pixel image. The method can encode the high-frequency three-dimensional information of the sample in a manner of spatial angle information coupled acquisition during down-sampling imaging, thereby improving the imaging performance of the down-sampling; a sparse-fRL three-dimensional reconstruction algorithm is used to perform phase space reconstruction on the multi-pixel image, and the reconstructed image is upsampled to obtain a high-resolution three-dimensional two-photon imaging result, wherein the reconstruction error estimation is performed after the multi-pixel image is up-sampled to suppress reconstruction artifacts, thereby greatly improving the imaging speed of the two-photon synthetic aperture microscope without losing spatial resolution.

[0038] In one embodiment, the 2pSAM includes a dual-axis scanning system of a resonant galvanometer and a linear galvanometer, fixed frequency scanning is performed by the resonant galvanometer in the two-photon synthetic aperture microscope, and sparse sampling of the two-photon excitation projection image is performed by the linear galvanometer in the two-photon synthetic aperture microscope.

[0039] The 2pSAM used in this embodiment includes a dual-axis scanning system of a resonant galvanometer and a linear galvanometer. No downsampling operation is required in the fast axis (resonant galvanometer scanning) direction, and its scanning frequency is fixed at 8kHz; while sparse sampling can be performed in the slow axis (linear galvanometer scanning) direction. The fewer the number of lines scanned by the slow axis of a single frame image, the faster the image frame rate. Taking an image with a pixel number of 1024×1024 as an example, for 2pSAM, the image acquisition frame rate of this size is 15Hz; if the sampling is doubled, the image acquisition frame rate of 1024×512 is 30Hz; and so on, when the sampling is 8 times, the image acquisition frame rate of 1024×128 is 120Hz. According to the Nyquist sampling theorem, because the optimal spatial resolution of 2pSAM is about 500nm, in order to achieve diffraction-limited resolution imaging, the physical spacing between adjacent pixels should be 250nm. At this time, the image size corresponding to 1024×1024 pixels is 256μm×256μm.

[0040] The method provided in this embodiment can utilize the efficient information acquisition capability of the two-photon synthetic aperture microscopy imaging system to encode the high-frequency three-dimensional information of the sample and improve the light flux by performing sparse sampling in the linear galvanometer scanning direction.

[0041] In one embodiment, the sparse sampling of the two-photon excitation projection image by the linear galvanometer in the two-photon synthetic aperture microscope further includes the following steps:

[0042] The physical distance between two adjacent pixels after downsampling is evenly divided according to the number of preset sampling angles.

[0043] The two-photon excitation projection images of each equally divided gap position are collected respectively by presetting a plurality of sampling viewing angles, wherein each corresponding adjacent two pixels in different sampling viewing angles are misaligned.

[0044] like Figure 2 As shown, it is a schematic diagram of the implementation process of a two-photon synthetic aperture microscopy imaging method provided by one or more embodiments of the present specification. By controlling the position of the scanning galvanometer, images at different angles are spatially displaced at a submicron level during acquisition, which is mainly reflected in the scanning direction of the slow-axis linear galvanometer. Without sacrificing the size of the imaging field of view by downsampling by 8 times, the physical distance between two adjacent rows after downsampling can be equally divided into 13 parts. The images at 13 angles are sampled separately by spatial dislocation. The blank positions that are not collected due to sparse sampling are collected by different angle images. Finally, a three-dimensional image is reconstructed from the 13 angle images. As long as the downsampling rate is less than the number of angles (13), it can be ensured that the spatial information of all samples in the large image field of view will be scanned and collected successively by the angle images. The data of the imaging system point spread function have the said dislocation consistent with the said projection image. Each viewing angle image scans a row of gap positions, and each corresponding row in the preset multiple sampling viewing angle images has a small dislocation. When performing three-dimensional reconstruction, the corresponding PSF data also has a corresponding dislocation, which is consistent with the projection image.

[0045] The method provided in this embodiment optimizes the control process of sparse sampling. Specifically, in the process of acquiring angle projection images, the images of multiple sampling perspectives are preset when scanning, and the position of the linear galvanometer is controlled so that there is a spatial misalignment between the images of multiple sampling perspectives preset during acquisition. The misalignment is used to supplement the acquisition of three-dimensional information of the sample lost due to sparse sampling.

[0046] The technical effects of the method of the present application are as follows: Figure 3 As shown in , for two-photon imaging, downsampling will lead to serious image quality degradation. Figure 3(a) shows the comparison of sparse sampling high-resolution reconstruction results of 2pSAM. Due to the coupled acquisition of spatial angle information by 2pSAM, a large amount of image information can be restored by upsampling the projection image and then performing fRL 3D reconstruction. Using the sparse-fRL reconstruction method, the high-frequency details of the image are further restored. Finally, the sparse-fRL reconstruction method using staggered sparse scanning can restore the high-resolution image information lost after downsampling to the maximum extent. Figure 3 (b) shows the statistical results of multiple groups of simulation results and the structural similarity index results compared with the true value (satisfying the Nyquist sampling theorem) image.

[0047] System Example

[0048] According to an embodiment of the present invention, a two-photon synthetic aperture microscopy imaging system is provided, such as Figure 4 As shown, it is a structural diagram of a two-photon synthetic aperture microscopy imaging system provided in this embodiment. A two-photon synthetic aperture microscopy imaging system according to an embodiment of the present invention includes:

[0049] The acquisition module 410 is used to acquire two-photon excitation projection images of preset multiple sampling angles through a two-photon synthetic aperture microscope, and to up-sample the two-photon excitation projection images to obtain multi-pixel images.

[0050] The reconstruction module 420 is used to use the sparse-fRL three-dimensional reconstruction algorithm to perform phase space reconstruction on the multi-pixel image, and upsample the reconstructed image to obtain a high-resolution three-dimensional two-photon imaging result, wherein the reconstruction error is estimated after the multi-pixel image is upsampled to suppress reconstruction artifacts.

[0051] In the system provided in this embodiment, the acquisition module 410 acquires two-photon excitation projection images of preset multiple sampling angles through a two-photon synthetic aperture microscope, and upsamples the two-photon excitation projection images to obtain a multi-pixel image, which can encode the high-frequency three-dimensional information of the sample in a manner of spatial angle information coupled acquisition during down-sampling imaging, thereby improving the down-sampling imaging performance; the reconstruction module 420 performs phase space reconstruction on the multi-pixel image by adopting a sparse-fRL three-dimensional reconstruction algorithm, and upsamples the reconstructed image to obtain a high-resolution three-dimensional two-photon imaging result, wherein the multi-pixel image is upsampled and then a reconstruction error estimation is performed to suppress reconstruction artifacts, and the image is upsampled after the error estimation step. This method can reduce error amplification, thereby suppressing artifacts in the reconstruction result, and greatly improves the imaging speed of the two-photon synthetic aperture microscope without losing spatial resolution.

[0052] In one embodiment, the 2pSAM includes a dual-axis scanning system of a resonant galvanometer and a linear galvanometer, fixed frequency scanning is performed by the resonant galvanometer in the two-photon synthetic aperture microscope, and sparse sampling of the two-photon excitation projection image is performed by the linear galvanometer in the two-photon synthetic aperture microscope.

[0053] The system provided in this embodiment can utilize the efficient information acquisition capability of the two-photon synthetic aperture microscopy imaging system to encode the high-frequency three-dimensional information of the sample and improve the light flux by performing sparse sampling in the linear galvanometer scanning direction.

[0054] In one embodiment, the acquisition module 410 is further configured to equally divide the physical distance between two adjacent pixels after downsampling according to a number of preset sampling angles.

[0055] The two-photon excitation projection images of each equally divided gap position are collected respectively by presetting a plurality of sampling viewing angles, wherein each corresponding adjacent two pixels in different sampling viewing angles are misaligned.

[0056] The reconstruction module 420 is further configured to ensure that the data of the imaging system point spread function have the misalignment that is consistent with the projection image.

[0057] The system provided in this embodiment optimizes the control process of sparse sampling. Specifically, in the process of acquiring angle projection images, the images of multiple sampling perspectives are preset when scanning, and the position of the linear galvanometer is controlled so that there is a spatial misalignment between the images of multiple sampling perspectives preset during acquisition. The misalignment is used to supplement the acquisition of three-dimensional information of the sample lost due to sparse sampling.

[0058] The embodiment of the present invention is a system embodiment corresponding to the above-mentioned method embodiment. The specific operations of the processing steps of each module can be understood by referring to the description of the method embodiment, which will not be repeated here.

[0059] like Figure 5 As shown, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which implements the two-photon synthetic aperture microscopy imaging method in the above-mentioned embodiment when the computer program is executed by a processor, or implements the two-photon synthetic aperture microscopy imaging method in the above-mentioned embodiment when the computer program is executed by a processor.

[0060] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0061] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and the contents not described in detail in the specification of the present invention belong to the common knowledge of those skilled in the art.

Claims

1. A two-photon synthetic aperture microscopy imaging method, characterized in that: The following steps are involved: Collecting two-photon excitation projection images of preset multiple sampling angles through a two-photon synthetic aperture microscope, and up-sampling the two-photon excitation projection images to obtain multi-pixel images; A sparse-fRL three-dimensional reconstruction algorithm is used to perform phase space reconstruction on the multi-pixel image, and the reconstructed image is upsampled to obtain a high-resolution three-dimensional two-photon imaging result, wherein the multi-pixel image is upsampled and then a reconstruction error is estimated to suppress reconstruction artifacts.

2. The two-photon synthetic aperture microscopy imaging method according to claim 1, characterized in that: Fixed frequency scanning is performed by a resonant galvanometer in a two-photon synthetic aperture microscope, and sparse sampling of the two-photon excitation projection image is performed by a linear galvanometer in a two-photon synthetic aperture microscope.

3. The two-photon synthetic aperture microscopy imaging method according to claim 2, characterized in that: The sparse sampling of the two-photon excitation projection image by the linear galvanometer in the two-photon synthetic aperture microscope also includes the following steps: Evenly dividing the physical distance between two adjacent pixels after downsampling according to the preset multiple sampling angles; The two-photon excitation projection images of each equally divided gap position are respectively collected through the preset multiple sampling viewing angles, wherein each corresponding two adjacent pixels in the projection images collected at different viewing angles are misaligned.

4. The two-photon synthetic aperture microscopy imaging method according to claim 3, characterized in that: The phase space reconstruction of the multi-pixel image using the sparse-fRL three-dimensional reconstruction algorithm also includes the following steps: The data of the imaging system point spread function have the misalignment that is consistent with the projection image.

5. A two-photon synthetic aperture microscopy system, characterized in that It includes an acquisition module and a reconstruction module; The acquisition module is used to acquire two-photon excitation projection images of preset multiple sampling angles through a two-photon synthetic aperture microscope, and to upsample the two-photon excitation projection images to obtain multi-pixel images; The reconstruction module is used to use a sparse-fRL three-dimensional reconstruction algorithm to perform phase space reconstruction on the multi-pixel image, and upsample the reconstructed image to obtain a high-resolution three-dimensional two-photon imaging result, wherein the reconstruction error is estimated after the multi-pixel image is upsampled to suppress reconstruction artifacts.

6. The two-photon synthetic aperture microscopy imaging system according to claim 5, characterized in that: Fixed frequency scanning is performed by a resonant galvanometer in a two-photon synthetic aperture microscope, and sparse sampling of the two-photon excitation projection image is performed by a linear galvanometer in a two-photon synthetic aperture microscope.

7. The two-photon synthetic aperture microscopy imaging system according to claim 6, characterized in that: The acquisition module is further configured to equally divide the physical distance between two adjacent pixels after downsampling according to the number of the preset multiple sampling angles; The two-photon excitation projection images of each equally divided gap position are respectively collected through the preset multiple sampling angles, wherein each of the two corresponding adjacent pixels in different sampling angles is misaligned.

8. The two-photon synthetic aperture microscopy imaging system according to claim 7, characterized in that: The reconstruction module is further configured to ensure that the data of the point spread function of the imaging system have the misalignment that is consistent with the projection image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the two-photon synthetic aperture microscopy imaging method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the two-photon synthetic aperture microscopy imaging method according to any one of claims 1 to 4 are implemented.

Images