Super-resolution tomography method, system, terminal and medium based on address scanning

Abstract

The present invention provides a method, system, terminal, and medium for super-resolution tomography based on address scanning, specifically relating to the field of optical microscopy technology. This solution generates a series of multi-focus two-dimensional scanning dot matrices based on a wide-field fluorescence image of a target sample; scans the target sample using the multi-focus two-dimensional scanning dot matrices, and synchronously acquires fluorescence signals emitted by the target sample to obtain a series of two-dimensional dot matrix fluorescence images; based on the structural characteristics of the target sample, axially slices the target sample by moving the focal plane, and each time the focal plane is moved, the step of acquiring a series of two-dimensional dot matrix fluorescence images is repeated until a preset slice coordinate range threshold is reached; and using a preset reconstruction algorithm, the two-dimensional dot matrix fluorescence image is reconstructed according to the focal plane to obtain a three-dimensional super-resolution image of each region of interest. This solution can perform two-dimensional address scanning of the sample layer by layer, achieving rapid three-dimensional tomographic super-resolution imaging without being limited by depth of field.

Description

Technical Field

[0001] The present invention relates to the field of optical microscopic imaging technology, and in particular to a super-resolution tomography method, system, terminal and medium based on addressing scanning. Background Art

[0002] Optical microscopy enables real-time, non-destructive, and non-contact observation of cells at the nanoscale, and is one of the most basic and important research tools in the fields of life sciences and biomedicine. Because structured illumination microscopy's super-resolution approach offers fast imaging speeds and has no special requirements for fluorescent probes or excitation light energy density, it has a natural advantage in live cell super-resolution imaging applications. In the development of structured illumination microscopy super-resolution imaging, the earliest technique to appear was SIM (Structured Illumination Microscopy), which uses cosine fringe structured light to illuminate samples for super-resolution imaging. This technique, compared to conventional widefield imaging, can achieve twice the resolution. However, the excitation light energy density under this illumination mechanism is low and cannot penetrate the sample surface, resulting in poor imaging depth and inability to perform three-dimensional imaging of thicker samples.

[0003] The subsequent emergence of image scanning microscopy (ISM) combined traditional confocal microscopy with enhanced tomographic capabilities while maintaining the resolution enhancements of SIM, enabling greater imaging depth. However, ISM's single-point scanning method severely limited imaging speed. Building on this, multifocal structured illumination microscopy (MSIM) emerged, significantly improving ISM imaging speed by scanning samples at multiple focal points using parallel excitation. However, when imaging thick samples, sample scattering and out-of-focus background significantly impact the imaging results. Two-photon microscopy offers excellent optical tomography capabilities and the advantages of deep imaging. Two-photon multifocal structured illumination microscopy (2P-MSIM), which combines two-photon excitation with fluorescence, further enhances imaging performance by reducing the impact of sample scattering and out-of-focus background on imaging. Despite this, practical applications of MSIM and 2P-MSIM still face the trade-off between imaging depth and three-dimensional imaging speed. Summary of the Invention

[0004] In view of the above-mentioned deficiencies in the prior art, the object of the present invention is to provide a super-resolution tomography method, system, terminal and medium based on address scanning, aiming to solve the problem of mutual restriction between imaging depth and three-dimensional imaging speed in the prior art three-dimensional super-resolution imaging technology.

[0005] To achieve the above objectives, the present invention provides, in a first aspect, a super-resolution tomography method based on address scanning, which is applied to a super-resolution tomography device. The device mainly includes a laser, an acousto-optic deflector, an electric focus lens, and a microscope objective lens, which are arranged in sequence along the optical path. The method mainly includes the following steps:

[0006] Acquiring a wide-field fluorescence image of the target sample, and selecting at least one region of interest based on the wide-field fluorescence image;

[0007] Based on the size and position coordinate information of the circumscribed rectangle of the selected region of interest, as well as the preset scanning point number information and scanning step length information, a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest are generated;

[0008] Based on a mapping relationship between a preset transverse coordinate and an acoustic wave frequency of an acousto-optic deflector, the target sample is scanned according to the multi-focus two-dimensional scanning dot matrix, and fluorescence signals emitted by the target sample are synchronously collected to obtain a series of two-dimensional dot matrix fluorescence images;

[0009] Based on the structural features of the target sample in the wide-field fluorescence image and the mapping relationship between the preset axial coordinates and the electrical signal of the electric focusing lens, the electric focusing lens is controlled to move the focal plane to perform axial slicing of the target sample, and each time the focal plane is moved, the step of acquiring a series of two-dimensional dot matrix fluorescence images is repeated until a preset slicing coordinate range threshold is reached;

[0010] The two-dimensional dot matrix fluorescence image is reconstructed according to the focal plane using a preset reconstruction algorithm to obtain a three-dimensional super-resolution image of each region of interest.

[0011] Optionally, acquiring a wide-field fluorescence image of the target sample includes:

[0012] Adjusting the microscope objective lens to a focal plane, and controlling the acousto-optic deflector to scan the target sample according to each of the preset full-field scanning multi-focus two-dimensional scanning lattice coordinates and a mapping relationship between the preset lateral coordinates and the acoustic wave frequency of the acousto-optic deflector, thereby obtaining a series of two-dimensional lattice fluorescence images;

[0013] The two-dimensional dot matrix fluorescence images are superimposed to obtain a wide-field fluorescence image of the target sample.

[0014] Optionally, the step of generating a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest based on the size and position coordinate information of the circumscribed rectangle of the selected region of interest, as well as preset scanning point number information and scanning step length information, includes:

[0015] Obtaining, based on the size and position coordinate information of the selected bounding rectangle of the region of interest, the X and Y coordinate information of the starting point of the bounding rectangle of the region of interest, as well as the width information along the X direction and the width information along the Y direction;

[0016] Based on the X and Y coordinate information of the starting point, the width information along the X direction, the width information along the Y direction, and the preset number of scanning points and scanning step information in the X direction, a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest are obtained.

[0017] Optionally, based on the X and Y coordinate information of the starting point, the width information along the X direction, the width information along the Y direction, and the preset number of scanning points and scanning step length information in the X direction, a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest are generated, including:

[0018] Divide the width information along the X direction by the preset number of scanning points in the X direction and round down to obtain the point spacing;

[0019] Divide the width information along the Y direction by the point spacing and round up to obtain the scanning point number information in the Y direction;

[0020] Based on the X and Y coordinate information of the starting point and the preset number of scanning points in the X direction and the number of scanning points in the Y direction, a multi-focus two-dimensional scanning dot matrix is ​​obtained;

[0021] Based on the scanning step information, the X and Y coordinate information of the starting point are adjusted to obtain a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest.

[0022] Optionally, scanning the target sample according to the multi-focus two-dimensional scanning dot matrix and synchronously collecting the fluorescence signal emitted by the target sample to obtain a series of two-dimensional dot matrix fluorescence images include:

[0023] Scanning the target sample according to the multi-focus two-dimensional scanning dot matrix, and after each scan of the multi-focus two-dimensional scanning dot matrix, synchronously collecting the fluorescence signal emitted by the target sample corresponding to the most recently scanned multi-focus two-dimensional scanning dot matrix to obtain a two-dimensional dot matrix fluorescence image;

[0024] The step of acquiring the two-dimensional dot matrix fluorescence image is repeated until all the multi-focus two-dimensional scanning dot matrices on the current focal plane are scanned, thereby obtaining a series of two-dimensional dot matrix fluorescence images.

[0025] Optionally, based on the structural features of the target sample in the wide-field fluorescence image and the mapping relationship between the preset axial coordinates and the electrical signal of the electric focus lens, the electric focus lens is controlled to move the focal plane to perform axial slicing on the target sample, and each time the focal plane is moved, the step of acquiring a series of two-dimensional dot matrix fluorescence images is repeated until a preset slicing coordinate range threshold is reached, including:

[0026] Based on the structural characteristics of the target sample in the wide-field fluorescence image, setting the axial slice coordinate range threshold and inter-layer spacing information;

[0027] Based on the inter-layer spacing information and a mapping relationship between a preset axial coordinate and an electrical signal of the electric focus lens, the electric focus lens is controlled to move by one inter-layer spacing each time, so as to move the focal plane once;

[0028] Each time the focal plane is moved, an axial slice is performed on the target sample, and the step of acquiring a series of two-dimensional dot matrix fluorescence images is repeated until a preset slice coordinate range threshold is reached.

[0029] Optionally, the reconstructing the two-dimensional dot matrix fluorescence image according to the focal plane using a preset reconstruction algorithm to obtain a three-dimensional super-resolution image of each region of interest includes:

[0030] Using a preset reconstruction algorithm, super-reconstruct the two-dimensional dot matrix fluorescence image of each focal plane of the target sample to obtain a two-dimensional super-resolved reconstructed image;

[0031] The two-dimensional super-resolution reconstructed images of different focal planes are three-dimensionally reconstructed to obtain a three-dimensional super-resolution image of each region of interest.

[0032] A second aspect of the present invention provides a super-resolution tomography system based on address scanning, the system comprising:

[0033] a region selection module, configured to acquire a wide-field fluorescence image of a target sample and select at least one region of interest based on the wide-field fluorescence image;

[0034] a two-dimensional scanning dot matrix generation module, configured to generate a series of multi-focus two-dimensional scanning dot matrices corresponding to the region of interest based on the size and position coordinate information of the circumscribed rectangle of the selected region of interest, as well as preset scanning point number information and scanning step length information;

[0035] a two-dimensional dot matrix fluorescence image acquisition module, configured to scan the target sample according to the multi-focus two-dimensional scanning dot matrix based on a mapping relationship between a preset transverse coordinate and the acoustic wave frequency of the acousto-optic deflector, and synchronously collect fluorescence signals emitted by the target sample to obtain a series of two-dimensional dot matrix fluorescence images;

[0036] an axial sectioning module, configured to control the electric focus lens to move the focal plane to perform axial sectioning on the target sample based on the structural features of the target sample in the wide-field fluorescence image and a mapping relationship between the preset axial coordinates and the electrical signals of the electric focus lens, and to repeat the step of acquiring a series of two-dimensional dot matrix fluorescence images each time the focal plane is moved until a preset section coordinate range threshold is reached;

[0037] The three-dimensional super-resolution image generation module is used to reconstruct the two-dimensional dot matrix fluorescence image according to the focal plane using a preset reconstruction algorithm to obtain a three-dimensional super-resolution image of each region of interest.

[0038] A third aspect of the present invention provides an intelligent terminal, comprising a memory, a processor, and an address scanning-based super-resolution tomography program stored in the memory and executable on the processor, wherein the address scanning-based super-resolution tomography program, when executed by the processor, implements any one of the steps of the above-mentioned address scanning-based super-resolution tomography method.

[0039] A fourth aspect of the present invention provides a computer-readable storage medium, on which a super-resolution tomography program based on address scanning is stored. When the super-resolution tomography program based on address scanning is executed by a processor, any one of the steps of the above-mentioned super-resolution tomography method based on address scanning is implemented.

[0040] Compared with the existing technology, the beneficial effects of this solution are as follows:

[0041] The present invention selects at least one region of interest in a wide-field fluorescence image of a target sample, and generates a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest based on the size and position coordinate information of the circumscribed rectangle of the selected region of interest, as well as preset scanning point number information and scanning step length information; based on a mapping relationship between preset lateral coordinates and the acoustic wave frequency of an acousto-optic deflector, synchronously scans the multi-focus two-dimensional scanning dot array and collects corresponding fluorescence signals to obtain a series of two-dimensional dot array fluorescence images on the same focal plane; and based on the structural characteristics of the target sample in the wide-field fluorescence image and a mapping relationship between preset axial coordinates and the electrical signal of an electrically adjustable focus lens, axially slices the target sample by moving the focal plane, and repeats the step of acquiring a series of two-dimensional dot array fluorescence images each time the focal plane is moved, so as to facilitate the acquisition of two-dimensional dot array fluorescence images of the target sample at different focal planes; then, using a preset reconstruction algorithm, the two-dimensional dot array fluorescence image is reconstructed according to the focal plane to obtain a three-dimensional super-resolution image of each region of interest.

[0042] Because both the acousto-optic deflector and the electric focusing lens have fast response characteristics, and the electric focusing lens' response rate is an order of magnitude slower than that of the acousto-optic deflector, combining the two for addressing and scanning enables two-dimensional addressing and scanning of the sample layer by layer, enabling rapid three-dimensional tomographic super-resolution imaging without being limited by depth of field. This invention effectively solves the existing problem of the mutual constraints between imaging depth and three-dimensional imaging speed in super-resolution dynamic imaging of samples, enabling rapid tomographic imaging of any region of interest. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 This is a flow chart of the super-resolution tomography method based on address scanning of the present invention;

[0045] Figure 2 Schematic diagram of synchronous control signals of the control program of the present invention for an electrically tunable lens (ETL), an sCMOS camera, and a two-dimensional acousto-optic deflector (2D-AOD);

[0046] Figure 3 (a) is a schematic diagram of a transverse scanning of the acousto-optic deflector (AOD) of the present invention;

[0047] Figure 3 (b) is a schematic diagram of the axial scanning of the electrically tunable lens (ETL) of the present invention;

[0048] Figure 3 (c) is a schematic diagram of the addressing scanning layer-cut imaging of the acousto-optic deflector and electrically tunable lens (i.e., AOD and ETL) of the present invention;

[0049] Figure 4 (a) is a schematic diagram of the selected area and layer cut of the present invention;

[0050] Figure 4 (b) is a schematic diagram of the dot matrix scanning method corresponding to a single exposure of the present invention;

[0051] Figure 4 (c) is a schematic diagram of a sequence of dot matrix fluorescence images obtained by multiple exposures of the present invention;

[0052] Figure 5 (a) is a schematic diagram of a single-layer wide-field fluorescence image of the present invention;

[0053] Figure 5 (b) is a schematic diagram of a single-layer two-dimensional super-resolution image of the present invention;

[0054] Figure 5 (c) is a schematic diagram of a multi-layer two-dimensional super-resolution image and a three-dimensional reconstructed image according to the present invention;

[0055] Figure 6 Schematic diagram of the structure of the super-resolution tomography system based on address scanning of the present invention;

[0056] Figure 7 It is a schematic diagram of the structure of the intelligent terminal of the present invention. DETAILED DESCRIPTION

[0057] In the following description, specific details such as particular system structures and techniques are provided for purposes of illustration, not limitation, to facilitate a thorough understanding of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

[0058] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0059] It should also be understood that the terms used in the present specification are only for the purpose of describing particular embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0060] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0061] The following is a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0063] The present invention addresses the problem of mutual constraints between imaging depth and three-dimensional imaging speed in existing super-resolution imaging systems. A super-resolution tomography method based on addressing scanning is proposed. This method first utilizes a femtosecond laser and a two-dimensional acousto-optic deflector (2D-AOD) for rapid multi-focus scanning imaging and generates a wide-field image of the sample. On this basis, one or more regions of interest are selected. The acoustic wave frequency of the acousto-optic deflector (AOD) is controlled to generate a two-dimensional dot matrix corresponding to the selected region for rapid MSIM imaging. Rapid switching of the focal plane is achieved by controlling the electrical signal of the electrically tunable focusing lens (ETL), thereby rapidly obtaining a three-dimensional super-resolution image of the sample. This method can address the problem of mutual constraints between imaging depth and three-dimensional imaging speed in super-resolution dynamic imaging of living biological samples. It aims to provide a technology for life science research that can perform rapid MSIM super-resolution tomography on any region of interest, enabling super-resolution imaging monitoring of certain specific structures within cells.

[0064] Exemplary Methods

[0065] The embodiment of the present invention provides a super-resolution tomography method based on address scanning, which is deployed on electronic devices such as computers and servers. The application scenario is a super-resolution tomography device, which mainly includes a laser, an acousto-optic deflector, an electric focus lens, a microscope objective lens, etc. arranged in sequence according to the direction of the optical path. This embodiment proposes a super-resolution tomography method based on address scanning (three Dimensional Optical Sectioning Multifocal Structured Illumination Microscopy, 3D-OS-MSIM), specifically, as shown in FIG. Figure 1 As shown, it mainly includes the following steps:

[0066] Step S100: Acquire a wide-field fluorescence image of a target sample, and select at least one region of interest based on the wide-field fluorescence image.

[0067] Specifically, obtaining a wide-field fluorescence image of a target sample includes: adjusting the microscope objective to the focal plane, controlling the acousto-optic deflector to scan the target sample according to each preset full-field scanning multi-focus two-dimensional scanning lattice coordinate based on the preset full-field scanning multi-focus two-dimensional scanning lattice coordinates and the mapping relationship between the lateral coordinates and the acoustic wave frequency of the acousto-optic deflector, and obtaining a series of two-dimensional lattice fluorescence images that correspond one-to-one to the preset full-field scanning multi-focus two-dimensional scanning lattice coordinates; and superimposing the two-dimensional lattice fluorescence images to obtain a wide-field fluorescence image of the target sample.

[0068] In one embodiment, the electric focus lens is not turned on and the microscope objective lens is adjusted to the focal plane of the sample. A control program is written in LabView to automatically superimpose the two-dimensional dot matrix fluorescence images of a series of two-dimensional scanning dot matrices to obtain a wide-field fluorescence image of the target sample with a large field of view. One or more arbitrary regions of interest can be manually selected on the wide-field fluorescence image displayed on the control program interface. The region shape can be selected as a default rectangle or an arbitrary shape. In order to simplify the difficulty of subsequent super-resolution reconstruction of the region of interest and improve processing efficiency, this embodiment automatically adds a bounding rectangle to the selected rectangular or other arbitrary shaped region of interest during actual operation, and scans and slices the bounding rectangle area containing the region of interest. The control program written in LabView is used to automatically save the pixel coordinate information of the selected region of interest and its bounding rectangle.

[0069] It should be noted that LabView is selected to write the control program in this embodiment. As other preferred implementations, other existing graphic software development platforms can also be selected to write the control program according to actual applications.

[0070] Step S200: Based on the size and position coordinate information of the circumscribed rectangle of the selected region of interest, as well as the preset scanning point number information and scanning step length information, a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest are generated.

[0071] Specifically, based on the size and position coordinate information of the circumscribed rectangle of the selected region of interest, the X- and Y-direction coordinate information of the starting point of the circumscribed rectangle of the region of interest, as well as the width information Lx along the X direction and the width information Ly along the Y direction are obtained; the number of scanning points Nx and the scanning step length M in the X direction are set based on experience; and based on the X- and Y-direction coordinate information of the starting point, the width information Lx along the X direction and the width information Ly along the Y direction, as well as the preset number of scanning points Nx and the scanning step length M in the X direction, a series of multi-focus two-dimensional scanning dot arrays are generated.

[0072] Based on the X and Y coordinate information of the starting point, the width information Lx along the X direction and the width information Ly along the Y direction, as well as the preset number of scanning points Nx and scanning step length M in the X direction, a series of multi-focus two-dimensional scanning dot arrays are generated, including:

[0073] The width information Lx along the X direction is divided by the preset number of scanning points in the X direction Nx and rounded down to obtain the point spacing D. The width information Ly along the Y direction is divided by the point spacing D and rounded up to obtain the number of scanning points in the Y direction Ny. Based on the X and Y coordinate information of the starting point, the preset number of scanning points in the X direction Nx, and the obtained number of scanning points in the Y direction Ny, a multi-focus two-dimensional scanning dot matrix consisting of Nx×Ny scanning points is generated to obtain the coordinates of each point in the two-dimensional scanning dot matrix. Based on the preset scanning step information M, the X and Y coordinate information of the starting point is adjusted to generate the next multi-focus two-dimensional scanning dot matrix. Based on the preset scanning step information M, the X and Y coordinate information of the starting point is continuously adjusted until a series of multi-focus two-dimensional scanning dot matrices that meet the MSIM super-resolution imaging requirements are generated. It is easy to understand that the width information, scanning point number information, scanning step information, and point spacing in this embodiment are all positive integers. Furthermore, the X direction and Y direction referred to in this embodiment represent two mutually orthogonal lateral directions parallel to the focal plane, corresponding to the deflection directions of the laser beam by two orthogonal acousto-optic deflectors, respectively, and the opposite Z direction represents the axial direction perpendicular to the focal plane.

[0074] In one embodiment, a control program written in LabView automatically generates the multi-focus two-dimensional scanning dot matrix required for MSIM imaging by reading the X and Y coordinate information of the starting point of the circumscribed rectangular area of ​​the region of interest, the width information Lx and Ly of the rectangular area along the X direction, the preset number of scanning points Nx in the X direction, and the preset scanning step size M. This generates a coordinate sequence of the multi-focus two-dimensional scanning dot matrix corresponding to the same focal plane. It is easy to understand that the coordinate sequence of the multi-focus two-dimensional scanning dot matrix here includes the order in which the scanning points are subsequently scanned, and the scanning order can be flexibly set according to actual needs.

[0075] Step S300: Based on the preset mapping relationship between the lateral coordinate and the acoustic wave frequency of the acousto-optic deflector, the target sample is scanned according to the multi-focus two-dimensional scanning dot matrix, and the fluorescence signal emitted by the target sample is synchronously collected to obtain a series of two-dimensional dot matrix fluorescence images.

[0076] Specifically, based on the preset mapping relationship between the lateral coordinates and the acoustic wave frequency of the acousto-optic deflector, the coordinate sequence of each two-dimensional scanning dot matrix is ​​converted into an acoustic wave frequency to drive a pair of orthogonal acousto-optic deflectors to scan the target sample according to each multi-focus two-dimensional scanning dot matrix; each time a multi-focus two-dimensional scanning dot matrix is ​​scanned, the image detector is controlled to synchronously collect the fluorescence signal emitted by the target sample corresponding to the multi-focus two-dimensional scanning dot matrix scanned most recently, and a two-dimensional dot matrix fluorescence image is obtained; the steps of obtaining the two-dimensional dot matrix fluorescence image are repeated until all the multi-focus two-dimensional scanning dot matrices on the current focal plane are scanned, and a series of two-dimensional dot matrix fluorescence images corresponding one to one to the multi-focus two-dimensional scanning dot matrices are obtained.

[0077] In one embodiment, in a control program written in LabView, the coordinate sequence of each two-dimensional scanning dot matrix is ​​converted into an acoustic wave frequency by mapping the calibrated lateral coordinates to the acoustic wave frequency of the acousto-optic deflector, and the data acquisition card is controlled to output a 32-bit digital signal corresponding to the acoustic wave frequency to drive a pair of orthogonal acousto-optic deflectors (2D-AOD) to scan the sample according to the multi-focus two-dimensional scanning dot matrix coordinate sequence; at the same time, the image detector (such as an sCMOS camera, etc.) is controlled by the analog signal output by the control data acquisition card to synchronously collect the fluorescence signal emitted by the target sample corresponding to each two-dimensional scanning dot matrix, that is, each time a pair of orthogonal acousto-optic deflectors scans a dot matrix, the image detector also completes an exposure (generally 5-10ms to avoid overexposure), thereby capturing a series of two-dimensional dot matrix fluorescence images corresponding to the multi-focus two-dimensional scanning dot matrix. It should be noted that in this embodiment, the data acquisition card outputs a 32-bit digital signal. As other preferred embodiments, other suitable digital signals can also be selected according to actual application requirements to regulate the acoustic wave frequency loaded on the acousto-optic deflector.

[0078] In the simulation experiment, the control program controls the synchronous control signals of ETL, sCMOS camera and 2D-AOD as follows: Figure 2 As shown, after the pulse laser passes through the 2D-AOD, it performs point excitation on the target sample. Each point excitation corresponds to a pulse of the 2D-AOD synchronization control signal. Each Nx*Ny (Nx and Ny are both positive integers) pulses constitute a pulse sequence. Each pulse sequence corresponds to an exposure of the sCMOS, and a two-dimensional dot matrix fluorescence image corresponding to the focal plane is obtained. When the electrical signal on the ETL remains unchanged, that is, at the same focal plane, a total of k (k is a positive integer) exposures occur, generating k two-dimensional dot matrix fluorescence images corresponding to the focal plane, thereby realizing MSIM two-dimensional super-resolution imaging of the focal plane.

[0079] Step S400: Based on the structural features of the target sample in the wide-field fluorescence image and the mapping relationship between the preset axial coordinates and the electrical signal of the electric focusing lens, the electric focusing lens is controlled to move the focal plane to perform axial sectioning on the target sample, and step S300 is repeated each time the focal plane is moved.

[0080] Specifically, based on the structural features of the target sample in the widefield fluorescence image, an axial (i.e., Z-direction) slice coordinate range threshold and slice spacing information are set. Based on the axial slice coordinate range threshold and slice spacing information, as well as the preset mapping relationship between the axial coordinate and the electrical signal of the electric focus lens, the electric focus lens is controlled to move one slice spacing at a time to move the focal plane once. Each time the focal plane moves, an axial slice is performed on the target sample, and step S300 is repeated until the preset slice coordinate range threshold is reached. The electrical signal of the electric focus lens in this embodiment refers to the current and / or voltage value applied to the electric focus lens.

[0081] In one embodiment, the electric focusing lens is turned on, and a control program written in LabView is used to set the axial slice coordinate range threshold and layer spacing information based on experience according to the sample structural characteristics in the wide-field image of the selected area of ​​interest. The control program generates a series of electrical signals corresponding to the axial positions of different focal planes to control the electric focusing lens to move the focal plane.

[0082] The following further describes the layer cutting principle of the super-resolution tomography method based on address scanning of the present invention. The principle of using the control program to control AOD and ETL to achieve address scanning tomography is as follows: Figure 3 As shown. Among them, Figure 3 (a) Schematic diagram of AOD transverse scanning. The pulsed laser passes through the acousto-optic deflector (AOD) located on the conjugate plane of the microscope objective lens, along the optical path, and then illuminates the sample plane (Sample plane) to achieve addressable scanning of single-point excitation. Figure 3 (b) Schematic diagram of the axial scanning of the ETL, showing the change in the distance between the excited focal plane in the sample and the microscope objective before and after the focal plane is moved; Figure 3 (c) is a schematic diagram of the AOD+ETL addressing scanning layer section imaging. Since the response rate of ETL is one order of magnitude slower than that of AOD, after combining with the AOD addressing scanning, the AOD can scan and obtain several two-dimensional dot matrices at the same focal plane of the ETL (Layer 1 or Layer 2). In this way, by changing the focal length of the ETL and performing two-dimensional addressing scanning at each focal plane of the sample, a series of two-dimensional dot matrix fluorescence images can be obtained.

[0083] In the simulation experiment, the schematic diagram of the generation, scanning and slice imaging of the multi-focus two-dimensional dot array of any area of ​​interest is shown in the figure. Figure 4 As shown. Among them, Figure 4 (a) shows the schematic diagram of the selected area and layer cutting. Figure 4 (b) shows a schematic diagram of the dot matrix scanning method corresponding to a single exposure. Figure 4 (c) is a schematic diagram of a sequence of two-dimensional dot matrix fluorescence images obtained through multiple exposures.

[0084] Step S500: reconstructing the two-dimensional dot matrix fluorescence image according to the focal plane using a preset reconstruction algorithm to obtain a three-dimensional super-resolution image of each region of interest.

[0085] Specifically, a two-dimensional super-resolution reconstruction algorithm is used to reconstruct all two-dimensional dot matrix fluorescence images of the target sample on the same focal plane to obtain a two-dimensional super-resolution image; based on the mapping relationship between the preset axial coordinates and the electrical signal of the electric focusing lens, all two-dimensional super-resolution images on a series of different focal planes are reconstructed into three dimensions to obtain a three-dimensional super-resolution image of each area of ​​interest.

[0086] In one embodiment, a control program written in LabView is used to first perform two-dimensional super-resolution reconstruction on a series of multi-focus two-dimensional dot matrix fluorescence images corresponding to the same plane of the target sample using an MSIM reconstruction algorithm (such as the MSBL algorithm) to generate a single-layer two-dimensional super-resolution image. Then, a three-dimensional reconstruction is performed on the series of single-layer two-dimensional super-resolution images to obtain a three-dimensional super-resolution image corresponding to the selected region of interest. The schematic diagram of the reconstructed image is shown in FIG. Figure 5 As shown, Figure 5 (a) shows a schematic diagram of a single-layer wide-field fluorescence image. Figure 5 (b) shows a schematic diagram of a single-layer two-dimensional super-resolution image. Figure 5 (c) shows a schematic diagram of multi-layer two-dimensional super-resolution images and three-dimensional reconstructed images.

[0087] In summary, the method of the present embodiment first controls the data acquisition card to output digital signals and analog signals without turning on the electric focus lens, wherein the digital signal controls the acousto-optic deflector to generate a lattice corresponding to the full-field scan and quickly scans, and the analog signal controls the image detector to synchronously collect the fluorescence image corresponding to each scanning lattice, and imports the wide-field image superimposed by the collected series of fluorescence images into a control program (written in LabView) for selecting a single or multiple regions of interest in the image for MSIM super-resolution imaging; then, when the electric focus lens is turned on, the control program determines the two-dimensional coordinate information of each point in each scanning lattice according to the first pixel coordinate information, the width of the circumscribed rectangle and the lattice parameter information of the selected region of interest, and calculates the two-dimensional coordinate information of each point in the scanning lattice according to the calibrated lateral coordinate and the acoustic wave frequency. A mapping relationship is established to control the data acquisition card to generate and output corresponding digital and analog signals, wherein the sound wave frequency corresponding to the digital signal regulates the acousto-optic deflector to scan the selected region of interest according to the coordinate information of the two-dimensional scanning lattice; one analog signal controls the image detector to synchronously collect the fluorescence image corresponding to each two-dimensional scanning lattice, which is used to reconstruct a single-layer super-resolution image; the other analog signal controls the current or voltage of the electric focusing lens to achieve axial layer cutting; finally, a two-dimensional super-resolution reconstruction algorithm is used to reconstruct all multi-focus two-dimensional lattice fluorescence images on the same plane of the target sample respectively, and obtain single-layer two-dimensional super-resolution images corresponding to each plane of the target sample, and then all single-layer two-dimensional super-resolution images are reconstructed into three dimensions, thereby obtaining a three-dimensional super-resolution image corresponding to the selected region of interest in the sample.

[0088] Since both the acousto-optic deflector and the electric focusing lens have the characteristics of fast response, and the response rate of the electric focusing lens is one order of magnitude slower than that of the acousto-optic deflector, this solution can realize two-dimensional addressing scanning of the sample layer by layer after combining it with the addressing scanning of the acousto-optic deflector. It can achieve fast three-dimensional tomographic super-resolution imaging as a whole without being restricted by the depth of field. This effectively solves the problem of mutual restriction between imaging depth and three-dimensional imaging speed in the existing technology of super-resolution dynamic imaging of samples, and can quickly perform tomographic imaging of any area of ​​interest.

[0089] Exemplary Systems

[0090] like Figure 6 As shown, corresponding to the above-mentioned super-resolution tomography method based on address scanning, an embodiment of the present invention further provides a super-resolution tomography system based on address scanning, the system comprising:

[0091] A region selection module 610 is configured to obtain a wide-field fluorescence image of a target sample and select at least one region of interest based on the wide-field fluorescence image;

[0092] A two-dimensional scanning dot matrix generation module 620 is configured to generate a series of multi-focus two-dimensional scanning dot matrices corresponding to the region of interest based on the size and position coordinate information of the circumscribed rectangle of the selected region of interest, as well as preset scanning point number information and scanning step length information;

[0093] a two-dimensional dot matrix fluorescence image acquisition module 630 for scanning the target sample according to the multi-focus two-dimensional scanning dot matrix based on a mapping relationship between a preset transverse coordinate and the acoustic wave frequency of the acousto-optic deflector, and synchronously collecting the fluorescence signal emitted by the target sample to obtain a series of two-dimensional dot matrix fluorescence images;

[0094] An axial sectioning module 640 is configured to control the electric focus lens to move the focal plane to perform axial sectioning of the target sample based on the structural features of the target sample in the wide-field fluorescence image and the mapping relationship between the preset axial coordinates and the electrical signals of the electric focus lens, and to repeat the step of acquiring a series of two-dimensional dot-matrix fluorescence images each time the focal plane is moved until a preset section coordinate range threshold is reached;

[0095] The three-dimensional super-resolution image generation module 650 is configured to reconstruct the two-dimensional dot matrix fluorescence image according to the focal plane using a preset reconstruction algorithm to obtain a three-dimensional super-resolution image of each region of interest.

[0096] Specifically, in this embodiment, the specific functions of the above-mentioned super-resolution tomography system based on address scanning can also refer to the corresponding description of the above-mentioned super-resolution tomography method based on address scanning, and will not be repeated here.

[0097] Based on the above embodiment, the present invention also provides an intelligent terminal, whose principle block diagram can be shown as follows: Figure 7 As shown. The above-mentioned intelligent terminal includes a processor, a memory, a network interface and a display screen connected via a system bus. The processor of the intelligent terminal is used to provide computing and control capabilities. The memory of the intelligent terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a super-resolution tomography program based on addressing scanning. The internal memory provides an environment for the operation of the operating system and the super-resolution tomography program based on addressing scanning in the non-volatile storage medium. The network interface of the intelligent terminal is used to communicate with an external terminal via a network connection. When the super-resolution tomography program based on addressing scanning is executed by the processor, the steps of any one of the above-mentioned super-resolution tomography methods based on addressing scanning are implemented. The display screen of the intelligent terminal can be a liquid crystal display or an electronic ink display.

[0098] Those skilled in the art will understand that Figure 7The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention and does not constitute a limitation on the smart terminal to which the solution of the present invention is applied. The specific smart terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0099] In one embodiment, a smart terminal is provided. The smart terminal includes a memory, a processor, and an address scanning-based super-resolution tomography program stored in the memory and executable on the processor. When the address scanning-based super-resolution tomography program is executed by the processor, the steps of any one of the address scanning-based super-resolution tomography methods provided in the embodiments of the present invention are implemented.

[0100] An embodiment of the present invention further provides a computer-readable storage medium, on which a super-resolution tomography program based on address scanning is stored. When the super-resolution tomography program based on address scanning is executed by a processor, the steps of any one of the super-resolution tomography methods based on address scanning provided in the embodiments of the present invention are implemented.

[0101] It should be understood that the sequence numbers of the steps in the above embodiments do not imply a specific order of execution; the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0102] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0103] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0104] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0105] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal device and method can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units described above is merely a logical functional division. In actual implementation, other division methods may be used. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented.

[0106] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the scope of protection of the present invention.

Claims

1. A super-resolution tomography method based on address scanning, applied to a super-resolution tomography device, the device mainly comprising a laser, an acousto-optic deflector, an electric focus lens, and a microscope objective lens arranged in sequence according to the direction of the optical path, characterized in that: The main steps include: Acquiring a wide-field fluorescence image of the target sample, and selecting at least one region of interest based on the wide-field fluorescence image; Based on the size and position coordinate information of the circumscribed rectangle of the selected region of interest, as well as the preset scanning point number information and scanning step length information, a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest are generated; Based on a mapping relationship between a preset transverse coordinate and an acoustic wave frequency of an acousto-optic deflector, the target sample is scanned according to the multi-focus two-dimensional scanning dot matrix, and after each scan of the multi-focus two-dimensional scanning dot matrix, a fluorescence signal emitted by the target sample corresponding to the most recently scanned multi-focus two-dimensional scanning dot matrix is ​​synchronously collected to obtain a two-dimensional dot matrix fluorescence image; Repeating the step of acquiring the two-dimensional dot matrix fluorescence image until all the multi-focus two-dimensional scanning dot matrices on the current focal plane are scanned to obtain a series of two-dimensional dot matrix fluorescence images; Based on the structural characteristics of the target sample in the wide-field fluorescence image, setting the axial slice coordinate range threshold and inter-layer spacing information; Based on the inter-layer spacing information and a mapping relationship between a preset axial coordinate and an electrical signal of the electric focus lens, the electric focus lens is controlled to move by one inter-layer spacing each time, so as to move the focal plane once; Each time the focal plane is moved, an axial slice is performed on the target sample, and the step of acquiring a series of two-dimensional dot matrix fluorescence images is repeated until a preset slice coordinate range threshold is reached; Using a preset reconstruction algorithm, super-reconstruct the two-dimensional dot matrix fluorescence image of each focal plane of the target sample to obtain a two-dimensional super-resolved reconstructed image; The two-dimensional super-resolution reconstructed images of different focal planes are three-dimensionally reconstructed to obtain a three-dimensional super-resolution image of each region of interest.

2. The super-resolution tomography method based on address scanning according to claim 1, characterized in that: The step of obtaining a wide-field fluorescence image of a target sample comprises: Adjusting the microscope objective lens to a focal plane, and controlling the acousto-optic deflector to scan the target sample according to each of the preset full-field scanning multi-focus two-dimensional scanning lattice coordinates and a mapping relationship between the preset lateral coordinates and the acoustic wave frequency of the acousto-optic deflector, thereby obtaining a series of two-dimensional lattice fluorescence images; The two-dimensional dot matrix fluorescence images are superimposed to obtain a wide-field fluorescence image of the target sample.

3. The super-resolution tomography method based on address scanning according to claim 1, characterized in that: The step of generating a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest based on the size and position coordinate information of the circumscribed rectangle of the selected region of interest, as well as the preset scanning point number information and scanning step length information, includes: Obtaining, based on the size and position coordinate information of the selected bounding rectangle of the region of interest, the X and Y coordinate information of the starting point of the bounding rectangle of the region of interest, as well as the width information along the X direction and the width information along the Y direction; Based on the X and Y coordinate information of the starting point, the width information along the X direction, the width information along the Y direction, and the preset number of scanning points and scanning step information in the X direction, a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest are obtained.

4. The super-resolution tomography method based on address scanning according to claim 3, characterized in that: Based on the X and Y coordinate information of the starting point, the width information along the X direction, the width information along the Y direction, and the preset number of scanning points and scanning step information in the X direction, a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest are generated, including: Divide the width information along the X direction by the preset number of scanning points in the X direction and round down to obtain the point spacing; Divide the width information along the Y direction by the point spacing and round up to obtain the scanning point number information in the Y direction; Based on the X and Y coordinate information of the starting point and the preset number of scanning points in the X direction and the number of scanning points in the Y direction, a multi-focus two-dimensional scanning dot matrix is ​​obtained; Based on the scanning step information, the X and Y coordinate information of the starting point are adjusted to obtain a series of multi-focus two-dimensional scanning dot arrays corresponding to the region of interest.

5. A super-resolution tomography system based on address scanning, characterized in that: The system is used to implement the steps of the super-resolution tomography method based on address scanning according to any one of claims 1 to 4, and the system comprises: a region selection module, configured to acquire a wide-field fluorescence image of a target sample and select at least one region of interest based on the wide-field fluorescence image; a two-dimensional scanning dot matrix generation module, configured to generate a series of multi-focus two-dimensional scanning dot matrices corresponding to the region of interest based on the size and position coordinate information of the circumscribed rectangle of the selected region of interest, as well as preset scanning point number information and scanning step length information; a two-dimensional dot matrix fluorescence image acquisition module, configured to scan the target sample according to the multi-focus two-dimensional scanning dot matrix based on a mapping relationship between a preset transverse coordinate and the acoustic wave frequency of the acousto-optic deflector, and synchronously collect fluorescence signals emitted by the target sample to obtain a series of two-dimensional dot matrix fluorescence images; an axial sectioning module, configured to control the electric focus lens to move the focal plane to perform axial sectioning on the target sample based on the structural features of the target sample in the wide-field fluorescence image and a mapping relationship between the preset axial coordinates and the electrical signals of the electric focus lens, and to repeat the step of acquiring a series of two-dimensional dot matrix fluorescence images each time the focal plane is moved until a preset section coordinate range threshold is reached; The three-dimensional super-resolution image generation module is used to reconstruct the two-dimensional dot matrix fluorescence image according to the focal plane using a preset reconstruction algorithm to obtain a three-dimensional super-resolution image of each region of interest.

6. An intelligent terminal, characterized in that: The intelligent terminal includes a memory, a processor, and an address scanning-based super-resolution tomography program stored in the memory and executable on the processor. When the address scanning-based super-resolution tomography program is executed by the processor, the steps of the address scanning-based super-resolution tomography method according to any one of claims 1 to 4 are implemented.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a super-resolution tomography program based on address scanning, which, when executed by a processor, implements the steps of the super-resolution tomography method based on address scanning according to any one of claims 1 to 4.

Images