Microscopic automatic focusing method based on single photon counting

Abstract

The invention discloses a microscopic automatic focusing method based on single photon counting, and relates to the field of automatic focusing, and the method comprises the steps: employing a photon counter to obtain multi-channel signals, carrying out the single-pixel imaging according to the multi-channel signals, and determining a gray-scale map of a sample; the definition evaluation operator and the grey-scale map are adopted for convolution, the definition evaluation value of the position is obtained, the definition evaluation value of the current position is compared with the definition evaluation value of the previous position, the initial focus position is determined, and the nanometer piezoelectric translation table is moved to the initial focus position; taking the initial focus position as a central point of a next search interval, determining a target area, adjusting the target area by adopting an improved hill-climbing search algorithm, and determining a final focus position; and moving the nanometer piezoelectric translation stage from the initial focus position to the final focus position to realize automatic focusing.

Description

Technical Field

[0001] The present invention relates to the technical field of autofocus, and particularly to a microscopic autofocus method based on single-photon counting. Background Art

[0002] Fluorescence confocal and stimulated emission microscopes are important means for high-resolution imaging in current biomedical research. In order to reduce the impact of phototoxicity on samples, a lower laser light source power and a shorter irradiation time are used. Ordinary CCDs and CMOSs are not sensitive to weak fluorescence signals and are not applicable to atomic fluorescence detection technology. However, a single-photon counter based on an avalanche photodiode can achieve highly sensitive detection of weak light.

[0003] In the super-resolution imaging process, a laser beam is scanned point by point on the sample. At each scanned point, the laser excites the fluorescent substances in the sample to generate fluorescence signals, and the fluorescence signals are accurately counted by a single-photon counter. The fluorescence signal excited at the focus is the strongest and can restore the actual sample information to the greatest extent. Therefore, autofocus is required to obtain a high-quality image. However, the weak fluorescence signal under a high-magnification objective lens makes manual focusing difficult. Summary of the Invention

[0004] The present invention provides a microscopic autofocus method based on single-photon counting to solve the above problems existing in the prior art, that is, the problem of difficult manual focusing in super-resolution high-quality imaging in the prior art. The present invention provides a microscopic autofocus method based on single-photon counting, and the method includes:

[0005] Obtain multi-channel signals by using a photon counter, perform single-pixel imaging based on the multi-channel signals, and determine the grayscale image of the sample;

[0006] Convolve the sharpness evaluation operator with the grayscale image to obtain the sharpness evaluation value at this position. Compare the sharpness evaluation value at the current position with the sharpness evaluation value at the previous position, take the position corresponding to the maximum value as the preliminary focus position, and move the nano-piezoelectric translation stage to the preliminary focus position;

[0007] Take the preliminary focus position as the center point of the next search interval, determine the target area, adopt an improved hill-climbing search algorithm, perform multi-scale search on the target area by gradually reducing the step size, step distance, and pixel lattice of the nano-piezoelectric translation stage, sequentially determine the search positions corresponding to different step sizes, step distances, and pixel lattices, and use the sharpness evaluation value setting to compare the two search positions before and after to adjust the target area and determine the final focus position;

[0008] Move the nano-piezoelectric translation stage from the preliminary focus position to the final focus position to complete autofocus and obtain the optimal imaging position of the sample.

[0009] Optionally, the sharpness evaluation operator is convolved with the grayscale image to obtain the sharpness evaluation value at this position, and the sharpness evaluation value at the current position is compared with the sharpness evaluation value at the previous position, and the position corresponding to the maximum value is used as the preliminary focus position, which specifically includes:

[0010] The sharpness evaluation operator is obtained through the following formula:

[0011]

[0012] In the formula, H is the sharpness evaluation operator, F(x, y) is the image grayscale value, * is the convolution operation, and S i represents templates in each direction;

[0013] Where:

[0014]

[0015] The sharpness evaluation operator is convolved with the grayscale image to obtain the sharpness evaluation value at this position;

[0016] The grayscale image evaluation values are calculated in sequence, the obtained grayscale image evaluation values are sorted, and the position corresponding to the maximum value is used as the preliminary focus position.

[0017] Optionally, the acquisition of the grayscale image of the sample specifically includes:

[0018] Laser is emitted according to the excitation light continuous laser, the sample is scanned point by point using a preset time sequence, and multi-channel signals generated during scanning are obtained through a single photon counter to obtain the grayscale image of the sample; wherein, the multi-channel signals include dark noise, laser A fluorescence signal, laser A and B fluorescence signals, and laser B fluorescence signal.

[0019] It also includes:

[0020] Signal denoising and normalization processing are performed on the multi-channel signals.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention provides a microscopic autofocus method based on single photon counting. This method reconstructs an image through signals collected by a single photon counter at different stages, calculates a sharpness evaluation function, and through an improved hill climbing search method, can obtain the step direction by determining the sharpness evaluation values before and after, and determine the focus position at this stage through the next evaluation value, driving the piezoelectric translation stage to reach the target position, realizing automatic focusing of a fast and accurate fluorescence super-resolution system; in addition, through multi-scale search and fine adjustment, the real focus image is restored to the greatest extent, improving the imaging accuracy. Description of the Drawings

[0022] The accompanying drawings here are incorporated into the specification and form a part of this specification, showing embodiments in line with the present invention and, together with the specification, are used to explain the principles of the present invention.

[0023] Figure 1 It is a flowchart of a microscopic autofocus method based on single-photon counting provided for an embodiment of the present invention;

[0024] Figure 2 It is a flowchart of an improved hill-climbing search algorithm provided for an embodiment of the present invention;

[0025] Figure 3 It is a schematic diagram of focusing on a 500nm microsphere using a large step size, large step distance, and small dot matrix provided for an embodiment of the present invention;

[0026] Among them, Figure 3 (a) of it is a clear picture of focusing on a 500nm microsphere using a large step size, large step distance, and small dot matrix; Figure 3 (b) of it is a blurred picture of focusing on a 500nm microsphere using a large step size, large step distance, and small dot matrix;

[0027] Figure 4 It is a clear picture of focusing on a 500nm microsphere using a medium step size, medium step distance, and medium dot matrix provided for an embodiment of the present invention;

[0028] Figure 5 It is a clear picture of focusing on a 500nm microsphere using a small step size, small step distance, and large dot matrix provided for an embodiment of the present invention;

[0029] Figure 6 It is a schematic diagram of a nano-piezoelectric translation stage scanning device provided for an embodiment of the present invention.

[0030] In the figure, 100 is the host computer autofocus module, 110 is the slave computer scanning control module, 120 is the confocal and STED microscope optical path module, and 130 is the photon counter signal acquisition module;

[0031] 1 is the host computer, 2 is the STM32H743 single-chip microcomputer, 3 is the continuous excitation light laser, 4 is the continuous loss light laser, 5 is the first acousto-optic modulator, 6 is the second acousto-optic modulator, 7 is the first small aperture diaphragm, 8 is the second small aperture diaphragm, 9 is the first spatial light filter, 10 is the second spatial light filter, 11 is the third small aperture diaphragm, 12 is the fourth small aperture diaphragm, 13 is the first quarter-wave plate, 14 is the second quarter-wave plate, 15 is the first half-wave plate, 16 is the second half-wave plate, 17 is the spatial light modulator, 18 is the dichroic mirror, 19 is the first reflector, 20 is the second reflector, 21 is the filter, 22 is the semi-reflective and semi-transmissive lens, 23 is the objective lens, 24 is the nano-piezoelectric translation stage, 25 is the lens, 26 is the single-mode optical fiber, and 27 is the single-photon counter. Detailed implementation mode

[0032] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0033] The technical solutions of the present invention and how the technical solutions of the present invention solve the above technical problems will be described in detail below with specific embodiments. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0034] Figure 1 is a flowchart of a microscopic autofocus method based on single-photon counting provided by an embodiment of the present invention. As Figure 1 shown, a microscopic autofocus method based on single-photon counting shown in this embodiment includes:

[0035] S1: Use a photon counter to obtain multi-channel signals, perform single-pixel imaging based on the multi-channel signals, and determine the grayscale image of the sample.

[0036] Generally, since the single-photon counter is a digital signal fitting image, the size of the fitting image pixel block corresponding to the actual image can be flexibly set. When autofocusing, the moving step, step distance, and pixel grid of the nano-piezoelectric translation stage are changed according to the focusing accuracy. The multi-scale fitting image can not only reduce the amount of calculation and improve the focusing process, but also more realistically reconstruct the original image of the excited sample.

[0037] Exemplarily, the obtained multi-channel signals can be subjected to signal denoising and normalization processing; the fitted grayscale image is normalized, the actual image is subtracted from the blank image for denoising, and then threshold segmentation is used to reduce discrete noise; the multi-channel signals can include dark noise, laser A fluorescence signal, laser A and B fluorescence signals, and laser B fluorescence signal.

[0038] S2: Convolve the sharpness evaluation operator with the grayscale image to obtain the sharpness evaluation value at this position. Compare the sharpness evaluation value at the current position with the sharpness evaluation value at the previous position, take the position corresponding to the maximum value as the preliminary focus position, and move the nano-piezoelectric translation stage to the preliminary focus position.

[0039] Exemplarily, move the piezoelectric translation stage to the lowest point and move it from top to bottom in accordance with the set large step length m0. Stay for a period of time at each step for point scanning. The main purpose is to search near the sample focus. The scanning parameters can be set as a large step distance n1, and the scanning dot matrix is k1×k1. Control the optical switch to excite the sample according to the designed timing sequence. The excited fluorescence signal is coupled into the single-photon counter through an optical fiber. Four signals are collected at each point, namely, the dark noise of the photon counter without laser irradiating the sample, the fluorescence signal collected when laser A irradiates the sample, the fluorescence signal collected when lasers A and B irradiate the sample, and the fluorescence signal collected when laser B irradiates the sample.

[0040] Optionally, convolve the sharpness evaluation operator with the grayscale image to obtain the sharpness evaluation value at this position. Compare the sharpness evaluation value at the current position with that at the previous position to determine the preliminary focus position, specifically including:

[0041] Obtain the sharpness evaluation operator through the following formula:

[0042]

[0043] In the formula, H is the sharpness evaluation operator, F(x, y) is the image grayscale value, * is the convolution operation, and S i represents the templates in each direction;

[0044] Among them:

[0045]

[0046] Perform convolution calculation on the sharpness evaluation operator and the grayscale image to obtain the sharpness evaluation value at this position;

[0047] Calculate the grayscale image evaluation values in sequence, sort the obtained grayscale image evaluation values, and take the position corresponding to the maximum value as the preliminary focus position.

[0048] Exemplarily, the piezoelectric translation stage can be driven to search the entire focusing range with a large step length m0 (z-axis), scan the sample with a fixed large step distance n1 (pixel pitch) and a fixed dot matrix k1×k1 (number of scanning dots), transmit the collected number of photons back to the host computer to fit the image of each step length, bind it with the absolute position information, calculate its evaluation value based on the sharpness evaluation function, compare the obtained evaluation values to obtain the maximum value, and record the absolute position information km1 corresponding to the maximum value of this evaluation value, which is the preliminary focus position.

[0049] Exemplarily, when it is necessary to calculate the clarity evaluation value to judge the next moving direction of the translation stage, the following operations can be performed on the original scanned data: (1) Separate the scanning parameters in the data; (2) Rearrange the data into a scanning dot matrix of size k1×k1; (3) For confocal fluorescence images, use matrix subtraction, that is, subtract the data of the first channel from the data of the second channel point by point to remove the dark noise of the single-photon counter. Similarly, for STED fluorescence images, use matrix subtraction, that is, subtract the data of the first channel from the data of the third channel point by point; (4) Median filtering for denoising. Due to the influence of the external environment and the transmission of the acquired data, there will be several discrete large values, so the method of threshold segmentation is selected to filter out invalid signals; (5) Normalize the data matrix to obtain the grayscale image of the sample; then use the clarity evaluation operator to convolve with the grayscale image to obtain the clarity evaluation value at this position.

[0050] S3: Use the preliminary focal position as the center point of the next search interval to determine the target area. Adopt an improved hill climbing search algorithm. By gradually reducing the step size, step distance, and pixel lattice of the nano-piezoelectric translation stage, perform multi-scale search on the target area, and sequentially determine the search positions corresponding to different step sizes, step distances, and pixel lattices. Use the clarity evaluation value setting to compare the search positions before and after to adjust the target area and determine the final focal position.

[0051] Optionally, adopt an improved hill climbing search algorithm to finely adjust the target area to determine the final focal position, specifically including:

[0052] Perform multi-scale search on the target area by gradually reducing the step size, step distance, and pixel lattice, sequentially determine the search positions corresponding to different step sizes, step distances, and pixel lattices, use the clarity evaluation value setting to compare the search positions before and after, and determine the next stepping direction of the nano-piezoelectric translation stage.

[0053] As Figure 2 shown, adopt an improved hill climbing search method to determine the next stepping position. Specifically: First, the clarity evaluation value at the current position is V1, and the clarity evaluation value at the previous translation stage position is V2. At this time, the latter evaluation value V2 is greater than the previous evaluation value V1. It is necessary to take another step to calculate the evaluation value V3 to judge which is the real focal position. Through convolution operation, it is obtained that V2 is greater than V3. Therefore, it is considered that V1 is the best focal point. If V2 is less than or equal to V3, it is considered that V2 is the best focal position. Otherwise, continue the search until the search boundary is reached. At this time, perform evaluation value sorting, and the position corresponding to the maximum value is considered the best focal position km2, as Figure 3 shown in (a) of Figure 3The (b) of it is the corresponding out-of-focus blurred image. Then move the translation stage to km2. Next, use a medium step size of 200, the point scanning parameters are a medium step distance of 40, a medium dot matrix of 125×125, and the scanning range is (km2 - 600, km2 + 600). Use the improved hill climbing method in the previous step to search for the corresponding focal position km3, and move the translation stage to km3, as Figure 4 shown.

[0054] S4: Move the nano-piezoelectric translation stage from the preliminary focal position to the final focal position to complete autofocus and obtain the best imaging position of the sample.

[0055] As Figure 5 shown, based on the final focal position km4 obtained by the improved hill climbing search algorithm, use a small step size of 100, the point scanning parameters are a medium step distance of 20, a medium dot matrix of 250×250, and the scanning range is (km3 - 300, km3 + 300). Use the improved hill climbing method in the second step to search for the corresponding focal position to complete the autofocus process.

[0056] Exemplarily, at the preliminary focal position km1, the search area is self-selected as (km1 - q1m1, km1 + q1m1), a large step size m1 (m1 ≤ m0 / 2), a large step distance n1, and a fixed dot matrix k1×k1. Calculate the sharpness evaluation value V1 of the step km1 - q1m1 and compare it with the sharpness evaluation value V2 of km1 - (q1 - 1)m2. If V2 > V1, continue the step search. If V2 ≤ V1, continue to step further to the position km1 - (q1 - 2)m1, calculate the sharpness evaluation value V3 at this position. If V3 ≥ V2, then it is considered that the position of V2 is the focal position at this time. If V3 < V2, then it is considered that V1 is the focal position, and drive the piezoelectric translation stage to the best position V1 or V2;

[0057] At this time, due to the relatively large scanning step distance, the fitted image cannot fully reflect the real focal position information. Therefore, scan the sample again with a medium step size m2, a medium step distance n2 (n2 < n1), and a dot matrix k2×k2 (k2 > k1) to obtain more detailed image details in the search area (km2 - q2m2, km2 + q2m2). Calculate the sharpness evaluation value according to the above method and compare, and drive the piezoelectric translation stage to the best position km3;

[0058] Finally, in order to reduce the influence of the translation stage hysteresis effect and the nonlinear response of the servo system on the autofocus performance, scan the sample with a small step size m3 (m3 < m2), a small step distance n3, and a dot matrix k3×k3 (k3 > k3) to obtain the image within the range of (km3 - q3m3, km3 + q3m3). The remaining steps are as above, and the finally moved position km4 is the best imaging position to complete the focusing process.

[0059] As Figure 6As shown, a continuous laser with a wavelength of 491 nm can be used as the excitation light. At this time, the first acousto-optic modulator 5 is turned on and the second acousto-optic modulator 6 is turned off. After acousto-optic modulation, the first-order diffraction spot is selected as the experimental beam, and the first small aperture diaphragm 7 is placed to exclude the influence of other diffraction orders on the first-order diffraction spot. At this time, the beam is not uniform enough. After passing through the first spatial light filter 9 for beam shaping and beam expansion, a uniform Gaussian spot is obtained. Then, the polarization state is adjusted successively through the first half-wave plate 13 and the first quarter-wave plate 15. Finally, it enters the objective lens 23 through the dichroic mirror 18 and the semi-reflective semi-transmissive lens 22. A 100× objective lens is selected to observe 500-nm green fluorescent microspheres. This microsphere has a relatively narrow fluorescence emission band. After the laser excites the sample, it passes through the semi-reflective semi-transmissive lens 22 and the 510 nm / 10 nm filter, and the fluorescence signal in this band is collected, and single-photon fitting imaging and an autofocus method are implemented according to the set program. The host computer 1 issues an autofocus command, and the slave computer 2 receives it and runs according to the established program.

[0060] Exemplarily, the laser emitted by the continuous excitation light laser 3 is shaped by the spatial light modulator 9 to obtain a uniform Gaussian spot as the illumination light source; for the continuous excitation light laser 3, the laser of the loss light continuous laser 4 is introduced as the de-excitation light, and an annular light is obtained by using the spatial light modulator 17, and then it is combined by the dichroic mirror 18 and irradiated on the sample through the semi-reflective semi-transmissive lens 22 to generate a fluorescence signal.

[0061] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope recorded by the present invention.

Claims

1. A microscopic autofocus method based on single photon counting, characterized in that: include: A photon counter is used to obtain multi-channel signals, and single-pixel imaging is performed based on the multi-channel signals to determine the grayscale image of the sample; The clarity evaluation value of the position is obtained by convolving the clarity evaluation operator with the grayscale image, the clarity evaluation value of the current position is compared with the clarity evaluation value of the previous position, the position corresponding to the maximum value is taken as the preliminary focus position, and the nanopiezoelectric shift stage is moved to the preliminary focus position; The initial focus position is used as the center point of the next search interval to determine the target area. The improved hill climbing search algorithm is used to gradually reduce the step length, step size and pixel array of the nanopiezoelectric shift stage to perform a multi-scale search on the target area. The search positions corresponding to different step lengths, step sizes and pixel arrays are determined in turn. The clarity evaluation value setting is used to compare the two search positions before and after to adjust the target area and determine the final focus position. Move the nanopiezoelectric shift stage from the preliminary focus position to the final focus position, complete autofocus, and obtain the optimal imaging position of the sample.

2. The microscopic automatic focusing method based on single photon counting as claimed in claim 1, characterized in that: The clarity evaluation value of the position is obtained by convolving the clarity evaluation operator with the grayscale image, comparing the clarity evaluation value of the current position with the clarity evaluation value of the previous position, and taking the position corresponding to the maximum value as the preliminary focus position, which specifically includes: The clarity evaluation operator is obtained by the following formula: In the formula, H is the clarity evaluation operator, F(x, y) is the image grayscale value, * is the convolution operation, S i Represents templates in various directions; in: Perform convolution calculation on the clarity evaluation operator and the grayscale image to obtain the clarity evaluation value of the position; The grayscale image evaluation values ​​are calculated in sequence, the obtained grayscale image evaluation values ​​are sorted, and the position corresponding to the maximum value is used as the preliminary focus position.

3. The microscopic automatic focusing method based on single photon counting as claimed in claim 1, characterized in that: The acquisition of the grayscale image of the sample specifically includes: According to the excitation light continuous laser, the sample is scanned point by point using a preset timing, and the multi-channel signal generated during the scanning is acquired by a single photon counter to obtain a grayscale image of the sample; wherein the multi-channel signal includes dark noise, laser A fluorescence signal, laser A and B fluorescence signals, and laser B fluorescence signal.

4. The microscopic automatic focusing method based on single photon counting as claimed in claim 3, characterized in that: Also includes: The multi-channel signals are subjected to signal denoising and normalization processing.

Images