A single-photon counting based microscopic autofocusing method

Abstract

The application discloses a kind of micro automatic focusing methods based on single-photon counting, it is related to the field of automatic focusing, including using photon counter to obtain multi-channel signal, according to multi-channel signal single-pixel imaging is carried out, and the gray scale of sample is determined;By using definition evaluation operator and gray scale are convolved, the definition evaluation value of this position is obtained, the definition evaluation value of current position and last position is compared, and the preliminary focus position is determined, and the nanometer piezoelectric translation stage is moved to preliminary focus position;Preliminary focus position is used as the center point of next search interval, and the target area is determined, and the improved hill-climbing search algorithm is used to adjust the target area, and the final focus position is determined;Nanometer piezoelectric translation stage is moved from preliminary focus position to final focus position, and automatic focusing is realized.

Description

Technical Field

[0001] This invention relates to the field of autofocus technology, and in particular to a microscopic autofocus method based on single-photon counting. Background Technology

[0002] Fluorescence confocal microscopy and stimulated emission microscopy are important high-resolution imaging tools in biomedical research today. In order to reduce the impact of phototoxicity on samples, laser light sources with lower power and shorter irradiation time are used. Ordinary CCD and CMOS are not sensitive to weak fluorescence signals and are not suitable for atomic fluorescence detection technology. However, single-photon counters based on avalanche photodiodes can achieve highly sensitive detection of weak light.

[0003] Super-resolution imaging involves scanning the sample point-by-point with a laser beam. At each scanning point, the laser excites fluorescent substances in the sample, generating a fluorescence signal. This fluorescence signal is then precisely counted using a single-photon counter. The fluorescence signal emitted at the focal point is the strongest, maximizing the reproduction of the actual sample information. Therefore, focusing is crucial for obtaining a high-quality image. However, the weak fluorescence signal under high-magnification objectives makes manual focusing difficult. Summary of the Invention

[0004] This invention provides a microscopic autofocus method based on single-photon counting to solve the aforementioned problems in the prior art, namely, the difficulty of manual focusing in super-resolution, high-quality imaging. The invention provides a microscopic autofocus method based on single-photon counting, which includes:

[0005] A photon counter is used to acquire multi-channel signals, and single-pixel imaging is performed based on the multi-channel signals to determine the grayscale image of the sample.

[0006] By convolving the sharpness evaluation operator with the grayscale image, the sharpness evaluation value at that position is obtained. The sharpness evaluation value at the current position is compared with the sharpness evaluation value at the previous position. The position corresponding to the maximum value is taken as the initial focal position, and the nanopiezoelectric level shift stage is moved to the initial focal position.

[0007] The initial focal position is used as the center point of the next search interval to determine the target area. An improved hill-climbing search algorithm is adopted to perform multi-scale search on the target area by gradually reducing the step size, step distance, and pixel array of the nanometer-pole level shifting stage. The search positions corresponding to different step sizes, step distances, and pixel arrays are determined in turn. The sharpness evaluation value is used to compare the two search positions to adjust the target area and determine the final focal position.

[0008] The nanopiezoelectric level shift stage is moved from the initial focal position to the final focal position to complete autofocus and obtain the optimal imaging position of the sample.

[0009] Optionally, the step of convolving a sharpness evaluation operator with a grayscale image to obtain a sharpness evaluation value at that location, comparing the sharpness evaluation value at the current location with the sharpness evaluation value at the previous location, and taking the location corresponding to the maximum value as the initial focus location, specifically includes:

[0010] The sharpness evaluation operator is obtained through the following formula:

[0011]

[0012] In the formula, H is the sharpness evaluation operator, F(x, y) is the image gray value, * is the convolution operation, and S i Templates representing various directions;

[0013] in:

[0014]

[0015] The sharpness evaluation operator is convolved with the grayscale image to obtain the sharpness evaluation value at that location;

[0016] The grayscale image evaluation values ​​are calculated sequentially, and the obtained grayscale image evaluation values ​​are sorted. The position corresponding to the maximum value is taken as the initial focus position.

[0017] Optionally, obtaining the grayscale image of the sample specifically includes:

[0018] The sample is scanned point by point using a preset timing sequence based on the laser emitted by the excitation continuous laser. The multi-channel signals generated during the scanning are obtained by a single photon counter to obtain the grayscale image of the sample. The multi-channel signals include dark noise, fluorescence signal of laser A, fluorescence signals of laser A and B, and fluorescence signal of laser B.

[0019] Also includes:

[0020] The multi-channel signals are subjected to signal denoising and normalization processing.

[0021] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention provides a microscopic autofocus method based on single-photon counting. This method reconstructs the image by collecting signals from single-photon counters at different stages, calculates the sharpness evaluation function, and, through an improved hill-climbing search method, can obtain two sharpness evaluation values ​​to determine the stepping direction. It then uses the next evaluation value to determine the focal position at that stage, driving the piezoelectric level shift stage to reach the target position, thus achieving fast and accurate autofocus in a fluorescence super-resolution system. Furthermore, through multi-scale search and fine adjustment, the true focal image is restored to the greatest extent, improving imaging accuracy. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0023] Figure 1 A flowchart of a microscopic autofocusing method based on single-photon counting provided for an embodiment of the present invention;

[0024] Figure 2 A flowchart of the improved hill-climbing search algorithm provided in an embodiment of the present invention;

[0025] Figure 3 The present invention provides a schematic diagram of focusing 500nm microspheres using a large step length, large step spacing, and small dot matrix.

[0026] in, Figure 3 (a) is a clear image of a 500nm microsphere using a large step size, large step spacing, and small dot matrix focusing. Figure 3 (b) is a blurred image of a 500nm microsphere using a large step size, large step spacing, and small dot matrix focusing.

[0027] Figure 4 Clear images of 500nm microspheres with medium step length, medium step distance, and medium dot matrix focusing are provided for embodiments of the present invention.

[0028] Figure 5 The 500nm microspheres provided in this embodiment of the invention use small step size, small step distance, and large dot matrix focusing to produce clear images.

[0029] Figure 6 This is a schematic diagram of a nanopiezoelectric level shifting stage scanning device provided in an embodiment of the present invention.

[0030] In the diagram, 100 is the host computer autofocus module, 110 is the slave computer scanning control module, 120 is the confocal and STED microscope optical path module, and 130 is the photon counter signal acquisition module.

[0031] 1 is the host computer, 2 is the STM32H743 microcontroller, 3 is the excitation continuous laser, 4 is the loss continuous laser, 5 is the first acousto-optic modulator, 6 is the second acousto-optic modulator, 7 is the first pinhole aperture, 8 is the second pinhole aperture, 9 is the first spatial light filter, 10 is the second spatial light filter, 11 is the third pinhole aperture, 12 is the fourth pinhole aperture, 13 is the first quarter-wave plate, 14 is the second quarter-wave plate, 15 is the first half-wave plate, 16 is the second half-wave plate, 17 is the spatial light modulator, 18 is the dichroic mirror, 19 is the first reflecting mirror, 20 is the second reflecting mirror, 21 is the filter, 22 is the semi-reflective half-lens, 23 is the objective lens, 24 is the nanometer piezoelectric level shift stage, 25 is the lens, 26 is the single-mode fiber, and 27 is the single-photon counter. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0033] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0034] Figure 1 This is a flowchart of a microscopic autofocus method based on single-photon counting provided by an embodiment of the present invention, as shown below. Figure 1 As shown in the figure, this embodiment illustrates a microscopic autofocus method based on single-photon counting, comprising:

[0035] S1: A photon counter is used to acquire multi-channel signals, and single-pixel imaging is performed based on the multi-channel signals to determine the grayscale image of the sample.

[0036] Generally, since a single-photon counter is a digital signal that fits an image, the size of the fitted image pixel block corresponding to the actual image can be flexibly set. During autofocus, the moving step size, step distance, and pixel grid of the nanoscale voltage level shift stage can be changed according to the focusing accuracy. The multi-scale fitted image can not only reduce the amount of computation and improve the focusing process, but also reconstruct the original image of the excited sample more realistically.

[0037] For example, the acquired multi-channel signals can be denoised and normalized; the fitted grayscale image can be normalized; the actual image can be denoised by subtracting the blank image from the actual image; and then threshold segmentation can be used to reduce discrete noise; the multi-channel signals may include dark noise, the fluorescence signal of laser A, the fluorescence signals of laser A and B, and the fluorescence signal of laser B.

[0038] S2: By convolving the sharpness evaluation operator with the grayscale image, the sharpness evaluation value at this position is obtained. The sharpness evaluation value at the current position is compared with the sharpness evaluation value at the previous position. The position corresponding to the maximum value is taken as the initial focal position. The nano-piezoelectric level shift stage is moved to the initial focal position.

[0039] For example, the voltage level shift stage is moved to the lowest point and moved from top to bottom according to the set large step size m0. At each step, it pauses for a period of time to perform point scanning. The main purpose is to search for the vicinity of the sample focal point. The scanning parameters can be set to a large step size n1 and a scanning point array of k1×k1. The photoswitch is controlled to excite the sample according to the designed timing. The excited fluorescence signal is coupled into the single photon counter through the optical fiber. At each point, four signals are collected: the dark noise of the photon counter of the sample without laser irradiation, the fluorescence signal collected by the sample irradiated by laser A, the fluorescence signal collected by the sample irradiated by lasers A and B, and the fluorescence signal collected by the sample irradiated by laser B.

[0040] Optionally, a sharpness evaluation value is obtained by convolving the sharpness evaluation operator with the grayscale image. The sharpness evaluation value at the current position is then compared with the sharpness evaluation value at the previous position to determine the preliminary focus position. This specifically includes:

[0041] The sharpness evaluation operator is obtained through the following formula:

[0042]

[0043] In the formula, H is the sharpness evaluation operator, F(x, y) is the image gray value, * is the convolution operation, and S i Templates representing various directions;

[0044] in:

[0045]

[0046] The sharpness evaluation operator is convolved with the grayscale image to obtain the sharpness evaluation value at that location;

[0047] The grayscale image evaluation values ​​are calculated sequentially, and the obtained grayscale image evaluation values ​​are sorted. The position corresponding to the maximum value is taken as the initial focus position.

[0048] For example, a piezoelectric level shift stage can be driven to search the entire focus range with a large step size m0 (z-axis). The sample is scanned with a fixed large step size n1 (pixel spacing) and a fixed dot matrix k1×k1 (number of scan dots). The number of photons collected is transmitted back to the host computer to fit the image for each step size, which is then bound to the absolute position information. The evaluation value is calculated based on the sharpness evaluation function. The obtained evaluation values ​​are compared to obtain the maximum value, and the absolute position information km1 corresponding to the maximum value is recorded, which is the preliminary focus position.

[0049] For example, to calculate the sharpness evaluation value to determine the next direction of the translation stage movement, the following operations can be performed on the original scanning data: (1) Separate the scanning parameters in the data; (2) Rearrange the data into a scanning dot matrix of size k1×k1; (3) Use matrix subtraction for confocal fluorescence images, that is, subtract the data of the first channel from the data of the second channel point by point to remove the dark noise of the single photon counter. Similarly, use matrix subtraction for STED fluorescence images, that is, subtract the data of the first channel from the data of the third channel point by point; (4) Median filtering for noise reduction. Due to the influence of the external environment and the large discrete values ​​in the data acquisition and transmission, the threshold segmentation method is selected to filter out invalid signals; (5) Normalize the data matrix to obtain the grayscale image of the sample; and then use the sharpness evaluation operator to convolve with the grayscale image to obtain the sharpness evaluation value at that position.

[0050] S3: Using the initial focal position as the center point of the next search interval, the target area is determined. An improved hill-climbing search algorithm is adopted. By gradually reducing the step size, step distance, and pixel array of the nanometer-level voltage level shifting stage, the target area is searched at multiple scales. The search positions corresponding to different step sizes, step distances, and pixel arrays are determined in turn. The sharpness evaluation value is used to compare the two search positions to adjust the target area and determine the final focal position.

[0051] Optionally, an improved hill-climbing search algorithm is used to fine-tune the target area and determine the final focal point, specifically including:

[0052] By gradually reducing the step size, step distance, and pixel array, the target area is searched at multiple scales. The search positions corresponding to different step sizes, step distances, and pixel arrays are determined in sequence. The sharpness evaluation value is used to compare the two search positions before and after, and the step direction of the nanopiezoelectric level shift stage is determined for the next step.

[0053] like Figure 2 As shown, an improved hill-climbing search method is used to determine the next step position. Specifically: First, the sharpness evaluation value of the current position is V1, and the sharpness evaluation value of the previous translation position is V2. At this time, the latter evaluation value V2 is greater than the former evaluation value V1, so it is necessary to step again to calculate the evaluation value V3 to determine which is the true focal position. Through convolution operation, V2 is found to be greater than V3, so V1 is considered to be the optimal focal position. If V2 is less than or equal to V3, V2 is considered to be the optimal focal position; otherwise, the search continues until the search boundary is reached. At this time, the evaluation values ​​are sorted, and the position corresponding to the maximum value is considered to be the optimal focal position km2. Figure 3 As shown in (a), Figure 3The (b) of it is the corresponding out-of-focus blurred image. Then move the translation stage to km2. Next, use a medium step size of 200, a point scanning parameter of a medium step distance of 40, a medium dot matrix of 125×125, and a scanning range of (km2 - 600, km2 + 600). Use the improved hill climbing method in the previous step to search for the corresponding focal position km3, and move the translation stage to km3, as Figure 4 shown.

[0054] S4: Move the nano-piezoelectric translation stage from the preliminary focal position to the final focal position, complete the autofocus, and obtain the best imaging position of the sample.

[0055] As Figure 5 shown, based on the final focal position km4 obtained by the improved hill climbing search algorithm, use a small step size of 100, a point scanning parameter of a medium step distance of 20, a medium dot matrix of 250×250, and a scanning range of (km3 - 300, km3 + 300). Use the improved hill climbing method in the second step to search for the corresponding focal position and complete the autofocus process.

[0056] Exemplarily, at the preliminary focal position km1, arbitrarily select a search area of (km1 - q1m1, km1 + q1m1), a large step size m1 (m1 ≤ m0 / 2), a large step distance n1, and a fixed dot matrix k1×k1. Calculate the sharpness evaluation value V1 of the step km1 - q1m1 and compare it with the sharpness evaluation value V2 of km1 - (q1 - 1)m2. If V2 > V1, continue the step-by-step search. If V2 ≤ V1, continue to take one more step to reach the position km1 - (q1 - 2)m1, calculate the sharpness evaluation value V3 at this position. If V3 ≥ V2, then consider the position V2 as the focal position at this time. If V3 < V2, then consider V1 as the focal position, and drive the piezoelectric translation stage to the optimal position V1 or V2;

[0057] At this time, due to the relatively large scanning step distance, the fitted image cannot fully reflect the real focal position information. Therefore, scan the sample again with a medium step size m2, a medium step distance n2 (n2 < n1), and a dot matrix k2×k2 (k2 > k1) to obtain more detailed image details in the search area (km2 - q2m2, km2 + q2m2). Calculate the sharpness evaluation value according to the above method and compare it, and drive the piezoelectric translation stage to the optimal position km3;

[0058] Finally, in order to reduce the influence of the translation stage hysteresis effect and the nonlinear response of the servo system on the autofocus performance, scan the sample with a small step size m3 (m3 < m2), a small step distance n3, and a dot matrix k3×k3 (k3 > k3) to obtain an image within the range of (km3 - q3m3, km3 + q3m3). The remaining steps are as above. Finally, the last moved position km4 is the best imaging position, and the focusing process is completed.

[0059] As Figure 6As shown, a 491nm continuous laser can be used as the excitation light. The first acousto-optic modulator 5 is turned on, and the second acousto-optic modulator 6 is turned off. After acousto-optic modulation, the first-order diffraction spot is selected as the experimental beam. A first pinhole aperture 7 is placed to eliminate the influence of other diffraction orders on the first-order diffraction spot. At this point, the beam is not uniform. It is then shaped and expanded by the first spatial light filter 9 to obtain a uniform Gaussian beam. The beam then passes through the first half-wave plate 13 and the first quarter-wave plate 15 to adjust the polarization state. Finally, it passes through a dichroic mirror 18 and a semi-reflective lens 22 before entering the objective lens 23. A 100× objective lens is selected to observe a 500nm green fluorescent microsphere. This microsphere has a narrow fluorescence emission band. After laser excitation of the sample, it passes through the semi-reflective lens 22 and a 510nm / 10nm filter to collect the fluorescence signal in this band. Single-photon fitting imaging is performed according to the set program, and an autofocus method is implemented. The host computer 1 issues an autofocus command, and the slave computer 2 receives it and runs according to the predetermined program.

[0060] For example, the laser emitted by the excitation continuous laser 3 is shaped by the spatial light modulator 9 to obtain a uniform Gaussian spot as an illumination source; the excitation continuous laser 3 introduces the laser from the depletion continuous laser 4 as the excitation light, uses the spatial light modulator 17 to obtain a ring light, and then passes through the dichroic mirror 18 for beam combining and the semi-reflective mirror 22 to irradiate the sample, generating a fluorescence signal.

[0061] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this invention.

Claims

1. A microscopic autofocusing method based on single-photon counting, characterized in that, include: A photon counter is used to acquire multi-channel signals, and single-pixel imaging is performed based on the multi-channel signals to determine the grayscale image of the sample. By convolving the sharpness evaluation operator with the grayscale image, the sharpness evaluation value at the current position is obtained. The sharpness evaluation value at the current position is compared with the sharpness evaluation value at the previous position. The position corresponding to the maximum value is taken as the initial focal position, and the nanopiezoelectric level shift stage is moved to the initial focal position. The initial focal position is used as the center point of the next search interval to determine the target area. An improved hill-climbing search algorithm is adopted to perform multi-scale search on the target area by gradually reducing the step size, step distance, and pixel array of the nanometer-pole level shifting stage. The search positions corresponding to different step sizes, step distances, and pixel arrays are determined in turn. The sharpness evaluation value is used to compare the two search positions to adjust the target area and determine the final focal position. The improved hill-climbing search algorithm compares the sharpness evaluation value of the current position, the sharpness evaluation value of the previous position, and the sharpness evaluation value of the next position of the nanopiezoelectric level shift stage, and takes the position corresponding to the maximum value of the sharpness evaluation value as the optimal focus position to drive the nanopiezoelectric level shift stage to complete autofocus. The nanopiezoelectric level shift stage is moved from the initial focal position to the final focal position to complete autofocus and obtain the optimal imaging position of the sample.

2. The microscopic autofocus method based on single-photon counting as described in claim 1, characterized in that, The process involves convolving a sharpness evaluation operator with a grayscale image to obtain a sharpness evaluation value at the current position. This value is then compared to the sharpness evaluation value at the previous position, and the position corresponding to the maximum value is taken as the initial focus position. Specifically, this includes: The sharpness evaluation operator is obtained through the following formula: ； In the formula, H For the sharpness evaluation operator, The image grayscale value, For convolution operations, Templates representing various directions; in: 、 ； The sharpness evaluation operator is convolved with the grayscale image to obtain the sharpness evaluation value at the current position; The grayscale image evaluation values ​​are calculated sequentially, and the obtained grayscale image evaluation values ​​are sorted. The position corresponding to the maximum value is taken as the initial focus position.

3. The microscopic autofocus method based on single-photon counting as described in claim 1, characterized in that, The acquisition of the grayscale image of the sample specifically includes: The sample is scanned point by point using a preset timing sequence based on the laser emitted by the excitation continuous laser. The multi-channel signals generated during the scanning are obtained by a single photon counter to obtain the grayscale image of the sample. The multi-channel signals include dark noise, fluorescence signal of laser A, fluorescence signals of laser A and B, and fluorescence signal of laser B.

4. The microscopic autofocus method based on single-photon counting as described in claim 3, characterized in that, Also includes: The multi-channel signals are subjected to signal denoising and normalization processing.

Images