A Deep Learning-Based Focal Plane Reflection-Free Microscopic Focusing Method

Abstract

A focal plane non-return microscopic focusing method based on deep learning belongs to the field of microscopic imaging technology. The present invention solves the problem that existing focusing methods require multiple return oscillations of the focal plane, resulting in long focusing time and low focusing efficiency. The present invention first constructs a focusing measurement network including an encoder and a decoder, and then uses the constructed training data set to train the focusing measurement network. When a microscopic image of an imaging target is input into the trained focusing measurement network, the focusing measurement network outputs a focusing measurement map that describes the focusing degree of each area of ​​the original image; the pixel mean in the area of ​​interest is delineated and counted from the focusing measurement map as the clarity measurement value of the input image, and a variable step-size search strategy is implemented based on the clarity measurement value and the target clarity and a PID-controlled search strategy, thereby realizing a focusing process without returning the focal plane. The method of the present invention can be applied to the microscopic focusing process in microscopic imaging.

Description

Technical Field

[0001] This invention belongs to the field of microscopic imaging technology, specifically relating to a focal plane non-reflective microscopic focusing method based on deep learning. Background Technology

[0002] Microscopes are essential tools for exploring the microscopic world, with wide applications in biomedicine, electronic component inspection, and other fields. However, due to limitations in the optical structure of microscopes, their depth of field is typically small. This causes the image to blur rapidly when the target deviates from the focal plane. Therefore, focusing is a necessary operation for microscopes to function properly and obtain high-quality images. Microscope focusing can be broadly categorized into two approaches: those based on hardware modifications and those based on software algorithms. Hardware modifications typically involve altering the illumination source and introducing light field modulation elements (SLM, phase plate, etc.) to obtain encoded defocus information, as illustrated in the literature (Silvestri L, Müllenbroich MC, Costantini I, et al. Universal autofocus for quantitative volumetric microscopy of whole mouse brains[J]. Nature Methods, 2021, 18(8):953-958. and Ge Z, Wei H, Xu F, et al. Millisecond autofocusing microscopy using neuromorphic event sensing[J]. Optics and Lasers in Engineering, 2023, 160:107247.). The defocus information is then decoded to achieve focusing. While these hardware-based methods generally offer good efficiency and accuracy, their versatility is limited by the need to modify the microscope structure, making them difficult to apply to commercial microscopes. In contrast, software algorithms only require processing the digital images acquired by the microscope to achieve focusing. Due to their ease of deployment, they have received widespread attention and research.

[0003] A relatively primitive software focusing method involves moving the entire target in a certain step size before scanning, and then selecting the image with the highest sharpness value as the focused image. However, this method requires acquiring a large number of images, resulting in a large focal plane movement distance, long focusing time, and low efficiency. More commercial software adopts a multi-scale search method (Chen Chuan, Zhang Min, Qiu Hao, et al. Microscopic autofocus system based on improved hill-climbing algorithm [J]. Journal of Jiangsu University of Technology, 2023, 29(02):10-18.DOI:10.19831 / j.cnki.2095-7394.2023.02.002.). This method first scans the target with a large step size. When the sharpness value is found to change from small to large and then back to small, it indicates that the optimal focusing point has been exceeded. At this time, a small step size is used for backtracking search. This method significantly reduces the focal plane movement distance. However, this method will repeatedly search near the optimal focusing position during the search process, causing the focal plane to oscillate back and forth multiple times. Besides multi-scale search methods, existing focusing algorithms generally require the focal plane to cross the optimal focusing position and then return, which wastes unnecessary time and is inefficient. Therefore, to solve the above problems, it is essential to propose a new microscopic focusing method that does not require focal plane reversal. Summary of the Invention

[0004] The purpose of this invention is to solve the problem that existing focusing methods require multiple oscillations of the focal plane, resulting in long focusing time and low focusing efficiency. Therefore, a deep learning-based focal plane non-oscillating microscopic focusing method is proposed.

[0005] The technical solution adopted by this invention to solve the above-mentioned technical problems is: a focal plane non-reflective microscopic focusing method based on deep learning, the method specifically including the following steps:

[0006] Step 1: Construct a focused measurement network including an encoder and a decoder;

[0007] Step 2: Generate a point spread function based on the microscope parameters, and then generate a training dataset for the focusing measurement network based on the point spread function;

[0008] Step 3: Use the training dataset generated in Step 2 to train the constructed focusing measurement network to obtain the trained focusing measurement network;

[0009] Step 4: Acquire an image of the imaging target under a microscope. After cropping the acquired image, input the image patches obtained through cropping into the trained focusing measurement network to obtain the semantic segmentation map corresponding to the acquired image.

[0010] And extract the region of interest image from the semantic segmentation map;

[0011] Step 5: Calculate the mean value of each pixel in the region of interest image extracted in Step 4, and use the calculated mean value as the sharpness value of the region of interest.

[0012] Step 6: Calculate the difference between the sharpness value obtained in Step 5 and the target sharpness value, and determine whether the difference is less than the set threshold γ;

[0013] If the difference is less than the set threshold γ, then focusing is complete;

[0014] If the difference is greater than or equal to the set threshold γ, then proceed to step seven;

[0015] Step 7: Calculate the focal plane movement step size based on the sharpness value obtained in Step 5. Move the focal plane according to the movement direction and the calculated movement step size, and then re-acquire the image of the imaging target at the new focal plane position.

[0016] Return to step four for the re-acquired imaging target image.

[0017] Furthermore, the encoder of the focusing measurement network includes N sequentially connected attention mechanism modules with the same structure, and each attention mechanism module includes a self-attention mechanism layer, a convolutional layer with a kernel size of 3×3, and an image patch overlapping region processing unit; the image patch overlapping region processing unit includes a convolutional layer with a kernel size of 7×7 and a convolutional layer with a kernel size of 3×3.

[0018] The decoder consists of two multilayer perceptrons.

[0019] Preferably, the number N of the attention mechanism modules is 4.

[0020] Furthermore, in step two, the method for generating the training dataset is as follows:

[0021] For any training data:

[0022] Step 1: Generate a point spread function based on the microscope parameters and the Born & Wolf model in ImageJ software;

[0023] Step 2: Obtain a slice with texture information and manually capture an image of the slice in focus under a microscope;

[0024] Step 3: Obtain the depth map corresponding to the slice from Step 2, specifically:

[0025] Perlin noise is generated to simulate the height variations of objects in the slices from step 2, thus obtaining a depth map.

[0026] Step 4: Combine the image obtained in Step 2 with the depth map obtained in Step 3 to obtain a 3D surface texture map;

[0027] Step 5: Convolve the 3D surface texture map with the point spread function to generate a virtual degradation map. Use the generated virtual degradation map as the input part of the training data. Calculate the absolute distance between the value of each pixel in the depth map and the position of the focal plane. Use the absolute distances corresponding to each pixel as the output part of the training data. That is, use the absolute distances corresponding to each pixel to form a label image.

[0028] Furthermore, the working process of the focused measurement network is as follows:

[0029] Each image patch obtained by cropping in step four is used as the input to the focusing measurement network. Within the focusing measurement network, each image patch first passes through the encoder, and then the outputs of the N attention mechanism modules of the encoder are all input to the first multilayer perceptron of the decoder.

[0030] The first multilayer perceptron scales N feature maps to the same dimension, then concatenates the scaled N feature maps along the channel dimension into a single feature map, and uses the concatenated feature map as the input to the second multilayer perceptron.

[0031] The semantic segmentation map is output through a second multilayer perceptron.

[0032] Furthermore, each image patch needs to be preprocessed by the Sobel operator before being input into the focusing measurement network.

[0033] Furthermore, the step size for calculating the focal plane movement based on the sharpness value obtained in step five is achieved using a PID controller.

[0034] Further, the step size for calculating the focal plane movement based on the sharpness value obtained in step five is specifically as follows:

[0035] Z _step =K p (C _set -C _cur )+K d C _diff +K i C _intgral

[0036] Among them, Z _step C is the step size of the focal plane. _cur The sharpness value obtained in step five, C _set It is the target sharpness value, C _diff It is the differential term of the PID controller, C _intgral K is the integral term of the PID controller. p ,Kd and K i These are all control parameters for the PID controller.

[0037] Furthermore, the method for calculating the target sharpness value is as follows:

[0038] Obtain a sample with the same texture as the imaging target, capture a focused image of the sample under a microscope, and then crop the captured focused image to obtain individual image blocks.

[0039] Each image patch is input into a trained focusing measurement network. The region of interest (ROI) image is then extracted from the image output by the focusing measurement network. The mean value of each pixel in the extracted ROI image is then calculated, and the calculated mean value is used as the target sharpness value.

[0040] Furthermore, the following process is included before step four:

[0041] When the focal plane is located at position A, an image of the target at position A is acquired using a microscope. A Then, move the focal plane from position A by a step size ΔA. After the movement, the focal plane is located at position B. Use a microscope to acquire the image of the target at position B. B ;

[0042] For image x A After cropping, the obtained image patches are input into the trained focus measurement network to obtain image x. A The corresponding semantic segmentation map is used to extract the region of interest (ROI) image. A Then calculate the region of interest (ROI) image. A The mean value of each pixel within the range l A ;

[0043] For image x B After cropping, the obtained image patches are input into the trained focus measurement network to obtain image x. B The corresponding semantic segmentation map is used to extract the region of interest (ROI) image. B Then calculate the region of interest (ROI) image. B The mean value of each pixel within the range l B ;

[0044] Determine if l is satisfied A -C _set <γ or l B -C _set <γ;

[0045] If the conditions are met, then focusing is complete;

[0046] If not satisfied, continue comparing l.A With l B Size:

[0047] If l A Less than l B Then the direction of movement of the focal plane is determined to be from position B to position A, and then step four is executed. That is, step four is the image acquired when the focal plane moves to a new position along the direction of movement, based on position A.

[0048] If l A Greater than l B Then the direction of movement of the focal plane is determined to be from position A to position B. Then, step four is executed. That is, step four is the image acquired when the focal plane moves to a new position along the direction of movement, based on position B.

[0049] The beneficial effects of this invention are:

[0050] 1. This invention first constructs and trains a focusing measurement network. When a microscopic image of an imaging target is input, the focusing measurement network outputs a focusing measurement map describing the focusing degree of each region of the original image, also known as a semantic segmentation map. After delineating the region of interest from the focusing measurement map, the average pixel value within the region of interest is statistically analyzed as the sharpness measurement value of the input image. Finally, a search strategy with variable step size is implemented based on a PID control search strategy.

[0051] 2. This invention achieves a focusing process that does not require the focal plane to be turned back. Compared with the traditional multi-scale search algorithm, under the premise of the same accuracy, the focal plane movement distance is reduced by 60%, the focusing time is saved by more than 70%, and the focusing efficiency is improved. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the training data generation process for a focused measurement network;

[0053] Figure 2 This is a schematic diagram illustrating the process of calculating the focal plane movement step size;

[0054] Figure 3 This is a flowchart of the focusing process of the present invention;

[0055] Figure 4 This is a schematic diagram of a 3D surface texture. Detailed Implementation

[0056] Specific implementation method one: Combining Figure 3 This embodiment describes a deep learning-based focal plane non-reflective microscopic focusing method, which specifically includes the following steps:

[0057] Step 1: Build a focused measurement network based on the mmsegmentation platform, including an encoder and a decoder;

[0058] Step 2: Generate a point spread function based on the microscope parameters, and then generate a training dataset for the focusing measurement network based on the point spread function;

[0059] Step 3: Use the training dataset generated in Step 2 to train the constructed focusing measurement network to obtain the trained focusing measurement network;

[0060] The training process is as follows: each virtual degradation map in the training dataset is cropped to obtain image patches of the same size, and there is an overlapping area between two adjacent image patches in the same virtual degradation map;

[0061] Each image patch is input into the focusing measurement network. Within the focusing measurement network, each image patch first passes through the encoder, and then the outputs of the N attention mechanism modules of the encoder are all input into the first multilayer perceptron of the decoder.

[0062] The first multilayer perceptron scales N feature maps to the same dimension, then concatenates the scaled N feature maps along the channel dimension into a single feature map, and uses the concatenated feature map as the input to the second multilayer perceptron.

[0063] The second multilayer perceptron outputs the semantic segmentation map corresponding to each virtual degenerate map;

[0064] The loss is calculated based on the semantic segmentation map and the label image. Training is stopped when the loss function converges or the maximum number of training iterations is reached, and a trained focus measurement network is obtained.

[0065] Step 4: Acquire an image of the imaging target under a microscope. After cropping the acquired image, input the image patches obtained through cropping into the trained focusing measurement network to obtain the semantic segmentation map corresponding to the acquired image.

[0066] And extract the region of interest (ROI) image from the semantic segmentation map;

[0067] Step 5: Calculate the mean value of each pixel in the region of interest image extracted in Step 4, and use the calculated mean value as the sharpness value of the region of interest.

[0068] Step 6: Calculate the difference between the sharpness value obtained in Step 5 and the target sharpness value, and determine whether the difference is less than the set threshold γ;

[0069] If the difference is less than the set threshold γ, then focusing is complete;

[0070] If the difference is greater than or equal to the set threshold γ, then proceed to step seven;

[0071] Step 7: Calculate the focal plane movement step size based on the sharpness value obtained in Step 5. Move the focal plane according to the movement direction and the calculated movement step size, and then re-acquire the image of the imaging target at the new focal plane position.

[0072] Return to step four for the re-acquired imaging target image.

[0073] The tag file of this invention is obtained from the absolute distance between the focal plane position and the depth map. Thus, the absolute distance represents the degree of focus of the degradation map; an absolute distance of 0 represents perfect focus, while a larger absolute distance indicates greater blurriness. By learning texture features, the problem of focal plane reflection during focusing is solved, improving focusing efficiency and saving focusing time, demonstrating the significant application value of this invention in the field of multi-target imaging.

[0074] Specific Implementation Method Two: This implementation method differs from Specific Implementation Method One in that the encoder of the focusing measurement network includes N sequentially connected attention mechanism modules with identical structures. Each attention mechanism module includes a self-attention mechanism layer, a convolutional layer with a kernel size of 3×3 (the convolutional layer is used to represent positional relationships), and an image patch overlapping region processing unit. The image patch overlapping region processing unit includes a convolutional layer with a kernel size of 7×7 and a convolutional layer with a kernel size of 3×3.

[0075] The decoder consists of two multilayer perceptrons.

[0076] The other steps and parameters are the same as in Specific Implementation Method 1.

[0077] It should be noted that the sequential connection in this embodiment means that the output of the first attention mechanism module is used as the input of the second attention mechanism module, the output of the second attention mechanism module is used as the input of the third attention mechanism module, and so on, with the output of the (N-1)th attention mechanism module used as the input of the Nth attention mechanism module. Within an attention mechanism module, the input image passes sequentially through the attention mechanism module, the convolutional layer, and the image patch overlap region processing unit; that is, the output of the image patch overlap region processing unit is used as the output of the attention mechanism module.

[0078] Specific Implementation Method 3: This implementation method differs from Specific Implementation Method 1 or 2 in that the number N of the attention mechanism modules is 4.

[0079] Other steps and parameters are the same as in specific implementation method one or two.

[0080] In this invention, the number of attention mechanism modules is preferably four, but is not limited to four.

[0081] Specific implementation method four: Combination Figure 1 This embodiment is described below. The difference between this embodiment and one of the specific embodiments one to three is that, in step two, the method for generating the training dataset is as follows:

[0082] For any training data:

[0083] Step 1: Generate a point spread function based on the microscope parameters and the Born & Wolf model in ImageJ software;

[0084] Step 2: Obtain a slice with texture information and manually capture an image of the slice in focus under a microscope;

[0085] Step 3: Obtain the depth map corresponding to the slice from Step 2, specifically:

[0086] Perlin noise is generated to simulate the height variations of objects in the slices from step 2, thus obtaining a depth map.

[0087] Step 4: Combine the image obtained in Step 2 with the depth map obtained in Step 3 to obtain a 3D surface texture map, such as... Figure 4 As shown;

[0088] Step 5: Convolve the 3D surface texture map with the point spread function to generate a virtual degradation map. Use the generated virtual degradation map as the input part of the training data. Calculate the absolute distance between the value of each pixel in the depth map and the position of the focal plane. Use the absolute distance of each pixel as the output part of the training data. That is, use the absolute distance of each pixel to form a label image (for any training data, after inputting the training data into the focus measurement network, the label image is the training target of the focus measurement network).

[0089] The other steps and parameters are the same as those in one of the specific implementation methods one to three.

[0090] The method described in this implementation can generate a series of training data, ultimately yielding a training dataset. The parameters set in the Born & Wolf model of the ImageJ software include the microscope's numerical aperture, pixel size, wavelength, and sampling z-axis step size. These specific parameter values ​​vary depending on the microscope. For example, a microscope might have a numerical aperture (NA) of 0.45, a pixel size of 325 nm, a wavelength of 610 nm, and a sampling z-axis step size of 500 nm.

[0091] Specific Implementation Method Five: This implementation method differs from Specific Implementation Methods One to Four in that the working process of the focusing measurement network is as follows:

[0092] Each image patch obtained by cropping in step four is used as the input to the focusing measurement network. Within the focusing measurement network, each image patch first passes through the encoder, and then the outputs of the N attention mechanism modules of the encoder are all input to the first multilayer perceptron of the decoder.

[0093] The first multilayer perceptron scales N feature maps to the same dimension, then concatenates the scaled N feature maps along the channel dimension into a single feature map, and uses the concatenated feature map as the input to the second multilayer perceptron.

[0094] The semantic segmentation map is output through a second multilayer perceptron.

[0095] The other steps and parameters are the same as those in one of the specific implementation methods one to four.

[0096] It should be noted that the image cropping method is the same for all parts of this invention.

[0097] Specific Implementation Method Six: This implementation method differs from Specific Implementation Methods One to Five in that each image block needs to be preprocessed by the Sobel operator before being input into the focusing measurement network.

[0098] The other steps and parameters are the same as those in one of the specific implementation methods one to five.

[0099] The purpose of preprocessing the input image using the Sobel operator is to make the network focus on the image's texture features.

[0100] Specific Implementation Method Seven: This implementation method differs from Specific Implementation Methods One to Six in that the step size of the focal plane movement calculated based on the sharpness value obtained in step five is achieved using a PID controller.

[0101] The other steps and parameters are the same as those in one of the specific implementation methods one to six.

[0102] Specific implementation method eight: Combination Figure 2 This embodiment is described below. The difference between this embodiment and any one of specific embodiments one through seven is that the step size for calculating the focal plane movement based on the sharpness value obtained in step five is specifically as follows:

[0103] Z _step =K p (C _set -C _cur )+K d C _diff +K i C _intgral

[0104] Among them, Z _step C is the step size of the focal plane._cur The sharpness value obtained in step five, C _set It is the target sharpness value, C _diff It is the differential term of the PID controller, C _intgral K is the integral term of the PID controller. p ,K d and K i These are all control parameters for the PID controller.

[0105] The other steps and parameters are the same as those in any of the specific implementation methods one to seven.

[0106] In the PID controller section, the control parameters can be adjusted according to actual conditions. By adjusting the PID control parameters, it is possible to avoid excessive output step size that exceeds the extreme point. Under appropriate PID control parameters, the focal plane will continuously approach the optimal focusing position, and the closer it gets to the optimal focusing position, the smaller the corresponding output step size. Focusing is completed when the sharpness measurement value is sufficiently close to the target sharpness value. Through this focusing strategy, focal plane reflection is eliminated during the focusing process.

[0107] Specific Implementation Method Nine: This implementation method differs from Specific Implementation Methods One to Eight in that the method for calculating the target sharpness value is as follows:

[0108] Obtain a sample with the same texture as the imaging target, capture a focused image of the sample under a microscope, and then crop the captured focused image to obtain individual image blocks.

[0109] Each image patch is input into a trained focusing measurement network. The region of interest (ROI) image is then extracted from the image output by the focusing measurement network. The mean value of each pixel in the extracted ROI image is then calculated, and the calculated mean value is used as the target sharpness value.

[0110] The other steps and parameters are the same as those in one of the specific implementation methods one to eight.

[0111] Because the focusing measurement network primarily learns the richness of image texture, and the texture feature richness of samples with similar textures is basically the same, the sharpness value of samples with similar textures is constant. Therefore, only one calibration of the samples is needed to obtain the target sharpness value C of all imaging targets with similar textures. _set The sharpness value C is obtained by moving the focal plane to one side from the focusing position. _set The focal plane position L1 corresponds to +γ. Then, the focal plane is moved to the other side from the focusing position to obtain the sharpness value C. _set The focal plane position L2 corresponding to +γ is obtained, and then the interval length △L between L1 and L2 is obtained, which provides prior information for focusing.

[0112] Specific Implementation Method Ten: This implementation method differs from Specific Implementation Methods One to Nine in that it includes the following process before step four:

[0113] When the focal plane is located at position A, an image of the target at position A is acquired using a microscope. A Then, move the focal plane from position A by a step size ΔA. After the movement, the focal plane is located at position B. Use a microscope to acquire the image of the target at position B. B ;

[0114] For image x A After cropping, the obtained image patches are input into the trained focus measurement network to obtain image x. A The corresponding semantic segmentation map is used to extract the region of interest (ROI) image. A Then calculate the region of interest (ROI) image. A The mean value of each pixel within the range l A ;

[0115] For image x B After cropping, the obtained image patches are input into the trained focus measurement network to obtain image x. B The corresponding semantic segmentation map is used to extract the region of interest (ROI) image. B Then calculate the region of interest (ROI) image. B The mean value of each pixel within the range l B ;

[0116] Determine if l is satisfied A -C _set <γ or l B -C _set <γ;

[0117] If the conditions are met, focusing is complete; at this point, at least one of position B and position A is between position L1 and position L2.

[0118] If not satisfied, continue comparing l. A With l B Size:

[0119] If l A Less than l B Then the direction of movement of the focal plane is determined to be from position B to position A, and then step four is executed. That is, step four is the image acquired when the focal plane moves to a new position along the direction of movement, based on position A.

[0120] If l A Greater than l BThen the direction of movement of the focal plane is determined to be from position A to position B. Then, step four is executed. That is, step four is the image acquired when the focal plane moves to a new position along the direction of movement, based on position B.

[0121] The other steps and parameters are the same as those in any of the specific implementation methods one to nine.

[0122] In this invention, the movement step size △A is controlled to be less than △L.

[0123] The above examples of the present invention are merely illustrative of the computational model and process of the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is impossible to exhaustively list all possible implementations here. Any obvious variations or modifications derived from the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A deep learning-based focal plane non-reflective microscopic focusing method, characterized in that, The method specifically includes the following steps: Step 1: Construct a focused measurement network including an encoder and a decoder; Step 2: Generate a point spread function based on the microscope parameters, and then generate a training dataset for the focusing measurement network based on the point spread function; Step 3: Use the training dataset generated in Step 2 to train the constructed focusing measurement network to obtain the trained focusing measurement network; The process preceding step four includes the following steps: When the focal plane is located at position A, an image of the target at position A is acquired using a microscope. A Then, move the focal plane from position A by a step size ΔA. After the movement, the focal plane is located at position B. Use a microscope to acquire the image of the target at position B. B ; For image x A After cropping, the obtained image patches are input into the trained focus measurement network to obtain image x. A The corresponding semantic segmentation map is used to extract the region of interest (ROI) image. A Then calculate the region of interest (ROI) image. A The mean value of each pixel within the range l A ; For image x B After cropping, the obtained image patches are input into the trained focus measurement network to obtain image x. B The corresponding semantic segmentation map is used to extract the region of interest (ROI) image. B Then calculate the region of interest (ROI) image. B The mean value of each pixel within the range l B ; Determine if l is satisfied A -C _set <γ or l B -C _set <γ; If the conditions are met, then focusing is complete; If not satisfied, continue comparing l. A With l B Size: If l A Less than l B Then the direction of movement of the focal plane is determined to be from position B to position A, and then step four is executed. That is, step four is the image acquired when the focal plane moves to a new position along the direction of movement, based on position A. If l A Greater than l B Then the direction of movement of the focal plane is determined to be from position A to position B, and then step four is executed. That is, step four is the image captured when the focal plane moves to a new position along the direction of movement based on position B. Step 4: Acquire an image of the imaging target under a microscope. After cropping the acquired image, input the image patches obtained through cropping into the trained focusing measurement network to obtain the semantic segmentation map corresponding to the acquired image. And extract the region of interest image from the semantic segmentation map; Step 5: Calculate the mean value of each pixel in the region of interest image extracted in Step 4, and use the calculated mean value as the sharpness value of the region of interest. Step 6: Calculate the difference between the sharpness value obtained in Step 5 and the target sharpness value, and determine whether the difference is less than the set threshold γ; If the difference is less than the set threshold γ, then focusing is complete; If the difference is greater than or equal to the set threshold γ, then proceed to step seven; Step 7: Calculate the focal plane movement step size based on the sharpness value obtained in Step 5. Move the focal plane according to the movement direction and the calculated movement step size, and then re-acquire the image of the imaging target at the new focal plane position. The calculation of the focal plane movement step size based on the sharpness value obtained in step five uses a PID controller, and the calculation method for the movement step size is as follows: Z _step =K p (C _set -C _cur )+K d C _diff +K i C _intgral Among them, Z _step C is the step size of the focal plane. _cur The sharpness value obtained in step five, C _set It is the target sharpness value, C _diff It is the differential term of the PID controller, C _intgral K is the integral term of the PID controller. p ,K d and K i These are all control parameters for the PID controller; The method for calculating the target sharpness value is as follows: Obtain a sample with the same texture as the imaging target, capture a focused image of the sample under a microscope, and then crop the captured focused image to obtain individual image blocks. Each image patch is input into the trained focus measurement network, and then the region of interest image is extracted from the image output by the focus measurement network. The mean value of each pixel in the extracted region of interest image is calculated, and the calculated mean value is used as the target sharpness value. Return to step four for the re-acquired imaging target image.

2. The deep learning-based focal plane non-reflective microscopic focusing method according to claim 1, characterized in that, The encoder of the focusing measurement network includes N sequentially connected attention mechanism modules with the same structure. Each attention mechanism module includes a self-attention mechanism layer, a convolutional layer with a kernel size of 3×3, and an image patch overlapping region processing unit. The image patch overlapping region processing unit includes a convolutional layer with a kernel size of 7×7 and a convolutional layer with a kernel size of 3×3. The decoder consists of two multilayer perceptrons.

3. The deep learning-based focal plane non-reflective microscopic focusing method according to claim 2, characterized in that, The number N of the attention mechanism modules is 4.

4. The deep learning-based focal plane non-reflective microscopic focusing method according to claim 1, characterized in that, In step two, the training dataset is generated as follows: For any training data: Step 1: Generate a point spread function based on the microscope parameters and the Born & Wolf model in ImageJ software; Step 2: Obtain a slice with texture information and manually capture an image of the slice in focus under a microscope; Step 3: Obtain the depth map corresponding to the slice from Step 2, specifically: Perlin noise is generated to simulate the height variations of objects in the slices from step 2, thus obtaining a depth map. Step 4: Combine the image obtained in Step 2 with the depth map obtained in Step 3 to obtain a 3D surface texture map; Step 5: Convolve the 3D surface texture map with the point spread function to generate a virtual degradation map. Use the generated virtual degradation map as the input part of the training data. Calculate the absolute distance between the value of each pixel in the depth map and the position of the focal plane. Use the absolute distances corresponding to each pixel as the output part of the training data. That is, use the absolute distances corresponding to each pixel to form a label image.

5. The deep learning-based focal plane non-reflective microscopic focusing method according to claim 1, characterized in that, The working process of the focused measurement network is as follows: Each image patch obtained by cropping in step four is used as the input to the focusing measurement network. Within the focusing measurement network, each image patch first passes through the encoder, and then the outputs of the N attention mechanism modules of the encoder are all input to the first multilayer perceptron of the decoder. The first multilayer perceptron scales N feature maps to the same dimension, then concatenates the scaled N feature maps along the channel dimension into a single feature map, and uses the concatenated feature map as the input to the second multilayer perceptron. The semantic segmentation map is output through a second multilayer perceptron.

6. The deep learning-based focal plane non-reflective microscopic focusing method according to claim 1, characterized in that, Before each image patch is input into the focusing measurement network, it needs to be preprocessed using the Sobel operator.

Images