Microscope automatic focusing method and device and computer readable storage medium

Abstract

The invention provides a microscope automatic focusing method and device and a computer readable storage medium, and the method comprises the steps: S1, capturing a laser reflection light spot image on a sample through a camera, so as to determine the focusing position of an objective lens; s2, integrating the light spots through threshold processing and morphological operation, analyzing the gradient of a longitudinal gray distribution map to determine the edge and orientation, and finally identifying the maximum contour and framing the maximum contour with a minimum bounding rectangle to determine the width of the light spots; and S3, according to the linear relation between the light spot width and the defocusing distance and by analyzing the semi-circle orientation and width change of the light spot, controlling the focusing motor to reach the focusing position. The distance and the direction between the objective lens and the focal plane are determined by acquiring the reflected light spot image of the laser irradiated on the sample through the microscope objective lens, and the focusing error caused by the nonlinearity of the spot size when the point-semicircle spot approaches the focal plane is avoided according to the linear relation between the spot width and the defocusing distance. And the focusing process is high in efficiency and precision.

Description

Technical Field

[0001] The present invention relates to the field of microscope focusing technology, and in particular to a microscope automatic focusing method, device and computer-readable storage medium. Background Art

[0002] Traditional autofocus methods face challenges: contrast autofocus methods have problems when searching for focus depth, while laser autofocus methods have problems with uneven spot changes near the focus point.

[0003] For a system with a shallow depth of field such as a microscope, it is particularly difficult to find the focus, especially when the surface of the object to be measured is uneven. Currently, there are two main methods for automatic focusing of microscopes: image contrast search method and laser assisted focusing method.

[0004] The image contrast method collects images within a certain range and calculates the contrast information, and determines the optimal focal plane based on the contrast peak. The laser-assisted focusing method uses the reflected signal of the laser on the sample to determine the distance from the focal plane, including point laser semicircular dispersion and line laser line width.

[0005] However, both the image contrast method and the laser-assisted focusing method have technical defects. The image contrast method relies on the contrast information of the image in autofocus. Its limitation is that it cannot accurately capture the optimal focal plane, especially when the surface features of the sample are not obvious or the contrast is low, it is difficult to accurately determine the focal plane.

[0006] If the point laser method is used in the laser-assisted focusing method, it is difficult to confirm the defocus distance because the light spot is too small when close to the focal plane; if the line laser method is used, it is only possible to quickly determine the defocus distance when near the focal plane, resulting in the problem of not being able to directly determine the defocus direction.

[0007] Therefore, it is necessary to provide a more accurate and efficient focusing strategy to solve the above problems. Summary of the invention

[0008] In view of the deficiencies in the prior art, the object of the present invention is to provide a microscope autofocus method, device and computer-readable storage medium to solve the problems raised in the above background technology.

[0009] In order to achieve the above object, the present invention is implemented by the following technical solution: a microscope autofocus method, comprising the steps of:

[0010] S1. Spot image acquisition

[0011] Use a camera to capture the laser reflection spot image on the sample to determine the focus position of the objective lens;

[0012] S2. Spot image processing and feature extraction

[0013] The light spot image is converted into a binary image containing only two gray values ​​by setting a first threshold, and then a morphological closing operation is performed to integrate the light spot. Then, the gray value of each column of the light spot image is calculated to obtain a longitudinal gray distribution map of light intensity, and its gradient change is analyzed to determine the edge and direction of the light spot. Finally, the edge recognition technology is used to frame the maximum contour, determine the width of the light spot, and output it;

[0014] S3, Defocus distance, direction determination and focus completion

[0015] According to the linear relationship between the light spot width and the defocus distance, the defocus amount between the current objective lens and the focal plane is determined, and then the specific direction of defocus is determined by analyzing the semicircular orientation and width change of the light spot. Finally, the shape of the light spot is identified. When the light spot shape is a semicircular spot, the camera position is adjusted so that the light spot appears as a semicircle when focusing, so as to ensure that the shape of the light spot is accurately captured when focusing. When the light spot shape is a linear spot, the linear relationship between the light spot width and the defocus distance is used to calculate the specific distance that the focus motor needs to move, and control the focus motor to reach the quasi-focus position.

[0016] As a second aspect of the present invention, a microscope autofocus device is proposed, comprising a memory and a processor, wherein the memory comprises a microscope autofocus program, and when the microscope autofocus program is executed by the processor, the above-mentioned microscope autofocus method is implemented.

[0017] As a third aspect of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores executable instructions, and when the executable instructions are executed, the processor is used to execute the microscope autofocus method.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] 1. To solve the problem of focus depth search and uneven spot change near the focus in the traditional focusing method in the shallow depth of field system, the present invention obtains the reflected spot image of the laser irradiated on the sample through the microscope objective lens, determines the distance and direction between the objective lens and the focal plane, and uses the characteristics of the line laser to propose that the defocus amount of the microscope sample can be converted into the spot width of the line spot according to the linear relationship between the spot width and the defocus distance, avoiding the focusing error caused by the nonlinearity of the spot size when the point-semicircle spot is close to the focal plane. At the same time, the light source has strong energy, which can realize the recognizability of a larger range of spots, and accurately move the focus motor to the quasi-focus position, so as to complete the focusing process with high efficiency and high precision;

[0020] 2. In view of the problem that it is difficult to determine the defocus direction and focal plane in the prior art, the present invention adopts an image processing algorithm, including spot binarization processing, morphological closing operation, and edge recognition technology, to accurately extract the features of the spot, and then accurately judge the semicircular direction and width of the spot by calculating the maximum brightness of each column of the spot image and the longitudinal grayscale distribution map of the light intensity, and analyzing the gradient change of the distribution map, thereby realizing the accurate determination of the defocus direction and focal plane;

[0021] 3. In order to cope with different sample surface characteristics and lighting conditions, the present invention accurately controls the scanning step and pause time of the focus motor by dynamically adjusting the laser power, camera exposure time and gain setting, and setting the minimum threshold of the line spot width. Specifically, when the CMOS detects that the center width of the spot reaches a preset threshold, the system determines that the focus is completed, so that the focus system can quickly respond to different focusing requirements and achieve a fast and stable focusing effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The disclosure of the present invention is described with reference to the accompanying drawings. It should be understood that the drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present invention. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0023] Figure 1 Schematic diagram of the overall process of the microscope auto-focusing method proposed in one embodiment of the present invention;

[0024] Figure 2 A schematic diagram of a scanning logic flow of a focus motor proposed in an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of using another light intensity-X coordinate line graph method to calculate the spot width proposed in one embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram of the linear spot width identified when finding the maximum contour in the spot image using the edge recognition technology proposed in one embodiment of the present invention;

[0027] Figure 5 It is a schematic diagram of a spot image after being processed by binarization and morphological closing operation in the process of spot image processing and feature extraction proposed in one embodiment of the present invention;

[0028] Figure 6 It is a schematic diagram of a light spot that is very close to a line light spot when it is difficult to determine the semicircular direction of the light spot according to an embodiment of the present invention;

[0029] Figure 7It is a schematic diagram of the widening of the spot when the lens is away from the focal plane when calculating the width change of the line spot proposed in one embodiment of the present invention;

[0030] Figure 8 This is a schematic diagram of the structure of a focusing device constructed when performing effect verification according to an embodiment of the present invention;

[0031] Fig. 9 This is a linear relationship diagram of line spot width-defocus amount proposed in an embodiment of the present invention, where the horizontal axis is the height unit (10um) of the object being measured, and the vertical axis is the width of the spot;

[0032] Fig.10 A schematic diagram of a semicircular spot when comparing the microscope auto-focusing method proposed in one embodiment of the present invention with the centerline spot focusing method in the prior art;

[0033] Fig.11 The figure is a schematic diagram of the complete process from image acquisition to defocus calculation proposed in one embodiment of the present invention. DETAILED DESCRIPTION

[0034] It is easy to understand that according to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can propose a variety of interchangeable structural modes and implementation modes. Therefore, the following specific implementation modes and drawings are only exemplary descriptions of the technical solution of the present invention, and should not be regarded as the whole of the present invention or as a limitation or restriction to the technical solution of the present invention.

[0035] The present invention is further described in detail below with reference to the accompanying drawings, but is not intended to limit the present invention.

[0036] As an understanding of the technical concept and implementation principle of the present invention, the main purpose of the microscope autofocus method, device and computer-readable storage medium proposed by the present invention is to solve the efficiency problem of the traditional contrast autofocus method in the focus depth search, and the problem of uneven spot change near the focus of the traditional laser autofocus method, especially when the surface of the object to be measured is uneven and there is a certain height difference, to solve the problem of cumbersome and inaccurate traditional methods in finding the focus. Therefore, the present invention obtains the reflected spot image of the laser irradiated on the sample through the microscope objective lens to determine the distance and direction between the objective lens and the focal plane, and then uses the characteristics of the line laser to accurately move the focus motor to the quasi-focus position through the linear relationship between the spot width and the defocus distance. At the same time, a computer image processing algorithm is used, including spot binarization processing, morphological closing operation and edge recognition technology, to accurately extract the spot features, thereby judging the semicircular direction and width of the spot, realizing the accurate determination of the defocus direction and focal plane, and solving the problem of focusing error caused by the unclear change of the spot size and nonlinearity when approaching the focal plane in the prior art.

[0037] As a first aspect of the present invention, Figure 1 - Figure 2 as well as Fig.11 As shown, a microscope autofocus method is proposed, comprising the following steps:

[0038] S1. Acquisition of spot image. The specific process is as follows:

[0039] The camera is used to capture the image of the laser reflected spot on the sample to determine the focus position of the objective lens. The focus position of the objective lens includes the distance and direction between the objective lens and the focal plane.

[0040] It should be noted that, in the specific implementation, a common 785nm or 660nm line laser is used as a light source in one embodiment of the present invention. A filter is set on the microscope camera to avoid the influence of the focusing light source. The line laser is a near-infrared laser for marking the line of mechanical equipment. In this process, it is necessary to adjust the line spot with appropriate light intensity and length through the slit and collimator, and then reflect it to the sample surface observed by the microscope through the beam splitter. It can be understood that due to the sophisticated design of the light source, it reaches the sample surface after 200mm transmission, ensuring that the light spot and the camera focus position are the same. The advantage of this focusing method is that the defocus distance and defocus direction can be determined according to the light spot, and the shortest distance of the focus motor movement can be quickly determined. When approaching the focus position, the light spot appears as a clear thin line. By setting the shortest light spot width threshold, the line laser is irradiated to the CMOS that detects the focus spot after collimation focusing, slit spatial filtering, and beam splitter reflection. Once the CMOS detects that the center of the light spot reaches this threshold, the focusing process can be completed, thereby achieving fast and accurate automatic focusing.

[0041] S2, spot image processing and feature extraction, the specific process includes:

[0042] S2-1, such as Figure 5 - Figure 6 As shown, the spot image is binarized by setting a first threshold value, which is used to convert the spot image into a binary image containing only two gray values, gray value 0 (black) and gray value 255 (white), to adapt to different brightness conditions. The purpose is to simplify the image and facilitate subsequent processing. The specific implementation method is as follows:

[0043] Based on the average gray value of the spot image f mean The adjustment coefficient scale is multiplied by the first threshold th set by the adjustment coefficient scale according to the brightness and contrast of the actual spot image through the formula th=255-(f mean *scale)(1), to adjust to obtain the best binarization effect. When the pixel value of the spot image is greater than the first threshold th, it becomes a grayscale value of 255, and when it is less than the first threshold th, it becomes a grayscale value of 0.

[0044] Since the energy of the light spot will be dispersed during the defocusing process, causing the light spot to become split, it is necessary to perform the following S2-2 image closing operation step to combine the light spot into a complete computer-recognizable light spot.

[0045] S2-2, perform morphological closing operation, first reduce the edge of the object in the spot image by corrosion processing, and then expand the edge by dilation processing, so as to connect the split spot parts in the spot image to form a complete spot shape.

[0046] In specific implementation, according to the characteristics of the spot image and the required processing effect, a matrix of appropriate size is selected as the structural element or structural matrix for erosion and expansion operations. In one embodiment of the present invention, it is preferred to construct a 3X3 or 5X5 structural matrix. After the structural element selection is completed, the erosion process is first performed, that is, the entire spot image pixels are traversed, and the structural element is placed at each pixel position of the spot image. For example, when the (x, y) position is traversed, the structural matrix is ​​placed at the (x, y) position. If the spot image at the pixel position in the area covered by the structural element is all white (that is, the gray value is 255), the pixel value of the spot image is retained, otherwise, the central pixel is set to black (that is, the gray value is 0), and the above steps are repeated until the entire spot image is processed, and an image after the edge of an object is eroded by a layer is obtained, and the erosion process is completed. Secondly, if there is only one white area in the spot image at the pixel position in the area covered by the structural element, the dilation process is performed. It can be understood that the process of dilation is similar to that of corrosion, and the same structure matrix is ​​traversed. The difference is that as long as there is a white area (that is, the gray value is 255) in the effective area of ​​the matrix, the entire matrix area becomes a gray value of 255. After the dilation operation, the edge of the entire target spot image will expand. Combining the corrosion operation with the dilation operation will result in a spot image with unchanged size and internal connectivity.

[0047] Since the light spot is distributed longitudinally, it is also necessary to perform the following step S2-3 of determining the direction of the light spot to analyze the shape and direction of the light spot in order to determine the defocus direction of the objective lens.

[0048] S2-3. Calculate the grayscale value of each column of the spot image to obtain the longitudinal grayscale distribution diagram of the light intensity. It can be understood that the longitudinal grayscale distribution diagram of the light intensity refers to a distribution diagram representing the longitudinal change of the light intensity along the image obtained by adding the grayscale value of each column of the preprocessed spot image. This distribution diagram is used to analyze the shape and direction of the spot so as to determine the defocus direction of the objective lens. In specific implementation, the longitudinal grayscale distribution diagram of the light intensity is obtained by adding the grayscale value of each column of the preprocessed spot image through the formula Addition is used to generate, where f(x) is a function that represents the cumulative value of the light intensity along the longitudinal direction (y direction) of the spot image at a given x-coordinate position, f(x,y k ) is the coordinate (x,y) in the spot image k ) is the gray value of the pixel point, x is a fixed coordinate, y is k is the vertical pixel coordinate, k is the index, indicating the kth pixel row, and then after smoothing, in order to reduce noise, by analyzing the gradient change of the vertical grayscale distribution diagram, that is, the slope between two adjacent points, when the gradient change exceeds the set second threshold, the straight edge of the light spot is determined, and then the semicircular direction of the light spot is determined. In specific implementation, first, for the obtained vertical grayscale distribution diagram of light intensity, a sliding window method is used for smoothing. The specific operation is as follows: a function smooth_curve(data,window_size) is defined, and a sliding window size of wi is used on the data. A window of ndow_size is used to calculate the average value in the window, thereby obtaining the smoothed curve data. Secondly, a second threshold is set, and the gradient value of the smoothed curve data point is calculated by taking the derivative or using the difference method. When the calculated gradient value exceeds the second threshold, it is considered that the x position of the spot image corresponding to the position is the straight edge of the spot. It can be understood that the setting of the second threshold is crucial for the accurate judgment of the direction of the spot. The second threshold needs to be optimized based on the experimental data and the characteristics of the actual spot image. Finally, the semicircular direction of the spot is determined by determining the position of the straight edge.

[0049] S2-4, such as Figure 4 As shown, on the basis of preprocessing the image, it is also necessary to use edge recognition method to obtain the maximum contour in the image, that is, to find continuous boundaries between black and white pixels, and finally form a contour, and frame the contour with the minimum horizontal circumscribed rectangle to determine the spot width and output.

[0050] In the specific implementation, first, edge recognition technology, such as Canny edge detector or Sobel operator, is used to process the spot image and identify edge features; secondly, the findContours function in the OpenCV library is used to detect all contours in the spot image, and the contour with the largest area or the most consistent with specific features is screened out. It can be understood that usually this largest contour represents the main spot; thirdly, for the largest contour, a boundingRect function is used to determine a minimum horizontal circumscribed rectangle surrounding the largest contour. At this time, the width of the circumscribed rectangle is used as the width of the spot; it should be noted that in the above-mentioned contour detection and screening process, a threshold needs to be set to exclude noise or irrelevant small contours. The threshold can be adjusted according to the characteristics of the spot image and environmental conditions. Finally, the detected spot width is verified to ensure that it is consistent with the physical size of the actual spot. If the detection result is inaccurate, the parameters of edge recognition and contour detection are readjusted, or the image preprocessing step is checked. Otherwise, the calculated spot width is used as output to achieve precise focusing. Through the above steps, it can be ensured that the measurement of the spot width is both accurate and reliable, providing key input parameters for the autofocus system.

[0051] In one embodiment of the present invention, it can be understood that in the automatic focusing system, the line spot and the semicircular spot refer to the shapes of the spot in different focusing states, and these two shapes provide different information about the distance and direction between the objective lens and the focal plane. Therefore, it is also necessary to perform the following S3 step to analyze the linear relationship between the line width and the defocus distance, as well as the line width and the spot width and the semicircular direction, to complete the precise control of the focus motor.

[0052] S3, the defocus distance and direction are determined and the focusing is completed. The specific process is as follows:

[0053] S3-1. Determine the defocus amount between the current objective lens and the focal plane based on the linear relationship between the spot width and the defocus distance. It can be understood that by measuring the width of the spot image and using the known linear relationship between the spot width and the defocus distance to calculate the defocus amount between the objective lens and the focal plane, an accurate starting point is provided for the focusing operation. That is, it is necessary to establish the following formula (4) for defocus amount and spot width, and calculate the spot width based on methods such as image binarization, morphological processing, and edge detection. The defocus amount of the current detection object is calculated based on formula (4), specifically, Where: f1 and f2 are the focal lengths of the objective lens and eyepiece, tanθ is a constant, P Zis a constant of the camera pixel size, and d is the spot width. From formula (4), it can be seen that there is a linear relationship between the spot width d detected by the camera and the defocus distance δ, and through the zero point, the linear slope is related to parameters such as the objective lens NA and the camera pixel size. During the focusing process, the defocus distance δ can be quickly calculated based on the spot width d.

[0054] S3-2, then determine the specific direction of defocus by analyzing the semicircular direction and width change of the light spot. It can be understood that this step is to perform morphological analysis on the light spot image and identify the geometric characteristics and change trend of the light spot, so as to infer the offset direction of the objective lens relative to the focal plane, which is crucial for the subsequent focus motor control and ensures that the focus motor can move in the correct direction. The specific implementation method is that the laser emitted by the focus laser is collimated by a collimator, filtered by a slit metal sheet and blocked by a semicircular light shield. When the detection object is in the focal plane of the objective lens, the light spot detected by the focus camera is a linear light spot, that is, the width of the slit; when the detection object is above the focal plane, the light spot detected by the focus camera is a semicircular light spot facing left; when the detection object is below the focal plane, the light spot detected by the focus camera is a semicircular light spot facing right; wherein, if the trend of the semicircular shape of the light spot is not obvious when the detection object is close to the focal plane of the objective lens, it is necessary to adjust according to the width of the light spot. The specific adjustment process is: calculate the system depth of field according to the current objective lens NA and magnification, move the Z-axis motor according to 3 to 5 times the depth of field, and observe the change in the width of the light spot. The direction in which the width decreases is the direction towards the focal plane.

[0055] S3-3, identifying the shape of the light spot, wherein, when the light spot shape is a semicircular light spot, adjusting the camera position so that the light spot appears as a semicircle when focusing, to ensure that the shape of the light spot is accurately captured when focusing. Figure 7 As shown, when the spot shape is a line spot, the linear relationship between the spot width (line width) and the defocus distance is used to calculate the specific distance the focus motor needs to move, and control the focus motor to reach the quasi-focus position.

[0056] Based on the above technical concept, it can be understood that when controlling the focus motor to complete the focus stage, the focus system needs to adjust the scanning step size and pause time of the focus motor according to different exposure times to adapt to the brightness conditions of different light spots and ensure the accuracy of the focusing process. At the same time, it is also necessary to set a minimum light spot width threshold. When the CMOS detects that the center width of the light spot reaches the shortest light spot width threshold, the focus system determines that the focus has been completed. So far, accurate focusing is achieved by quickly determining the shortest distance that the focus motor moves to ensure clear imaging. In specific implementation, when the light spot is close to the line spot, it is difficult for the system to determine the defocus direction of the microscope. Therefore, according to the different models of the microscope, the motor can be driven to move in a small range first. The moving distance can be 3 to 4 times the depth of field distance. For example, the depth of field of the 10X0.25NA objective lens is 8.5um, and the moving distance can be 25.5um. Then, according to the change in the width of the line spot, the direction to move is determined.

[0057] At the same time, Fig.10 As shown in (a), (b), and (c), for a semicircular spot, if the direction of the semicircle cannot be determined based on the acquired spot shape at this stage, its orientation (the direction of the semicircle) and width change provide information about the defocus direction. When the spot approaches the focal plane, its size change is not obvious and does not have a linear relationship. Therefore, the specific process of determining the accuracy of the focusing system to ensure the focusing process is as follows: First, control the focus motor to move a certain distance randomly and then calculate the width change of the line spot, such as Figure 8 As shown, Figure 8 The camera of the structure moves 100um, that is, when the microscope is focused, the light spot is not a point, but a semicircle. Fig.10 As shown in (e)(f)(g)), if the light spot becomes wider, it means that the lens is far away from the focal plane, and if the light spot becomes narrower, it means that the lens is close to the focal plane; then, according to the random moving distance, the direction of the focus motor movement is determined; finally, by setting the shortest light spot width threshold, when it is detected that the light spot width is less than the threshold, it is determined that the focus system has completed the focusing process. The focusing error caused by the nonlinearity of the light spot size when the point-semicircle light spot is close to the focal plane can be avoided. At the same time, the light source has strong energy, and the identifiability of a larger range of light spots can be achieved. At the same time, compared with the line light spot focusing method, the present invention can have an obvious semicircular shape within a large defocus range, and the defocus direction can be quickly determined.

[0058] As an embodiment of the present invention, a verification step for a microscope autofocus method is proposed:

[0059] First, if Figure 8As shown, the system light source uses a 150mw high-power 785nm or 660nm line laser. The line laser is first collimated and then shaped by a slit filter. It is then projected on the first beam splitter, reflected by the second beam splitter and reaches the sample stage. It is then reflected from the sample stage back to the second beam splitter and reflected by the first beam splitter to the CMOS camera.

[0060] Secondly, if Fig. 9 As shown in the figure, it is an xy line graph of the calculated spot width and height information. The lowest point in the center is the focal plane of the microscope (because the optical path is set to 200mm, the slit is conjugate with the microscope plane, so that when the microscope is focused, the spot is also the thinnest line spot). The sample stage height increases from 290um to 650um. The spot width obtained in the process of defocusing, focusing and then defocusing is different. The process from 290um to 490um is called negative defocus, and the process from 490um to 650um is called positive defocus. Fig. 9 It can be seen that the image width changes during negative defocus and positive defocus are linear, so when the spot width at any position is obtained, the current position can be calculated, thereby driving the motor to quickly move to the position of the focal plane.

[0061] Finally, for comparison, if it is a semicircular spot solution, when the spot is close to the focal plane, Fig.10 As shown in (a)(b)(c), the spot size does not change significantly and does not have a linear relationship. A feasible solution is to Figure 8 The camera of the structure moves 100um, that is, when the microscope is focused, the light spot is not a point, but a semicircle. Fig.10 (e)(f)(g) As shown. So far, the nonlinear problem of spot size and defocus distance has been solved, and an obvious advantage is also provided: that is, within a large defocus range, the semicircular shape of the spot can quickly determine the defocus direction, thereby improving the accuracy of focusing. In addition, due to the strong energy of the light source, the present invention can achieve the identifiability of a larger range of spots, thereby enhancing the applicability and robustness of the focusing system under different conditions.

[0062] As an embodiment of the present invention, the microscope focusing method used in the present invention is that when approaching the focal plane, the light spot is in the center of the field of view, and the width is approximately 1 / 20 of the horizontal pixel of the camera. In specific implementation, in order to cope with surfaces with different reflectivity, the laser power and the camera exposure time and gain setting can be dynamically adjusted to achieve a detection light spot with sufficient light intensity. If the reflectivity of different positions of the sample is different, more noise will remain inside the obtained spot image, but the size and width of the spot image will not be affected, and the spot is finally concentrated in the center of the field of view when close to focusing, which does not affect the successful focusing of the system at the center of the field of view. At the same time, in view of the uneven surface of the sample, the xy-axis stage can be moved to focus on the center position multiple times.

[0063] As an embodiment of the present invention, Figure 3 As shown, different from the above-mentioned binarization processing and contour recognition method, the present invention proposes that when calculating the spot width, a light intensity-X coordinate line graph method can also be used. The specific implementation process includes: image preprocessing, performing necessary preprocessing on the acquired spot image, including noise reduction and contrast enhancement, to ensure that the edge and features of the spot are more obvious; brightness value extraction: traversing each column of pixels in the spot image, analyzing all pixel values ​​in each column, and extracting the maximum brightness of each column; constructing a line graph: taking the maximum brightness value of each column as a data point, taking the X coordinate of the column (i.e., the X axis of the image) as the horizontal coordinate, and the maximum brightness (Y axis) as the vertical coordinate, and drawing a light intensity-X coordinate line graph; threshold setting: setting a suitable threshold according to the characteristics of the image and environmental conditions, Used to identify the protruding part of the line graph that represents the width of the light spot; Protruding part identification: In the line graph, identify the X-axis coordinate points corresponding to the brightness values ​​higher than the set threshold, and these points constitute the boundary of the light spot; Light spot width calculation: Calculate the distance between the two outermost points of the protruding part of the line graph (i.e., the left and right edges of the light spot) on the X-axis. This distance is the width of the light spot, that is, calculate the gradient change of the line graph. When the gradient is greater than a certain threshold, it can be judged that the corresponding position of the line graph is the straight edge of the semicircle. After determining the straight edge, if the light spot is distributed on the right side of the straight edge, it is judged as the right semicircle, and if the light spot is distributed on the left side, it is judged as the left semicircle; Data output: The calculated light spot width is used as output for subsequent focusing operations, such as controlling the focus motor to move to the corresponding position.

[0064] It can be understood that, through the above steps, the proposed light intensity-X coordinate line graph method can accurately measure the width of the light spot and provide key parameter input for the autofocus system. It is also applicable to the case of line spots and can quickly and accurately determine the width of the light spot, thereby guiding the precise movement of the focus motor.

[0065] As a second aspect of the present invention, a microscope autofocus device is proposed, including a memory and a processor, wherein the memory includes a microscope autofocus program, and when the microscope autofocus program is executed by the processor, the above-mentioned microscope autofocus method is implemented.

[0066] As a third aspect of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores executable instructions, and when the executable instructions are executed, the processor is used to execute the above microscope autofocus method.

[0067] The technical scope of the present invention is not limited to the contents in the above description. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical idea of ​​the present invention, and these deformations and modifications should all fall within the protection scope of the present invention.

Claims

1. A microscope autofocus method, characterized in that: Included steps S1. Spot image acquisition Use a camera to capture the laser reflection spot image on the sample to determine the focus position of the objective lens; S2. Spot image processing and feature extraction The light spot image is converted into a binary image containing only two gray values ​​by setting a first threshold, and then a morphological closing operation is performed to integrate the light spot. Then, the gray value of each column of the light spot image is calculated to obtain a longitudinal gray distribution map of light intensity, and its gradient change is analyzed to determine the edge and direction of the light spot. Finally, the edge recognition technology is used to frame the maximum contour, determine the width of the light spot, and output it; S3, Defocus distance, direction determination and focus completion According to the linear relationship between the light spot width and the defocus distance, the defocus amount between the current objective lens and the focal plane is determined, and then the specific direction of defocus is determined by analyzing the semicircular orientation and width change of the light spot. Finally, the shape of the light spot is identified. When the light spot shape is a semicircular spot, the camera position is adjusted so that the light spot appears as a semicircle when focusing, so as to ensure that the shape of the light spot is accurately captured when focusing. When the light spot shape is a linear spot, the linear relationship between the light spot width and the defocus distance is used to calculate the specific distance that the focus motor needs to move, and control the focus motor to reach the quasi-focus position.

2. The microscope autofocus method according to claim 1, characterized in that: According to the formula Determine the defocus between the current objective lens and the focal plane, where f1 and f2 are the focal lengths of the objective lens and the eyepiece, respectively, tanθ is a constant, and P Z is a constant of the camera pixel size, and d is the spot width.

3. The microscope autofocus method according to claim 1, characterized in that: The process of analyzing the semicircular direction and width change of the light spot and determining the specific defocus direction is as follows: the laser emitted by the focus laser is collimated by the collimator, filtered by the slit metal sheet, and blocked by the semicircular light shield. When the detection object is in the focal plane of the objective lens, the light spot detected by the focus camera is a linear spot, that is, the width of the slit. When the detection object is above the focal plane, the light spot detected by the focus camera is a semicircular light spot facing left; When the detection object is below the focal plane, the light spot detected by the focus camera is a semicircular light spot facing right; If the semicircular shape of the light spot is not obvious when the detected object is close to the focal plane of the objective lens, it is necessary to adjust the light spot width. The specific adjustment process is: calculate the system depth of field according to the current objective lens NA and magnification, move the Z-axis motor according to the depth of field, and observe the change of the light spot width. The direction in which the width decreases is the direction towards the focal plane.

4. The microscope autofocus method according to claim 1, characterized in that: When controlling the focus motor to complete the focus stage, the focus system needs to adjust the scanning step length and pause time of the focus motor according to different exposure times to adapt to the brightness conditions of different light spots and ensure the accuracy of the focus process. At the same time, a minimum light spot width threshold needs to be set. When the CMOS detects that the center width of the light spot reaches the shortest light spot width threshold, the focus system determines that the focus has been completed. If the direction of the semicircle cannot be determined based on the acquired spot shape at this stage, the specific process of the focusing system to ensure the accuracy of the focusing process is as follows: first, the focus motor is controlled to move randomly for a certain distance and then the width change of the line spot is calculated. If the spot becomes wider, it means that the lens is far away from the focal plane, and if the spot becomes narrower, it means that the lens is close to the focal plane. Then, the direction of movement of the focus motor is determined based on the randomly moved distance. Finally, by setting the shortest spot width threshold, when it is detected that the spot width is less than the threshold, it is determined that the focusing system has completed the focusing process.

5. The microscope autofocus method according to claim 1, characterized in that: The first threshold th is based on the average gray value f of the spot image mean The adjustment coefficient scale is multiplied by the actual brightness and contrast of the spot image to obtain the best binarization effect. The two grayscale values ​​are grayscale value 0 and grayscale value 255. When the pixel value of the light spot image is greater than the first threshold value th, it becomes the grayscale value 255, and when it is less than the first threshold value th, it becomes the grayscale value 0.

6. The microscope autofocus method according to claim 1, characterized in that: Before performing the morphological closing operation, it is necessary to select a matrix of a preset size as a structural element for erosion and expansion operations according to the characteristics of the spot image and the required processing effect. After the structural element selection is completed, the erosion process is performed. The specific operation is as follows: traverse the entire spot image pixels and place the structural element at each pixel position of the spot image. If the spot image at the pixel position in the area covered by the structural element is all white, the pixel value of the spot image is retained, and the above steps are repeated until the entire spot image is processed, and an image is obtained after a layer of the edge of the object is eroded away, and the erosion process is completed; if the spot image at the pixel position in the area covered by the structural element has only one white area, the area covered by the entire structural element is changed to a grayscale value of 255, and an expansion operation is performed to expand the edge of the object in the spot image outward to fill small holes and breaks.

7. The microscope autofocus method according to claim 1, characterized in that: The light intensity longitudinal grayscale distribution diagram is generated by adding the grayscale value of each column of the preprocessed light spot image; The specific process of determining the semicircular direction of the light spot is as follows: first, the obtained longitudinal grayscale distribution diagram of the light intensity is smoothed using the sliding window method, and the specific operation is as follows: define a function smooth_curve(data, window_size), and slide a window with a size of window_size on the data to calculate the average value in the window, thereby obtaining the smoothed curve data; secondly, set a second threshold, and simultaneously calculate the gradient value of the smoothed curve data point. When the calculated gradient value exceeds the second threshold, it is considered that the x position of the light spot image corresponding to the position is the straight edge of the light spot. The second threshold is optimized based on the experimental data and the characteristics of the actual light spot image; finally, by determining the position of the straight edge, the semicircular direction of the light spot is judged.

8. The microscope autofocus method according to claim 1, characterized in that: The specific process of determining the spot width output is as follows: first, the spot image is processed using edge recognition technology and edge features are identified; second, the findContours function is used to detect all contours in the spot image and select the contour with the largest area or the one that best meets the specific features; Again, for the maximum contour, use the boundingRect function to determine a minimum horizontal circumscribed rectangle to surround this maximum contour. At this time, the width of the circumscribed rectangle is used as the width of the light spot. Finally, the detected light spot width is verified to ensure that it is consistent with the physical size of the actual light spot. If the detection result is inaccurate, readjust the parameters of edge recognition and contour detection, or check the image preprocessing steps. Otherwise, the calculated light spot width will be used as output to achieve precise focus.

9. A microscope autofocus device, characterized in that: include: A memory and a processor, wherein the memory includes a microscope autofocus program, and when the microscope autofocus program is executed by the processor, the microscope autofocus method according to claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that: The storage medium stores executable instructions, which, when executed, are used to cause a processor to execute the microscope autofocus method according to any one of claims 1 to 8.

Images