A microscope automatic focusing method and system based on image processing

Abstract

The present invention discloses an image processing-based microscope autofocus method and system. This method treats an image as a collection of local regions, called windows or tiles, each 5 pixels by 5 pixels in size. An evaluation function is applied independently to each window. Taking the center pixel as a reference, the grayscale gradient difference is calculated in the X and Y axes, taking its absolute value. The grayscale gradient difference is also calculated in the main diagonal and secondary diagonal directions, and these differences are used as weighting factors. These calculated grayscale gradient differences are then weighted together, and the calculated values ​​for each window are finally summed to obtain a quantified image sharpness value. This method addresses the time-consuming and inaccurate nature of traditional focusing methods.

Description

Technical Field

[0001] The present invention belongs to the technical field of microscope focusing, and in particular relates to an image processing-based microscope automatic focusing method and system. Background Art

[0002] Optical microscopes, as effective tools for observing the microscopic world, are widely used in biomedicine and scientific research. With the advancement of automation technology, the development of automated inspection systems can significantly reduce manual labor, improve inspection efficiency and accuracy, and save significant time. Automatic focusing of microscopes is a crucial key technology in computer vision-based automated inspection systems.

[0003] For optical microscopes, autofocus refers to the process by which the system adjusts mechanical structures (such as image distance or object distance) to achieve the clearest captured image. Compared to ordinary light microscopes, fluorescence microscopes use more powerful light sources and filter light through two sets of filters: one set is used to control the light reaching the sample, and the other set is used to filter the fluorescence signal from the sample. The light beam reflected from the surface is collected by a specially designed dichroic mirror, objective lens, and filter into a CMOS camera for fluorescence detection.

[0004] Since fluorescence microscopy requires scanning multiple images within the sample volume, this process is time-consuming and increases the risk of sample photobleaching, photodamage, or phototoxicity. Therefore, fluorescence microscopy autofocus must be able to quickly determine the optimal focus plane for each field of view (FOV) in "real time" during image acquisition.

[0005] Image processing-based autofocus processes the sequence of images captured by the image sensor in real time to obtain an evaluation value that can represent the image clarity. The clarity of the image is determined based on the evaluation value, and a corresponding feedback signal is given to drive the motor to control the movement of the stage until the clearest image is obtained, completing the autofocus.

[0006] Image analysis-based passive focusing relies on image analysis algorithms. This approach can often be directly adapted to existing microscope systems and is relatively low-cost, making it more popular. Passive focusing can be divided into three main categories:

[0007] (1) Autofocus based on phase detection; the phase detection method is only applicable to a small number of wide-field microscopes.

[0008] (2) Depth from Defocus (DFD) estimation: Depth from Defocus (DFD) methods require prior knowledge of a clear image of the focused object, compare the current image with the clear image, and estimate the defocus parameters. Another DFD method used in microscopy extracts defocus information through deep learning methods, which is less versatile and only applicable to specific targets.

[0009] (3) Depth-from-focus (DFF). The depth-from-focus method uses image clarity to determine whether focus is achieved. When the maximum image clarity is reached, focus is considered achieved. Therefore, the clarity evaluation function (CEF) is a key research topic in image processing-based autofocus technology.

[0010] Selection of clarity evaluation function: There is a dependency between image clarity and changes in depth of field or focal length. This relationship is called a clarity function, which is characterized by a maximum value at the image focus point. Existing clarity evaluation functions include statistical, frequency domain, and spatial domain types:

[0011] Statistical functions include contrast statistics, image information entropy, and the Vollath cross-correlation function. These functions all obey Shannon's theorem. Larger function values ​​indicate greater image content and can be used as indicators of image clarity. These functions are relatively unbiased, but the calculations are complex, sensitivity is low, and the function curves tend to be too flat, with unclear extremes. This can lead to missed optimal positions during the subsequent focal plane search.

[0012] Frequency-domain functions primarily include the discrete Fourier transform (DFT) and discrete wavelet transform. Using the DFT or wavelet transform, an image is converted from the time domain to the frequency domain. The more high-frequency information an image contains within the frequency domain, the richer the edge information within the ROI region, and the higher the image clarity. This type of function has high sensitivity and can effectively separate high-frequency from low-frequency information. While it is feasible to determine image clarity based on the richness of high-frequency information, some impulse noise is also high-frequency and can disrupt the original, true high-frequency information of the image. Furthermore, the calculations are relatively complex, resulting in poor real-time performance and limited practical application.

[0013] The spatial domain class function evaluation criteria focuses on using the first and second order differentials of the image and extracting edge gradient information using differential operators such as Roberts, Sobel, and Laplacian operators. Compared with blurred images, clear images usually have larger grayscale differences, resulting in higher variance, so the variance (VAR) can be used as the CEF. Images with sharper edges are considered to be clearer. Several CEFs have been proposed, including the summed gradient (GRA), summed Laplace gradient (LAP), squared gradient (SG), Brenner gradient, and Tenengrad gradient. However, traditional CEFs are not ideal solutions for processing various types of sample images because the spatial or frequency domain selection of all CEFs is fixed, and their properties cannot be changed, which will reduce the accuracy.

[0014] Another key technology in autofocus systems is the focus search strategy. In digital image processing-based autofocus systems, focus search involves finding a peak on the autofocus evaluation function curve. Rapidly and accurately determining the focus position determines the accuracy and real-time performance of autofocus. However, existing methods are slow due to the high number of searches and are susceptible to local extreme points. Summary of the Invention

[0015] One of the purposes of the present invention is to provide a microscope autofocus method based on image processing, which improves and optimizes the image clarity evaluation function and the autofocus extreme value search algorithm, aiming to solve the problems of time-consuming and inaccurate focusing of traditional methods.

[0016] First, a focus evaluation function that is more suitable for fluorescence microscopy images is proposed. Compared with other spatial domain functions, the proposed function has higher sensitivity and stability, better unimodality and unbiasedness, and can process images faster.

[0017] Secondly, to avoid interference from local extrema on the focusing curve on autofocus, we proposed an improved hill climbing algorithm to achieve precise search for microscope autofocus. Simulation results show that this method has good results in the field of microscope autofocus and can meet the needs of microscope autofocus.

[0018] The specific technical solutions adopted by the present invention to achieve the above-mentioned purpose are as follows:

[0019] A microscope autofocus method based on image processing considers the image as an integration of local areas, which are called windows or tiles. Each local area is 5 pixels × 5 pixels in size. The evaluation function calculation formula will act independently on each window. Taking the center pixel as the reference, the grayscale gradient difference after taking the absolute value is calculated in the X-axis and Y-axis directions respectively. At the same time, the grayscale gradient difference after taking the absolute value is calculated in the main diagonal and sub-diagonal directions respectively and used as the weight factor. Then, the grayscale gradient differences obtained from these calculations are weighted. Finally, the calculated values ​​of each window are accumulated to obtain the image clarity quantification value.

[0020] The grayscale gradient evaluation function applies a specified convolution template to the image grayscale to characterize the clarity of the image.

[0021] Assume that the size of image I is M×N, the grayscale value of a pixel (x, y) in image I is f(x, y), the quantitative value of image clarity evaluation is F, and the focus evaluation function expression is as follows:

[0022] At x=5k-2, k∈N * , y=5k-2,k∈N * Under the premise that, N* is a set of positive integers, k is the set N * An element value in

[0023]

[0024] In the evaluation function expression: F proposed That is, the evaluation function. The subscript proposed is used to indicate a new evaluation function that is different from other existing evaluation functions. M is the number of pixels in the horizontal direction of the image, N is the number of pixels in the vertical direction of the image, x is the horizontal coordinate of the center pixel of the window, and y is the vertical coordinate of the center pixel of the window. and is the basic value of grayscale gradient statistics, which is the grayscale value of the new pixel after convolution of the central grayscale pixel and its surrounding pixels at 1 (INT) or 2 (EXT) positions on the X-axis or Y-axis with the corresponding differential operator. The subscripts X and Y respectively indicate that the window is currently operating on the X-axis and Y-axis. The superscript INT indicates that the current center pixel of the window is operating with the pixel at 1 adjacent position. The superscript EXT indicates that the current center pixel of the window is operating with the pixel at 2 adjacent positions.

[0025]

[0026] In the above formula, f(x, y) represents the grayscale value of the pixel at the coordinate (x, y), and the symbol is the convolution operator, and the differential operator matrix in the formula is as follows:

[0027]

[0028] Among them, m, n=1, 2, 3, 4, 5, δ ij It is a binary function used to simplify the representation of the differential operator matrix. Its independent variable, i.e., input value, is generally two integers. If the two are equal, the output value is 1, otherwise it is 0.

[0029]

[0030] and is the weight factor of grayscale gradient statistics, which is the grayscale value of the new pixel after convolution of the original grayscale pixel and the adjacent 1 (INT) or 2 (EXT) pixels on the main diagonal (main-diag) or sub-diagonal (anti-diag) with the corresponding differential operator. The subscripts main-diag and anti-diag indicate that the window is currently operating on the main diagonal and sub-diagonal, respectively.

[0031]

[0032]

[0033] The differential operator matrix in the formula is as follows,

[0034]

[0035] The core of the evaluation function proposed in this invention is:

[0036] The image is regarded as an integration of local areas. The local areas can be called windows or tiles. The size of each local area is 5 pixels × 5 pixels. The function calculation formula will act independently on each window. Taking the center pixel as the reference, the grayscale gradient difference S after taking the absolute value is calculated in the X-axis and Y-axis directions respectively. At the same time, the grayscale gradient difference after taking the absolute value is calculated in the main diagonal and sub-diagonal directions and used as the weight factor W. Then these calculated values ​​are weighted. Finally, the calculated values ​​of each window are accumulated to obtain the image clarity quantization value F.

[0037] It is not difficult to find that in the evaluation function of the present invention, all 16 pixels adjacent to the central pixel participate in the function's calculation process. Therefore, the proposed function fully accounts for the anisotropy of the edge direction of the ROI region in the fluorescence image, weakening the function's sensitivity to noise, thereby improving its sensitivity. In addition, the concept of image windowing eliminates the lengthy calculation methods used in traditional methods, further enhancing the real-time performance of the function.

[0038] In this invention, the absolute values ​​of the basic values ​​and weighting factors in the grayscale gradient statistics are taken. This is to prevent negative terms from offsetting the accumulated quantized values ​​of positive terms in the gradient statistics of the entire image. However, negative terms are also part of the grayscale gradient, and failing to take their absolute values ​​would weaken the sensitivity of the entire function. Many traditional focus evaluation functions do not fully consider this point.

[0039] The present invention also proposes an improved focus search strategy, which divides the focus search process into two search stages: coarse adjustment and fine adjustment. The purpose of the coarse adjustment search stage is to quickly find the peak point range, and this stage still uses the traditional hill climbing search algorithm. In the fine adjustment search stage, an improved hill climbing algorithm is used, and the stepper motor drives the stage to move slightly. At this time, the image grayscale changes little, and the evaluation function is easily disturbed by external environmental noise and local extreme values ​​appear. Therefore, this stage starts with continuously shooting three images to obtain their evaluation values ​​to determine the search direction. The peak point range can be quickly determined through the initial large step search, and then in the small step search stage, three images are continuously shot to calculate their clarity evaluation values ​​to determine the search direction, which can effectively avoid falling into local extreme value situations.

[0040] The basic steps for focus search are as follows:

[0041] S1: Determine the moving step of the Z-axis stage by calculating the depth of field D of the current objective magnification, initialize the clarity value at the current position, and initialize the moving direction.

[0042] S2: Enter the coarse adjustment search stage and move the stage in steps of 2D.

[0043] S3: Obtain the current image and evaluate the clarity.

[0044] S4: Update the old clarity value with the current position and clarity.

[0045] S5: Determine whether the clarity value at the current position is less than the corresponding clarity value at the previous position.

[0046] If the clarity value at the current position is less than the corresponding clarity value at the previous position, it indicates that the possible best focus position has been exceeded. It is necessary to continue moving the stage in steps of 2D to collect the clarity value corresponding to the next position. If the clarity value corresponding to the next position is less than the clarity value at the current position, enter step S6; otherwise, continue to execute steps S2 to S5 in sequence.

[0047] S6: The stage retreats and enters the fine search stage.

[0048] S7: Move the stage in smaller steps of D for fine search, and continuously capture three images to obtain their clarity evaluation values F.

[0049] S9: Compare the clarity evaluation values F1, F2, and F3 at three consecutive positions to determine the next moving direction.

[0050] If F1 < F2 < F3, it indicates that the current is in the stage of increasing clarity value, and continue to move the stage in the current direction and enter step S10; if F1 > F2 and F2 < F3, it indicates that the middle value is a local minimum and can be ignored and continue to move in the current direction and enter step S10; if F1 < F2 and F2 > F3, it indicates that the middle value is a possible local maximum, but it cannot be determined as the global maximum, and further exploration is needed. Move in the current direction and enter step S10; if F1 > F2 > F3, it indicates that the maximum point has been passed, and it is necessary to change the direction and make fine adjustments to accurately find the maximum point, and enter step S11.

[0051] S10: Move the stage in the original direction in steps of D, obtain the current new image, evaluate the clarity and update the clarity evaluation values F1, F2, and F3, and enter step S9 for comparison.

[0052] S11: moving the stage in the opposite direction with a step size of 0.5D, continuously capturing three images and obtaining updated clarity evaluation values ​​Fb1, Fb2, and Fb3;

[0053] S12: comparing the clarity evaluation value Fb1, the clarity evaluation value Fb2, and the clarity evaluation value Fb3 at three consecutive positions;

[0054] If Fb1>Fb2>Fb3, it is considered that the global maximum point Fb1 is found, and step S13 is executed; otherwise, step S11 and step S12 are continued in sequence;

[0055] S13: Move the stage to the currently considered global maximum point Fb1, successfully find the focal plane and exit.

[0056] The present invention adopts an optimized search strategy: a staged coarse-tuning and fine-tuning search algorithm is used to quickly locate the peak point range, and the search direction is accurately determined by continuously shooting three images, which can effectively avoid the influence of local extreme values.

[0057] A second objective of the present invention is to provide a microscope focusing system that utilizes the aforementioned focusing method. The system primarily consists of four modules: image acquisition, image analysis, stage control, and result analysis. The image acquisition module captures images using a USB camera connected to an intelligent fluorescence microscope; the image analysis module receives and processes the images, calculating their clarity; the stage control module controls the movement and position of the Z-axis stage via serial communication based on the image analysis results; and the result analysis module determines whether the image has reached optimal clarity and provides feedback based on a search strategy until focus is achieved.

[0058] The focusing system of the present invention is developed using the C++ language, QT framework, and OpenCV image processing software library. It can realize the automated focusing process, integrate complex steps, significantly reduce the time and effort of manual operation, and make the experimental tool more efficient and reliable.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] 1. The present invention introduces a focus evaluation function designed for microscopic images, which is particularly suitable for fluorescence microscopy and has higher sensitivity, stability, and unimodality, ensuring fast and accurate image processing and enhancing the adaptability of the microscope.

[0061] 2. The present invention optimizes the search strategy, adopts a staged coarse-tuning and fine-tuning search algorithm to quickly locate the peak point range, and accurately determines the search direction by continuously shooting three images, effectively avoiding the influence of local extreme values.

[0062] 3. The focusing system of the present invention can realize an automated focusing process, integrate complex steps, significantly reduce the time and effort of manual operation, and make the experimental tool more efficient and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 This is a workflow diagram of a microscope autofocus method based on image processing according to the present invention;

[0064] Figure 2 This is a framework diagram of an image processing-based microscope autofocus system of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solutions and advantages of the present invention more apparent, exemplary embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments of the present invention, and it should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described in the present invention, all other embodiments obtained by those skilled in the art without creative work should fall within the scope of protection of the present invention.

[0066] In the following description, numerous specific details are provided to provide a more thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be practiced without one or more of these details. In other instances, certain technical features well known in the art are not described to avoid confusion with the present invention.

[0067] It should be understood that the present invention can be implemented in different forms and should not be interpreted as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to make disclosure thorough and complete and to fully convey the scope of the present invention to those skilled in the art.

[0068] A microscope autofocus method based on image processing considers the image as an integration of local areas, which are called windows or tiles. Each local area is 5 pixels × 5 pixels in size. The evaluation function calculation formula will act independently on each window. Taking the center pixel as the reference, the grayscale gradient difference after taking the absolute value is calculated in the X-axis and Y-axis directions respectively. At the same time, the grayscale gradient difference after taking the absolute value is calculated in the main diagonal and sub-diagonal directions respectively and used as the weight factor. Then, the grayscale gradient differences obtained from these calculations are weighted. Finally, the calculated values ​​of each window are accumulated to obtain the image clarity quantification value.

[0069] The grayscale gradient evaluation function applies a specified convolution template to the image grayscale to characterize the clarity of the image.

[0070] Assume that the size of image I is M×N, the grayscale value of a pixel (x, y) in image I is f(x, y), the quantitative value of image clarity evaluation is F, and the focus evaluation function expression is as follows:

[0071] At x=5k-2, k∈N * , y=5k-2,k∈N * Under the premise that, N * is a set of positive integers, k is the set N * An element value in

[0072]

[0073] In the evaluation function expression: F proposed That is, the evaluation function. The subscript proposed is used to indicate a new evaluation function that is different from other existing evaluation functions. M is the number of pixels in the horizontal direction of the image, N is the number of pixels in the vertical direction of the image, x is the horizontal coordinate of the center pixel of the window, and y is the vertical coordinate of the center pixel of the window. and is the basic value of grayscale gradient statistics, which is the grayscale value of the new pixel after convolution of the central grayscale pixel and its surrounding pixels at 1 (INT) or 2 (EXT) positions on the X-axis or Y-axis with the corresponding differential operator. The subscripts X and Y respectively indicate that the window is currently operating on the X-axis and Y-axis. The superscript INT indicates that the current center pixel of the window is operating with the pixel at 1 adjacent position. The superscript EXT indicates that the current center pixel of the window is operating with the pixel at 2 adjacent positions.

[0074]

[0075] In the above formula, f(x, y) represents the grayscale value of the pixel at the coordinate (x, y), and the symbol is the convolution operator, and the differential operator matrix in the formula is as follows:

[0076]

[0077] Among them, m, n=1, 2, 3, 4, 5, δ ij It is a binary function used to simplify the representation of the differential operator matrix. Its independent variable, i.e., input value, is generally two integers. If the two are equal, the output value is 1, otherwise it is 0.

[0078]

[0079] and is the weight factor of grayscale gradient statistics, which is the grayscale value of the new pixel after convolution of the original grayscale pixel and the adjacent 1 (INT) or 2 (EXT) pixels on the main diagonal (main-diag) or sub-diagonal (anti-diag) with the corresponding differential operator. The subscripts main-diag and anti-diag indicate that the window is currently operating on the main diagonal and sub-diagonal, respectively.

[0080]

[0081]

[0082] The differential operator matrix in the formula is as follows,

[0083]

[0084] As a preferred embodiment, the focus search process is divided into two search stages: coarse adjustment and fine adjustment. The purpose of the coarse adjustment search stage is to quickly find the peak point range, and this stage still uses the traditional hill climbing search algorithm. In the fine adjustment search stage, an improved hill climbing algorithm is used, and the stepper motor drives the stage to move slightly. At this time, the image grayscale changes little, and the evaluation function is easily disturbed by external environmental noise and has local extreme values. Therefore, this stage starts with continuously shooting three images to obtain their evaluation values ​​to determine the search direction. The peak point range can be quickly determined through the initial large step search, and then in the small step search stage, three images are continuously shot to calculate their clarity evaluation values ​​to determine the search direction, which can effectively avoid falling into local extreme value situations.

[0085] like Figure 1 As shown, the basic steps of focus search are as follows:

[0086] S1: Determine the Z-axis stage movement step by obtaining the depth of field D of the current objective lens magnification, initialize the clarity value of the current position, and initialize the movement direction;

[0087] S2: Enter the coarse search phase and move the stage with a step size of 2D;

[0088] S3: Get the current image and evaluate its clarity;

[0089] S4: Update the old clarity value with the current position and clarity;

[0090] S5: Determine whether the clarity value of the current position is less than the clarity value corresponding to the previous position;

[0091] If the clarity value at the current position is less than the clarity value corresponding to the previous position, it indicates that the possible best focus position has been exceeded. The stage needs to continue moving the stage by a step of 2D to collect the clarity value corresponding to the next position. If the clarity value corresponding to the next position is less than the clarity value at the current position, proceed to step S6; otherwise, continue to execute steps S2 to S5 in sequence;

[0092] S6: The stage retreats and enters the fine search stage;

[0093] S7: Move the stage with a smaller step of D for fine search, and continuously capture three images to obtain their clarity evaluation values F;

[0094] S9: Compare the clarity evaluation values F1, F2, and F3 at three consecutive positions to determine the next movement direction;

[0095] If F1 < F2 < F3, it indicates that the current is in the stage of increasing clarity value, and continue to move the stage in the current direction and proceed to step S10; if F1 > F2 and F2 < F3, it indicates that the middle value is a local minimum and can be ignored, and continue to move in the current direction and proceed to step S10; if F1 < F2 and F2 > F3, it indicates that the middle value is a possible local maximum, but it cannot be determined as the global maximum, and further exploration is needed. Move in the current direction and proceed to step S10; if F1 > F2 > F3, it indicates that the maximum point has been passed, and the direction needs to be changed and fine-tuned to accurately find the maximum point, and proceed to step S11;

[0096] S10: Move the stage in the original direction with a step of D, obtain the current new image, evaluate the clarity, and update the clarity evaluation values F1, F2, and F3, and enter step S9 for comparison;

[0097] S11: Move the stage in the reverse direction with a step of 0.5D, continuously capture three images, and obtain the updated clarity evaluation values Fb1, Fb2, and Fb3;

[0098] S12: Compare the clarity evaluation values Fb1, Fb2, and Fb3 at three consecutive positions;

[0099] If Fb1 > Fb2 > Fb3, it is considered that the global maximum point Fb1 has been found, and execute step S13; otherwise, continue to execute steps S11 and S12 in sequence;

[0100] S13: Move the stage to the position of the currently considered global maximum point Fb1, successfully find the focal plane, and exit.

[0101] Next, we design and implement a systematic test plan to verify the technical effect of the microscope autofocus method.

[0102] Experimental Example 1 compares the performance of the improved function with traditional functions in fluorescence image focus evaluation to assess whether the proposed focus evaluation function can provide higher clarity, sensitivity, unimodality, and processing speed. The proposed function was tested alongside the Brenner function, Roberts function, Sum of Modified Differences (SMD), Energy of Gradient (EOG), Sobel function, Brenner2d function, Improved-Tenengrad function, Bre2d_Rob function, and SMD_MRob function. Through a series of quantitative and qualitative analyses, it was demonstrated that the improved function can significantly improve the accuracy and efficiency of the autofocus system.

[0103] Experimental Example 2 verifies the effectiveness of the improved hill climbing algorithm in microscope autofocus, specifically its ability to quickly locate the peak point range through a staged coarse and fine search strategy, and tests whether the algorithm can effectively avoid the influence of local extreme values, thereby improving focusing accuracy and efficiency.

[0104] Experimental preparation: image dataset construction.

[0105] To ensure the reliability and wide applicability of the experimental results, a diverse sample image library with different step lengths was constructed. This dataset was captured by an intelligent fluorescence microscope and covers the following settings:

[0106] Step size: Three different stage movement step sizes of 2D, D and 0.5D are used.

[0107] Number of samples: 12 independent sets of sample data were taken under each step length condition.

[0108] Contents of each sample set: Each sample set includes 21 consecutive fluorescence images, which gradually change from a defocused state to the clearest state (the 11th image is in focus), and then gradually return to a defocused state.

[0109] Experimental instrument: intelligent fluorescence microscope.

[0110] The microscope used in this experiment is equipped with a motorized stage, allowing precise sample movement in set steps, simulating the various focusing requirements encountered in real-world applications. The motorized turret allows the host computer to switch between different objectives and calculate the depth of field (D), thereby determining the Z-axis stage movement step size. Furthermore, the microscope's built-in high-performance camera ensures the capture of high-quality fluorescence images.

[0111] Experimental Example 1:

[0112] The performance of different functions is tested using different image datasets, including the clarity ratio R, sensitivity S, number of local peaks α, focus search interval β, and algorithm time t.

[0113] 1. Preprocessing stage

[0114] Calibrate the microscope to ensure it is in optimal working condition. Set the experimental parameters, including step size, exposure time, etc.

[0115] 2. Image acquisition

[0116] For each sample, the stage is moved sequentially at a specified step size, and a fluorescence image is captured at each position. This ensures consistent lighting conditions before and after each capture, minimizing the impact of external factors on experimental results.

[0117] 3. Data processing

[0118] The focus evaluation function mentioned above is used to calculate the evaluation value of each image. For each set of samples, the following five key performance indicators are recorded and analyzed:

[0119] Sharpness ratio R: The higher the sharpness ratio, the easier it is for the evaluation function to distinguish between in-focus and out-of-focus images.

[0120] Sensitivity S: reflects the sensitivity of the function to small changes.

[0121] Number of local peaks α: Count the number of local extreme points.

[0122] Focus search interval β: Determines the effective search range.

[0123] Algorithm time t: measures the time required to calculate the evaluation value.

[0124] 4. Data Analysis

[0125] All data is organized into a table to facilitate intuitive comparison of the performance of different functions.

[0126] Calculate the average value of each indicator under each step condition to evaluate the consistency and stability of the function. Conclusion of Experiment 1:

[0127] By quantifying it according to the performance evaluation indicators, we can get the following table:

[0128]

[0129] Through a detailed analysis of a large amount of experimental data under different step length conditions, the following conclusions can be drawn: the focus evaluation function of the present invention shows better performance in most cases, especially in terms of clarity ratio, sensitivity and processing speed.

[0130] The evaluation function of the present invention can effectively reduce the number of local peaks and improve the accuracy of focus search. The function has strong anti-interference ability and high stability.

[0131] In general, the evaluation function of the present invention not only improves the speed and accuracy of autofocus, but also enhances the stability and robustness of the system.

[0132] Experimental results show that the evaluation function of the present invention is particularly suitable for fluorescence microscopy applications requiring high resolution and fast processing.

[0133] Experimental Example 2:

[0134] Three samples A, B, and C corresponding to a certain image are selected to represent light, medium, and heavy defocus states, respectively, to facilitate the simulation algorithm's logic of changing the step size during the focusing process.

[0135] Parameter setting: The initial step size is set to 2D, representing a severe defocus state.

[0136] 1. Coarse search stage: After each movement, two images are continuously collected and their clarity evaluation values ​​are calculated.

[0137] 2. Fine-tuning search stage: When the specific conditions of the invention are met, the search direction is changed and the fine-tuning search stage is entered. The moving step size is changed to D, three images are continuously collected and the clarity evaluation value is calculated.

[0138] 3. Termination condition: If the condition is further met, the step size is reduced to 0.5D and the search direction is adjusted until the clearest image is achieved and the focusing process is stopped.

[0139] Data Recording and Analysis: For this sample, the clarity ratings of all images, along with their corresponding position information and movement steps, were recorded in detail throughout the entire process. Focus speed, accuracy, and stability were analyzed and compared.

[0140] The initial step size is set to 2D and the coarse adjustment search phase is directly entered. In this phase, two images are continuously collected and the clarity evaluation value is calculated. When a local maximum point is encountered during the third step size, the algorithm of the present invention effectively avoids the local maximum point. The fourth step size is continued for comparison. After the local maximum point is determined, step S2 is entered.

[0141] When moving the 11th step length, a maximum point appears in the comparison, indicating that the possible best focus position has been exceeded. At this time, to avoid collecting the next image for the local extreme point and continuing the comparison, see step S5;

[0142] At this time, if the specific conditions of the present invention are met, that is, after comparison in step S5, F2 < F1 indicates that the extreme point position has been crossed. When the 13th step length is implemented to retreat, when moving to the 14th step length, the search direction is changed and the fine-tuning search stage step S7 is entered. The moving step length is changed to D, and three images are continuously collected and the clarity evaluation value is calculated;

[0143] Compare the clarity evaluation values F1, clarity evaluation value F2, and clarity evaluation value F3 at three consecutive positions to determine the next moving direction. F1 > F2 > F3 indicates that the maximum point has been passed. When moving the 16th step length, change the direction and reduce the step length for fine-tuning to accurately find the maximum point, and enter step S11;

[0144] Move the stage in the reverse direction with a step length of 0.5D, continuously take three images and obtain the updated clarity evaluation values Fb1, clarity evaluation value Fb2, and clarity evaluation value Fb3 until moving the 21st step length. If Fb1 > Fb2 > Fb3, it is considered that the global maximum point Fb1 has been found, and execute step S13: move the stage to the position of the currently considered global maximum point Fb1, successfully find the focal plane and exit.

[0145] The moving steps and clarity evaluation values are shown in the following table:

[0146]

[0147]

[0148]

[0149] The traditional hill climbing algorithm finds the best focus by setting the initial scanning direction and search step length, gradually acquiring images and calculating the focus evaluation function value. However, when encountering a situation where the focus evaluation value decreases, it simply reverses the direction and reduces the step length, which makes it easy to stagnate at local extreme positions such as "-14D" and misjudge it as the optimal solution, resulting in autofocus failure.

[0150] There is also a focus search method using the minimum step global search in the prior art. Although it can ensure finding the global optimal solution, its efficiency is low. It takes 41 steps to complete the search within 20D. In contrast, the method of the present invention improves the retrieval efficiency by about twice.

[0151] The above application experiments demonstrate that the improved hill-climbing algorithm can achieve focus in a shorter time, demonstrating significant advantages, especially under severe defocus conditions. Regardless of the initial defocus state, the algorithm effectively avoids interference from local extrema and stably locates the global maximum sharpness point, ensuring accurate focus. Compared to traditional methods, the proposed algorithm reduces unnecessary searches and improves overall efficiency by alternating between appropriately designed large and small step sizes.

[0152] The improved hill climbing algorithm of the present invention not only improves the speed and accuracy of microscope autofocus, but also enhances the reliability and adaptability of the system. The algorithm is an effective solution and can be applied to actual microscope autofocus systems.

[0153] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A microscope autofocus method based on image processing, characterized in that: The image is regarded as an integration of local regions, which are called windows or tiles. Each local region has a size of 5 pixels × 5 pixels. The evaluation function calculation formula will act independently on each window. Based on the central pixel point, the absolute value of the gray - level gradient difference is calculated in the X - axis and Y - axis directions respectively. At the same time, the absolute value of the gray - level gradient difference is calculated in the main diagonal and sub - diagonal directions respectively and used as a weight factor. Then, the weighted operation is performed on the gray - level gradient differences obtained from these calculations. Finally, the calculated values of each window are accumulated to obtain the clarity quantization value of the image. The gray - level gradient evaluation function applies a specified convolution template to the image gray - level to characterize the clarity of the image. Assume that the size of image I is M×N, the gray - level value of a certain pixel point (x, y) in image I is f(x, y), and the quantization value of image clarity evaluation is F. The expression of the focus evaluation function is as follows: At x=5k-2, k∈N * , y=5k-2,k∈N * Under the premise that, N * is a set of positive integers, k is a set N * An element value in In the evaluation function expression: F proposed That is, the evaluation function. The subscript proposed is used to indicate a new evaluation function that is different from other existing evaluation functions. M is the number of pixels in the horizontal direction of the image, N is the number of pixels in the vertical direction of the image, x is the horizontal coordinate of the center pixel of the window, and y is the vertical coordinate of the center pixel of the window. and is the basic value of grayscale gradient statistics, which is the grayscale value of the new pixel after convolution of the central grayscale pixel and its surrounding pixels at one or two adjacent positions on the X-axis or Y-axis with the corresponding differential operator. The subscripts Y and Y indicate that the window is currently operating on the X-axis and Y-axis respectively. The superscript INT indicates that the current center pixel of the window is operating with the pixel at one adjacent position. The superscript EXT indicates that the current center pixel of the window is operating with the pixel at two adjacent positions. In the above formula, f(x, y) represents the grayscale value of the pixel at the coordinate (x, y), and the symbol is the convolution operator, and the differential operator matrix in the formula is as follows: Among them, m, n=1, 2, 3, 4, 5, δ ij It is a binary function used to simplify the representation of the differential operator matrix. Its independent variable, i.e., input value, is generally two integers. If the two are equal, the output value is 1, otherwise it is 0. and is the weight factor of grayscale gradient statistics, which is the grayscale value of the new pixel after convolution of the original grayscale pixel and the adjacent 1 or 2 pixels on the main diagonal or sub-diagonal line with the corresponding differential operator. The subscripts main-diag and anti-diag indicate that the window is currently operating on the main diagonal and sub-diagonal lines, respectively. The differential operator matrix in the formula is as follows:

2. The microscope autofocus method based on image processing according to claim 1, characterized in that: The focus search process is divided into two search stages: coarse adjustment and fine adjustment. The purpose of the coarse - adjustment search stage is to quickly find the peak point range. In this stage, the traditional hill - climbing search algorithm is used; in the fine - adjustment search stage, an improved hill - climbing algorithm is used. The steps of focus search are as follows: S1: Calculate and determine the moving step of the Z - axis stage by obtaining the depth of field D of the current objective magnification, initialize the clarity value of the current position, and initialize the moving direction. S2: Enter the coarse - adjustment search stage and move the stage in steps of 2D. S3: Obtain the current image and evaluate its clarity. S4: Update the old clarity value with the current position and clarity. S5: Determine whether the clarity value of the current position is less than the clarity value corresponding to the previous position. If the clarity value of the current position is less than the clarity value corresponding to the previous position, it indicates that the possible best focus position has been exceeded. It is necessary to continue to move the stage in steps of 2D to collect the clarity value corresponding to the next position. If the clarity value corresponding to the next position is less than the clarity value of the current position, enter step S6; otherwise, continue to execute steps S2 to S5 in sequence. S6: The stage retreats and enters the fine - search stage. S7: Move the stage with a smaller step D for fine - search, and continuously capture three images to obtain their clarity evaluation values F. S9: Compare the clarity evaluation values F1, F2, and F3 of three consecutive positions to determine the next moving direction. If F1 < F2 < F3, it indicates that the current is in the stage of increasing clarity value, and continue to move the stage along the current direction and enter step S10; if F1 > F2 and F2 < F3, it indicates that the middle value is a local minimum, ignore it and continue to move along the current direction and enter step S10; if F1 < F2 and F2 > F3, it indicates that the middle value is a possible local maximum, but it cannot be determined as the global maximum, and further exploration is needed. Move along the current direction and enter step S10; if F1 > F2 > F3, it indicates that the maximum point has been passed, and it is necessary to change the direction and fine - adjust to accurately find the maximum point, and enter step S11. S10: Move the stage in the original direction with a step size D to obtain the current new image, evaluate the clarity and update the clarity evaluation value F1, clarity evaluation value F2 and clarity evaluation value F3, and proceed to step S9 for comparison; S11: moving the stage in the opposite direction with a step size of 0.5D, continuously capturing three images and obtaining updated clarity evaluation values ​​Fb1, Fb2, and Fb3; S12: comparing the clarity evaluation value Fb1, the clarity evaluation value Fb2, and the clarity evaluation value Fb3 at three consecutive positions; If Fb1>Fb2>Fb3, it is considered that the global maximum point Fb1 is found, and step S13 is executed; otherwise, step S11 and step S12 are continued in sequence; S13: Move the stage to the currently considered global maximum point Fb1, successfully find the focal plane and exit.

3. A focusing system using the microscope autofocus method based on image processing according to claim 1 or 2, characterized in that: It includes an image acquisition module, an image analysis module, a stage control module and a result analysis module.

4. The focusing system according to claim 3, wherein: The image acquisition module acquires images through a USB camera connected to the intelligent fluorescence microscope; the image analysis module receives and processes the images and calculates the clarity; the stage control module controls the movement of the Z-axis stage and adjusts its position through serial communication based on the results of the image analysis; The result analysis module determines whether the image has reached the clearest state and gives a feedback signal according to the search strategy until the focus is successfully achieved.

5. The focusing system according to claim 4, wherein: The focusing system uses C++ language, QT framework and OpenCV image processing software library.

Images