A fast autofocus method for electron microscope based on image processing

Abstract

The present invention proposes a fast automatic focusing method for an electron microscope based on image processing, which can perform automatic focusing efficiently, in real time and accurately. Specifically, a small number of images at different focal lengths are collected, the images are preprocessed, the image clarity evaluation function is improved, binary search and curve fitting are combined, the corresponding relationship between image clarity and focal length is obtained, a Gaussian curve is fitted, and the focal length corresponding to the maximum clarity is selected to complete automatic focusing.

Description

Technical Field

[0001] The invention belongs to the field of instruments and meters, and in particular relates to an automatic focusing method for an electron microscope. Background Art

[0002] A scanning electron microscope (SEM) is a large precision instrument used for high-resolution micro-area morphology analysis. It is mainly used to observe the surface morphology and structure of samples. It has become an important means of studying the microstructure of the body and has been widely used in microscopic research in the fields of life sciences, physics, chemistry, justice, earth sciences, materials science, and industrial production.

[0003] Autofocus is a key technology of fully automatic scanning electron microscopes, which determines the accuracy and real-time performance of microscope imaging. Common autofocus methods in electron microscopes are divided into active autofocus and passive autofocus. Active autofocus relies on specific sensors or distance measuring devices. It determines the distance between the object and the focusing device by actively emitting light waves and receiving reflected light waves, so as to achieve the focusing purpose; passive autofocus determines the focusing direction and focusing position by analyzing the image information at each position to achieve the focusing purpose. Passive autofocus is the most commonly used autofocus method for electron microscopes because of its simple structural requirements and low cost.

[0004] The performance of passive autofocus depends on the image clarity evaluation function and the performance of the focus search algorithm. The clarity evaluation function must meet the characteristics of single peak, fast speed and strong anti-interference.

[0005] The evaluation of image clarity includes methods based on image grayscale and methods based on frequency domain. A clear image has more details and more grayscale value changes than a blurred image, so the image grayscale distribution can be used as a standard for evaluating image clarity. Functions for evaluating image clarity based on image grayscale include Sobel gradient function, Laplace gradient function, full image variance function, etc. The clarity of an image is reflected in the frequency domain as the size of the high-frequency component. The frequency domain-based method uses discrete Fourier transform or discrete cosine transform to convert the spatial domain image into a frequency domain image, then extracts and analyzes the high-frequency information to obtain the image clarity.

[0006] According to the value of the clarity evaluation function, the focus search algorithm controls the electron beam through the electron microscope to focus on the position where the clarity evaluation function is the largest, thereby achieving focus. Currently, the commonly used methods include global search method, hill climbing search method and function approximation method.

[0007] The global search method, also known as the exhaustive search method, is the simplest and easiest to implement of all focus search methods. This method starts from one end point of the search and moves toward the other end point with a fixed step size. With each step, the clarity of the position is calculated. After the search is completed, the position with the highest clarity is the focus position to be found.

[0008] The hill climbing algorithm is the most widely used focus search algorithm on the market. The hill climbing algorithm determines the next search direction by comparing the clarity of images at different focus positions. When focusing starts, a larger step size is set, and the clarity of the image is evaluated while acquiring the image. When the clarity of the new image is less than the previous value, the direction is changed and the step size is reduced until the peak position is found.

[0009] The function approximation method finds the optimal focus position through curve fitting. Its basic principle is: given the evaluation values ​​of several points on the evaluation function curve, a second-order or third-order function is used to fit the evaluation values ​​of these points, and finally the focus position is obtained according to the extreme points of the fitting curve.

[0010] The existing technologies have the following main disadvantages:

[0011] The image clarity evaluation function is easily affected by image brightness and image noise. The brightness of the scanning electron microscope image changes with the distance from the focus point. At the same time, in order to focus quickly, the dwell time of a single pixel is generally shorter, and the image noise is larger than that of a normal image, resulting in inaccurate evaluation function results and easy focus on the wrong position. The evaluation function of the frequency domain transform has a large amount of calculation and is not suitable for occasions with high real-time requirements.

[0012] The global search algorithm needs to traverse the entire journey, which is inefficient. The hill climbing search algorithm can shorten the focusing time, but it still needs to collect more images. At the same time, the local peaks caused by environmental noise make the algorithm prone to misfocusing. The function approximation method collects fewer points and can significantly improve the focusing speed, but it is highly dependent on the data around the extreme points and is rarely used in practice. Summary of the invention

[0013] In view of this, the present invention proposes a fast auto-focusing method for an electron microscope based on image processing, which can perform auto-focusing efficiently, in real time and accurately.

[0014] The specific technical solutions are as follows:

[0015] A fast autofocus method for an electron microscope based on image processing is proposed. A small number of images at different focal lengths are collected, and the images are preprocessed to improve the image clarity evaluation function. The corresponding relationship between image clarity and focal length is obtained by combining binary search and curve fitting. A Gaussian curve is fitted, and the focal length corresponding to the maximum clarity is selected to complete autofocus.

[0016] Further, the following steps are included:

[0017] Step 1, calculate the search range of focal length;

[0018] Step 2: Automatically adjust exposure parameters to obtain image A that reaches target brightness;

[0019] Step 3, search the imaging range and determine a focal length Fi within the imaging range;

[0020] Step 4: Fit the clarity curve, select the focal length corresponding to the maximum clarity, and complete autofocus.

[0021] Furthermore, the focal length search range is [f-1, f+0.2], where f is equal to the distance between the upper surface of the stage and the pole shoe minus the thickness of the sample.

[0022] Furthermore, step 2 includes first setting the focal length and target brightness of the image, with the focal length being set to the middle value of the entire search range, and then adjusting the exposure parameters of the electron microscope under the condition that the focal length remains unchanged to obtain image A that reaches the target brightness.

[0023] Furthermore, step 3 includes, on the basis of the exposure parameters automatically obtained in step 2 remaining unchanged, using a binary search method to set different focal lengths within the focal length search range, obtaining N images B at different focal lengths, and performing brightness alignment on the obtained image B with the image A obtained in step 2 to obtain a brightness aligned image B', wherein the brightness alignment is used to ensure that the images are relatively clear at the same brightness; when the i-th brightness aligned image B' i When the clarity of the image exceeds the set threshold relative to the clarity of other brightness-aligned images, it means that the i-th brightness-aligned image B' i The focal length is within the imaging range. At this time, the binary search ends. At this time, the i-th brightness aligned image B' i The focal length is recorded as Fi.

[0024] Further, the comparison method of clarity is,

[0025] The brightness aligned image B' is subjected to different degrees of rapid noise reduction, wherein the i-th brightness aligned image B' i There are four specific methods for fast noise reduction: iThe adjacent pixels are 4-in-1, 9-in-1, 16-in-1, 25-in-1, i=1, ..., N;

[0026] Each brightness-aligned image and its image after rapid noise reduction form a group of images C, where the i-th brightness-aligned image B' i and B' i The i-th group of images obtained by fast denoising is denoted as C i ;

[0027] Calculate C according to the focal length relationship when the image is taken. i and C i+1 The change value of the image clarity includes: the i-th image B' after brightness alignment i After 4-in-1, 9-in-1, 16-in-1, and 25-in-1, four denoised images are generated. These four denoised images and the image B' after brightness alignment of the i-th image i The i-th group of images is formed, and the i+1-th group of images is formed, and B' is compared respectively. i and B' i+1 The clarity of the two images produced by 4-in-1, the clarity of the two images produced by 9-in-1, the clarity of the two images produced by 16-in-1, and the clarity of the two images produced by 25-in-1.

[0028] Furthermore, the brightness alignment method is specifically to multiply the image brightness of image B by an adjustment coefficient α, wherein the brightness adjustment coefficient α is calculated by the following formula: min(A1-αB1) 2 , where A1 represents the brightness of image A, B1 represents the brightness of image B, and N≥3.

[0029] Furthermore, step 4 includes that the clarity curve of the image has a single peak, and the search is continued with a fixed step size along the direction of increasing clarity based on the focal length Fi until the clarity drops for the first time, and at least 4 focal lengths and their corresponding clarity have been searched. At this time, the search ends, and the clarity curve is fitted using these 4 focal lengths and their corresponding clarity, and the curve fitting adopts Gaussian distribution.

[0030] Beneficial Effects

[0031] 1. The present invention reduces the impact of image brightness by aligning image brightness;

[0032] 2. The present invention uses multiple images with different noise reduction levels for clarity calculation at the same time, avoiding noise reduction models and complex noise reduction algorithms.

[0033] 3. The present invention completes a rapid search of the imaging range through dichotomy and clarity, and finds the global maximum point through the local optimum (i.e., a point with the relatively maximum clarity). The global maximum point corresponds to the focal length, thereby quickly completing the search for points in the imaging area. The present invention is suitable for occasions with high real-time requirements.

[0034] 4. The present invention adopts Gaussian curve for fitting, which is more in line with the distribution law of the clarity curve and has higher accuracy.

[0035] 5. The present invention can complete automatic focusing by only collecting a small number of images at different focal lengths, which is simple to implement and convenient to operate. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 , flow chart of the method of the present invention;

[0037] Figure 2 , schematic diagram of electron microscope stage and pole shoe;

[0038] Figure 3 , Flowchart of automatic adjustment of exposure parameters;

[0039] Figure 4a , schematic diagram of 4-in-1 adjacent pixels;

[0040] Figure 4b , schematic diagram of 9 adjacent pixels in 1;

[0041] Figure 4c , schematic diagram of 16 adjacent pixels in 1;

[0042] Figure 4d , schematic diagram of adjacent pixels 25-in-1;

[0043] Figure 5 , Schematic diagram of the electron microscope imaging area;

[0044] Figure 6 , schematic diagram of imaging area search;

[0045] Figure 7 , schematic diagram of Gaussian function curve fitting results. DETAILED DESCRIPTION

[0046] A fast autofocus method for an electron microscope based on image processing collects a small number of images at different focal lengths, preprocesses the images, improves the image clarity evaluation function, combines binary search and curve fitting, obtains the corresponding relationship between image clarity and focal length, fits a Gaussian curve, and quickly completes focusing.

[0047] The present invention adopts the methods of obtaining electron microscope parameters, automatically adjusting exposure parameters, image noise reduction, image clarity calculation, imaging area search, and clarity curve fitting for focusing the scanning electron microscope. The main process is shown in the figure below.

[0048] 1. Calculate the search range for focal length

[0049] The imaging depth of field of the electron microscope is very shallow, and the step length of the focus search can only be set to a relatively small value. The step length is generally set manually and is related to the sample being photographed. It is set between 10 and 100 microns, and there will be some fine-tuning according to the shooting parameters. Quickly determining the initial value of the search can effectively increase the focusing speed. Obtaining the distance D between the upper surface of the stage and the pole shoe can quickly determine the search range of the focal length. The focal length is the distance from the pole shoe to the surface of the sample. The sample is placed on the stage, so the distance D minus the thickness of the sample (assuming 0.3mm) is the focal length, f=D-0.3. However, the thickness of the sample and whether the sample is horizontal are not accurate, so this value f is only an estimate, and then the search range [f-1,f+0.2] is determined by this value. The focal length must be within this range, and then this distance is divided into N parts, and the length of each part is the step length.

[0050] 2. Automatically adjust exposure parameters to obtain image A that reaches the target brightness

[0051] The scanning electron microscope needs to adjust the exposure parameters first to ensure that the image with sufficient clarity is obtained. First, set the focal length to the middle value of the entire search range and the target brightness of the image, and then adjust the exposure parameters of the electron microscope to make the brightness of the acquired image close to the target brightness. The process is as follows: Figure 3 shown.

[0052] Many electron microscope parameters need to be set to obtain images. After the sample is determined, they are rarely adjusted. However, the voltage of the sensor and the signal amplification factor related to the image brightness need to be adjusted every time you shoot. The adjustment goal is to set a suitable brightness (for example, the average brightness value is 160, assuming the maximum brightness is 255), and then adjust the voltage and amplification factor to achieve this brightness.

[0053] 3. After automatically adjusting the exposure parameters, search the imaging range

[0054] On the basis of keeping the exposure parameters automatically obtained in step 2 unchanged, within the focal length search range, a binary search method is used to set different focal lengths to obtain N images B at different focal lengths. The obtained image B is brightness aligned with the image A obtained in step 2 to obtain a brightness aligned image B'. Brightness alignment is used to ensure that the images are relatively clear at the same brightness. When the i-th brightness aligned image B' iWhen the clarity of the image exceeds the set threshold relative to the clarity of other brightness-aligned images, it means that the i-th brightness-aligned image B' i The focal length is within the imaging range. At this time, the binary search ends. At this time, the i-th brightness aligned image B' i The focal length is recorded as Fi.

[0055] Regarding image clarity: the calculation of image clarity needs to take into account the change in image brightness and the impact of image noise. Brightness will cause overall changes in clarity, and the focal length corresponding to the maximum clarity will shift. Image noise will reduce the difference in direct image clarity and even cause the calculation method of clarity to fail.

[0056] The brightness of electron microscope images will vary with the depth of field. Images collected at different focal lengths have brightness differences. Before calculating the clarity, brightness alignment is required to compare the clarity of two images at the same brightness. Taking the first collected image A as a reference, solve the brightness adjustment coefficient α of each collected image B so that (A1-αB1) 2 Minimize the impact of brightness differences on clarity.

[0057] Image noise is related to the dwell time of the image, the beam current size, the acceleration voltage, the imaging sensor, and the amplifier circuit. The usual practice is to calculate the noise intensity model based on these parameters, and then use some noise reduction algorithms such as non-local mean, guided filtering, BM3D, deep learning, etc. to reduce noise. There are two problems with this. First, the model calculated based on the noise intensity is complex and the calculation result is not accurate. Second, the noise reduction algorithm is time-consuming, and the image after noise reduction cannot achieve the desired effect.

[0058] The present invention adopts a method of calculating the average of multiple groups of regions, which can effectively avoid the estimation of the noise model, and then takes the clarity of the best group of images calculated to fit the curve, as follows:

[0059] The image B' after brightness alignment is used as the original image. The images used to calculate the clarity include the original image, and five groups of images, namely, 4-in-1, 9-in-1, 16-in-1, and 25-in-1, of the adjacent pixels of the original image. Among them, 4-in-1, 9-in-1, 16-in-1, and 25-in-1 are equivalent to performing different degrees of rapid noise reduction on the original image. The clarity of the five groups of images is calculated respectively. The clarity adopts the Sobel operator, and the statistical average is the clarity of the image. Each time, five different images are compared with the previous five images. Due to the influence of noise and excessive noise reduction, the clarity change of images with strong noise and over-blurred images is relatively small. The group with the largest change in clarity is the most suitable group for noise reduction.

[0060] Regarding the search for the imaging area: the imaging depth of the electron microscope is very shallow. In the entire search range of the focal length, there is imaging only at a part of the focal length. At other focal lengths, the image has basically no texture, only noise in the image. The present invention needs to quickly find the effective imaging area, and then perform a secondary search within the area to avoid excessive searches in invalid areas.

[0061] The binary search method is used here. According to the binary search, different focal lengths are set each time, images are collected, and the clarity of the image is calculated. When the clarity of a point collected changes significantly compared to the clarity of the points that have been collected, it means that the point with the highest clarity must be in the imaging range. At this time, the binary search ends.

[0062] 4. Fitting the clarity curve

[0063] The fitting of the clarity curve requires points within the imaging area to reduce the interference of points outside the area on the fitting area. At the same time, in order to ensure the accuracy of the fitting, the point with the highest clarity must be within the range of the points that have been searched.

[0064] 4.1 Searching within the imaging area

[0065] The clarity curve of the image has a single peak. After finding the first point in the imaging interval, a direction needs to be randomly selected, a point is collected, the direction in which the clarity increases is calculated, and then the search continues in the direction of greater clarity with a fixed step size until the first point where the clarity decreases appears and there are enough points (greater than or equal to 4 points). At this time, the search ends and the clarity curve is fitted.

[0066] 4.2 Curve Fitting

[0067] The depth of field of an electron microscope is shallow, with images existing only within a certain focal length range, and its clarity curve is more in line with a Gaussian distribution rather than a quadratic or cubic function.

[0068] Assume Gaussian function:

[0069]

[0070] Among them, a, b, c, d are the parameters to be solved, focal length x, clarity y, and focal length d corresponding to the maximum clarity.

[0071] In gallium nitride defect detection, defects at N points of a gallium nitride sample are automatically detected, and each point needs to be automatically focused. The present invention realizes fast and accurate automatic focusing in the defect detection.

Claims

1. A fast automatic focusing method for an electron microscope based on image processing, characterized in that: By collecting a small number of images at different focal lengths, preprocessing the images, improving the image clarity evaluation function, combining binary search and curve fitting, obtaining the corresponding relationship between image clarity and focal length, fitting the Gaussian curve, selecting the focal length corresponding to the maximum clarity, and completing automatic focusing.

2. The method for rapid automatic focusing of an electron microscope based on image processing according to claim 1, characterized in that: The following steps are included: Step 1, calculate the search range of focal length; Step 2: Automatically adjust exposure parameters to obtain image A that reaches target brightness; Step 3, search the imaging range and determine a focal length Fi within the imaging range; Step 4: Fit the clarity curve, select the focal length corresponding to the maximum clarity, and complete autofocus.

3. The method for rapid automatic focusing of an electron microscope based on image processing according to claim 2, characterized in that: The focal length search range is [f-1, f+0.2], where f is equal to the distance between the upper surface of the stage and the pole piece minus the thickness of the sample.

4. The method for rapid automatic focusing of an electron microscope based on image processing according to claim 2, characterized in that: Step 2 further includes first setting the focal length and target brightness of the image, with the focal length being set to the middle value of the entire search range, and then adjusting the exposure parameters of the electron microscope under the condition that the focal length remains unchanged to obtain image A that reaches the target brightness.

5. The method for rapid automatic focusing of an electron microscope based on image processing according to claim 2, characterized in that: Step 3 further includes, on the basis of keeping the exposure parameters automatically obtained in step 2 unchanged, using a binary search method to set different focal lengths within the focal length search range, obtaining N images B at different focal lengths, and performing brightness alignment on the obtained image B with the image A obtained in step 2 to obtain a brightness aligned image B', wherein the brightness alignment is used to ensure that the images have a relatively high definition at the same brightness; When the i-th image is aligned with brightness, image B' i When the clarity of the image exceeds the set threshold relative to the clarity of other brightness-aligned images, it means that the i-th brightness-aligned image B' i The focal length is within the imaging range. At this time, the binary search ends. At this time, the i-th brightness aligned image B' i The focal length is recorded as Fi.

6. The method for rapid automatic focusing of an electron microscope based on image processing according to claim 5, characterized in that: The comparison method of clarity is: The brightness aligned image B' is subjected to different degrees of rapid noise reduction, wherein the i-th brightness aligned image B' i There are four specific methods for fast noise reduction: i The adjacent pixels are 4-in-1, 9-in-1, 16-in-1, 25-in-1, i=1, ..., N; Each brightness-aligned image and its image after rapid noise reduction form a group of images C, where the i-th brightness-aligned image B' i and B' i The i-th group of images obtained by fast denoising is denoted as C i ; Calculate C according to the focal length relationship when the image is taken. i and C i+1 The change value of the image clarity includes: the i-th image B' after brightness alignment i After 4-in-1, 9-in-1, 16-in-1, and 25-in-1, 4 denoised images are generated. These 4 denoised images and the image B' after brightness alignment of the i-th image i The i-th group of images is formed, and the i+1-th group of images is formed, and B' is compared respectively. i and B' i+1 The clarity of the two images produced by 4-in-1, the clarity of the two images produced by 9-in-1, the clarity of the two images produced by 16-in-1, and the clarity of the two images produced by 25-in-1.

7. The method for rapid automatic focusing of an electron microscope based on image processing according to claim 5, characterized in that: The brightness alignment method is specifically to multiply the image brightness of image B by an adjustment coefficient α, where the brightness adjustment coefficient α is calculated by the following formula: min(A1-αB1) 2 , where A1 represents the brightness of image A, B1 represents the brightness of image B, and N≥3.

8. A method for rapid automatic focusing of an electron microscope based on image processing according to any one of claims 2 to 7, characterized in that: Step 4 further includes that the clarity curve of the image has a single peak, and the search is continued with a fixed step size along the direction of increasing clarity based on the focal length Fi until the clarity drops for the first time, and at least 4 focal lengths and their corresponding clarity have been searched. At this time, the search ends, and the clarity curve is fitted using these 4 focal lengths and their corresponding clarity, and the curve fitting adopts Gaussian distribution.

Images