A transmission electron microscope automatic focusing method and system based on image edge contrast monitoring

Abstract

The application provides a transmission electron microscope automatic focusing method and system based on image edge contrast monitoring, and relates to the technical field of transmission electron microscope, and comprises the following steps: S1, acquiring a current magnetic lens current value, a magnification and a transmission electron micrograph; S2, performing model segmentation or threshold segmentation on the current transmission electron micrograph to extract a mask graph, and performing sample edge extraction processing on the mask graph; S3, obtaining a normal line based on a tangent direction; S4, obtaining a relationship curve between a serial number and an average gray value; S5, extracting a focusing criterion parameter based on the relationship curve; S6, judging whether the relationship curve meets a preset focusing state condition; and S7, if the current cycle number is not less than a threshold value, determining that the automatic focusing fails and outputting error information. The application can realize automatic focusing while detecting the focusing state, thereby improving the robustness, efficiency and verifiability of the transmission electron microscope automatic focusing.

Description

Technical Field

[0001] This invention relates to the field of transmission electron microscopy, and more particularly to an automatic focusing method and system for transmission electron microscopy based on image edge contrast monitoring. Background Technology

[0002] Transmission electron microscopy (TEM) is a microscopic analytical instrument that uses a high-energy electron beam to penetrate ultrathin samples and form magnified images. It is widely used in materials science, semiconductor manufacturing, nanostructure characterization, and biological ultrastructure research. Focusing is a crucial step in the TEM imaging process. The focusing state directly determines image sharpness, sample edge resolution, the rendering of fine structural details, and the accuracy of subsequent quantitative measurements and analyses. In practical applications, to obtain high-quality TEM images, it is usually necessary to finely adjust the magnetic lens current under different magnifications, sample thicknesses, and imaging conditions to ensure that the sample edges or target structures are in ideal focus.

[0003] Traditional transmission electron microscopes rely primarily on manual adjustment of the magnetic lens current and observation of image sharpness changes by the operator. This method is highly dependent on the operator's experience and has poor repeatability. To automate the focusing process, several automatic focusing schemes have been developed. One type of method estimates the magnitude and direction of defocusing based on electron beam image shift, beam tilt, or related displacement, and then calculates the required current compensation value. Another method acquires multiple images at different focal positions and uses a sharpness evaluation function to calculate the optimal focal position that maximizes the sharpness index for each image. Commonly used sharpness evaluation functions include image gradient energy, Laplacian operator response, gray-level variance, high-frequency components of Fourier transform, and autocorrelation function width. These schemes have been applied in some commercial transmission electron microscope systems, providing a feasible technical path for automatic focusing.

[0004] However, existing autofocus technologies still have many shortcomings in practical applications. First, defocus estimation methods based on image shift or beam tilt often require additional hardware or specific electron beam scanning procedures, increasing system complexity and calibration difficulty. Second, methods based on multi-frame focus sequence scanning require acquiring multiple images to locate the optimal focus, which significantly increases the electron irradiation dose to the sample and prolongs focusing time. Furthermore, for electron beam-sensitive materials, excessively long irradiation times may lead to sample damage or structural changes. Third, regardless of whether gradient, Laplacian, variance, or frequency domain energy is used as a sharpness indicator, these indicators are single scalars calculated for the entire image or a fixed region of interest. When there are contaminant particles, uneven background brightness, low signal-to-noise ratio, or sparse sample texture in the image, these global indicators are easily interfered with and prone to misjudgment. In addition, the edge extension morphology of images varies significantly at different magnifications, and the sensitivity and optimal value location of the same sharpness evaluation function often lack consistency at different magnifications, leading to a decrease in the robustness of focusing strategies based on fixed indicators after magnification switching. Finally, most existing methods lack explicit self-detection logic for focus status and failure exit mechanisms in the automatic acquisition process. When the focusing process fails to converge due to sample drift, charging effect, or environmental disturbances, the system continues to acquire images, wasting acquisition time and potentially rendering all subsequent data invalid due to improper focusing. Meanwhile, although there are many mature technologies in areas such as image segmentation, edge detection, and basic visual models, no publicly available technical solution has yet been found that organically combines sample edge extraction, edge normal direction grayscale profile alignment analysis, joint determination of white edge pixel width and curve peak count, and closed-loop focus status detection process into automatic focusing in transmission electron microscopy. Summary of the Invention

[0005] In view of this, the purpose of this invention is to propose an automatic focusing method and system for transmission electron microscopy based on image edge contrast monitoring, so as to solve the problems existing in the prior art, such as the automatic focusing process relying on global sharpness, unstable sample edge positioning, easy misjudgment under low signal-to-noise ratio and complex background conditions, lack of reliable state detection of focusing results, and difficulty in safely exiting during automatic acquisition.

[0006] The technical means employed in this invention are as follows: An autofocusing method for transmission electron microscopy based on image edge contrast monitoring includes the following steps: S1. Obtain the current magnetic lens current value, current magnification, and current transmission electron microscopy image; S2. Perform model segmentation or grayscale conversion, threshold segmentation and morphological processing on the current transmission electron microscopy image to extract the mask image, and perform sample edge extraction processing on the mask image to extract the sample edge contour. S3. Select multiple edge points at equal intervals on the sample edge contour of the current transmission electron microscopy image. Obtain the local tangent direction based on the contour curvature in the neighborhood of each edge point. Obtain the normal based on the tangent direction. Determine the background side according to the mask image. The direction from the inside of the sample to the background side outside the sample is the positive direction. S4. Using each edge point as a zero reference, align the grayscale sampling numbers in each normal direction to obtain multiple edge grayscale profiles, and average the multiple edge grayscale profiles to obtain the relationship curve between the number and the average grayscale value. S5. Extract focus criteria parameters based on the relationship curve. The focus criteria parameters include the number of peaks and the width of white edge pixels. S6. Determine whether the relationship curve meets the preset focus state conditions based on the focus criterion parameters. The preset focus state conditions include that the relationship curve is single-peaked and the width of the white edge pixels is within the specified range corresponding to the current magnification. If the conditions are met, the current focus state is determined to be good and the focus process ends. S7. If the preset focus state conditions are not met, determine whether the current loop count is less than the threshold. If it is less than the threshold, calculate the change value of the magnetic lens current based on the number of peaks, the width of the white edge pixels, and the current magnification, and add the change value of the magnetic lens current to the current magnetic lens current value. After the image stabilizes, return to S1. If the current loop count is not less than the threshold, determine that the autofocus has failed and output an error message.

[0007] Furthermore, in S2, the sample edge extraction process is implemented using a trained segmentation model or a traditional image processing method; The trained segmentation model is an image segmentation model based on SAM3. The SAM3 model is fine-tuned and trained using existing transmission electron microscopy images and sample region annotation masks corresponding to the transmission electron microscopy images. The trained segmentation model is used to output a segmentation mask of the target sample region based on the input transmission electron microscopy image. Traditional image processing methods include threshold segmentation, connected component filtering, Canny edge detection, and contour extraction.

[0008] Furthermore, when the sample edge extraction is performed using the SAM3-based image segmentation model, the transmission electron microscopy image to be processed is input into the SAM3-based image segmentation model to obtain the segmentation mask of the target sample region, and the boundary is extracted based on the segmentation mask to obtain the edge contour of the target sample. When traditional image processing methods are used for sample edge extraction, the transmission electron microscope image is grayscaled and binarized with a fixed threshold to obtain a binary image. The binary image is then inverted to highlight the dark sample area. Subsequently, the contour of the external connected region is extracted, and the connected region with the largest area is retained as the target sample region. Canny edge detection is then performed on the binary image of the target sample region, and the contour of the detected edge image is extracted to obtain the edge contour of the target sample.

[0009] Furthermore, in S3, calculating the normal direction of each edge point includes the following steps: Calculate the tangent vector based on the adjacent contour points before and after each edge point, and then perform a perpendicular transformation on the tangent vector to obtain the normal vector.

[0010] Furthermore, in S4, the edge grayscale profile is obtained by performing grayscale sampling at fixed pixel intervals or sub-pixel intervals along the normal direction of each edge point.

[0011] Furthermore, in S5, the number of peaks is obtained by peak detection using the relationship curve, the first derivative of the relationship curve, or the second derivative of the relationship curve; the average gray value at the zero position of each edge point is used as the average gray value of the edge, and the sampling points in the relationship curve that are greater than the average gray value of the edge are identified as white edge pixels, and the width of the white edge pixels is determined by the width of the continuous interval that satisfies this condition.

[0012] Furthermore, in S6, the specified range corresponding to the current magnification is established by calibration experiments and stored in the form of a lookup table, empirical model, regression model or neural network model. Different magnifications correspond to different target ranges of white edge pixel width.

[0013] Furthermore, in S7, the change value of the magnetic lens current is obtained through a pre-established correction mapping relationship, which takes the number of peaks, the width of the white edge pixels and the current magnification as inputs, and outputs the current correction amplitude and correction direction.

[0014] Furthermore, the modified mapping relationship adopts a piecewise linear model: ΔI t = σ t × [ a(M t )×(W t -W0(M t )) + b(M t )×(P t -1) + c(M t ) ] Among them, W0(M t ) represents the target white border width corresponding to the optimal focus state at the current magnification, a(M t ), b(Mt ) and c(M t σ is the correlation coefficient obtained through calibration. t To correct the direction; The correction direction is determined based on one or more of the following: the skewness of the relationship curve relative to the zero position, the positional relationship of the bright edge on the inside and outside of the sample, the adjustment direction of the previous iteration, and the correction rules pre-calibrated at the current magnification.

[0015] The present invention also provides an autofocusing system for transmission electron microscopy based on image edge contrast monitoring, for implementing the autofocusing method for transmission electron microscopy based on image edge contrast monitoring described in any one of the above claims, comprising: The image acquisition module is used to acquire the current magnetic lens current value, the current magnification, and the current transmission electron microscopy image; The edge extraction module is used to perform sample edge extraction processing on the current transmission electron microscopy image to extract the sample edge contour; The edge normal analysis module is used to select multiple edge points at equal intervals on the sample edge contour, calculate the normal direction of each edge point, and define the side where the gray value increases along the normal direction as the positive direction. The parameter extraction module is used to align and average the grayscale sampling numbers in each normal direction with each edge point as the zero reference, to obtain the relationship curve between the number and the average grayscale value, and to extract the number of peaks and the width of white edge pixels based on the relationship curve. The focus determination module is used to determine whether the relationship curve meets the preset focus state conditions. The preset focus state conditions include at least the relationship curve being a single peak and the width of the white edge pixels being within the specified range corresponding to the current magnification. The magnetic lens control module is used to calculate the change in magnetic lens current based on the number of peaks, the width of white edge pixels, and the current magnification when the focus determination module determines that the preset focus state conditions are not met, and to control the magnetic lens to perform current adjustment. The exception handling module is used to output an autofocus failure message when the current loop count reaches a threshold.

[0016] Compared with the prior art, the present invention has the following advantages: This invention introduces sample edge extraction technology, which can stably acquire sample edges in complex transmission electron microscopy images. Compared with methods that rely on fixed hot zones, overall image clarity, or manual selection, it has better target locking and anti-interference capabilities. The sample edge extraction technology can be implemented using a trained segmentation model, or by using traditional image processing methods such as Canny, or other edge recognition and segmentation methods.

[0017] This invention transforms the two-dimensional edge focusing problem into a one-dimensional stable curve analysis problem by aligning and averaging the grayscale profile along the normal direction of the sample edge, significantly reducing the impact of random noise, isolated contamination points, and local abnormal textures on focusing evaluation.

[0018] This invention, by combining the number of curve peaks and the width of white edge pixels as focusing criteria, can more accurately characterize the edge extension morphology in transmission electron microscopy than a single gradient, variance, or frequency domain energy index, thereby improving the robustness of focusing judgment.

[0019] This invention establishes a current correction model associated with magnification, which can adaptively calculate the current change value based on the current magnification and edge morphology parameters, thereby improving the consistency and convergence speed of autofocus at different magnifications.

[0020] This invention, by setting up a closed-loop process for focusing status detection, can proactively determine whether focusing is successful during automatic acquisition tasks, avoiding continuous image acquisition in a defocused state, and improving system security and traceability.

[0021] In some embodiments, the present invention only requires edge analysis and closed-loop correction based on the current image, without the need to pre-acquire a complete multifocal image stack, thus helping to reduce electron dose, shorten autofocus time and reduce additional perturbation to the sample. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of the automatic focusing process of the transmission electron microscope of the present invention; Figure 2 This is the sample edge extraction mask image of the present invention; Figure 3 This is a diagram illustrating edge extraction and edge point selection in this invention; Figure 4 This is the normal generation diagram for the present invention; Figure 5 This is a graph showing the alignment and average curve of the edge grayscale cross-section of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] In this invention, sample edge extraction refers to obtaining sample boundaries, edge contours, edge pixel sets, mask boundaries, or equivalent contour information from transmission electron microscopy images for focused analysis. This can be achieved through various methods, including but not limited to trained segmentation models, traditional edge detection algorithms, threshold segmentation algorithms, active contour models, morphological boundary extraction algorithms, or combinations thereof. Therefore, the trained SAM3 segmentation model is merely a preferred embodiment and not a necessary limitation on the scope of this invention; similarly, traditional image processing methods such as Canny edge detection can also be used as alternative implementation methods.

[0027] In this invention, the width of the white edge pixels can also be simply referred to as the white edge width, which is used to characterize the width of the bright edge extension region at the sample edge; the number of peaks is used to characterize the number of main peaks and / or sub-peaks of the average gray-scale curve of the edge. Preferably, the average gray-scale value at the zero point of each edge point is used as the average gray-scale of the edge, and sampling points in the average gray-scale curve that are greater than the average gray-scale of the edge are identified as white edge pixels; the white edge width can be determined by the number of consecutive pixels or the width of a consecutive interval that meets the above conditions. Both can be configured by calibration experiments in combination with specific microscope models, sample types, and imaging modes.

[0028] like Figure 1 As shown, this invention provides an autofocusing method for transmission electron microscopy based on image edge contrast monitoring, comprising the following steps: S1. Obtain the current magnetic lens current value, current magnification, and current transmission electron microscopy image.

[0029] The magnetic lens current value is preferably the target magnetic lens current value that has a major influence on the current imaging focal plane, such as the objective lens current value, or an equivalent focusing control value formed by the combination of multiple lens currents. When the autofocus process starts, it first reads the current magnetic lens current value, current magnification, and the currently acquired transmission electron microscope image from the transmission electron microscope control system. After each magnetic lens current adjustment, it is necessary to wait for the image to stabilize before returning to S1 to reacquire the current magnetic lens current value, current magnification, and a new transmission electron microscope image to enter the next iteration. After each superposition of the magnetic lens current change value, image drift, global brightness fluctuation, and inter-frame differences can also be monitored. When the drift and brightness fluctuation are below the stabilization threshold, the image is determined to be stable, and the next round of focusing analysis is initiated.

[0030] S2. Perform model segmentation or grayscale conversion, threshold segmentation and morphological processing on the current transmission electron microscopy image to extract the mask image, and perform sample edge extraction processing on the mask image to extract the sample edge contour. Sample edge extraction can be achieved through a trained segmentation model or traditional image processing methods. The trained segmentation model can be a SAM3-based image segmentation model, used to receive transmission electron microscopy images and output a segmentation mask for the target sample region. After obtaining the segmentation mask, boundary extraction is performed to obtain the target sample edge contour. Traditional image processing methods may include threshold segmentation, connected component filtering, Canny edge detection, and contour extraction. Specifically, the transmission electron microscopy image can first be grayscaled and thresholded to obtain a binary image, which is then inverted to highlight dark sample regions. Subsequently, the contours of external connected regions are extracted, retaining the connected region with the largest area as the target sample region. Canny edge detection is then performed on the binary image of the target sample region, and contour extraction is performed on the detected edge image to obtain the target sample edge contour for focused analysis.

[0031] S3. Select multiple edge points at equal intervals on the sample edge contour of the current transmission electron microscopy image. Obtain the local tangent direction based on the contour curvature in the neighborhood of each edge point. Obtain the normal based on the tangent direction. Determine the background side according to the mask image. The direction from the inside of the sample to the background side outside the sample is the positive direction. S4. Using each edge point as a zero reference, align the grayscale sampling numbers in each normal direction to obtain multiple edge grayscale profiles, and average the multiple edge grayscale profiles to obtain the relationship curve between the number and the average grayscale value.

[0032] Each edge point is designated as a zero point, and grayscale sampling is performed along the normal direction of each edge point at fixed pixel or sub-pixel intervals. Before grayscale sampling, one or more preprocessing operations can be performed on the transmission electron microscope image, including noise reduction, normalization, adaptive filtering, histogram enhancement, or bandpass filtering. The grayscale sequences obtained from all edge points are aligned according to the zero points of each edge point, and all sequences are averaged to obtain an average grayscale curve. This average grayscale curve represents the relationship between the sequence number and the average grayscale value, and is used for subsequent extraction of focusing criterion parameters.

[0033] S5. Extract focus criteria parameters based on the relationship curve. The focus criteria parameters include the number of peaks and the width of white edge pixels.

[0034] Peak detection is performed on the average grayscale curve to obtain the number of peaks. Peak detection can be achieved through the average grayscale curve itself, the first derivative of the average grayscale curve, or the second derivative of the average grayscale curve. The average grayscale value at the zero point of each edge is used as the edge average grayscale. Sampling points in the average grayscale curve that are greater than the edge average grayscale value are identified as edge white pixels. The width of the white edge pixels is determined by the width of the continuous interval that meets this condition. The number of peaks is used to characterize the number of main peaks and sub-peaks of the edge grayscale average curve, and the width of the white edge pixels is used to characterize the width of the bright edge extension region at the sample edge. Both serve as quantitative criteria for determining the current focus state.

[0035] S6. Based on the focus criterion parameters, determine whether the relationship curve meets the preset focus state conditions. The preset focus state conditions include that the relationship curve is single-peaked and the width of the white edge pixels is within the specified range corresponding to the current magnification. If the conditions are met, determine that the current focus state is good and end the focus process.

[0036] A permissible width range corresponding to the current magnification is defined. This range is a threshold range associated with the magnification, pre-established by calibration experiments and stored in the form of a lookup table, empirical model, regression model, or neural network model. Different magnifications correspond to different target ranges for the white edge pixel width. When the number of peaks is 1 and the white edge pixel width falls within the permissible width range corresponding to the current magnification, the current image is considered to be in good focus, a focus completion signal is output, and the focusing process ends. If the number of peaks is not equal to 1 or the white edge pixel width exceeds the permissible range for the corresponding magnification, the current focus condition is not met, and the process proceeds to S7 for iterative correction.

[0037] S7. If the preset focus state conditions are not met, determine whether the current loop count is less than the threshold. If it is less than the threshold, calculate the change value of the magnetic lens current based on the number of peaks, the width of the white edge pixels, and the current magnification, and add the change value of the magnetic lens current to the current magnetic lens current value. After the image stabilizes, return to S1. If the current loop count is not less than the threshold, determine that the autofocus has failed and output an error message.

[0038] When the number of peaks is not equal to 1 or the width of the white edge pixels exceeds the allowable range of the corresponding magnification, it is determined whether the current iteration number is less than a preset threshold. If the current iteration number is greater than or equal to the threshold, autofocus is determined to have failed, an error message is output, and the process exits. If the current iteration number is less than the threshold, the change value of the magnetic lens current is calculated based on the number of peaks, the width of the white edge pixels, and the current magnification. The change value of the magnetic lens current is obtained through a pre-established correction mapping relationship, which takes at least the number of peaks, the width of the white edge pixels, and the current magnification as input, and outputs the current correction amplitude and correction direction. The correction direction is determined based on one or more of the following: the skewness of the average grayscale curve relative to zero, the positional relationship of the bright edge on the inside and outside of the sample, the adjustment direction of the previous iteration, and the pre-calibrated correction rules at the current magnification. The corrected magnetic lens current is updated to the sum of the current magnetic lens current value and the change value of the magnetic lens current. The transmission electron microscope is controlled to perform current adjustment, and after the image stabilizes, it re-enters S1 to continue iterating until the focusing state conditions are met or the failure threshold is reached.

[0039] This invention also provides an automatic focusing system for transmission electron microscopy based on image edge contrast monitoring, including an image acquisition module, an edge extraction module, an edge normal analysis module, a parameter extraction module, a focusing determination module, a magnetic lens control module, and an anomaly handling module. The image acquisition module acquires the current magnetic lens current value, the current magnification, and the current transmission electron microscope image. The edge extraction module performs sample edge extraction processing on the current transmission electron microscope image to extract the sample edge contour. The edge normal analysis module selects multiple edge points at equal intervals on the sample edge contour, calculates the normal direction of each edge point, and defines the side where the gray value increases along the normal direction as positive. The parameter extraction module uses each edge point as a zero-point reference, aligns and averages the gray-level sampling numbers in each normal direction to obtain a relationship curve between the number and the average gray value, and extracts the number of peaks and the width of white edge pixels based on the relationship curve. The focusing determination module determines whether the relationship curve meets preset focusing conditions, which at least include the relationship curve being single-peaked and the width of white edge pixels being within a specified range corresponding to the current magnification. The magnetic lens control module calculates the change in magnetic lens current based on the number of peaks, the width of white edge pixels, and the current magnification when the focus determination module determines that the preset focus conditions are not met, and then controls the magnetic lens to perform current adjustment. The exception handling module outputs an autofocus failure message when the current loop count reaches a threshold. All modules work together to execute the above methods.

[0040] Example 1: Automatic Focusing Method Flow like Figure 1 As shown, this embodiment discloses an autofocusing method for transmission electron microscopy based on image edge contrast monitoring. This method operates within a closed-loop system consisting of transmission electron microscope control software, an edge recognition program, and a magnetic lens control unit, and includes the following steps: Step 101: Start the autofocus process and read the current magnetic lens current value I. t Current magnification M t And the currently acquired image I(x,y). The magnetic lens current value is preferably a target magnetic lens current value that has a major influence on the current imaging focal plane, such as the objective lens current value, or an equivalent focusing control quantity formed by a combination of multiple lens currents.

[0041] Step 102: Perform sample edge extraction processing on the current image I(x,y) to obtain the main sample region and / or sample edge contour C. The sample edge extraction processing can be implemented using various techniques: In a preferred embodiment, the current image can be input into the SAM3 model to obtain a sample main region segmentation mask S(x,y), and then boundary extraction can be performed on the target sample mask to obtain the sample edge contour C; In another optional embodiment, the Canny edge detection algorithm can be used to directly extract the sample edges, combined with edge connection, contour filtering, morphological repair, or region closure processing to obtain the target edge contour C for subsequent analysis; In other embodiments, threshold segmentation, active contour models, traditional image segmentation algorithms, or other basic visual models can also be used to achieve sample edge extraction.

[0042] When the sample edge extraction results contain multiple candidate regions or multiple candidate contours, the region or contour corresponding to the target sample can be selected as the object of subsequent focused analysis based on area, connectivity, aspect ratio, position prior, edge continuity, contour integrity, output confidence, or consistency with historical frames.

[0043] Step 103: Select N edge points p evenly along the sample edge contour C according to the arc length. i For each edge point p i Calculate the local tangent vector t based on its adjacent contour points. i And from this, the normal vector n is obtained. i Preferably, first in n i and -n i Take a set of neighboring sampling points in each of the two directions, and compare the average gray values ​​on both sides. If along n i If the direction grayscale value is higher, then n is retained. i As a positive sign; conversely, -n i Defined as positive, to ensure that the "direction of grayscale value increase" is positive.

[0044] Step 104, for each edge point p i The corresponding position number is recorded as 0, and a grayscale sequence g of length 2K+1 is extracted along its normal direction at a set sampling interval Δs. i (k)=I(p i +k·Δs·n i ), where k takes values ​​in the range [-K, K]. All grayscale sequences are zero-aligned with k=0 and then averaged to obtain the average grayscale curve: g avg (k) = (1 / N) × Σ g i (k) In practice, Gaussian filtering, median filtering, Savitzky-Golay smoothing, or normalization can be applied to each grayscale sequence first, and then the average grayscale curve can be obtained to further suppress noise.

[0045] Step 105, based on the average grayscale curve g avg (k) Extract focusing parameters. Preferably, first extract the focusing parameters for g. avg (k) Perform peak detection and obtain the number of peaks P. t Then, the average gray value at the zero point of each edge point is taken as the edge average gray value T. e and will satisfy g avg (k)>T e The sampling points are identified as edge white pixels. For example, those satisfying g can be... avg (k)>T e The maximum width of a continuous interval is defined as the width of the white border W. t In other implementations, the number of consecutive sampling points that meet this condition can also be used as the white border width.

[0046] Step 106: Perform focus state determination. Preferably, set the magnification M... t The corresponding allowable width range [W] min (M t ), W max (M t When P t =1 and W t When the image falls within the allowed width range, the current image is determined to be in good focus, a focus completion signal is output, and the process ends.

[0047] Step 107, when P t Not equal to 1 or W t When the value exceeds the allowable range of the corresponding multiplier, check whether the current iteration number cnt is less than the preset threshold cnt. max If cnt is greater than or equal to cnt max If autofocus fails, an error message is output and the process exits; if cnt is less than cnt max Then proceed to the current correction step.

[0048] Step 108, according to P t W t and M t Calculate the change in magnetic lens current ΔI t Preferably, the correction function is established in advance through calibration: ΔI t = f(P t W t M t , σt ) Where, σ t The correction direction information can be obtained from the skew characteristics of the average grayscale curve relative to zero, the positional relationship of the bright edge on the inside and outside of the sample, the correction direction of the previous round, the empirical calibration rules of the corresponding magnification, or a combination thereof. When the direction does not need to be analyzed separately, the correction function can also directly output the signed current change value.

[0049] Step 109, update the corrected magnetic lens current to I. t+1 =I t +ΔI t The transmission electron microscope is controlled to perform current adjustment. After the image stabilizes, it is re-acquired and returned to step 101 to continue iterating until the focusing condition is met or the failure threshold is reached.

[0050] Through the above steps, this embodiment realizes an automatic focusing process based on changes in sample edge morphology, wherein sample edge extraction, parameter extraction, and focusing state detection are completed in the same closed loop.

[0051] Example 2: Edge Parameter Extraction Method In one alternative implementation, combined with Figures 2 to 5 The number of peaks P in step 105 t and white border width W t The following methods can be used to determine this: (1) such as Figure 2 As shown, region filtering or contour filtering is performed on the sample edge extraction results, preferably retaining the main region or main contour corresponding to the target sample, to obtain a mask or edge set for edge analysis; (2) For example Figure 3 As shown, the edge contour is extracted from the edge of the mask or the edge set, and multiple edge points are selected evenly according to the arc length; (3) such as Figure 4 As shown, normals are constructed at each edge point based on the local tangent vector, and the direction of grayscale increase is uniformly defined as positive. (4) such as Figure 5 As shown, the grayscale profiles in all normal directions are aligned and smoothed according to the zero point of the edge to obtain the average grayscale curve g. s (k); (5) The average gray value at the zero position of each edge point is taken as the edge average gray value T. e , will satisfy g s (k)>T e The sampling points are identified as edge white pixels, and the number of consecutive pixels or the width of a consecutive interval that meets this condition is defined as the white edge width W. t ; (6) For g s (k) Perform local maximum detection and retain only peaks whose height exceeds the threshold T. p Peak spacing exceeds threshold D p The peaks are obtained, and the number of peaks P is obtained. t ; (7) When P t =1 and W t When it falls within the normal range of the corresponding magnification, it is determined to be in focus; when P t Greater than 1, W t If the width is too wide or too narrow, it is judged as out of focus or abnormal focus.

[0052] In another alternative implementation, peak detection can also be performed on the first derivative curve or the second derivative curve to obtain a more sensitive edge diffusion criterion.

[0053] Example 3: Current Correction Mapping Relationship In one alternative implementation, a correction mapping table for different magnifications can be established through offline calibration. Specifically, at several representative magnifications M, using the manually confirmed optimal focusing current as a reference point, the number of peaks, white edge width, and required correction current corresponding to different defocus states are recorded to form a lookup table or regression model.

[0054] For example, a piecewise linear model can be used: ΔI t = σ t × [ a(M t )×(W t -W0(M t )) + b(M t )×(P t -1) + c(M t ) ] Among them, W0(M t ) represents the target white border width corresponding to the optimal focus state at the current magnification, a(M t ), b(M t ) and c(M t σ is the correlation coefficient obtained through calibration. t To correct the direction.

[0055] In another alternative implementation, the corrected mapping relationship can also be established through lightweight neural networks, support vector regression, random forests or other supervised learning models, thereby making the corrected current estimation more adaptive for different sample types.

[0056] Example 4: System Structure Corresponding to the above method, the present invention also provides an automatic focusing system for a transmission electron microscope, which includes at least the following modules: 1. Image acquisition module, used to acquire the current magnetic lens current value, current magnification, and current transmission electron microscopy image; 2. Edge extraction module, used to call the sample edge extraction algorithm to identify the main sample region and edge contour; the edge extraction algorithm can be a pre-trained segmentation model, a traditional edge detection algorithm, or other edge recognition and segmentation algorithm; 3. Edge normal analysis module, used to select points at equal intervals on the edge contour and calculate the normal direction, and define the direction of grayscale increase as positive; 4. Parameter extraction module, used to perform zero-position alignment and averaging of the normal grayscale profile, and extract focusing parameters such as the number of peaks and the width of the white edge; 5. Focus determination module, used to determine the focus status based on the number of peaks, white border width, and magnification-related thresholds; 6. Magnetic lens control module, used to calculate the current change value based on the parameter extraction results when the lens is out of focus, and control the magnetic lens to perform current adjustment; 7. An exception handling module is used to output an autofocus failure message when the number of loops reaches a threshold, and terminate subsequent automatic acquisition tasks or trigger manual takeover.

[0057] Example 5: Electronic Devices and Storage Media This invention can also be implemented by an electronic device. The electronic device may be a host computer for a transmission electron microscope, an image workstation, an edge computing controller, or an embedded control platform. The electronic device includes a processor, a memory, and a bus. The memory stores program instructions, which, when executed by the processor, are used to implement the aforementioned autofocusing method.

[0058] The present invention can also be implemented by a computer-readable storage medium storing a program for performing the above method. When the program is executed by a processor, it completes sample edge extraction, edge grayscale profile analysis, focusing state judgment, and magnetic lens current closed-loop correction.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatic focusing in transmission electron microscopy based on image edge contrast monitoring, characterized in that, Includes the following steps: S1. Obtain the current magnetic lens current value, current magnification, and current transmission electron microscopy image; S2. Perform model segmentation or grayscale conversion, threshold segmentation and morphological processing on the current transmission electron microscopy image to extract the mask image, and perform sample edge extraction processing on the mask image to extract the sample edge contour. S3. Select multiple edge points at equal intervals on the sample edge contour of the current transmission electron microscopy image. Obtain the local tangent direction based on the contour curvature in the neighborhood of each edge point. Obtain the normal based on the tangent direction. Determine the background side according to the mask image. The direction from the inside of the sample to the background side outside the sample is the positive direction. S4. Using each edge point as a zero reference, align the grayscale sampling numbers in each normal direction to obtain multiple edge grayscale profiles, and average the multiple edge grayscale profiles to obtain the relationship curve between the number and the average grayscale value. S5. Extract focus criteria parameters based on the relationship curve. The focus criteria parameters include the number of peaks and the width of white edge pixels. S6. Determine whether the relationship curve meets the preset focus state conditions based on the focus criterion parameters. The preset focus state conditions include that the relationship curve is single-peaked and the width of the white edge pixels is within the specified range corresponding to the current magnification. If the conditions are met, the current focus state is determined to be good and the focus process ends. S7. If the preset focus state conditions are not met, determine whether the current loop count is less than the threshold. If it is less than the threshold, calculate the change value of the magnetic lens current based on the number of peaks, the width of the white edge pixels, and the current magnification, and add the change value of the magnetic lens current to the current magnetic lens current value. After the image stabilizes, return to S1. If the current loop count is not less than the threshold, determine that the autofocus has failed and output an error message.

2. The automatic focusing method for transmission electron microscopy based on image edge contrast monitoring according to claim 1, characterized in that, In S2, the sample edge extraction process is implemented through a trained segmentation model or a traditional image processing method; The trained segmentation model is an image segmentation model based on SAM3. The SAM3 model is fine-tuned and trained using existing transmission electron microscopy images and sample region annotation masks corresponding to the transmission electron microscopy images. The trained segmentation model is used to output a segmentation mask of the target sample region based on the input transmission electron microscopy image. Traditional image processing methods include threshold segmentation, connected component filtering, Canny edge detection, and contour extraction.

3. The automatic focusing method for transmission electron microscopy based on image edge contrast monitoring according to claim 2, characterized in that, When the sample edge extraction is performed using the SAM3-based image segmentation model, the transmission electron microscope image to be processed is input into the SAM3-based image segmentation model to obtain the segmentation mask of the target sample region, and the boundary is extracted based on the segmentation mask to obtain the edge contour of the target sample. When traditional image processing methods are used for sample edge extraction, the transmission electron microscope image is grayscaled and binarized with a fixed threshold to obtain a binary image. The binary image is then inverted to highlight the dark sample area. Subsequently, the contour of the external connected region is extracted, and the connected region with the largest area is retained as the target sample region. Canny edge detection is then performed on the binary image of the target sample region, and the contour of the detected edge image is extracted to obtain the edge contour of the target sample.

4. The automatic focusing method for transmission electron microscopy based on image edge contrast monitoring according to claim 1, characterized in that, In S3, calculating the normal direction of each edge point includes the following steps: Calculate the tangent vector based on the adjacent contour points before and after each edge point, and then perform a perpendicular transformation on the tangent vector to obtain the normal vector.

5. The automatic focusing method for transmission electron microscopy based on image edge contrast monitoring according to claim 1, characterized in that, In S4, the edge grayscale profile is obtained by performing grayscale sampling at fixed pixel intervals or sub-pixel intervals along the normal direction of each edge point.

6. The automatic focusing method for transmission electron microscopy based on image edge contrast monitoring according to claim 1, characterized in that, In S5, the number of peaks is obtained by peak detection using the relationship curve, the first derivative of the relationship curve, or the second derivative of the relationship curve; the average gray value at the zero position of each edge point is used as the average gray value of the edge, and the sampling points in the relationship curve that are greater than the average gray value of the edge are identified as white edge pixels, and the width of the white edge pixels is determined by the width of the continuous interval that satisfies this condition.

7. The automatic focusing method for transmission electron microscopy based on image edge contrast monitoring according to claim 1, characterized in that, In S6, the specified range corresponding to the current magnification is established by calibration experiments and stored in the form of lookup tables, empirical models, regression models or neural network models. Different magnifications correspond to different target ranges of white edge pixel width.

8. The automatic focusing method for transmission electron microscopy based on image edge contrast monitoring according to claim 1, characterized in that, In S7, the change value of the magnetic lens current is obtained through a pre-established correction mapping relationship. The correction mapping relationship takes the number of peaks, the width of the white edge pixels and the current magnification as inputs, and outputs the current correction amplitude and correction direction.

9. The automatic focusing method for transmission electron microscopy based on image edge contrast monitoring according to claim 8, characterized in that, The corrected mapping relationship adopts a piecewise linear model: ΔI t = σ t × [ a(M t )×(W t -W0(M t )) + b(M t )×(P t -1) + c(M t ) ] Among them, W0(M t ) represents the target white border width corresponding to the optimal focus state at the current magnification, a(M t b(M) t ) and c(M t σ is the correlation coefficient obtained through calibration. t To correct the direction; The correction direction is determined based on one or more of the following: the skewness of the relationship curve relative to the zero position, the positional relationship of the bright edge on the inside and outside of the sample, the adjustment direction of the previous iteration, and the correction rules pre-calibrated at the current magnification.

10. A transmission electron microscope autofocusing system based on image edge contrast monitoring, used to implement the transmission electron microscope autofocusing method based on image edge contrast monitoring as described in any one of claims 1-9, characterized in that, include: The image acquisition module is used to acquire the current magnetic lens current value, the current magnification, and the current transmission electron microscopy image; The edge extraction module is used to perform sample edge extraction processing on the current transmission electron microscopy image to extract the sample edge contour; The edge normal analysis module is used to select multiple edge points at equal intervals on the sample edge contour, calculate the normal direction of each edge point, and define the side where the gray value increases along the normal direction as the positive direction. The parameter extraction module is used to align and average the grayscale sampling numbers in each normal direction with each edge point as the zero reference, to obtain the relationship curve between the number and the average grayscale value, and to extract the number of peaks and the width of white edge pixels based on the relationship curve. The focus determination module is used to determine whether the relationship curve meets the preset focus state conditions. The preset focus state conditions include at least the relationship curve being a single peak and the width of the white edge pixels being within the specified range corresponding to the current magnification. The magnetic lens control module is used to calculate the change in magnetic lens current based on the number of peaks, the width of white edge pixels, and the current magnification when the focus determination module determines that the preset focus state conditions are not met, and to control the magnetic lens to perform current adjustment. The exception handling module is used to output an autofocus failure message when the current loop count reaches a threshold.

Images