Image drift calibration method and device based on scanning electron microscope

Abstract

The present invention relates to the field of image processing, and provides an image drift calibration method and device based on a scanning electron microscope. The image drift calibration method based on a scanning electron microscope comprises: obtaining an image to be calibrated and a reference image taken by a scanning electron microscope; matching feature points of the image to be calibrated and the reference image to obtain a plurality of first matching point pairs; calculating the drift amount of the image to be calibrated relative to the reference image according to each first matching point pair; calculating the electric displacement calibration step according to the drift amount and the relationship between the drift amount and the electric displacement step, and the electric displacement calibration step is used to control the scanning electron microscope to realize drift calibration of the image to be calibrated. Through the present invention, automatic calibration of the image generated by the scanning electron microscope is realized, and the image drift amount is reduced.

Description

Technical Field

[0001] The invention relates to the field of image processing, and in particular to an image drift calibration method and device based on a scanning electron microscope. Background Art

[0002] The scanning electron microscope is an electronic optical instrument that plays an important role in the fields of materials, biology, physics, chemistry, etc. The principle of the scanning electron microscope is that the electron beam emitted from the cathode of the electron gun is converged into a cross spot under the action of the grid, and under the action of the anode acceleration, it is focused by three electromagnetic lenses and converged into a very fine electron probe incident beam to reach the surface of the sample. Among them, under the drive of the scanning coil above the objective lens, the incident beam can be orderly raster scanned on the surface of the sample, thereby stimulating a variety of information such as secondary electrons, backscattered electrons and X-rays, and then amplified by the corresponding detector detection and signal processing and amplification system, and transmitted to the cathode ray tube for imaging.

[0003] During high-magnification imaging under a scanning electron microscope, the image will move irregularly and continuously due to electron beam drift, electromagnetic interference and other reasons, resulting in obvious, large-scale and continuous drift in the image, which seriously affects the characterization of the sample morphology and the user experience. At present, this problem can be solved by waiting, spraying a large amount of gold on the sample, and increasing the scanning speed. However, all three methods have certain shortcomings. For example, spraying a large amount of gold will affect the image surface. If the sample is small and conductive, it cannot be sprayed with gold in large quantities, and you can only wait; although increasing the scanning speed can temporarily solve the image drift problem, it will reduce the image quality. Summary of the invention

[0004] In order to calibrate an image generated by a scanning electron microscope and reduce the amount of image drift, the present invention proposes an image drift calibration method and device based on a scanning electron microscope.

[0005] In a first aspect, the present invention provides an image drift calibration method based on a scanning electron microscope, the method comprising:

[0006] Acquire an image to be calibrated and a reference image taken by a scanning electron microscope;

[0007] Perform feature point matching on the image to be calibrated and the reference image to obtain a plurality of first matching point pairs;

[0008] Calculating the drift of the image to be calibrated relative to the reference image according to each first matching point pair;

[0009] The electric displacement calibration step is calculated according to the drift amount and the relationship between the drift amount and the electric displacement step. The electric displacement calibration step is used to control the scanning electron microscope to achieve drift calibration of the image to be calibrated.

[0010] Through the above method, by comparing the feature points of the image to be calibrated with the reference image, a first feature point matching pair is obtained, and the drift of the image to be calibrated relative to the reference image is calculated based on the feature points in the first feature point matching pair. Further, based on the relationship between the drift amount and the electric displacement step, the electric displacement calibration step is calculated, thereby adjusting the scanning electron microscope to overcome image drift and improve user experience.

[0011] In an optional implementation, feature point matching is performed on the image to be calibrated and the reference image to obtain a plurality of first matching point pairs, including:

[0012] Extracting features from the image to be calibrated and the reference image respectively to obtain a plurality of first feature points of the image to be calibrated and a plurality of second feature points of the reference image;

[0013] Performing feature point matching on each first feature point and each second feature point to obtain a plurality of second matching point pairs;

[0014] The second matching point pairs are screened to obtain the first matching point pairs that meet the preset conditions.

[0015] Through the above-mentioned implementation, by screening the second matching point pairs that meet the preset conditions, some inaccurate or low-quality matching point pairs can be removed, thereby further improving the accuracy and reliability of the matching, and facilitating more accurate calculation of the drift amount and the electrical displacement calibration step. In addition, the feature extraction algorithm and preset conditions can be adjusted according to different image features and application requirements to improve the adaptability and reliability in practical applications.

[0016] In an optional implementation, feature extraction is performed on the image to be calibrated and the reference image respectively to obtain a plurality of first feature points of the image to be calibrated and a plurality of second feature points of the reference image, including:

[0017] Extracting features from the image to be calibrated and the reference image respectively to obtain a plurality of third feature points of the image to be calibrated and a plurality of fourth feature points of the reference image;

[0018] Calculating the response value of each third characteristic point and the response value of each fourth characteristic point;

[0019] Sort the third feature points in descending order of response values, and select a preset number of third feature points with top response values ​​as first feature points;

[0020] The fourth feature points are sorted in descending order of response values, and a preset number of fourth feature points with top response values ​​are selected as second feature points.

[0021] Through the above implementation, by calculating the response values ​​of the third feature point and the fourth feature point, and sorting them from large to small according to the response values, the feature points with higher response values ​​are selected as the first feature point and the second feature point, and the influence of low-quality or noise feature points is eliminated. It can ensure that the selected feature points have high representativeness and reliability, and provide a basis for improving the accuracy of subsequent feature point matching and drift.

[0022] In an optional implementation, screening the second matching point pairs to obtain the first matching point pairs that meet the preset conditions includes:

[0023] Calculating the length and tilt angle of each second matching point pair, where the length and tilt angle of the second matching point pair are obtained by the coordinates of the first feature point and the second feature point in the second matching point pair;

[0024] The second matching point pairs whose length is smaller than the preset length and whose inclination angle is smaller than the preset angle are used as the first matching point pairs.

[0025] Through the above implementation, the length and tilt angle of the first matching point pair are calculated through the coordinates of the first feature point and the coordinates of the second feature point in the first matching point pair. The length represents the distance between the first feature point and the second feature point, and the tilt angle represents the relative position offset between the first feature point and the second feature point. The larger the length, the larger the distance between the first feature point and the second feature point, and the larger the tilt angle, the larger the relative position offset between the first feature point and the second feature point, indicating that the first feature point and the second feature point are less similar. Therefore, the second matching point pair is screened by length and tilt angle, so that the obtained first matching point pair is more accurate, thereby improving the accuracy of the drift amount and the electrical displacement calibration step.

[0026] In an optional implementation, calculating the drift amount of the image to be calibrated relative to the reference image according to each first matching point pair includes:

[0027] For each first matching point pair, calculating the length of the first matching point pair, where the length of the first matching point pair is obtained by the coordinates of the first feature point and the second feature point in the first matching point pair;

[0028] Calculate the average length of each first matching point pair;

[0029] Obtain the magnification of the scanning electron microscope, the width of the reference image, and the lateral resolution of the reference image;

[0030] The drift is calculated based on the mean, magnification, width, and lateral resolution.

[0031] Through the above implementation, the average length of the first matching point pair is used as the pixel drift amount, and the drift amount of the image to be calibrated is calculated based on the relationship between the obtained pixel drift amount, the magnification of the scanning electron microscope, the width of the reference image, and the lateral resolution of the reference image and the drift amount of the image.

[0032] In an optional implementation, the drift amount is calculated according to the mean, magnification, width and lateral resolution, including:

[0033]

[0034] Among them, SreenOffset is the drift amount, PixelOffset is the mean, SreenPixel is the lateral resolution, PhotoWidth is the width, and Mag is the magnification.

[0035] In an optional implementation, feature point matching is performed on each first feature point and each second feature point to obtain a plurality of second matching point pairs, including:

[0036] Obtaining a feature point descriptor of each first feature point and a feature point descriptor of each second feature point;

[0037] Feature point matching is performed on each first feature point and each second feature point according to the feature point descriptor of each first feature point and the feature point descriptor of each second feature point to obtain each second matching point pair.

[0038] Through the above implementation, the feature point descriptor can describe the detailed information of the feature points in the image, such as edges, corner points, etc., and use the feature point descriptor to match the first feature point and the second feature point, so as to more accurately identify and match similar feature points, thereby improving the matching accuracy.

[0039] In a second aspect, the present invention also provides an image drift calibration device based on a scanning electron microscope.

[0040] The device includes:

[0041] An acquisition module, used for acquiring the image to be calibrated and the reference image taken by the scanning electron microscope;

[0042] A matching module, used for matching feature points of the image to be calibrated and the reference image to obtain a plurality of first matching point pairs;

[0043] A first calculation module, used for calculating the drift amount of the image to be calibrated relative to the reference image according to each first matching point pair;

[0044] The second calculation module is used to calculate the electric displacement calibration step according to the drift amount and the relationship between the drift amount and the electric displacement step. The electric displacement calibration step is used to control the scanning electron microscope to achieve drift calibration of the image to be calibrated.

[0045] Through the above device, by comparing the feature points of the image to be calibrated with the reference image, a first feature point matching pair is obtained, and the drift of the image to be calibrated relative to the reference image is calculated based on the feature points in the first feature point matching pair. Further, based on the relationship between the drift amount and the electric displacement step, the electric displacement calibration step is calculated, thereby adjusting the scanning electron microscope to overcome image drift and improve user experience.

[0046] In a third aspect, the present invention further provides a computer device, comprising a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the steps of the image drift calibration method based on a scanning electron microscope according to the first aspect or any embodiment of the first aspect by executing the computer instructions.

[0047] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image drift calibration method based on a scanning electron microscope according to the first aspect or any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 is a flow chart of a method for image drift calibration based on a scanning electron microscope proposed according to an exemplary embodiment;

[0050] Figure 2 is a flow chart of a method for image drift calibration based on a scanning electron microscope in an example;

[0051] Figure 3 It is a structural schematic diagram of an image drift calibration device based on a scanning electron microscope proposed according to an exemplary embodiment;

[0052] Figure 4 It is a schematic diagram of the hardware structure of a computer device proposed according to an exemplary embodiment. DETAILED DESCRIPTION

[0053] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0055] In order to calibrate an image generated by a scanning electron microscope and reduce the amount of image drift, the present invention proposes an image drift calibration method and device based on a scanning electron microscope.

[0056] Figure 1 FIG. 1 is a flow chart of a method for image drift calibration based on a scanning electron microscope according to an exemplary embodiment. Figure 1 As shown, the image drift calibration method based on a scanning electron microscope includes the following steps S101 to S104.

[0057] Step S101: acquiring an image to be calibrated and a reference image taken by a scanning electron microscope.

[0058] In an optional embodiment, the reference image is a focused scanning image captured after the scanning electron microscope is adjusted to meet the imaging conditions.

[0059] In an optional embodiment, in order to more accurately match the feature points of the image to be calibrated and the reference image, a scale pyramid may be constructed to scale the initial image to be calibrated and the initial reference image captured by the scanning electron microscope based on a scale factor to obtain the image to be calibrated and the reference image with different resolutions. The scale factor and the number of pyramid layers are not specifically limited and can be set according to actual needs.

[0060] Step S102: performing feature point matching on the image to be calibrated and the reference image to obtain a plurality of first matching point pairs.

[0061] In an optional embodiment, feature points of the image to be calibrated and feature points of the reference image are obtained respectively, and feature point matching is performed based on the feature points of the image to be calibrated and the feature points of the reference image to obtain a plurality of first matching point pairs.

[0062] In an optional embodiment, a feature point detection algorithm (Oriented FAST and Rotated BRIEF, ORB) is used to extract features from the image to be calibrated and the reference image, respectively, to obtain feature points of the image to be calibrated and feature points of the reference image, and the feature points are used to reflect important information in the image, such as edge information, frequency information, etc.

[0063] In an optional embodiment, feature points of the image to be calibrated and the reference image are matched by feature descriptors corresponding to feature points of the image to be calibrated and feature descriptors corresponding to feature points of the reference image to obtain a plurality of first matching point pairs. The matching algorithm may be brute force matching, nearest neighbor matching, etc., and matching is performed between the image to be calibrated and the reference image according to the feature descriptors of the feature points to find the first matching point pairs that are close in spatial position and have high descriptor similarity.

[0064] In an optional embodiment, inaccurate or low-quality matching point pairs may be removed by using preset screening conditions (such as a distance threshold, an angle threshold, etc.), thereby improving the accuracy of the first matching point pair.

[0065] Step S103: Calculate the drift amount of the image to be calibrated relative to the reference image according to each first matching point pair.

[0066] In an optional embodiment, the drift amount represents the positional offset of each pixel in the image to be calibrated relative to each pixel in the reference image. The drift amount of the image to be calibrated relative to the reference image includes the drift amount in the horizontal direction and the drift amount in the vertical direction.

[0067] In an optional embodiment, the drift amount corresponding to each matching point pair is calculated based on the pixel point coordinates corresponding to the feature points in each first matching point pair; the drift amount of the image to be calibrated relative to the reference image is calculated based on the parameters of the scanning electron microscope, the parameters of the reference image, and the drift amount corresponding to each matching point pair.

[0068] Step S104: Calculate the electrical displacement calibration step according to the drift amount and the relationship between the drift amount and the electrical displacement step. The electrical displacement calibration step is used to control the scanning electron microscope to achieve drift calibration of the image to be calibrated.

[0069] In an optional embodiment, the relationship between the drift amount and the electrical displacement step can be obtained through experiments.

[0070] Through the above method, by comparing the feature points of the image to be calibrated with the reference image, a first feature point matching pair is obtained, and the drift of the image to be calibrated relative to the reference image is calculated based on the feature points in the first feature point matching pair. Further, based on the relationship between the drift amount and the electric displacement step, the electric displacement calibration step is calculated, thereby adjusting the scanning electron microscope to overcome image drift and improve user experience.

[0071] In one example, in the above step S102, feature point matching is performed on the image to be calibrated and the reference image through the following steps to obtain a plurality of first matching point pairs:

[0072] Step a1: extracting features from the image to be calibrated and the reference image respectively to obtain a plurality of first feature points of the image to be calibrated and a plurality of second feature points of the reference image.

[0073] Step a2: performing feature point matching on each first feature point and each second feature point to obtain a plurality of second matching point pairs.

[0074] Step a3: Screen the second matching point pairs to obtain the first matching point pairs that meet the preset conditions.

[0075] In an embodiment of the present invention, by screening the second matching point pairs that meet the preset conditions, some inaccurate or low-quality matching point pairs can be removed, thereby further improving the accuracy and reliability of the matching, and facilitating more accurate calculation of the drift amount and the electrical displacement calibration step. In addition, the feature extraction algorithm and preset conditions can be adjusted according to different image features and application requirements to improve the adaptability and reliability in practical applications.

[0076] In one example, in the above step a1, feature extraction is performed on the image to be calibrated and the reference image in the following manner to obtain a plurality of first feature points of the image to be calibrated and a plurality of second feature points of the reference image:

[0077] First, feature extraction is performed on the image to be calibrated and the reference image respectively to obtain a plurality of third feature points of the image to be calibrated and a plurality of fourth feature points of the reference image.

[0078] Next, the response value of each third characteristic point and the response value of each fourth characteristic point are calculated.

[0079] In an optional embodiment, the Harris corner response function can be used to calculate the response value of the third feature point and the response value of each fourth feature point. The Harris corner response function is a function for detecting corner points in an image. By calculating the gradient magnitude and direction of the window around each pixel in the image, as well as the distribution characteristics of these gradient magnitudes and directions, it is determined whether the pixel is a corner point (feature point).

[0080] Thirdly, the third feature points are sorted in descending order of response values, and a preset number of third feature points with top response values ​​are selected as first feature points.

[0081] Finally, the fourth feature points are sorted in descending order of response values, and a preset number of fourth feature points with top response values ​​are selected as second feature points.

[0082] In an optional embodiment, the number of the third feature points and the number of the fourth feature points can be set according to actual conditions and are not specifically limited here.

[0083] In an optional embodiment, the first feature point and the second feature point further include direction information. Exemplarily, the direction information in the feature point can be determined by using the grayscale centroid method / moment method.

[0084] In an embodiment of the present invention, by calculating the response values ​​of the third feature point and the fourth feature point, and sorting them from large to small according to the response values, the feature points with higher response values ​​are selected as the first feature point and the second feature point, and the influence of low-quality or noise feature points is eliminated. This ensures that the selected feature points have high representativeness and reliability, and provides a basis for improving the accuracy of subsequent feature point matching and drift.

[0085] In one example, in the above step a2, feature point matching is performed on each first feature point and each second feature point in the following manner to obtain a plurality of second matching point pairs:

[0086] First, a feature point descriptor of each first feature point and a feature point descriptor of each second feature point are obtained.

[0087] In an optional embodiment, several point pairs are randomly selected in the neighborhood of the feature point, the pixel sizes of the point pairs are compared, and binary values ​​are assigned in the following manner to form a binary code, which is used as a feature point descriptor of the feature point.

[0088]

[0089] Among them, τ represents the feature point descriptor, p(x) and p(y) represent the pixel value of random point x and the pixel value of random point y respectively.

[0090] Then, feature point matching is performed on each first feature point and each second feature point according to the feature point descriptor of each first feature point and the feature point descriptor of each second feature point to obtain each second matching point pair.

[0091] In an optional embodiment, the Hamming distance between the feature point descriptor of the first feature point and the feature point descriptor of the second feature point is calculated, and feature point matching is performed on each first feature point and each second feature point according to the Hamming distance to obtain each second matching point pair.

[0092] In an embodiment of the present invention, the feature point descriptor can describe the detailed information of the feature points in the image, such as edges, corner points, etc., and by matching the first feature point and the second feature point using the feature point descriptor, similar feature points can be more accurately identified and matched, thereby improving the matching accuracy.

[0093] In one example, in the above step a3, the second matching point pairs are screened by the following content to obtain the first matching point pairs that meet the preset conditions:

[0094] First, the length and tilt angle of each second matching point pair are calculated, and the length and tilt angle of the second matching point pair are obtained by the coordinates of the first feature point and the coordinates of the second feature point in the second matching point pair. Among them, the tilt angle is the angle between the line between the first feature point and the second feature point in the second matching point pair and the preset reference line. The preset reference line can be set according to actual needs and is not specifically limited here.

[0095] In an optional embodiment, the length and tilt angle of the second matching point pair are calculated by the following formula:

[0096] a=1024-Point x1 +Point x2

[0097] b=Abs(Point y2 -Point y1 )

[0098]

[0099] Angle=asin(b / Length)×180 / π

[0100] Among them, Length represents the length of the second matching point pair, Point x1 ,Point y1 Represents the coordinates of the first feature point in the second matching point pair, Point x2 ,Point y2 represents the coordinates of the second feature point, and Angle represents the inclination angle of the second matching point pair.

[0101] Then, the second matching point pairs whose length is less than the preset length and whose inclination angle is less than the preset angle are used as the first matching point pairs. The preset length and the preset angle are set according to the actual situation and are not specifically limited here.

[0102] In an embodiment of the present invention, the length and the tilt angle of the first matching point pair are calculated by using the coordinates of the first feature point and the coordinates of the second feature point in the first matching point pair. The length represents the distance between the first feature point and the second feature point, and the tilt angle represents the relative position offset between the first feature point and the second feature point. The larger the length, the larger the distance between the first feature point and the second feature point, and the larger the tilt angle, the larger the relative position offset between the first feature point and the second feature point, indicating that the first feature point and the second feature point are less similar. Therefore, the second matching point pair is screened by length and tilt angle, so that the obtained first matching point pair is more accurate, thereby improving the accuracy of the drift amount and the electrical displacement calibration step.

[0103] In one example, in the above step S103, the drift amount of the image to be calibrated relative to the reference image is calculated in the following manner:

[0104] First, for each first matching point pair, the length of the first matching point pair is calculated, and the length of the first matching point pair is obtained by the coordinates of the first feature point and the coordinates of the second feature point in the first matching point pair.

[0105] Next, the average length of each first matching point pair is calculated.

[0106] Again, obtain the magnification of the SEM, the width of the reference image, and the lateral resolution of the reference image.

[0107] Finally, the drift is calculated based on the mean, magnification, width, and lateral resolution.

[0108] In an optional embodiment, the specific formula for calculating the drift amount is as follows:

[0109]

[0110] Among them, SreenOffset is the drift amount, PixelOffset is the mean, SreenPixel is the lateral resolution, PhotoWidth is the width, and Mag is the magnification.

[0111] In an embodiment of the present invention, the average length of the first matching point pair is used as the pixel drift amount, and the drift amount of the image to be calibrated is calculated based on the relationship between the obtained pixel drift amount, the magnification of the scanning electron microscope, the width of the reference image, and the lateral resolution of the reference image and the drift amount of the image.

[0112] In one example, in the above step S104, the electrical displacement calibration step is calculated in the following manner:

[0113] Firstly, according to the relationship between the drift amount and the electrical displacement step, the drift amount corresponding to the unit electrical displacement step is calculated.

[0114] Then, the drift of the calibration image relative to the reference image is divided by the drift corresponding to the unit electrical displacement step to obtain the electrical displacement calibration step. The specific calculation formula is as follows:

[0115]

[0116] Among them, ElectricalOffset represents the electrical displacement calibration step, SreenOffset represents the drift amount, and ElectricalStep represents the drift amount corresponding to the unit electrical displacement step.

[0117] The working process of an image drift calibration method based on a scanning electron microscope is described below through a specific embodiment. Figure 2 This is the flowchart of the method, in which the image format conversion is to convert the collected image from CxImage format to Mat format of RGB three-color channels, the grayscale conversion refers to converting the Mat format image of RGB three-color channels into the corresponding grayscale image, and cropping is used to remove the bottom label of the SEM image overlap. In the implementation of this method, firstly, based on the ORB feature point detection algorithm, the matching point pairs are initially obtained; then, the corner points are screened using custom judgment conditions, and the wrong matching point pairs are discarded; finally, the electric displacement drive is performed based on the composite mapping calibration to realize the automatic calibration of image drift.

[0118] Among them, the specific steps of feature point detection and matching include:

[0119] Step b1: construct an n-level scale pyramid and scale it based on the scale factor to obtain n images.

[0120] Step b2: Use the Harris corner point response function to calculate the response value R and sort it, and extract the first N feature points with the largest R and retain them.

[0121] Step b3: Use the grayscale centroid method / moment method to determine the direction of the feature point.

[0122] Step b4: Randomly select several point pairs in the neighborhood of the feature point, compare the pixel sizes of the point pairs, perform the following binary assignment, and finally form a binary code, i.e., the feature descriptor.

[0123]

[0124] Among them, p(x), p(y) represent the pixel values ​​of random points x and y.

[0125] Step b5: Use the Hamming distance to calculate 1 to n in the feature description, and sort based on the distance to obtain matching point pairs.

[0126] Before performing corner point screening, the feature points are traversed and converted into coordinates, and then the corner point screening is performed. The specific steps include:

[0127] Step c1: Judgment basis: Based on the on-machine test using a scanning electron microscope, the characteristics of the incorrectly matched point pairs are determined, including the tilt angle and length.

[0128] Step c2: Use the Pythagorean theorem to calculate the length and inclination angle of the line connecting the matching point pairs.

[0129] a=1024-Point x1 +Point x2

[0130] b=Abs(Point y2 -Point y1 )

[0131]

[0132] Among them, Length represents the straight-line distance between matching point pairs. x1 ,Point y1 Represents the coordinates of the first feature point in a matching point pair, Point x2 ,Point y2 Characterizes the coordinates of the second feature point in the matching point pair.

[0133] Angle=asin(b / Length)×180 / π

[0134] Here, Angle represents the tilt angle.

[0135] Step c3: Customize the length range and tilt angle range to filter out outliers.

[0136] The specific steps of composite mapping calibration include:

[0137] Step d1: Based on the coordinates of each inlier point, the pixel drift amounts of all inliers are counted, and the mean value is used to represent the pixel drift length.

[0138] Step d2: Determine the drift amount of the image based on the relationship among the reference photo, the magnification, and the pixel drift length.

[0139] SreenOffset=PixelOffset / SreenPixel / PhotoWidth*Mag

[0140] Among them, SreenOffset represents the drift amount of the image, PixelOffset represents the pixel drift length, SreenPixel represents the scanned image width in pixels, PhotoWidth represents the physical quantity of the reference photo width, and Mag represents the magnification.

[0141] Step d3: Determine the drift amount corresponding to the electric displacement step based on the electric displacement and drift amount test.

[0142] Step d4: Drift calibration: Calculate the electrical displacement calibration step based on the screen drift amount corresponding to the electrical displacement step and the screen drift amount, and calibrate it.

[0143] ElectricalOffset=SreenOffset / ElectricalStep

[0144] Among them, ElectricalOffset represents the electrical displacement calibration step, SreenOffset represents the drift of the image, and ElectricalStep represents the drift corresponding to the electrical displacement step.

[0145] In an embodiment of the present invention, the image drift automatic calibration process is as follows: first, turn on the maintenance device (mechanical pump, molecular pump, etc.) required for the scanning electron microscope to work, open the host computer interface, and observe the vacuum degree until the image viewing conditions are met; then, adjust the high voltage, beam current, filament current, objective lens and other parameters until a clear scanning image is focused; then, click the image calibration button corresponding to the icon bar to enter the real-time calibration mode. During the real-time calibration process, it can be observed that the electric displacement is moving to reduce the degree of image drift caused by the electron beam, sample stage, etc.; finally, click the image calibration button again to exit the real-time calibration mode. In this process, no manual intervention is required, and photos of the current scanning image are automatically taken at a certain time interval to achieve image drift calibration. The method provided in the embodiment of the present invention was tested based on standard samples, and the test data showed that the image drift was reduced by 66% during the continuous 15-minute calibration process.

[0146] Based on the same inventive concept, the embodiment of the present invention also provides an image drift calibration device based on a scanning electron microscope, such as Figure 3 As shown, the device comprises:

[0147] The acquisition module 301 is used to acquire the image to be calibrated and the reference image taken by the scanning electron microscope; for details, please refer to the description of step S101 in the above embodiment, which will not be repeated here.

[0148] The matching module 302 is used to perform feature point matching on the image to be calibrated and the reference image to obtain a plurality of first matching point pairs; for details, please refer to the description of step S102 in the above embodiment, which will not be repeated here.

[0149] The first calculation module 303 is used to calculate the drift of the image to be calibrated relative to the reference image according to each first matching point pair; for details, please refer to the description of step S103 in the above embodiment, which will not be repeated here.

[0150] The second calculation module 304 is used to calculate the electric displacement calibration step according to the drift amount and the relationship between the drift amount and the electric displacement step. The electric displacement calibration step is used to control the scanning electron microscope to achieve drift calibration of the image to be calibrated. For details, please refer to the description of step S104 in the above embodiment, which will not be repeated here.

[0151] Through the above device, by comparing the feature points of the image to be calibrated with the reference image, a first feature point matching pair is obtained, and the drift of the image to be calibrated relative to the reference image is calculated based on the feature points in the first feature point matching pair. Further, based on the relationship between the drift amount and the electric displacement step, the electric displacement calibration step is calculated, thereby adjusting the scanning electron microscope to overcome image drift and improve user experience.

[0152] In one example, the matching module 302 includes:

[0153] The feature extraction submodule is used to extract features from the image to be calibrated and the reference image respectively to obtain multiple first feature points of the image to be calibrated and multiple second feature points of the reference image; for details, please refer to the description in the above embodiment and will not be repeated here.

[0154] The matching submodule is used to perform feature point matching on each first feature point and each second feature point to obtain a plurality of second matching point pairs; for details, please refer to the description in the above embodiment and will not be repeated here.

[0155] The screening submodule is used to screen the second matching point pairs to obtain the first matching point pairs that meet the preset conditions. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0156] In one example, the feature extraction submodule includes:

[0157] The feature extraction unit is used to extract features from the image to be calibrated and the reference image respectively to obtain multiple third feature points of the image to be calibrated and multiple fourth feature points of the reference image; for details, please refer to the description in the above embodiment and will not be repeated here.

[0158] The first calculation unit is used to calculate the response value of each third characteristic point and the response value of each fourth characteristic point; for details, please refer to the description in the above embodiment, which will not be repeated here.

[0159] The first sorting unit is used to sort the third feature points in descending order of response value, and select a preset number of third feature points with top response values ​​as the first feature points; for details, please refer to the description in the above embodiment, which will not be repeated here.

[0160] The second sorting unit is used to sort the fourth feature points in descending order of response value, and select a preset number of fourth feature points with top response values ​​as the second feature points. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0161] In one example, the screening submodule includes:

[0162] The second calculation unit is used to calculate the length and tilt angle of each second matching point pair, and the length and tilt angle of the second matching point pair are obtained by the coordinates of the first feature point and the coordinates of the second feature point in the second matching point pair; for details, please refer to the description in the above embodiment and will not be repeated here.

[0163] The screening unit is used to select the second matching point pairs whose length is less than the preset length and whose inclination angle is less than the preset angle as the first matching point pairs. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0164] In one example, the first calculation module 303 includes:

[0165] The third calculation unit is used to calculate the length of the first matching point pair for each first matching point pair, and the length of the first matching point pair is obtained by the coordinates of the first feature point and the coordinates of the second feature point in the first matching point pair; for details, please refer to the description in the above embodiment, which will not be repeated here.

[0166] The fourth calculation unit is used to calculate the average length of each first matching point pair; the details refer to the description in the above embodiment, which will not be repeated here.

[0167] The first acquisition unit is used to acquire the magnification of the scanning electron microscope, the width of the reference image and the lateral resolution of the reference image; the details are as described in the above embodiment and will not be repeated here.

[0168] The fifth calculation unit is used to calculate the drift amount according to the mean value, the magnification, the width and the lateral resolution. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0169] In one example, the fifth calculation unit calculates the drift amount by the following formula:

[0170]

[0171] Among them, SreenOffset is the drift amount, PixelOffset is the mean value, SreenPixel is the lateral resolution, PhotoWidth is the width, and Mag is the magnification. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0172] In one example, the matching submodule includes:

[0173] The second acquisition unit is used to acquire a feature point descriptor of each first feature point and a feature point descriptor of each second feature point; for details, please refer to the description in the above embodiment, which will not be repeated here.

[0174] The matching unit is used to match the first feature points with the second feature points according to the feature point descriptors of the first feature points and the feature point descriptors of the second feature points to obtain the second matching point pairs. For details, please refer to the description in the above embodiment, which will not be repeated here.

[0175] The specific definition and beneficial effects of the above-mentioned device can be found in the definition of the image drift calibration method based on the scanning electron microscope above, which will not be repeated here. The above-mentioned modules can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0176] Figure 4 FIG. 1 is a schematic diagram of a hardware structure of a computer device according to an exemplary embodiment. Figure 4 As shown, the device includes one or more processors 410 and a memory 420, and the memory 420 includes a persistent memory, a volatile memory, and a hard disk. Figure 4 A processor 410 is taken as an example. The device may also include: an input device 430 and an output device 440.

[0177] The processor 410, the memory 420, the input device 430 and the output device 440 may be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0178] The processor 410 may be a central processing unit (CPU). The processor 410 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips. A general-purpose processor may be a microprocessor or the processor may be any conventional processor.

[0179] The memory 420 is a non-transient computer-readable storage medium, including a persistent memory, a volatile memory, and a hard disk, and can be used to store non-transient software programs, non-transient computer executable programs, and modules, such as program instructions / modules corresponding to the image drift calibration method based on a scanning electron microscope in the embodiment of the present application. The processor 410 executes various functional applications and data processing of the server by running the non-transient software programs, instructions, and modules stored in the memory 420, that is, implementing any of the above-mentioned image drift calibration methods based on a scanning electron microscope.

[0180] The memory 420 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data used as needed, etc. In addition, the memory 420 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 420 may optionally include a memory remotely arranged relative to the processor 410, and these remote memories may be connected to the data processing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0181] The input device 430 can receive input digital or character information and generate signal input related to user settings and function control. The output device 440 can include display devices such as display screens.

[0182] One or more modules are stored in the memory 420, and when executed by one or more processors 410, the execution is as follows: Figure 1 The method shown.

[0183] The above-mentioned product can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not described in detail in this embodiment, please refer to Figure 1 Related description of the illustrated embodiment.

[0184] The embodiment of the present invention further provides a non-transitory computer storage medium, which stores computer executable instructions, and the computer executable instructions can execute the method in any of the above method embodiments. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0185] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0186] The above is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features applied herein.

Claims

1. A method for image drift calibration based on a scanning electron microscope, characterized in that: The method comprises: Acquire an image to be calibrated and a reference image taken by a scanning electron microscope; Performing feature point matching on the image to be calibrated and the reference image to obtain a plurality of first matching point pairs; Calculating the drift amount of the image to be calibrated relative to the reference image according to each of the first matching point pairs, including: for each of the first matching point pairs, calculating the length of the first matching point pair, the length of the first matching point pair being obtained by the coordinates of the first feature point and the coordinates of the second feature point in the first matching point pair; calculating the average of the lengths of each of the first matching point pairs; obtaining the magnification of the scanning electron microscope, the width of the reference image, and the lateral resolution of the reference image; calculating the drift amount according to the average, the magnification, the width, and the lateral resolution; The electrical displacement calibration step is calculated according to the drift amount and the relationship between the drift amount and the electrical displacement step. The electrical displacement calibration step is used to control the scanning electron microscope to achieve drift calibration of the image to be calibrated.

2. The method according to claim 1, characterized in that Performing feature point matching on the image to be calibrated and the reference image to obtain a plurality of first matching point pairs includes: Performing feature extraction on the image to be calibrated and the reference image respectively to obtain a plurality of first feature points of the image to be calibrated and a plurality of second feature points of the reference image; Performing feature point matching on each of the first feature points and each of the second feature points to obtain a plurality of second matching point pairs; The second matching point pairs are screened to obtain the first matching point pairs that meet a preset condition.

3. The method according to claim 2, characterized in that Extracting features from the image to be calibrated and the reference image respectively to obtain a plurality of first feature points of the image to be calibrated and a plurality of second feature points of the reference image, including: Performing feature extraction on the image to be calibrated and the reference image respectively to obtain a plurality of third feature points of the image to be calibrated and a plurality of fourth feature points of the reference image; Calculating the response value of each of the third characteristic points and the response value of each of the fourth characteristic points; Sorting the third feature points in descending order of response values, and selecting a preset number of third feature points with top response values ​​as the first feature points; The fourth feature points are sorted in descending order of response values, and the preset number of fourth feature points with top response values ​​are selected as the second feature points.

4. The method according to claim 2, characterized in that: Screening each of the second matching point pairs to obtain each of the first matching point pairs that meets a preset condition includes: Calculating the length and the tilt angle of each of the second matching point pairs, where the length and the tilt angle of the second matching point pairs are obtained by the coordinates of the first feature point and the second feature point in the second matching point pairs; A second matching point pair whose length is smaller than a preset length and whose inclination angle is smaller than a preset angle is used as each of the first matching point pairs.

5. The method according to claim 1, characterized in that Calculating the drift amount according to the mean value, the magnification, the width and the lateral resolution includes: Among them, SreenOffset is the drift amount, PixelOffset is the mean value, SreenPixel is the lateral resolution, PhotoWidth is the width, and Mag is the magnification.

6. The method according to claim 2, characterized in that Performing feature point matching on each of the first feature points and each of the second feature points to obtain a plurality of second matching point pairs, including: Obtaining a feature point descriptor of each of the first feature points and a feature point descriptor of each of the second feature points; Feature point matching is performed on each of the first feature points and each of the second feature points according to the feature point descriptors of each of the first feature points and the feature point descriptors of each of the second feature points to obtain each of the second matching point pairs.

7. An image drift calibration device based on a scanning electron microscope, characterized in that: The device comprises: An acquisition module, used for acquiring the image to be calibrated and the reference image taken by the scanning electron microscope; A matching module, used for matching feature points of the image to be calibrated and the reference image to obtain a plurality of first matching point pairs; A first calculation module is used to calculate the drift amount of the image to be calibrated relative to the reference image according to each of the first matching point pairs, including: for each of the first matching point pairs, calculating the length of the first matching point pair, the length of the first matching point pair is obtained by the coordinates of the first feature point and the coordinates of the second feature point in the first matching point pair; calculating the average of the lengths of each of the first matching point pairs; obtaining the magnification of the scanning electron microscope, the width of the reference image and the lateral resolution of the reference image; and calculating the drift amount according to the average, the magnification, the width and the lateral resolution; The second calculation module is used to calculate the electric displacement calibration step according to the drift amount and the relationship between the drift amount and the electric displacement step. The electric displacement calibration step is used to control the scanning electron microscope to achieve drift calibration of the image to be calibrated.

8. A computer device, characterized in that: The invention comprises a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the steps of the image drift calibration method based on a scanning electron microscope according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the scanning electron microscope-based image drift calibration method according to any one of claims 1 to 6 are implemented.

Images