Atomic force microscope morphology measurement error correction method and device

Abstract

The invention discloses an atomic force microscope morphology measurement error correction method and device, and belongs to the technical field of image processing. Comprising the steps that a first scanning image and a second scanning image are obtained, the first scanning image is obtained by scanning a sample through structured illumination obvious micro-imaging SIM, and the second scanning image is obtained by scanning the sample through an atomic force microscope AFM; according to the first scanning image, morphology measurement error detection is carried out on the second scanning image to obtain an error position, and the error position is a position where morphology measurement errors exist in the second scanning image; and correcting the morphology measurement error of the error position according to the first scanning image to obtain a target morphology image. Therefore, the morphology measurement error caused by the acting force between the probe and the sample can be guided and corrected through the SIM optical data, so that the accurate sample morphology can be obtained, and the accuracy and reliability of sample morphology measurement are improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and particularly to a method and device for correcting the topography measurement error of an atomic force microscope. Background Art

[0002] Atomic Force Microscopy (AFM) is an important tool for characterizing nano-scale topography and mechanical properties by the interaction between a probe and the surface of a sample, and is widely used in the fields of materials science, biomedicine, microelectronics, etc. However, during the observation process of an atomic force microscope, when the probe contacts the surface of the sample, the force between the probe and the sample may cause deformation of a local area of the sample or displacement of the overall structure, and this mechanical perturbation will cause systematic errors in the surface topography data (such as height and roughness distortion), especially significant in soft materials (such as biological tissues, polymers) or brittle materials (such as two-dimensional crystals).

[0003] In the related art, when measuring the topography of a sample by an atomic force microscope, the tapping mode is usually used to reduce the negative impact of the probe force. The tapping mode is a measurement mode that indirectly obtains the surface topography of the sample by adjusting the amplitude and phase changes of the probe. Compared with the traditional contact mode, it can reduce the lateral shear force and the risk of surface scratches.

[0004] In the above method, although optimizing the probe amplitude and phase signals can, to a certain extent, reduce the direct damage when the probe contacts the surface of the sample, however, the force between the probe and the sample will inevitably cause local or overall displacement, resulting in a certain error in the measured surface topography of the sample, making the sample topography measurement inaccurate. Summary of the Invention

[0005] This application provides a method, device, equipment and storage medium for correcting the topography measurement error of an atomic force microscope, which can detect the topography measurement error of the AFM mechanical image (the second scanned image) according to the SIM optical image (the first scanned image), and correct the detected topography measurement error according to the SIM optical image, so as to realize guiding the correction of the topography measurement error caused by the force between the probe and the sample through optical data, and further obtain an accurate sample topography, improving the accuracy and reliability of the sample topography measurement. The technical solutions include the following content.

[0006] In a first aspect, a method for correcting the topography measurement error of an atomic force microscope is provided, and the method includes:

[0007] Obtain a first scanned image and a second scanned image, where the first scanned image is obtained by scanning a sample through structured illumination microscopy (SIM), and the second scanned image is obtained by scanning the sample through atomic force microscopy (AFM);

[0008] According to the first scanned image, perform topographic measurement error detection on the second scanned image to obtain an error position, where the error position is the position in the second scanned image where there is a topographic measurement error;

[0009] According to the first scanned image, correct the topographic measurement error at the error position to obtain a target topographic image.

[0010] In this application, obtaining the first scanned image and the second scanned image is equivalent to obtaining an optical image obtained by scanning the sample through SIM and a mechanical image obtained by scanning the sample through AFM. Then, according to the first scanned image, perform topographic error detection on the second scanned image to obtain an error position, that is, using the SIM optical image as a guide to detect the position in the second scanned image where there is a topographic measurement error. Then, using the SIM optical image as a guide again, correct the topographic measurement error at the error position to obtain a target topographic image. In this application, by performing topographic measurement error detection on the AFM mechanical image (the second scanned image) according to the SIM optical image (the first scanned image) and correcting the detected topographic measurement error according to the SIM optical image, it is possible to achieve guiding the correction of the topographic measurement error caused by the force between the probe and the sample through the SIM optical data, thereby obtaining an accurate sample topography and improving the accuracy and reliability of sample topography measurement.

[0011] Optionally, the step of obtaining the second scanned image includes:

[0012] Determine a target area from the first scanned image;

[0013] Scan the target area of the sample through AFM to obtain the second scanned image.

[0014] Optionally, the performing topographic measurement error detection on the second scanned image according to the first scanned image to obtain an error position includes:

[0015] Extract features from the first scanned image and the second scanned image to obtain a first scanned feature and a second scanned feature;

[0016] According to the first scanned feature and the second scanned feature, determine a third scanned feature and a fourth scanned feature, where the third scanned feature and the fourth scanned feature are respectively the features that match each other in the first scanned feature and the second scanned feature;

[0017] Determine a target transformation matrix according to the third scanning feature and the fourth scanning feature;

[0018] Perform topography measurement error detection on the second scanned image according to the target transformation matrix, the first scanning feature, and the second scanning feature, to obtain the error position.

[0019] Optionally, the performing topography measurement error detection on the second scanned image according to the target transformation matrix, the first scanning feature, and the second scanning feature, to obtain the error position includes:

[0020] Determine a reference transformation feature according to the target transformation matrix and the first scanning feature;

[0021] Align the reference transformation feature with the second scanning feature to obtain a plurality of feature pairs, where any one of the plurality of feature pairs includes any one feature in the reference transformation feature and the feature at the corresponding position in the second scanning feature;

[0022] Determine the error position according to the plurality of feature pairs.

[0023] Optionally, the determining the error position according to the plurality of feature pairs includes:

[0024] For any one of the plurality of feature pairs, determine the difference between the two features in the feature pair to obtain the error corresponding to the feature pair;

[0025] Determine the position corresponding to the feature pair with an error exceeding a preset error threshold among the plurality of feature pairs as the error position.

[0026] Optionally, the correcting the topography measurement error at the error position according to the first scanned image to obtain a target topography image includes:

[0027] Replace the feature at the error position in the second scanning feature with the feature at the error position in the reference transformation feature to obtain the target topography feature;

[0028] Restore the target topography feature to obtain the target topography image.

[0029] Optionally, the method further includes:

[0030] Generate virtual mechanical features according to the motion law of the sample;

[0031] Interpolate the target topography feature according to the virtual mechanical features to obtain a high-resolution topography feature;

[0032] And restoring the target morphological features to obtain the target morphological image, including:

[0033] Restoring the high-resolution morphological features to obtain the target morphological image.

[0034] Optionally, the method further includes:

[0035] Scanning the sample multiple times through SIM to obtain multiple first scanned images;

[0036] Determining the motion law of the sample according to the multiple first scanned images.

[0037] Optionally, the method further includes:

[0038] Performing denoising and normalization processing on the first scanned image to obtain the preprocessed first scanned image;

[0039] Performing denoising and planarization processing on the second scanned image to obtain the preprocessed second scanned image;

[0040] And, the detecting the morphological measurement error of the second scanned image according to the first scanned image to obtain the error position, including:

[0041] Detecting the morphological measurement error of the preprocessed second scanned image according to the preprocessed first scanned image to obtain the error position.

[0042] In a second aspect, an atomic force microscope morphological measurement error correction device is provided, and the device includes:

[0043] A first acquisition module, configured to acquire a first scanned image and a second scanned image, where the first scanned image is obtained by scanning the sample through structured illumination microscopy SIM, and the second scanned image is obtained by scanning the sample through an atomic force microscope AFM;

[0044] An error detection module, configured to detect the morphological measurement error of the second scanned image according to the first scanned image to obtain an error position, where the error position is the position where the morphological measurement error exists in the second scanned image;

[0045] An error correction module, configured to correct the morphological measurement error at the error position according to the first scanned image to obtain a target morphological image.

[0046] Optionally, the first acquisition module is configured to:

[0047] Determine a target area from the first scanned image;

[0048] The target area of the sample is scanned by AFM to obtain the second scanned image.

[0049] Optionally, the error detection module is configured to:

[0050] Extract features from the first scanned image and the second scanned image to obtain a first scanned feature and a second scanned feature;

[0051] Determine a third scanned feature and a fourth scanned feature according to the first scanned feature and the second scanned feature, where the third scanned feature and the fourth scanned feature are respectively the features that match each other in the first scanned feature and the second scanned feature;

[0052] Determine a target transformation matrix according to the third scanned feature and the fourth scanned feature;

[0053] Perform topography measurement error detection on the second scanned image according to the target transformation matrix, the first scanned feature and the second scanned feature to obtain the error position.

[0054] Optionally, the error detection module is configured to:

[0055] Determine a reference transformation feature according to the target transformation matrix and the first scanned feature;

[0056] Align the reference transformation feature with the second scanned feature to obtain a plurality of feature pairs, where any one of the plurality of feature pairs includes any one feature in the reference transformation feature and the feature at the corresponding position in the second scanned feature;

[0057] Determine the error position according to the plurality of feature pairs.

[0058] Optionally, the error detection module is configured to:

[0059] For any one of the plurality of feature pairs, determine the difference between the two features in the feature pair to obtain the error corresponding to the feature pair;

[0060] Determine the position corresponding to the feature pair with an error exceeding a preset error threshold among the plurality of feature pairs as the error position.

[0061] Optionally, the error correction module is configured to:

[0062] Replace the feature at the error position in the second scanned feature with the feature at the error position in the reference transformation feature to obtain the target topography feature;

[0063] Restore the target topography feature to obtain the target topography image.

[0064] Optionally, the device further includes:

[0065] a generation module, configured to generate virtual mechanical features according to the motion law of the sample;

[0066] an interpolation module, configured to interpolate the target topography features according to the virtual mechanical features to obtain high-resolution topography features;

[0067] and the error correction module is configured to:

[0068] restore the high-resolution topography features to obtain the target topography image.

[0069] Optionally, the device further includes:

[0070] a second acquisition module, configured to perform multiple scans on the sample through SIM to obtain multiple first scan images;

[0071] a determination module, configured to determine the motion law of the sample according to the multiple first scan images.

[0072] Optionally, the device further includes:

[0073] a first preprocessing module, configured to perform denoising and normalization processing on the first scan image to obtain the preprocessed first scan image;

[0074] a second preprocessing module, configured to perform denoising and planarization processing on the second scan image to obtain the preprocessed second scan image;

[0075] and, the error detection module is configured to:

[0076] perform topography measurement error detection on the preprocessed second scan image according to the preprocessed first scan image to obtain the error position.

[0077] In a third aspect, a computer device is provided, where the computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the computer program is executed by the processor, the above-mentioned atomic force microscope topography measurement error correction method is implemented.

[0078] In a fourth aspect, a computer-readable storage medium is provided, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned atomic force microscope topography measurement error correction method is implemented.

[0079] In a fifth aspect, there is provided a computer program product containing instructions, which, when running on a computer, causes the computer to execute the steps of the above-mentioned method for correcting the topography measurement error of an atomic force microscope.

[0080] It can be understood that the beneficial effects of the above-mentioned second aspect, third aspect, fourth aspect, and fifth aspect can be referred to the relevant descriptions in the first aspect above, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0082] Figure 1 is a schematic diagram of the implementation environment of a method for correcting the topography measurement error of an atomic force microscope provided by an embodiment of the present application;

[0083] Figure 2 is a flowchart of a method for correcting the topography measurement error of an atomic force microscope provided by an embodiment of the present application;

[0084] Figure 3 is a flowchart of another method for correcting the topography measurement error of an atomic force microscope provided by an embodiment of the present application;

[0085] Figure 4 is a schematic structural diagram of an apparatus for correcting the topography measurement error of an atomic force microscope provided by an embodiment of the present application;

[0086] Figure 5 is a schematic structural diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0087] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the drawings.

[0088] It should be understood that the "multiple" mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solution of this application, the words "first" and "second" are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first" and "second" do not limit the quantity and execution order, and the words "first" and "second" do not limit them to be different.

[0089] The application scenario of the embodiment of the present application is first described.

[0090] In the related art, in order to solve the deformation of the local area of ​​the sample or the displacement of the overall structure caused by the force between the probe and the sample, it is mainly proposed to reduce the negative impact of the probe force by optimizing the scanning mode and mechanical feedback control.

[0091] First, the sample surface is scanned by tapping mode. Specifically, in tapping mode, the amplitude, frequency and other parameters of the mechanical cantilever can be adjusted so that the needle tip briefly contacts the sample surface periodically, thereby reducing direct damage when the probe contacts the sample surface to a certain extent.

[0092] However, when the tip periodically contacts the sample surface, although there is a force interaction, the force action process is relatively complicated, and the force measurement is not direct and continuous, making it difficult to accurately determine the quantitative relationship between force and sample deformation. Therefore, the tapping mode has limited quantitative analysis capabilities for mechanical parameters, and the complexity of the phase signal makes the interpretation of mechanical data dependent on experience.

[0093] Second, by real-time monitoring of the phase response of the interaction between the probe and the sample, a phase and mechanical property mapping model is established. Specifically, the sample surface is scanned by a high-frequency probe to collect dynamic phase signals. Then, a theoretical correlation between phase changes and local stiffness is established (phase and mechanical property mapping model). Finally, the phase signal is converted into mechanical data, thereby inferring mechanical data such as elastic modulus and adhesion without direct contact with the sample surface.

[0094] However, in the above methods, the sensitivity of the phase signal is affected by the characteristics of the sample material (viscoelasticity, surface charge) and environmental conditions (such as humidity, temperature), resulting in insufficient model generalization ability. For example, for biological samples or non-uniform composite materials, the phase-mechanics relationship is difficult to describe by a unified calibration curve. Secondly, the measurement of the phase relies on high-precision probe calibration, which requires extremely high hardware stability. In addition, the detection sensitivity of the phase to very low forces is insufficient to meet the mechanical measurement requirements of single molecules or ultrathin films.

[0095] Thirdly, optimize the measurement accuracy of AFM by combining multimodal sensing and machine learning techniques. For example, by training a hybrid model based on dynamic deviation and convolutional neural network, then synchronously collecting parameters such as probe amplitude, phase and resonance frequency, and inputting the collected data into the hybrid model. Predict the local mechanical properties and automatically optimize the mechanical parameters through the hybrid model.

[0096] However, the above method is highly sensitive to the quality and diversity of training data, and the computational complexity increases significantly, making it difficult to achieve real-time measurement.

[0097] Although the above three methods can, to a certain extent, reduce the direct damage when the probe contacts the sample surface, they still fail to effectively solve the measurement error caused by sample displacement. Moreover, for biological samples, they may undergo dynamic changes themselves, which makes the error during the measurement process more difficult to control. Therefore, in related technologies, it is still impossible to completely avoid the error caused by sample displacement, especially when measuring relatively soft or deformable samples, the error is more significant.

[0098] In addition, when AFM is used to measure the sample topography, its sampling frequency is low, making it difficult to accurately measure rapidly changing dynamic samples. Specifically, the data acquisition frequency of the atomic force microscope is limited by its point-by-point scanning mechanism and the dynamic characteristics of the electromechanical system, making it difficult to accurately capture rapidly changing topographies. At the same time, the adjustment of feedback control parameters needs to balance the response speed and system stability, further restricting the improvement of resolution. Although the introduction of high-frequency resonant scanning and high-speed optoelectronic detection technologies has improved the resolution of some improved AFM to the millisecond level, under the physical limitations of probe dynamic stability and signal-to-noise ratio, the current AFM is difficult to meet the accurate characterization requirements of ultrafast dynamic processes.

[0099] To this end, the embodiments of the present application propose an atomic force microscope topography measurement error correction method, which can detect topography measurement errors in the AFM mechanical image according to the SIM optical image, and correct the detected topography measurement errors according to the SIM optical image, so as to realize guiding the correction of topography measurement errors caused by the force between the probe and the sample through optical data, and then an accurate sample topography can be obtained, improving the accuracy and reliability of sample topography measurement.

[0100] Next, the technical solutions in the embodiments of the present application will be described with reference to the accompanying drawings in the embodiments of the present application.

[0101] First, the implementation environment involved in an atomic force microscope topography measurement error correction method provided by the embodiments of the present application will be described.

[0102] For example, Figure 1 is a schematic diagram of the implementation environment of an atomic force microscope topography measurement error correction method provided by the embodiments of the present application. Refer to Figure 1 , Figure 1 which includes an atomic force microscope 101, a structured illumination microscope 102, and a computer device 103.

[0103] The atomic force microscope 101 is used to measure the surface topography of the sample to obtain an AFM mechanical image. The atomic force microscope 101 is an important tool for studying the microscopic structure and properties of the surface of substances. Its operation is based on the interaction force between the probe and the atoms on the surface of the sample. Its core component is a cantilever that is very sensitive to weak forces. One end of the cantilever is fixed, and the other end has a sharp probe. When the probe approaches the surface of the sample, an interaction force will be generated between the probe and the atoms on the surface of the sample, such as van der Waals force, electrostatic force, etc. This interaction force will cause the cantilever to bend or vibrate slightly. By detecting changes such as the deformation amount or vibration frequency of the cantilever, the surface topography of the sample can be obtained.

[0104] The structured illumination microscope 102 is used to optically scan the surface topography of the sample through structured light to obtain a SIM optical image. The structured illumination microscope (SIM) is a super-resolution fluorescence microscope that can break through the diffraction limit of traditional optical microscopes and achieve higher-resolution imaging. Specifically, the structured illumination microscope 102 irradiates the sample with light of a specific structure (such as sinusoidal fringe light), so that the high-frequency information of the sample and the structured light are mutually modulated to generate new frequency components. These new frequency components are within the detectable range. By collecting modulated images at different angles and behaviors and then using algorithms to process and reconstruct these images, high-resolution images can be obtained.

[0105] The computer device 103 is used to correct the topography measurement error in the AFM mechanical image by applying an atomic force microscope topography measurement error correction method provided by an embodiment of the present application. Optionally, the computer device 103 may be a terminal device such as a desktop computer or a laptop computer.

[0106] The atomic force microscope 101 and the computer device 103 can communicate through wireless connection or wired connection, and the structured light microscopy 102 and the computer device 103 can communicate through wireless connection or wired connection.

[0107] In the embodiment of the present application, first, the atomic force microscope 101 is used to perform mechanical measurement on the sample to obtain an AFM mechanical image, and the structured light microscopy 102 is used to perform optical scanning on the sample to obtain a SIM optical image. Then, the computer device 103 can obtain the sample topography (AFM mechanical image) measured by the atomic force microscope 101 and the SIM optical image obtained by the optical scanning of the structured light microscopy 102. Then, the topography measurement error in the AFM mechanical image is corrected under the guidance of the SIM optical image, so as to obtain the accurate sample topography.

[0108] Next, the terms involved in the embodiment of the present application are explained.

[0109] 1. Topography measurement error

[0110] In the process of measuring the sample topography by the atomic force microscope, the force between the probe and the sample may cause deformation, damage, etc. in the local area of the sample, which will cause errors in the measured sample topography. This error can be called topography measurement error.

[0111] 2. Feature matching algorithm

[0112] The feature matching algorithm is an algorithm for finding corresponding feature points between different images or different parts of the same image, that is, an algorithm for finding similar feature points in different images. Its working principle mainly involves three key steps: feature extraction, feature description, and feature matching. The main role of feature extraction is to find representative and stable feature points or regions in the image, such as corner points. Feature description can generate a descriptor for each extracted feature point. The descriptor contains the local information of the image around the feature point and has a certain invariance. Feature matching is used to measure the similarity between feature points in different images according to the descriptors of each feature, so as to find matching feature point pairs.

[0113] Common feature matching algorithms include: Scale-Invariant Feature Transform (SIFT), Speeded Up Robust Features (SURF), Oriented Fast and Rotated Brief (ORB), etc.

[0114] 3. Random Sample Consensus (RANSAC)

[0115] RANSAC is used to robustly estimate the parameters of a mathematical model from a set of data containing noise and outliers. Its core idea is to randomly sample a small number of data points from the dataset to estimate the model parameters, and then use this model to test other points in the dataset to see how many points can conform to this model. After multiple samplings and tests, the model with the most inliers is selected as the final result.

[0116] In the embodiments of this application, for the SIM optical image and the AFM mechanical image, a small number of feature point pairs are randomly selected from the set of feature points of the SIM optical image and the AFM mechanical image. Assume there is a transformation matrix that can map the feature points in the SIM optical image to the corresponding feature points in the AFM mechanical image through a certain geometric transformation (such as translation, rotation, scaling, affine transformation, or perspective transformation, etc.). Through this transformation matrix, the conversion between SIM data and AFM data can be realized, and accordingly, the alignment of the SIM optical image and the AFM optical image can be achieved.

[0117] The following provides a detailed explanation of an atomic force microscope topography measurement error correction method provided by the embodiments of this application.

[0118] Figure 2 is a flowchart of an atomic force microscope topography measurement error correction method provided by the embodiments of this application. This method can be applied to a computer device. Refer to Figure 2 and the method includes the following steps.

[0119] Step 201: Obtain a first scanned image and a second scanned image. The first scanned image is obtained by scanning the sample through structured illumination microscopy (SIM), and the second scanned image is obtained by scanning the sample through an atomic force microscope (AFM).

[0120] It should be understood that the first scanned image is an optical image obtained by scanning the sample through SIM, and the second scanned image is a mechanical image obtained by scanning the sample through AFM.

[0121] When scanning a sample through SIM, a movable diffraction grating is placed in the path of the laser beam so that the light from the diffraction grating irradiates the sample, and the optical path is adjusted to ensure that the light uniformly irradiates the wide field of view area of the sample. Then, by changing the angle of the movable diffraction grating, the structured light irradiates the sample at different angles or directions. Using the imaging system of SIM, fluorescence images of the sample are collected at different structured light illumination angles. Finally, the fluorescence images collected at multiple angles are stitched or fused to obtain the optical image obtained by SIM scanning the sample, that is, the first scanned image. Then the computer device can obtain the first scanned image obtained by scanning from the SIM device.

[0122] As a possible way, AFM can directly scan the sample to obtain a second scanned image, and then the computer device obtains the second scanned image from the AFM device.

[0123] Specifically, when scanning a sample through AFM, the probe on the cantilever of the AFM scans the surface of the sample, records the interaction force between the probe and the sample, and simultaneously generates the surface topography information of the sample according to the interaction force between the probe and the sample, that is, generates the second scanned image.

[0124] As another possible way, after scanning and obtaining the first scanned image, the target area can be determined from the first scanned image; the target area of the sample is scanned through AFM to obtain the second scanned image.

[0125] The target area can be an area with a special structure in the sample, such as the area where two cells are in contact. In some embodiments, it is not necessarily to measure the entire picture of the sample, and it may be to measure the topography of a certain part of the sample. Therefore, the target area can also be a specified topography measurement area in the sample.

[0126] In the above methods, it is equivalent to first determining the area to be measured for topography or the area with a special structure from the first scanned image obtained by SIM scanning, and then only scanning the target area through AFM. In this way, repeated scanning of unnecessary areas can be avoided, thereby reducing the contact frequency between the probe and the sample, and further reducing the damage of the probe to the sample.

[0127] Among them, the operation of determining the target area from the first scanned image can be: performing target recognition on the first scanned image to determine the position of the target area in the first scanned image.

[0128] Optionally, the target recognition can be performed on the first scanned image through a target detection algorithm to determine the position of the target area in the first scanned image. Specifically, the feature representation and classifier of the target can be learned through training to detect the target in the image, and the coordinate values of the bounding box where the target area is located can be given, so as to obtain the position of the target area in the first scanned image.

[0129] Exemplarily, the target detection algorithm can be Faster R-CNN (Faster Region-based Convolutional Neural Network), YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), etc., and the embodiments of the present application do not limit this.

[0130] After determining the position of the target area in the first scanned image through the above method, the scanning range of the AFM can be adjusted according to this position, so that the AFM can scan the target area of the sample to obtain the second scanned image.

[0131] It should be noted that after obtaining the first scanned image and the second scanned image, the first scanned image and the second scanned image can be preprocessed to exclude the influence of noise on the scanning result and obtain more accurate first scanned image and second scanned image.

[0132] As an implementation manner, the operation of preprocessing the first scanned image can be: denoising and normalizing the first scanned image to obtain the preprocessed first scanned image.

[0133] In this case, by denoising the first scanned image, the background noise can be eliminated to exclude the influence of the background noise on the scanning result. In addition, by normalizing the first scanned image, the usability of the first scanned image can be improved, and the consistency between the first scanned image and the second scanned image can be maintained, so that the subsequent first scanned image and the second scanned image can be easily compatible.

[0134] Among them, the methods for denoising the first scanned image include but are not limited to mean filtering, median filtering, Gaussian filtering, Wiener filtering, non-local mean filtering or denoising methods based on neural network models. Preferably, non-local mean filtering can be used for denoising the first scanned image.

[0135] Non - local means filtering searches for regions similar to the neighborhood of the current pixel in the entire image, and then the pixels in these pixel regions are weighted and averaged to estimate the value of the current pixel. Since it utilizes the similarity widely existing in the image, it can better preserve the texture and structural information of the image while removing noise, especially for some complex textures and details in SIM optical images, having a good denoising effect. Thus, the image quality of the first scanned image can be better improved.

[0136] Among them, the ways to normalize the first scanned image may include but are not limited to pixel normalization, gray - level normalization, image - size normalization, illumination normalization, etc. Preferably, illumination normalization can be performed on the first scanned image.

[0137] Since the first scanned image is an optical image generated based on images scanned at different structured - light angles, the first scanned image is affected by illumination changes, and there are also significant differences in illumination changes between the first scanned image and the second scanned image.

[0138] Illumination normalization refers to unifying the illumination conditions in the image through a certain algorithm or technology, making it have similar appearance characteristics under different illumination conditions, so as to improve the image quality by eliminating or weakening the influence of illumination changes on the image. Optionally, the ways of illumination normalization may include but are not limited to histogram equalization, Gaussian pyramid method, bilateral filtering, etc.

[0139] As another implementation manner, the operation of pre - processing the second scanned image can be: denoising and flattening the second scanned image to obtain the pre - processed second scanned image.

[0140] During the process of scanning the sample by AFM, interference factors such as uneven substrate, non - perpendicularity between the tip and the sample, non - linearity of the scanning tube, and thermal drift will cause the collected data to be distorted, such as scanning - line tilt, plane distortion, etc., which will lead to inaccurate topographic measurement errors of the second scanned image in the subsequent process. Therefore, it is necessary to flatten the second scanned image.

[0141] In this case, by denoising and flattening the second scanned image, the influence caused by factors such as uneven substrate or tilt introduced during the scanning process, as well as the influence of background noise, can be eliminated, and the image quality of the second scanned image can be improved, so that the true topography of the sample surface can be more accurately analyzed subsequently.

[0142] Among them, the ways to flatten the second scanned image may include 0 - order flattening, 1 - order flattening, polynomial flattening, etc. Different flattening processing methods can be selected according to different topographic features of the sample surface. The embodiments of the present application do not limit this.

[0143] The 0th-order flattening is used to adjust the global plane height value of the image. Specifically, the misalignment between different scan lines is eliminated by subtracting an average height from each scan line. The 1st-order flattening can not only remove height differences but also compensate for the tilt of the scan lines. The polynomial flattening corrects wavy or arc-shaped distortions by fitting higher-order polynomials, which can handle complex surface topographies.

[0144] When flattening the second scanned image, corresponding smoothing parameters can be set. The smoothing parameters correspond to the 0th-order flattening, 1st-order flattening, or polynomial flattening. That is to say, the 0th-order flattening, 1st-order flattening, or polynomial flattening has corresponding smoothing parameters. When a corresponding smoothing parameter is set, the corresponding flattening operation can be performed, thereby realizing the flattening process of the second scanned image.

[0145] Among them, the specific method for denoising the second scanned image is similar to the method for denoising the first scanned image described above, which will not be elaborated here.

[0146] Furthermore, after preprocessing the first scanned image and the second scanned image, the topographic measurement error of the preprocessed second scanned image can be detected based on the preprocessed first scanned image, so that more accurate topographic measurement error detection can be achieved.

[0147] Step 202: Detect the topographic measurement error of the second scanned image according to the first scanned image to obtain the error position, which is the position where there is a topographic measurement error in the second scanned image.

[0148] Since the method of structured light scanning will not damage the sample, the surface image of the sample obtained by scanning the sample through SIM is relatively accurate.

[0149] In this case, by detecting the topographic measurement error of the second scanned image according to the first scanned image, the first scanned image can be used as a guide to detect the topographic measurement error in the second scanned image. In this way, the topographic measurement error existing in the second scanned image can be accurately detected, and then a relatively accurate error position can be determined.

[0150] Specifically, the operation of step 202 may include the following steps (1)-(4).

[0151] (1) Extract features from the first scanned image and the second scanned image to obtain the first scan features and the second scan features.

[0152] The first scan feature is obtained by extracting features from the first scan image, that is, the first scan feature is the image feature of the first scan image. The second scan feature is obtained by extracting features from the second scan image, that is, the second scan feature is the image feature of the second scan image.

[0153] In some embodiments, key feature points and surface contour lines of the first scan image and the second scan image can be extracted. For example, the key feature points can be special parts such as protrusions, depressions, and corners on the surface of the sample, which can characterize the key features of the sample morphology. The surface contour line can be the general planar structure and boundary of the sample surface, which can characterize information such as the surface structure and boundary of the sample.

[0154] Since the key feature points and surface contour lines can effectively describe the features of the sample surface, therefore, by extracting the key features and surface contour lines, the image features of the first scan image and the second scan image can be better characterized, making the first scan feature and the second scan feature more accurate.

[0155] As a possible way, the operation of step (1) above can be: respectively extracting features from the first scan image and the second scan image through an encoder to obtain the first scan feature and the second scan feature.

[0156] Among them, the encoder can be composed of multiple convolutional layers. Since the convolutional layer can extract context features in the image, the shallow convolutional layer can better retain the global features of the image, and the deep convolutional layer can better extract the deep and local features of the image. Therefore, through the encoder, shallow global features and deep local features can be extracted, so that more comprehensive features can be extracted, making it possible to extract accurate first scan features and second scan features.

[0157] As another possible way, the operation of step (1) above can also be: respectively extracting features from the first scan image and the second scan image through a corner detection algorithm and an edge detection algorithm to obtain the first scan feature and the second scan feature.

[0158] Among them, the corner detection algorithm is used to detect the key feature points in the first scan image and the second scan image, and the edge detection algorithm is used to detect the edge contour lines in the first scan image and the second scan image.

[0159] For example, for the first scanned image, key feature points in the first scanned image can be detected by Harris corner detection, and edge contour points in the first scanned image can be detected by the Canny operator, so that the first scanned features can be obtained. Similarly, for the second scanned image, key feature points in the second scanned image can be detected by Harris corner detection, and edge contour points in the second scanned image can be detected by the Canny operator, so that the second scanned features can be obtained.

[0160] (2) Determine the third scanned features and the fourth scanned features according to the first scanned features and the second scanned features.

[0161] The third scanned features and the fourth scanned features are respectively the features that match each other in the first scanned features and the second scanned features. In other words, the third scanned features and the fourth scanned features are respectively the features with similarity at the corresponding positions in the first scanned features and the second scanned features.

[0162] Since both the first scanned image and the second scanned image are images obtained by scanning the same sample, there will be features with similarity between the first scanned features and the second scanned features to a certain extent. By finding these features with similarity, it is easier to align the first scanned image and the second scanned image, and then the accurate detection of the topography measurement error can be realized.

[0163] A possible way is to perform feature matching on the first scanned features and the second scanned features to determine the third scanned features and the fourth scanned features.

[0164] Optionally, the first scanned features and the second scanned features can be feature-matched by a feature matching algorithm. For example, the feature matching algorithm can be SIFT, SUFT, etc.

[0165] Specifically, the operation of feature matching the first scanned features and the second scanned features can be: for each feature point in the first scanned features, generate a corresponding first descriptor for each feature point in the first scanned features, and for each feature point in the second scanned features, generate a corresponding second descriptor for each feature point in the second scanned features; for any one first descriptor, calculate the similarity between this first descriptor and each second descriptor; in the case where the similarity between a first descriptor and a second descriptor is greater than or equal to a preset similarity threshold, the feature corresponding to this first descriptor can be determined as the third scanned feature, and the feature corresponding to this second descriptor can be determined as the fourth scanned feature.

[0166] Among them, a descriptor is a vector. A descriptor can contain local information of the image around a feature point. Since the extracted feature points themselves cannot represent sufficient information, a descriptor needs to be generated for each feature point to represent the local information of the image around the feature point.

[0167] As a possible way, for example, a 16×16 region can be taken around a feature point through a SIFT descriptor. Then each region is divided into 4×4 sub-regions, and the gradient histograms in 8 directions within each sub-region are calculated. Finally, a 128-dimensional description vector is formed, and this 128-dimensional description vector can be used as the descriptor corresponding to this feature point.

[0168] Optionally, the operation of calculating the similarity between a first descriptor and a second descriptor may include but is not limited to calculating the Euclidean distance, Hamming distance, cosine similarity, etc. between a first descriptor and a second descriptor. The embodiments of the present application do not limit this.

[0169] When the similarity between a first descriptor and a second descriptor is greater than or equal to a preset similarity threshold, it indicates that this first descriptor and this second descriptor are relatively similar. That is to say, the local information of the image around a feature point in the first scan feature is relatively similar to the local information of the image around a feature point in the second scan feature. Thus, it can be shown that the feature corresponding to this first descriptor and the feature corresponding to this second descriptor are matched. Therefore, the feature corresponding to the first descriptor can be determined as the third scan feature, and the feature corresponding to this second descriptor can be determined as the fourth scan feature.

[0170] It should be understood that the third scan feature and the fourth scan feature obtained in the above manner are corresponding, specifically manifested as a one-to-one correspondence between multiple feature points in the third scan feature and multiple feature points in the fourth scan feature.

[0171] (3) Determine the target transformation matrix according to the third scan feature and the fourth scan feature.

[0172] Since the third scan feature is a feature in the SIM optical image (the first scan image), that is, the SIM optical feature, and the fourth scan feature is a feature in the AFM mechanical image (the second scan image), that is, the AFM mechanical feature. The target transformation matrix can realize the conversion from the SIM optical feature to the AFM mechanical feature. Therefore, the target transformation matrix is used to represent the conversion relationship between the SIM optical feature and the AFM mechanical feature.

[0173] In addition, since the first scanned image is an optical image obtained by structured light scanning, while the second scanned image is an image corresponding to mechanical data obtained by AFM scanning. Therefore, for the first scanning feature and the second scanning feature, their corresponding feature representation methods are different. The first scanning feature can be a feature corresponding to optical data, while the second scanning feature can be a feature corresponding to mechanical data. Then, when performing topography error detection on the two, their comparison is involved. However, when directly comparing the two, they cannot be directly compared due to different data types.

[0174] In this case, in the embodiments of the present application, by finding a target transformation matrix, it is equivalent to finding the corresponding relationship between optical data and mechanical data, so that the SIM optical features can be easily converted into AFM mechanical features, breaking through the technical barrier that cannot be compared due to different data types, and thus providing convenience for topography error detection.

[0175] Optionally, the random sample consensus algorithm can be used to determine the target transformation matrix according to the third scanning feature and the fourth scanning feature.

[0176] As a possible way, the operation of determining the target transformation matrix according to the third scanning feature and the fourth scanning feature can be: randomly select a plurality of first feature points from the third scanning feature, and select a corresponding plurality of second feature points from the fourth scanning feature; randomly generate a transformation matrix; for any one of the plurality of first feature points, and the corresponding second feature point, transform this first feature point through this transformation matrix to obtain a third feature point; calculate the similarity between this third feature point and the corresponding second feature point. When the similarity between this third feature point and the corresponding second feature point is greater than or equal to the target similarity threshold, determine this third feature point as the inlier corresponding to this transformation matrix, otherwise, determine it as an outlier; through multiple iterations of the above operation, determine the transformation matrix with the largest number of inliers among the plurality of transformation matrices as the target transformation matrix.

[0177] By performing the above transformation operation on each first feature point, a plurality of third feature points can be obtained, and the similarity between each third feature point and the corresponding second feature point can be calculated. Subsequently, according to the similarity, the number of inliers among the plurality of third feature points can be determined. The number of inliers corresponds to this transformation matrix. And by performing the above method multiple times, the number of inliers corresponding to a plurality of transformation matrices can be obtained. Then, determine the transformation matrix with the largest number of inliers among the plurality of transformation matrices as the target transformation matrix.

[0178] The third feature point is the AFM mechanical feature corresponding to the converted SIM optical feature.

[0179] The inlier is used to identify that after a SIM optical feature is converted into an AFM mechanical feature, it is close to the mechanical feature at the corresponding position in the fourth scan feature. That is, a transformation matrix can be used to convert a SIM optical feature into the corresponding AFM mechanical feature.

[0180] In this case, the target transformation matrix is determined by the above possible methods, so that the conversion between the SIM optical feature and the AFM mechanical feature can be realized through the target transformation matrix. In this way, the technical barrier that cannot be compared due to different data types can be broken through, making it easier to detect the topography measurement error subsequently.

[0181] (4) According to the target transformation matrix, the first scan feature, and the second scan feature, perform topography measurement error detection on the second scan image to obtain the error position.

[0182] In the above steps (1)-(3), a conversion relationship that can convert the SIM optical feature into the corresponding AFM mechanical feature is found. Then, through this relationship, the feature corresponding to the optical data can be easily converted into the feature corresponding to the mechanical data, so that the second scan feature and the first scan feature can be easily compared. That is, from the perspective of data consistency, a convenient detection method for topography measurement error detection is provided, so that the error position can be conveniently determined.

[0183] Specifically, the operation of step (4) can be: according to the target transformation matrix and the first scan feature, determine the reference transformation feature; align the reference transformation feature with the second scan feature to obtain multiple feature pairs; according to the multiple feature pairs, determine the error position.

[0184] The reference transformation feature is the feature corresponding to the mechanical data obtained by converting the first scan feature corresponding to the optical data through the target transformation matrix.

[0185] Any one of the multiple feature pairs includes any one feature in the reference transformation feature and the feature at the corresponding position in the second scan feature.

[0186] The above method is that first, the first scan feature corresponding to the optical data is transformed and converted into the feature corresponding to the mechanical data. Then, the reference transformation feature and the second scan feature are aligned in space. Specifically, the positions where the matching features of the two are located can be aligned first, so as to realize the correspondence between the features at other positions. After the reference transformation feature and the second scan feature are aligned, multiple feature pairs can be obtained. The multiple feature pairs are equivalent to the features at the corresponding positions after the reference transformation feature and the second scan feature are aligned. Then, according to the features at the corresponding positions after the reference transformation feature and the second scan feature are aligned, it can be determined whether there is a topography measurement error at the corresponding position of the second scan image.

[0187] In this way, the first scanned feature is transformed by the target transformation matrix, and then the transformed first scanned feature is aligned with the second scanned feature, so as to obtain the features at the corresponding positions after the reference transformation feature is aligned with the second scanned feature, enabling accurate detection of the topography measurement error based on the features at the corresponding positions subsequently, and further determining the accurate error position.

[0188] Among them, the operation of determining the error position according to the multiple feature pairs can be: for any one of the multiple feature pairs, determine the difference between the two features in this feature pair to obtain the error corresponding to this feature pair; determine the positions corresponding to the feature pairs with errors exceeding the preset error threshold among the multiple feature pairs as the error position.

[0189] The preset error threshold can be set in advance, and the preset error threshold can be set to be relatively large.

[0190] In the case where the error corresponding to a feature pair exceeds the preset error threshold, it indicates that the error corresponding to this feature pair is relatively large, that is, the difference between the two features in this feature pair is relatively large. Then it means that the difference between the feature at this position in the reference transformation feature and the feature at this position in the second scanned feature is relatively large, that is, there is a problem with the topography at this position in the second scanned image, that is, there is a topography measurement error at this position, so this position can be determined as an error position.

[0191] In the case where the error corresponding to a feature pair does not exceed the preset error threshold, it indicates that the error corresponding to this feature pair is relatively small, that is, the difference between the two features in this feature pair is relatively small. Then it means that the difference between the feature at this position in the reference transformation feature and the feature at this position in the second scanned feature is relatively small, that is, the topography at this position in the second scanned image is basically the same as the topography at the corresponding position in the first scanned image, and it can be determined that there is no problem with the topography at this position in the second scanned image.

[0192] Among them, the operation of determining the difference between the two features in this feature pair to obtain the error corresponding to this feature pair can be: determine the distance between the two features in this feature pair to obtain the error corresponding to this feature pair.

[0193] Among them, the distance between the two features can be obtained by calculating the Euclidean distance, Manhattan distance, etc. between the two features, so as to obtain the error corresponding to this feature pair.

[0194] It should be noted that through the above step 202, the second scanned image can be guided by the first scanned image to accurately detect the topography measurement error in the second scanned image. After detecting the topography measurement error in the second scanned image, the detected topography measurement error can be corrected to improve the accuracy of the sample topography in the second scanned image, that is, continue to execute the following step 203.

[0195] Step 203: According to the first scanned image, correct the topography measurement error at the error position to obtain a target topography image.

[0196] In this case, by correcting the topography measurement error at the error position according to the first scanned image, it is equivalent to correcting the detected topography measurement error according to the SIM optical image, so as to realize guiding and correcting the topography measurement error caused by the force between the probe and the sample through optical data, and then an accurate sample topography can be obtained, improving the accuracy and reliability of the sample topography measurement.

[0197] A possible way is that the operation of step 203 can be: replace the feature at the error position in the second scanned feature with the feature at the same error position in the reference transformation feature to obtain a target topography feature; restore the target topography feature to obtain a target topography image.

[0198] Since the reference transformation feature is the feature obtained after transforming the first scanned feature, and the first scanned image is obtained by structured light scanning, that is to say, the first scanned image can represent the accurate sample surface topography. Therefore, the first scanned feature can accurately characterize the features of the sample surface topography, and thus the reference transformation feature can also accurately characterize the features of the sample surface topography.

[0199] In this case, by replacing the feature at the error position in the second scanned feature with the feature at the same error position in the reference transformation feature, the second scanned feature can also accurately characterize the features of the sample surface topography, realizing the accurate correction of the topography measurement error, and thus an accurate sample surface topography can be obtained.

[0200] Since the positions with topography measurement errors in the second scanned feature are not necessarily one, there may be multiple error positions. Then the above operation is specifically for each error position, replacing the feature at each error position in the second scanned feature with the feature at the corresponding position in the reference transformation feature, so as to obtain a target topography feature. In this way, each topography measurement error in the second scanned feature can be corrected, thus improving the measurement accuracy of the sample surface topography.

[0201] Among them, the operation of restoring the target morphological features to obtain the target morphological image can be: restoring the target morphological features through a decoder to obtain the target morphological image.

[0202] The decoder can be composed of multiple deconvolution layers. By performing upsampling on the target morphological features through multiple deconvolution layers, the target morphological features can be restored to a morphological image with the same size as the second scanned image, that is, the target morphological image is obtained.

[0203] It should be noted that after obtaining the above-mentioned target morphological features, virtual mechanical features can also be generated according to the motion law of the sample; according to the virtual mechanical features, interpolation is performed on the target morphological features to obtain high-resolution features.

[0204] The virtual mechanical features refer to the features corresponding to the mechanical data generated based on the motion of the sample when the AFM probe contacts the sample.

[0205] For some biological tissue samples (such as cells), the inside of such samples may be in a moving state, and for different samples, there may be different motion laws inside the samples. Then, when scanning the morphology of such samples by AFM, it may not be possible to accurately measure the morphology that conforms to the motion law of the sample. Therefore, interpolation based on the virtual mechanical features can enrich the detailed features of the target morphological features, so that the target morphological features can be more in line with the motion law of the sample, and the resolution of the image can be improved by the interpolation method.

[0206] In this case, the operation of restoring the target morphology to obtain the target morphological image can be: restoring the high-resolution morphological features to obtain the target morphological image.

[0207] In this way, by restoring the high-resolution morphological features to obtain the target morphological image, the details of the sample morphology can be restored, so that a detailed morphology that conforms to the motion law of the sample can be generated, thereby improving the time resolution of mechanical imaging when measuring the morphology by AFM, and while ensuring the nanoscale spatial resolution of AFM, achieving high-precision mechanical characterization of fast dynamic processes, and improving the ability of AFM to measure fast dynamic morphological changes.

[0208] Among them, the operation of generating virtual mechanical features according to the motion law of the sample can be: inputting the motion law of the sample into a mechanical feature generation model, and generating virtual mechanical features corresponding to the motion law of the sample through the mechanical feature generation model.

[0209] The mechanical feature generation model can be a neural network model based on machine learning. Before inputting the motion law of the sample into the mechanical feature generation model, the mechanical feature generation model is trained with the mechanical feature data corresponding to various motion laws, so that the mechanical feature generation model can learn the corresponding relationship between various motion laws and mechanical features. Thus, after inputting a motion law into the mechanical feature generation model subsequently, the mechanical feature generation model can output the mechanical features corresponding to this motion law.

[0210] In this case, the corresponding virtual mechanical features are output through the mechanical feature generation model, so that the virtual mechanical features can be quickly generated, thereby improving the correction efficiency of the sample morphology.

[0211] A possible way to determine the motion law of the sample may include: scanning the sample multiple times through SIM to obtain multiple first scan images; determining the motion law of the sample according to the multiple first scan images.

[0212] Among them, the operation of determining the motion law of the sample according to the multiple first scan images can be: determining the motion trajectory of the sample according to the multiple first scan images, inputting the motion trajectory of the sample into the motion law recognition model, and outputting the motion law of the sample through the motion law recognition model.

[0213] The motion law recognition model is a statistical model, which can be pre-trained according to the motion parameters corresponding to various motion laws, so that the motion law recognition model learns the motion parameters corresponding to various motion laws, summarizes the corresponding relationship between various motion parameters and various motion laws, so that the motion law recognition model has the ability to recognize motion laws, and then the motion law of the sample can be output through the motion law recognition model.

[0214] In the above method, the translation distance of the feature points can be determined first according to the translation amount of the feature points in different images among the multiple first scan images, and the geometric relationship between the matching feature points in different first scan images can also be used to calculate the rotation angle, etc. Then, the motion trajectory of the sample, such as linear motion, curved motion, etc., is drawn according to the calculated motion parameters such as translation distance and rotation angle.

[0215] Then, the motion trajectory of the sample is output to the sample law recognition model. In the sample law recognition model, first, the motion trajectory of the sample is fitted through the corresponding target model to fit the corresponding trajectory equation. Then, the motion parameters such as the speed, acceleration, period, and frequency of the sample motion are analyzed according to the parameters in the fitted trajectory equation. Finally, the motion laws are summarized for the analyzed motion parameters, so as to determine the corresponding motion laws.

[0216] The target model is a mathematical model, such as a linear model, a non - linear model, a polynomial model, etc. The target model can be adaptively updated according to the motion trajectory of the sample. For example, if the motion trajectory is a straight line, a linear model can be selected as the target model.

[0217] In this way, through the above - mentioned modeling method, the more accurate motion law of the sample can be determined.

[0218] Through the above steps 201 - 203, the correction of the topography measurement error in the sample topography can be realized, so as to obtain a more accurate sample topography. After correcting the topography measurement error, the target topography image can also be displayed so that the user can see the accurate sample topography.

[0219] Since the mechanical data obtained by AFM scanning may include height, the target topography image can be a three - dimensional image. Therefore, by displaying the target topography image, the user can see the detailed structure of the sample, thereby enhancing the user experience.

[0220] It should be noted that in the embodiments of the present application, the sample is quickly scanned by SIM, and the target area of the sample is identified. Then, AFM is guided to scan only the topography of the target area. Compared with the traditional AFM scanning technology, the physical contact of the probe with the sample is reduced, thereby reducing the risk of damage caused by the probe.

[0221] In addition, using the high - resolution SIM optical image as a guide to capture the dynamic deformation trajectory of the sample, virtual mechanical data that conforms to the real motion law of the sample can be generated between the original sparse sampling points of the AFM mechanical image. Thus, while improving the resolution of the AFM mechanical image, the high - precision mechanical characterization of the fast - dynamic process is realized, and further, the measurement accuracy of the sample topography can be improved.

[0222] In addition, the method for correcting the topography measurement error of an atomic force microscope provided in the embodiments of the present application mainly improves the topography measurement accuracy by detecting and correcting the topography measurement error in the AFM mechanical image. This is not affected by factors such as the sample material and the complexity of mechanical analysis. Therefore, the stability and reliability of the sample topography measurement can be improved.

[0223] For the sake of easy understanding, the following combines Figure 3 to give an exemplary illustration of the method for correcting the topography measurement error of an atomic force microscope provided in the embodiments of the present application. For example, Figure 3 is the flowchart of another method for correcting the topography measurement error of an atomic force microscope provided in the embodiments of the present application. Refer to Figure 3 , Figure 3 which includes steps 301 - 311.

[0224] Step 301: Scan the surface of the sample through SIM to obtain a first scanned image.

[0225] Step 302: Obtain a target area from the first scanned image, and scan the target area of the sample through AFM to obtain a second scanned image.

[0226] Step 303: Preprocess the first scanned image and the second scanned image.

[0227] Step 304: Extract the features of the first scanned image and the second scanned image to obtain a first scanned feature and a second scanned image.

[0228] Step 305: Convert the first scanned feature into a reference transformed feature through a target transformation matrix.

[0229] Step 306: Align the reference transformed feature with the second scanned feature to obtain multiple feature pairs.

[0230] Step 307: Calculate the difference between the two features in each feature pair to obtain the error corresponding to each feature pair.

[0231] Step 308: Determine whether the error corresponding to each feature pair is greater than a preset error threshold. If the error corresponding to a feature pair is greater than the preset error threshold, execute the following Step 309.

[0232] Step 309: Replace the feature at this position in the second scanned feature with the feature at the corresponding position in the reference transformed feature to obtain a target morphology feature.

[0233] Step 310: Restore the target morphology feature to obtain a target morphology image.

[0234] Step 311: Display the target morphology image.

[0235] In an embodiment of the present application, a computer device acquires a first scanned image and a second scanned image, that is, an optical image obtained by scanning a sample through SIM and a mechanical image obtained by scanning the sample through AFM. Then, based on the first scanned image, the second scanned image is subjected to topography error detection to obtain an error position, that is, using the SIM optical image as a guide to detect the position where there is a topography measurement error in the second scanned image. Then, using the SIM optical image as a guide, the topography measurement error at the error position is corrected to obtain a target topography image. By performing topography measurement error detection on the AFM mechanical image (the second scanned image) according to the SIM optical image (the first scanned image) and correcting the detected topography measurement error according to the SIM optical image in the present application, it is possible to achieve guiding and correcting the topography measurement error caused by the force between the probe and the sample through the SIM optical data, so as to obtain an accurate sample topography and improve the accuracy and reliability of sample topography measurement.

[0236] Figure 4 FIG. 4 is a schematic structural diagram of an apparatus for correcting topography measurement error of an atomic force microscope provided in an embodiment of the present application. The apparatus for correcting topography measurement error of the atomic force microscope can be implemented as part or all of a computer device by software, hardware, or a combination of both. The computer device can be the computer device shown below Figure 5 as shown. Referring to Figure 4 , the apparatus includes: a first acquisition module 401, an error detection module 402, and an error correction module 403.

[0237] The first acquisition module 401 is configured to acquire a first scanned image and a second scanned image. The first scanned image is obtained by scanning a sample through structured illumination microscopy (SIM), and the second scanned image is obtained by scanning the sample through an atomic force microscope (AFM);

[0238] The error detection module 402 is configured to perform topography measurement error detection on the second scanned image based on the first scanned image to obtain an error position, where the error position is the position where there is a topography measurement error in the second scanned image;

[0239] The error correction module 403 is configured to correct the topography measurement error at the error position based on the first scanned image to obtain a target topography image.

[0240] Optionally, the first acquisition module 401 is configured to:

[0241] Determine a target area from the first scanned image;

[0242] Scan the target area of the sample through AFM to obtain a second scanned image.

[0243] Optionally, the error detection module 402 is configured to:

[0244] Feature extraction is performed on the first scanned image and the second scanned image to obtain a first scanned feature and a second scanned feature;

[0245] Based on the first scanned feature and the second scanned feature, a third scanned feature and a fourth scanned feature are determined, and the third scanned feature and the fourth scanned feature are respectively the features that match each other in the first scanned feature and the second scanned feature;

[0246] Based on the third scanned feature and the fourth scanned feature, a target transformation matrix is determined;

[0247] Based on the target transformation matrix, the first scanned feature, and the second scanned feature, topographic measurement error detection is performed on the second scanned image to obtain an error position.

[0248] Optionally, the error detection module 402 is configured to:

[0249] Based on the target transformation matrix and the first scanned feature, a reference transformation feature is determined;

[0250] The reference transformation feature is aligned with the second scanned feature to obtain a plurality of feature pairs, and any one of the plurality of feature pairs includes any one feature in the reference transformation feature and the feature at the corresponding position in the second scanned feature;

[0251] Based on the plurality of feature pairs, the error position is determined.

[0252] Optionally, the error detection module 402 is configured to:

[0253] For any one of the plurality of feature pairs, the difference between the two features in the feature pair is determined to obtain the error corresponding to the feature pair;

[0254] The position corresponding to the feature pair with an error exceeding a preset error threshold among the plurality of feature pairs is determined as the error position.

[0255] Optionally, the error correction module 403 is configured to:

[0256] Replace the feature at the error position in the second scanned feature with the feature at the error position in the reference transformation feature to obtain a target topographic feature;

[0257] Restore the target topographic feature to obtain a target topographic image.

[0258] Optionally, the apparatus further includes:

[0259] A generation module, configured to generate virtual mechanical features according to the motion law of the sample;

[0260] An interpolation module for interpolating the target morphology features according to the virtual mechanical features to obtain high-resolution morphology features;

[0261] And the error correction module 403 is used for:

[0262] Restoring the high-resolution morphology features to obtain the target morphology image.

[0263] Optionally, the device further includes:

[0264] A second acquisition module for scanning the sample multiple times through SIM to obtain multiple first scanned images;

[0265] A determination module for determining the motion law of the sample according to the multiple first scanned images.

[0266] Optionally, the device further includes:

[0267] A first preprocessing module for denoising and normalizing the first scanned image to obtain the preprocessed first scanned image;

[0268] A second preprocessing module for denoising and flattening the second scanned image to obtain the preprocessed second scanned image;

[0269] And, the error detection module 402 is used for:

[0270] Performing morphology measurement error detection on the preprocessed second scanned image according to the preprocessed first scanned image to obtain the error position.

[0271] In the embodiments of the present application, obtaining the first scanned image and the second scanned image is also obtaining the optical image obtained by scanning the sample through SIM and the mechanical image obtained by scanning the sample through AFM. Then, according to the first scanned image, morphology error detection is performed on the second scanned image to obtain the error position, that is, using the SIM optical image as a guide to detect the position where there is a morphology measurement error in the second scanned image. Then, using the SIM optical image as a guide, the morphology measurement error at the error position is corrected to obtain the target morphology image. In the present application, by performing morphology measurement error detection on the AFM mechanical image (the second scanned image) according to the SIM optical image (the first scanned image) and correcting the detected morphology measurement error according to the SIM optical image, it is possible to realize guiding and correcting the morphology measurement error caused by the force between the probe and the sample through the SIM optical data, so as to obtain an accurate sample morphology and improve the accuracy and reliability of sample morphology measurement.

[0272] It should be noted that: when the atomic force microscope topography measurement error correction device provided in the above embodiment corrects the topography measurement error of the AFM mechanical image, only the division of the above functional modules is used for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0273] Each functional unit and module in the above embodiment can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of the functional units and modules are only for the convenience of mutual distinction and do not limit the protection scope of the embodiments of the present application.

[0274] The atomic force microscope topography measurement error correction device provided in the above embodiment and the embodiment of the atomic force microscope topography measurement error correction method belong to the same concept. For the specific working process and the technical effects brought by the units and modules in the above embodiment, reference can be made to the method embodiment part, which will not be elaborated here.

[0275] Figure 5 This is a schematic structural diagram of a computer device provided by an embodiment of the present application. As Figure 5 shown, the computer device 500 includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, the steps in an atomic force microscope topography measurement error correction method in the above embodiment are implemented.

[0276] The computer device 500 can be a general computer device or a special computer device. In specific implementations, the computer device 500 can be a desktop computer, a portable computer, a handheld computer, a tablet computer and other terminal devices. The embodiments of the present application do not limit the type of the computer device 500. Those skilled in the art can understand that Figure 5 this is only an example of the computer device 500 and does not constitute a limitation on the computer device 500. It may include more or fewer components than shown, or combine some components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0277] The processor 50 may be a Central Processing Unit (CPU), or the processor 50 may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0278] In some embodiments, the memory 51 may be an internal storage unit of the computer device 500, such as the hard disk or memory of the computer device 500. In other embodiments, the memory 51 may also be an external storage device of the computer device 500, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. equipped on the computer device 500. Further, the memory 51 may also include both the internal storage unit and the external storage device of the computer device 500. The memory 51 is used to store the operating system, application programs, Boot Loader, data, and other programs, etc. The memory 51 may also be used to temporarily store data that has been output or is to be output.

[0279] The embodiments of the present application also provide a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments can be implemented.

[0280] The embodiments of the present application provide a computer program product, which when running on a computer, causes the computer to execute the steps in the above method embodiments.

[0281] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such understanding, to implement all or part of the processes in the above method embodiments of the present application, a computer program can be used to instruct relevant hardware to complete. This computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can at least include: any entity or device that can carry the computer program code to the photographing device / terminal device, recording medium, computer memory, ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk and optical data storage device, etc. The computer-readable storage medium mentioned in the present application can be a non-volatile storage medium, in other words, it can be a non-transitory storage medium.

[0282] It should be understood that all or part of the steps to implement the above embodiments can be achieved by software, hardware, firmware or any combination thereof. When implemented using software, it can be achieved in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. These computer instructions can be stored in the above computer-readable storage medium.

[0283] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0284] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0285] In the embodiments provided in the present application, it should be understood that the disclosed device / computer equipment and method can be implemented in other ways. For example, the device / computer equipment embodiments described above are only illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0286] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0287] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features. And these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for correcting the topography measurement error of an atomic force microscope, characterized in that, The method includes: Obtaining a first scanned image and a second scanned image, where the first scanned image is obtained by scanning a sample through structured illumination microscopy (SIM), and the second scanned image is obtained by scanning the sample through an atomic force microscope (AFM); Performing a topographic measurement error detection on the second scanned image according to the first scanned image to obtain an error position, where the error position is the position in the second scanned image where there is a topographic measurement error; Correcting the topographic measurement error at the error position according to the first scanned image to obtain a target topographic image.

2. The method according to claim 1, characterized in that, The step of obtaining the second scanned image includes: Determining a target region from the first scanned image; Scanning the target region of the sample through AFM to obtain the second scanned image.

3. The method according to claim 1, wherein The performing a topographic measurement error detection on the second scanned image according to the first scanned image to obtain an error position includes: Performing feature extraction on the first scanned image and the second scanned image to obtain a first scanned feature and a second scanned feature; Determining a third scanned feature and a fourth scanned feature according to the first scanned feature and the second scanned feature, where the third scanned feature and the fourth scanned feature are respectively the features that match each other in the first scanned feature and the second scanned feature; Determining a target transformation matrix according to the third scanned feature and the fourth scanned feature; Performing a topographic measurement error detection on the second scanned image according to the target transformation matrix, the first scanned feature, and the second scanned feature to obtain the error position.

4. The method according to claim 3, wherein The performing a topographic measurement error detection on the second scanned image according to the target transformation matrix, the first scanned feature, and the second scanned feature to obtain the error position includes: Determining a reference transformation feature according to the target transformation matrix and the first scanned feature; Aligning the reference transformation feature with the second scanned feature to obtain a plurality of feature pairs, where any one of the plurality of feature pairs includes any one feature in the reference transformation feature and the feature at the corresponding position in the second scanned feature; Determining the error position according to the plurality of feature pairs.

5. The method according to claim 4, wherein The determining the error position according to the plurality of feature pairs includes: For any one of the plurality of feature pairs, determining the difference between the two features in the feature pair to obtain the error corresponding to the feature pair; Determining the position corresponding to the feature pair with an error exceeding a preset error threshold among the plurality of feature pairs as the error position.

6. The method according to claim 5, wherein The correcting the topographic measurement error at the error position according to the first scanned image to obtain a target topographic image includes: Replacing the feature at the error position in the second scanned feature with the feature at the error position in the reference transformation feature to obtain the target topographic feature; Restoring the target topographic feature to obtain the target topographic image.

7. The method according to claim 6, wherein The method further includes: Generating a virtual mechanical feature according to the motion law of the sample. Interpolate the target morphology features according to the virtual mechanical features to obtain high-resolution morphology features; And restoring the target morphology features to obtain the target morphology image, including: Restoring the high-resolution morphology features to obtain the target morphology image.

8. The method according to claim 7, wherein The method further includes: Performing multiple scans on the sample by SIM to obtain multiple first scan images; Determine the motion law of the sample according to the multiple first scan images.

9. The method according to claim 1, wherein The method further includes: Denoise and normalize the first scan image to obtain the first scan image after preprocessing; Denoise and planarize the second scan image to obtain the second scan image after preprocessing; And, detecting the morphology measurement error of the second scan image according to the first scan image to obtain the error position, including: Detect the morphology measurement error of the second scan image after preprocessing according to the first scan image after preprocessing to obtain the error position.

10. A computer device, characterized in that, The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the method described in any one of claims 1 to 9 is implemented.

Images