Feature region-based cross-scale data fusion method, apparatus, device and storage medium

Abstract

The present invention discloses a feature-region-based cross-scale data fusion method, which includes step S1: preprocessing AFM and WLI datasets, identifying a reference surface in the measurement data, and calibrating for errors between different data sources; step S2: extracting feature region descriptors using an iterative similar region algorithm; step S3: registering the two datasets based on the extracted feature region descriptors using an attention-based registration mechanism; step S4: fusing the WLI and AFM data based on the registration results, registering the height references, and then replacing the overlapping AFM measurement data with the WLI measurement data. S4: smoothing the edges using linear interpolation to complete the data fusion. This method solves the problem of cross-scale data registration accuracy and improves the accuracy and efficiency of micro- and nanostructure measurement.

Description

[Technical Field]

[0001] The present application relates to the technical field of three-dimensional measurement of microscopic surfaces, and more particularly to a method, apparatus, device and storage medium for feature-region-based cross-scale data fusion. [Background technology]

[0002] Development of measurement techniques for nano- and micro-scale structures is crucial because they are widely used in engineering applications. Multi-sensor data fusion is a promising approach to expand the range of measurement techniques and improve measurement resolution and efficiency by combining the advantages of various technologies.

[0003] Although much progress has been made in the research of multi-sensor data fusion methods, existing data fusion methods face challenges in effectively extracting matching criteria for cross-scale structures between datasets with excessively large resolution differences, such as a factor of 10 or more. For example, the lateral resolution difference between tip scanning measurement data from an atomic force microscope (AFM) and vertical scanning measurement data from a white light interferometer (WLI) ranges from 1:10 to 1:600. Effective fusion is not possible using existing cross-scale methods, and registration accuracy is also limited by the resolution difference. Therefore, there is a pressing need for a method that can effectively fuse cross-scale data with large lateral resolution differences (e.g., AFM and WLI).

[0004] Furthermore, there are prior art technologies relating to general point cloud registration (see Patent Document 1) and robustness in complex scenes (see Patent Document 2), but these technologies have significantly different application scenarios and technical fields from the present invention, which is specialized in the high-precision fusion of specific sensors in microscopic measurement technology. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Chinese Patent Application Publication No. 112950682 [Patent Document 2] Chinese Patent Application Publication No. 116912296 Summary of the Invention [Problem to be solved by the invention]

[0006] Furthermore, the prior art techniques described in Patent Documents 1 and 2 rely on scale adjustment using wavelet decomposition (Patent Document 1) and on enhancing location recognition using deep learning models (Patent Document 2), respectively, and do not mention the description of the hierarchical structure of feature regions or alternative fusion techniques for specific sensor data.

[0007] Furthermore, the prior art described in Patent Documents 1 and 2 addresses general-purpose scale differences (Patent Document 1) and robustness in complex situations (Patent Document 2), and the technical problems that the present invention aims to solve are not overlapping.

[0008] Furthermore, the innovative point of the present invention is focused on the registration mechanism based on the description of feature regions and attention-driven, while the prior art does not cover this type of method, and in particular, does not mention specific processing techniques for AFM / WLI data.

[0009] In response to at least one deficiency or need for improvement in the existing techniques, the present invention provides a feature region-based cross-scale data fusion method, apparatus, device and storage medium, which can solve at least one of the problems in the background art described above. [Means for solving the problem]

[0010] To achieve the above object, according to a first aspect of the present invention, there is provided a feature region-based cross-scale data fusion method, the method comprising: Step S1: preprocessing an AFM dataset, which is probe scanning measurement data by an atomic force microscope, and a WLI dataset, which is vertical scanning measurement data by a white light interferometer, to identify a reference surface in the measurement data and calibrate errors of different data sources; Step S2: extracting feature region descriptors based on the preprocessed data using an iterative similar region algorithm, the feature region descriptors including location, region center point, descriptor, region size, boundary and feature parameters; Step S3: realizing registration between the AFM dataset and the WLI dataset based on the extracted feature region descriptors using an attention-based registration mechanism; The method includes step S4 of fusing the WLI data set and the AFM data set based on the registration result, registering the height reference, replacing the AFM measurement data with the WLI measurement data of the overlapping portion, and performing smoothing processing by linear interpolation at the edge portion to complete the fusing of the AFM data set and the WLI data set.

[0011] Furthermore, in the above-mentioned feature region-based cross-scale data fusion method, the error correction of the different data sources can be specifically performed by: Bat-wing effect suppression was performed on the WLI dataset to reduce noise at step edges due to diffraction effects in white light interferometry; and This involves reconstructing a tip model using a scanning electron microscope and deconvolving the tip against the AFM data set to reduce measurement errors due to tip shape and tilt rate.

[0012] Furthermore, in the above-mentioned feature region-based cross-scale data fusion method, extracting the feature region descriptor specifically includes: Step S2.1 of generating a parameter matrix of the segmentation target and initializing the clustering target as a height matrix Z and its maximum number of clusters Nmax; Step S2.2, where n cluster centers are selected, n∈[1, Nmax]; Assign each data point to the nearest cluster center,

[0013] (Number 1) JPEG0007730230000002.jpg1052

[0014] where x is a data point and z i is the i-th cluster center when performing height parameter clustering, S2.3, and Recalculate the centers of each cluster,

[0015] (Number 2) JPEG0007730230000003.jpg1433

[0016] where M i is the data point set of the i-th cluster, and |M i step S2.4, where | is the number of data points in the set; Step S2.5: Repeating steps S2.3 to S2.4 until a preset termination condition is met to obtain the sum of squared error losses corresponding to the current number n of cluster centers;

[0017] (Number 3) JPEG0007730230000004.jpg1653

[0018] Repeat steps S2.2–S2.5 for SSE n Step S2.6: obtain the curve, calculate the SSE decay rate to obtain the optimal number of clusters N, and perform connected region analysis on the cluster results to achieve data set division; and step S2.7 of extracting feature regions using RANSAC fitting on the divided data set, extracting a set of points in the feature regions corresponding to the current parameters to generate a feature region descriptor, and repeating steps S2.2 to S2.5 for the remaining set of points using a new parameter matrix.

[0019] Furthermore, in the above-mentioned cross-scale data fusion method based on feature regions, generating the parameter matrix of the segmentation target specifically includes: Sequentially analyzing the height parameter matrix, the gradient parameter matrix, and the curvature parameter matrix, and outputting the measurement results in the form of a height matrix z; If the region is non-planar, the method includes dividing the region by a gradient matrix and calculating the gradient matrix and the curvature using a floating interval.

[0020] Furthermore, in the above-mentioned cross-scale data fusion method based on feature regions, initial feature extraction is performed on identifiable fine structure regions to obtain primary feature region extraction, and secondary feature region extraction is performed on regions that meet predetermined conditions in the results of the primary feature region extraction to obtain detailed regions.

[0021] Furthermore, in the above feature region-based cross-scale data fusion method, the attention-based registration mechanism is utilized to realize the registration between the AFM dataset and the WLI dataset based on the extracted feature region descriptors, specifically: Descriptor classification is performed on the feature extraction results of WLI and AFM, respectively, and the self-attention matrix C is calculated as follows: s , A s Get

[0022] (Number 4) JPEG0007730230000005.jpg755

[0023] where S i is the feature point set, y is a feature point, and f y are the feature parameters, and F i is the i-th element of the feature parameter set, and g y is the size of the feature region, and G i , G i+1 are the minimum and maximum values ​​of the size of the feature region, respectively, and the value of the mth row and nth column of the self-attention matrix represents the probability that the set containing feature point m also contains feature point n. Step S3.1; Cross-matching between point sets to obtain a transformation matrix. Different matching methods calculate confidence factors based on the number of matching point pairs and position errors to obtain a cross-attention matrix with matching probability information.

[0024] (Number 5) JPEG0007730230000006.jpg4383

[0025] Here, C si JPEG0007730230000007.jpg45A sj Set C si and Set A sj indicates that O matches Cm and O An are set C si The position coordinates of the mth feature region in set A sj The position coordinates of the nth feature region in R ij and T ij are the rotation and translation transformation matrices, respectively, and ξ ij Set C si and Set A sj is the matching trust factor, and d th is the threshold value of the position error to determine whether two points are a matching point pair, JPEG0007730230000008.jpg1013 is the sum of the position errors of the matching point pairs, and the position error d <d th where N is the number of matching point pairs, JPEG0007730230000009.jpg716, P ij Set C si and Set A sj is the confidence probability of matching, Step S3.2, which is JPEG0007730230000010.jpg718; The rotation transformation matrix R ij and the translation transformation matrix T ijCalculate the matching error d of the feature points in the other point set under the rotation transformation using the formula, adjust the region size threshold of the point set classification process, and update the self-attention matrix;

[0026] (Number 6) JPEG0007730230000011.jpg3381

[0027] Here, [G i , G i+1 ) is the initial threshold range, and [G i ', G i+1 ') is the updated threshold range in step S3.3; The method includes step S3.4 of repeating steps S3.2 to S3.3 until the probability of the cross-attention matrix no longer changes due to the iterative processing, and acquiring the matching result with the highest probability as the optimal matching method.

[0028] Furthermore, the above-mentioned cross-scale data fusion method based on feature regions further includes a step of verifying the registration accuracy by comparing the registration results of the ICP algorithm and the SIFT algorithm with the registration results of the iterative similar region algorithm (ISR).

[0029] According to a second aspect of the present invention, there is further provided a feature region based cross-scale data fusion apparatus, comprising: a pre-processing module configured to pre-process an AFM dataset, which is probe scanning measurement data by an atomic force microscope, and a WLI dataset, which is vertical scanning measurement data by a white light interferometer, to identify a reference surface in the measurement data, and to calibrate errors of different data sources; a feature extraction module that uses an iterative similar region algorithm to extract feature region descriptors based on the preprocessed data, the feature region descriptors being configured to include location, region center points, descriptors, region size, boundaries, and feature parameters; a data registration module arranged to achieve registration of the AFM dataset and the WLI dataset based on the extracted feature region descriptors using an attention-based registration mechanism; The data fusion module is configured to fuse the WLI data set and the AFM data set based on the registration result, register the height reference, and then replace the AFM measurement data with the WLI measurement data of the overlapping portion, and perform linear interpolation smoothing processing at the edges to complete the fusion of the AFM data set and the WLI data set.

[0030] According to a third aspect of the present invention, there is further provided a feature region-based cross-scale data fusion device, the device including at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program that, when executed by the processing unit, causes the processing unit to perform the steps of any of the methods described above.

[0031] According to a fourth aspect of the present invention, there is further provided a storage medium storing a computer program executable by a feature region-based cross-scale data fusion device, the computer program causing the feature region-based cross-scale data fusion device to perform the steps of any of the above methods when executed by the feature region-based cross-scale data fusion device. [Effects of the Invention]

[0032] In general, the above technical solution conceived by the present invention can achieve the following beneficial effects compared with the existing technology. The feature region-based cross-scale data fusion method provided by this invention precisely fuses high-resolution AFM data and large-scale WLI data through an iterative similar region algorithm and an attention-based registration mechanism, solving the problem of cross-scale data registration accuracy and improving the accuracy and efficiency of micro- and nanostructure measurements.

[0033] In order to more clearly explain the technical solutions in the embodiments of the present application, the drawings necessary for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without paying any creative work. [Brief explanation of the drawings]

[0034] [Figure 1] FIG. 1 is a schematic diagram of a flow chart of a feature region-based cross-scale data fusion method provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0035] In order to clarify the objectives, technical solutions and advantages of the present invention, the present invention will be described in more detail below with reference to the drawings and examples. However, the specific examples described here are intended to aid in understanding the present invention and are not intended to limit the present invention. Furthermore, the technical features described in each of the following embodiments can be implemented in any combination as long as they are not mutually contradictory.

[0036] The terms "first," "second," "third," etc. in the specification, claims, and drawings of this application are used to distinguish between different objects and are not intended to describe a particular order. Furthermore, the terms "comprise" and "have" and variations thereof are intended to cover an exclusive inclusion. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the recited steps or units, and may also include steps or units not recited, or may include other steps or units inherent to such process, method, product, or apparatus.

[0037] 1 is a schematic diagram of a flow chart of the feature region-based cross-scale data fusion method provided by the embodiment of the present application. As shown in FIG. 1, the feature region-based cross-scale data fusion method provided by the embodiment of the present application includes: Step S1: preprocessing an AFM dataset, which is probe scanning measurement data by an atomic force microscope, and a WLI dataset, which is vertical scanning measurement data by a white light interferometer, to identify a reference surface in the measurement data and calibrate errors of different data sources; Step S2: extracting feature region descriptors based on the preprocessed data using an iterative similar region algorithm, the feature region descriptors including location, region center point, descriptor, region size, boundary and feature parameters; Step S3: realizing registration between the AFM dataset and the WLI dataset based on the extracted feature region descriptors using an attention-based registration mechanism; The method includes step S4 of fusing the WLI data set and the AFM data set based on the registration result, registering the height reference, replacing the AFM measurement data with the WLI measurement data of the overlapping parts, performing smoothing processing by linear interpolation at the edge parts, and completing the fusion of the AFM data and the WLI data.

[0038] Specifically, atomic force probe scanning (AFM) datasets and vertical scanning white light interferometry (WLI) datasets are both point cloud data. While the vertical resolutions of WLI and AFM are similar, the horizontal resolution spans a much larger range. The horizontal resolution of atomic force microscopy generally depends on the scanning step length and ranges from 1 nm to 50 nm. However, due to the limited size of the probe tip, the measurement results are affected by the tip convolution effect. Meanwhile, WLI is limited by the diffraction limit, and the horizontal resolution is typically less than 1 μm. Furthermore, in step-edge regions, WLI measurements are susceptible to the "bat-wing effect," which results in noisy data.

[0039] In the data preprocessing stage, we first identify a reference surface in the measurement data, flatten the surface using the least-squares plane method, and calibrate errors from different data sources. For WLI data, we perform bat-wing effect suppression processing to reduce noise at step edges caused by diffraction effects. For AFM data, we use a scanning electron microscope (SEM) to reconstruct a tip model and deconvolve the tip to reduce measurement errors caused by the tip shape and tilt rate.

[0040] The accuracy of feature region segmentation in WLI, i.e., low-resolution data, is affected by the low resolution of the data source, limiting the accuracy of cross-scale data matching. Therefore, instead of feature point extraction as in traditional methods, we extract feature regions and generate feature region descriptors to classify between feature regions, and combine them with position matching error to adjust the classification threshold to obtain the rotational and translational relationship between WLI and AFM data.

[0041] Based on the preprocessed data, an iterative similar region (ISR) algorithm is used to extract feature region descriptors from the data. The extraction of feature region descriptors involves the optimization of K-means clustering and the RANSAC algorithm to improve the accuracy of feature region extraction. The feature region descriptor includes the location, region center point, descriptor, range size, boundary, and feature parameters (e.g., height, gradient, curvature, etc.).

[0042] An attention-based registration mechanism is used to precisely position the AFM region within the WLI coordinate system. This mechanism incorporates the concepts of cross-attention and self-attention, and uses an alternating iterative process to eliminate the influence of noise regions and optimize the registration accuracy between cross-scale data.

[0043] By combining feature region descriptors and attention mechanism matching, registration errors caused by insufficient segmentation accuracy of feature regions in low-resolution data can be effectively reduced.

[0044] Based on the registration results, the WLI and AFM data are fused. Because the accuracy of AFM measurement data is much higher than that of WLI measurement data, the overlapping area fusion process first registers the height reference, and then replaces the AFM measurement data with the WLI measurement data in the overlapping area. A uniform registration process is performed on the boundary between the AFM and WLI data to make the final fused data set smoother.

[0045] The feature region-based cross-scale data fusion method provided in the embodiments of this application precisely fuses high-resolution AFM data and large-scale WLI data through an iterative similar region algorithm and an attention-based registration mechanism, solving the problem of cross-scale data registration accuracy and improving the accuracy and efficiency of micro- and nanostructure measurements.

[0046] In addition, in the cross-scale data fusion method based on feature regions provided by the embodiments of the present application, the error correction of the different data sources can be specifically performed by: Bat-wing effect suppression was performed on the WLI dataset to reduce noise at step edges due to diffraction effects in white light interferometry; and This may include using a scanning electron microscope to reconstruct a tip model and deconvolve the tip against the AFM data to reduce measurement errors due to tip shape and tilt rate.

[0047] Specifically, the bat-wing effect is a phenomenon that occurs in white light scanning interferometry and is primarily caused by diffraction. When the height of a step in the measurement target is smaller than the coherence length of the light source, bat-wing-like artifacts appear at the step edges. This phenomenon is called the bat-wing effect. The bat-wing effect can be eliminated or compensated for by software or hardware improvements, such as adjusting the system's hardware parameters or applying filtering correction to the three-dimensional shape obtained by demodulating the coherent signal, thereby improving the accuracy of the measurement results.

[0048] A tip model is reconstructed using a scanning electron microscope, and the tip shape of the atomic force probe is measured. Using the tip model, deconvolution of the tip is performed on the measurement data of the atomic force microscope, thereby reducing measurement errors caused by the tip shape and tilt rate.

[0049] In addition, in the feature region-based cross-scale data fusion method provided by the embodiments of the present application, extracting the feature region descriptor specifically includes: Step S2.1 of generating a parameter matrix of the segmentation target and initializing the clustering target as a height matrix Z and its maximum number of clusters Nmax; Step S2.2, selecting n cluster centers, among which n∈[1, Nmax]; Assign each data point to the nearest cluster center,

[0050] (Number 7) JPEG0007730230000012.jpg1254

[0051] where x is a data point and z i is the i-th cluster center when performing height parameter clustering, S2.3, and Recalculate the centers of each cluster,

[0052] (Number 8) JPEG0007730230000013.jpg1633

[0053] where M i is the data point set of the i-th cluster, and |M i step S2.4, where | is the number of data points in the set; Step S2.5: Repeating steps S2.3 to S2.4 until a preset termination condition is met to obtain the sum of squared error losses corresponding to the current number n of cluster centers;

[0054] (Number 9) JPEG0007730230000014.jpg1961

[0055] Repeat steps S2.2–S2.5 for SSE n Step S2.6: obtain the curve, calculate the SSE decay rate to obtain the optimal number of clusters N, and perform connected region analysis on the cluster results to achieve data set division; The method may include S2.7, which extracts feature regions using RANSAC fitting on the divided data set, extracts a set of points in the feature regions corresponding to the current parameters, generates a feature region descriptor, and repeats steps S2.2 to S2.5 for the remaining set of points using a new parameter matrix.

[0056] Specifically, the multidimensional clustering method of K-means clustering is used to segment the feature regions of WLI and AFM measurement data, laying the foundation for data registration. K-means clustering is a classical unsupervised learning algorithm. In the examples of this application, an optimization method of K-means clustering specialized for surface measurement data is used to automatically determine the clustering dimension and number of clusters by analyzing the decay rate of surface fitting error. The RANSAC algorithm is also used to limit the clustering error within the feature regions, ensuring more accurate segmentation.

[0057] The parameter matrix of the segmentation object is generated, and three types of parameter matrices, namely, the height parameter matrix, the gradient parameter matrix, and the curvature parameter matrix, are analyzed sequentially, and the measurement results are output in the form of a height matrix Z. The clustering object is initialized as the height matrix Z and its maximum cluster number Nmax, and the maximum cluster number can be selected as a relatively large value, so that the optimal cluster number and the corresponding clustering result can be automatically obtained in the subsequent analysis process.

[0058] When a feature region point set corresponding to the current parameters is extracted and a feature region descriptor is generated, the result is as shown in Table 1 below.

[0059] (Table 1) Table 1: Elements included in the feature region descriptor JPEG0007730230000015.jpg2277

[0060] In addition, in the cross-scale data fusion method based on feature regions provided by the embodiments of the present application, generating the parameter matrix of the segmentation target specifically includes: Sequentially analyzing the height parameter matrix, the gradient parameter matrix, and the curvature parameter matrix, and outputting the measurement results in the form of a height matrix z; If the region is non-planar, the method may include dividing the region by a gradient matrix and calculating the gradient matrix and curvature using a floating interval.

[0061] Specifically, based on prior knowledge of surface metrology, feature types are classified into four types: flat, sloped, spherical, and irregular surfaces, and then sorted according to their occurrence probability. All unclassified regions are designated as noise points. In the feature extraction process, surface features are classified using a hierarchical structure of three matrices: height (Z), gradient (D), and curvature (Q). The surface type error is used to evaluate the classification accuracy, which helps to improve the segmentation of feature regions. Regions that cannot be classified as flat, sloped, or spherical are labeled as irregular surface regions or local noise points based on their connected areas.

[0062] If some regions cannot be classified as planar, we can use the gradient matrix D to divide these regions. The gradient matrix D and curvature matrix Q can be calculated through floating intervals. For example, the 2D gradient matrix D and 4-neighbor curvature matrix Q can be calculated using the following formulas:

[0063] (Number 10) JPEG0007730230000016.jpg2274

[0064] where r(i, j) is the radius of the circle that passes through the fitting point Z(i, j) and its four neighboring points.

[0065] In addition, in the cross-scale data fusion method based on feature regions provided by the embodiments of the present application, initial feature extraction may be performed on identifiable fine structure regions, which may be used as primary feature region extraction, and secondary feature region extraction may be performed on regions that satisfy predetermined conditions in the results of the primary feature region extraction to obtain detailed regions.

[0066] Specifically, the feature extraction process can accurately segment the fine structure regions in the WLI data. In cross-scale measurements, initial feature extraction identifies the fine structure regions, which is called primary feature region extraction. Then, the relatively large regions identified by the primary extraction are subjected to secondary extraction to extract detailed regions. The two-stage extraction process is similar, but the secondary extraction uses a stricter RANSAC fitting threshold to refine the detailed regions in the WLI data. This two-stage extraction process forms the basis for the registration of WLI and AFM data.

[0067] In addition, the feature region-based cross-scale data fusion method provided by the embodiments of the present application utilizes the attention-based registration mechanism to realize registration of two datasets based on the extracted feature region descriptors, specifically, Descriptor classification is performed on the feature extraction results of WLI and AFM, respectively, and the self-attention matrix C is calculated as follows: s , A s Get

[0068] (Number 11) JPEG0007730230000017.jpg755

[0069] where S i is the feature point set, y is a feature point, and f y are the feature parameters, and F i is the i-th element of the feature parameter set, and g y is the size of the feature region, and G i , G i+1 are the minimum and maximum values ​​of the size of the feature region, respectively, and the value of the mth row and nth column of the self-attention matrix represents the probability that the set containing feature point m also contains feature point n. Step S3.1; Cross-matching between point sets is performed to obtain a transformation matrix. Different matching methods calculate confidence factors based on the number of matching point pairs and position errors, and obtain a cross-attention matrix with matching probability information.

[0070] (Number 12) JPEG0007730230000018.jpg4383

[0071] Here, C si JPEG0007730230000019.jpg45A sj Set C si and Set A sj indicates that O matches Cm and O An are set C si The position coordinates of the mth feature region in set A sj The position coordinates of the nth feature region in R ij and T ij are the rotation and translation transformation matrices, respectively, and ξ ij Set C si and Set A sj is the matching trust factor, and d th is the threshold value of the position error to determine whether two points are a matching point pair, JPEG0007730230000020.jpg1013 is the sum of the position errors of the matching point pairs, and the position error d <d th where N is the number of matching point pairs, JPEG0007730230000021.jpg716, P ij Set C si and Set A sj is the confidence probability of matching, Step S3.2, which is JPEG0007730230000022.jpg718; The rotation transformation matrix R ij and the translation transformation matrix T ij Calculate the matching error d of the feature points in the other point set due to the rotation transformation using the formula, adjust the region size threshold of the point set classification process, and update the self-attention matrix;

[0072] (Number 13) JPEG0007730230000023.jpg3889

[0073] Here, [G i , G i+1 ) is the initial threshold range, and [G i ', G i+1 ') is the updated threshold range in step S3.3; The method may include step S3.4 of repeating steps S3.2 to S3.3 until the probability of the cross-attention matrix no longer changes due to the iterative processing, and acquiring the matching result with the highest probability as the optimal matching method.

[0074] Specifically, the superglue method based on graph convolutional neural networks (GCNs) proposes the concepts of cross-attention and self-attention, taking into account the structural characteristics of the feature region arrangement in surface measurement data. In the feature matching process, the cross-matching matrix and the self-matching matrix are alternately processed iteratively to eliminate the influence of noise regions on feature matching. The process of feature descriptor matching and position matching are combined and repeated to optimize the registration accuracy between cross-scale data.

[0075] In addition, the feature region-based cross-scale data fusion method provided by the embodiments of the present application may further include verifying the registration accuracy by comparing the registration results of the ICP algorithm and the SIFT algorithm with the results of the ISR algorithm.

[0076] In the examples of this application, a new cross-scale data fusion method based on feature region registration is proposed, which fuses large-scale white light interferometer (WLI) data with high-resolution atomic force microscope (AFM) data to measure microscopic nanocomposite structures at different scales with high accuracy. The core of this method is the proposed ISR algorithm, which uses feature region descriptors instead of feature point descriptors to optimize the traditional data fusion process and ensure the accuracy of cross-scale registration.

[0077] The ISR algorithm uses an optimized K-means clustering method to extract feature regions from the WLI and AFM data, generate region descriptors, and then utilizes an attention mechanism to perform matching. The process alternates between self-matching and cross-matching to improve registration and compensate for errors due to resolution differences. After registration, the WLI data is replaced with the higher-resolution AFM data for overlapping regions, and linear interpolation smoothing is applied to the edges to ensure seamless fusion.

[0078] The ISR method exhibits superior performance in processing cross-scale data with significant resolution differences, and can provide consistent feature region extraction on a unified physical scale. Furthermore, even when ICP is optimized using artificially selected coarse registration points, the ISR algorithm can demonstrate superior registration accuracy.

[0079] In an embodiment of the present application, a feature region-based cross-scale data fusion device is further provided, a pre-processing module configured to pre-process an AFM dataset, which is probe scanning measurement data by an atomic force microscope, and a WLI dataset, which is vertical scanning measurement data by a white light interferometer, to identify a reference surface in the measurement data, and to calibrate errors of different data sources; a feature extraction module that uses an iterative similar region algorithm to extract feature region descriptors based on the preprocessed data, the feature region descriptors being configured to include location, region center points, descriptors, region size, boundaries, and feature parameters; a data registration module arranged to achieve registration of the AFM dataset and the WLI dataset based on the extracted feature region descriptors using an attention-based registration mechanism; and a data fusion module configured to fuse the WLI data set and the AFM data set based on the registration result, register the height reference, then replace the AFM measurement data with the WLI measurement data of the overlapping part, perform linear interpolation smoothing processing at the edges, and complete the fusion of the AFM data set and the WLI data set.

[0080] The present invention also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the above method, wherein the computer readable storage medium includes, but is not limited to, a floppy disk, an optical disk, a DVD, a CD-ROM, a microdrive, a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic or optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0081] Although the above method embodiments are described as a combination of a series of operations for simplicity, those skilled in the art will understand that the present application is not limited to the order of operations described. In the present application, certain steps may be performed in other orders or simultaneously. Furthermore, those skilled in the art will understand that all embodiments described herein are merely preferred embodiments, and that the described operations or modules are not necessarily essential to the present application.

[0082] In the above embodiments, the description of each embodiment focuses on different aspects, and for configurations that are not described in detail in one embodiment, reference can be made to related descriptions in other embodiments.

[0083] It should be understood that some embodiments disclosed in this application may be realized by other means. For example, the above-described device embodiments are merely examples, and the division of the units is also an example shown as a logical functional division. In actual implementation, a different division scheme may be adopted. For example, multiple units or components may be integrated or incorporated into other systems, or some components may be omitted or not implemented. Another point is that the illustrated or described mutual couplings or direct couplings or communication connections may be indirect couplings or communication connections via some service interfaces, devices, or units, which may be electrical or in other forms.

[0084] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., the units may be located in the same place or may be distributed across multiple network units. Some or all of these units can be selected appropriately according to actual needs to achieve the technical purpose of the present embodiment.

[0085] Furthermore, the functional units in each embodiment described in this application may be integrated into a single processing unit, or each unit may be physically provided separately, or two or more units may be integrated into a single unit. These integrated units may be implemented as hardware or software functional units.

[0086] When the above-mentioned integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the essential part of the technical solution of the present application, or the part that contributes to the existing technology, or all or part of the technical solution, can be realized in the form of a software product, which is stored in a memory and includes a plurality of instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. Meanwhile, the memory includes any medium capable of storing program code, such as a USB memory, a read-only memory (ROM), a random access memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.

[0087] Those skilled in the art can understand that all or part of the steps of the various methods in the above embodiments can be completed by a program that instructs relevant hardware, and the program can be stored in a computer-readable memory, including a flash disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0088] The foregoing description is merely illustrative of the present disclosure and is not intended to limit the scope of the present disclosure. In other words, all equivalent changes and modifications made in accordance with the teachings of the present disclosure remain within the scope of the present disclosure. Other embodiments of the present disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure, including common knowledge or customary technical means known in the art but not described herein, in accordance with the general principles of the present disclosure. The specification and examples are considered to be exemplary only, with the scope and spirit of the present disclosure being defined by the following claims.

[0089] The technical features of the above embodiments can be combined in any way, and for the sake of brevity, not all possible combinations of the various technical features of the above embodiments are described, but as long as there is no contradiction in the combination of these technical features, all are intended to be included within the scope described in this specification.

[0090] Those skilled in the art can easily understand that the above content is only a preferred embodiment of the present invention, and does not limit the present invention, and all modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. Step S1: preprocessing an AFM dataset, which is probe scanning measurement data by an atomic force microscope, and a WLI dataset, which is vertical scanning measurement data by a white light interferometer, to identify a reference surface in the measurement data and calibrate errors of different data sources; Step S2: extracting feature region descriptors based on the preprocessed data using an iterative similar region algorithm, the feature region descriptors including location, region center point, descriptor, region size, boundary and feature parameters; Step S3: realizing registration between the AFM dataset and the WLI dataset based on the extracted feature region descriptors using an attention-based registration mechanism; Step S4: based on the registration result, combine the WLI data set and the AFM data set, register the height reference, replace the AFM measurement data with the WLI measurement data of the overlapping part, and perform smoothing processing by linear interpolation at the edge part to complete the combination of the AFM data set and the WLI data set; A feature region-based cross-scale data fusion method comprising:

2. Specifically, the error correction of the different data sources includes: Bat-wing effect suppression was performed on the WLI dataset to reduce step-edge noise due to diffraction effects in white light interferometry; and and reconstructing the tip model using a scanning electron microscope and deconvolving the tip against the AFM data set to reduce measurement errors due to tip shape and tilt rate. The feature region based cross-scale data fusion method of claim 1 .

3. Specifically, extracting the feature region descriptor includes: Step S2.1 of generating a parameter matrix of the segmentation target and initializing the clustering target as a height matrix Z and its maximum number of clusters Nmax; Step S2.2, selecting n cluster centers, n∈[1, Nmax]; Assign each data point to the nearest cluster center, (Equation 1) where x is a data point and z ｉ is the i-th cluster center when performing height parameter clustering, S2.3, Recalculate the centers of each cluster, (Equation 2) Here, M ｉ is the data point set of the i-th cluster, and |M ｉ step S2.4, where | is the number of data points in the set; Step S2.5: repeating steps S2.3 and S2.4 until a preset termination condition is met to obtain the sum of squared error losses corresponding to the current number n of cluster centers; (Equation 3) Repeat steps S2.2 to S2.5 to obtain the SSE. ｎ Step S2.6: obtaining the curve, calculating the SSE decay rate to obtain the optimal number of clusters N, and performing connected region analysis on the cluster results to realize the division of the data set; Step S2.7: extracting feature regions using a RANSAC fit on the divided data set, extracting a set of feature region points corresponding to the current parameters to generate feature region descriptors, and repeating steps S2.2 to S2.5 on the remaining set of points using the new parameter matrix; characterized in that it comprises The feature region based cross-scale data fusion method of claim 1 .

4. Specifically, generating the parameter matrix to be divided includes: Sequentially analyzing the height parameter matrix, the gradient parameter matrix, and the curvature parameter matrix, and outputting the measurement results in the form of a height matrix z; If the region is non-planar, the method further comprises dividing the region by a gradient matrix and calculating the gradient matrix and the curvature using a floating interval. The feature region based cross-scale data fusion method according to claim 3 .

5. The method is characterized in that initial feature extraction is performed on a distinguishable fine structure region to extract a primary feature region, and secondary feature region extraction is performed on a region that matches a predetermined condition in the result of the primary feature region extraction to obtain a detailed region. The feature region based cross-scale data fusion method according to claim 3 .

6. Specifically, utilizing the attention-based registration mechanism to realize registration between the AFM dataset and the WLI dataset based on the extracted feature region descriptors includes: Descriptor classification is performed on the feature extraction results of WLI and AFM, and the self-attention matrix C is calculated as follows: ｓ , A ｓ Get (Equation 4) Here, S ｉ is the feature point set, y is a feature point, and f ｙ is a feature parameter, and F ｉ is the i-th element of the feature parameter set, and g ｙ is the size of the feature region, and G ｉ , G ｉ+1 are the minimum and maximum values ​​of the size of the feature region, respectively, and the value of the mth row and nth column of the self-attention matrix represents the probability that the set including feature point m simultaneously includes feature point n. Cross-matching between point sets is performed to obtain a transformation matrix. In different matching methods, a confidence factor is calculated based on the number of matching point pairs and the position error, and a cross-attention matrix with matching probability information is obtained. (Equation 5) Here, C ｓｉ A ｓｊ Set C ｓｉ and Set A ｓｊ indicates that it matches, and O Ｃｍ and O Ａｎ are set C ｓｉ The position coordinates of the mth feature region in set A ｓｊ The position coordinates of the nth feature region in R ｉｊ and T ｉｊ are the rotation and translation transformation matrices, respectively, and ξ ｉｊ Set C ｓｉ and Set A ｓｊ is the matching confidence factor, and d ｔｈ is the threshold value of the position error for determining whether two points are a matching point pair, is the sum of the position errors of the matching point pairs, and the position error d<d ｔｈ where N is the number of matching point pairs, , P ｉｊ Set C ｓｉ and Set A ｓｊ is the confidence probability of matching, Step S3.2, where The rotation transformation matrix R ｉｊ and the translation transformation matrix T ｉｊ Calculate the matching error d of the feature points in the other point set due to the rotation transformation using the formula (2), adjust the region size threshold in the point set classification process, and update the self-attention matrix; (Equation 6) Here, [G ｉ , G ｉ＋１ ) is the initial threshold range, and [G ｉ ', G ｉ＋１ ') is the updated threshold range in step S3.3; Step S3.4: repeating steps S3.2 to S3.3 until the probability of the cross-attention matrix no longer changes due to the iterative process, and obtaining the matching result with the highest probability as the optimal matching method; characterized in that it comprises The feature region based cross-scale data fusion method according to claim 3 .

7. The method further includes a step of verifying the registration accuracy by comparing the registration results of the ICP algorithm and the SIFT algorithm with the registration results of the iterative similar region algorithm. The feature region based cross-scale data fusion method of claim 1 .

8. a pre-processing module configured to pre-process an AFM dataset, which is probe scanning measurement data by an atomic force microscope, and a WLI dataset, which is vertical scanning measurement data by a white light interferometer, to identify a reference surface in the measurement data, and to calibrate errors of the different data sources; a feature extraction module that uses an iterative similar region algorithm to extract feature region descriptors based on the preprocessed data, the feature region descriptors being configured to include location, region center points, descriptors, region size, boundaries, and feature parameters; a data registration module arranged to achieve registration of the AFM dataset and the WLI dataset based on the extracted feature region descriptors using an attention-based registration mechanism; a data fusion module configured to fuse the WLI data set and the AFM data set based on the registration result, register the height reference, then replace the AFM measurement data with the WLI measurement data of the overlapping portion, and perform linear interpolation smoothing processing at the edges to complete the fusion of the AFM data set and the WLI data set; A feature region-based cross-scale data fusion device comprising:

9. A feature region-based cross-scale data fusion device, comprising at least one processing unit and at least one storage unit, A feature region-based cross-scale data fusion device, characterized in that the storage unit stores a computer program, which, when executed by the processing unit, causes the processing unit to perform the steps of the method of any one of claims 1 to 7.

10. 1. A storage medium storing a computer program executable by a feature region-based cross-scale data fusion device, comprising: When the computer program is executed on a feature region-based cross-scale data fusion device, the feature region-based cross-scale data fusion device A storage medium for causing a computer to execute the steps of the method according to any one of claims 1 to 7.

Images