Method, apparatus, device, and storage medium for cross-scale data fusion based on feature regions

Abstract

This invention provides a method to improve the accuracy and efficiency of micro- and nanostructure measurements by resolving the registration accuracy problem between cross-scale data with large differences in lateral resolution (e.g., atomic force microscopy (AFM) and white light interferometry (WLI)). [Solution] The method involves preprocessing the AFM dataset and WLI dataset to identify the reference surface in the measurement data and calibrate the errors of different data sources S1, extracting feature region descriptors using an iterative similarity region algorithm S2, registering the two datasets based on the extracted feature region descriptors using an attention-based registration mechanism S3, fusing the WLI data and AFM data based on the registration results and registering the height reference, then replacing the AFM measurement data with the WLI measurement data in the overlapping portion, and performing smoothing processing by linear interpolation at the edge portions to complete the data fusing S4.

Description

Technical Field

[0001] This application relates to the field of three-dimensional measurement technology for micro surfaces, and more specifically, to a cross-scale data fusion method, apparatus, device, and storage medium based on feature regions.

Background Art

[0002] Since nano-micro cross-scale structures are widely used in engineering applications, the development of their measurement technology is very important. Multi-sensor data fusion is a promising approach to expand the scope of measurement technology and improve measurement resolution and efficiency by combining the advantages of various technologies.

[0003] Currently, although there have been many advancements in the research of multi-sensor data fusion methods, for data sets with overly large differences in resolution, such as when the difference in resolution is more than 10 times, existing data fusion methods have problems in effectively extracting the matching criteria for cross-scale structures. For example, the difference in the lateral resolution between the probe scanning measurement data by an atomic force microscope (AFM) and the vertical scanning measurement data by a white light interferometer (WLI) ranges from about 1:10 to 1:600. However, effective fusion cannot be achieved using existing cross-scale methods, and the registration accuracy is also limited by the difference in resolution. Therefore, there is an urgent need for a method that can achieve effective fusion between cross-scale data with a large difference in lateral resolution (such as AFM and WLI). <000008​​​​​​​​​​​​​ Specification of Chinese Patent Application Publication No. 112950682 [Patent Document 2] Specification of Chinese Patent Application Publication No. 116912296 [Summary of the Invention] [Problems to be Solved by the Invention]

[0006] In addition, the prior arts described in Patent Document 1 and Patent Document 2 respectively rely on scale adjustment by wavelet decomposition (Patent Document 1) and enhancement of position recognition by a deep learning model (Patent Document 2), and do not mention the description of the hierarchical structure of the feature region and alternative fusion technology solutions for specific sensor data.

[0007] In addition, the prior arts described in Patent Document 1 and Patent Document 2 respectively address general scale differences (Patent Document 1) and robustness in complex scenarios (Patent Document 2), and the technical problems to be solved by the present invention do not overlap with these.

[0008] Furthermore, the innovation point of the present invention focuses on the description of the feature region and the registration mechanism based on attention drive. The prior arts do not cover this type of approach, and in particular, do not mention specific processing technology solutions for AFM / WLI data.

[0009] In response to at least one defect or improvement need of the existing technology, the present invention provides a cross-scale data fusion method, apparatus, device, and storage medium based on a feature region, which can solve at least one of the problems in the above background technology. [Means for Solving the Problems]

[0010] To achieve the above object, according to a first aspect of the present invention, a cross-scale data fusion method based on a feature region is provided, and the method includes: Step S1 involves preprocessing the AFM dataset, which is probe scanning measurement data from an atomic force microscope, and the WLI dataset, which is vertical scanning measurement data from a white light interferometer, to identify the reference surface in the measurement data and calibrate the errors of the different data sources. Step S2 involves using an iterative similarity region algorithm to extract feature region descriptors based on preprocessed data, wherein the feature region descriptor includes location, region center point, descriptor, range size, boundary, and feature parameters. Step S3 involves using an attention-based registration mechanism to perform registration between the AFM dataset and the WLI dataset based on the extracted feature region descriptors. The process includes step S4, in which, based on the registration results, the WLI dataset and the AFM dataset are merged, a height reference is registered, the AFM measurement data is replaced with the overlapping WLI measurement data, and the edges are smoothed by linear interpolation to complete the fusion of the AFM dataset and the WLI dataset.

[0011] Furthermore, in the cross-scale data fusion method based on the above feature regions, the calibration of errors between the different data sources is, specifically, To suppress the bat wing effect on the WLI dataset and reduce noise at step edges caused by diffraction effects in white light interferometry, This includes reconstructing the tip model using a scanning electron microscope, performing deconvolution of the probe tip against the AFM dataset, and reducing measurement errors due to the shape and tilt of the probe tip.

[0012] Furthermore, in the cross-scale data fusion method based on the feature regions described above, extracting the feature region descriptors specifically involves, Step S2.1 involves generating a parameter matrix to be partitioned and initializing the clustering targets as a height matrix Z and its maximum number of clusters Nmax, Step S2.2 selects n cluster centers such that n ∈ [1, Nmax], Assign each data point to the nearest cluster center.

[0013] (Math 1) JPEG2026112370000002.jpg1052

[0014] Here, x is a data point, z i S2.3 is the i-th cluster center when performing height parameter clustering, Recalculate the center of each cluster,

[0015] (Math 2) JPEG2026112370000003.jpg1433

[0016] Here, M i |M is the data set of the i-th cluster. i | is the number of data points in that set, in step S2.4, Step S2.5 involves repeating steps S2.3 to S2.4 until a pre-set termination condition is met, thereby obtaining the sum of squared error losses corresponding to the current number of cluster centers n.

[0017] (Math 3) JPEG2026112370000004.jpg1653

[0018] Repeat steps S2.2 to S2.5, SSE n Step S2.6 involves obtaining a curve, calculating the SSE decay rate to obtain the optimal number of clusters N, performing connected region analysis on the cluster results, and realizing the partitioning of the dataset. Step S2.7 includes extracting feature regions from the divided dataset using RANSAC fitting, extracting a set of points from the feature regions corresponding to the current parameters to generate a feature region descriptor, and repeating steps S2.2 to S2.5 for the remaining set of points using a new parameter matrix.

[0019] Furthermore, in the cross-scale data fusion method based on the above characteristic region, generating the parameter matrix of the object to be segmented is specifically sequentially analyzing the parameter matrix of height, the parameter matrix of gradient, and the parameter matrix of curvature, and outputting the measurement result in the form of a height matrix z, and when the region is non-planar, dividing the region with the gradient matrix, and calculating the gradient matrix and the curvature using the floating interval.

[0020] Furthermore, in the cross-scale data fusion method based on the above characteristic region, initial feature extraction is performed on the distinguishable fine structure region as the primary feature region extraction, and secondary feature region extraction is performed on the region that meets the preset conditions in the result of the primary feature region extraction to obtain the detailed region.

[0021] Furthermore, in the cross-scale data fusion method based on the above characteristic region, using the attention-based registration mechanism to realize the registration of the AFM data set and the WLI data set based on the extracted characteristic region descriptor is specifically performing descriptor classification on the feature extraction results of WLI and AFM respectively, and obtaining the self-attention matrices C s and A s as follows,

[0022] (Equation 4) JPEG2026112370000005.jpg755

[0023] Here, S i is the feature point set, y is the feature point, f y is the feature parameter, F i is the i-th element of the feature parameter set, g y is the size of the feature region, G i and G i+1 are respectively the minimum value and the maximum value of the size of the feature region, and the value of the m-th row and n-th column of the self-attention matrix represents the probability of simultaneously including the feature point n in the set including the feature point m, step S3.1 Cross-matching is performed between sets of points to obtain a transformation matrix. Different matching methods calculate a confidence factor based on the number of matching point pairs and positional error, and obtain a cross-attention matrix containing matching probability information.

[0024] (Math 5) JPEG2026112370000006.jpg4383

[0025] Here C si JPEG2026112370000007.jpg45A sj is set C si and set A sj This indicates that they match, O Cm and O An These are each set C si The position coordinates of the mth feature region and set A sj In the position coordinates of the nth feature region, R ij and T ij These are the rotation transformation matrix and the translation transformation matrix, respectively, ξ ij is set C si and set A sj d is a matching confidence factor, th This is the position error threshold for determining whether two points are a matching point pair. JPEG2026112370000008.jpg1013 is the sum of the positional errors of the matching point pairs, and the positional error d <d th And N is the number of matching point pairs, JPEG2026112370000009.jpg716, P ij is set C si and set A sj This is the confidence probability of matching. Step S3.2 is JPEG2026112370000010.jpg718, The rotation transformation matrix R ij and the translation transformation matrix T ijUsing this, the matching error d of feature points in other point sets under the rotational transformation is calculated, the region size threshold of the point set classification process is adjusted, and the self-attention matrix is ​​updated.

[0026] (Math 6) JPEG2026112370000011.jpg3381

[0027] Here [G i , G i+1 ) is the initial threshold range, and [G i ', G i+1 ') is the updated threshold range in step S3.3, Steps S3.2 to S3.3 are repeated until no change in probability occurs due to iteration in the cross-attention matrix, and step S3.4 is taken to obtain the matching result with the highest probability as the optimal matching method.

[0028] Furthermore, the cross-scale data fusion method based on the feature regions described above further includes a step to verify registration accuracy by comparing the registration results of the ICP algorithm and the SIFT algorithm with the registration results of the Iterative Similarity Region (ISR) algorithm.

[0029] A second aspect of the present invention further provides a cross-scale data fusion device based on feature regions. A preprocessing module is configured to preprocess AFM datasets, which are probe scanning measurement data from an atomic force microscope, and WLI datasets, which are vertical scanning measurement data from a white light interferometer, to identify reference surfaces in the measurement data, and to calibrate errors between different data sources. Using an iterative similarity region algorithm, feature region descriptors are extracted based on the preprocessed data, and the feature region descriptors are arranged in a feature extraction module that includes location, region center point, descriptor, range size, boundary, and feature parameters. A data registration module is configured to enable registration of AFM datasets and WLI datasets based on extracted feature region descriptors, utilizing an attention-based registration mechanism. The system includes a data fusion module that, based on registration results, merges the WLI dataset and the AFM dataset, registers the height reference, replaces the AFM measurement data with the overlapping WLI measurement data, and performs linear interpolation smoothing on the edges to complete the fusion of the AFM dataset and the WLI dataset.

[0030] A third aspect of the present invention further provides a feature region-based cross-scale data fusion device, the device comprising at least one processing unit and at least one storage unit, wherein a computer program is stored in the storage unit, and when the computer program is executed by the processing unit, the processing unit is made to perform the steps of any of the above methods.

[0031] A fourth aspect of the present invention is to provide a storage medium that stores a computer program executable by a feature-region-based cross-scale data fusion device, and when the computer program is executed by the feature-region-based cross-scale data fusion device, the feature-region-based cross-scale data fusion device is made to perform the steps described in any of the above methods. [Effects of the Invention]

[0032] In general, the above-described technical proposal conceived by the present invention can achieve the following beneficial effects compared to existing technologies. The feature region-based cross-scale data fusion method provided by the present invention precisely fuses high-resolution AFM data and large-scale WLI data through an iterative similar region algorithm and an attention-based registration mechanism, solving the problem of cross-scale data registration accuracy and improving the accuracy and efficiency of fine and nanostructure measurements.

[0033] To more clearly explain the technical concepts in the embodiments of this application, the necessary drawings for use in the embodiments are briefly introduced below. However, obviously, the drawings described below represent only a few embodiments of this application, and those skilled in the art can obtain other drawings based on these without paying any creative work. [Brief explanation of the drawing]

[0034] [Figure 1] This is a schematic flowchart of a cross-scale data fusion method based on feature regions provided by an embodiment of this application. [Modes for carrying out the invention]

[0035] To further clarify the purpose, technical concept, and advantages of the present invention, the present invention will be described in more detail below with reference to the drawings and examples. The specific examples described herein are for the purpose of aiding understanding the present invention and do not limit it. Furthermore, the technical features described in each of the following embodiments can be combined in any way, as long as they do not contradict each other.

[0036] The terms “first,” “second,” “third,” etc., in the specification, claims, and drawings of this application are used to distinguish different subjects and are not intended to describe a specific order. Furthermore, the terms “includes” and “have,” and their variations, are intended to cover exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units already listed, and may also include steps or units not listed, or other steps or units specific to those processes, methods, products, or apparatus.

[0037] Figure 1 is a schematic flowchart of a feature region-based cross-scale data fusion method provided by an embodiment of this application. As shown in Figure 1, the feature region-based cross-scale data fusion method provided by an embodiment of this application is Step S1 involves preprocessing the AFM dataset, which is probe scanning measurement data from an atomic force microscope, and the WLI dataset, which is vertical scanning measurement data from a white light interferometer, to identify the reference surface in the measurement data and calibrate the errors of the different data sources. Step S2 involves using an iterative similarity region algorithm to extract feature region descriptors based on preprocessed data, wherein the feature region descriptor includes location, region center point, descriptor, range size, boundary, and feature parameters. Step S3 involves using an attention-based registration mechanism to perform registration between the AFM dataset and the WLI dataset based on the extracted feature region descriptors. The process includes step S4, in which, based on the registration results, the WLI dataset and the AFM dataset are merged, the height reference is registered, the AFM measurement data is replaced with the overlapping WLI measurement data, and the edge portions are smoothed by linear interpolation to complete the fusion of the AFM data and the WLI data.

[0038] Specifically, both Atomic Force Probe Scanning (AFM) datasets and Vertical Scan White Light Interferometry (WLI) datasets are point cloud data. While the vertical resolution of WLI and AFM is similar, the horizontal resolution span is very large. Generally, the horizontal resolution of nuclear microscopes depends on the scanning step length and varies from 1 nm to 50 nm. However, due to the limited size of the probe tip, the measurement results are affected by the convolution effect of the probe tip. On the other hand, WLI is limited by the diffraction limit, and its horizontal resolution is generally less than 1 μm. Furthermore, in step-edge regions, WLI measurements are susceptible to the "bat wing effect," which generates noise data.

[0039] In the data preprocessing stage, a reference surface is first identified in the measurement data, and the surface is planarized using the least-squares plane method to calibrate errors between different data sources. For WLI data, bat wing effect suppression is performed to reduce noise from step edges caused by diffraction effects. For AFM data, the tip model is reconstructed using a scanning electron microscope (SEM), and deconvolution of the probe tip is performed to reduce measurement errors due to the shape and tilt of the probe tip.

[0040] The accuracy of feature region segmentation in WLI, or low-resolution data, is affected by the low resolution of the data source, limiting the matching accuracy of cross-scale data. Therefore, instead of feature point extraction in conventional methods, feature regions are classified by extracting them and generating feature region descriptors. This classification threshold is then adjusted by combining it with the position matching error to obtain the rotation and translation relationship between WLI and AFM data.

[0041] Based on the preprocessed data, feature region descriptors are extracted from the data using the Iterative Similarity Region (ISR) algorithm. K-means clustering and RANSAC algorithm optimization are involved in the extraction of feature region descriptors to improve the accuracy of feature region extraction. Feature region descriptors include location, region center point, descriptor, range size, boundary, and feature parameters (e.g., height, gradient, curvature).

[0042] The AFM region is precisely positioned within the WLI coordinate system using an attention-based registration mechanism. This mechanism incorporates the concepts of cross-attention and self-attention, and optimizes registration accuracy between cross-scale data by eliminating the influence of noise regions through alternating iterative processing.

[0043] By combining the matching of feature region descriptors and attention mechanisms, registration errors caused by insufficient feature region segmentation accuracy in low-resolution data can be effectively reduced.

[0044] Based on the registration results, WLI and AFM data are merged. Since the accuracy of AFM measurement data is much higher than that of WLI measurement data, in the merging process for overlapping regions, the height criterion is first registered, and then the AFM measurement data is replaced with the WLI measurement data for the overlapping portion. A uniform registration process is performed on the boundary between AFM and WLI data to make the final fused dataset smoother.

[0045] The feature region-based cross-scale data fusion method provided by the embodiments of this application precisely fuses high-resolution AFM data and large-scale WLI data through an iterative similar region algorithm and an attention-based registration mechanism, solving the problem of cross-scale data registration accuracy and improving the accuracy and efficiency of fine and nanostructure measurements.

[0046] Furthermore, in the cross-scale data fusion method based on feature regions provided by the embodiments of this application, the error of the different data sources is specifically: To suppress the bat wing effect on the WLI dataset and reduce noise at step edges caused by diffraction effects in white light interferometry, This may include reconstructing the tip model using a scanning electron microscope, performing deconvolution of the probe tip on the AFM data, and reducing measurement errors due to the shape and tilt of the probe tip.

[0047] Specifically, the bat wing effect is a phenomenon that occurs in white light scanning interferometry (SAS) and is mainly caused by diffraction. When the height of the step of the object being measured is smaller than the coherence length of the light source, an artifact resembling a bat wing appears at the edge of the step, and this phenomenon is called the bat wing effect. The bat wing effect can be eliminated or compensated for by software or hardware improvements, such as adjusting the system's hardware parameters or applying filtering correction to the three-dimensional shape obtained by demodulating the coherent signal, thereby improving the accuracy of the measurement results.

[0048] A scanning electron microscope is used to reconstruct the tip model and measure the tip shape of the atomic force probe. Using this tip model, deconvolution of the tip is performed on the measurement data from the atomic force microscope to reduce measurement errors caused by the tip shape and tilt.

[0049] Furthermore, in the cross-scale data fusion method based on feature regions provided by the embodiment of this application, the extraction of the feature region descriptor is specifically: Step S2.1 involves generating a parameter matrix for the partitioning target and initializing the clustering target as a height matrix Z and its maximum number of clusters Nmax, Step S2.2 selects n cluster centers, and among them n∈[1, Nmax], Assign each data point to the nearest cluster center.

[0050] (Number 7) JPEG2026112370000012.jpg1254

[0051] Here, x is a data point, z i S2.3 is the i-th cluster center when performing height parameter clustering, Recalculate the center of each cluster,

[0052] (Math 8) JPEG2026112370000013.jpg1633

[0053] Here, M i |M is the data set of the i-th cluster. i | is the number of data points in that set, in step S2.4, Step S2.5 involves repeating steps S2.3 to S2.4 until a pre-set termination condition is met, thereby obtaining the sum of squared error losses corresponding to the current number of cluster centers n.

[0054] (Math 9) JPEG2026112370000014.jpg1961

[0055] Repeat steps S2.2 to S2.5, SSE n Step S2.6 involves obtaining a curve, calculating the SSE decay rate to obtain the optimal number of clusters N, performing connected region analysis on the cluster results, and realizing the partitioning of the dataset. The process may include step S2.7, in which feature regions are extracted from the partitioned dataset using RANSAC fitting, a set of points representing the feature regions corresponding to the current parameters is extracted to generate a feature region descriptor, and steps S2.2 to S2.5 are repeated for the remaining set of points using a new parameter matrix.

[0056] Specifically, the foundation for data registration is laid by partitioning the feature regions of WLI and AFM measurement data using a multidimensional clustering method of K-means clustering. K-means clustering is a classical unsupervised learning algorithm, and in this embodiment, an optimized K-means clustering method specifically for surface measurement data is used to automatically determine the clustering dimension and number of clusters by analyzing the decay rate of surface fitting errors. Furthermore, the RANSAC algorithm is used to limit the clustering error within the feature regions, ensuring more accurate partitioning.

[0057] The system generates a parameter matrix for the partitioning target, sequentially analyzes three types of parameter matrices—a height parameter matrix, a gradient parameter matrix, and a curvature parameter matrix—and outputs the measurement results in the form of a height matrix Z. The clustering target is initialized as the height matrix Z and its maximum number of clusters Nmax. A relatively large value can be selected for the maximum number of clusters, and the optimal number of clusters and the corresponding clustering results can be automatically obtained in the subsequent analysis process.

[0058] When a set of points in the feature region corresponding to the current parameters is extracted and a feature region descriptor is generated, the result is as shown in Table 1 below.

[0059] (Table 1) Table 1: Elements included in feature region descriptors JPEG2026112370000015.jpg2277

[0060] Furthermore, in the cross-scale data fusion method based on feature regions provided by the embodiment of this application, generating the parameter matrix to be divided is, specifically, The height parameter matrix, the gradient parameter matrix, and the curvature parameter matrix are analyzed sequentially, and the measurement results are output in the form of the height matrix z. If the region is non-planar, the method may include dividing the region with a gradient matrix and calculating the gradient matrix and curvature using floating intervals.

[0061] Specifically, based on prior knowledge of surface metrology, feature types are classified into four categories: planar, inclined, spherical, and irregular surfaces, and then sorted according to their probability of occurrence. All unclassified regions are designated as noise points. In the feature extraction process, surface features are classified using a hierarchical structure of three matrices: height (Z), gradient (D), and curvature (Q). The accuracy of the classification is evaluated using the surface type error, which helps to improve the segmentation of feature regions. Regions that cannot be classified as planar, inclined, or spherical are labeled as irregular surface regions or local noise points based on their connected regions.

[0062] If some regions cannot be classified as planes, these regions can be partitioned using a gradient matrix D. The gradient matrix D and curvature matrix Q can be calculated over floating intervals. For example, a 2D gradient matrix D and a 4-neighbor curvature matrix Q can be calculated using the following formulas.

[0063] (Number 10) JPEG2026112370000016.jpg2274

[0064] Here, r(i, j) is the radius of the circle passing through the fitting point Z(i, j) and its four neighboring points.

[0065] Furthermore, in the cross-scale data fusion method based on feature regions provided by the embodiment of this application, initial feature extraction may be performed on identifiable microstructure regions, which may be designated as primary feature region extractions. Secondary feature region extraction may then be performed on regions that satisfy pre-set conditions in the results of the primary feature region extractions to obtain detailed regions.

[0066] Specifically, the feature extraction process allows for the precise segmentation of fine structure regions within WLI data. In cross-scale measurements, fine structure regions can be identified through initial feature extraction, which is called primary feature region extraction. Next, secondary extraction is performed on the relatively large regions identified by the primary extraction to extract more detailed regions. While the two extraction processes are similar, secondary extraction uses a more stringent RANSAC fitting threshold to refine the detailed regions within the WLI data. These two extraction processes form the basis for registering WLI and AFM data.

[0067] Furthermore, in the feature region-based cross-scale data fusion method provided by the embodiment of this application, the registration of two datasets based on the extracted feature region descriptors is achieved by utilizing the attention-based registration mechanism, specifically, Descriptor classification is performed on the feature extraction results of WLI and AFM respectively, and the self-attention matrix C is defined as follows: s , A s Obtain,

[0068] (Math 11) JPEG2026112370000017.jpg755

[0069] Here, S i is a set of feature points, y is a feature point, and f y F is a feature parameter, i is the i-th element of the feature parameter set, and g y G is the size of the feature region. i , G i+1 These are the minimum and maximum values ​​of the feature region size, respectively, and the value in the mth row and nth column of the self-attention matrix represents the probability of simultaneously containing feature point n within a set containing feature point m, in step S3.1, Cross-matching is performed between sets of points to obtain a transformation matrix. Different matching methods calculate a confidence factor based on the number of matching point pairs and positional error to obtain a cross-attention matrix containing matching probability information.

[0070] (Math 12) JPEG2026112370000018.jpg4383

[0071] Here C si JPEG2026112370000019.jpg45A sj is set C si and set A sj This indicates that they match, O Cm and O An These are each set C si The position coordinates of the mth feature region and set A sj In the position coordinates of the nth feature region, R ij and T ij These are the rotation transformation matrix and the translation transformation matrix, respectively, ξ ij is set C si and set A sj d is a matching confidence factor, th This is the position error threshold for determining whether two points are a matching point pair. JPEG2026112370000020.jpg1013 is the sum of the positional errors of the matching point pairs, and the positional error d <d th And N is the number of matching point pairs, JPEG2026112370000021.jpg716, P ij is set C si and set A sj This is the confidence probability of matching. Step S3.2 is JPEG2026112370000022.jpg718, The rotation transformation matrix R ij and the aforementioned translation transformation matrix T ij Using this, the matching error d of feature points in other point sets due to the rotation transformation is calculated, the region size threshold of the point set classification process is adjusted, and the self-attention matrix is ​​updated.

[0072] (Math 13) JPEG2026112370000023.jpg3889

[0073] Here [G i , G i+1 ) is the initial threshold range, and [G i ', G i+1 ') is the updated threshold range in step S3.3, The process may also include step S3.4, which involves repeating steps S3.2 to S3.3 until no change in probability occurs due to iteration in the cross-attention matrix, and obtaining the matching result with the highest probability as the optimal matching method.

[0074] Specifically, the superglue method based on graph convolutional neural networks (GCNs) proposes the concepts of cross-attention and self-attention. Considering that the arrangement of feature regions in surface measurement data has structural features, the influence of noise regions on feature matching is eliminated by iteratively processing the cross-matching matrix and the self-matching matrix alternately during the feature matching process. The registration accuracy between cross-scale data is optimized by combining the processes of feature descriptor matching and position matching and iterating between them.

[0075] Furthermore, the feature region-based cross-scale data fusion method provided by the embodiments of this application may further include verifying the registration accuracy by comparing the registration results of the ICP algorithm and the SIFT algorithm with the results of the ISR algorithm.

[0076] In the embodiments of this application, a novel cross-scale data fusion method based on feature region registration is proposed, enabling high-precision measurement of fine nanocomposite structures at different scales by fusing large-scale white light interferometry (WLI) data with high-resolution atomic force microscopy (AFM) data. At the core of this method is the proposed ISR algorithm, which optimizes the conventional data fusion process and ensures the accuracy of cross-scale registration by using feature region descriptors instead of feature point descriptors.

[0077] The ISR algorithm utilizes an optimized K-means clustering method to extract feature regions from WLI and AFM data, generate region descriptors, and perform matching using an attention mechanism. This process is iterated alternately between self-matching and cross-matching to improve registration and compensate for errors due to differences in resolution. For overlapping regions after registration, the WLI data is replaced with higher-resolution AFM data, and linear interpolation smoothing is applied to the edges to ensure seamless fusion.

[0078] The ISR method excels in processing cross-scale data with significant resolution differences, providing consistent feature region extraction on a unified physical scale. Furthermore, even when ICP is optimized using artificially selected coarse registration points, the ISR algorithm demonstrates superior registration accuracy.

[0079] The embodiments of this application further provide a cross-scale data fusion device based on feature regions. A preprocessing module is configured to preprocess AFM datasets, which are probe scanning measurement data from an atomic force microscope, and WLI datasets, which are vertical scanning measurement data from a white light interferometer, to identify reference surfaces in the measurement data, and to calibrate errors between different data sources. Using an iterative similarity region algorithm, feature region descriptors are extracted based on the preprocessed data, and the feature region descriptors are arranged in a feature extraction module that includes location, region center point, descriptor, range size, boundary, and feature parameters. A data registration module is configured to enable registration of AFM datasets and WLI datasets based on extracted feature region descriptors, utilizing an attention-based registration mechanism. The system includes a data fusion module that, based on registration results, fuses the WLI dataset and the AFM dataset, registers the height reference, replaces the AFM measurement data with the overlapping WLI measurement data, performs linear interpolation smoothing on the edges, and completes the fusion of the AFM dataset and the WLI dataset.

[0080] The present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the above method. The computer-readable storage medium includes, but is not limited to, floppy disks, optical disks, DVDs, CD-ROMs, microdrives, magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0081] While the embodiments of each of the above methods are described as a combination of operations for the sake of brevity, those skilled in the art will understand that this application is not limited to the described order of operations. In this application, certain steps may be performed in other orders or simultaneously. Furthermore, those skilled in the art will understand that all embodiments described herein are merely preferred embodiments, and that the described operations or modules are not necessarily essential in this application.

[0082] In the above embodiments, each embodiment focuses on a different aspect, and for configurations not described in detail in one embodiment, the relevant descriptions in other embodiments can be referenced.

[0083] With respect to some of the embodiments disclosed in this application, it should be understood that such devices can be realized by other means. For example, the embodiments of the above devices are merely illustrative, and the division of the units is, for example, just one example of a logical functional division, and in actual implementation, different division schemes may be adopted, for example, multiple units or components may be integrated, incorporated into other systems, or some components may be omitted or not performed. Another point is that the mutual coupling, direct coupling, or communication connection illustrated or described may be an indirect coupling or communication connection via some service interface, device, or unit, which may be of electrical or other form.

[0084] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units; that is, such units may be located in the same location or distributed across multiple network units. Some or all of these units can be appropriately selected according to actual needs to achieve the technical objectives of this embodiment.

[0085] Furthermore, each functional unit in each embodiment described in this application may be integrated into a single processing unit, or each unit may be physically provided individually, or two or more units may be integrated into a single unit. These integrated units may be implemented as hardware or as software functional units.

[0086] If the above-mentioned integrated unit is implemented in the form of a software function unit and sold or used as an independent product, it can be stored in computer-readable memory. Based on this understanding, the essential parts of the proposed technology of this application, or parts that contribute to existing technology, or all or part of the proposed technology, may appear in the form of a software product, which is stored in memory and contains a number of instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of this application. On the other hand, the memory includes any medium capable of storing program code, such as USB memory, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0087] Those skilled in the art will understand that all or some of the steps of the various methods of the above embodiments can be completed by a program that gives instructions to the relevant hardware, and that such program can be stored in computer-readable memory, including flash disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0088] The foregoing description is merely an exemplary embodiment of the Disclosure and does not limit the scope of the Disclosure. That is, all equivalent changes and modifications made in accordance with the teachings of the Disclosure are still included in the scope of the Disclosure. A person skilled in the art will readily come up with other ways of carrying out the Disclosure, given the description and the practices disclosed herein. This application is intended to cover any variations, uses, or adaptations of the Disclosure, which, in accordance with the general principles of the Disclosure, include common and conventional technical means known in the art not described herein. The description and examples are merely illustrative, and the scope and spirit of the Disclosure are defined by the claims.

[0089] Each of the technical features of the above embodiments can be combined in any way, and for the sake of brevity, not all possible combinations of the various technical features of the above embodiments are described. However, as long as these combinations of technical features are inconsistent, they are all included within the scope described herein.

[0090] Those skilled in the art will readily understand that the above is merely a preferred embodiment of the present invention and does not limit it, and that all modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are all within the scope of protection of the present invention.

Claims

1. Step S1 involves preprocessing the AFM dataset, which is probe scanning measurement data from an atomic force microscope, and the WLI dataset, which is vertical scanning measurement data from a white light interferometer, to identify the reference surface in the measurement data and to calibrate the errors of the different data sources. Step S2 involves using an iterative similarity region algorithm to extract feature region descriptors based on preprocessed data, wherein the feature region descriptor includes location, region center point, descriptor, range size, boundary, and feature parameters. Step S3 involves using an attention-based registration mechanism to perform registration between the AFM dataset and the WLI dataset based on the extracted feature region descriptor. Step S4 involves merging the WLI dataset and the AFM dataset based on the registration results, registering the height reference, replacing the AFM measurement data with the overlapping WLI measurement data, and performing a smoothing process using linear interpolation at the edges to complete the merging of the AFM dataset and the WLI dataset. A cross-scale data fusion method based on feature regions, characterized by including the following.

2. Calibrating the errors of the aforementioned different data sources specifically involves, To suppress the bat wing effect on the WLI dataset and reduce noise at step edges caused by diffraction effects in the white light interferometer, This method is characterized by reconstructing the tip model using a scanning electron microscope, performing deconvolution of the probe tip on the AFM dataset, and reducing measurement errors due to the shape and tilt of the probe tip. A cross-scale data fusion method based on the feature region described in claim 1.

3. Extracting the aforementioned feature region descriptors specifically means, Step S2.1 involves generating a parameter matrix for the partitioning target and initializing the clustering target as a height matrix Z and its maximum number of clusters Nmax, Step S2.2 selects n cluster centers such that n ∈ [1, Nmax], Assign each data point to the nearest cluster center. (Math 1) Here, x is a data point, z ｉ S2.3 is the i-th cluster center when performing height parameter clustering, Recalculate the center of each cluster, (Math 2) Here, M ｉ This is the data set for the i-th cluster, |M ｉ | is the number of data points in that set, in step S2.4, Step S2.5 involves repeating steps S2.3 to S2.4 until a predetermined termination condition is met, thereby obtaining the sum of squared error losses corresponding to the current number of cluster centers n. (Math 3) Repeat steps S2.2 to S2.5, SSE ｎ Step S2.6 involves obtaining a curve, calculating the SSE decay rate to obtain the optimal number of clusters N, performing a connected region analysis on the cluster results, and realizing the partitioning of the dataset. Step S2.7 involves extracting feature regions from the divided dataset using RANSAC fitting, extracting a set of points from the feature regions corresponding to the current parameters to generate a feature region descriptor, and repeating steps S2.2 to S2.5 for the remaining set of points using a new parameter matrix. Features that include A cross-scale data fusion method based on the feature region described in claim 1.

4. The process of generating the parameter matrix to be partitioned specifically involves, The parameter matrices for height, gradient, and curvature are analyzed sequentially, and the measurement results are output in the form of the height matrix z. If the region is non-planar, the method includes dividing the region with a gradient matrix and calculating the gradient matrix and curvature using floating intervals. A cross-scale data fusion method based on the feature region described in claim 3.

5. This method involves performing initial feature extraction on identifiable microstructure regions to obtain primary feature region extraction, and then performing secondary feature region extraction on regions that match pre-set conditions in the results of the primary feature region extraction to obtain detailed regions. A cross-scale data fusion method based on the feature region described in claim 3.

6. Specifically, using the attention-based registration mechanism described above, and based on the extracted feature region descriptors, registering the AFM dataset and the WLI dataset can be achieved as follows: Descriptor classification is performed on the feature extraction results of WLI and AFM respectively, and the self-attention matrix C is defined as follows: ｓ A ｓ Obtain, (Math 4) Here, S ｉ is a feature point set, y is a feature point, and f ｙ is a feature parameter, F ｉ is the i-th element of the feature parameter set, g ｙ is the size of the feature region, G ｉ , G ｉ+1 are respectively the minimum value and the maximum value of the size of the feature region. The value of the m-th row and n-th column of the self-attention matrix represents the probability of simultaneously including the feature point n in the set including the feature point m. Step S3.1 Cross-matching is performed between sets of points to obtain a transformation matrix. Different matching methods calculate a confidence factor based on the number of matching point pairs and positional error to obtain a cross-attention matrix containing matching probability information. (Math 5) Here C ｓｉ A ｓｊ Set C ｓｉ and Set A ｓｊ This indicates that they match, O Ｃｍ and O Ａｎ These are each set C ｓｉ The position coordinates of the mth feature region and set A ｓｊ In the position coordinates of the nth feature region, R ｉｊ and T ｉｊ These are the rotation transformation matrix and the translation transformation matrix, respectively, ξ ｉｊ Set C ｓｉ and Set A ｓｊ d is the matching confidence factor, ｔｈ This is the position error threshold used to determine whether two points are a matching point pair. is the sum of the positional errors of the matching point pairs, where positional error d < d ｔｈ And N is the number of matching point pairs, , P ｉｊ Set C ｓｉ and Set A ｓｊ This is the confidence probability of matching. Step S3.2 is, The rotation transformation matrix R ｉｊ and the translation transformation matrix T ｉｊ Using this, the matching error d of feature points in other point sets due to the rotation transformation is calculated, the region size threshold of the point set classification process is adjusted, and the self-attention matrix is ​​updated. (Math 6) Here [G ｉ G ｉ＋１ ) is the initial threshold range, and [G ｉ ', G ｉ＋１ ') is the updated threshold range in step S3.3, Steps S3.2 to S3.3 are repeated until no change in probability occurs due to iteration in the cross-attention matrix, and step S3.4 is performed to obtain the matching result with the highest probability as the optimal matching method. Features that include A cross-scale data fusion method based on the feature region described in claim 3.

7. The method further includes a step of verifying registration accuracy by comparing the registration results of the ICP algorithm, the registration results of the SIFT algorithm, and the registration results of the iterative similarity domain algorithm. A cross-scale data fusion method based on the feature region described in claim 1.

8. A preprocessing module is configured to preprocess AFM datasets, which are probe scanning measurement data from an atomic force microscope, and WLI datasets, which are vertical scanning measurement data from a white light interferometer, to identify reference surfaces in the measurement data, and to calibrate errors between different data sources. Using an iterative similarity region algorithm, feature region descriptors are extracted based on the preprocessed data, and the feature region descriptors are arranged in a feature extraction module that includes location, region center point, descriptor, range size, boundary, and feature parameters. A data registration module is configured to enable registration of AFM datasets and WLI datasets based on extracted feature region descriptors, utilizing an attention-based registration mechanism. Based on the registration results, the data fusion module is positioned to merge the WLI dataset and AFM dataset, register the height reference, replace the overlapping WLI measurement data with AFM measurement data, and perform linear interpolation smoothing at the edges to complete the fusion of the AFM dataset and WLI dataset. A cross-scale data fusion device based on a feature region, characterized by including the following.

9. A feature-region-based cross-scale data fusion device comprising at least one processing unit and at least one storage unit, A cross-scale data fusion device based on a feature region, characterized in that a computer program is stored in the storage unit, and when the computer program is executed by the processing unit, the processing unit is made to perform the steps of the method according to any one of claims 1 to 7.

10. A storage medium for storing a computer program executable by a feature-domain-based cross-scale data fusion device, When the aforementioned computer program is executed on a cross-scale data fusion device based on feature regions, the cross-scale data fusion device based on feature regions A storage medium that causes the steps of the method according to any one of claims 1 to 7 to be performed.

Images