A method, device, equipment and storage medium for cross-scale data fusion based on feature regions

Abstract

The present application discloses a cross-scale data fusion method based on feature regions, including: S1 pre-processing AFM data and WLI data, determining the reference surface in the measurement data, and correcting the errors of different data sources; S2 using an iterative similarity region algorithm to extract feature region descriptors; S3 using an attention-based alignment mechanism combined with the extracted feature region descriptors to achieve registration of the two data sets; S4 based on the registration results, fusing the WLI and AFM data, and after aligning the height reference, replacing the WLI measurement data of the overlapping parts with the AFM measurement data, performing linear interpolation smoothing at the edges, and completing the data fusion. The present invention also discloses a cross-scale data fusion device based on feature regions, corresponding equipment, and storage medium. The cross-scale data fusion method based on feature regions provided by the present invention can solve the problem of cross-scale data registration accuracy and improve the accuracy and efficiency of micro-nanostructure measurement.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional measurement of microscopic surfaces, and more specifically, to a method, apparatus, device, and storage medium for cross-scale data fusion based on feature regions. Background Art

[0002] Nanoscale and microscale structures are ubiquitous in engineering applications, making the development of measurement techniques crucial. Multi-sensor data fusion is a promising approach that combines the strengths of different technologies to expand the scope of measurement techniques and improve measurement resolution and efficiency.

[0003] Research on multi-sensor data fusion methods has made significant progress, but existing data fusion methods face challenges in effectively extracting matching benchmarks for cross-scale structures when the resolution of datasets differs significantly, for example, by a factor of ten or more. For example, the lateral resolution difference between atomic force probe scanning (AFM) data and vertical scanning white light interferometry (WLI) data ranges from 1:10 to 1:600. Existing cross-scale methods cannot effectively fuse these data, and the registration accuracy is also limited by the resolution difference. Therefore, a method is urgently needed to effectively fuse cross-scale data with large lateral resolution differences, such as those between AFM and WLI. Summary of the Invention

[0004] In response to at least one defect or improvement need in the prior art, the present invention provides a method, apparatus, device and storage medium for cross-scale data fusion based on feature regions, which can solve at least one of the problems existing in the above-mentioned background technology.

[0005] To achieve the above object, according to a first aspect of the present invention, a cross-scale data fusion method based on feature regions is provided, the method comprising:

[0006] S1 pre-processes AFM data and WLI data, determines the reference surface in the measurement data, and corrects the errors of different data sources;

[0007] S2 uses an iterative similarity region algorithm to extract feature region descriptors based on preprocessed data. The feature region descriptors include position, region center point, descriptor, range size, boundary and feature parameters.

[0008] S3 uses an attention-based alignment mechanism combined with extracted feature region descriptors to achieve registration of the two datasets;

[0009] Based on the registration results, S4 fuses the WLI and AFM data. After aligning the height reference, the AFM measurement data replaces the WLI measurement data of the overlapping parts, and linear interpolation and smoothing are performed on the edges to complete the fusion of the AFM data and WLI data.

[0010] Furthermore, in the above-mentioned cross-scale data fusion method based on feature regions, the error correction of different data sources specifically includes:

[0011] Bat-wing effect suppression is performed on WLI data to reduce step edge noise caused by diffraction effects in white light interferometers;

[0012] The tip model was reconstructed using a scanning electron microscope and the tip was deconvoluted on the AFM data to reduce the measurement errors caused by the tip shape and slope.

[0013] Furthermore, in the above-mentioned cross-scale data fusion method based on feature regions, the extraction of feature region descriptors specifically includes:

[0014] S2.1 Generate the parameter matrix to be divided, initialize the clustering object to the height matrix Z and its maximum number of clusters Nmax;

[0015] S2.2 select n cluster centers, where n∈[1,Nmax];

[0016] S2.3 Assign each data point to the nearest cluster center

[0017]

[0018] Where x is the data point, z i is the i-th cluster center when performing high-parameter clustering;

[0019] S2.4 Recalculate the center of each cluster

[0020]

[0021] Among them, M i is the set of data points of the ith cluster, |M i | is the number of data points in the set;

[0022] S2.5 Repeat steps S2.3-S2.4 until the preset termination condition is met, and obtain the error loss square sum corresponding to the current number of cluster centers n

[0023]

[0024] S2.6 Repeat steps S2.2-S2.5 to obtain SSE nThe curve is used to calculate the SSE decay rate to obtain the optimal number of clusters N, and the connected domain analysis is performed on the clustering results to achieve data set partitioning;

[0025] S2.7 uses RANSAC fitting to extract feature regions from the divided data set, extracts the feature region click set corresponding to the current parameters to generate a feature region descriptor, and repeats steps S2.2-S2.5 for the remaining point sets with a new parameter matrix.

[0026] Furthermore, in the above-mentioned cross-scale data fusion method based on feature regions, generating the parameter matrix to be divided specifically includes:

[0027] Analyze the parameter matrices of height, gradient and curvature in sequence, and output the measurement results in the form of height matrix z;

[0028] If the region is non-planar, the region is divided using a gradient matrix, and the gradient matrix and curvature are calculated using a floating interval.

[0029] Furthermore, the above-mentioned cross-scale data fusion method based on feature regions performs initial feature extraction on identifiable microstructure regions as primary feature region extraction; and performs secondary feature region extraction on regions that meet preset conditions in the results of the primary feature region extraction to obtain detail regions.

[0030] Furthermore, the above-mentioned cross-scale data fusion method based on feature regions utilizes an attention-based alignment mechanism in combination with the extracted feature region descriptors to achieve registration of the two datasets, specifically including:

[0031] S3.1 respectively classifies the feature extraction results of WLI and AFM into descriptors and obtains the self-attention matrix C s 、A s ,have

[0032] S i ={y|f y =F i ,g y ∈[G i ,G i+1 )}

[0033] Among them, S i is the feature point set, y is the feature point, f y Characteristic parameter, g y is the size of the feature area, and the value of the mth row and nth column in the self-attention matrix represents the probability that the set containing feature point m also contains feature point n;

[0034] S3.2 cross-matches the point sets to obtain the transformation matrix. Different matching schemes calculate the confidence factor based on the number of matching point pairs and position error to obtain the cross attention matrix with matching probability information.

[0035]

[0036] in Represents the set C si With set A sj Match, O Cm , O An The set C si The position coordinates of the mth feature region and the set A sj The position coordinates of the nth feature region in R ij 、T ij are the rotation transformation matrix and the translation transformation matrix respectively, ξ ij For set C si With set A sj Matching confidence factor, d th To determine whether two points are matching point pairs, the position error threshold is used. ∑d is the sum of the position errors of the matching point pairs. There is a position error d <d th ,N is the number of matching point pairs, N≥4, P ij For set C si With set A sj The matching confidence probability is ∑P=1;

[0037] S3.3 The rotation and translation transformation matrix R ij 、T ij Calculate the matching error d of the feature points in other point sets under the rotation transformation to adjust the regional size threshold of the point set classification process and update the self-attention matrix

[0038]

[0039] Among them [G i ,G i+1 ) is the initial threshold range, [G i ′,G i+1 ') is the updated threshold range;

[0040] S3.4 Repeat steps S3.2-S3.3 until the iterative process in the cross-attention matrix does not cause probability changes, and obtain the matching result with the highest probability as the optimal matching solution.

[0041] Furthermore, the above-mentioned cross-scale data fusion method based on feature regions also includes verifying the registration accuracy by comparing the registration results of the ICP algorithm and the SIFT algorithm with the result of the ISR algorithm.

[0042] According to a second aspect of the present invention, a cross-scale data fusion device based on feature regions is also provided, comprising:

[0043] Preprocessing module, used to preprocess AFM data and WLI data, determine the reference surface in the measurement data, and correct the errors of different data sources;

[0044] A feature extraction module is used to extract feature region descriptors based on the preprocessed data using an iterative similarity region algorithm. The feature region descriptors include position, region center point, descriptor, range size, boundary, and feature parameters.

[0045] The data registration module is used to register two datasets using an attention-based alignment mechanism combined with extracted feature region descriptors;

[0046] The data fusion module is used to fuse the WLI and AFM data based on the registration results. After aligning the height reference, the AFM measurement data is used to replace the WLI measurement data of the overlapping part, and linear interpolation and smoothing are performed on the edges to complete the fusion of the AFM data and WLI data.

[0047] According to the third aspect of the present invention, a cross-scale data fusion device based on feature regions is also provided, which includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit performs the steps of any one of the above methods.

[0048] According to a fourth aspect of the present invention, a storage medium is also provided, which stores a computer program that can be executed by a cross-scale data fusion device based on feature regions. When the computer program runs on the cross-scale data fusion device based on feature regions, the cross-scale data fusion device based on feature regions executes the steps of any one of the above methods.

[0049] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0050] The cross-scale data fusion method based on feature regions provided by the present invention accurately fuses high-resolution AFM data and large-scale WLI data through an iterative similarity region algorithm and an attention-based alignment mechanism, thereby solving the problem of cross-scale data registration accuracy and improving the accuracy and efficiency of micro-nanostructure measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 A flowchart of a cross-scale data fusion method based on feature regions provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0054] The terms "first," "second," "third," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements, but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0055] Figure 1 A flow chart of a cross-scale data fusion method based on feature regions provided in an embodiment of the present application is shown as follows: Figure 1 As shown, an embodiment of the present application provides a cross-scale data fusion method based on feature regions, including:

[0056] S1 pre-processes AFM data and WLI data, determines the reference surface in the measurement data, and corrects the errors of different data sources;

[0057] S2 uses an iterative similarity region algorithm to extract feature region descriptors based on preprocessed data. The feature region descriptors include position, region center point, descriptor, range size, boundary and feature parameters.

[0058] S3 uses an attention-based alignment mechanism combined with extracted feature region descriptors to achieve registration of the two datasets;

[0059] Based on the registration results, S4 fuses the WLI and AFM data. After aligning the height reference, the AFM measurement data replaces the WLI measurement data of the overlapping parts, and linear interpolation and smoothing are performed on the edges to complete the fusion of the AFM data and WLI data.

[0060] Specifically, both atomic force probe scanning (AFM) datasets and vertical scanning white light interferometry (WLI) datasets are point cloud data. WLI and AFM have similar vertical resolutions, but their horizontal resolutions vary significantly. Generally speaking, the horizontal resolution of an AFM depends on the scanning step size, ranging from 1nm to 50nm. However, due to the limited size of the probe tip, the measurement results are affected by the tip convolution effect. WLI, on the other hand, is limited by the diffraction limit, with a horizontal resolution generally below 1μm. Furthermore, WLI measurements are susceptible to the "bat-wing effect" at the edge of steps, resulting in noisy data.

[0061] During data preprocessing, a reference surface is first determined for the measurement data and then flattened using the least-squares planar method to correct for errors between different data sources. WLI data are then subjected to bat-wing effect suppression to reduce step edge noise caused by diffraction effects. AFM data are then subjected to tip deconvolution using a scanning electron microscope (SEM) tip model to minimize measurement errors caused by tip shape and slope.

[0062] Since the accuracy of feature region demarcation in WLI (low-resolution data) is affected by the low resolution of the data source, this limits the accuracy of cross-scale data matching. Therefore, we extract feature regions and generate feature region descriptors instead of traditional feature point extraction methods. We classify feature regions and adjust the classification threshold based on the position matching error to obtain the rotation and translation relationship between WLI and AFM data.

[0063] Based on the preprocessed data, the Iterative Similarity Region (ISR) algorithm is used to extract feature region descriptors from the data. This extraction involves K-means clustering and RANSAC algorithm optimization to improve the accuracy of feature region extraction. Feature region descriptors include location, region center, descriptor, range size, boundary, and characteristic parameters such as height, gradient, and curvature.

[0064] An attention-based alignment mechanism is used to accurately locate the AFM region in the WLI coordinate system. This mechanism includes the concepts of cross-attention and self-attention, which optimizes the registration accuracy between cross-scale data by alternately eliminating the influence of noisy regions on feature matching.

[0065] By combining feature region descriptors with attention mechanism matching, the registration error caused by insufficient accuracy of feature region division in low-resolution data can be effectively reduced.

[0066] Based on the registration results, the WLI and AFM data are fused. Because AFM data is far more accurate than WLI data, the fusion process for the overlapping areas involves first aligning the height reference and then replacing the WLI data with the AFM data. Uniform alignment is performed at the interface between the AFM and WLI data, resulting in a smoother final fused dataset.

[0067] The cross-scale data fusion method based on feature regions provided in the embodiments of the present application accurately fuses high-resolution AFM data and large-scale WLI data through an iterative similarity region algorithm and an attention-based alignment mechanism, thereby solving the problem of cross-scale data registration accuracy and improving the accuracy and efficiency of micro-nanostructure measurement.

[0068] Optionally, in the cross-scale data fusion method based on feature regions provided in an embodiment of the present application, the correction of errors between different data sources specifically includes:

[0069] Bat-wing effect suppression is performed on WLI data to reduce step edge noise caused by diffraction effects in white light interferometers;

[0070] The tip model was reconstructed using a scanning electron microscope and the tip was deconvoluted on the AFM data to reduce the measurement errors caused by the tip shape and slope.

[0071] Specifically, the bat-wing effect is a phenomenon encountered in white-light scanning interferometry, primarily caused by diffraction. When the measured step height is less than the coherence length of the light source, an artifact resembling a bat's wing is generated at the step edge. This phenomenon is known as the bat-wing effect. This effect can be eliminated or compensated for through software or hardware improvements, such as adjusting system hardware parameters and filtering the 3D topography obtained by coherent signal demodulation, thereby improving measurement accuracy.

[0072] A scanning electron microscope (SEM) was used to reconstruct a tip model to measure the morphology of the atomic force probe tip. This tip model was then used to perform tip deconvolution on the AFM data to reduce measurement errors caused by tip shape and slope.

[0073] Optionally, in the cross-scale data fusion method based on feature regions provided in an embodiment of the present application, extracting feature region descriptors specifically includes:

[0074] S2.1 Generate the parameter matrix to be divided, initialize the clustering object to the height matrix Z and its maximum number of clusters Nmax;

[0075] S2.2 select n cluster centers, where n∈[1,Nmax];

[0076] S2.3 Assign each data point to the nearest cluster center

[0077]

[0078] Where x is the data point, z i is the i-th cluster center when performing high-parameter clustering;

[0079] S2.4 Recalculate the center of each cluster

[0080]

[0081] Among them, M i is the set of data points of the ith cluster, |M i | is the number of data points in the set;

[0082] S2.5 Repeat steps S2.3-S2.4 until the preset termination condition is met, and obtain the error loss square sum corresponding to the current number of cluster centers n

[0083]

[0084] S2.6 Repeat steps S2.2-S2.5 to obtain SSE n The curve is used to calculate the SSE decay rate to obtain the optimal number of clusters N, and the connected domain analysis is performed on the clustering results to achieve data set partitioning;

[0085] S2.7 uses RANSAC fitting to extract feature regions from the divided data set, extracts the feature region click set corresponding to the current parameters to generate a feature region descriptor, and repeats steps S2.2-S2.5 for the remaining point sets with a new parameter matrix.

[0086] Specifically, a multidimensional K-means clustering method is used to segment characteristic regions in WLI and AFM measurement data, thus laying the foundation for data registration. K-means clustering is a classic unsupervised learning algorithm. This embodiment of the application employs a K-means clustering optimization method for surface metrology data. By analyzing the decay rate of surface fitting error, the clustering dimension and number of clusters are automatically determined. Furthermore, the RANSAC algorithm is used to limit clustering errors within characteristic regions, thereby ensuring more accurate segmentation.

[0087] Generate the parameter matrix to be partitioned. The three parameter matrices of height, gradient, and curvature will be analyzed in sequence. The measurement results are output in the form of height matrix Z. The clustering object is initialized to the height matrix Z and its maximum number of clusters Nmax. The maximum number of clusters can be selected to a larger value. The optimal number of clusters and the corresponding clustering results can be automatically obtained through the subsequent analysis process.

[0088] Extract the feature area click set corresponding to the current parameters to generate the feature area descriptor as shown in Table 1.

[0089] Table 1: Elements contained in the feature region descriptor

[0090]

[0091] Optionally, in the cross-scale data fusion method based on feature regions provided in an embodiment of the present application, generating a parameter matrix to be divided specifically includes:

[0092] Analyze the parameter matrices of height, gradient and curvature in sequence, and output the measurement results in the form of height matrix z;

[0093] If the region is non-planar, the region is divided using a gradient matrix, and the gradient matrix and curvature are calculated using a floating interval.

[0094] Specifically, based on prior knowledge of surface metrology, feature types are divided into four categories: planes, slopes, spheres, and irregular surfaces, and ranked by their likelihood of occurrence. Unclassified areas are designated as noise points. The feature extraction process uses a hierarchy of three matrices: height (Z), gradient (D), and curvature (Q) to classify surface features. The surface type error is used to evaluate the accuracy of the classification, which helps to refine the division of feature regions. Regions that cannot be classified as planes, slopes, or spheres are marked as irregular surface regions or local noise points based on their connection domains.

[0095] If there are areas that cannot be classified as planes, these areas can be demarcated using the gradient matrix D. The gradient matrix D and the curvature matrix Q can be calculated using floating intervals. For example, the two-dimensional gradient matrix D and the four-neighbor curvature matrix Q can be calculated using the following formulas.

[0096]

[0097] where r(i,j) is the radius of the circle fitting point Z(i,j) and its four neighbors.

[0098] Optionally, the cross-scale data fusion method based on feature regions provided in an embodiment of the present application performs initial feature extraction on identifiable microstructure regions as primary feature region extraction; and performs secondary feature region extraction on regions that meet preset conditions in the results of the primary feature region extraction to obtain detail regions.

[0099] Specifically, the feature extraction process precisely delineates microstructural regions in WLI data. In cross-scale measurements, initial feature extraction identifies microstructural regions, referred to as first-level feature region extraction. A second-level extraction then extracts detail regions from the larger regions identified in the first-level results. The two-level extraction process is similar, but the second-level extraction uses a stricter RANSAC fitting threshold to refine the detail regions in the WLI data. This two-level extraction process lays the foundation for aligning WLI and AFM data.

[0100] Optionally, the cross-scale data fusion method based on feature regions provided in an embodiment of the present application utilizes an attention-based alignment mechanism in combination with extracted feature region descriptors to achieve registration of two data sets, specifically including:

[0101] S3.1 respectively classifies the feature extraction results of WLI and AFM into descriptors and obtains the self-attention matrix C s 、A s ,have

[0102] S i ={y|f y =F i ,g y ∈[G i ,G i+1 )}

[0103] Among them, S i is the feature point set, y is the feature point, f y Characteristic parameter, g y is the size of the feature area, and the value of the mth row and nth column in the self-attention matrix represents the probability that the set containing feature point m also contains feature point n;

[0104] S3.2 cross-matches the point sets to obtain the transformation matrix. Different matching schemes calculate the confidence factor based on the number of matching point pairs and position error to obtain the cross attention matrix with matching probability information.

[0105]

[0106] in Represents the set C si With set A sj Match, O Cm , O An The set C si The position coordinates of the mth feature region and the set A sj The position coordinates of the nth feature region in R ij 、T ij are the rotation transformation matrix and the translation transformation matrix respectively, ξ ij For set C siWith set A sj Matching confidence factor, d th To determine whether two points are matching point pairs, the position error threshold is used. ∑d is the sum of the position errors of the matching point pairs. There is a position error d <d th ,N is the number of matching point pairs, N≥4, P ij For set C si With set A sj The matching confidence probability is ∑P=1;

[0107] S3.3 The rotation and translation transformation matrix R ij 、T ij Calculate the matching error d of the feature points in other point sets under the rotation transformation to adjust the regional size threshold of the point set classification process and update the self-attention matrix

[0108]

[0109] Among them [G i ,G i+1 ) is the initial threshold range, [G i ′,G i+1 ') is the updated threshold range;

[0110] S3.4 Repeat steps S3.2-S3.3 until the iterative process in the cross-attention matrix does not cause probability changes, and obtain the matching result with the highest probability as the optimal matching solution.

[0111] Specifically, the Superglue method, based on a graph convolutional neural network, proposes the concepts of cross-attention and self-attention. Taking into account the structural characteristics of the arrangement of feature regions in surface metrology data, a cross-matching matrix and a self-matching matrix are introduced during the feature matching process for alternating iterations to eliminate the influence of noise regions on feature matching. Feature descriptor matching is coupled with position matching, and through mutual iteration, the registration accuracy between cross-scale data is optimized.

[0112] Optionally, the cross-scale data fusion method based on feature regions provided in the embodiment of the present application further includes verifying the registration accuracy by comparing the registration results of the ICP algorithm and the SIFT algorithm with the results of the ISR algorithm.

[0113] This application proposes a novel cross-scale data fusion method based on feature region alignment. This method fuses large-scale white light interferometry (WLI) data with high-resolution atomic force microscopy (AFM) data, enabling precise measurement of micro-nanocomposite structures at different scales. The core of this method is the proposed ISR algorithm, which optimizes the traditional data fusion process by replacing feature point descriptors with feature region descriptors, ensuring the accuracy of cross-scale registration.

[0114] The ISR algorithm uses an optimized K-means clustering method to extract feature regions from the WLI and AFM data, generating region descriptors that are then matched using an attention mechanism. This process iteratively alternates between self-matching and cross-matching to refine the registration and compensate for errors caused by resolution differences. For overlapping regions after registration, the higher-resolution AFM data replaces the WLI data, and linear interpolation is applied to the edges to ensure seamless fusion.

[0115] The ISR method excels at processing cross-scale data with significantly different resolutions, providing consistent feature region extraction at a uniform physical scale. Furthermore, the ISR algorithm demonstrates superior registration accuracy even when optimizing ICP using manually selected coarse alignment points.

[0116] The present application also provides a cross-scale data fusion device based on feature regions, including:

[0117] Preprocessing module, used to preprocess AFM data and WLI data, determine the reference surface in the measurement data, and correct the errors of different data sources;

[0118] A feature extraction module is used to extract feature region descriptors based on the preprocessed data using an iterative similarity region algorithm. The feature region descriptors include position, region center point, descriptor, range size, boundary, and feature parameters.

[0119] The data registration module is used to register two datasets using an attention-based alignment mechanism combined with extracted feature region descriptors;

[0120] The data fusion module is used to fuse the WLI and AFM data based on the registration results. After aligning the height reference, the AFM measurement data is used to replace the WLI measurement data of the overlapping part, and linear interpolation and smoothing are performed on the edges to complete the fusion of the AFM data and WLI data.

[0121] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above method. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a microdrive, a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0122] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0123] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0124] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interface, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0125] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0126] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0127] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0128] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0129] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure herein, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art that are not recorded in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

[0130] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A cross-scale data fusion method based on feature regions, characterized in that: include: S1 pre-processes the atomic force probe scanning data and vertical scanning white light interferometry data, determines the reference surface in the measurement data, and corrects the errors of different data sources; S2 uses an iterative similarity region algorithm to extract feature region descriptors based on preprocessed data. The feature region descriptors include position, region center point, descriptor, range size, boundary and feature parameters. S3 uses an attention-based alignment mechanism combined with extracted feature region descriptors to achieve registration of the two datasets; S4 fuses the vertical scanning white light interferometry data and the atomic force probe scanning data based on the registration results. After aligning the height reference, the vertical scanning white light interferometry data of the overlapping parts are replaced by the atomic force probe scanning measurement data. Linear interpolation and smoothing are performed on the edges to complete the fusion of the atomic force probe scanning data and the vertical scanning white light interferometry data. The attention-based alignment mechanism is combined with the extracted feature region descriptors to achieve the registration of the two datasets, specifically including: S3.1 performs descriptor classification on the feature extraction results of vertical scanning white light interferometry and atomic force probe scanning respectively to obtain the self-attention matrix and feature point set; S3.2 Cross-matching the feature point sets to obtain a transformation matrix. Different matching schemes calculate confidence factors based on the number of matching point pairs and position errors to obtain a cross-attention matrix with matching probability information. The cross-attention matrix includes rotation transformation matrix items and translation transformation matrix items. S3.3 uses the rotation transformation matrix and the translation transformation matrix to calculate the matching error of feature points in other feature point sets under the rotation transformation to adjust the region size threshold of the feature point set classification process and update the self-attention matrix; S3.4 Repeat steps S3.2-S3.3 until the iterative process in the cross-attention matrix does not cause probability changes, and obtain the matching result with the highest probability as the optimal matching solution.

2. The cross-scale data fusion method based on feature regions according to claim 1, characterized in that: Correcting errors from different data sources specifically includes: Suppress the bat-wing effect of vertical scanning white light interferometer data to reduce the step edge noise caused by the diffraction effect in the white light interferometer; The tip model was reconstructed using a scanning electron microscope and the tip deconvolution was performed on the atomic force probe scanning data to reduce the measurement error caused by the tip shape and slope.

3. The cross-scale data fusion method based on feature regions according to claim 1, characterized in that: The extraction of feature region descriptors specifically includes: S2.1 Generate the parameter matrix to be divided, initialize the clustering object to the height matrix Z and its maximum number of clusters Nmax; S2.2 Select n cluster centers, where ; S2.3 Assign each data point to the nearest cluster center in, is a data point, is the i-th cluster center when performing high-parameter clustering; S2.4 Recalculate the center of each cluster in, is the set of data points of the ith cluster, is the number of data points in the set; S2.5 Repeat steps S2.3-S2.4 until the preset termination condition is met, and obtain the error loss square sum corresponding to the current number of cluster centers n S2.6 Repeat steps S2.2-S2.5 to obtain The curve is used to calculate the SSE decay rate to obtain the optimal number of clusters N, and the connected domain analysis is performed on the clustering results to achieve data set partitioning; S2.7 uses RANSAC fitting to extract feature regions from the divided data set, extracts the feature region click set corresponding to the current parameters to generate a feature region descriptor, and repeats steps S2.2-S2.5 for the remaining point sets with a new parameter matrix.

4. The cross-scale data fusion method based on feature regions according to claim 3, characterized in that: The generating of the parameter matrix to be divided specifically includes: Analyze the parameter matrices of height, gradient and curvature in sequence, and output the measurement results in the form of height matrix z; If the region is non-planar, the region is divided using a gradient matrix, and the gradient matrix and curvature are calculated using a floating interval.

5. The cross-scale data fusion method based on feature regions according to claim 3, characterized in that: Initial feature extraction is performed on the identifiable microstructure area as the first-level feature area extraction; second-level feature area extraction is performed on the area that meets the preset conditions in the results of the first-level feature area extraction to obtain the detail area.

6. The cross-scale data fusion method based on feature regions according to claim 3, characterized in that: The attention-based alignment mechanism is combined with the extracted feature region descriptors to achieve the registration of the two datasets, specifically including: S3.1 performs descriptor classification on the feature extraction results of vertical scanning white light interferometry and atomic force probe scanning respectively to obtain the self-attention matrix ,have in, is the feature point set, is the feature point, characteristic parameters, is the size of the feature area, and the value of the mth row and nth column in the self-attention matrix represents the probability that the set containing feature point m also contains feature point n; S3.2 cross-matches the point sets to obtain the transformation matrix. Different matching schemes calculate the confidence factor based on the number of matching point pairs and position error to obtain the cross attention matrix with matching probability information. in Representing a collection With collection Match, Set The position coordinates and set of the mth feature area in The position coordinates of the nth feature area in , are the rotation transformation matrix and the translation transformation matrix respectively, For collection With collection The matching confidence factor, To determine whether two points are matching point pairs, the position error threshold is: is the sum of the position errors of the matching point pairs, and there is a position error ,N is the number of matching point pairs, N≥4, For collection With collection The confidence probability that matches is =1; S3.3 The rotation and translation transformation matrix Calculate the matching error of feature points in other points under the rotation transformation To adjust the region size threshold of the point set classification process and update the self-attention matrix in is the initial threshold range, is the updated threshold range; S3.4 Repeat steps S3.2-S3.3 until the iterative process in the cross-attention matrix does not cause probability changes, and obtain the matching result with the highest probability as the optimal matching solution.

7. The cross-scale data fusion method based on feature regions according to claim 1, characterized in that: It also includes verification of the registration accuracy by comparing the registration results of the ICP algorithm and SIFT algorithm with the results of the ISR algorithm.

8. A cross-scale data fusion device based on feature regions, characterized in that: include: A preprocessing module is used to preprocess the atomic force probe scanning data and vertical scanning white light interferometry data, determine the reference surface in the measurement data, and correct the errors of different data sources; A feature extraction module is used to extract feature region descriptors based on the preprocessed data using an iterative similarity region algorithm. The feature region descriptors include position, region center point, descriptor, range size, boundary, and feature parameters. The data registration module is used to register two datasets using an attention-based alignment mechanism combined with extracted feature region descriptors; The data fusion module is used to fuse the vertical scanning white light interferometry data and the atomic force probe scanning data based on the registration results. After aligning the height reference, the vertical scanning white light interferometry measurement data of the overlapping parts is replaced by the atomic force probe scanning measurement data. Linear interpolation and smoothing are performed on the edges to complete the fusion of the atomic force probe scanning data and the vertical scanning white light interferometry data. The attention-based alignment mechanism is combined with the extracted feature region descriptors to achieve the registration of the two datasets, specifically including: S3.1 performs descriptor classification on the feature extraction results of vertical scanning white light interferometry and atomic force probe scanning respectively to obtain the self-attention matrix and feature point set; S3.2 Cross-matching the feature point sets to obtain a transformation matrix. Different matching schemes calculate confidence factors based on the number of matching point pairs and position errors to obtain a cross-attention matrix with matching probability information. The cross-attention matrix includes rotation transformation matrix items and translation transformation matrix items. S3.3 uses the rotation transformation matrix and the translation transformation matrix to calculate the matching error of feature points in other feature point sets under the rotation transformation to adjust the region size threshold of the feature point set classification process and update the self-attention matrix; S3.4 Repeat steps S3.2-S3.3 until the iterative process in the cross-attention matrix does not cause probability changes, and obtain the matching result with the highest probability as the optimal matching solution.

9. A cross-scale data fusion device based on feature regions, characterized in that: The method comprises at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit executes the steps of the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: It stores a computer program that can be executed by a cross-scale data fusion device based on feature regions. When the computer program runs on the cross-scale data fusion device based on feature regions, the cross-scale data fusion device based on feature regions executes the steps of the method according to any one of claims 1 to 7.

Images