Cross-scale data fusion method and device based on feature region, equipment and storage medium

Abstract

The invention discloses a cross-scale data fusion method based on a feature region, and the method comprises the steps: S1, carrying out the preprocessing of AFM data and WLI data, determining a reference surface in measurement data, and correcting the errors of different data sources; s2, extracting a feature region descriptor by adopting an iterative similarity region algorithm; s3, realizing registration of the two data sets by utilizing an attention-based alignment mechanism and combining the extracted feature region descriptors; and S4, on the basis of a registration result, fusing WLI and AFM data, after aligning a height reference, replacing WLI measurement data of a coincident part with AFM measurement data, and carrying out linear interpolation smoothing processing at an edge to complete data fusion. The invention further discloses a cross-scale data fusion device based on the feature region, corresponding equipment and a storage medium. According to the cross-scale data fusion method based on the feature region, the problem of cross-scale data registration precision can be solved, and the accuracy and efficiency of micro-nano structure measurement are improved.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional measurement of microscopic surfaces, and more specifically, to a method, device, equipment and storage medium for cross-scale data fusion based on feature regions. Background Art

[0002] Nanoscale and microscale structures are ubiquitous in engineering applications, so developing their measurement techniques is crucial. Multi-sensor data fusion is a promising approach to expand the scope of measurement techniques and improve measurement resolution and efficiency by combining the advantages of different technologies.

[0003] At present, there have been many advances in the research of multi-sensor data fusion methods. However, for data sets with large resolution differences, such as when the resolution difference is more than ten times, existing data fusion methods are challenging in effectively extracting matching benchmarks for cross-scale structures. For example, the lateral resolution difference between atomic force probe scanning (AFM) data and vertical scanning white light interferometry (WLI) data ranges from 1:10 to 1:600. The existing cross-scale methods cannot be used for effective fusion, and the registration accuracy is also limited by the resolution difference. There is an urgent need for a method to achieve effective fusion between cross-scale data (such as AFM and WLI) with large lateral resolution differences. Summary of the invention

[0004] In response to at least one defect or improvement need in the prior art, the present invention provides a cross-scale data fusion method, device, equipment and storage medium based on feature regions, which can solve at least one of the problems existing in the above-mentioned background technology.

[0005] To achieve the above object, according to a first aspect of the present invention, a cross-scale data fusion method based on feature regions is provided, the method comprising:

[0006] S1 pre-processes AFM data and WLI data, determines the reference surface in the measurement data, and corrects the errors of different data sources;

[0007] S2 uses an iterative similarity region algorithm to extract feature region descriptors based on preprocessed data. The feature region descriptors include position, region center point, descriptor, range size, boundary and feature parameters.

[0008] S3 uses an attention-based alignment mechanism combined with extracted feature region descriptors to achieve registration of the two datasets;

[0009] Based on the registration results, S4 fuses the WLI and AFM data. After aligning the height reference, the AFM measurement data replaces the WLI measurement data of the overlapping part, and linear interpolation and smoothing are performed on the edges to complete the fusion of AFM data and WLI data.

[0010] Furthermore, in the above-mentioned cross-scale data fusion method based on feature regions, the correction of errors of different data sources specifically includes:

[0011] Bat-wing effect suppression is performed on WLI data to reduce step edge noise caused by diffraction effects in white light interferometers;

[0012] The tip model was reconstructed using scanning electron microscopy and the tip was deconvoluted on the AFM data to reduce the measurement errors caused by the tip shape and slope.

[0013] Furthermore, in the above-mentioned cross-scale data fusion method based on feature regions, the extracting feature region descriptors specifically includes:

[0014] S2.1 Generate the parameter matrix to be divided, initialize the clustering object as the height matrix Z and its maximum number of clusters Nmax;

[0015] S2.2 select n cluster centers, where n∈[1,Nmax];

[0016] S2.3 Assign each data point to the nearest cluster center

[0017]

[0018] Where x is the data point, z i is the i-th cluster center when performing high-parameter clustering;

[0019] S2.4 Recalculate the center of each cluster

[0020]

[0021] Among them, M i is the set of data points of the ith cluster, |M i | is the number of data points in the set;

[0022] S2.5 Repeat steps S2.3-S2.4 until the preset termination condition is met, and obtain the error loss square sum corresponding to the current number of cluster centers n

[0023]

[0024] S2.6 Repeat steps S2.2-S2.5 to obtain SSE nThe curve is used to calculate the SSE decay rate to obtain the optimal number of clusters N, and the connected domain analysis is performed on the clustering results to achieve data set division;

[0025] S2.7 uses RANSAC fitting to extract feature regions from the divided data set, extracts the feature region click set corresponding to the current parameters to generate a feature region descriptor, and repeats steps S2.2-S2.5 for the remaining point sets with a new parameter matrix.

[0026] Furthermore, in the above-mentioned cross-scale data fusion method based on feature regions, the generating of the parameter matrix to be divided specifically includes:

[0027] Analyze the parameter matrices of height, gradient and curvature in turn, and output the measurement results in the form of height matrix z;

[0028] If the region is non-planar, the region is divided by a gradient matrix, and the gradient matrix and curvature are calculated by a floating interval.

[0029] Furthermore, the above-mentioned cross-scale data fusion method based on feature regions performs initial feature extraction on identifiable microstructure regions as primary feature region extraction; and performs secondary feature region extraction on regions that meet preset conditions in the results of the primary feature region extraction to obtain detail regions.

[0030] Furthermore, the above-mentioned cross-scale data fusion method based on feature regions, using the attention-based alignment mechanism in combination with the extracted feature region descriptors to achieve the registration of two data sets, specifically includes:

[0031] S3.1 Describe the subclasses of the feature extraction results of WLI and AFM respectively to obtain the self-attention matrix C s , A s ,have

[0032] S i ={y|f y =F i ,g y ∈[G i ,G i+1 )}

[0033] Among them, S i is a feature point set, y is a feature point, f y Characteristic parameter, g y is the feature area size, and the value of the mth row and nth column in the self-attention matrix represents the probability that the set containing feature point m also contains feature point n;

[0034] S3.2 cross-matches the point sets to obtain the transformation matrix. Different matching schemes calculate the confidence factor based on the number of matching point pairs and the position error to obtain the cross-attention matrix with matching probability information.

[0035]

[0036] in Represents the set C si With set A sj Match, O Cm , O An The set C si The position coordinates of the mth feature region in the set A sj The position coordinates of the nth feature region in R ij , T ij are the rotation transformation matrix and the translation transformation matrix respectively, ξ ij For set C si With set A sj The matching confidence factor, d th To determine whether two points are matching point pairs, the position error threshold is ∑d, which is the sum of the position errors of the matching point pairs. <d th ,N is the number of matching point pairs, N≥4,P ij For set C si With set A sj The matching confidence probability is ∑P=1;

[0037] S3.3 The rotation and translation transformation matrix R ij , T ij Calculate the matching error d of feature points in other point sets under the rotation transformation to adjust the area size threshold of the point set classification process and update the self-attention matrix

[0038]

[0039] Among them [G i ,G i+1 ) is the initial threshold range, [G i ′,G i+1 ′) is the updated threshold range;

[0040] S3.4 Repeat steps S3.2-S3.3 until the iterative process in the cross-attention matrix does not cause probability changes, and obtain the matching result with the highest probability as the optimal matching solution.

[0041] Furthermore, the above-mentioned cross-scale data fusion method based on feature regions also includes verifying the registration accuracy by comparing the registration results of the ICP algorithm and the SIFT algorithm with the results of the ISR algorithm.

[0042] According to a second aspect of the present invention, there is also provided a cross-scale data fusion device based on feature regions, comprising:

[0043] A preprocessing module is used to preprocess AFM data and WLI data, determine the reference surface in the measurement data, and correct the errors of different data sources;

[0044] A feature extraction module, for extracting a feature region descriptor based on the preprocessed data by using an iterative similarity region algorithm, wherein the feature region descriptor includes a position, a region center point, a descriptor, a range size, a boundary, and feature parameters;

[0045] The data registration module is used to register two datasets using an attention-based alignment mechanism combined with the extracted feature region descriptors;

[0046] The data fusion module is used to fuse the WLI and AFM data based on the registration results. After aligning the height reference, the AFM measurement data is used to replace the WLI measurement data of the overlapping part, and linear interpolation and smoothing are performed on the edges to complete the fusion of the AFM data and the WLI data.

[0047] According to the third aspect of the present invention, there is also provided a cross-scale data fusion device based on feature regions, which includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit executes the steps of any one of the above-mentioned methods.

[0048] According to a fourth aspect of the present invention, a storage medium is also provided, which stores a computer program that can be executed by a cross-scale data fusion device based on feature regions. When the computer program runs on the cross-scale data fusion device based on feature regions, the cross-scale data fusion device based on feature regions executes the steps of any one of the above-mentioned methods.

[0049] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0050] The cross-scale data fusion method based on feature regions provided by the present invention accurately fuses high-resolution AFM data and large-scale WLI data through an iterative similarity region algorithm and an attention-based alignment mechanism, thereby solving the problem of cross-scale data registration accuracy and improving the accuracy and efficiency of micro-nanostructure measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 A flowchart of a cross-scale data fusion method based on feature regions provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0054] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0055] Figure 1 A schematic diagram of a cross-scale data fusion method based on feature regions provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, a cross-scale data fusion method based on feature regions provided in an embodiment of the present application includes:

[0056] S1 pre-processes AFM data and WLI data, determines the reference surface in the measurement data, and corrects the errors of different data sources;

[0057] S2 uses an iterative similarity region algorithm to extract feature region descriptors based on preprocessed data. The feature region descriptors include position, region center point, descriptor, range size, boundary and feature parameters.

[0058] S3 uses an attention-based alignment mechanism combined with extracted feature region descriptors to achieve registration of the two datasets;

[0059] Based on the registration results, S4 fuses the WLI and AFM data. After aligning the height reference, the AFM measurement data replaces the WLI measurement data of the overlapping part, and linear interpolation and smoothing are performed on the edges to complete the fusion of AFM data and WLI data.

[0060] Specifically, the atomic force probe scanning (AFM) data set and the vertical scanning white light interferometry (WLI) data set are both point cloud data. The vertical resolution of WLI and AFM is similar, but the resolution span in the horizontal direction is huge. Generally speaking, the horizontal resolution of the atomic force microscope depends on the scanning step size, ranging from 1nm to 50nm, but due to the limited size of the probe tip, the measurement result will be affected by the tip convolution effect, and WLI is limited by the diffraction limit, and the horizontal resolution is generally less than 1μm. In addition, in the step edge area, WLI measurement is easily affected by the "bat wing effect" and produces noisy data.

[0061] In the data preprocessing stage, the reference surface in the measurement data is first determined, and the surface is flattened using the least squares plane method to correct the errors of different data sources. The bat wing effect is suppressed on the WLI data to reduce the step edge noise caused by the diffraction effect. The tip deconvolution is performed on the AFM data using a scanning electron microscope (SEM) to reconstruct the tip model to reduce the measurement error caused by the tip shape and slope.

[0062] Since the accuracy of feature region division of WLI, i.e. low-resolution data, is affected by the low resolution of the data source, the matching accuracy of cross-scale data is limited. Therefore, the feature region is extracted to generate a descriptor of the feature region instead of the feature point extraction in the traditional method, and the feature regions are classified and the classification threshold is adjusted in combination with the position matching error to obtain the rotation and translation relationship between WLI and AFM data.

[0063] Based on the preprocessed data, the Iterative Similarity Region (ISR) algorithm is used to extract feature region descriptors from the data. The extraction of feature region descriptors involves K-means clustering and RANSAC algorithm optimization to improve the accuracy of feature region extraction. Feature region descriptors include location, region center point, descriptor, range size, boundary and feature parameters such as height, gradient and curvature.

[0064] The AFM region is precisely located in the WLI coordinate system using an attention-based alignment mechanism, which includes the concepts of cross-attention and self-attention, and optimizes the registration accuracy between cross-scale data by alternately iteratively eliminating the influence of noise regions on feature matching.

[0065] By combining the feature region descriptor with the attention mechanism, the registration error caused by insufficient accuracy in feature region division of low-resolution data can be effectively reduced.

[0066] Based on the registration results, the WLI and AFM data are fused. Since the accuracy of AFM measurement data is much greater than that of WLI measurement data, the fusion process of the overlapping area is to align the height reference first, and then replace the WLI measurement data of the overlapping part with the AFM measurement data. The interface between AFM and WLI data is uniformly aligned to make the final fused data set smoother.

[0067] The cross-scale data fusion method based on feature regions provided in the embodiment of the present application accurately fuses high-resolution AFM data and large-scale WLI data through an iterative similarity region algorithm and an attention-based alignment mechanism, thereby solving the problem of cross-scale data alignment accuracy and improving the accuracy and efficiency of micro-nanostructure measurement.

[0068] Optionally, in the cross-scale data fusion method based on feature regions provided in an embodiment of the present application, the correction of errors of different data sources specifically includes:

[0069] Bat-wing effect suppression is performed on WLI data to reduce step edge noise caused by diffraction effects in white light interferometers;

[0070] The tip model was reconstructed using scanning electron microscopy and the tip was deconvoluted on the AFM data to reduce the measurement errors caused by the tip shape and slope.

[0071] Specifically, the bat-wing effect is a phenomenon encountered in white light scanning interferometry technology, mainly caused by the diffraction effect. When the measured step height is less than the coherence length of the light source, an artifact shaped like a bat wing will appear on the edge of the step. This phenomenon is called the bat-wing effect. The bat-wing effect can be eliminated or compensated by improving software or hardware, such as adjusting system hardware parameters and filtering and correcting the three-dimensional morphology obtained by demodulating the coherent signal, to improve the accuracy of the measurement results.

[0072] The tip model is reconstructed using a scanning electron microscope to measure the morphology of the atomic force probe tip. The tip model is used to perform tip deconvolution on the atomic force microscope measurement data to reduce the measurement error caused by the tip shape and slope.

[0073] Optionally, in the cross-scale data fusion method based on feature regions provided in an embodiment of the present application, extracting feature region descriptors specifically includes:

[0074] S2.1 Generate the parameter matrix to be divided, initialize the clustering object as the height matrix Z and its maximum number of clusters Nmax;

[0075] S2.2 select n cluster centers, where n∈[1,Nmax];

[0076] S2.3 Assign each data point to the nearest cluster center

[0077]

[0078] Where x is the data point, z i is the i-th cluster center when performing high-parameter clustering;

[0079] S2.4 Recalculate the center of each cluster

[0080]

[0081] Among them, M i is the set of data points of the ith cluster, |M i | is the number of data points in the set;

[0082] S2.5 Repeat steps S2.3-S2.4 until the preset termination condition is met, and obtain the error loss square sum corresponding to the current number of cluster centers n

[0083]

[0084] S2.6 Repeat steps S2.2-S2.5 to obtain SSE n The curve is used to calculate the SSE decay rate to obtain the optimal number of clusters N, and the connected domain analysis is performed on the clustering results to achieve data set division;

[0085] S2.7 uses RANSAC fitting to extract feature regions from the divided data set, extracts the feature region click set corresponding to the current parameters to generate a feature region descriptor, and repeats steps S2.2-S2.5 for the remaining point sets with a new parameter matrix.

[0086] Specifically, a multidimensional clustering method of K-means clustering is used to segment the characteristic regions in the WLI and AFM measurement data, thereby laying the foundation for data alignment. K-means clustering is a classic unsupervised learning algorithm. The embodiment of the present application adopts a K-means clustering optimization method for surface metrology data, which automatically determines the clustering dimension and the number of clusters by analyzing the decay rate of the surface fitting error. In addition, the RANSAC algorithm is also used to limit the clustering error in the characteristic region, thereby ensuring more accurate segmentation.

[0087] Generate the parameter matrix to be divided, analyze the three parameter matrices of height, gradient and curvature in turn, and output the measurement results in the form of height matrix Z. Initialize the clustering object as the height matrix Z and its maximum number of clusters Nmax. The maximum number of clusters can be selected as a larger value. The optimal number of clusters and the corresponding clustering results can be automatically obtained through the subsequent analysis process.

[0088] Extract the feature area click set corresponding to the current parameters to generate the feature area descriptor as shown in Table 1.

[0089] Table 1: Elements contained in the feature region descriptor

[0090]

[0091] Optionally, in the cross-scale data fusion method based on feature regions provided in an embodiment of the present application, generating a parameter matrix to be divided specifically includes:

[0092] Analyze the parameter matrices of height, gradient and curvature in turn, and output the measurement results in the form of height matrix z;

[0093] If the region is non-planar, the region is divided by a gradient matrix, and the gradient matrix and curvature are calculated by a floating interval.

[0094] Specifically, based on the prior knowledge of surface metrology, feature types are divided into four categories: plane, inclined surface, spherical surface and irregular surface, and are ranked according to their likelihood of occurrence. Unclassified areas are designated as noise points. The feature extraction process uses a hierarchy of three matrices: height (Z), gradient (D) and curvature (Q) to classify surface features. The accuracy of the classification is evaluated using the surface type error, which helps to refine the division of feature areas. Areas that cannot be classified as planes, inclined surfaces or spherical surfaces are marked as irregular surface areas or local noise points based on their connection domains.

[0095] If there are some areas that cannot be classified as planes, these areas can be divided using the gradient matrix D. The gradient matrix D and the curvature matrix Q can be calculated using a floating interval. For example, the two-dimensional gradient matrix D and the four-neighbor curvature matrix Q can be calculated using the following formula.

[0096]

[0097] where r(i,j) is the radius of the circle fitting point Z(i,j) and its four neighbors.

[0098] Optionally, the cross-scale data fusion method based on feature regions provided in the embodiment of the present application performs initial feature extraction on identifiable microstructure regions as primary feature region extraction; and performs secondary feature region extraction on regions that meet preset conditions in the results of the primary feature region extraction to obtain detail regions.

[0099] Specifically, the feature extraction process can accurately delineate the microstructural regions in the WLI data. In cross-scale measurements, the initial feature extraction can identify the microstructural regions, which is called the first-level feature region extraction. Then, the larger areas identified in the first-level results are subjected to the second-level extraction to extract the detail regions. The two-level extraction process is similar, but the second-level extraction uses a stricter RANSAC fitting threshold to refine the detail regions in the WLI data. This two-level extraction process lays the foundation for aligning the WLI and AFM data.

[0100] Optionally, the cross-scale data fusion method based on feature regions provided in the embodiment of the present application, wherein the registration of two data sets is achieved by utilizing an attention-based alignment mechanism in combination with extracted feature region descriptors, specifically includes:

[0101] S3.1 Describe the subclasses of the feature extraction results of WLI and AFM respectively to obtain the self-attention matrix C s , A s ,have

[0102] S i ={y|f y =F i ,g y ∈[G i ,G i+1 )}

[0103] Among them, S i is a feature point set, y is a feature point, f y Characteristic parameter, g y is the feature area size, and the value of the mth row and nth column in the self-attention matrix represents the probability that the set containing feature point m also contains feature point n;

[0104] S3.2 cross-matches the point sets to obtain the transformation matrix. Different matching schemes calculate the confidence factor based on the number of matching point pairs and the position error to obtain the cross-attention matrix with matching probability information.

[0105]

[0106] in Represents the set C si With set A sj Match, O Cm , O An The set C si The position coordinates of the mth feature region in the set A sj The position coordinates of the nth feature region in R ij , T ij are the rotation transformation matrix and the translation transformation matrix respectively, ξ ij For set C siWith set A sj The matching confidence factor, d th To determine whether two points are matching point pairs, the position error threshold is ∑d, which is the sum of the position errors of the matching point pairs. <d th ,N is the number of matching point pairs, N≥4,P ij For set C si With set A sj The matching confidence probability is ∑P=1;

[0107] S3.3 The rotation and translation transformation matrix R ij , T ij Calculate the matching error d of feature points in other point sets under the rotation transformation to adjust the area size threshold of the point set classification process and update the self-attention matrix

[0108]

[0109] Among them [G i ,G i+1 ) is the initial threshold range, [G i ′,G i+1 ′) is the updated threshold range;

[0110] S3.4 Repeat steps S3.2-S3.3 until the iterative process in the cross-attention matrix does not cause probability changes, and obtain the matching result with the highest probability as the optimal matching solution.

[0111] Specifically, the concepts of cross attention and self attention are proposed in the superglue method based on graph convolutional neural network. Considering that the arrangement of feature areas in surface metrology data has structural characteristics, the cross matching matrix and the self matching matrix are introduced in the feature matching process for alternating iteration to eliminate the influence of noise areas on feature matching. The feature descriptor matching and position matching processes are coupled to optimize the registration accuracy between cross-scale data through mutual iteration.

[0112] Optionally, the feature region-based cross-scale data fusion method provided in the embodiment of the present application further includes verifying the registration accuracy by comparing the registration results of the ICP algorithm and the SIFT algorithm with the results of the ISR algorithm.

[0113] The embodiment of the present application proposes a new cross-scale data fusion method based on feature region alignment, which fuses large-scale white light interferometer (WLI) data with high-resolution atomic force microscope (AFM) data to accurately measure micro-nano composite structures of different scales. The core of this method is the proposed ISR algorithm, which optimizes the traditional data fusion process by replacing feature point descriptors with feature region descriptors, ensuring the accuracy of cross-scale registration.

[0114] The ISR algorithm uses an optimized K-means clustering method to extract feature regions from WLI and AFM data, generate region descriptors, and use an attention mechanism for matching. This process iteratively alternates between self-matching and cross-matching to refine the registration and compensate for errors caused by resolution differences. For overlapping areas after registration, the higher-resolution AFM data will replace the WLI data, and linear interpolation smoothing will be applied at the edges to ensure seamless fusion.

[0115] The ISR method performs well in processing cross-scale data with significant resolution differences, providing consistent feature region extraction at a uniform physical scale. In addition, the ISR algorithm shows superior registration accuracy even when ICP is optimized using manually selected coarse alignment points.

[0116] The embodiment of the present application also provides a cross-scale data fusion device based on feature regions, comprising:

[0117] A preprocessing module is used to preprocess AFM data and WLI data, determine the reference surface in the measurement data, and correct the errors of different data sources;

[0118] A feature extraction module, for extracting a feature region descriptor based on the preprocessed data by using an iterative similarity region algorithm, wherein the feature region descriptor includes a position, a region center point, a descriptor, a range size, a boundary, and feature parameters;

[0119] The data registration module is used to register two datasets using an attention-based alignment mechanism combined with the extracted feature region descriptors;

[0120] The data fusion module is used to fuse the WLI and AFM data based on the registration results. After aligning the height reference, the AFM measurement data is used to replace the WLI measurement data of the overlapping part, and linear interpolation and smoothing are performed on the edges to complete the fusion of the AFM data and the WLI data.

[0121] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0122] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0123] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0124] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0125] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0126] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0127] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program code.

[0128] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0129] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

[0130] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A cross-scale data fusion method based on feature regions, characterized in that: include: S1 pre-processes the atomic force probe scanning data and vertical scanning white light interferometry data, determines the reference surface in the measurement data, and corrects the errors of different data sources; S2 uses an iterative similarity region algorithm to extract feature region descriptors based on preprocessed data. The feature region descriptors include position, region center point, descriptor, range size, boundary and feature parameters. S3 uses an attention-based alignment mechanism combined with extracted feature region descriptors to achieve registration of the two datasets; Based on the registration results, S4 fuses the vertical scanning white light interferometry and atomic force probe scanning data. After aligning the height reference, the atomic force probe scanning measurement data replaces the vertical scanning white light interferometry measurement data of the overlapping part, and linear interpolation smoothing is performed on the edges to complete the fusion of the atomic force probe scanning data and the vertical scanning white light interferometry data.

2. The cross-scale data fusion method based on feature regions according to claim 1, characterized in that: The error correction of different data sources specifically includes: Suppress the bat-wing effect of vertical scanning white light interferometer data to reduce the step edge noise caused by the diffraction effect in the white light interferometer; The tip model was reconstructed using a scanning electron microscope and the tip was deconvoluted on the atomic force probe scanning data to reduce the measurement error caused by the tip shape and slope.

3. The cross-scale data fusion method based on feature regions according to claim 1, characterized in that: The extracting feature region descriptor specifically includes: S2.1 Generate the parameter matrix to be divided, initialize the clustering object as the height matrix Z and its maximum number of clusters Nmax; S2.2 select n cluster centers, where n∈[1,Nmax]; S2.3 Assign each data point to the nearest cluster center Where x is the data point, z i is the i-th cluster center when performing high-parameter clustering; S2.4 Recalculate the center of each cluster Among them, M i is the set of data points of the ith cluster, |M i | is the number of data points in the set; S2.5 Repeat steps S2.3-S2.4 until the preset termination condition is met, and obtain the error loss square sum corresponding to the current number of cluster centers n S2.6 Repeat steps S2.2-S2.5 to obtain SSE n The curve is used to calculate the SSE decay rate to obtain the optimal number of clusters N, and the connected domain analysis is performed on the clustering results to achieve data set division; S2.7 uses RANSAC fitting to extract feature regions from the divided data set, extracts the feature region click set corresponding to the current parameters to generate a feature region descriptor, and repeats steps S2.2-S2.5 for the remaining point sets with a new parameter matrix.

4. The cross-scale data fusion method based on feature regions according to claim 3, characterized in that: The generating of the parameter matrix to be divided specifically includes: Analyze the parameter matrices of height, gradient and curvature in turn, and output the measurement results in the form of height matrix z; If the region is non-planar, the region is divided by a gradient matrix, and the gradient matrix and curvature are calculated by a floating interval.

5. The cross-scale data fusion method based on feature regions according to claim 3, characterized in that: Initial feature extraction is performed on the identifiable microstructure area as the primary feature area extraction; secondary feature area extraction is performed on the area that meets the preset conditions in the results of the primary feature area extraction to obtain the detail area.

6. The cross-scale data fusion method based on feature regions according to claim 3, characterized in that: The attention-based alignment mechanism is combined with the extracted feature region descriptors to achieve the registration of the two data sets, specifically including: S3.1 classifies the feature extraction results of vertical scanning white light interferometry and atomic force probe scanning respectively, and obtains the self-attention matrix C s , A s ,have S i ={y|f y =F i ,g y ∈[G i ,G i+1 )} Among them, S i is a feature point set, y is a feature point, f y Characteristic parameter, g y is the feature area size, and the value of the mth row and nth column in the self-attention matrix represents the probability that the set containing feature point m also contains feature point n; S3.2 cross-matches the point sets to obtain the transformation matrix. Different matching schemes calculate the confidence factor based on the number of matching point pairs and the position error to obtain the cross-attention matrix with matching probability information. in Represents the set C si With set A sj Match, O Cm , O An The set C si The position coordinates of the mth feature region in the set A sj The position coordinates of the nth feature region in R ij , T ij are the rotation transformation matrix and the translation transformation matrix respectively, ξ ij For set C si With set A sj The matching confidence factor, d th To determine whether two points are matching point pairs, the position error threshold is ∑d, which is the sum of the position errors of the matching point pairs. <d th , N is the number of matching point pairs, N ≥ 4, P ij For set C si With set A sj The matching confidence probability is ∑P=1; S3.3 The rotation and translation transformation matrix R ij , T ij Calculate the matching error d of feature points in other point sets under the rotation transformation to adjust the area size threshold of the point set classification process and update the self-attention matrix Among them [G i , G i+1 ) is the initial threshold range, [G i ′,G i+1 ′) is the updated threshold range; S3.4 Repeat steps S3.2-S3.3 until the iterative process in the cross-attention matrix does not cause probability changes, and obtain the matching result with the highest probability as the optimal matching solution.

7. The cross-scale data fusion method based on feature regions according to claim 1, characterized in that: It also includes verification of the registration accuracy by comparing the registration results of the ICP algorithm and the SIFT algorithm with the results of the ISR algorithm.

8. A cross-scale data fusion device based on feature regions, characterized in that: include: A preprocessing module is used to preprocess the atomic force probe scanning data and vertical scanning white light interferometry data, determine the reference surface in the measurement data, and correct the errors of different data sources; A feature extraction module, for extracting a feature region descriptor based on the preprocessed data by using an iterative similarity region algorithm, wherein the feature region descriptor includes a position, a region center point, a descriptor, a range size, a boundary, and feature parameters; The data registration module is used to register two datasets using an attention-based alignment mechanism combined with the extracted feature region descriptors; The data fusion module is used to fuse the vertical scanning white light interferometry and atomic force probe scanning data based on the alignment results. After aligning the height reference, the atomic force probe scanning measurement data replaces the vertical scanning white light interferometry measurement data of the overlapping part, and linear interpolation smoothing is performed on the edges to complete the fusion of the atomic force probe scanning data and the vertical scanning white light interferometry data.

9. A cross-scale data fusion device based on feature regions, characterized in that: The method comprises at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit executes the steps of the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that: It stores a computer program that can be executed by a cross-scale data fusion device based on feature regions. When the computer program runs on the cross-scale data fusion device based on feature regions, the cross-scale data fusion device based on feature regions executes the steps of the method described in any one of claims 1 to 7.

Images