Three-dimensional surface topography rapid splicing method based on improved ORB

Abstract

The invention discloses a three-dimensional surface topography rapid splicing method based on an improved ORB. The method comprises the following steps: determining an overlapping region of a micro-nano surface structure to be spliced based on a block matching method; extracting an initial feature point set based on a FAST algorithm, calculating Harris corner response values of the feature points after non-maximum suppression processing, and performing screening according to the response values; describing the plurality of to-be-matched feature points with the maximum response values to obtain feature descriptors; based on the feature descriptors of the to-be-matched feature points, rough matching point pairs are obtained, fine matching is conducted on the rough matching point pairs through a decoupling type RANSAC algorithm, a three-dimensional space rigid body transformation model is obtained, and micro-nano surface structure splicing fusion is conducted through transformation interpolation and overlapping area data fusion. The invention further discloses a three-dimensional surface topography rapid splicing device based on the improved ORB, corresponding equipment and a storage medium. According to the method, the splicing efficiency and precision can be remarkably improved, and the problem of scale difference of micro-nano surface structure data in different directions is effectively solved.

Description

Technical Field

[0001] This application relates to the field of three-dimensional topography measurement technology, and more specifically, to a fast stitching method for three-dimensional surface topography based on improved ORB. Background Art

[0002] In modern industry and scientific research, the accurate measurement and characterization of the surface topography of objects are of great significance. The surface topography of an object includes surface roughness, waviness, and structure, etc. These characteristics have an important impact on the performance and function of the object in fields such as engineering, biomedicine, and materials science. With the development of technology, non-contact three-dimensional topography measurement technology has gradually become the mainstream, which has the advantages of fast measurement speed, high accuracy, and no damage to the surface of the measured object. However, due to the field of view limitation of the measurement device, a single measurement often cannot cover a large area of the surface topography. Therefore, stitching technology is needed to combine the data of multiple measurements into a complete three-dimensional topography map.

[0003] Currently, three-dimensional surface topography stitching technology is mainly divided into stitching methods based on auxiliary devices and free stitching methods. Although the stitching method with the help of auxiliary devices has a simple algorithm, the system flexibility is poor and the cost is high. Although the free stitching method is flexible to use, it is prone to cumulative errors. In addition, when dealing with micro-nano structures, the existing stitching methods often have difficulty in balancing stitching efficiency and accuracy due to the complexity of their height information and large amount of data. For example, traditional image stitching methods are only applicable to two-dimensional images and are difficult to effectively handle the stitching of micro-nano surface structure height maps, easily causing height information distortion. Therefore, it is particularly urgent to develop a stitching technology that is efficient, highly accurate, and applicable to micro-nano surface structures. Summary of the Invention

[0004] In view of at least one defect or improvement requirement of the prior art, this application provides a fast stitching method for three-dimensional surface topography based on improved ORB, which can solve at least one of the problems existing in the above background art.

[0005] To achieve the above object, according to the first aspect of this application, a fast stitching method for three-dimensional surface topography based on improved ORB is provided, and the method includes: Performing a horizontal sliding search on the micro-nano surface structure to be stitched based on the block matching method to determine the overlapping area; In the overlapping area, extracting an initial set of feature points based on the FAST algorithm, performing non-maximum suppression processing on the initial feature points to remove redundant feature points, calculating the Harris corner response value of the feature points after non-maximum suppression processing, and screening according to the response value to obtain multiple feature points to be matched with the largest response value; Describing the multiple feature points to be matched with the largest response value to obtain feature descriptors; Based on the feature descriptors of the feature points to be matched, the feature points to be matched are filtered through the K-nearest neighbor search algorithm to obtain rough matching point pairs, and the rough matching point pairs are precisely matched through the decoupled RANSAC algorithm to obtain a three-dimensional space rigid body transformation model; Based on the three-dimensional space rigid body transformation model, micro-nano surface structure stitching and fusion are performed through transformation interpolation and overlapping area data fusion.

[0006] Furthermore, for the above-mentioned fast three-dimensional surface topography stitching method based on improved ORB, the initial feature point set is extracted based on the FAST algorithm, and non-maximum suppression processing is performed on the initial feature points to remove redundant feature points, specifically including: For each coordinate point in the overlapping area , whose height value is , taking this point as the center point, select the first number m of coordinate points on a circle with the first radius ; Preset threshold t , if there are consecutive second numbers of coordinate points whose height values differ from the center point by more than the first threshold t , that is

[0007] then take the coordinate point as a candidate feature point; For the candidate feature points , calculate its response value

[0008]

[0009] Compare the response value of the candidate feature point with the response values of all other candidate feature points within the range. If the response value is the local maximum among them, then retain as the initial feature point, otherwise suppress it.

[0010] Furthermore, for the above-mentioned fast three-dimensional surface topography stitching method based on improved ORB, calculating the Harris corner response value and screening according to the response value to obtain multiple feature points to be matched with the largest response value, specifically including: For the height map of the overlapping area , calculate through the Prewitt operator , the gradients

[0011] Among them, represents convolution; For each initial coordinate point, calculate the local autocorrelation matrix

[0012] Among them, is the local window, with a size of ; Calculate the Harris corner response value

[0013]

[0014] Among them, is the determinant of the matrix , is the trace of the matrix , is an empirical constant; Sort the calculated corner response values and retain multiple candidate matching feature points with the largest response values.

[0015] Furthermore, for the above-mentioned fast 3D surface topography stitching method based on improved ORB, the description of the multiple candidate matching feature points with the largest response values to obtain feature descriptors specifically includes: For the current candidate matching feature point, determine a local neighborhood with a size of around it; Generate random coordinate point pairs within the local neighborhood ; For each pair of random coordinate points , compare their values and in the height map of the micro-nano surface structure, and there is

[0016] Connect all the comparison results of the coordinate point pairs in sequence to form a binary vector with a length of , which is the feature descriptor of this feature point .

[0017] Furthermore, for the above-mentioned fast 3D surface topography stitching method based on improved ORB, filtering the candidate matching feature points through the K-nearest neighbor search algorithm based on the feature descriptors of the candidate matching feature points to obtain rough matching point pairs specifically includes: For the set of candidate matching feature points, select the normalized Hamming distance for similarity measurement of features

[0018] Among them, and are 256-bit feature descriptors; For the feature points of the height map , find two nearest neighbor points in the feature points of the height map , . For the feature points and its two nearest neighbor points in , ; Cross-validate and to see if they are a pair of roughly matched points, including verifying that and are nearest neighbors of each other, the normalized Hamming distance of the nearest neighbors is less than the threshold, and the distance ratio of the nearest neighbor and the second nearest neighbor is less than the threshold, that is

[0019] Among them, , , are respectively , , 's feature descriptors.

[0020] Furthermore, for the above-mentioned fast stitching method of three-dimensional surface topography based on improved ORB, the rough-matched point pairs are precisely matched by the decoupled RANSAC algorithm to obtain a three-dimensional space rigid body transformation model, which specifically includes: Randomly select from rough-matched point pairs, calculate the two-dimensional plane rigid body transformation model, use this model to map all point pairs, calculate the Euclidean distance in the horizontal direction as the reprojection error. If the reprojection error is less than the first preset threshold, then consider this point pair as an inlier. Through multiple iterations of calculation, obtain a two-dimensional optimal inlier set that satisfies horizontal alignment, and the number of inliers is ; Randomly select from two-dimensional optimal inliers, calculate the three-dimensional plane rigid body transformation model, use this model to map all point pairs, calculate the Z-axis distance of the corresponding point pairs as the reprojection error. If the reprojection error is less than the second preset threshold, then consider this point pair as an inlier. Through multiple iterations of calculation, obtain a three-dimensional optimal inlier set that satisfies spatial alignment; Based on the three-dimensional optimal interior point set, a three-dimensional space rigid body transformation model is fitted by the least squares method

[0021] .

[0022] Furthermore, for the above-mentioned fast stitching method of three-dimensional surface topography based on improved ORB, based on the three-dimensional space rigid body transformation model, micro-nano surface structure stitching and fusion are performed through transformation interpolation and overlapping region data fusion, specifically including:[[]] Based on the bilinear interpolation principle, the Z-direction height of the point O to be interpolated is calculated as follows:[[]]

[0023] where 、 、 、 are the height values of the 4 mapping points around the point to be interpolated respectively; After completing the interpolation and registration of the data, fusion processing of the overlapping region is performed through weighted fusion. The weighted fusion assigns different weights to the two sets of data according to the distance of the feature points from the edge of the overlapping region. The weight of the overlapping region is:[[]]

[0024] where is the total width of the overlapping region, is the data point distance from the left boundary; The micro-nano surface structure data and are stitched and fused, and the stitching and fusion result is , and there is .

[0025] According to the second aspect of the present application, a fast stitching device for three-dimensional surface topography based on improved ORB is also provided, including:[[]] A block matching module for performing a horizontal sliding search on the micro-nano surface structure to be stitched based on the block matching method to determine the overlapping region; A feature extraction module for extracting an initial feature point set based on the FAST algorithm within the overlapping region, performing non-maximum suppression processing on the initial feature points to remove redundant feature points, calculating the Harris corner response value of the feature points after non-maximum suppression processing, and screening according to the response value to obtain multiple feature points to be matched with the largest response value; A descriptor acquisition module for describing the multiple feature points to be matched with the largest response value to obtain feature descriptors; A matching module, configured to filter the feature points to be matched based on the feature descriptors of the feature points to be matched through the K-nearest neighbor search algorithm, obtain rough matching point pairs, and perform fine matching on the rough matching point pairs through a decoupled RANSAC algorithm to obtain a three-dimensional space rigid body transformation model; A splicing and fusion module, configured to perform micro-nano surface structure splicing and fusion based on the three-dimensional space rigid body transformation model through transformation interpolation and overlapping region data fusion.

[0026] According to the third aspect of the present application, there is also provided a three-dimensional surface topography rapid splicing device based on improved ORB, which includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit is caused to execute the steps of the method described in any one of the above.

[0027] According to the fourth aspect of the present application, there is also provided a storage medium storing a computer program executable by a three-dimensional surface topography rapid splicing device based on improved ORB. When the computer program runs on the three-dimensional surface topography rapid splicing device based on improved ORB, the three-dimensional surface topography rapid splicing device based on improved ORB is caused to execute the steps of the method described in any one of the above.

[0028] Generally speaking, compared with the prior art by the above technical solution conceived by the present application, the following beneficial effects can be achieved: A three-dimensional surface topography rapid splicing method provided by an embodiment of the present application can effectively reduce the number of feature points and improve the quality of feature points by combining the FAST algorithm and the Harris corner detection algorithm for feature extraction, thereby significantly improving the splicing efficiency and accuracy. At the same time, by performing fine matching on the rough matching point pairs through a decoupled RANSAC algorithm, the scale difference problem of micro-nano surface structure data in the horizontal and vertical directions can be solved, further improving the accuracy and reliability of matching. In addition, based on the transformation interpolation and overlapping region data fusion of the three-dimensional space rigid body transformation model, efficient splicing and fusion of the micro-nano surface structure can be realized, ensuring smooth transition and high-precision alignment of the splicing result. By optimizing the feature extraction and fine matching processes, the splicing efficiency and accuracy can be significantly improved, effectively solving the scale difference problem of micro-nano surface structure data in different directions, and realizing efficient and high-precision surface topography splicing. Description of the Drawings

[0029] To more clearly illustrate the technical solutions in the embodiments of the present application, the accompanying drawings required for the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0030] Figure 1 It is a schematic flowchart of a fast stitching method for three-dimensional surface topography based on improved ORB provided by an embodiment of the present application. Detailed implementation manners

[0031] In order to make the objectives, technical solutions and advantages of the present application more clear and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0032] The terms "first", "second", "third", etc. in the specification and claims of the present application and the above accompanying drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0033] Figure 1 It is a schematic flowchart of a fast stitching method for three-dimensional surface topography based on improved ORB provided by an embodiment of the present application. As Figure 1 shown, the fast stitching method for three-dimensional surface topography based on improved ORB provided by the embodiment of the present application includes: S1 Horizontally slide and search the micro-nano surface structure to be stitched based on the block matching method to determine the overlapping area; S2 In the overlapping area, extract the initial feature point set based on the FAST algorithm, perform non-maximum suppression processing on the initial feature points to remove redundant feature points, calculate the Harris corner response value of the feature points after non-maximum suppression processing, and screen according to the response value to obtain multiple feature points to be matched with the largest response value; S3 Describe the multiple feature points to be matched with the largest response value to obtain a feature descriptor; S4 filters the feature points to be matched through the K-nearest neighbor search algorithm based on the feature descriptors of the feature points to be matched, obtains the rough matching point pairs, and performs fine matching on the rough matching point pairs through the decoupled RANSAC algorithm to obtain the three-dimensional space rigid body transformation model; S5 performs micro-nano surface structure stitching and fusion through transformation interpolation and overlapping region data fusion based on the three-dimensional space rigid body transformation model.

[0034] Specifically, the micro-nano surface structure stitching algorithm proposed in this application introduces the ORB algorithm. To ensure high stitching efficiency, the feature extraction and feature matching algorithms are improved according to the characteristics of white light interference data. In the feature extraction stage, the FAST algorithm in ORB is used for feature extraction. If the number of feature points is too large, the algorithm efficiency will decrease. This application proposes a feature extraction algorithm combining FAST and Harris, and filters according to the Harris response value of the feature points extracted by FAST to control the number of feature points and improve the algorithm efficiency. In the fine matching process of feature matching, due to the inconsistent scales of the micro-nano surface structure data obtained by white light interference in the horizontal and vertical directions, the standard RANSAC algorithm is inaccurate and ineffective. This application proposes a decoupled RANSAC algorithm, which decouples the three-dimensional error into the Euclidean distance error in the horizontal direction and the height error in the vertical direction, corresponding to the two stages of two-dimensional horizontal registration and high-precision vertical height registration respectively, and excludes the interference caused by the accuracy differences in the horizontal and vertical directions of the data.

[0035] The micro-nano surface structure stitching based on the improved ORB mainly includes the following steps: determining the overlapping region, feature extraction and description, feature matching, and stitching and fusion. First, for the micro-nano surface structures to be stitched, the overlapping region is initially determined through the block matching method. Then, the overlapping region is subjected to feature extraction by combining FAST and Harris and the detected feature points are described by BRIEF. Then, the preliminary matching point pairs are obtained through rough matching, and the point pairs are screened through the decoupled RANSAC fine matching, and the transformation model is calculated using the final matching point pairs. Finally, the micro-nano surface structure stitching and fusion are realized using the transformation model.

[0036] In one embodiment, the overlapping region can be determined by the block matching method, for example and are the height maps of two adjacent micro-nano surface structures on the left and right, the number of sampling points is , in select a block with a size of The size of the block needs to be reasonably selected to ensure is within the overlapping region of In slide horizontally to search, so that and and The blocks in have the minimum Mean Squared Error (MSE):

[0037] Where and are the points in the blocks and respectively.

[0038] According to and 's positions, the width of the overlapping region can be quickly determined: :

[0039] Where is the distance between the block and 's right boundary, is the distance between the block and 's left boundary, is the width of the block.

[0040] Since only horizontal sliding search is involved, the time complexity of the block matching method for determining the overlapping region is:

[0041] In the case of full search, the time complexity is:

[0042] The computational complexity of horizontal sliding search is only of that of full search, which can significantly reduce the computational amount. Especially when the vertical search range is large, this difference is more significant. For micro-nano surface structures mainly with horizontal displacement, the block matching method of horizontal search is superior to full search block matching.

[0043] A fast 3D surface topography stitching method based on improved ORB provided by an embodiment of the present application extracts features by combining the FAST algorithm and the Harris corner detection algorithm, which can effectively reduce the number of feature points and improve the quality of feature points, thus significantly improving the stitching efficiency and accuracy. At the same time, the decoupled RANSAC algorithm is used to perform fine matching on the coarsely matched point pairs, which can solve the scale difference problem of micro-nano surface structure data in the horizontal and vertical directions, and further improve the accuracy and reliability of matching. In addition, the transformation interpolation and overlapping region data fusion based on the 3D space rigid body transformation model can achieve efficient stitching and fusion of micro-nano surface structures, ensuring smooth transition and high-precision alignment of the stitching results. By optimizing the feature extraction and fine matching processes, the stitching efficiency and accuracy can be significantly improved, effectively solving the scale difference problem of micro-nano surface structure data in different directions, and realizing efficient and high-precision surface topography stitching.

[0044] Optionally, for the fast 3D surface topography stitching method based on improved ORB provided by an embodiment of the present application, the initial feature point set is extracted based on the FAST algorithm, and non-maximum suppression processing is performed on the initial feature points to remove redundant feature points, which specifically includes: For each coordinate point within the overlapping region , whose height value is , with this point as the center point, select the first number m of coordinate points on a circle with the first radius ; Preset threshold t , if there are consecutive second number of coordinate points whose height value difference from the center point is greater than the first threshold t , that is

[0045] then take the coordinate point as a candidate feature point; For the candidate feature point , calculate its response value

[0046]

[0047] Compare the response value of the candidate feature point with the response values of all other candidate feature points within the range. If the response value is the local maximum among them, then retain as an initial feature point, otherwise suppress it.

[0048] Specifically, key features such as corner points, edges, and textures are extracted from the micro-nano surface structure data. Whether the feature points are significant directly affects the subsequent matching and stitching quality. Compared with dense texture and edge features, corner points are fewer in number but contain rich information, which can significantly reduce the computational amount and improve the algorithm efficiency. Corner points generally correspond to local extrema, representing significant changes in the surface structure data. When a point has significant differences with multiple surrounding points in multiple directions, it indicates that the point is at the intersection of multiple edges or texture changes and can be considered a corner point.

[0049] The FAST (Features from Accelerated Segment Test) algorithm is a fast corner detection method based on the numerical comparison of coordinate points. By comparing the numerical differences between the central point and the surrounding coordinate points, it determines whether the point is a corner point. The specific process of the FAST algorithm is as follows: For the coordinate points within the overlapping region of the micro-nano surface structure height map , whose height value is . Taking the point as the center, 16 coordinate points are selected on a circle with a radius of 3, and the corresponding height values are . Set a threshold . If there are consecutive coordinate points that satisfy

[0050] or

[0051] , then the coordinate point is considered a candidate corner point.

[0052] The main parameters of the FAST algorithm are the threshold and the number of consecutive coordinate points in corner point determination, both of which will directly affect the corner point determination. Selecting a larger threshold can, to a certain extent, suppress the influence of noise and improve the stability of corner points, but it will cause some real corner points to be missed. The threshold can be determined according to the root mean square of the micro-nano surface structure height map:

[0053]

[0054] Among them, is a constant, and the empirical range is 0.2 - 0.4. RMS is the root mean square of the overlapping region, , are the width and height of the overlapping region respectively, and is The height value at is the average height of the overlapping area.

[0055] The number of consecutive coordinate points in corner point judgment determines the strictness of FAST corner point determination. A larger can improve the reliability of corner point detection, reduce the overall number of corner points, but will make the algorithm more sensitive to noise, easily suppress the detection of real corner points due to noise, and will also increase the computational complexity and reduce the algorithm efficiency. Taking 9 is a balance between speed and accuracy.

[0056] The FAST algorithm achieves efficient corner point detection through rapid comparison of coordinate point values. However, a corner point itself is a local region feature rather than an isolated point, such as the corners of an object or the intersection points of edges. In actual detection, the coordinate points that meet the corner point determination conditions often gather together, and multiple adjacent corner points will be detected in the same area, resulting in the problem of redundant corner points. Redundant corner points will increase the computational complexity of subsequent matching and will also lead to inaccurate feature positioning, affecting the matching accuracy. To solve the problem of corner point redundancy, non-maximum suppression (NMS) is performed after the FAST algorithm. By retaining the corner point with the largest local response value and suppressing other candidate points in its neighborhood, redundancy is removed, the positioning accuracy of corner points is improved, which is conducive to improving the efficiency and accuracy of subsequent matching. The process of non-maximum suppression is as follows: For candidate feature points , its response value can be expressed as:

[0057] reflects the degree of difference between the candidate feature point and the surrounding coordinate points. The greater the difference, the larger the response value, and the more likely it is to be a real corner point. In the neighborhood centered on the candidate feature point with a size of , compare with the response values of all candidate points within this range. If 's response value is locally the largest, then it is retained as a FAST feature point; otherwise, it is suppressed.

[0058] The neighborhood size of non-maximum suppression will affect the number of feature points. Selecting a larger neighborhood can effectively reduce the number of feature points and suppress the problem of corner point redundancy, but if it is too large, it will cause valid corner points to be mis-suppressed and real feature points to be missed. The neighborhood range of non-maximum suppression should not be too large, generally 3×3 or 5×5.

[0059] The FAST algorithm needs to process each point in the overlapping area, and each point is compared at most 16 times. So the time complexity is:

[0060] Among them, and are the width and height of the overlapping region respectively.

[0061] Optionally, for the three-dimensional surface topography rapid stitching method based on improved ORB provided by the embodiments of the present application, the calculating of the Harris corner response value, screening according to the response value, and obtaining multiple to-be-matched feature points with the largest response value specifically include: For the height map of the overlapping region , calculate in and directions of the gradient

[0062] Among them, represents convolution; For each initial coordinate point, calculate the local autocorrelation matrix

[0063] Among them, is the local window, with a size of ; Calculate the Harris corner response value

[0064]

[0065] Among them, is the determinant of the matrix , is the trace of the matrix , is an empirical constant; Sort the calculated corner response values and retain multiple to-be-matched feature points with the largest response values.

[0066] Specifically, the Harris algorithm is a corner detection algorithm based on the height change in the neighborhood of coordinate points. By monitoring the height change through gradient information, if a certain point causes significant height changes in all directions, then this point is very likely to be a corner.

[0067] The height map of the overlapping region of the micro-nano surface structure to be stitched is , calculate in and directions of the gradient. For each coordinate point in the overlapping region, calculate the local autocorrelation matrix , and after obtaining , calculate the Harris corner response value, where is an empirical constant (usually with a value between 0.04 and 0.06).

[0068] The larger the Harris corner response value, the more significant the gradient change in the , direction, and the greater the likelihood of being a corner. Harris analyzes the height changes in two directions by calculating the autocorrelation matrix of the local window, suppressing the influence of noise. Compared with the FAST algorithm that is vulnerable to noise, the Harris algorithm has higher corner extraction accuracy. The Harris algorithm needs to calculate the gradients of all points in the overlapping region in the , direction, corresponding to . When calculating the autocorrelation matrix, the window summation also needs to be calculated. Therefore, its total time complexity is:

[0069] where , are the width and height of the overlapping region respectively, and is the size of the Harris local window.

[0070] Quickly obtain the set of candidate feature points through the FAST algorithm and non-maximum suppression, calculate the Harris corner response values of the candidate feature points, sort according to the response values, and retain some of the feature points with the largest response values. This method can balance the relationship between calculation efficiency and feature point quality, use the relatively fast FAST algorithm for global calculation, avoid the huge calculation burden brought by directly applying the Harris algorithm to process micro-nano surface structure data, and can also improve the reliability of feature points through the secondary screening of the Harris algorithm.

[0071] The time complexity of the feature extraction algorithm combining FAST and Harris is:

[0072] where , are the width and height of the overlapping region respectively, is a constant ( ), and are the numbers of feature points before and after non-maximum suppression respectively, and is the size of the Harris local window.

[0073] The time complexity of the feature extraction algorithm combining FAST and Harris is slightly higher than that of the FAST algorithm, but it can reduce the number of feature points while ensuring the quality of feature points, greatly improving the efficiency of subsequent matching and the overall efficiency of the algorithm.

[0074] Optionally, for the three-dimensional surface topography rapid stitching method based on improved ORB provided by the embodiments of the present application, the step of describing a plurality of to-be-matched feature points with the largest response values to obtain feature descriptors specifically includes: For the current to-be-matched feature point, determine a local neighborhood with a size of around it; Generate pairs of random coordinate points in the local neighborhood; For each pair of random coordinate points , compare their values and in the height map of the micro-nano surface structure. There is

[0075] Connect the comparison results of all pairs of coordinate points in sequence to form a binary vector with a length of , which is the feature descriptor of this feature point .

[0076] Specifically, the purpose of feature extraction is to identify significant feature points in the micro-nano surface structure. However, these feature points only contain position information and cannot meet the requirements of feature matching. Therefore, a unique descriptor, that is, a feature descriptor, needs to be generated for each detected feature point. Feature descriptors usually exist in the form of numerical vectors, representing the local region structure information around the feature points, so that the feature points have uniqueness. Commonly used feature description methods include gradient-based, binary-based, and histogram-based methods.

[0077] The BRIEF descriptor generates a binary vector through the comparison of a series of random coordinate point pairs to describe the local information around the feature points. It is convenient and efficient without complex calculations. The specific process of obtaining the BRIEF descriptor of the feature point is as follows: For the current feature point, determine a local neighborhood with a size of around it. The size of the neighborhood needs to be weighed according to the specific application scenario, ensuring that enough local information can be captured while avoiding introducing too much noise.

[0078] Generate pairs of random coordinate points: in the local neighborhood. The positions of these coordinate point pairs relative to the central feature point are fixed.

[0079] For each pair of random coordinate points , compare their values and If is greater than , then record the comparison result of this pair of coordinate points as 1; otherwise, record the comparison result as 0:

[0080] Construction of the binary descriptor: Connect the comparison results of all pairs of coordinate points in order to form a binary vector with a length of , which is the BRIEF descriptor of this feature point:

[0081] The longer the BRIEF descriptor length, the higher the discrimination, but the greater the matching calculation amount. Shorter descriptors are faster, but may reduce the matching accuracy. To balance accuracy and efficiency, here take 8, that is, the length of the BRIEF feature descriptor is 256. Correspondingly, the size of the local neighborhood can be set to 31×31.

[0082] Optionally, for the three-dimensional surface topography fast stitching method based on the improved ORB provided in the embodiments of the present application, the feature descriptors based on the feature points to be matched are used to filter the feature points to be matched through the K-nearest neighbor search algorithm to obtain rough matching point pairs, specifically including: For the set of feature points to be matched, the normalized Hamming distance is selected for the similarity measurement of the features

[0083] where and are 256-bit feature descriptors; For the feature points of the height map , find two nearest neighbor points in the feature points of the height map , ; For the feature points and its two nearest neighbor points in , ; Cross-validation and whether they are rough matching point pairs, including verifying that and are the nearest neighbors of each other, the normalized Hamming distance of the nearest neighbors is less than the threshold, and the distance ratio of the nearest neighbor and the second nearest neighbor is less than the threshold, that is

[0084] Among them, , , are respectively , , 's feature descriptors.

[0085] Specifically, after feature extraction, matching points are found through feature matching to establish the corresponding relationship between the regions to be stitched. Feature matching is usually divided into two stages: rough matching and fine matching. Rough matching quickly determines the initial feature matching pairs, providing a basis for subsequent fine matching; in the fine matching stage, geometric constraint conditions are used to optimize the rough matching results, eliminating mis-matched points (outliers) and improving the matching accuracy.

[0086] The BRIEF feature descriptor represents the local information of the feature points. The smaller the difference between the descriptors, the higher the matching degree of the two feature points. For the set of feature points to be matched, the normalized Hamming distance is selected to measure the similarity of the features:

[0087] Among them, , are both 256-bit BRIEF feature descriptors. The smaller the value of and 's matching degree is higher.

[0088] Rough matching uses K-nearest neighbor search and cross-validation, which can quickly and efficiently filter out useless points. For the feature points of the height map , find two nearest neighbor points in the feature points of the height map , . The feature points and its two nearest neighbor points in are , . and are the rough matching point pairs that need to pass cross-validation, that is, and 's nearest neighbors are each other, and at the same time, it is also necessary to satisfy that the normalized Hamming distance of the nearest neighbors is less than the threshold, and the distance ratio of the nearest neighbor and the second nearest neighbor is less than the threshold, that is:

[0089] Among them, , , are respectively , , BRIEF descriptor.

[0090] The rough matching based on K-nearest neighbor search can be divided into constructing KD-Tree, bidirectional K-nearest neighbor search and cross-validation, and the time complexity is:

[0091] Among them, N is the number of feature points. When N takes 300, the time complexity of rough matching is very small.

[0092] Optionally, for the three-dimensional surface topography fast stitching method provided by the embodiments of the present application, the decoupled RANSAC algorithm is used to perform fine matching on the rough matching point pairs to obtain a three-dimensional space rigid body transformation model, which specifically includes: Randomly select from rough matching point pairs, calculate the two-dimensional plane rigid body transformation model, use this model to map all point pairs, calculate the Euclidean distance in the horizontal direction as the reprojection error. If the reprojection error is less than the first preset threshold, then consider this point pair as an inlier. Through multiple iterations of calculation, obtain a two-dimensional optimal inlier set that satisfies horizontal alignment, and the number of inliers is ; Randomly select from two-dimensional optimal inliers, calculate the three-dimensional plane rigid body transformation model, use this model to map all point pairs, calculate the Z-axis distance of the corresponding point pairs as the reprojection error. If the reprojection error is less than the second preset threshold, then consider this point pair as an inlier. Through multiple iterations of calculation, obtain a three-dimensional optimal inlier set that satisfies spatial alignment; Based on the three-dimensional optimal inlier set, fit the three-dimensional space rigid body transformation model by the least squares method

[0093] .

[0094] Specifically, when the number of retained feature points in feature extraction is relatively large, there will be obvious matching errors, with a large spatial distance deviation, and there are also unobvious matching errors as shown in the local enlarged view, which are close to the correct matching but do not conform to global consistency. In order to improve the stitching accuracy, it is necessary to eliminate the mismatched point pairs through fine matching.

[0095] The RANSAC algorithm performs excellently in two-dimensional image matching. However, when applied to three-dimensional matching of micro-nano surface structures, its performance will be affected by the data characteristics of micro-nano surface structures. There are obvious accuracy differences in the vertical and horizontal directions of micro-nano surface structure data. The RANSAC algorithm usually uses the Euclidean distance threshold to eliminate mis-matched points. This single-threshold setting method is difficult to meet the accuracy differences in both directions simultaneously: if a smaller threshold is set to meet the high-precision requirements in the vertical direction, the number of inliers will be insufficient in the horizontal direction due to the lower resolution; if a larger threshold is set, it can meet the matching requirements in the horizontal direction, but it cannot fully utilize the high-precision information in the vertical direction, resulting in difficulty in eliminating mis-matches caused by noise.

[0096] Aiming at the limitations of the standard RANSAC algorithm for white-light interference measurement data, this application proposes a decoupled RANSAC algorithm, which decouples the three-dimensional error into the Euclidean distance error in the horizontal direction and the height error in the vertical direction, corresponding to two stages of two-dimensional horizontal registration and high-precision vertical height registration respectively, excluding the interference caused by the accuracy differences in the horizontal and vertical directions of the data. The implementation process is as follows: From randomly select from the coarse-matched point pairs, without considering the Z-direction height value, and calculate the two-dimensional planar rigid body transformation model. Subsequently, use this model to map all point pairs, calculate the Euclidean distance in the horizontal direction as the reprojection error. If the reprojection error is less than the preset threshold, then regard this point pair as an inlier. Through multiple iterative calculations, obtain the two-dimensional optimal inlier set that meets the horizontal alignment, and the number of inliers is .

[0097] From randomly select from the two-dimensional optimal inliers, calculate the three-dimensional planar rigid body transformation model. Subsequently, use this model to map all point pairs, calculate the Z-axis distance of the corresponding point pairs as the reprojection error. If the reprojection error is less than the preset threshold, then regard this point pair as an inlier. Through multiple iterative calculations, obtain the three-dimensional optimal inlier set that meets the spatial alignment.

[0098] Based on the three-dimensional optimal inlier set, fit the three-dimensional spatial rigid body transformation model by the least squares method :

[0099] Although the decoupled RANSAC algorithm is divided into two stages, the first stage is two-dimensional registration, and the number of samples processed in the high-precision vertical height registration in the second stage is small. Its time complexity is:

[0100] Among them, 、 are the inlier ratios in two matches respectively, , is the size of the randomly selected minimum sample set in two matches, which are 2 and 3 respectively, is the number of rough matching point pairs.

[0101] Optionally, for the fast stitching method of three-dimensional surface topography based on improved ORB provided in the embodiments of the present application, based on the three-dimensional space rigid body transformation model, micro-nano surface structure stitching and fusion are performed through transformation interpolation and overlapping region data fusion, which specifically includes: Based on the bilinear interpolation principle, the Z-direction height of the point O to be interpolated is calculated as follows:

[0102] Wherein, , , , are the height values of 4 mapping points around the point to be interpolated respectively; After completing the interpolation and registration of the data, fusion processing of the overlapping region is performed through weighted fusion. The weighted fusion assigns different weights to the two sets of data according to the distance of the feature points from the edge of the overlapping region. The weight of the overlapping region is:

[0103] Wherein, is the total width of the overlapping region, is the data point distance from the left boundary; The micro-nano surface structure data and are stitched and fused, and the stitching and fusion result is , and there is .

[0104] Specifically, after obtaining the transformation model, the micro-nano surface structure stitching and fusion includes two steps: transformation interpolation and data fusion of the overlapping region. Since there is a deviation between the data of the target region after transformation and the data of the reference region, interpolation needs to be performed first to ensure data alignment before subsequent fusion operations can be carried out. Commonly used interpolation methods include nearest point interpolation and bilinear interpolation.

[0105] Nearest - point interpolation selects the value of the known point closest to the target point as the value of the target point. It is fast, but the interpolation result may have obvious edges and discontinuities, leading to accuracy loss. Bilinear interpolation uses the height values of four mapped points around the point to be interpolated for weighted averaging. The weights are determined by the distances between the mapped coordinates and these four coordinate points. Compared with nearest - point interpolation, bilinear interpolation can produce a smoother transition, but the computational cost will increase. Bilinear interpolation performs linear interpolation in two directions separately and then performs another linear interpolation on the results to obtain the estimated value of the target point. According to the principle of bilinear interpolation, the Z - direction height of the point O to be interpolated is calculated as follows:

[0106] Among them, 、 、 、 are the height values of the 4 mapped points around the point to be interpolated respectively.

[0107] After completing the interpolation registration of the data, fusion processing of the overlapping area is carried out through weighted fusion, which can achieve a smooth transition. Weighted fusion assigns different weights to the two sets of data according to the distance of the data points from the edge of the overlapping area. The weight of the overlapping area is:

[0108] Among them, is the total width of the overlapping area, is the data point the distance from the left boundary.

[0109] Micro - nano surface structure data and are stitched and fused. The stitched and fused result is , If it is one of the points, then there is:

[0110] The stitching accuracy is evaluated by the RMSE (Root mean square error) of the height of the overlapping area. The root mean square error is an effective index to measure the deviation between the stitching result and the reference area. The root mean square error of the height of the overlapping area is used as the evaluation standard for the stitching quality, and its formula is as follows:

[0111] Among them, 、 are the heights of the overlapping area before and after stitching, corresponding to the reference area and the stitched and fused result respectively.

[0112] Optionally, an embodiment of the present application further provides a three-dimensional surface topography rapid stitching device based on improved ORB, including: A block matching module, configured to perform a horizontal sliding search on the micro-nano surface structure to be stitched based on a block matching method to determine the overlapping area; A feature extraction module, configured to extract an initial set of feature points based on the FAST algorithm within the overlapping area, perform non-maximum suppression processing on the initial feature points to remove redundant feature points, calculate the Harris corner response value of the feature points after non-maximum suppression processing, and perform screening according to the response value to obtain multiple feature points to be matched with the largest response value; A descriptor acquisition module, configured to describe the multiple feature points to be matched with the largest response value to obtain a feature descriptor; A matching module, configured to filter the feature points to be matched based on the feature descriptors of the feature points to be matched through the K-nearest neighbor search algorithm to obtain a pair of roughly matched points, and perform fine matching on the pair of roughly matched points through the decoupled RANSAC algorithm to obtain a three-dimensional space rigid body transformation model; A stitching and fusion module, configured to perform micro-nano surface structure stitching and fusion based on the three-dimensional space rigid body transformation model through transformation interpolation and overlapping area data fusion.

[0113] The present application also provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. Among them, the computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, micro drives, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nano-systems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0114] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0115] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0116] In several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some service interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0117] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0118] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0119] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned memory includes: USB flash drive, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0120] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. This program can be stored in a computer-readable memory. The memory can include: flash drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0121] The foregoing are only exemplary embodiments of the present disclosure, and thus cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure shall still fall within the scope covered by the present disclosure. Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not described in the present disclosure. The specification and embodiments are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

[0122] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered that the scope described in this specification is covered.

[0123] It is easy for those skilled in the art to understand that the foregoing is only a preferred embodiment of the present application, and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included within the protection scope of the present application.

Claims

1. A fast stitching method for three-dimensional surface topography based on improved ORB, characterized in that, Including: Performing a lateral sliding search on the micro-nano surface structure to be spliced based on the block matching method to determine the overlapping region; Within the overlapping region, extracting an initial set of feature points based on the FAST algorithm, performing non-maximum suppression processing on the initial feature points to remove redundant feature points, calculating the Harris corner response values of the feature points after non-maximum suppression processing, and screening according to the response values to obtain multiple feature points to be matched with the largest response values; Describing the multiple feature points to be matched with the largest response values to obtain a feature descriptor; Based on the feature descriptors of the feature points to be matched, filtering the feature points to be matched through the K-nearest neighbor search algorithm to obtain a set of rough matching point pairs, and performing fine matching on the set of rough matching point pairs through the decoupled RANSAC algorithm to obtain a three-dimensional space rigid body transformation model; Based on the three-dimensional space rigid body transformation model, performing micro-nano surface structure splicing and fusion through transformation interpolation and overlapping region data fusion.

2. The rapid stitching method for three-dimensional surface topography based on improved ORB according to claim 1, wherein, The extracting of the initial set of feature points based on the FAST algorithm, performing non-maximum suppression processing on the initial feature points to remove redundant feature points specifically includes: For each coordinate point within the overlapping region , its height value is . Taking this point as the center, select a first number m of coordinate points on the circle with the first radius; Preset threshold t If there are a consecutive second number of coordinate points whose height values differ from the center point by more than a first threshold t That is Then the coordinate point is used as a candidate feature point; For candidate feature points , calculate their response values Compare the response value of the candidate feature point with the response values of all other candidate feature points within the range. If the response value is the local maximum among them, then retain as the initial feature point; otherwise, suppress it.

3. The rapid stitching method for three-dimensional surface topography based on improved ORB according to claim 2, wherein, The calculating of the Harris corner response values, screening according to the response values to obtain multiple feature points to be matched with the largest response values specifically includes: Height map of the overlapping region , calculated by the Prewitt operator In , Gradient in the direction Among them, represents convolution; For each initial coordinate point, calculating the local autocorrelation matrix Among them, is a local window with a size of ; Calculate the Harris corner response value Among them, is the determinant of the matrix , is the trace of the matrix , is an empirical constant; Sorting the calculated corner response values and retaining multiple feature points to be matched with the largest response values.

4. The fast stitching method for three-dimensional surface topography based on improved ORB according to claim 3, wherein The describing of the multiple feature points to be matched with the largest response values to obtain a feature descriptor specifically includes: For the current feature points to be matched, a local neighborhood with a size of is determined around them; Generate within the local neighborhood for random coordinate point pairs ; For each pair of random coordinate points , compare their values in the height map of the micro-nano surface structure and , there is Connect all the comparison results of coordinate point pairs in sequence to form a binary vector with a length of , which is the feature descriptor of this feature point 。 5. The rapid stitching method for three-dimensional surface topography based on improved ORB according to claim 4, wherein The filtering of the feature points to be matched through the K-nearest neighbor search algorithm based on the feature descriptors of the feature points to be matched to obtain a set of rough matching point pairs specifically includes: For the set of feature points to be matched, using the normalized Hamming distance to measure the similarity of features Among them, and are 256-bit feature descriptors; For the height map 's feature points , find two nearest neighbor points among the feature points of the height map and ; 's feature points and its two nearest neighbor points in and ;​​ Cross-validation and whether it is a rough matching point pair, including verifying and the nearest neighbors of each other are each other, the normalized Hamming distance of the nearest neighbors is less than the threshold, and the distance ratio of the nearest neighbor and the second nearest neighbor is less than the threshold, that is Among them, , , are respectively , , 's feature descriptors.

6. The rapid stitching method for three-dimensional surface topography based on improved ORB according to claim 5, characterized in that The performing of fine matching on the set of rough matching point pairs through the decoupled RANSAC algorithm to obtain a three-dimensional space rigid body transformation model specifically includes: From Randomly select from the rough matching point pairs, calculate the 2D planar rigid body transformation model, use this model to map all point pairs, calculate the Euclidean distance in the horizontal direction as the reprojection error. If the reprojection error is less than the first preset threshold, then consider this point pair as an inlier. Through multiple iterative calculations, obtain a 2D optimal inlier set that satisfies horizontal alignment, and the number of inliers is ; Randomly select from two-dimensional optimal inliers, calculate the three-dimensional plane rigid body transformation model, use this model to map all point pairs, calculate the Z-axis distance of the corresponding point pairs as the reprojection error. If the reprojection error is less than the second preset threshold, then regard this point pair as an inlier. Through multiple iterative calculations, obtain a set of three-dimensional optimal inliers that satisfy spatial alignment; Based on the three-dimensional optimal inlier set, the three-dimensional rigid body transformation model is fitted by the least squares method 。 7. The fast stitching method for three-dimensional surface topography based on improved ORB as claimed in claim 6, wherein, The performing of micro-nano surface structure splicing and fusion through transformation interpolation and overlapping region data fusion based on the three-dimensional space rigid body transformation model specifically includes: Based on the bilinear interpolation principle, the Z-direction height of the point O to be interpolated is calculated as follows: Among them, , , , are the height values of 4 mapping points around the interpolation point respectively; After completing the interpolation registration of the data, performing fusion processing on the overlapping region through weighted fusion. The weighted fusion assigns different weights to the two sets of data according to the distance of the feature points from the edge of the overlapping region. The weight of the overlapping region is: Among them, is the total width of the overlapping region, is the data point distance from the left boundary; Splice and fuse the micro-nano surface structure data and The splicing and fusion result is There is 。 8. A three-dimensional surface topography rapid stitching device based on improved ORB, characterized in that, Including: A block matching module for performing a lateral sliding search on the micro-nano surface structure to be spliced based on the block matching method to determine the overlapping region; A feature extraction module for extracting an initial set of feature points within the overlapping region based on the FAST algorithm, performing non-maximum suppression processing on the initial feature points to remove redundant feature points, calculating the Harris corner response values of the feature points after non-maximum suppression processing, and screening according to the response values to obtain multiple feature points to be matched with the largest response values; A descriptor acquisition module for describing the multiple feature points to be matched with the largest response values to obtain a feature descriptor; A matching module, configured to filter the feature points to be matched based on the feature descriptors of the feature points to be matched through the K-nearest neighbor search algorithm, obtain the rough matching point pairs, and perform fine matching on the rough matching point pairs through the decoupled RANSAC algorithm to obtain a three-dimensional space rigid body transformation model; A splicing and fusion module, configured to perform micro-nano surface structure splicing and fusion through transformation interpolation and overlapping region data fusion based on the three-dimensional space rigid body transformation model.

9. A three-dimensional surface topography fast stitching device based on improved ORB, characterized in that, It includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit is caused to execute the steps of the method according to any one of claims 1 to 7.

10. A storage medium, characterized in that, It stores a computer program executable by a three-dimensional surface topography rapid splicing device based on improved ORB. When the computer program runs on the three-dimensional surface topography rapid splicing device based on improved ORB, the three-dimensional surface topography rapid splicing device based on improved ORB is caused to execute the steps of the method according to any one of claims 1 to 7.

Images