Image plane-based feature point gathering method and device, and electronic device

Abstract

The application provides a feature point gathering method and device based on an image plane and electronic equipment. The method comprises: extracting feature points of a first image and a second image to be matched, and performing matching processing based on the Euclidean distance between the descriptors of each feature point to obtain an initial matching pair set U; performing near neighbor point matching on the feature points after the matching processing to obtain a matching pair set S after filtering outliers; randomly extracting a set number of matching pairs from the matching pair set S to perform transformation matrix estimation, substituting the estimated transformation matrix into the initial matching pair set U to obtain a multi-plane inlier set I, and dynamically moving the matching points according to the distance of the matching points in the multi-plane inlier set I to the center of gravity of the point set to obtain a multi-plane gathered point set PR; gathering the feature points in the first image and the second image according to a clustering algorithm and the multi-plane gathered point set PR to obtain a plane data cluster, and obtaining a multi-model based on the plane data cluster to realize multi-model matching of the images.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus and electronic device for feature point aggregation based on an image plane. Background Technology

[0002] Planar structures are quite common in daily life and learning. Numerous planes of different shapes and sizes constitute complex scenes. Understanding the structure of indoor and outdoor environments is important in many applications of machine vision, such as 3D reconstruction, autonomous landing of detectors, augmented reality, and robot navigation. Feature points extracted from images are distributed throughout the entire image. Filtering feature points that do not belong to the planes of the main objects in the image and incorrect matching pairs, and then clustering the remaining feature points based on planes to achieve multi-model matching is a key research focus.

[0003] Existing feature point extraction algorithms mostly rely on clustering algorithms and their improved algorithms to divide plane matching points to obtain multiple models, but the efficiency and accuracy of multiple model estimation are relatively low. Summary of the Invention

[0004] This application provides a feature point aggregation method, apparatus, and electronic device based on image plane to solve the problem of low efficiency and accuracy of existing multi-model estimation based on feature point aggregation.

[0005] In a first aspect, embodiments of this application provide a feature point aggregation method based on an image plane, including:

[0006] Feature points are extracted from the first and second images to be matched, and matching is performed based on the Euclidean distance between the descriptors of each feature point to obtain an initial set of matching pairs U.

[0007] Nearest neighbor matching is performed on the feature points after matching to achieve out-point filtering, resulting in a set of matching pairs S after filtering out-points.

[0008] A set number of matching pairs are randomly selected from the matching pair set S to estimate the transformation matrix. The estimated transformation matrix is ​​then substituted back into the initial matching pair set U to obtain a multi-plane point set I. The matching points are then dynamically moved according to the distance from the matching points of the first image and the second image in the multi-plane point set I to the centroid of the point set to obtain a multi-plane aggregation point set PR.

[0009] The feature points in the first image and the second image are clustered according to the clustering algorithm and the multi-plane clustering point set PR, respectively, to obtain the first planar data clique C of the first image. L and the second planar data cluster C of the second image R And based on the first planar data group C L and the second planar data group C RMultiple models are obtained to achieve multi-model matching of images.

[0010] In one possible implementation, the step of randomly selecting a predetermined number of matching pairs from the matching pair set S, estimating the transformation matrix, and substituting the estimated transformation matrix back into the initial matching pair set U to obtain the multi-plane point set I includes:

[0011] Randomly select z matching pairs from the matching pair set S to form a set Z; where z∈[6,9];

[0012] The transformation matrix M is calculated using the least squares method based on the given set Z. i ;

[0013] The current transformation matrix M i Substituting back into the given set Z, the distance error DE is calculated:

[0014]

[0015] Where DE is the distance error of the matching pair after transformation by the homography matrix H or the fundamental matrix F; It is the epipolar line of the matching point in the first image in the second image. It is the epipolar line of the matching point of the second image in the first image; and yes The squares of the first and second terms, and yes The squares of the first and second terms;

[0016] If the distance error DE between all z matching pairs is greater than the distance error threshold Δ1, then discard them; otherwise, M will be... i Substitute back into the initial set of matching pairs U, obtain matching pairs whose distance error DE is less than or equal to the distance error threshold Δ2, and obtain the interior point set I. i ;

[0017] Repeat the operation of randomly selecting z matching pairs from the matching pair set S to form a set Z, and then sequentially calculating the interior point set I corresponding to the transformation model within the local region of each matching pair in the matching pair set S. i And store it in sequence I to obtain the point set I = {I1, I2, ..., I} in the multiplane. n}; where n is the number of interior points; Δ2 is greater than Δ1; Δ1 ∈ [1.0, 1.5]; Δ2 ∈ [2.0, 5.0].

[0018] In one possible implementation, the step of dynamically moving the matching points based on the distance from the matching points of the first and second images in the multi-plane point set I to the centroid of the point set, to obtain the multi-plane aggregation point set PR, includes:

[0019] The i-th subset I of the multiplane point set I i All matching points are stored in their respective images. Right now Obtain the matching point set for each of the two images. Where k = L, R, j = 1, 2, ..., m; Store the i-th subset I i The matching point in the first image; Store the i-th subset I i The matching point in the second image;

[0020] calculate Distance between matching points and centroids of point sets

[0021]

[0022] in,

[0023] Calculate the distance |Δd| that each matching point moves towards the centroid:

[0024]

[0025] in,

[0026] Move the matching points according to the following formula until the change in the position of the matching points is less than the threshold ε, and obtain the multi-plane aggregation point set PR:

[0027]

[0028] in, The distance of the j-th matching point from the centroid The distance; and Let be the j-th matching point in the i-th matching point subset of the first image and the second image; n is the number of matching subsets; m is the number of matching points in the i-th matching point subset; Let be the centroid of the i-th matching point subset in the two images; and This is the sum of the x-coordinates and y-coordinates of the matching points in the current matching point subset of the two images; The x-coordinate of the matching point; For the ordinate of the matching point; Let θ be the coordinates of the matching point after displacement; θ be the direction angle of the matching point's movement; |Δd| be the distance the matching point moves towards the centroid; if The length of movement is Otherwise, the movement length is It is the distance threshold, which is set according to the distribution of distances from the matching point to the centroid; α is the probability of change; α∈[0,1.0].

[0029] In one possible implementation, the feature points in the first image and the second image are clustered according to a clustering algorithm and the multi-plane clustering point set PR, respectively, to obtain a first planar data cluster C of the first image. L and the second planar data cluster C of the second image R ,include:

[0030] The meanshift algorithm is used to calculate the meanshift vector for each feature point in the multi-plane aggregation point set PR.

[0031] Each feature point is... Move the feature point and update its position x. i Until Until the distance error threshold Δ3 is less than or equal to the distance error threshold;

[0032] The feature points in the first image are clustered into a first planar data group C. L ={C L1 C L2 ...,C Ln}, and aggregate the feature points in the second image into a second planar data cluster C. R ={C R1 C R2 ,...,C Rm}; where n is the number of the first image data clusters, m is the number of the second image data clusters, and h is the kernel radius.

[0033] In one possible implementation, the first planar data cluster C L and the second planar data group C R Find multiple models, including:

[0034] Establish a sequence M that stores multiple models and a sequence G that stores the set of points within each model;

[0035] The required number of matching pairs m for the calculation model; the maximum inlier distance error threshold Δ; and the threshold m for determining the acceptable number of inliers in the current model. min If the number of interior points conforming to the current model is not less than m minIf yes, save the model; otherwise, discard it.

[0036] In the first planar data group C respectively L The i-th subset C Li and the second planar data group C R The j-th subset C Rj Extract m matching pairs from the sample to obtain the current transformation model M. now Determine M now In the data cluster {C Li C Rj The set of interior points G of} now Is the length greater than m? min If the condition is not met, a new match will be extracted; otherwise, M will be removed. now Put in M, and put G now Place G;

[0037] The process ends when either the first or second condition is met. Li C Rj Extract matching operations;

[0038] The first condition is:

[0039]

[0040] Where q is the confidence rate, k is the number of matching pairs sampled in each iteration, n is the number of matching pairs in the corresponding data clique, and m now This represents the number of points within the current transformation matrix.

[0041] The second condition is:

[0042] m now ≥m min

[0043] Where, m now m is the number of points within the current transformation matrix. min The threshold m for the number of interior points in the current model min .

[0044] In one possible implementation, the step of performing nearest neighbor matching on the matched feature points to achieve outlier filtering, resulting in a set S of matched pairs after outlier filtering, includes:

[0045] Based on the K-nearest neighbor algorithm, α is searched for the first image. L The nearest neighbor points are used to search for α in the second image. R One nearest neighbor point;

[0046] Determine the nearest neighbor intersection S of each matching point i ;

[0047] When the set conditions are met, adjust the number of nearest neighbors to search, and re-search for nearest neighbors based on the adjusted number and update the intersection S of nearest neighbors. i ;

[0048] Merged and updated nearest neighbor intersection S i This yields a set of matching pairs S.

[0049] In one possible implementation, the number of nearest neighbors is adjusted when certain conditions are met, including:

[0050] Nearest neighbor intersection S i If the intersection point is not empty, calculate the average distance from the intersection point to each matching point in the first image. and the average distance to each matching point in the second image

[0051] make If γ≥1, then α L =γα L ;otherwise

[0052] Where i = 1, 2, ..., n, and n is the intersection length.

[0053] Secondly, embodiments of this application provide a feature point aggregation device based on an image plane, comprising:

[0054] The initial matching module is used to extract feature points from the first and second images to be matched, and to perform matching processing based on the Euclidean distance between the descriptors of each feature point to obtain an initial set of matching pairs U.

[0055] The filtering module is used to perform nearest neighbor matching on the feature points after matching to achieve out-point filtering, and obtain the set of matching pairs S after filtering out-points.

[0056] The moving module is used to randomly select a set number of matching pairs from the matching pair set S, estimate the transformation matrix, substitute the estimated transformation matrix back to the initial matching pair set U to obtain a multi-plane point set I, and dynamically move the matching points according to the distance from the matching points of the first image and the second image in the multi-plane point set I to the centroid of the point set to obtain a multi-plane aggregation point set PR.

[0057] The clustering module is used to cluster feature points in the first image and the second image according to the clustering algorithm and the multi-plane clustering point set PR, respectively, to obtain the first planar data cluster C of the first image. L and the second planar data cluster C of the second image R And based on the first planar data group C L and the second planar data group CR Multiple models are obtained to achieve multi-model matching of images.

[0058] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect or any possible implementation of the first aspect above.

[0059] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method as described in the first aspect or any possible implementation of the first aspect.

[0060] This application provides a feature point aggregation method, apparatus, and electronic device based on an image plane. By extracting feature points from a first image and a second image to be matched, an initial matching pair set U is obtained through matching processing. An adaptive out-point filtering method based on the consistency of the distribution of nearest neighbor internal points is used to perform nearest neighbor point matching to achieve out-point filtering, resulting in a matching pair set S after filtering out-points. Then, the matching point positions are moved using a coplanar point aggregation method to improve the degree of coplanar point aggregation. Subsequently, a clustering algorithm is used to divide the aggregated matching points into planes. Finally, multiple transformation models are obtained based on the corresponding plane matching pairs, thereby improving the efficiency and accuracy of multi-model estimation. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a flowchart illustrating the implementation of a feature point aggregation method based on an image plane according to an embodiment of this application.

[0063] Figure 2 This is a flowchart illustrating the implementation of a feature point aggregation method based on an image plane, provided in another embodiment of this application.

[0064] Figure 3 This is a flowchart illustrating the implementation of a feature point aggregation method based on an image plane, provided in another embodiment of this application.

[0065] Figure 4 This is an algorithm effect diagram provided by a specific embodiment of this application;

[0066] Figure 5This is a comparison diagram of the solution provided in a specific embodiment of this application with other algorithms;

[0067] Figure 6 This is a schematic diagram of the structure of a feature point aggregation device based on an image plane provided in an embodiment of this application;

[0068] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0069] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0070] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0071] Unless otherwise stated, the term "multiple" means two or more.

[0072] In this embodiment, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0073] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0074] Various feature point extraction algorithms for image feature extraction have been disclosed in the prior art, and each algorithm has different characteristics in the feature point extraction process.

[0075] Feature point extraction algorithms like SIFT extract feature points across the entire image. Problems like 3D reconstruction and autonomous landing of probes only process local areas of the image (such as buildings or celestial bodies). Pre-filtering feature points outside the study area is beneficial for subsequent operations. RANSAC is the preferred method for feature point filtering. It randomly selects 4 or 7 pairs of matching points from the matching pair sets of two images, calculates the transformation matrix, and substitutes it back into the matching pair set to obtain the optimal set of interior points (correct matching pairs) for the current matrix. The transformation matrix with the largest number of supported points is used as the final model, and the interior point set corresponding to this model is retained to filter out outliers (incorrect matching pairs). The algorithm efficiency is related to the proportion of interior points in the matching pair set; selecting the smallest subset to calculate the transformation model may not result in optimal model parameters.

[0076] The NAPSAC algorithm, a derivative of the RANSAC algorithm, assumes that inliers are closer than outliers in an image and replaces uniform random sampling with random sampling from nearest-neighbor regions, making RANSAC suitable for high-dimensional data. The PROSAC algorithm sorts the set of matching pairs based on a similarity metric and semi-randomly samples from the subset of data with the highest similarity metric. This algorithm can find all inliers in a shorter time, with the worst-case scenario being the same as RANSAC. The LO-RANSAC algorithm resamples the inlier set based on different error thresholds to improve the quality of the transformation model. The MLESAC algorithm uses maximum likelihood estimation to select the optimal model, improving sensitivity to thresholds. The NG-RANSAC algorithm combines RANSAC with neural networks, using the neural network to predict the weights of each matching pair, which can better filter out outliers with large errors. For two images with wide baselines and large viewpoint differences, multiple transformation models exist between the images. Feature points on different planes of the images belong to different transformation models, but during multi-model computation, points within the same plane may belong to different inlier sets of different models. Multi-model matching algorithms include Sequential RANSAC, MultiRANSAC, J-Linkage, T-Linkage, Multi-H, PEARL, AdaLam, GMS, and SuperGlue. Sequential RANSAC and MultiRANSAC are affected by the low inlier rate of each homography model; J-Linkage and T-Linkage algorithms randomly select clusters in the preference space and determine the importance of a plane based on the number of associated inliers; PEARL and Multi-H algorithms transform the multi-model matching problem into a global energy optimization problem; AdaLam selects seed points based on the distance ratio of the nearest neighbor matching pairs for each keypoint, ensuring the uniqueness and coverage of seed points and guaranteeing high-speed parallelism; GMS divides the image into a fixed-size grid, counts the number of matched feature points in each small region, and uses this for outlier filtering; this method is not suitable when the two images have significant differences in viewpoint or scaling; SuperGlue performs feature point matching and outlier filtering simultaneously, improving local feature differences through self-attention and cross-attention layers.

[0077] The aforementioned multi-model matching algorithm hardly modifies the positions of multi-plane feature points, relying solely on clustering algorithms and their improvements to divide the plane matching points to obtain multiple models. In fact, when the inlier rate in the matching pair set is high, the distance from the matching point to the centroid of the point set is almost the same for each pair of inliers in the transformed model, while the distance from the matching point to the centroid of the point set varies relatively more among the outliers. Matching points closer to the centroid are more likely to be coplanar points, while matching points farther from the centroid are less likely to be coplanar points.

[0078] Based on the above characteristics of the distribution of correctly matched points on the same plane, this application aims to dynamically adjust the position of the matched points according to the distance from the matched point to the centroid of the point set, thereby improving the clustering of points on the same plane. The points on each plane in the image are clustered together, and a clustering algorithm is used to cluster the matched points to obtain a multi-model with higher accuracy.

[0079] To make the objectives, technical solutions, and advantages of this application clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0080] Figure 1 This is a flowchart illustrating the implementation of a feature point aggregation method based on an image plane according to an embodiment of this application, as shown below. Figure 1 As shown, the method includes the following steps:

[0081] S101, extract feature points from the first and second images to be matched, and perform matching processing based on the Euclidean distance between the descriptors of each feature point to obtain the initial set of matching pairs U.

[0082] S102, perform nearest neighbor matching on the feature points after matching to achieve out-point filtering, and obtain the set of matching pairs S after filtering out-points.

[0083] In steps S101 and S102, feature points are extracted and described for both images. An initial set of matching pairs is obtained using the Euclidean distance between feature points, and outliers are filtered out. To obtain more representative feature points and higher-precision matching pairs, different feature point extraction and matching algorithms can be selected based on the specific situation. For example, Harris corner points and AKAZE points can be selected for short-baseline images, while Sift and Surf feature points can be selected for wide-baseline images. By analyzing the distribution patterns of inliers and outliers, matching pairs with inconsistent distributions in local areas are filtered out, while inliers with consistent distributions are retained, thereby improving the inlier rate.

[0084] S103, randomly select a set number of matching pairs from the matching pair set S to estimate the transformation matrix, substitute the estimated transformation matrix back into the initial matching pair set U to obtain the multi-plane point set I, and dynamically move the matching points according to the distance from the matching points of the first and second images in the multi-plane point set I to the centroid of the point set to obtain the multi-plane aggregation point set PR.

[0085] In this process, points on the same plane in the 2D image are clustered, and the distance of each matching point from its centroid is calculated. In sets of matching pairs with a high inlier rate, the distances from two matching points within an inlier to their respective centroids are almost identical, while the distances from two matching points outside an inlier to their respective centroids differ relatively significantly. The distance histogram approximates a chi-square distribution, and a distance threshold is applied. Set to the average distance from the matching point to the center of gravity. The smaller the distance from the matching point to the center of gravity, the greater the probability that they belong to the same plane, and the greater the distance to the center of gravity. Conversely, the greater the distance from the matching point to the center of gravity, the less likely they belong to the same plane, and the smaller the distance to the center of gravity. If the distance between the matching point and the center of gravity is no greater than... If a matching point is found, it moves towards the centroid by the original distance; otherwise, it moves towards the centroid by α times the original distance, where α ∈ [0, 1.0] and decreases as the distance between the matching point and the centroid increases. Aggregation stops when the change in the position of the matching point is less than the threshold ε ∈ [0, 2.0].

[0086] Specifically, the matching point position is dynamically moved according to the distance from the matching point to the centroid of the point set. The iteration stops when the change in the matching point position is less than the threshold ε, where the threshold ε is the distance between the matching points before and after each round of movement.

[0087] S104, cluster the feature points in the first image and the second image according to the clustering algorithm and the multi-plane clustering point set PR respectively, to obtain the first plane data clique C of the first image. L The second plane data group C of the second image R And based on the first plane data group C L Second plane data group C R Multiple models are obtained to achieve multi-model matching of images.

[0088] Optionally, the meanshift algorithm is used for clustering. The meanshift algorithm is used to divide the data into different planar clusters. Transformation models are calculated based on matching pairs between corresponding data clusters in two images, ensuring that each matching pair belongs to only one plane's transformation model. This results in a multi-model sequence, improving the accuracy and efficiency of the proposed scheme and providing a technical foundation for subsequent 3D reconstruction and the selection of flat regions for small celestial bodies.

[0089] In this embodiment, feature points are extracted from the first and second images to be matched, and an initial set of matching pairs U is obtained through matching processing. An adaptive out-point filtering method based on the consistency of the distribution of nearest neighbor in-place points is used to perform nearest neighbor point matching to achieve out-point filtering, resulting in a set of matching pairs S after filtering out-points. Then, the position of the matching points is moved using a coplanar point aggregation method to improve the degree of coplanar point aggregation. Afterward, a clustering algorithm is used to divide the aggregated matching points into planes. Finally, multiple transformation models are obtained based on the corresponding plane matching pairs, which improves the efficiency and accuracy of multi-model estimation.

[0090] In one possible implementation, step S102 involves performing nearest neighbor matching on the feature points after matching to achieve outlier filtering, resulting in a set S of matched pairs after outlier filtering, including:

[0091] The K-nearest neighbor algorithm is used to search for α in the first image. L 1 nearest neighbor point, for the second image search α R One nearest neighbor point;

[0092] Determine the nearest neighbor intersection S of each matching point i ;

[0093] When the set conditions are met, adjust the number of nearest neighbors to search, and re-search for nearest neighbors based on the adjusted number and update the intersection S of nearest neighbors. i ;

[0094] Merged and updated nearest neighbor intersection S i This yields a set of matching pairs S.

[0095] Specifically, the number of nearest neighbor points searched, α, is automatically adjusted based on the ratio γ of the average distances γ from the nearest neighbor intersection points to each matching point. L α R This improves the automation of the filtering process.

[0096] In one possible implementation, step S102, adjusting the number of searched nearest neighbors when a set condition is met, includes:

[0097] Nearest neighbor intersection S i If the intersection point is not empty, calculate the average distance from the intersection point to each matching point in the first image. and the average distance to each matching point in the second image

[0098] make If γ≥1, then α L =α L ;otherwise

[0099] Where i = 1, 2, ..., n, and n is the intersection length.

[0100] In one possible implementation, a predetermined number of matching pairs are randomly selected from the matching pair set S for transformation matrix estimation. The estimated transformation matrix is ​​then substituted back into the initial matching pair set U to obtain a multi-plane point set I, including:

[0101] Randomly select z matching pairs from the matching pair set S to form a set Z; where z∈[6,9];

[0102] The transformation matrix M is calculated using the least squares method based on the constituent set Z. i ;

[0103] The current transformation matrix M i Back-substitution of the set Z to calculate the distance error DE:

[0104]

[0105] Where DE is the distance error of the matching pair after transformation by the homography matrix H or the fundamental matrix F; The matching point in the first image is the epipolar line in the second image. The matching point of the second image is the epipolar line in the first image; and yes The squares of the first and second terms, and yes The squares of the first and second terms;

[0106] If the distance error DE between all z matching pairs is greater than the distance error threshold Δ1, then discard them; otherwise, M will be... i Substitute back into the initial set of matching pairs U, obtain matching pairs whose distance error DE is less than or equal to the distance error threshold Δ2, and obtain the inlier set I. i ;

[0107] Repeat the operation of randomly selecting z matching pairs from the matching pair set S to form a set Z, and then calculating the interior point set corresponding to the transformation model in the local region for each matching pair in the matching pair set S, and storing it in sequence I, to obtain the multiplane interior point set I = {I1, I2, ..., I...} n}; where n is the number of interior points; Δ2 is greater than Δ1; Δ1∈[1.0,1.5]; Δ2∈[2.0,5.0].

[0108] Since the coordinates of feature points in the image are two-dimensional, to address changes in a three-dimensional environment, each feature point needs to be assigned a z-axis coordinate to transform it into homogeneous coordinates. The distance error thresholds Δ1 and Δ2 are manually set. Experiments show that Δ1 should not exceed 1.5 pixels, and Δ2 performs best within the range of 2.0-5.0 pixels. The matching point change threshold ε performs best within the range of 0-2.0 pixels. To improve algorithm efficiency, the homography matrix is ​​calculated using 4 matching pairs or the fundamental matrix is ​​calculated using 7 matching pairs instead of the least squares method to calculate the transformation model.

[0109] In this embodiment, the transformation matrices that meet the conditions in the set of matched pairs after filtering outgoing points are substituted back into the initial set of matched pairs to obtain the set of interior points that conform to the transformation matrix. Based on the characteristic that the closer a matched point is to the centroid of the point set, the greater the probability that they belong to the same plane, the matched points are dynamically moved according to their distance from the centroid, thereby improving the clustering of matched points on the same plane. By observing the histogram of the distances from the matched points in the interior point set of the transformation matrix to the centroid of the point set, it is found that the histogram approximates a chi-square distribution. The distance threshold is then set... The distance is set to the mean, and the transformation probability α is set to decrease as the distance from the matching point to the centroid increases, thereby improving the automaticity of the algorithm.

[0110] In one possible implementation, matching points are dynamically moved based on the distance from the centroid of the point set to the matching points of the first and second images in the multi-plane point set I, to obtain the multi-plane aggregation point set PR, which includes:

[0111] The i-th subset I of the point set I in the multiplane i All matching points are stored in their respective images. Right now Obtain the matching point set for each of the two images. Where k = L, R, j = 1, 2, ..., m; Store the matching points of the first image; Store the matching points of the second image;

[0112] calculate Distance between matching points and centroids of point sets

[0113]

[0114] in,

[0115] Calculate the distance |Δd| that each matching point moves towards the centroid:

[0116]

[0117] in,

[0118] Move the matching points according to the following formula until the change in the position of the matching points is less than the threshold ε, and obtain the multi-plane aggregation point set PR:

[0119]

[0120] in, The distance of the j-th matching point from the centroid The distance; and Let be the j-th matching point in the i-th matching point subset of the first and second images; n is the number of matching subsets; m is the number of matching points in the i-th matching point subset. Let be the centroid of the i-th matching point subset in the two images; and This is the sum of the x-coordinates and y-coordinates of the matching points in the current matching point subset of the two images; The x-coordinate of the matching point; For the ordinate of the matching point; Let θ be the coordinates of the matching point after displacement; θ be the direction angle of the matching point's movement; |Δd| be the distance the matching point moves towards the centroid; if The length of movement is Otherwise, the movement length is It is the distance threshold, which is set according to the distribution of distances from the matching point to the centroid; α is the probability of change; α∈[0,1.0].

[0121] In one possible implementation, feature points in the first and second images are clustered using a clustering algorithm and a multi-plane clustering point set PR, respectively, to obtain a first planar data cluster C of the first image. L The second plane data group C of the second image R ,include:

[0122] The meanshift algorithm is used to calculate the meanshift vector for each feature point in the multiplane aggregation point set PR.

[0123]

[0124] Each feature point is... Move the feature point and update its position x. i Until Until the distance error threshold Δ3 is less than or equal to the distance error threshold;

[0125] The feature points in the first image are clustered into a first planar data group C. L ={C L1 C L2 ...,C Ln}, and aggregate the feature points in the second image into a second planar data cluster C. R ={C R1 C R2 ,...,C Rm}; where n is the number of data clusters in the first image, m is the number of data clusters in the second image, and h is the kernel radius.

[0126] The kernel function bandwidth parameter h is set according to the actual situation, and the termination condition threshold Δ3 is generally set to 0.0000005 pixels.

[0127] In this implementation, meanshift clustering is used to cluster data clusters in two images. The core of the meanshift clustering algorithm is to obtain the meanshift vector based on the attributes of the data objects, and then change the position of the data objects, that is, to move the planar feature points in the image in the direction pointed to by the meanshift vector. Through multiple iterations, the cluster center is obtained when the length of the meanshift vector is less than the threshold Δ3, and all feature points are assigned to different clusters.

[0128] In one possible implementation, based on the first planar data cluster C L Second plane data group C R Find multiple models, including:

[0129] Create a sequence M to store multiple models and a sequence G to store the set of points within each model; both have an initial length of 0.

[0130] The required number of matching pairs m for the calculation model; the maximum inlier distance error threshold Δ; and the threshold m for determining the acceptable number of inliers in the current model. min If the number of interior points conforming to the current model is not less than m min If yes, save the model; otherwise, discard it.

[0131] The i-th subset C of the first plane data clique Li The j-th subset C of the second plane data clique Rj Extract m matching pairs from the sample to obtain the current transformation model M. now Determine M now In the data cluster {C Li C Rj The set of interior points G of} now Is the length greater than m? min If the condition is not met, a new match will be extracted; otherwise, M will be removed. now Put in M, and put G now Place G;

[0132] The process ends when either the first or second condition is met. Li C Rj Extract matching operations;

[0133] The first condition is:

[0134]

[0135] Where q is the confidence rate, k is the number of matching pairs sampled in each iteration, n is the number of matching pairs in the corresponding data clique, and m now This represents the number of points within the current transformation matrix.

[0136] The second condition is:

[0137] m now ≥m min

[0138] Where, m now m is the number of points within the current transformation matrix. min The threshold m for the number of interior points in the current model min .

[0139] In this process, the transformation model of the corresponding data cluster is calculated and stored in the model sequence. Matching points in different planes are obtained through multiple transformation matrices during the matching point aggregation process in step S103. A relatively large distance error threshold is set when solving the transformation model between the current planes, thereby classifying as many matching pairs as possible as inliers in the corresponding data cluster. If the number of inliers in the current transformation matrix is ​​greater than or equal to m... min If the matrix is ​​found to be true, it is stored in the model sequence; otherwise, it is discarded.

[0140] In this implementation, there are subtle differences in the transformations of the same-plane matching pairs in the two images, and multiple transformation matrices exist between the same-plane data clusters. Therefore, a relatively large distance error threshold is set when calculating the interior points of the transformation model in this step. Since the interior point rate after matching different plane data clusters in the two images is extremely low, the number of interior points m corresponding to the obtained model is less than m0. min If the data is of the same plane, it is discarded. After matching the data clusters, the inlier rate is relatively high, and the number of inliers corresponding to the obtained model is not less than m. min Thus, it is stored in a multi-model sequence.

[0141] Figure 2 This is a flowchart illustrating the implementation of a feature point aggregation method based on an image plane according to an embodiment of this application. Compared to... Figure 1 The steps are shown in more detail below. The specific process is as follows:

[0142] Feature points are extracted from the images to be matched, and descriptors are used to describe the matching to obtain matching pairs.

[0143] Adaptive error matching pair removal is performed based on the consistency of the distribution of nearest neighbor internal points;

[0144] Extract matching pairs within a local region to estimate the transformation matrix, and then substitute the sampled matching pairs back into the matrix.

[0145] Determine whether the distance error threshold is met. If the distance error threshold is met, substitute back into the initial matching pair set to select interior points; otherwise, repeat the steps of "extracting matching pairs in the local region to estimate the transformation matrix and substituting back into the sampled matching pairs".

[0146] Determine whether the current matching pair meets the distance error threshold. If it does, obtain the set of interior points in the current transformation matrix; otherwise, repeat the step of "substituting back to the initial matching pair set to select interior points".

[0147] Determine if the degree of change in the position of the matching point is less than the threshold ε; if the degree of change in the position of the matching point is less than the threshold ε, obtain the planar clustered matching point set;

[0148] When the degree of change in the position of the matching point is greater than or equal to the threshold ε, calculate the centroid of the matching point set in the left and right images, the distance from each matching point to the centroid, and the distance threshold. The calculated distance value is less than or equal to the distance threshold. If the original distance is not reached, move the original distance towards the center of gravity; otherwise, move the original distance by α times until the change in the position of the matching point is less than the threshold ε, thus obtaining a planar clustered matching point set.

[0149] The meanshift algorithm is used to aggregate feature points on different planes of the image;

[0150] Calculate and store the transformation model between corresponding planar data clusters to obtain a multi-model sequence.

[0151] In this embodiment, out-point filtering is performed based on the consistency of point distribution within the neighborhood, improving the in-point rate and the automation of the filtering algorithm. The transformation model of each matching pair within the local region of the filtered in-point set is calculated, and this model is used to substitute back into the initial matching set to calculate the in-points. Since the distance between matching points on the same plane and the centroid varies in different images, to avoid affecting the in-plane point aggregation effect, the distance from the matching point to the centroid in the model's in-points is calculated. A distance threshold is set according to the distance distribution, and the movement amplitude is adjusted according to the distance from each matching point to the centroid, giving each matching point a certain degree of initiative in its aggregation behavior, thus obtaining a point set with better in-plane aggregation effect.

[0152] Figure 3 This is a flowchart illustrating the implementation of a feature point aggregation method based on an image plane according to an embodiment of this application. Wherein, in Figure 2 Based on the flowchart shown, the steps are divided into four main modules: Module 1, Data Filtering; Module 2, Clustering of Points in the Same Plane; Module 3, Clustering Data Clusters and Performing Mean Shift Clustering; Module 4, Obtaining Multiple Models.

[0153] Additionally, the diagram also shows the image processing results for each module.

[0154] Two application examples of this application are given below.

[0155] Experiment 1: Verifying Real Images

[0156] There are two sets of real images: NESS and BreadCartoyChips. Experiments were conducted on each set of images. In this experiment, the NESS images were reduced to their original size. Keeping the breadcartoy chips size constant, the resolutions of the two images are 852*1136 pixels and 480*640 pixels, respectively. SIFT feature points are extracted from the images and matched according to Euclidean distance. The number of matching pairs for the two sets of images are 8952 and 2359, respectively. In the filtering algorithm, the number of nearest neighbors of the left and right matching points is set to α. L =α R =10, minimum number of matching pairs z is set to 6, distance error threshold Δ1 is set to 2.0 pixels and 3.0 pixels, distance error threshold Δ2 is set to 1.0 pixels and 1.5 pixels, threshold ε for the degree of change in matching point position is set to 0.5 pixels, meanshift aggregation algorithm kernel function bandwidth h is set to 350 and 50, meanshift aggregation termination condition threshold Δ3 is set to 0.0000005 pixels, and the process of obtaining multiple models ( Figure 3 The minimum number of interior points is set to 4, and the distance error threshold Δ is set to 200.0 pixels and 50.0 pixels. Figure 2 The paper presents the aggregation effect diagrams and multi-plane matching point color maps of two sets of images. This application can distinguish different plane matching points between images with wide baselines and large viewpoint differences, and calculate the transformation models of each plane, achieving ideal results.

[0157] Experiment 2: Comparative Experiment

[0158] Five architectural images from the Adelaidermf dataset were selected: nese, neem, unihouse, ladysymon, and johnsona. The unihouse image is 735*980 pixels, and the other four images are 2272*1704 pixels. In this experiment, all images except unihouse were resized to their original size. Each set of image feature points is matched using Euclidean distance. The method in this application is named "Proposed" and compared with the PEARL, Multi-H, and Sequential RANSAC algorithms. In the experiments, the homography matrix is ​​calculated using the 4-point method, and the number of nearest neighbors α for each matched point in the filtering algorithm is... L α RThe threshold values ​​are set to 10. In the coplanar point clustering algorithm, the number of matching pairs z extracted from the local region of each matching pair is 6. The distance error threshold Δ1 is set to 1.0-1.5 pixels, the distance error threshold Δ2 is set to 2.0-5.0 pixels, the degree of change in matching point position is set to 0-2.0 pixels, the termination condition threshold Δ3 in the meanshift clustering algorithm is set to 0.0000005 pixels, the Gaussian kernel bandwidth h is 250-350 pixels, and the threshold m for the number of interior points in the multi-model computation step is... min Set to 4. Select two evaluation metrics: the average misclassification rate and the median misclassification rate. Calculate the value of each metric after running each method 10 times on each set of images.

[0159] Figure 5 The comparison results between this application and three other methods are presented. From Figure 5 It can be seen that the method in this application is quite similar to the Multi-H method in terms of performance, and both methods are significantly inferior to other methods. This indicates that the method in this application has relatively more stable performance.

[0160] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0161] The following are device embodiments of this application. For details not described in detail, please refer to the corresponding method embodiments described above.

[0162] Figure 6 This is a schematic diagram of the structure of a feature point aggregation device based on an image plane according to an embodiment of this application, as shown below. Figure 6 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown, such as... Figure 6 As shown, the device includes: an initial matching module 601, a filtering module 602, a moving module 603, and a clustering module 604.

[0163] The initial matching module 601 is used to extract feature points from the first and second images to be matched, and to perform matching processing based on the Euclidean distance between the descriptors of each feature point to obtain an initial matching pair set U.

[0164] The filtering module 602 is used to perform nearest neighbor matching on the feature points after matching to achieve out-point filtering, and obtain the set of matching pairs S after filtering out-points.

[0165] The moving module 603 is used to randomly select a set number of matching pairs from the matching pair set S to estimate the transformation matrix, substitute the estimated transformation matrix back to the initial matching pair set U to obtain the multi-plane point set I, and dynamically move the matching points according to the distance from the matching points of the first image and the second image in the multi-plane point set I to the centroid of the point set to obtain the multi-plane aggregation point set PR.

[0166] Clustering module 604 is used to cluster feature points in the first image and the second image according to the clustering algorithm and the multi-plane clustering point set PR, respectively, to obtain the first plane data cluster C of the first image. L The second plane data group C of the second image R And based on the first plane data group C L Second plane data group C R Multiple models are obtained to achieve multi-model matching of images.

[0167] In this embodiment, feature points are extracted from the first and second images to be matched, and an initial set of matching pairs U is obtained through matching processing. An adaptive out-point filtering method based on the consistency of the distribution of nearest neighbor in-place points is used to perform nearest neighbor point matching to achieve out-point filtering, resulting in a set of matching pairs S after filtering out-points. Then, the position of the matching points is moved using a coplanar point aggregation method to improve the degree of coplanar point aggregation. Afterward, a clustering algorithm is used to divide the aggregated matching points into planes. Finally, multiple transformation models are obtained based on the corresponding plane matching pairs, which improves the efficiency and accuracy of multi-model estimation.

[0168] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 7 As shown, the electronic device 7 of this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. When the processor 70 executes the computer program 72, it implements the steps in the various image plane-based feature point aggregation method embodiments described above, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 70 executes the computer program 72, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 6 The functions of modules 601 to 604 are shown.

[0169] For example, the computer program 72 can be divided into one or more modules / units, which are stored in the memory 71 and executed by the processor 70 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 72 in the electronic device 7. For example, the computer program 72 can be divided into... Figure 6 Modules 601 to 604 are shown.

[0170] The electronic device 7 can be a desktop computer, laptop, handheld computer, cloud server, or other computing device. The electronic device 7 may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 7 This is merely an example of electronic device 7 and does not constitute a limitation on electronic device 7. It may include more or fewer components than shown, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.

[0171] The processor 70 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASiCs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0172] The memory 71 can be an internal storage unit of the electronic device 7, such as a hard disk or memory. The memory 71 can also be an external storage device of the electronic device 7, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device 7. Furthermore, the memory 71 can include both internal and external storage units of the electronic device 7. The memory 71 is used to store the computer program and other programs and data required by the electronic device. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0173] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0174] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0175] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0176] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0177] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0179] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above-described embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various image plane-based feature point aggregation method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0180] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A feature point gathering method based on an image plane, characterized by, Comprise: Perform feature point extraction on the first image and the second image to be matched, and perform matching processing based on the Euclidean distance between the descriptors of each feature point to obtain an initial matching pair set U; Perform nearest neighbor point matching on the feature points after matching processing to realize outlier filtering, and obtain a matching pair set S after filtering out outliers; Randomly draw a set number of matching pairs from the matching pair set S for transformation matrix estimation, back-substitute the estimated transformation matrix to the initial matching pair set U to obtain a multi-plane inlier set I, and dynamically move the matching points according to the distance of the matching points of the first image and the second image in the multi-plane inlier set I to the inlier set barycenter to obtain a multi-plane cluster point set ; According to the clustering algorithm and the multi-plane cluster point set Respectively, the feature points in the first image and the second image are clustered to obtain a first plane data group of the first image And a second plane data group of the second image And based on the first plane data group And the second plane data group A multi-model is obtained to realize multi-model matching of the image; The matching points are dynamically moved according to the distances from the matching points of the first image and the second image in the multi-plane inlier point set I to the point set barycenter, to obtain a multi-plane aggregated point set , comprising: storing all the matched points of the first subset in the first image and the second subset in the second image respectively according to the images to which the matched points belong, obtaining the matched point set of each of the two images respectively; wherein, i ; i i ​​​​​​​​​​​​ Computing the distance of the matching point to the center of mass of the point set : wherein ; calculating the distance each matching point moves toward the center of gravity : wherein ; Moving the matching point according to the following formula until the matching point position changes by less than a threshold until the matching point position changes by less than a threshold : wherein, ; is the first j distance between the distance between the and is the i is the j is the n is the number of matching subsets; m is the number of matching points in the i is the is the i is the and is the sum of the horizontal coordinates and the sum of the vertical coordinates of the matching points in the current matching point subset of the two images; is the horizontal coordinate of the matching point; is the vertical coordinate of the matching point; is the coordinate of the matching point after displacement; is the direction angle of the movement of the matching point; is the distance of the movement of the matching point to the center of gravity; if , the length of the movement is ; otherwise, the length of the movement is ; is the distance threshold value, which is set according to the distance distribution of the matching points to the center of gravity; is the change probability.

2. The image plane-based feature point clustering method of claim 1, wherein, The processor executes the computer program to realize the steps of the method of any one of claims 1 to 6. randomly extracting a matching pair from the matching pair set S to form a set Z; calculating a transformation matrix using least squares from the set of compositions Z ; substituting the current transformation matrix back-substitute the set of compositions Z to compute distance error The computer program is executed by the processor to realize the steps of the method of any one of claims 1 to 6. : wherein, Perform feature point extraction on the first image and the second image to be matched, and perform matching processing based on the Euclidean distance between the descriptors of each feature point to obtain an initial matching pair set U; is the distance error after the homography matrix H or the fundamental matrix F transformation; is the epipolar line of the matching point in the second image, is the epipolar line of the matching point in the first image; and is the square of the first and second terms of and is the square of the first and second terms of If the distance error between z matched pairs Perform nearest neighbor point matching on the feature points after matching processing to realize outlier filtering, and obtain a matching pair set S after filtering out outliers; All are greater than the distance error threshold If so, discard it; otherwise, Substitute back into the initial set of matching pairs U to obtain the distance error. The processor executes the computer program to realize the steps of the method of any one of claims 1 to 6. Less than or equal to the distance error threshold The matching pairs yield the interior point set. ; Repeat the random extraction of a matching pair from the matching pair set S and the operations after it, sequentially calculate the inner point set corresponding to the transformation model in the local area of each matching pair in the matching pair set S , and store it in sequence I, to obtain a multi-plane inner point set ; wherein, n is the number of inner point sets; the is greater than the . 3.The image plane-based feature point clustering method of claim 1, wherein, The first image and the second image are respectively clustered to obtain a first planar data cluster of the first image and a second planar data cluster of the second image , comprising:​ using a meanshift algorithm on the multi-planar cluster of points meanshift vector is computed for each feature point in the cluster ; Each feature point is updated with moving, updating feature point position until less than or equal to distance error threshold ​ aggregating feature points in the first image into and aggregating feature points in the second image into ; wherein, n is the number of data clusters of the first image, m is the number of data clusters of the second image; h is the kernel radius.

4. The image plane-based feature point clustering method of claim 3, wherein, The first plane data group is based on the first plane data The second plane data group is based on the second plane data The multi-model is obtained by solving The computer program is executed by the processor to realize the steps of the method of any one of claims 1 to 6. The number of matching pairs required for the calculation model m ; the maximum inner point distance error threshold is ; a threshold value of the number of inner points acceptable to the current model is set , if the number of inner points of the current model is not less than , the model is saved, otherwise discarded; extracting m matching pairs from a first subset of the first plane data group and a second subset of the second plane data group, respectively, to obtain a current transformation model i j ​​​​​​​​​​​​​ ending the processing of the data bundle when either the first condition or the second condition is met extraction matching operation; Perform feature point extraction on the first image and the second image to be matched, and perform matching processing based on the Euclidean distance between the descriptors of each feature point to obtain an initial matching pair set U; wherein, q is the confidence rate, k is the number of matched pairs sampled per iteration, n is the number of matched pairs in the corresponding data cluster, is the number of inliers in the current transformation matrix; Perform nearest neighbor point matching on the feature points after matching processing to realize outlier filtering, and obtain a matching pair set S after filtering out outliers; wherein, is the number of inliers in the current transformation matrix; is the threshold of the number of inliers of the current model .

5. The image plane-based feature point clustering method according to any one of claims 1 to 4, characterized in that, The processor executes the computer program to realize the steps of the method of any one of claims 1 to 6. searching for a number of nearest neighbor points for the first image based on a K-Nearest Neighbor algorithm, searching for a number of nearest neighbor points for the second image, searching for a number of nearest neighbor points for the second image, determining the intersection S of the near neighbors of each matching point i ; adjusting the number of search neighbor points when a set condition is met, and re-searching the neighbor points and updating the neighbor point intersection S according to the adjusted number i ; The updated near neighbor point intersection S is merged i to obtain the matching pair set S.

6. The image plane-based feature point clustering method of claim 5, wherein, The computer program is executed by the processor to realize the steps of the method of any one of claims 1 to 6. Nearest point intersection S i If not empty, then compute average distance of intersection point to each matching point of the first image and to each matching point of the second image ; Let , if , then ; otherwise ; wherein ，n is the intersection length.

7. An image plane-based feature point clustering apparatus for performing the image plane-based feature point clustering method according to any one of claims 1 to 6, characterized by Perform feature point extraction on the first image and the second image to be matched, and perform matching processing based on the Euclidean distance between the descriptors of each feature point to obtain an initial matching pair set U; Perform nearest neighbor point matching on the feature points after matching processing to realize outlier filtering, and obtain a matching pair set S after filtering out outliers; The processor executes the computer program to realize the steps of the method of any one of claims 1 to 6. A mobile module is used to randomly extract a set number of matching pairs from the matching pair set S for transformation matrix estimation, and the estimated transformation matrix is substituted back into the initial matching pair set U to obtain a multi-plane inlier set I, and the distance of the matching points of the first image and the second image in the multi-plane inlier set I to the inlier set barycenter is dynamically moved to obtain a multi-plane aggregated point set ; a clustering module, configured to cluster feature points in the first image and the second image respectively to obtain a first planar data cluster of the first image and a second planar data cluster of the second image a clustering module, configured to cluster feature points in the first image and the second image respectively to obtain a first planar data cluster of the first image and a second planar data cluster of the second image a clustering module, configured to cluster feature points in the first image and the second image respectively to obtain a first planar data cluster of the first image and a second planar data cluster of the second image a clustering module, configured to cluster feature points in the first image and the second image respectively to obtain a first planar data cluster of the first image and a second planar data cluster of the second image a clustering module, configured to cluster feature points in the first image and the second image respectively to obtain a first planar data cluster of the first image and a second planar data cluster of the second image a clustering module, configured to cluster feature points in the first image and the second image respectively to obtain a 8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The computer program is executed by the processor to realize the steps of the method of any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. ​

Images