An image feature matching optimization method based on intra-class spatial consistency

Abstract

The present invention discloses an image feature matching optimization method based on intra-class spatial consistency in the field of computer vision and image processing technology, which comprises the following steps: feature point extraction and preliminary matching: extracting feature points from a query image and a reference image and performing preliminary matching; initialization and transformation model estimation: initializing a matching point set and residuals, and calculating an initial transformation model; error calculation and matching point set update: calculating the errors of matching point pairs, and updating the matching point set using a dynamic screening method; residual optimization: judging whether the optimal result is achieved based on the residuals, and deciding whether to continue iteration; intra-class spatial consistency clustering and isolated cluster elimination: performing cluster analysis after obtaining the optimal residuals, separating substructures based on intra-class spatial consistency, and eliminating isolated clusters; outputting results: outputting a matching point set after eliminating isolated clusters, which solves the problem that traditional feature matching methods are difficult to completely eliminate false matches in complex scenarios and are sensitive to noise.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and in particular to an image feature matching optimization method based on intra-class spatial consistency. Background Art

[0002] Image feature matching is a fundamental task in computer vision. Its core goal is to achieve precise alignment and scene understanding between images by extracting salient feature points from two images and establishing correspondences. Traditional methods rely primarily on extracting and similarity-measuring local feature descriptors. For example, these methods use algorithms like the Scale-Invariant Feature Transform (SIFT) or Oriented Robust Features (ORB) to detect keypoints and generate descriptors that are rotation- and scale-invariant. Based on the similarity of the descriptors, preliminary matching point pairs are screened and an initial set of correspondences is formed.

[0003] In practical applications, due to factors such as noise, illumination variations, perspective differences, and object motion during image acquisition, feature point matching results inevitably contain mismatches. To improve matching robustness, existing technologies typically employ iterative optimization strategies. These strategies construct a geometric model (such as a fundamental matrix or homography) by randomly sampling a small number of matching points. The consistency of other matching points with this model is then verified, gradually eliminating outliers that do not conform to the geometric constraints. By repeatedly optimizing model parameters, these methods can improve matching accuracy to a certain extent.

[0004] However, with the increasing complexity of application scenarios (such as multiple moving targets, dynamic backgrounds, large-scale perspective changes, etc.), traditional feature matching methods face new challenges. For example: 1. Traditional algorithms rely on epipolar constraint models, but in the case of pure camera rotation or uneven distribution of feature points, mismatched points may still exist. For example, when the camera undergoes pure rotation, the pixel displacement of the feature points is independent of the depth, resulting in the inability of traditional algorithms to effectively eliminate mismatched points. 2. Traditional algorithms eliminate mismatched points globally and lack optimization of local feature points, resulting in insufficient matching accuracy. Especially in the case of uneven distribution of feature points, global optimization is difficult to meet the matching requirements of local feature points. 3. Traditional methods usually use a fixed threshold to screen matching points. When the image noise is large or the feature points are unevenly distributed, the fixed threshold is difficult to adapt, resulting in a decrease in the quality of the matching point set. Summary of the Invention

[0005] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid obscuring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0006] Therefore, the purpose of the present invention is to provide an image feature matching optimization method based on intra-class spatial consistency to solve the problems that traditional feature matching methods are difficult to completely eliminate mismatches in complex scenes and are sensitive to noise.

[0007] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0008] An image feature matching optimization method based on intra-class spatial consistency, the steps are as follows:

[0009] S1. Feature point extraction and preliminary matching: Extract feature points from the query image and reference image and perform preliminary matching;

[0010] S2, initialization and transformation model estimation: Initialize the matching point set and residual, and calculate the initial transformation model;

[0011] S3, error calculation and matching point set update: calculate the error of matching point pairs and update the matching point set using dynamic screening method;

[0012] S4, residual optimization: Based on the residual, determine whether the optimal state is reached and decide whether to continue iteration;

[0013] S5. Intra-class spatial consistency clustering and isolated cluster elimination: After obtaining the optimal residual, cluster analysis is performed to separate substructures based on intra-class spatial consistency and eliminate isolated clusters;

[0014] S6. Output result: Output the set of matching points after removing isolated clusters.

[0015] As a preferred solution of the image feature matching optimization method based on intra-class spatial consistency described in the present invention, in the feature point extraction and preliminary matching steps, the ORB algorithm is used to extract key points from the query image and the reference image, and the key points are stored in C respectively. k and R k , and perform feature point matching based on the Hamming distance to obtain the initial matching set M. The Hamming distance formula is:

[0016]

[0017] in, and are the descriptors of the query image and the reference image respectively, ⊕ represents the binary XOR operation, is the descriptor length. The smaller the Hamming distance, the higher the matching similarity.

[0018] As a preferred solution of the image feature matching optimization method based on intra-class spatial consistency described in the present invention, in the initialization and transformation model estimation step, all matching point pairs in the initial matching set M are regarded as candidate point sets, the initialization preferred point set is empty, and the residual is initialized. , randomly select 8 pairs of matching points from the matching set M, and use the 8-point algorithm to calculate the initial basic matrix , the fundamental matrix Satisfy the epipolar constraint: ;

[0019] in, and To match the homogeneous coordinates of the point pairs, construct the coefficient matrix , solve the linear equations Get the basic matrix The vectorized form of , then perform singular value decomposition on the initial solution, set the minimum singular value to 0, and reconstruct the basic matrix .

[0020] As a preferred solution of the image feature matching optimization method based on intra-class spatial consistency described in the present invention, in the error calculation and matching point set update step, the epipolar geometric error of the matching point pair is calculated. , the formula is:

[0021] ;

[0022] Introduce the quartile method to dynamically screen the optimal points and calculate the error set Quartiles and , and the interquartile range ;

[0023] Set dynamic upper and lower thresholds: ;

[0024] If the error of the matching point pair If the point is within the dynamic upper and lower bounds, it will be added to the preferred point set. Otherwise, it will be marked as a candidate outlier. The preferred point set will be used to re-estimate the basic matrix F, and the optimization problem will be solved by the least squares method. The singular value decomposition of F will be corrected, and the minimum singular value will be set to 0.

[0025] As a preferred solution of the image feature matching optimization method based on intra-class spatial consistency described in the present invention, in the residual optimization step, the average residual of the preferred point set is calculated:

[0026] ;

[0027] in, is the number of points in the optimal point set, is the geometric error of each matching point pair, the residual Reflects the current base matrix For the degree of fitting of the preferred point set, the smaller the residual, the higher the model quality;

[0028] The calculated residual and the current optimal residual For comparison, The initial value is , if the current residual , indicating that the current model is better than the previous model, so the optimal residual is updated And save the current fundamental matrix And the optimal point set, then return to the initialization and transformation model estimation step to continue the iteration, otherwise, if This indicates that the current model has not brought significant improvement, so the iteration is stopped and the next step of cluster analysis is entered.

[0029] As a preferred solution of the image feature matching optimization method based on intra-class spatial consistency described in the present invention, in the intra-class spatial consistency clustering and isolated cluster elimination steps, multi-dimensional features are extracted from the current preferred point set to construct a 6-dimensional feature vector , the multidimensional features include the pixel coordinates of the matching point pairs in the two images and , geometric error and pixel shift ;

[0030] Use K-Means clustering method, preset the number of clusters K, and randomly select K initial cluster centers , for each eigenvector , calculate its Euclidean distance to the cluster center:

[0031]

[0032] The feature vector Assign to the nearest cluster:

[0033] ;

[0034] Update cluster centers:

[0035] ;

[0036] After iterating until the center is stable or the maximum number of iterations is reached, check the number of points in each cluster ,If the number of points is less than the set threshold, the cluster is regarded as an isolated cluster and the points in the isolated cluster are removed.

[0037] Compared with the existing technology, the present invention has the following beneficial effects: the present invention uses the quartile method to dynamically screen feature points and eliminates mismatches through K-Means clustering analysis based on intra-class spatial consistency, thereby addressing the limitations of traditional methods in complex scenarios. This method uses epipolar geometry to establish projection relationship constraints between feature points, effectively filtering outliers that violate the epipolar geometry principle; it uses a clustering algorithm to analyze the spatial distribution consistency of feature points and eliminate mismatches in isolated classes. On the basis of global optimization, local feature points are further optimized to improve matching accuracy, and by dynamically adjusting the weights of feature points, the impact of noise and other factors on matching accuracy is ultimately reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort. Among them:

[0039] Figure 1 This is a flowchart of an image feature matching optimization method based on intra-class spatial consistency of the present invention. DETAILED DESCRIPTION

[0040] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0041] like Figure 1 As shown, the present invention provides an image feature matching optimization method based on intra-class spatial consistency, the steps are as follows:

[0042] S1. Feature point extraction and preliminary matching: Extract feature points from the query image and reference image and perform preliminary matching;

[0043] S2, initialization and transformation model estimation: Initialize the matching point set and residual, and calculate the initial transformation model;

[0044] S3, error calculation and matching point set update: calculate the error of matching point pairs and update the matching point set using dynamic screening method;

[0045] S4, residual optimization: Based on the residual, determine whether the optimal state is reached and decide whether to continue iteration;

[0046] S5. Intra-class spatial consistency clustering and isolated cluster elimination: After obtaining the optimal residual, cluster analysis is performed to separate substructures based on intra-class spatial consistency and eliminate isolated clusters;

[0047] S6. Output result: Output the set of matching points after removing isolated clusters.

[0048] In order to describe the technical solution of the present invention in more detail, each step is described in further detail below, specifically:

[0049] S1. Feature point extraction and preliminary matching: Use the ORB algorithm (Oriented FAST and Rotated BRIEF) to extract key points from the query image and the reference image, and store them in vectors C respectively. k and R k ORB generates feature points through FAST corner detection and BRIEF binary descriptor, which has the characteristics of rotation invariance and high computational efficiency. It matches feature points based on Hamming distance to obtain the initial matching set M. The Hamming distance is defined as:

[0050] (1)

[0051] in, and are the descriptors of the query image and the reference image respectively, ⊕ represents the binary XOR operation, is the descriptor length (usually 256 bits for ORB). The smaller the Hamming distance, the higher the matching similarity.

[0052] S2. Initialization and transformation model estimation: The goal of this step is to lay the foundation for subsequent optimization, initialize the matching point set and estimate the initial transformation model. First, all matching point pairs in the initial matching set M are regarded as candidate point sets for subsequent screening of high-quality matching points. At this time, the preferred point set (high-quality matching point set) is initialized to empty, indicating that no matching points that meet the conditions have been screened out. At the same time, the residual is initialized. , as the benchmark for subsequent iterative optimization, the initial value of the residual is set to positive infinity to ensure that the residual of subsequent calculations can be gradually reduced, so as to find a better model. In order to estimate the initial transformation model, 8 pairs of matching points are randomly selected from the matching set M, and the classic 8-point algorithm is used to calculate the initial basic matrix . Basic Matrix Satisfy the epipolar constraint:

[0053] (2)

[0054] in, and To match the homogeneous coordinates of point pairs, the 8-point algorithm is implemented by constructing a coefficient matrix , where each row corresponds to a pair of matching points in the form of , by solving the linear equations , and get the basic matrix The vectorized form of , to ensure Satisfy the rank constraint of epipolar geometry (i.e. ), perform singular value decomposition (SVD) on the initial solution, set the minimum singular value to 0, and reconstruct the basic matrix This initial transformation model provides the basis for subsequent error calculation and optimization. Although the initial estimate may contain many mismatched points, the model quality can be gradually improved through iterative optimization in subsequent steps.

[0055] S3, Error calculation and matching point set update: The goal of this step is to update the optimal point set and improve the quality of matching points through error calculation and dynamic screening. First, all matching point pairs are updated. Calculate its epipolar geometric error to measure its deviation from the current basic matrix The error is defined as:

[0056] (3)

[0057] This error reflects whether the matching point pair meets the epipolar geometry constraint. The smaller the error, the more consistent the matching point pair is with the current model. To avoid the limitations of fixed thresholds, this paper introduces the quartile method to dynamically screen the optimal points. Specifically, the error set of all matching point pairs is Perform statistical analysis and calculate its quartiles:

[0058] (4)

[0059] in, is the 25th percentile, is the 75th percentile, Is the interquartile range, which is used to measure the degree of dispersion of the error distribution. Based on the quartile method, set the dynamic upper and lower thresholds:

[0060] (5)

[0061] For each matching point pair, if its error ∈[lower bound, upper bound], it is added to the preferred point set, indicating that the point pair meets the geometric constraints of the current model; otherwise, it is marked as a candidate outlier and temporarily removed. Compared with the traditional fixed threshold method, the quartile method can adaptively adjust the threshold to adapt to the error distribution in different scenarios and improve the robustness of the screening. After screening the preferred point set, use these points to re-estimate the basic matrix , solve the optimization problem by the least squares method:

[0062] (6)

[0063] To satisfy the rank constraint, singular value decomposition (SVD) is used to Corrections are made to set the minimum singular value to 0 to ensure that the fundamental matrix meets epipolar geometry requirements. This process gradually improves the quality of the matching point set through dynamic screening and model optimization, providing more reliable input for subsequent steps.

[0064] S4, residual optimization: The goal of this step is to evaluate the quality of the current model through the residual and decide whether to continue iterative optimization to obtain a better matching point set and transformation model. Based on the current preferred point set, calculate the average residual , as a measure of model quality. The residual is defined as the average geometric error of all matching point pairs in the preferred point set:

[0065] (7)

[0066] in, is the number of points in the optimal point set, is the geometric error of each matching point pair, the residual Reflects the current base matrix For the degree of fitting of the preferred point set, the smaller the residual, the higher the model quality. and the current optimal residual For comparison, The initial value is , if the current residual , indicating that the current model is better than the previous model, so the optimal residual is updated And save the current fundamental matrix And the optimal point set, then return to step S2, reselect matching point pairs, estimate the new basic matrix, and continue iterative optimization. On the contrary, if This indicates that the current model has not brought significant improvement, so the iteration is stopped and the next step of cluster analysis is entered.

[0067] The iterative process of residual optimization ensures that the fundamental matrix Gradually approach the optimal solution while continuously improving the quality of the selected point set. To avoid meaningless iterations, you can set a maximum number of iterations (e.g., 1000) or a residual change threshold (e.g., stop when it is less than 0.001) to balance computational efficiency and model accuracy. This step, through global optimization, provides high-quality input data for subsequent intra-class spatial consistency clustering.

[0068] S5, Intra-class spatial consistency clustering and isolated cluster elimination: This step uses the K-Means clustering method to separate potential substructures based on intra-class spatial consistency and eliminate mismatched points in isolated clusters. First, multi-dimensional features are extracted from the current preferred point set for cluster analysis. Features include spatial location and , represents the pixel coordinates of the matching point pair in the two images; Geometric error , reflecting the geometric consistency of the matching point pair; pixel displacement , represents the displacement of the matching point pair between the two images. Combining these features, a 6-dimensional feature vector is constructed .

[0069] Intra-class spatial consistency means that the matching points under the same geometric transformation relationship should be highly consistent in spatial position, geometric error, and pixel displacement. For example, in a multi-plane scene, matching points on the same plane should be close in spatial position, and the geometric error and pixel displacement should be similar, while mismatched points often show inconsistency. K-Means clustering is based on this principle: the number of clusters K is preset, and K initial cluster centers are randomly selected. For each eigenvector , calculate its Euclidean distance to the cluster center:

[0070] (8)

[0071] Assign it to the nearest cluster:

[0072] (9)

[0073] Update cluster centers:

[0074] (10)

[0075] Iterate until the center is stable or the maximum number of iterations is reached. After clustering, check the number of points in each cluster ,like , it is considered an isolated cluster (the number of points is insufficient to estimate the fundamental matrix), and the points in it are marked as mismatched points and removed. K-Means minimizes the intra-cluster variance, ensuring that the matching points in the same cluster are highly consistent in spatial features, reflecting intra-class spatial consistency.

[0076] S6. Output: The set of matching points after K-Means clustering, excluding isolated clusters, is taken as the final optimization result and directly output for subsequent tasks (such as 3D reconstruction and image stitching). This method retains high-quality matching points while eliminating mismatched points through intra-cluster spatial consistency clustering.

[0077] In summary, compared with the prior art, the present invention has the following advantages:

[0078] 1. Dynamic matching point screening: The quartile method is introduced to dynamically screen matching points. Compared with the fixed threshold method, it is more robust and can adapt to the error distribution in different scenarios.

[0079] 2. Intra-class spatial consistency clustering: Through K-Means clustering, the intra-class consistency of spatial position, geometric error, and pixel displacement is separated into substructures, and mismatched points in isolated clusters are eliminated, significantly improving the matching accuracy in complex scenarios.

[0080] 3. High computational efficiency: By directly eliminating isolated clusters and outputting the results, the algorithm process is simplified and the computational overhead is reduced.

[0081] 4. Improved robustness: Through the quartile method and intra-class spatial consistency clustering, the impact of noise and complex scenes on matching accuracy is reduced.

[0082] Although the present invention has been described above with reference to embodiments, various modifications may be made thereto and equivalent components may be substituted without departing from the scope of the present invention. In particular, as long as there are no structural conflicts, the various features of the embodiments disclosed herein may be combined with each other in any manner, and the omission of an exhaustive description of such combinations in this specification is solely for the sake of space and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. An image feature matching optimization method based on intra-class spatial consistency, characterized in that: Here are the steps: S1. Feature point extraction and preliminary matching: Extract feature points from the query image and reference image and perform preliminary matching; S2, initialization and transformation model estimation: Initialize the matching point set and residual, and calculate the initial transformation model; S3, error calculation and matching point set update: calculate the error of matching point pairs and update the matching point set using dynamic screening method; S4, residual optimization: Based on the residual, determine whether the optimal state is reached and decide whether to continue iteration; S5. Intra-class spatial consistency clustering and isolated cluster elimination: After obtaining the optimal residual, cluster analysis is performed to separate substructures based on intra-class spatial consistency and eliminate isolated clusters; S6. Output result: Output the set of matching points after removing isolated clusters; In the error calculation and matching point set update step, the epipolar geometric error e of the matching point pair is calculated. i , the formula is: e i =|p i′ ·F·p i |; Among them, p i =(x i ,y i ,1) and p i′ =(x i′ ,y i′ ,1) is the homogeneous coordinates of the matching point pair; Introduce the quartile method to dynamically select the optimal points and calculate the error set {e i The quartiles Q1 and Q3, and the interquartile range IQR = Q3-Q1; Set dynamic upper and lower thresholds: lower limit = Q1-1.5×IQR, upper limit = Q3+1.5×IQR; If the error e of the matching point pair i If the point is within the dynamic upper and lower bounds, it will be added to the preferred point set. Otherwise, it will be marked as a candidate outlier. The preferred point set will be used to re-estimate the basic matrix F, and the optimization problem will be solved by the least squares method. The singular value decomposition of F will be modified, and the minimum singular value will be set to 0. In the intra-class spatial consistency clustering and isolated cluster elimination steps, multi-dimensional features are extracted from the current optimal point set to construct a 6-dimensional feature vector v i =(x i ,y i ,x i′ ,y i′ ,e i ,d i ), the multidimensional features include the pixel coordinates (x i ,y i ) and (x i ′,y i ′), geometric error e i =|p i′ ·F·p i | and pixel displacement Among them, |p i′ ·F·p i | is the geometric error of each matching point pair, and the residual r(t) reflects the degree of fit of the current basic matrix F to the preferred point set. The smaller the residual, the higher the model quality; The K-Means clustering method is used, the number of clusters K is preset, and K initial cluster centers {c1, c2, ..., c K }, for each eigenvector v i , calculate its Euclidean distance to the cluster center: The feature vector v i Assign to the nearest cluster: label(v i )=argmin k dist(v i ,c k ); Update cluster centers: After iterating until the center is stable or the maximum number of iterations is reached, check the number of points in each cluster |S k |, if the number of points is less than the set threshold, the cluster is regarded as an isolated cluster and the points in the isolated cluster are removed.

2. The image feature matching optimization method based on intra-class spatial consistency according to claim 1, characterized in that: In the feature point extraction and preliminary matching steps, the ORB algorithm is used to extract key points from the query image and the reference image, and the key points are stored in C k and R k , and perform feature point matching based on the Hamming distance to obtain the initial matching set M. The Hamming distance formula is: Among them, d i ∈{0,1} L and d i′ ∈{0,1} L are the descriptors of the query image and the reference image, respectively. Represents binary XOR operation, L is the descriptor length, and the smaller the Hamming distance, the higher the matching similarity.

3. The image feature matching optimization method based on intra-class spatial consistency according to claim 2, characterized in that: In the initialization and transformation model estimation step, all matching point pairs in the initial matching set M are regarded as candidate point sets, the initial preferred point set is empty, and the residual r(0) is initialized to +∞. Eight pairs of matching points are randomly selected from the matching set M, and the initial basic matrix F is calculated using the 8-point algorithm. The basic matrix F satisfies the epipolar constraint: p i′ ·F·p i =0; By constructing the coefficient matrix A and solving the linear equation system A·vec(F)=0, the vectorized form vec(F) of the basic matrix F is obtained. Then, the initial solution is subjected to singular value decomposition, the minimum singular value is set to 0, and the basic matrix F is reconstructed.

4. The image feature matching optimization method based on intra-class spatial consistency according to claim 3, characterized in that: In the residual optimization step, the average residual of the optimal point set is calculated: Among them, N 优选点 is the number of points in the optimal point set; The calculated residual r(t) is compared with the current optimal residual r best For comparison, r best The initial value is r(0) = +∞, if the current residual r(t) < r best , indicating that the current model is better than the previous model, so the optimal residual r is updated best = r(t) and save the current basic matrix F and the optimal point set, then return to the initialization and transformation model estimation step to continue the iteration. Conversely, if r(t) ≥ r best This indicates that the current model has not brought significant improvement, so the iteration is stopped and the next step of cluster analysis is entered.

Images