Image feature matching optimization method based on intra-class space consistency

Abstract

The invention discloses an image feature matching optimization method based on intra-class space consistency in the technical field of computer vision and image processing. The method comprises the following steps: feature point extraction and preliminary matching: extracting feature points from a query image and a reference image and performing preliminary matching; initialization and transformation model estimation: initializing a matching point set and a residual error, and calculating an initial transformation model; error calculation and matching point set updating: calculating the error of the matching point pair, and updating the matching point set by adopting a dynamic screening method; performing residual optimization: judging whether the optimal condition is reached or not based on the residual, and deciding whether to continue iteration or not; performing intra-class space consistency clustering and isolated cluster elimination: performing clustering analysis after the optimal residual error is obtained, and eliminating isolated clusters based on an intra-class space consistency separation ion structure; and outputting a result: outputting a matching point set after the isolated clusters are removed. The method solves the problem that a traditional feature matching method is difficult to completely remove mismatching in a complex scene and is sensitive to noise.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and image processing, and particularly to an optimization method for image feature matching based on intra-class spatial consistency. Background Art

[0002] Image feature matching is one of the fundamental tasks in the field of computer vision. Its core objective is to achieve precise alignment and scene understanding between images by extracting significant feature points in two images and establishing corresponding relationships. Traditional methods mainly rely on the extraction of local feature descriptors and similarity measurement. For example, key points are detected through algorithms such as Scale-Invariant Feature Transform (SIFT) or Oriented FAST and Rotated BRIEF (ORB), and descriptors with rotation and scale invariance are generated. Based on the similarity of the descriptors, initial matching point pairs can be screened to form an initial set of corresponding relationships.

[0003] In practical applications, due to factors such as noise interference, lighting changes, perspective differences, and target object movement during image acquisition, the feature point matching results inevitably contain mismatched points. To improve the robustness of the matching, existing technologies usually adopt an iterative optimization strategy. By randomly sampling a small number of matching points to construct a geometric model (such as a fundamental matrix or a homography matrix), and verifying the consistency of other matching points with this model, the abnormal points that do not conform to the geometric constraints are gradually eliminated. Such methods can improve the matching accuracy to a certain extent by iteratively optimizing the model parameters.

[0004] However, with the complexity of application scenarios (such as multiple moving targets, dynamic backgrounds, large-scale perspective changes, etc.), traditional feature matching methods face new challenges. For example: 1. Traditional algorithms rely on the epipolar constraint model, but in the case of pure camera rotation or uneven feature point distribution, mismatched points may still exist. For example, when the camera performs pure rotation, the pixel displacement of feature points is independent of depth, resulting in the inability of traditional algorithms to effectively eliminate mismatched points. 2. Traditional algorithms eliminate mismatched points globally, lacking optimization of local feature points, resulting in insufficient matching accuracy. Especially in the case of uneven feature point distribution, global optimization is difficult to meet the matching requirements of local feature points. 3. Traditional methods usually use a fixed threshold to screen matching points. When the image noise is large or the feature point distribution is uneven, the fixed threshold is difficult to adapt, resulting in a decline in the quality of the matching point set. Summary of the Invention

[0005] The purpose of this part is to outline some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Simplifications or omissions may be made in this part, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this part, the abstract, and the title, and such simplifications or omissions shall not be used to limit the scope of the present invention.

[0006] Therefore, an object of the present invention is to provide an optimization method for image feature matching based on intra-class spatial consistency to solve the problems that it is difficult to completely eliminate false matches and the method is sensitive to noise in complex scenarios in traditional feature matching methods.

[0007] To solve the above technical problems, according to one aspect of the present invention, the following technical solutions are provided: An optimization method for image feature matching based on intra-class spatial consistency, the steps are as follows: S1. Feature point extraction and preliminary matching: Extract feature points from the query image and the reference image and perform preliminary matching; S2. Initialization and transformation model estimation: Initialize the matching point set and the residual, and calculate the initial transformation model; S3. Error calculation and matching point set update: Calculate the error of the matching point pairs, and update the matching point set by using a dynamic screening method; S4. Residual optimization: Based on the residual, determine whether the optimal state is reached and decide whether to continue iteration; S5. Intra-class spatial consistency clustering and isolated cluster elimination: After obtaining the optimal residual, perform clustering analysis, separate sub-structures based on intra-class spatial consistency, and eliminate isolated clusters; S6. Output result: Output the matching point set after eliminating isolated clusters.

[0008] As a preferred solution of the optimization method for image feature matching based on intra-class spatial consistency according to the present invention, in the step of feature point extraction and preliminary matching, the ORB algorithm is used to extract key points from the query image and the reference image, and they are respectively stored in C k and R k , and feature point matching is performed based on the Hamming distance to obtain the initial matching set M. The Hamming distance formula is:

[0009] where and are the descriptors of the query image and the reference image respectively, ⊕ represents the binary exclusive OR operation, is the descriptor length, and the smaller the Hamming distance, the higher the matching similarity.

[0010] As a preferred solution of the optimization method for image feature matching based on intra-class spatial consistency according to the present invention, in the step of initialization and transformation model estimation, all matching point pairs in the initial matching set M are regarded as the candidate point set, the preferred point set is initialized to be empty, and at the same time the residual is initialized. Randomly select 8 pairs of matching points from the matching set M, and use the 8-point algorithm to calculate the initial fundamental matrix , and the fundamental matrix Satisfy the epipolar constraint: ; Wherein, and are the homogeneous coordinates of the matching point pairs. By constructing the coefficient matrix , solve the linear equation system to obtain the vectorized form of the fundamental matrix . Then perform singular value decomposition on the initial solution, set the minimum singular value to 0, and reconstruct the fundamental matrix .

[0011] As a preferred solution of an image feature matching optimization method based on intra-class spatial consistency according to the present invention, wherein, in the error calculation and matching point set update step, calculate the epipolar geometric error of the matching point pairs, and the formula is: ; Introduce the quartile method to dynamically screen and select the preferred points, and calculate the quartiles of the error set and , as well as the interquartile range ; Set the dynamic upper and lower bound thresholds: ; If the error of the matching point pair is within the dynamic upper and lower bound thresholds, add it to the preferred point set, otherwise mark it as a candidate outlier, re-estimate the fundamental matrix F using the preferred point set, solve the optimization problem by the least squares method, and perform singular value decomposition correction on F, setting the minimum singular value to 0.

[0012] As a preferred solution of an image feature matching optimization method based on intra-class spatial consistency according to the present invention, wherein, in the residual optimization step, calculate the average residual of the preferred point set: ; wherein, is the number of points in the preferred point set, is the geometric error of each matching point pair, and the residual reflects the fitting degree of the current fundamental matrix to the preferred point set. The smaller the residual, the higher the model quality; Compare the calculated residual with the current optimal residual , wherein the initial value is . If the current residual , it indicates that the current model is better than the previous model. Therefore, update the optimal residual and save the current fundamental matrix and the optimal point set. Subsequently, return to the initialization and transformation model estimation steps for continued iteration. Conversely, if it indicates that the current model has not brought significant improvement, stop the iteration, and enter the next clustering analysis step.

[0013] As a preferred solution of the image feature matching optimization method based on intra-class spatial consistency described in the present invention, in the intra-class spatial consistency clustering and isolated cluster elimination step, extract multi-dimensional features from the current optimal point set to construct a 6-dimensional feature vector , where the multi-dimensional features include the pixel coordinates of the matching point pairs in the two images and geometric error and pixel displacement ; Adopt the K-Means clustering method, preset the number of clusters K, and randomly select K initial clustering centers , for each feature vector , calculate its Euclidean distance from the clustering center:

[0014] Assign the feature vector to the nearest cluster: ; Update the clustering center: ; Iterate until the center is stable or the maximum number of iterations is reached, then check the number of points in each cluster , if the number of points is less than the set threshold, regard this cluster as an isolated cluster and eliminate the points in the isolated cluster.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention dynamically screens feature points using the quartile method and eliminates false matches through K-Means clustering analysis based on intra-class spatial consistency, thereby solving the limitations of traditional methods in complex scenarios. This method uses epipolar geometry to establish the projection relationship constraints between feature points, effectively filtering out abnormal points that violate the epipolar geometry principle; uses the clustering algorithm to analyze the spatial distribution consistency of feature points and eliminates false matching points in isolated classes. On the basis of global optimization, further optimize local feature points, improve the matching accuracy, and finally reduce the influence of noise, etc. on the matching accuracy. Description of the Drawings

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings. Among them: Figure 1 This is a flowchart of an image feature matching optimization method based on intra-class spatial consistency of the present invention. Specific embodiments

[0017] To make the above objects, features, and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

[0018] As Figure 1 shown, the present invention provides an image feature matching optimization method based on intra-class spatial consistency, and the steps are as follows: S1. Feature point extraction and preliminary matching: Extract feature points from the query image and the reference image and perform preliminary matching; S2. Initialization and transformation model estimation: Initialize the matching point set and the residual, and calculate the initial transformation model; S3. Error calculation and matching point set update: Calculate the error of the matching point pairs, and update the matching point set using a dynamic screening method; S4. Residual optimization: Based on the residual, determine whether the optimal state is reached and decide whether to continue iteration; S5. Intra-class spatial consistency clustering and isolated cluster elimination: After obtaining the optimal residual, perform clustering analysis, separate the sub-structures based on intra-class spatial consistency, and eliminate the isolated clusters; S6. Output result: Output the set of matching points after eliminating the isolated clusters.

[0019] To describe the technical solutions of the present invention in more detail, each step will be further described in detail below, specifically: S1. Feature point extraction and preliminary matching: Use the ORB algorithm (Oriented FAST and Rotated BRIEF) to extract key points from the query image and the reference image, and store them in vectors C k and R k respectively. ORB generates feature points through FAST corner detection and BRIEF binary descriptors, and has the characteristics of rotation invariance and high computational efficiency. Feature point matching is performed based on the Hamming Distance to obtain the initial matching set M. The Hamming Distance is defined as: (1) Where and They are the descriptors of the query image and the reference image respectively. ⊕ represents the binary exclusive OR operation. is the descriptor length (usually 256 bits for ORB). The smaller the Hamming distance, the higher the matching similarity.

[0020] S2. Initialization and transformation model estimation: The goal of this step is to lay the foundation for subsequent optimization, initialize the set of matching points and estimate the initial transformation model. First, all the matching point pairs in the initial matching set M are regarded as the candidate point set for subsequent screening of high-quality matching points. At this time, the preferred point set (high-quality matching point set) is initialized to be empty, indicating that no qualified matching points have been screened out yet. At the same time, the residual is initialized as the benchmark for subsequent iterative optimization. The initial value of the residual is set to positive infinity to ensure that the residual calculated subsequently can gradually decrease, so as to find a better model. To estimate the initial transformation model, 8 pairs of matching points are randomly selected from the matching set M, and the classical 8-point algorithm is used to calculate the initial fundamental matrix . The fundamental matrix satisfies the epipolar constraint: (2) where and are the homogeneous coordinates of the matching point pair. The specific implementation of the 8-point algorithm is to construct the coefficient matrix , where each row corresponds to a pair of matching points, in the form of . By solving the linear equation system , the vectorized form of the fundamental matrix is obtained. To ensure that satisfies the rank constraint of the epipolar geometry (i.e., ), the initial solution is subjected to singular value decomposition (SVD), the smallest singular value is set to 0, and the fundamental matrix is reconstructed. This initial transformation model provides the basis for subsequent error calculation and optimization. Although the initial estimate may contain many mismatched points, the model quality can be gradually improved through the iterative optimization in subsequent steps.

[0021] S3. Error calculation and update of the matching point set: The goal of this step is to update the preferred point set and improve the quality of the matching points through error calculation and dynamic screening. First, for all matching point pairs , calculate their epipolar geometry error to measure the degree of their deviation from the current fundamental matrix . The error is defined as: (3) This error reflects whether the matching point pairs satisfy the epipolar geometry constraint. The smaller the error, the more the matching point pairs conform to the current model. To avoid the limitations of a fixed threshold, the present invention introduces the quartile method to dynamically screen and select optimal points. Specifically, for the error set of all matching point pairs statistical analysis is performed to calculate its quartiles: (4) where, is the 25th percentile, is the 75th percentile, is the interquartile range, which is used to measure the dispersion of the error distribution. Based on the quartile method, dynamic upper and lower bound thresholds are set: (5) For each matching point pair, if its error ∈[lower bound, upper bound], it is added to the set of optimal points, indicating that the point pair conforms to the geometric constraints of the current model; otherwise, it is marked as a candidate outlier and temporarily excluded. Compared with the traditional fixed-threshold method, the quartile method can adaptively adjust the threshold to adapt to the error distribution in different scenarios and improve the robustness of the screening. After screening out the set of optimal points, these points are used to re-estimate the fundamental matrix , and the optimization problem is solved by the least squares method: (6) To satisfy the rank constraint, singular value decomposition (SVD) is used to correct , and the smallest singular value is set to 0 to ensure that the fundamental matrix meets the requirements of the epipolar geometry. This process gradually improves the quality of the matching point set through dynamic screening and model optimization, providing a more reliable input for subsequent steps.

[0022] S4. Residual optimization: The goal of this step is to evaluate the quality of the current model through residuals and decide whether to continue iterative optimization to obtain a better matching point set and transformation model. Based on the current set of optimal points, the average residual is calculated as a measure of the model quality. The residual is defined as the average of the geometric errors of all matching point pairs within the set of optimal points: (7) where, is the number of points in the set of optimal points, is the geometric error of each matching point pair, and the residual reflects the fitting degree of the current fundamental matrix to the set of optimal points. The smaller the residual, the higher the model quality. The calculated residual is compared with the current optimal residual , where the initial value is , if the current residual , indicating that the current model is superior to the previous model, so the optimal residual is updated and save the current fundamental matrix and the preferred point set. Subsequently, return to step S2, reselect the matching point pairs, estimate the new fundamental matrix, and continue the iterative optimization. Conversely, if it indicates that the current model has not brought significant improvement, stop the iteration, and enter the next clustering analysis.

[0023] The iterative process of residual optimization ensures that the fundamental matrix gradually approaches the optimal solution, and at the same time, the quality of the preferred point set is continuously improved. To avoid meaningless iterations, the maximum number of iterations (such as 1000 times) or the residual change threshold (such as stopping when less than 0.001) can be set to balance the computational efficiency and model accuracy. This step provides high-quality input data for the subsequent within-class spatial consistency clustering through global optimization.

[0024] S5. Within-class Spatial Consistency Clustering and Isolated Cluster Elimination: In this step, the K-Means clustering method is used to separate potential substructures based on within-class spatial consistency and eliminate the mis-matched points in the isolated clusters. First, extract multi-dimensional features from the current preferred point set for clustering analysis. The features include spatial position and , representing the pixel coordinates of the matching point pairs in the two images; geometric error , reflecting the geometric consistency of the matching point pairs; pixel displacement , representing the displacement between the matching point pairs in the two images. Combining these features, construct a 6-dimensional feature vector .

[0025] Within-class spatial consistency means that the matching points under the same geometric transformation relationship should be highly consistent in terms of spatial position, geometric error, and pixel displacement. For example, in a multi-plane scene, the matching points on the same plane should be adjacent in spatial position, and the geometric error and pixel displacement should be similar, while the mis-matched points often show inconsistencies. The K-Means clustering groups based on this principle: preset the number of clusters K and randomly select K initial clustering centers . For each feature vector , calculate its Euclidean distance from the clustering center: (8) Assign it to the nearest cluster: (9) Update the clustering center: (10) Iterate until the center is stable or the maximum number of iterations is reached. After clustering, check the number of points in each cluster , if , it is regarded as an isolated cluster (the number of points is insufficient to estimate the fundamental matrix), and the points in it are marked as mismatched points and removed. K-Means ensures that the matching points within the same cluster are highly consistent in spatial features by minimizing the within-cluster variance, reflecting the within-class spatial consistency.

[0026] S6. Output result: The set of matching points after removing the isolated clusters by K-Means clustering is used as the final optimized result and directly output for subsequent tasks (such as 3D reconstruction, image stitching). This method clusters based on within-class spatial consistency, retains a high-quality set of matching points, and at the same time removes the mismatched points.

[0027] In summary, compared with the prior art, the present invention has the following advantages: 1. Dynamic matching point screening: The quartile method is introduced to dynamically screen matching points, which is more robust than the fixed threshold method and can adapt to the error distribution in different scenarios.

[0028] 2. Within-class spatial consistency clustering: Through K-Means clustering, the sub-structures are separated based on the within-class consistency of spatial position, geometric error, and pixel displacement, and the mismatched points in the isolated clusters are removed, significantly improving the matching accuracy in complex scenarios.

[0029] 3. High computational efficiency: By directly removing the isolated clusters and outputting the results, the algorithm process is simplified and the computational overhead is reduced.

[0030] 4. Improved robustness: Through the quartile method and within-class spatial consistency clustering, the influence of noise and complex scenarios on the matching accuracy is reduced.

[0031] Although the present invention has been described above with reference to the embodiments, various improvements can be made to it and components therein can be replaced with equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the disclosed embodiments of the present invention can be combined with each other in any way. The exhaustive description of these combinations is omitted in this specification only for the sake of saving space and resources. Therefore, the present invention is not limited to the specific embodiments disclosed in the text, but includes all technical solutions falling within the scope of the claims.

Claims

1. An image feature matching optimization method based on intra-class spatial consistency, characterized in that The steps are as follows: S1. Feature point extraction and preliminary matching: Extract feature points from the query image and the reference image and perform preliminary matching; S2. Initialization and transformation model estimation: Initialize the set of matching points and the residuals, and calculate the initial transformation model; S3. Error calculation and update of the set of matching points: Calculate the errors of the matching point pairs, and update the set of matching points using a dynamic screening method; S4. Residual optimization: Based on the residuals, determine whether the optimal state has been reached and decide whether to continue the iteration; S5. Intra-class spatial consistency clustering and elimination of isolated clusters: After obtaining the optimal residuals, perform clustering analysis, separate the sub-structures based on intra-class spatial consistency, and eliminate the isolated clusters; S6. Output result: Output the set of matching points after eliminating the isolated clusters.

2. The optimized method for image feature matching based on intra-class spatial consistency according to claim 1, characterized in that In the feature point extraction and preliminary matching step, the ORB algorithm is used to extract key points from the query image and the reference image, and they are stored in C k and R k , respectively. Feature point matching is performed based on the Hamming distance to obtain the initial matching set M. The Hamming distance formula is as follows: ； wherein, and are descriptors of a query image and a reference image respectively, ⊕ represents a binary exclusive OR operation, is the descriptor length, and the smaller the Hamming distance, the higher the matching similarity.

3. An image feature matching optimization method based on intra-class spatial consistency according to claim 1, characterized in that, In the initialization and transformation model estimation step, all the matching point pairs in the initial matching set M are regarded as the candidate point set, the preferred point set is initialized to be empty, and the residual is initialized at the same time. Eight pairs of matching points are randomly selected from the matching set M, and the initial fundamental matrix is calculated using the eight-point algorithm. The fundamental matrix satisfies the epipolar constraint: ; Among them, and are the homogeneous coordinates of the matching point pairs. By constructing the coefficient matrix , solving the linear equations to obtain the vectorized form of the fundamental matrix , then performing singular value decomposition on the initial solution, setting the smallest singular value to 0, and reconstructing the fundamental matrix .

4. An optimization method for image feature matching based on intra-class spatial consistency according to claim 1, characterized in that In the error calculation and matching point set update step, calculate the epipolar geometric error of the matching point pairs , and the formula is: ； Introduce the quartile method to dynamically screen and optimize points, and calculate the error set quartiles of and , as well as the interquartile range ; Set dynamic upper and lower bound thresholds: ; If the error of the matching point pair is within the dynamic upper and lower bound thresholds, it is added to the preferred point set; otherwise, it is marked as a candidate outlier. The fundamental matrix F is re - estimated using the preferred point set, the optimization problem is solved by the least - squares method, and F is corrected by singular value decomposition, with the smallest singular value set to 0.

5. An optimization method for image feature matching based on intra-class spatial consistency according to claim 1, characterized in that, In the residual optimization step, calculate the average residual of the preferred point set: ； Among them, is the number of points in the optimal point set, is the geometric error of each matching point pair, the residual reflects the fitting degree of the current fundamental matrix to the optimal point set. The smaller the residual, the higher the model quality; The calculated residual is compared with the current optimal residual , where the initial value is . If the current residual , it indicates that the current model is better than the previous model. Therefore, update the optimal residual and save the current fundamental matrix and the preferred point set. Subsequently, return to the initialization and transformation model estimation steps to continue the iteration. Otherwise, if it indicates that the current model has not brought significant improvement, stop the iteration, and enter the next clustering analysis.

6. A method for optimizing image feature matching based on intra-class spatial consistency according to claim 1, characterized in that In the in-class space consistency clustering and isolated cluster elimination steps, multi-dimensional features are extracted from the current preferred point set to construct a 6-dimensional feature vector , where the multi-dimensional features include the pixel coordinates of the matching point pairs in the two images and geometric error and pixel displacement ; Using the K-Means clustering method, preset the number of clusters K, and randomly select K initial cluster centers , for each feature vector , calculate its Euclidean distance from the cluster center: ； Assign the feature vector to the nearest cluster: ； Update the clustering center: ； Iterate until the center is stable or the maximum number of iterations is reached, and then check the number of points in each cluster , if the number of points is less than the set threshold, then consider this cluster as an isolated cluster and remove the points in the isolated cluster.

Images