An image feature matching method based on affine transformation and ORB

Abstract

The application discloses an image feature matching method based on affine transformation and ORB, and particularly relates to the technical field of image processing, and comprises the following steps: step one: constructing an affine attention module, extracting the optimal affine transformation between images, and inserting the affine attention module into an ORB framework; step two: using the ORB framework to perform feature detection and description; step three: using FLANN to compare the descriptors to generate the corresponding relationship between the image pairs; and step four: using GMS to quickly distinguish true correspondences from false correspondences, and finally screening out correct mutual matching points from the corresponding relationship. In the feature description stage, the affine attention module is used to obtain the affine information between images to increase the robustness of feature description. In order to overcome the problem that the affine attention module increases the amount of calculation, in the feature matching stage, GMS is used to quickly screen out correct mutual matching points, so that the application has high matching precision and fast matching speed, and achieves the effect of high-quality feature matching.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to an image feature matching method based on affine transformation and ORB. Background Technology

[0002] Image feature matching, a core task in computer vision, aims to determine the correspondence between two images in the same scene, and is of great value for tasks such as SFM, SLAM, and real-time image stitching. Scene changes and real-time constraints make feature matching challenging: two images of the same scene may differ significantly due to changes in viewpoint, rotation, scaling, and lighting; some real-time applications not only demand higher matching accuracy but also impose strict limitations on matching time.

[0003] Methods for solving feature matching tasks typically follow a detection-description-matching step. Commonly used algorithms include SIFT, SURF, and ORB. ORB is an effective alternative to SIFT and SURF, improved upon the FAST and BRIEF algorithms. It can improve feature extraction speed while maintaining feature robustness, and simultaneously improve feature matching speed based on binary string Hamming distance, meeting the real-time requirements of feature matching. However, the descriptors obtained after ORB detection and description lack affine mutability. These descriptors are invariant to scene changes, only containing information such as lighting, viewpoint, rotation, and scaling equally, making matching difficult. Therefore, ORB actually reduces feature matching accuracy under scene changes. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention proposes an image feature matching method based on affine transformation and ORB. It employs an affine attention module, and based on the optimal affine transformation parameters between images, an encoder and decoder infuse the descriptors with affine information between the images, thereby enabling the descriptors to possess affine variability. Using the affine transformation information of two images as a condition, the invariance of constant descriptors is converted into variability, allowing the descriptors to be modified according to the differences between images. This increases the robustness of feature description, improves matching accuracy, and thus quickly finds the correct correspondence between images.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an image feature matching method based on affine transformation and ORB, the method comprising the following steps: Step 1: Construct an affine attention module, extract the optimal affine transformation between images, and insert it into the ORB framework; Step 2: Use the ORB framework for feature detection and description; Step 3: Use the FLANN comparison descriptor to generate correspondences between image pairs; Step 4: Using GMS, true and false correspondences can be quickly distinguished, and finally, the correct matching points can be selected from these correspondences; Specifically, step 1 is as follows: Step 1-1: Construct and initialize the affine transformation space parameter network using the template image; Step 1-2: Generate the corresponding affine transformation matrix based on the parameter network, use a random algorithm to estimate the SAD of the template image and the target region of the image to be matched after each affine transformation, and retain the affine transformation parameters and the corresponding affine transformation matrix of the SAD within the threshold range. Steps 1-3: The affine transformation parameter network is re-initialized using the branch-bound method and the SAD is estimated. The entire parameter network is refined from coarse to fine until certain conditions are met (usually the SAD at a local minimum) to obtain the optimal affine transformation parameters. Steps 1-4: Based on the optimal affine parameters, insert the image to be matched into the ORB frame using an encoder and decoder, so that the descriptor contains affine information between the images; The two-dimensional affine transformation space needs to take into account all possible two-dimensional affine transformations, such as translation, scaling, rotation and other affine transformations; It involves 6 parameters, which correspond to the translation between images. Rotation between images Image scaling ; The corresponding affine transformation matrix can be obtained from these parameters. and It is a rotation transformation matrix. It is a scaling transformation matrix, and its relevant definition is as follows: Given 6 affine transformation parameters, there is a 3×3 affine transformation matrix. : Given a precision of random sampling Perform sampling initialization, where Number of sampling points for Estimated template image With affine transformation Later in the image The normalized SAD between the corresponding pixels is expressed as: Step 4 is based on the assumption that each pixel in the matching image has smooth motion, there are many matching points around the correct matching point, and there are many incorrect matching points around the incorrect matching point. First, the matching image is divided into multiple grids, and the number of matching points in any grid is... , A binomial distribution can be represented as: In the formula, Matching points within the grid; For matching points Number of surrounding grids; This represents the number of matching points within the grid. Indicates matching point The probability of a correct match; Indicates matching point The probability of a mismatch point; express For a correct match; To improve the accuracy of the interior point selection algorithm, the problem of selecting the correct matching points is transformed into a numerical optimization problem, i.e., optimization. Mean and variance functions ; Optimization function Proportional to the number of grid cells and the number of feature points within the grid, i.e. ; Next, we can use equation (6) to determine whether the matching point in the current grid area is a correct matching point: In the formula, For grid area And the total number of matching points within the eight surrounding grids, For the threshold, A constant factor, for The total number of feature points in each matching grid region.

[0006] In a preferred embodiment, in step 2: ORB is an improved algorithm framework based on the FAST and BRIEF algorithms. ORB is an improved algorithm framework based on the FAST and BRIEF algorithms. First, the FAST algorithm is used to detect feature points and determine the direction of feature points. Then, the rBRIEF algorithm is used to generate rotation-invariant feature descriptors. Finally, Hamming distance is used to complete feature point matching. In terms of feature point detection, ORB uses the FAST algorithm for image feature detection, at the pixel level. With the center of the circle, A circle is drawn with a radius equal to 1. If there are more grayscale difference points on the circumference than a set threshold, then the point is considered a candidate point. For feature points; to achieve rotation invariance of feature points, ORB uses neighborhood moments to calculate the orientation of FAST feature points, defining the feature point orientation as the gray level of the feature point and the orientation of the offset vector between the centroids. In terms of feature point description, ORB uses rBRIEF, which is rotation invariant, for feature description. To achieve rotation invariance, the direction of the feature descriptor is added. Based on the feature point direction information obtained from feature detection, it is added to the descriptor to finally form a 128-dimensional feature descriptor with rotation invariance.

[0007] In a preferred embodiment, in step 3: first, the points in an image to be matched are... Find the point in the baseline image that has the closest metric distance to it. The distance between them is denoted as Next, iterate through all the initial matching point pairs between the image to be matched and the reference image, and find the smallest similarity metric distance among them. Set a threshold The value depends on the matching situation; if the point With point The minimum distance between them is less than the threshold. Then the matching point pair will be Keep it, otherwise match the point pair. Discard the match and apply the same process to match other points in the image.

[0008] Compared with the prior art, the technical effects and advantages of the present invention are as follows: An affine attention module is constructed. This module uses the optimal affine transformation parameters between images and an encoder-decoder to make the descriptor contain affine information between images, thereby increasing the robustness of feature description and improving feature matching accuracy. By analyzing and comparing existing feature matching methods, GMS is identified as the most suitable module for effective screening in the matching stage, overcoming the problem of computational consumption and thus improving the feature matching speed. Attached Figure Description

[0009] Figure 1 This is an overall block diagram of the present invention; Figure 2 The final matching image effect of the present invention Figure 1 ; Figure 3 The final matching image effect of the present invention Figure 2 . Detailed Implementation

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] according to Figure 1 The image feature matching method shown includes the following steps: Step 1: Construct an affine attention module, extract the optimal affine transformation between images, and insert it into the ORB framework; Step 2: Use the ORB framework for feature detection and description; Step 3: Use the FLANN comparison descriptor to generate correspondences between image pairs; Step 4: Using GMS, true and false correspondences can be quickly distinguished, and finally, the correct matching points can be selected from these correspondences; Specifically, step 1 is as follows: Step 1-1: Construct and initialize the affine transformation space parameter network using the template image; Step 1-2: Generate the corresponding affine transformation matrix based on the parameter network, use a random algorithm to estimate the SAD of the template image and the target region of the image to be matched after each affine transformation, and retain the affine transformation parameters and the corresponding affine transformation matrix of the SAD within the threshold range. Steps 1-3: Reinitialize the affine transformation parameter network using the branch-bound method and estimate the SAD. The entire parameter network is refined from coarse to fine until certain conditions are met (usually the SAD at a local minimum) to obtain the optimal affine transformation parameters. Steps 1-4: Based on the optimal affine parameters, insert the image to be matched into the ORB frame using an encoder and decoder, so that the descriptor contains affine information between the images; The two-dimensional affine transformation space needs to consider all possible two-dimensional affine transformations, such as translation, scaling, and rotation; it involves six parameters, each corresponding to a translation between images. Rotation between images Image scaling ; The corresponding affine transformation matrix can be obtained from these parameters. and It is a rotation transformation matrix. It is a scaling transformation matrix, and its relevant definition is as follows: Given 6 affine transformation parameters, there is a 3×3 affine transformation matrix. : Given a precision of random sampling Perform sampling initialization, where Number of sampling points for Estimate template image With affine transformation The normalized SAD between the corresponding pixels in the image is then expressed as: Step 4 is based on the assumption that each pixel in the matching image has smooth motion, there are many matching points around the correct matching point, and there are many incorrect matching points around the incorrect matching point. First, the matching image is divided into multiple grids, and the number of matching points in any grid is... , A binomial distribution can be represented as: In the formula, Matching points within the grid; For matching points Number of surrounding grids; This represents the number of matching points within the grid. Indicates matching point The probability of a correct match; Indicates matching point The probability of a mismatch point; express For a correct match; To improve the accuracy of the interior point selection algorithm, the problem of selecting the correct matching points is transformed into a numerical optimization problem, i.e., optimization. Mean and variance functions ; Optimization function Proportional to the number of grid cells and the number of feature points within the grid, i.e. ; Next, we can use equation (6) to determine whether the matching point in the current grid area is a correct matching point: In the formula, For grid area And the total number of matching points within the eight surrounding grids, For the threshold, A constant factor, for The total number of feature points in each matching grid region.

[0012] In a preferred embodiment, in step 2: ORB is an improved algorithm framework based on the FAST and BRIEF algorithms. ORB is an improved algorithm framework based on the FAST and BRIEF algorithms. First, the FAST algorithm is used to detect feature points and determine the direction of feature points. Then, the rBRIEF algorithm is used to generate rotation-invariant feature descriptors. Finally, Hamming distance is used to complete feature point matching. In terms of feature point detection, ORB uses the FAST algorithm for image feature detection, at the pixel level. With the center of the circle, A circle is drawn with a radius equal to 1. If there are more grayscale difference points on the circumference than a set threshold, then the point is considered a candidate point. For feature points; to achieve rotation invariance of feature points, ORB uses neighborhood moments to calculate the orientation of FAST feature points, defining the feature point orientation as the gray level of the feature point and the orientation of the offset vector between the centroids. In terms of feature point description, ORB uses rBRIEF, which is rotation invariant, for feature description. To achieve rotation invariance, the direction of the feature descriptor is added. Based on the feature point direction information obtained from feature detection, it is added to the descriptor to finally form a 128-dimensional feature descriptor with rotation invariance.

[0013] In a preferred embodiment, in step 3: first, the points in an image to be matched are... Find the point in the baseline image that has the closest metric distance to it. Let the distance between them be denoted as Next, iterate through all the initial matching point pairs between the image to be matched and the reference image, and find the smallest similarity metric distance among them. Set a threshold ( The value depends on the matching situation; if the point With point The minimum distance between them is less than the threshold. Then the matching point pair will be Keep it, otherwise match the point pair. Discard the match and apply the same process to match other points in the image.

[0014] In summary, this invention provides an image feature matching method based on affine transformation and ORB. An affine attention module is constructed, which, through an encoder-decoder, uses optimal affine transformation parameters between images to ensure that descriptors contain affine information between images, increasing the robustness of feature description and thus improving feature matching accuracy. By analyzing and comparing existing feature matching methods, GMS is identified as the most suitable module for effective filtering in the matching stage, overcoming the problem of computational overhead and thereby improving feature matching speed.

[0015] Finally, the following points should be noted: First, in the description of this application, it should be noted that, unless otherwise specified and limited, the terms "installation", "connection", and "linkage" should be interpreted broadly, and can be mechanical or electrical connections, or internal connections between two components, or direct connections. "Up", "down", "left", "right", etc. are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may change. Secondly: The accompanying drawings of the embodiments disclosed in this invention only involve the structures involved in the embodiments disclosed in this invention. Other structures can refer to the general design. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other. In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image feature matching method based on affine transformation and ORB, characterized in that: The method includes the following steps: Step 1: Construct an affine attention module, extract the optimal affine transformation between images, and insert it into the ORB framework; Step 2: Use the ORB framework for feature detection and description; Step 3: Use the FLANN comparison descriptor to generate correspondences between image pairs; Step 4: Using GMS, true and false correspondences can be quickly distinguished, and finally, the correct matching points can be selected from these correspondences; Specifically, step 1 is as follows: Step 1-1: Construct and initialize the affine transformation space parameter network using the template image; Step 1-2: Generate the corresponding affine transformation matrix based on the parameter network, use a random algorithm to estimate the SAD of the template image and the target region of the image to be matched after each affine transformation, and retain the affine transformation parameters and the corresponding affine transformation matrix of the SAD within the threshold range. Steps 1-3: The affine transformation parameter network is re-initialized using the branch-bound method and the SAD is estimated. The entire parameter network is refined from coarse to fine until the SAD that satisfies the local minimum is obtained to obtain the optimal affine transformation parameters. Steps 1-4: Based on the optimal affine parameters, insert the image to be matched into the ORB frame using an encoder and decoder, so that the descriptor contains affine information between the images; The two-dimensional affine transformation space includes translation, rotation, and scaling transformations, involving six parameters, each corresponding to a translation between images. Rotation between images Image scaling ; The corresponding affine transformation matrix can be obtained from these parameters. and It is a rotation transformation matrix. It is a scaling transformation matrix, and its relevant definition is as follows: Given 6 affine transformation parameters, there is a 3×3 affine transformation matrix. : Given a precision of random sampling Perform sampling initialization, where The number of sampling points is Estimated template image With affine transformation Later in the image The normalized SAD between the corresponding pixels is expressed as: Step 4 is based on the motion smoothness assumption: each pixel in the matching image has motion smoothness; First, the matching image is divided into multiple grids, and the number of matching points in any grid is... , A binomial distribution can be represented as: In the formula, Matching points within the grid; This represents the number of matching points within the grid. Indicates matching point The probability of a correct match; Indicates matching point The probability of a mismatch point; express To ensure correct matching, and to increase the accuracy of the interior point selection algorithm, the problem of correctly matching point selection is transformed into a numerical optimization problem. Next, we can use equation (6) to determine whether the matching point in the current grid area is a correct matching point: In the formula, Grid area And the total number of matching points within the eight surrounding grids, For the threshold, A constant factor, for The total number of feature points in each matching grid region.

2. The image feature matching method based on affine transformation and ORB according to claim 1, characterized in that: In step 2: ORB is an improved algorithm framework based on the FAST and BRIEF algorithms. First, the FAST algorithm is used to detect feature points and determine the orientation of the feature points. Then, the rBRIEF algorithm is used to generate rotation-invariant feature descriptors. Finally, Hamming distance is used to complete feature point matching. In terms of feature point detection, ORB uses the FAST algorithm for image feature detection, at the pixel level. With the center of the circle, A circle is drawn with a radius equal to 1. If there are more grayscale difference points on the circumference than a set threshold, then the point is considered a candidate point. For feature points; to achieve rotation invariance of feature points, ORB uses neighborhood moments to calculate the orientation of FAST feature points, defining the feature point orientation as the gray level of the feature point and the orientation of the offset vector between the centroids. In terms of feature point description, ORB uses rBRIEF, which is rotation invariant, for feature description. To achieve rotation invariance, the direction of the feature descriptor is added. Based on the feature point direction information obtained from feature detection, it is added to the descriptor to finally form a 128-dimensional feature descriptor with rotation invariance.

3. The image feature matching method based on affine transformation and ORB according to claim 1, characterized in that: In step 3: First, the points in an image to be matched... Find the point in the baseline image that has the closest metric distance to it. The distance between them is denoted as Next, iterate through all the initial matching point pairs between the image to be matched and the reference image, and find the smallest similarity metric distance among them. Set a threshold If point With point The minimum distance between them is less than the threshold. Then the matching point pair will be Keep it, otherwise match the point pair. Discard the match and apply the same process to match other points in the image.

Images