Image matching method based on improved SURF features

Abstract

The application relates to the technical field of digital image processing, in particular to an image matching method based on improved SURF features. The method comprises the following steps: inputting a reference image and a to-be-matched image; detecting feature points with scale invariance in the reference image and the to-be-matched image respectively through a SURF algorithm; performing feature description on the feature points in the reference image and the to-be-matched image respectively through a DAISY descriptor; completing initial matching according to the Euclidean distance between two feature points; and eliminating the mis-matching points in the initial matching process according to a RANSAC algorithm. The method can effectively improve the accuracy and speed of image feature matching, reduce the rate of error matching, and retain higher-quality feature points, so as to facilitate subsequent image splicing, image fusion, three-dimensional reconstruction and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to an image matching method based on improved SURF features. BACKGROUND

[0002] Image matching technology is the basis of many computer vision field problem research, and image local feature matching occupies a crucial position in the field of image matching. The local feature descriptor of the image is the core part in the image feature extraction and matching process. At present, most of the methods for image local feature matching are based on feature point local feature matching. Among them, SURF algorithm is proposed by Bay et al. based on scale invariant feature transform (SIFT). Through the simplification of the Hessian determinant algorithm, the operation speed of SURF algorithm is greatly improved, but its accuracy is decreased, and the rotation invariance is poor. DAISY descriptor is a local feature descriptor for wide baseline stereo matching proposed by Engin Tola et al. Because of the structure of the descriptor, it is easy to obtain rotation invariance, and the matching effect is good. Therefore, the image matching technology based on SURF has been a research hotspot. SUMMARY

[0003] The purpose of the present application is to solve the problems of high false matching rate and low matching accuracy when extracting feature points in image feature matching based on SURF algorithm. Based on the feature point detection of SURF algorithm, the DAISY feature descriptor is used to replace the original SURF feature description operator, and the RANSAC algorithm is used to delete the false matching points, so as to improve the performance of the matching algorithm. The technical scheme is as follows:

[0004] An image matching method based on improved SURF features, comprising the following steps:

[0005] Step 1, input the reference image and the image to be matched; pre-process the image, which includes image graying and filtering processing, and then perform normalization processing;

[0006] Step 2, detect the feature points with scale invariance in the reference image and the image to be matched respectively by SURF algorithm;

[0007] Step 3, describe the feature points in the reference image and the image to be matched respectively by DAISY descriptor;

[0008] Step 4, match the descriptors of the feature points in the reference image and the descriptors of the feature points in the image to be matched to complete the initial matching;

[0009] Step 5, eliminate the false matching points in the initial matching process by using RANSAC algorithm;

[0010] Step 6, generate the matching relationship of the feature points in the reference image and the image to be matched.

[0011] Wherein, the step 2 includes detecting the feature points by using Hessian matrix determinant, for the point with image scale δ on the image P, the definition of Hessian matrix of the point is:

[0012]

[0013] Wherein, L xx (a,δ), L xy (a,δ) and L yy (a,δ) are the convolutions of the second-order partial derivatives of Gauss and the image P at the point , in order to improve the operation speed, the box filter is used, and the values after convolution operation are D xx , D xy , D yy , in order to balance the error caused by the approximation of the box filter, a weighted coefficient is multiplied on D xy , the value of the weighted coefficient is usually 0.9, therefore, the approximate value of the Hessian matrix determinant of each pixel is:

[0014] Det(H)=D xx D yy -(0.9D xy ) 2

[0015] The step 3 includes, first, assigning the principal direction to have rotation invariance, second, establishing the descriptor, selecting the direction in 45° units, extracting the maximum gradient histogram of the second layer of concentric circles, thereby obtaining the DAISY descriptor with rotation invariance.

[0016] The specific method is: taking the feature point to be described as the center, constructing three concentric circles with different radii outward, the Gaussian scale of each layer is the same, the Gaussian scale increases layer by layer outward, and a sampling point is taken every 45° in the range of 360°, so that 3×8+1=25 sampling points are obtained. Taking the pixel point (c, d) as an example, the gradient of the pixel point in 8 directions is represented by (o=1,2,3…,8 represents the direction of the gradient; (·) + represents (α) + =max(α,0)) represents. Then, the gradient image G o (c,d) is subjected to Gaussian convolution, and the formula is represented as Wherein G∑ represents a Gaussian kernel with a Gaussian scale value ∑, then a pixel point (c, d) obtains a vector with a length of 8, that is, the local gradient direction histogram of the pixel point (c, d) can be expressed as:

[0017]

[0018] Therefore, the formula of the DAISY descriptor is:

[0019]

[0020] In the formula, P=1, 2, 3 represents the structure layer number; Q=1, 2, …, 8 represents the direction of each layer division, l r (c,d,R k ) represents the position of the rth sampling point on the kth concentric ring with (c, d) as the structure center, and the local gradient direction histogram of the sampling point. Therefore, the descriptor feature vector dimension of the pixel point (c, d) is (8*3+1)*8=200.

[0021] The step 4 is to determine the reference image descriptor obtained in the step 3 and the image descriptor to be matched by using the Euclidean distance between the feature points, compare with a given threshold, exclude the feature points exceeding the given threshold, and complete the initial matching.

[0022] The step 5 is to eliminate the false matching points by using the RANSAC algorithm on the basis of the initial matching relationship between the two images obtained in the step 4, to avoid the one-to-many false matching phenomenon, and specifically includes:

[0023] (1) randomly extract 4 pairs of matching pairs from the data set as samples, calculate the homography matrix by using the formula, the model is denoted as P1, and a threshold is set.

[0024] (2) calculate the error between all data in the data set and the model P1, if the error is less than the set threshold, then the data is regarded as an inlier, and is added to the inlier set Q1.

[0025] (3) if enough points are regarded as inliers, then select the model P1 at this time, otherwise repeat the above steps until the requirement is met, select the model with the largest inlier ratio as the solution of the problem.

[0026] Compared with the prior art, the above technical scheme has the following technical effects:

[0027] The image matching method based on the improved SURF feature provided by the embodiment of the present application can effectively improve the accuracy and speed of image feature matching, reduce the rate of false matching, and retain higher quality feature points, so as to facilitate subsequent image stitching, image fusion, three-dimensional reconstruction and the like. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 Flow chart of the method of the present application

[0029] Figure 2 Block diagram of the DAISY descriptor structure of the present application

[0030] Figure 3 Table of matching data results of the method of the present application and the classic SURF algorithm under rotation angle change

[0031] Figure 4 Table of matching data results of the method of the present application and the classic SURF algorithm under image blur change

[0032] Figure 5 Table of matching data results of the method of the present application and the classic SURF algorithm under image JPEG compression ratio change

[0033] Figure 6 Table of matching data results of the method of the present application and the classic SURF algorithm under image illumination change

[0034] Figure 7 Matching results of the classic SURF under rotation angle

[0035] Figure 8 Matching results of the method of the present application under rotation angle

[0036] Figure 9 Matching results of the method of the present application under image blur change

[0037] Figure 10 Matching results of the method of the present application under image JPEG compression ratio change

[0038] Figure 11 Matching results of the method of the present application under image illumination change DETAILED DESCRIPTION

[0039] The technical solutions of the present application will be further described in detail below in combination with the drawings:

[0040] As shown in the figure, the method comprises five steps. Figure 1

[0041] Step 1. Input the reference image and the image to be matched.

[0042] Step 2. Feature extraction detection is performed by the SURF algorithm, and feature points with scale invariance of the reference image and the image to be matched are detected respectively.

[0043] ​Step 3. Feature description of the feature points in the reference image and the image to be matched respectively by DAISY descriptor;

[0044] Step 4. Initial matching by Euclidean distance

[0045] Step 5. Elimination of false matching points in the result of initial matching by RANSAC algorithm;

[0046] Step 6. Output of the matched image.

[0047] The experimental platform is: MATLAB (2018a), the running environment is: Inter(R) core(TM) i5-4200H CPU@2.80GHz, 8GB memory, 64-bit Windows 10 operating system.

[0048] The principle of DAISY descriptor is introduced below, combined with the attached Figure 2 Further illustrate the image matching method based on improved SURF features of the present application.

[0049] DAISY descriptor is a local feature descriptor, which is composed of a center-symmetric structure and adopts a circular field, and has better positioning performance. Taking the feature point to be described as the center, concentric circles of three different radii are constructed outward, the Gaussian scale of each layer is the same, the Gaussian scale outward increases layer by layer, and a sampling point is taken every 45° in the range of 360°, so that 3x8+1=25 sampling points are obtained.

[0050] The construction method of DAISY descriptor: taking a pixel point (c, d) as an example, the gradient of the pixel point in 8 directions is represented by (o = 1, 2, 3…, 8 represents the direction of the gradient; (·) + represents (alpha) + = max(alpha, 0)) represents. Then, the gradient image G o (c, d) is subjected to Gaussian convolution, and the formula is represented as where G ∑ represents a Gaussian kernel with a Gaussian scale value of sigma, then the pixel point (c, d) obtains a vector with a length of 8, that is, the local gradient direction histogram of the pixel point (c, d) can be represented as:

[0051]

[0052] Therefore, the formula of the DAISY descriptor is:

[0053]

[0054] In the formula, P = 1, 2, 3 represents the number of layers of the structure; Q = 1, 2, …, 8 represents the direction divided in each layer, and l r(c,d,R k () represents the location of the r-th sampling point on the k-th concentric ring with (c,d) as the structural center. This represents the local gradient direction histogram of the sampling point. Therefore, the dimension of the descriptor feature vector of pixel (c,d) is (8×3+1)×8=200.

[0055] After analyzing the construction method of the DAISY descriptor, it can be found that the centrally symmetric structure of the DAISY descriptor has good robustness to image illumination and affine transformations, and can be rotated quickly and processed conveniently. Furthermore, the weighting method of the gradient histogram by the DAISY operator is achieved using Gaussian kernel convolution. Since the Gaussian kernel is isotropic, after image rotation, only the order of the histogram bars changes, thus eliminating the need to recalculate the gradient histogram. Considering the poor performance of the SURF algorithm in rotation invariance, a method is adopted that, based on the SURF algorithm's feature point detection, the principal direction of the feature points is selected using the SURF algorithm, and then the rotation invariance of the DAISY descriptor is utilized to quickly obtain the DAISY descriptor. This maintains the fast computation speed of the original SURF algorithm while improving its matching accuracy in rotation invariance.

[0056] In one embodiment, the image matching method based on improved SURF features is implemented as follows:

[0057] The input image is converted to grayscale and filtered, and then normalized.

[0058] The SURF algorithm uses box filtering instead of Gaussian filtering and uses integral images to accelerate the integration process, thus maximizing computational speed while maintaining high accuracy.

[0059] For a point on image P with image scale δ The Hessian matrix is ​​defined as follows:

[0060]

[0061] Among them, L xx (a,δ) is the second-order partial derivative of Gaussian and the point... Convolution of the image at that location.

[0062] SURF uses a box filter to approximate a Gaussian filter, thereby improving computational speed. However, using a box filter introduces errors in the approximation, so in D... xy Multiply by a weighting factor of 0.9 to balance it.

[0063] Det(H) = D xx D yy -(0.9D xy) 2

[0064] Because of using box filter, all D xx is the approximate value of L xx , D xy is the approximate value of L xy , D yy is the approximate value of L yy .

[0065] In order to obtain the best effect, non-maximum suppression is performed on each detected interest point scale space, and the response value of each interest point is greater than or less than the value of its surrounding 26 points, then it is regarded as a feature point.

[0066] After using the Hessian matrix to obtain the feature point, the main direction of the feature point is determined by the Haar wavelet feature in the field of the feature point. Then the formula of the DAISY descriptor is used to construct the calculation of the feature descriptor, and the rotation invariance of the feature descriptor is easy to obtain.

[0067] According to the Euclidean distance between two feature points, the feature point set obtained in the last step is subjected to feature point coarse matching, and the coarse matching point is obtained.

[0068] After coarse matching, the RANSAC algorithm is used to remove the error matching points. The threshold is set to 0.03, and the specific steps are as follows:

[0069] (1) randomly select 4 pairs of matching pairs from the data set as a sample, calculate the homography matrix by using the formula, the model is P1, and the threshold is set.

[0070] (2) calculate the error between all data in the data set and the model P1, if the error is less than the set threshold, it is regarded as an inlier, and is added to the inlier set Q1.

[0071] (3) if there are enough points regarded as inliers, select the model P1 at this time, otherwise repeat the above steps until the requirement is met, select the model with the largest inlier ratio as the solution of the problem.

[0072] In order to further illustrate the image matching result of the present application, we respectively select the rotation angle, image blur, JPEG compression ratio and light change under the experiment, combine with the attached Figure 3 , attached Figure 4 , attached Figure 5 and attached Figure 6It can be seen that the application introduces DAISY descriptor on the basis of SURF algorithm feature detection, keeps the original SURF algorithm fast computing speed, improves the speed of obtaining rotation invariance, improves the correct matching rate, avoids the matching "one-to-many" phenomenon and the like; the false matching elimination algorithm is introduced after the rough matching is completed, and the generated false matching point pairs are eliminated. The algorithm has high matching precision, fast computing speed, short matching time, fast matching speed, is more robust compared with the classical SURF matching method, and has real-time performance.

[0073] The above examples only illustrate the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical scheme falls within the protection scope of the present application. The embodiments of the present application are described in detail above, but the present application is not limited to the above-mentioned embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the purpose of the present application.

Claims

1. An image matching method based on improved SURF features, characterized in that, The method comprises the following steps: Step 1, inputting a reference image and a to-be-matched image; pre-processing the images, which comprises image graying and filtering processing, and then normalizing processing; Step 2, detecting feature points with scale invariance in the reference image and the to-be-matched image respectively through a SURF algorithm; Step 3, the feature points in the reference image and the image to be matched are respectively described by DAISY descriptors, characterized in that a principal direction is assigned to have rotation invariance, a descriptor is established, a direction is selected in 45° units, a second layer of concentric circle maximum gradient histogram is extracted, thereby obtaining a DAISY descriptor having rotation invariance, taking the feature point to be described as the center, a concentric circle is constructed outwardly with three different radii, the Gaussian scale of each layer is the same, the Gaussian scale outwardly increases layer by layer, a sampling point is taken every 45° within a range of 360°, so that 3x8+1=25 sampling points are obtained in total, taking a pixel point (c,d) as an example, the gradient of the pixel point in 8 directions is represented by (o = 1, 2, 3…, 8 represents the direction of the gradient; (·) + represents (α) + = max(α, 0)) represents, then, Gaussian convolution is performed on the gradient image G o (c,d), and the formula is represented as wherein G ∑ represents a Gaussian kernel with a Gaussian scale value of ∑, then the pixel point (c,d) obtains a vector with a length of 8, that is, the local gradient direction histogram of the pixel point (c,d) can be represented as: Therefore, the formula of the DAISY descriptor is: In the formula, P = 1, 2, 3 represents the structure layer number; Q = 1, 2, …, 8 represents the direction of division of each layer, and l r (c, d, R k ) represents the position of the rth sampling point on the kth concentric ring with (c, d) as the structure center, represents the local gradient direction histogram of the sampling point, so the descriptor feature vector dimension of the pixel point (c, d) is (8 x 3 + 1) x 8 = 200. Step 4, matching the descriptors of the feature points in the reference image and the descriptors of the feature points in the to-be-matched image, and completing initial matching; Step 5, eliminating mis-matching points in the initial matching process through a RANSAC algorithm; Step 6, generating a matching relationship of the feature points in the reference image and the to-be-matched image.

2. The image matching method based on improved SURF features according to claim 1, characterized in that: Said step 2 is specifically, using Hessian matrix determinant to detect feature points, for the image P on the image scale for the point δ The definition of Hessian matrix is: wherein L xx (a, δ), L xy (a, δ) and L yy (a, δ) are the convolutions of the Gaussian second derivatives with the image P at the point To improve the operation speed, a box filter is used, and the values after the convolution operation are respectively D xx , D xy , and D yy . To balance the error caused by the approximation of the box filter, a weighted coefficient is multiplied on D xy , and the weighted coefficient is 0.

9. Therefore, the approximate value of the Hessian matrix discriminant of each pixel is: Det(H) = D xx D yy - (0.9D xy ) 2 .

3. The image matching method based on improved SURF features according to claim 1, characterized in that: The step 5 is based on the initial matching relationship between the two images obtained in the step 4, and further mis-matching point elimination is performed through the RANSAC algorithm to avoid one-to-many mis-matching phenomenon, and specifically comprises: (1) randomly extracting four pairs of matching pairs from the data set as samples, calculating a homography matrix through a formula, the model is denoted as P1, and a threshold is set; (2) calculating errors between all data in the data set and the model P1, if the error is less than the set threshold, the data is regarded as an inlier, and is added to an inlier set Q1; (3) if enough points are regarded as inliers, the model P1 at this time is selected, otherwise, the above steps are repeated until the requirement is met, and the model with the largest inlier ratio is selected as the solution of the problem.

Images