A fast method for extracting affine invariant features of images

Abstract

The present invention discloses a fast method for extracting affine invariant features of images, belonging to the field of computer vision. The method first determines the scaling relationship by judging the pixel length relationship of corresponding line segments on two images, and based on this, distinguishes the reference image and the target image; secondly, divides the feature extraction process into two times: first, performs the first feature extraction and matching in a more efficient way to obtain relatively optimal affine transformation parameters, and only performs simulated affine on the reference image, greatly reducing and minimizing the number and time consumption of simulated transformation, feature extraction and feature matching, and can introduce a scale parameter simulation and nearest neighbor interpolation method in the affine transformation process, effectively improving the number of feature point pair matches, accuracy and running efficiency; the experimental results show that the present invention has good affine invariance, can maintain excellent performance and accuracy at small viewing angles, can also resist large-angle viewing angle changes, and can greatly reduce memory occupancy and computational overhead.

Description

Technical Field

[0001] The present invention relates to a method for quickly extracting affine invariant features of images, belonging to the field of computer vision. Background Art

[0002] The existing mainstream image matching methods are mainly based on point feature methods, which are widely used in many important fields such as remote sensing image processing, target detection, and 3D reconstruction. However, if the point feature method is to achieve good results, one of the important conditions that need to be met is that the captured images need to ensure that there is sufficient overlap area in the content of adjacent image frames. However, when the image undergoes an affine transformation, it will cause significant changes in the gray information of the content overlap area between consecutive video image frames, which violates the prerequisite for point feature extraction, resulting in unsatisfactory feature extraction and matching effects even for the best affine image point feature methods currently. For this reason, researchers have given many improved results based on classical algorithms. Although some improved algorithms have good accuracy under affine transformation, the high computational complexity and long computational time have always been problems that trouble people.

[0003] The current image feature extraction methods for dealing with affine transformation (including translation, scale, rotation, flip, shear) can be classified into two categories: partial affine transformation and complete affine transformation. The first category of methods that support partial affine transformation includes Harris / Hessian-Affine, MSER (Maximally Stable Extremal Regions), etc. Such algorithms directly replace the feature detection step in the SIFT (Scale Invariant Feature Transform) algorithm with an affine invariant feature extraction operator with better performance, or use the second-order gradient moment representing the local shape to normalize the SIFT feature domain to remove the influence of affine variability, which can to some extent improve the problem that the SIFT algorithm cannot extract repeated features in different scale spaces due to large image transformation. However, since the initial scale selection and positioning of the features of the existing affine invariant feature extraction algorithms do not start in a completely affine invariant manner, this type of method cannot achieve true complete affine invariance and has insufficient adaptability to large affine deformations. The second category of methods that support complete affine transformation, represented by the ASIFT (Affine-SIFT) algorithm, solves this problem. ASIFT draws on the exhaustive strategy of SIFT to simulate the scale space. Its complete affine invariance is achieved by changing the longitude and latitude of the camera's principal optical axis direction to simulate different perspectives, and then using scale-invariant methods to extract features from the obtained image pairs. It has been proven to be completely affine invariant both theoretically and experimentally. However, ASIFT has the problems of large computational amount and long time consumption.

[0004] To solve the problem of large time consumption in ASIFT calculation, based on the ASIFT algorithm, the AORB (Affine-ORB) algorithm, the ASURF (Affine-SURF) algorithm, and the AFREAK (Affine-FREAK) algorithm respectively use the ORB (Oriented FAST and Rotated BRIEF), SURF (Speeded Up Robust Features), and FREAK (Fast Retina Keypoint) algorithms to replace the SIFT algorithm. Although the operation speed of the algorithm is effectively improved, there is still a problem of low feature extraction accuracy. Summary of the Invention

[0005] In order to further improve the accuracy of feature extraction while maintaining a relatively fast operation speed, the present invention provides a fast image affine invariant feature extraction method, and the technical solution is as follows:

[0006] The first object of the present invention is to provide a fast image affine invariant feature extraction method, which includes:

[0007] Step 1: Obtain the first image and the second image to be matched, extract the feature points of the two images respectively and match them. Select part of the feature points from the first image, form a line segment for every two feature points, and find the matching feature points in the second image to form a feature point line segment corresponding to the first image;

[0008] Step 2: Obtain the scaling relationship between the images according to the length relationship between the line segment pairs obtained in Step 1. From this, the distance relationship between the corresponding camera positions of the first image and the second image and the image center in the three-dimensional space can be known, and based on this, they are divided into a reference image and a target image;

[0009] Step 3: Perform simulated affine transformations with different parameters on the reference image to obtain a group of images corresponding to different sampling point positions. Use each simulated image and the target image to perform ORB feature extraction and matching. The sampling parameter corresponding to the largest number of matching point pairs obtained is recorded as the optimal sampling parameter;

[0010] Step 4: Based on the optimal sampling parameter, reproduce the simulated affine transformation on the reference image, and perform SIFT feature extraction and matching with the target image. After matching screening, obtain the feature point pairs and matching relationships between the reference image and the target image.

[0011] Optionally, the scaling relationship between the images in Step 2 is characterized by a scaling coefficient, and the scaling coefficient is:

[0012]

[0013] Among them, X = {x1, x2, x3, …, x n} represents the set of feature points on the first image, Y = {y1, y2, y3, …, y n} represents the set of feature points corresponding to X on the second image, Dis(x i , x i+1 ) represents the length of the line segment with points x i and x i+1 as endpoints in the first image, Dis(y i , y i+1 ) represents the length of the line segment corresponding to Dis(x i , x i+1 ) in the second image, and Des(i) represents the descriptor distance when performing feature matching between the feature point x i in the first image and the matching point y i in the second image.

[0014] Optionally, step 2 further includes: swapping the first image and the second image, recalculating the scaling coefficient f, comparing the two calculated scaling coefficients, the first image in the mapping with a larger scaling coefficient is closer to the camera optical center and is selected as the reference image, and the other image is used as the target image.

[0015] Optionally, step 1 uses the SIFT algorithm for feature point extraction and matching.

[0016] Optionally, the sampling parameters for the affine transformation in step 3 include: longitude, latitude, and scale parameters.

[0017] Optionally, step 4 further includes: using the RANSAC method to screen and remove mismatched point pairs.

[0018] Optionally, step 4 uses the Lanczos4 interpolation method to reproduce and simulate the reference image.

[0019] The second object of the present invention is to provide a three-dimensional reconstruction method. Using the fast image affine invariant feature extraction method described in any one of the above, first estimate the reference image and the target image using the scaling coefficient, and then perform three-dimensional reconstruction based on the feature matching result.

[0020] The third object of the present invention is to provide a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the method described in any one of the above is implemented.

[0021] The beneficial effects of the present invention are:

[0022] The image affine invariant feature extraction method of the present invention first determines the scaling relationship by using the pixel length relationship of corresponding line segments on two images, and based on this, distinguishes the reference image and the target image. Then, only the reference image is simulated for affine transformation, which greatly reduces the number and time consumption of simulation transformation, feature extraction, and feature matching.

[0023] In an embodiment of the present invention, the ORB algorithm is used to extract and match features of the simulated affine image and the target image, and a scale parameter simulation and nearest neighbor interpolation method are introduced in the affine transformation process, effectively improving the matching quantity, accuracy, and running efficiency of feature point pairs.

[0024] In an embodiment of the present invention, after obtaining the optimal parameters, a method with higher accuracy is used for precise matching. After reproducing the simulated affine transformation of the reference image with the optimal parameters combined with the Lanczos4 interpolation method, SIFT feature extraction and matching are performed to obtain the final result. The experimental results show that this embodiment has good affine invariance, can maintain the excellent performance and accuracy of SIFT at small viewing angles, can also resist large-angle viewing angle changes, and at the same time can greatly reduce the computational cost. The speed is more than 5 times that of ASIFT. In most scenarios, the accuracy can exceed ASIFT. And due to the high efficiency of the ORB algorithm itself, it can also greatly reduce the memory occupancy and computational overhead. Description of the Drawings

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0026] Figure 1 It is a flow structure diagram of a fast image affine invariant feature extraction method of the present invention.

[0027] Figure 2 It is an image set for testing the present invention.

[0028] Figure 3 It is a schematic diagram of the sampling point distribution in the process of obtaining the optimal simulation parameters of the present invention, where (a) is a three-dimensional schematic diagram and (b) is a top view.

[0029] Figure 4 For Embodiment 2 of the present invention, based on Figure 2Example diagram of experimental results of the first and sixth images, where (a) is the image of feature extraction in the estimated scaling scale, (b) is the schematic diagram of the matching between the image of simulated affine transformation and the target image during the acquisition process of the optimal simulation parameters, and (c) is the schematic diagram of the final image feature point matching effect of the embodiment of the present invention.

[0030] Figure 5 For the embodiment of the present invention Figure 2 Graphs of the matching accuracy results and running time results of feature point extraction between different images. Specific implementation manners

[0031] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0032] Embodiment 1:

[0033] This embodiment provides a fast method for extracting affine invariant features of images, including:

[0034] Step 1: Obtain the first image and the second image to be matched, extract the feature points of the two images respectively and match them. Select some feature points from the first image, form a line segment for every two feature points, and find the matching feature points in the second image to form a feature point line segment corresponding to the first image;

[0035] Step 2: Obtain the scaling relationship between the images according to the length relationship between the line segment pairs obtained in Step 1. From this, the distance relationship between the corresponding camera positions of the first image and the second image and the image center in the three-dimensional space can be known, and based on this, they are divided into a reference image and a target image;

[0036] Step 3: Perform simulated affine transformations with different parameters on the reference image to obtain a group of images corresponding to different sampling point positions. Use each simulated image and the target image for ORB feature extraction and matching. The sampling parameter corresponding to the largest number of matching point pairs obtained is recorded as the optimal sampling parameter;

[0037] Step 4: Based on the optimal sampling parameter, reproduce the simulated affine transformation on the reference image, perform SIFT feature extraction and matching with the target image, and after matching screening, obtain the feature point pairs and matching relationships between the reference image and the target image

[0038] Embodiment 2:

[0039] This embodiment provides a fast method for extracting affine invariant features of images. Refer to Figure 1 , including the following steps:

[0040] Step 1: Obtain two images. Extract and match feature points in the two images respectively through the SIFT algorithm. Let X = {x1, x2, x3, …, x n} represent the set of feature points on the first image, and Y = {y1, y2, y3, …, y n} represent the set of feature points corresponding to X on the second image. Take one-fourth of the points. For the selected points in the first image, every two form a feature line segment. Use Dis(x i , x i+1 ) to represent the length of the line segment with points x i and x i+1 as endpoints in the first image, and the two feature points that are matched in the second image form a feature line segment Dis(y i , y i+1 ) corresponding to the one in the first image;

[0041] Step 2: Use all the pairs of feature line segments in Step 1 to calculate the product of the length difference and weight of the matched line segment pairs in the first and second images one by one, and finally sum them as the scaling coefficient for mapping from the first image to the second image. The calculation formula is shown in Equation (1), where Des(i) represents the descriptor distance when the feature point x i in the first image is SIFT feature-matched with the corresponding point y i in the second image, which represents the reliability of the matching relationship of this pair of feature points. The smaller the descriptor distance, the more reliable the matching relationship of the feature point pair. Use the descriptor distance of the feature point pairs at both ends of the corresponding line segment and Des(i) + Des(i + 1) as the weight of the length difference of this pair of line segments;

[0042]

[0043] Step 3: Swap the first image and the second image, repeat Step 1 and Step 2 to calculate the scaling coefficient, and compare it with the scaling coefficient calculated in Step 2. The image in the mapping with the larger scaling coefficient is closer to the camera optical center and is selected as the reference image, and the other image is used as the target image;

[0044] Step 4: According to the reference image and the target image selected in Step 3, by changing the longitude, latitude, and scale parameters, quickly realize the simulated affine transformation of the reference image according to a certain sampling rule in combination with the nearest neighbor interpolation method. After each simulation, perform ORB feature extraction and matching on the simulated image and the target image, and retain the simulated parameters (longitude, latitude, and scale) corresponding to the simulated image with the largest number of matching point pairs;

[0045] Step 5: Use the simulated parameters retained in Step 4, combine with the Lanczos4 interpolation method to reproduce the simulation of the reference image, perform SIFT feature extraction and matching on the simulated image and the target image, and use the RANSAC method to screen and remove the mismatched point pairs;

[0046] Step 6: Use the real transformation matrix between the images in the data set to calculate the real matching point pairs from the reference image to the target image, and compare them with the matching point pairs in step 5 to verify the actual effect of the present invention.

[0047] Based on the above specific implementation method, combined with Figure 2 The image set shown in the figure is used for testing experiments to verify the effect of the present invention:

[0048] In the algorithm flow, SIFT feature extraction is performed on the two input images, and the scaling coefficients are calculated based on the matching relationship. The reference image and the target image are selected based on the size relationship of the scaling coefficients. The example results are as follows: Figure 4 (a). Then, the reference image is simulated and affine, and the sampling points are distributed as follows: Figure 3 The simulated image of the reference image is matched with the target image one by one to obtain the best simulation parameters. The matching result is shown in Figure 4 (b) Based on the optimal simulation parameters, combined with the Lanczos interpolation method, the reference image is simulated and SIFT features are extracted and matched with the target image. After RANSAC is used to filter feature point pairs, the example effect is as follows Figure 4 (c) as shown.

[0049] The method of the present invention is tested on a standard data set. According to the different degrees of image transformation, the accuracy and time consumption of the present invention on the image are as follows: Figure 5 As shown, it can be found that the accuracy of the present invention is above 90%, and the time consumption is basically controlled at about 7000ms, which can basically meet the requirements for feature point extraction and matching in actual 3D reconstruction.

[0050] This embodiment is completed using Visual Studio Code and OpenCV4.5.5 under the Ubuntu20.04 operating system installed in VMware Workstation 16Pro. The hardware environment is a laptop with a 2.3GHz i7 processor and 16GB of running memory. The experimental process is relatively stable.

[0051] Some steps in the embodiments of the present invention may be implemented using software, and the corresponding software program may be stored in a readable storage medium, such as a CD or a hard disk.

[0052] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A fast method for extracting affine invariant features of images, characterized in that, The method includes: Step 1: Obtain a first image and a second image to be matched, respectively extract feature points of the two images and match them. Select some feature points from the first image, form a line segment for every two feature points, and find the matching feature points in the second image to form a feature point line segment corresponding to the first image. Step 2: Obtain the scaling relationship between the images according to the length relationship between the line segment pairs obtained in Step 1. From this, the distance relationship between the corresponding camera positions of the first image and the second image and the image center in the three-dimensional space can be known, and based on this, they are divided into a reference image and a target image. Step 3: Perform simulated affine transformations on the reference image with different parameters to obtain a group of images corresponding to different sampling point positions. Use each simulated image and the target image to perform ORB feature extraction and matching. The sampling parameter corresponding to the largest number of matching point pairs obtained is recorded as the optimal sampling parameter. Step 4: Based on the optimal sampling parameter, reproduce the simulated affine transformation on the reference image, and perform SIFT feature extraction and matching with the target image. After performing matching screening, obtain the feature point pairs and matching relationships between the reference image and the target image.

2. The fast image affine invariant feature extraction method according to claim 1, wherein In Step 2, the scaling relationship between the images is characterized by a scaling coefficient, and the scaling coefficient is: where X = {x1, x2, x3, …, x n} represents the set of feature points on the first image, Y = {y1, y2, y3, …, y n} represents the set of feature points corresponding to X on the second image, Dis(x i , x i+1 ) represents the length of the line segment with endpoints x i and x i+1 in the first image, Dis(y i , y i+1 ) represents the length of the line segment corresponding to Dis(x i , x i+1 ) in the second image, and Des(i) represents the descriptor distance when performing feature matching between the feature point x i in the first image and the matching point y i in the second image.

3. The fast image affine invariant feature extraction method according to claim 2, characterized in that, Step 2 further includes: Swap the first image and the second image, recalculate the scaling coefficient f, compare the scaling coefficients calculated twice. The first image in the mapping with a larger scaling coefficient is closer to the camera optical center and is selected as the reference image, and the other image is used as the target image.

4. The fast image affine invariant feature extraction method according to claim 1, wherein In Step 1, the SIFT algorithm is used for feature point extraction and matching.

5. The fast image affine invariant feature extraction method according to claim 1, characterized in that, The sampling parameters for performing affine transformation in Step 3 include: longitude, latitude, and scale parameters.

6. The fast image affine invariant feature extraction method according to claim 1, wherein Step 4 further includes: Using the RANSAC method to screen and remove mismatched point pairs.

7. The fast image affine invariant feature extraction method according to claim 1, wherein Step 4 uses the Lanczos4 interpolation method to reproduce the simulation of the reference image.

8. A three-dimensional reconstruction method, characterized in that, The three-dimensional reconstruction method uses the fast image affine invariant feature extraction method according to any one of claims 1-7. First, estimate the reference image and the target image using the scaling coefficient, and then perform three-dimensional reconstruction based on the feature matching result.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images