Inclined image matching method of Affine-AKAZE algorithm
Through the tilt image matching method of the Affine-AKAZE algorithm, through the steps of simulated image generation, feature point extraction and descriptor calculation, and phased matching, the problems of low accuracy and low number of points in the existing technology are solved, and high-precision matching is achieved.
Patent Information
- Application Number
- CN202510061944.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-15
- Publication Date
- 2025-06-06
Smart Images

Figure CN120107060A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of photogrammetry, and in particular to an oblique image matching method of an Affine-AKAZE algorithm. Background Art
[0002] As an emerging photogrammetry technology, oblique photogrammetry is widely used in digital twins, national land space planning, flood prevention and disaster reduction, smart cities and other fields. Feature point matching is the key point in oblique image processing, and high-precision feature point matching results have a great impact on the oblique modeling effect. Compared with traditional orthophoto matching, the difficulty of oblique image matching is increased due to the large geometric deformation and color difference between the images to be matched.
[0003] At present, in the aspect of matching feature points of oblique images, there are many studies on affine invariance algorithms. For example, some scholars have proposed the Level Line Descriptor (LLD) algorithm, the Maximally Stable Extremal Region (MSER) algorithm, the Harris-Affine and Hessian-Affine invariant regional features, etc. These algorithms do not have complete affine invariance. When the geometric deformation between the images to be matched is large, the matching effect is poor and the correct matching point pairs can be obtained are few. In order to solve the problem of low matching accuracy and small number of matching points of oblique images, this paper proposes an Affine-AKAZE algorithm with affine invariance. Summary of the invention
[0004] In view of the shortcomings of the prior art, the present invention provides an oblique image matching method of the Affine-AKAZE algorithm, which solves the shortcomings of the existing affine invariance algorithms in the feature point matching of oblique images, such as the LLD, MSER, Harris-Affine and Hessian-Affine algorithms, which are not completely affine invariant, have poor matching effects when the geometric deformation is large, and have few correct matching points.
[0005] To achieve the above objectives, the present invention is implemented by the following technical solutions: an oblique image matching method of Affine-AKAZE algorithm, comprising the following steps:
[0006] S1. Simulated image generation: by determining the key factors of longitude and latitude angles, performing rotation, tilt and longitude and latitude angle sampling operations, simulating the affine deformation of the image to be matched due to the change of viewing angle to obtain simulated image data;
[0007] S2. Feature point extraction and descriptor calculation: Use the AKAZE algorithm to extract the feature points of the simulated image, and use the Opponent-FREAK color descriptor to fuse the color features to calculate the descriptor to enhance the feature description capability;
[0008] S3. Phased matching: Comprehensively use bidirectional matching, cosine similarity matching and VFC algorithm, eliminate some mismatched points through rough matching first, and then perform VFC precise matching based on this for further optimization.
[0009] Preferably, the step S1 includes:
[0010] S101. Determine the key factors: Determine the longitude angle φ and latitude angle θ of the view angle change between the images to be matched, which is the key factor in the generation of simulated images;
[0011] S102. Performing a rotation operation: performing a rotation operation on the simulated image based on the longitude angle φ, so that the angle of the image changes in the horizontal direction, simulating the image state under different viewing angles;
[0012] S103. Perform tilt processing: Combine the latitude angle θ with the simulated image and set the parameter to 1 / cost (t is the tilt parameter) to make the image tilt in the vertical direction, further simulating the image affine deformation caused by the change of camera viewing angle;
[0013] S104. Sampling operation: sampling the longitude angle φ and the latitude angle θ to simulate all possible perspective changes between the images to be matched, thereby obtaining image data close to the simulated image.
[0014] Preferably, the longitude angle sampling is specifically: when the tilt t changes, the longitude angle φ is changed arithmetic progression, that is, φ i+1 =φ i +Δφ (Δφ is the arithmetic difference, and |φ|<180°), by gradually changing the longitude angle, the simulated images at different horizontal viewing angles are obtained.
[0015] Preferably, the latitude angle sampling is specifically: based on the tilt t, the latitude angle θ is sampled in geometric proportion, that is, θ i+1 =aθ i (a>1 is a geometric coefficient). In this way, the image changes under different vertical viewing angles are simulated, thereby fully covering possible viewing angle changes.
[0016] Preferably, the step S2 includes: S201. Feature point extraction: using the AKAZE algorithm to perform feature point extraction operations on the simulated image.
[0017] Preferably, the step S2 further includes: S202. Descriptor calculation: using the Opponent-FREAK color descriptor to calculate the descriptor of the extracted feature point, including space transformation, wherein the space transformation is based on the RGB space of the color image, and converts it into the Opponent Color space, and the conversion model is shown as follows:
[0018]
[0019] Among them, O 1 , O 2 Channel represents the color information of the image to be matched, O 3 It is the intensity information.
[0020] Preferably, the step S3 includes: S301. Rough matching: firstly, a rough matching is performed, and the rough matching includes bidirectional matching and cosine similarity matching.
[0021] Preferably, the two-way matching is specifically:
[0022] Reference image and image to be matched selection: Select one of the images to be matched as the reference image and the other as the image to be matched, including first selecting image I 1 is the reference image, I 2 The image to be matched;
[0023] First matching: Use the Affine-AKAZE algorithm to obtain the matching point pair set A of the pair of images;
[0024] Exchange images: Then exchange the reference image and the image to be matched, that is, select image I 2 is the reference image, I 1 is the image to be matched;
[0025] Second matching: Affine-AKAZE algorithm is used again to extract and obtain the matching point pair set B;
[0026] Take the intersection: Perform an intersection operation on set A and set B, and the result is the bidirectional matching result.
[0027] Preferably, the cosine similarity matching is specifically:
[0028] Calculate cosine similarity: For a pair of feature points, calculate the cosine value of the angle between the corresponding feature vectors as the cosine similarity. Suppose the two feature vectors are and The formula for calculating cosine similarity is:
[0029]
[0030] in and is the inner product of the corresponding description vector, and is the vector length, and the cosine value C ranges from [-1,1];
[0031] Threshold screening: By traversing the cosine values of all feature point pairs and comparing them with the predetermined optimal threshold T, if the cosine value is greater than the threshold T, the point pair is retained as the correct matching point, otherwise it is judged as a wrong match, thereby obtaining an accurate matching result based on the cosine constraint.
[0032] Preferably, the step S3 also includes: S302. VFC algorithm exact matching: based on the results of bidirectional matching and cosine similarity constraints, VFC (Vector Filed Consensus, VFC) algorithm is used to achieve exact matching to obtain the final result. The VFC algorithm obtains the mapping f:X→Y through a regular optimization method. Based on the mapping, the results that meet the internal points are retained and the external points are eliminated, thereby further optimizing the matching results.
[0033] The present invention provides an oblique image matching method based on the Affine-AKAZE algorithm, which has the following beneficial effects:
[0034] 1. The present invention accurately simulates affine deformation by simulating the image generation link, providing a high-quality data basis for subsequent processing. The AKAZE algorithm and Opponent-FREAK color descriptor are used to enhance the ability to extract and describe feature points. The phased matching strategy further optimizes the results, significantly improves the accuracy of tilted image matching, obtains more correct matching points, effectively solves the problems of low matching accuracy and small number of points in existing algorithms, achieves high-precision matching, and meets high-precision application requirements.
[0035] 2. The affine deformation simulation in the simulated image generation of the present invention enables the algorithm to adapt to different geometric deformations, overcoming the limitation of the existing algorithm that the matching effect is poor when the geometric deformation is large. It performs well in various data experiments, with high average accuracy and small variance, indicating that it has strong adaptability and good stability in different scenarios, and can be widely used in oblique image processing in multiple fields. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a flow chart of the matching method of the present invention;
[0037] Figure 2 It is a schematic diagram of latitude angle sampling of the present invention;
[0038] Figure 3 It is a schematic diagram of longitude angle sampling of the present invention;
[0039] Figure 4It is a schematic diagram of RGB and Opponent channel conversion of the present invention;
[0040] Figure 5 It is a schematic diagram of the graf data set of the present invention;
[0041] Figure 6 This is a schematic diagram of the original image of the drone of the present invention;
[0042] Figure 7 A schematic diagram of the Affine-AKAZE drone image matching results of the present invention;
[0043] Figure 8 This is a schematic diagram of the drone image matching results of the AKAZE algorithm of the present invention;
[0044] Fig. 9 This is a schematic diagram of the UAV image matching results of the KAZE algorithm of the present invention;
[0045] Fig.10 This is a schematic diagram of the drone image matching results of the Hessian-affine algorithm of the present invention;
[0046] Fig.11 Schematic diagram of the average accuracy and variance of the four algorithms of the present invention. DETAILED DESCRIPTION
[0047] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0048] Example:
[0049] Please see attached Figure 1 -Attached Fig.11 The embodiment of the present invention provides an oblique image matching method of the Affine-AKAZE algorithm, comprising the following steps:
[0050] S1. Simulated image generation: by determining the key factors of longitude and latitude angles, performing rotation, tilt and longitude and latitude angle sampling operations, simulating the affine deformation of the image to be matched due to the change of viewing angle to obtain simulated image data;
[0051] S101. Determine the key factors: Determine the longitude angle φ and latitude angle θ of the view angle change between the images to be matched, which is the key factor in the generation of simulated images;
[0052] S102. Performing a rotation operation: performing a rotation operation on the simulated image based on the longitude angle φ, so that the angle of the image changes in the horizontal direction, simulating the image state under different viewing angles;
[0053] S103. Perform tilt processing: Combine the latitude angle θ and treat the simulated image with a parameter of 1 / cost (t is a tilt parameter) to make the image tilt in the vertical direction, further simulating the image affine deformation caused by the change of camera viewing angle;
[0054] S104. Sampling operation: sampling the longitude angle φ and the latitude angle θ to simulate all possible perspective changes between the images to be matched, thereby obtaining image data close to the simulated image;
[0055] The longitude angle sampling is specifically: when the tilt t changes, the longitude angle φ is changed arithmetic difference, that is, φ i+1 =φ i +Δφ (Δφ is an arithmetic difference, and |φ|<180°), by gradually changing the longitude angle, the simulated images at different horizontal viewing angles are obtained;
[0056] The latitude angle sampling is specifically: based on the tilt t, the latitude angle θ is sampled in a geometric ratio, that is, θ i+1 =aθ i (a>1 is a geometric coefficient). In this way, the image changes under different vertical viewing angles are simulated to fully cover possible viewing angle changes.
[0057] S2. Feature point extraction and descriptor calculation: Use the AKAZE algorithm to extract the feature points of the simulated image, and use the Opponent-FREAK color descriptor to fuse the color features to calculate the descriptor to enhance the feature description capability;
[0058] S201. Feature point extraction: Use the AKAZE algorithm to extract feature points from the simulated image. The AKAZE algorithm constructs a nonlinear scale space and detects feature points at different scales. It can more effectively adapt to various changes in the image and extract representative feature points. These feature points will serve as the basis for subsequent matching;
[0059] S202. Descriptor calculation: Use Opponent-FREAK color descriptor to calculate the descriptor of the extracted feature point, including space transformation, which is based on the RGB space of the color image, and converts it into the opposing color (OpponentColor) space. The conversion model is shown in the following formula:
[0060]
[0061] Among them, O 1 , O 2Channel represents the color information of the image to be matched, O 3 The spatial transformation is the intensity information. It can better integrate the color feature information and enhance the ability to describe the image features.
[0062] The descriptor calculation is specifically as follows: the descriptor of the feature is calculated in the Opponent space, so as to obtain a feature description containing color information. Compared with the traditional grayscale descriptor, it can better cope with the color changes and complex scenes in the image and improve the stability and accuracy of the algorithm;
[0063] S3. Phased matching: Comprehensively use bidirectional matching, cosine similarity matching and VFC algorithm to eliminate some mismatched points and improve accuracy through rough matching, and then further optimize VFC precise matching based on this to achieve high-precision matching of feature points of oblique images;
[0064] S301. Coarse matching: firstly, a coarse matching is performed, wherein the coarse matching includes a bidirectional matching and a cosine similarity matching;
[0065] The bidirectional matching is specifically:
[0066] Reference image and image to be matched selection: Select one of the images to be matched as the reference image and the other as the image to be matched, including first selecting image I 1 is the reference image, I 2 The image to be matched;
[0067] First matching: Use the Affine-AKAZE algorithm to obtain the matching point pair set A of the pair of images;
[0068] Exchange images: Then exchange the reference image and the image to be matched, that is, select image I 2 is the reference image, I 1 is the image to be matched;
[0069] Second matching: Affine-AKAZE algorithm is used again to extract and obtain the matching point pair set B;
[0070] Intersection: Perform intersection operation on set A and set B. The result is the bidirectional matching result, which can effectively reduce mismatched point pairs and improve matching accuracy.
[0071] The cosine similarity matching is specifically:
[0072] Calculate cosine similarity: For a pair of feature points, calculate the cosine value of the angle between the corresponding feature vectors as the cosine similarity. Suppose the two feature vectors are and The formula for calculating cosine similarity is:
[0073]
[0074] in and is the inner product of the corresponding description vector, and is the vector length, and the cosine value C ranges from [-1,1];
[0075] Threshold screening: By traversing the cosine values of all feature point pairs and comparing them with the predetermined optimal threshold T, if the cosine value is greater than the threshold T, the point pair is retained as the correct matching point, otherwise it is judged as a wrong match, thereby obtaining an accurate matching result based on the cosine constraint, further improving the matching accuracy;
[0076] S302. VFC algorithm accurate matching: Based on the results of bidirectional matching and cosine similarity constraints, the VFC (Vector Fielded Consensus, VFC) algorithm is used to achieve accurate matching to obtain the final result. The VFC algorithm obtains the mapping f:X→Y through a regular search method. Based on this mapping, the results that meet the internal points are retained and the external points are eliminated, thereby further optimizing the matching results and improving the accuracy and reliability of the matching.
[0077] Comparative experiment: Based on the Graf dataset and UAV oblique images, various algorithms are compared to evaluate the performance (accuracy and precision) of the Affine-AKAZE algorithm, verify the improvement effect (affine invariance and validity of color descriptors), and determine its applicability and stability, so as to provide reliable algorithm support for oblique image processing.
[0078] 1. Experimental Preparation
[0079] Experimental data selection: In order to fully verify the matching performance of the Affine-AKAZE algorithm, the experiment used two sets of data from different sources. One is to select the Graf dataset from the Mikolajczyk dataset. This dataset contains 6 images, which are mainly used to verify the effect of the algorithm on affine rotation changes. The affine angle between the images gradually increases, and the affine deformation becomes larger and larger; the second is to select 4 sets of drone oblique image data to better verify the stability of the improved algorithm in actual application scenarios.
[0080] Determination of algorithms involved in comparison: Affine-AKAZE algorithm, AKAZE algorithm, KAZE algorithm and Hessian-Affine algorithm are determined as the algorithms involved in the comparison. By comparing the performance of these algorithms on the same data, the advantages and improvement effects of the Affine-AKAZE algorithm are evaluated.
[0081] 2. Experimental Procedure
[0082] Graf dataset experiments
[0083] Data preprocessing: For each image in the Graf dataset, the Affine-AKAZE algorithm is used for processing. First, simulated image generation is performed. According to the longitude and latitude angles of the view angle between images, simulated images are obtained through rotation, tilt operations, and latitude and longitude sampling. Then, the AKAZE algorithm is used to extract feature points on the simulated image, and the Opponent-FREAK color descriptor is used to calculate the feature point descriptors.
[0084] Phased matching and result recording: Phased matching is performed, including rough matching (bidirectional matching and cosine similarity matching) and precise matching using the VFC algorithm. In rough matching, bidirectional matching is performed first, that is, different images are used as reference images and images to be matched, and the Affine-AKAZE algorithm is used twice to obtain the set of matching point pairs and take the intersection; then the cosine similarity is calculated, and the matching point pairs are screened according to the set threshold. Finally, the VFC algorithm is used for precise matching to obtain the final result. The number of correct matching points, accuracy, and root mean square error (RMSE) after the Affine-AKAZE algorithm matches different image pairs (such as Img1-Img2, Img1-Img3, etc.) are recorded. At the same time, the same processing flow is used for the AKAZE, KAZE, and Hessian-Affine algorithms, matching experiments are performed on the same image pairs, and the corresponding results are recorded.
[0085] Drone oblique imaging experiment
[0086] Data processing and matching: The four sets of UAV oblique image data were processed using the Affine-AKAZE, AKAZE, KAZE and Hessian-Affine algorithms according to the above process. First, simulated images were generated, then feature points were extracted and descriptors were calculated, and then staged matching was performed, and the number of correct matching points, accuracy and RMSE of each algorithm on each set of data (data 1, data 2, data 3, data 4) were recorded.
[0087] Visualization of matching results: In addition to recording data indicators, a schematic diagram of the matching results of each algorithm on the drone oblique image is also generated to intuitively display the distribution of matching points of different algorithms on the actual image, assisting in analyzing the algorithm performance.
[0088] 3. Results Analysis
[0089] Graf dataset results analysis
[0090] Comparison of matching points: Observe the correct matching points of different algorithms between different image pairs in the Graf dataset. When the perspective change of the image to be matched is small, the Affine-AKAZE algorithm obtains significantly more matching point pairs than other algorithms; as the perspective change difference increases, the AKAZE, KAZE and Hessian-Affine algorithms have large image distortion errors, and the number of matching points decreases or even cannot be obtained, while the Affine-AKAZE algorithm can still obtain more correct matching points by simulating the affine perspective distortion transformation of the image.
[0091] Accuracy and root mean square error analysis: The accuracy and RMSE of each algorithm are analyzed. When the viewing angle changes slightly, the accuracy of the Affine-AKAZE algorithm is similar to that of other algorithms. However, when the viewing angle changes greatly, its accuracy can still remain at a relatively high level, and the RMSE value is small, indicating that its matching accuracy is high.
[0092] Analysis of UAV oblique image results
[0093] Comparison of comprehensive indicators: Comparison of the number of correct matching points, accuracy, and RMSE of each algorithm on four sets of drone oblique image data. The Affine-AKAZE algorithm is significantly better than other algorithms in terms of matching points, while the accuracy is higher, and the accuracy of its matching point pairs mostly reaches the sub-pixel level (RMSE is less than one pixel), while the accuracy of other algorithms is mostly greater than one pixel.
[0094] Stability evaluation: By calculating the average and variance values of the matching results of the four algorithms, it is found that the average value of the Affine-AKAZE algorithm is significantly better than the other algorithms, and the variance value is smaller than the other algorithms, indicating that the algorithm has good stability for matching different images.
[0095] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. An oblique image matching method based on the Affine-AKAZE algorithm, characterized in that: The following steps are involved: S1. Simulated image generation: by determining the key factors of longitude and latitude angles, performing rotation, tilt and longitude and latitude angle sampling operations, simulating the affine deformation of the image to be matched due to the change of viewing angle to obtain simulated image data; S2. Feature point extraction and descriptor calculation: Use the AKAZE algorithm to extract the feature points of the simulated image, and use the Opponent-FREAK color descriptor to fuse the color features to calculate the descriptor to enhance the feature description capability; S3. Phased matching: Comprehensively use bidirectional matching, cosine similarity matching and VFC algorithm, eliminate some mismatched points through rough matching first, and then perform VFC precise matching based on this for further optimization.
2. The oblique image matching method of the Affine-AKAZE algorithm according to claim 1, characterized in that: The step S1 includes: S101. Determine the key factors: Determine the longitude angle φ and latitude angle θ of the view angle change between the images to be matched, which is the key factor in generating the simulated image; S102. Performing a rotation operation: performing a rotation operation on the simulated image based on the longitude angle φ, so that the angle of the image changes in the horizontal direction, simulating the image state under different viewing angles; S103. Perform tilt processing: Combine the latitude angle θ and treat the simulated image with a parameter of 1 / cost (t is a tilt parameter) to make the image tilt in the vertical direction, further simulating the image affine deformation caused by the change of camera viewing angle; S104. Sampling operation: sampling the longitude angle φ and the latitude angle θ to simulate all possible perspective changes between the images to be matched, thereby obtaining image data close to the simulated image.
3. The oblique image matching method of the Affine-AKAZE algorithm according to claim 2, characterized in that: The longitude angle sampling is specifically: when the tilt t changes, the longitude angle φ is changed arithmetic difference, that is, φ i+1 =φ i +Δφ (Δφ is the arithmetic difference, and |φ|<180°), by gradually changing the longitude angle, the simulated images at different horizontal viewing angles are obtained.
4. The oblique image matching method of the Affine-AKAZE algorithm according to claim 2, characterized in that: The latitude angle sampling is specifically: based on the tilt t, the latitude angle θ is sampled in a geometric ratio, that is, θ i+1 =aθ i (a>1 is a geometric coefficient). In this way, the image changes under different vertical viewing angles are simulated, thereby fully covering possible viewing angle changes.
5. The oblique image matching method of the Affine-AKAZE algorithm according to claim 1, characterized in that: The step S2 includes: S201. Feature point extraction: using the AKAZE algorithm to perform feature point extraction operations on the simulated image.
6. The oblique image matching method of the Affine-AKAZE algorithm according to claim 1, characterized in that: The step S2 also includes: S202. Descriptor calculation: using the Opponent-FREAK color descriptor to calculate the descriptor of the extracted feature point, including space transformation, the space transformation is based on the RGB space of the color image, and converts it into the opposing color (Opponent Color) space, the conversion model is shown as follows: Among them, O1 and O2 channels represent the color information of the image to be matched, and O3 represents the intensity information.
7. The oblique image matching method of the Affine-AKAZE algorithm according to claim 1, characterized in that: The step S3 includes: S301. Rough matching: firstly, a rough matching is performed, and the rough matching includes bidirectional matching and cosine similarity matching.
8. The oblique image matching method of the Affine-AKAZE algorithm according to claim 7, characterized in that: The bidirectional matching is specifically: Reference image and image to be matched selection: select one of a pair of images to be matched as the reference image and the other as the image to be matched, including first selecting image I1 as the reference image and I2 as the image to be matched; First matching: Use the Affine-AKAZE algorithm to obtain the matching point pair set A of the pair of images; Exchange images: Then exchange the reference image and the image to be matched, that is, select image I2 as the reference image and I1 as the image to be matched; Second matching: Affine-AKAZE algorithm is used again to extract the matching point pair set B; Take the intersection: Perform an intersection operation on set A and set B, and the result is the bidirectional matching result.
9. The oblique image matching method of the Affine-AKAZE algorithm according to claim 7, characterized in that: The cosine similarity matching is specifically: Calculate cosine similarity: For a pair of feature points, calculate the cosine value of the angle between the corresponding feature vectors as the cosine similarity. Suppose the two feature vectors are and The formula for calculating cosine similarity is: in and is the inner product of the corresponding description vector, and is the vector length, and the cosine value C ranges from [-1,1]; Threshold screening: By traversing the cosine values of all feature point pairs and comparing them with the predetermined optimal threshold T, if the cosine value is greater than the threshold T, the point pair is retained as the correct matching point, otherwise it is judged as a wrong match, thereby obtaining an accurate matching result based on the cosine constraint.
10. The oblique image matching method of the Affine-AKAZE algorithm according to claim 1, characterized in that: The step S3 also includes: S302. VFC algorithm precise matching: based on the results of bidirectional matching and cosine similarity constraints, the VFC (Vector Filed Consensus, VFC) algorithm is used to achieve precise matching to obtain the final result. The VFC algorithm obtains the mapping f:X→Y through a regular search method. Based on the mapping, the results that meet the internal points are retained and the external points are eliminated, thereby further optimizing the matching results.