A fast matching method for large-viewing angle images of elliptical targets

Abstract

The present invention relates to a method for fast wide-angle image matching of elliptical targets. The method comprises: S1, using prior knowledge to estimate the number of pyramid layers where the circular target is located, thereby reducing pyramid layer redundancy in the SIFT algorithm, improving feature point detection efficiency while reducing mismatches caused by background feature points; S2, in a coarse matching process, subjecting the actual image to simulated perspective deformation to obtain a simulated image and performing feature matching, with the rotation angle limited to conform to the actual camera rotation angle. When the number of correctly matched feature points between the simulated image and the reference image exceeds a certain threshold K, a fine matching process is entered; S3, using the correctly matched feature points to obtain a homography matrix between the actual image and the reference image, obtaining a resampled image, and performing final wide-angle matching using an improved SIFT algorithm. Compared with the existing technology, the present invention has the advantages of not only being highly robust to perspective changes but also improving the matching efficiency and accuracy of circular targets.

Description

Technical Field

[0001] The invention relates to an image matching method, in particular to a large-viewing-angle image fast matching method for an elliptical target. Background Art

[0002] Image feature point extraction and matching algorithms are one of the most important research modules in image processing, crucial for various research areas such as structure from motion, object detection, and image stitching. SIFT, a classic image feature point matching algorithm, is widely used in various matching applications due to its affine invariance to brightness and rotation. Furthermore, the SIFT features contain sufficient information to meet the needs of feature point description and search. While the SIFT algorithm is suitable for most scenarios, the number of feature points extracted by the algorithm decreases sharply when the matching image has a large perspective deviation. Furthermore, large rotation angles can cause significant image distortion, making it prone to mismatching between background and target points. To address these issues, Morel et al. proposed the Affine-SIFT (ASIFT) algorithm. This algorithm leverages an affine transformation model and uses parameter sampling to simulate the perspective distortion between the actual image and the reference image. By varying two camera axis parameters, namely longitude and latitude, it effectively increases the number of matching points, thereby achieving matching. However, ASIFT only performs an approximate affine transformation and does not achieve true three-dimensional rotation of the image, resulting in some differences from the actual perspective imaging model. In order to conform to the actual perspective projection situation, Cai et al. proposed the PSIFT algorithm, which changed the affine transformation model in ASIFT to a perspective transformation model, further improving the matching accuracy.

[0003] However, in practical applications, the SIFT algorithm itself is computationally complex. To maximize the number of valid feature points, the Gaussian linear pyramid layers established are excessive, resulting in redundant feature points extracted from the image. This is particularly true when matching a small number of key points, where excessive background feature points can negatively impact matching accuracy. ASIFT performs an exhaustive simulated transformation of the two camera axis direction parameters in the affine transformation, followed by SIFT matching. PSIFT shares the same spatial angle selection as the ASIFT algorithm. This predictably results in a certain degree of angular transformation that never occurs in reality. Eliminating these unpredictable transformation angles can significantly accelerate the algorithm. Furthermore, for close-range photogrammetry, the setup angles are often known, and limiting the simulated transformation angles to a known range can significantly accelerate matching efficiency. Furthermore, the PSIFT algorithm uses all simulated images from a set of simulated images obtained from the actual image to perform SIFT feature matching against the reference image, resulting in low computational efficiency. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method for fast matching of images of elliptical targets with a large viewing angle.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] According to one aspect of the present invention, a method for fast matching of images of an elliptical target with a large viewing angle is provided. The method comprises the following steps:

[0007] Step S1, using prior knowledge to estimate the number of pyramid layers where the circular target is located, thereby reducing pyramid layer redundancy in the SIFT algorithm, improving feature point detection efficiency while reducing mismatches caused by background feature points;

[0008] Step S2: In the coarse matching process, the actual image is subjected to simulated perspective deformation to obtain a simulated image and perform feature matching. The rotation angle is limited to the actual camera rotation angle. When the number of feature points of the simulated image and the reference image that are correctly matched is greater than a certain threshold K, the fine matching process is entered;

[0009] In step S3, the homography matrix of the actual image and the reference image is obtained by using the correctly matched feature points to obtain a resampled image, and the improved SIFT algorithm is used to complete the final large-viewing angle matching.

[0010] As a preferred technical solution, the step S1 of estimating the number of pyramid layers where the circular target is located using prior knowledge is specifically as follows:

[0011] An improved SIFT algorithm is used, that is, the traditional SIFT scale space division strategy is changed. By estimating the size of the circular target, it is determined that the circular target can be detected in the Nth layer of the DoG pyramid. Then the scale division can be changed to a three-layer pyramid of N, N+1, and N-1.

[0012] As a preferred technical solution, the specific process of the improved SIFT algorithm is as follows:

[0013] Step S101, constructing a three-layer pyramid: determining the N, N+1, and N-1 layers of the pyramid through prior knowledge and performing image position search;

[0014] Step S102, feature point positioning: searching for extreme points in the DoG space as image feature points in the constructed three-layer pyramid using the Gaussian difference function, and accurately locating the positions and scales of the feature points to sub-pixels by fitting a three-dimensional quadratic function;

[0015] Step S103, feature point direction assignment: For each candidate feature point, the gradient of the Gaussian scale space image where the feature point is located is calculated. Based on the local image gradient direction information, each key point position will be assigned one or more directions;

[0016] Step S104, key point descriptor: measuring the gradient of the local image in the area around each key point at a selected scale.

[0017] As a preferred technical solution, the image feature points in step S102 are scale and rotation invariant.

[0018] As a preferred technical solution, the gradient in step S104 is converted into a feature vector representation, and the feature vector is mapped to unit length.

[0019] As a preferred technical solution, the ability to eliminate image deformation during the coarse matching process in step S2 is related to the selection of the sampling interval.

[0020] As a preferred technical solution, the precise matching process in step S2 is specifically as follows:

[0021] First, all the correct matching point pairs obtained in the coarse matching stage are used to solve the transformation matrix between the actual captured image and the reference image coordinate system;

[0022] The homography matrix is ​​then used to transform the actual captured image to obtain a resampled image, further eliminating image deformation.

[0023] As a preferred technical solution, the transformation matrix is ​​a homography matrix.

[0024] As a preferred technical solution, the homography matrix is ​​obtained using the RANSAC algorithm.

[0025] As a preferred technical solution, the selection of the threshold K in step S2 determines the number of improved SIFT matches and the speed-up of the computational efficiency of the PSIFT algorithm.

[0026] Compared with the prior art, the present invention has the following advantages:

[0027] 1) The present invention matches a small number of key circular points without acquiring all feature points. Therefore, when a certain image in the simulated image set can effectively eliminate perspective distortion, other simulated images can be discontinued from the reference image for matching. Instead, the homography matrix is ​​obtained using the correct homonymous points for reprojection, achieving wide-view matching of the key points.

[0028] 2) The present invention is not only robust to changes in viewing angles, but also improves the matching efficiency and accuracy of circular targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Flowchart for circular feature extraction;

[0030] Figure 2 It is a schematic diagram of perspective projection relationship;

[0031] Figure 3 Schematic diagram of the position of the camera coordinate system and the world coordinate system;

[0032] Figure 4 This is a schematic diagram of the existing stereo matching results using ASIFT;

[0033] Figure 5 Schematic diagram of stereo matching results using the present invention. DETAILED DESCRIPTION

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort shall fall within the scope of protection of the present invention.

[0035] 1. To address the difficulty of using the Affine Scale Invariant Feature Transform (ASIFT) algorithm to stably and quickly match circular targets under close-range photogrammetry conditions when images exhibit large angular variations, this paper proposes an improved PSIFT coarse-fine accelerated matching algorithm for circular targets. First, the number of pyramid levels containing the circular target is estimated using prior knowledge, reducing pyramid level redundancy in the SIFT algorithm. This improves feature point detection efficiency while minimizing mismatches caused by background feature points. Subsequently, during the coarse matching process, the actual image is subjected to simulated perspective deformation to obtain a simulated image, which is then used for feature matching. The rotation angle is limited to the actual camera rotation angle. When the number of correctly matched feature points between the simulated image and the reference image exceeds a certain threshold, the fine matching process begins. Finally, the homography matrix of the actual and reference images is derived using the correctly matched feature points, resulting in a resampled image, ultimately achieving wide-angle matching. Experimental results demonstrate that compared to existing feature matching algorithms, the proposed algorithm is not only robust to perspective variations but also improves the matching efficiency and accuracy of circular targets.

[0036] 2. Circular feature point detection

[0037] Since the feature detector of the SIFT algorithm has an extreme response at the center of the circle, this feature can be used to locate circular marks. The main matching object of this patent is circular mark points under a large viewing angle, so the traditional SIFT scale space division strategy can be modified. That is, by estimating the size of the circular target, it can be determined that the circular target can be detected in the Nth layer of the DoG pyramid. Then the scale division can be changed to a three-layer pyramid of N, N+1, and N-1, which not only ensures the detection rate of the circular target, reduces the appearance of background feature points, but also greatly speeds up the overall matching speed. Figure 1 As shown, the improved SIFT algorithm is decomposed into the following four steps:

[0038] (1) Constructing a three-layer pyramid: Determine the N, N, N+1, and N-1 layers of the pyramid based on prior knowledge to search for image locations.

[0039] (2) Feature point positioning: In the constructed three-layer pyramid, the Gaussian difference function is used to search for the extreme points in the DoG space as image feature points. The position and scale of the feature points are accurately determined to sub-pixel by fitting a three-dimensional quadratic function. These feature points are scale- and rotation-invariant.

[0040] (3) Feature point direction assignment: For each candidate feature point, the gradient of the Gaussian scale space image where the feature point is located is calculated. Based on the local image gradient direction information, each key point position will be assigned one or more directions.

[0041] (4) Keypoint descriptor: The gradient of the local image in the region around each keypoint is measured at a selected scale. These gradients are converted into a feature vector representation, and the feature vector is mapped to unit length.

[0042] 3. Improved PSIFT coarse-fine accelerated matching algorithm

[0043] The ASIFT algorithm is very resistant to perspective changes, but affine transformations do not conform to the actual perspective projection model. This patent draws on the perspective model in PSIFT to accelerate the SIFT algorithm for circular target recognition while limiting the range of simulated transformations in the actual image to further reduce algorithm redundancy. Finally, the homography matrix between the actual image and the reference image is obtained by correctly matching feature points to obtain a resampled image. The improved SIFT algorithm is used to complete the final large-view matching, greatly increasing the efficiency of the matching algorithm.

[0044] 3.1 ASIFT Algorithm Overview

[0045] The main idea of ​​the ASIFT algorithm is to use an affine change model to simulate viewpoints in real images, create simulated images from these simulated viewpoints, and use SIFT to extract and match features. Its affine transformation model is as follows, where A is the transformation matrix, where κ is the rotation angle, and θ is the rotation along the camera optical axis. is the slope parameter. R θ ,R κ is the rotation matrix.

[0046]

[0047] The ASIFT algorithm matching process is as follows:

[0048] (1) By varying t and κ through interval sampling, a set of affine transformation matrices is obtained;

[0049] (2) Using the affine transformation matrix, perform affine transformation on the actual image and the reference image respectively to obtain two sets of simulated images;

[0050] (3) The two sets of simulated images are matched using the SIFT algorithm, the correct image groups are matched, the duplicate feature points are deleted, and all matching results are displayed on the original image to complete the matching.

[0051] 3.2 Perspective model construction and simulation transformation angle constraints

[0052] Because the ASIFT algorithm uses an affine transformation model to simulate images, this patent uses a perspective transformation model to further enhance the accuracy of perspective simulation and more accurately reflect the image deformation caused by changes in camera shooting angle. The coordinate systems used in camera pinhole imaging are the camera coordinate system and the image coordinate system.

[0053] The positions of the same point p in the object in the three coordinate systems have the following relationship:

[0054]

[0055] in It is the perspective transformation model, which describes the transformation relationship between the image coordinate system and the world coordinate system.

[0056] Since the image intrinsic parameters have been calibrated and remain unchanged for a certain period of time, we do not need to discuss them. In the perspective projection relationship, the camera extrinsic parameters are λ, θ, Three angles, angles Corresponding to the slope t. Then, if we roughly know the camera rotation angle, we can constrain the simulated transformation angle by limiting the transformation angle range, thereby reducing the amount of algorithm calculation.

[0057] 3.3 Coarse and Fine Accelerated Matching Algorithm

[0058] The PSIFT algorithm uses SIFT feature matching between all simulated images in a simulated image set and a reference image, resulting in low computational efficiency. When matching a small number of circular landmarks, if a certain image in the simulated image set effectively eliminates perspective distortion, the remaining simulated images can be discontinued from the matching process. This paper proposes a coarse-to-fine accelerated matching method. The coarse matching stage of the accelerated PSIFT algorithm, as described in the previous section, simulates perspective distortion by sampling the perspective transformation model parameters within a restricted range. This simulated image is then used with the reference image for improved SIFT feature extraction and description. When the number of correct matching point pairs between a simulated image and the reference image reaches K during the coarse matching stage, the coarse matching stage ends and the fine matching process begins.

[0059] The ability to eliminate image distortion during the coarse matching process is related to the choice of sampling interval. To minimize the impact of the sampling interval on the matching algorithm and improve matching performance, the acceleration algorithm incorporates a fine matching process. During the fine matching process, all correctly matched point pairs obtained during the coarse matching phase are first used to calculate the transformation matrix between the actual captured image and the reference image coordinate system, known as the homography. This homography is then used to transform the actual captured image to obtain a resampled image, further eliminating image distortion and improving matching accuracy. This homography is obtained using the RANSAC algorithm and is more accurate than the perspective transformation matrix obtained by sampling during the coarse matching phase.

[0060] The choice of the preset value K determines the number of SIFT matches and the efficiency of the PSIFT algorithm. The fine matching stage uses the RANSAC method to solve the homography matrix, which only requires four matching point pairs. Therefore, in theory, N ≥ 4 is sufficient. However, in practice, since the accuracy of the four matching point pairs obtained in the coarse matching stage cannot be guaranteed, it is necessary to appropriately increase this threshold.

[0061] 4. Experimental Results and Analysis

[0062] The binocular industrial camera calibration in a vibration table experiment in a certain experiment was taken as an example to verify the matching efficiency and accuracy of the wide-angle image fast matching method of elliptical targets proposed in this patent. The industrial high-speed camera used was equipped with a fixed-focus lens with a focal length of 20mm. The intersection angle of the two cameras was greater than 90°. The vibration table experiment scene contained a large number of complex facilities and uneven lighting. In this scene, the planar target was oriented using photogrammetry coding points and QR codes, and the matching success rate, efficiency and accuracy were analyzed.

[0063] (1) Feasibility of fast matching method for large-viewing angle images of elliptical targets

[0064] Figure 5 The final matching results of using ASIFT algorithm and improved PSIFT coarse-fine accelerated matching algorithm are shown, with a total of 8 circular targets.

[0065] Obviously, the ASIFT algorithm, due to the excessive number of background feature points, causes RANSAC to filter out correct matching results, resulting in correct local matching but lacking a correct global matching effect. The method proposed in this patent can correctly obtain correct global matching results. Table 1 shows the number of correct circular target matches.

[0066] Table 1

[0067] ASIFT This patent Correct number of matches 2 8

[0068] (2) The running time comparison between this patent and ASIFT algorithm is shown in Table 2;

[0069] Table 2

[0070] ASIFT This patent Runtime 48.27s 14.67s

[0071] For the same scene, under the same computer test environment and conditions, the two matching speeds are ASIFT: 48.27s, and this patent: 14.67s.

[0072] 5. Conclusion

[0073] To address the image matching problem of circular objects in scenes with large perspective variations, this paper proposes a method that limits the number of pyramid levels to ensure the recognition rate and subsequent matching efficiency of circular objects. By building a perspective projection model, the large-view matching model more closely matches actual perspective variations, mimicking the transformation angle constraints and reducing algorithmic redundancy. The proposed coarse-fine accelerated large-view matching strategy improves matching efficiency while significantly reducing computational complexity. Experiments demonstrate that this method demonstrates robustness in circular object matching experiments under large perspective variations and significantly improves matching efficiency compared to the ASIFT algorithm. However, some areas for improvement remain, such as the diversity of matching scenarios and multiple experimental tests.

[0074] The foregoing description is merely a specific embodiment of the present invention, and the scope of protection of the present invention is not limited thereto. Any person skilled in the art will readily conceive of various equivalent modifications or substitutions within the technical scope disclosed herein, and such modifications or substitutions are intended to be encompassed by the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for fast matching of large-viewing angle images of elliptical targets, characterized in that: The method comprises the following steps: Step S1, using prior knowledge to estimate the number of pyramid layers where the circular target is located, thereby reducing pyramid layer redundancy in the SIFT algorithm, improving feature point detection efficiency while reducing mismatches caused by background feature points; Step S2: In the coarse matching process, the actual image is subjected to simulated perspective deformation to obtain a simulated image and perform feature matching. The rotation angle is limited to the actual camera rotation angle. When the number of feature points of the simulated image and the reference image that are correctly matched is greater than a certain threshold K, the fine matching process is entered; Step S3, using the correctly matched feature points to obtain the homography matrix of the actual image and the reference image, obtain a resampled image, and use the improved SIFT algorithm to complete the final large-viewing angle matching; The step S1, using prior knowledge to estimate the number of pyramid layers where the circular target is located, is specifically: The improved SIFT algorithm is used, that is, the traditional SIFT scale space division strategy is changed. By estimating the size of the circular target, it is determined that the circular target can be detected in the Nth layer of the DoG pyramid. Then the scale division can be changed to a three-layer pyramid of N, N+1, and N-1. The specific process of the improved SIFT algorithm is as follows: Step S101, constructing a three-layer pyramid: determining the N, N+1, and N-1 layers of the pyramid through prior knowledge and performing image position search; Step S102, feature point positioning: searching for extreme points in the DoG space as image feature points in the constructed three-layer pyramid using the Gaussian difference function, and accurately locating the positions and scales of the feature points to sub-pixels by fitting a three-dimensional quadratic function; Step S103, feature point direction assignment: For each candidate feature point, the gradient of the Gaussian scale space image where the feature point is located is calculated. Based on the local image gradient direction information, each key point position will be assigned one or more directions; Step S104, key point descriptor: measuring the gradient of the local image in the area around each key point at a selected scale.

2. The method for fast matching of large viewing angle images of an elliptical target according to claim 1, characterized in that: The image feature points in step S102 are scale and rotation invariant.

3. The method for fast matching of large viewing angle images of an elliptical target according to claim 1, characterized in that: The gradient in step S104 is converted into a feature vector representation, and the feature vector is mapped to unit length.

4. The method for fast matching of large viewing angle images of an elliptical target according to claim 1, characterized in that: The ability to eliminate image deformation during the rough matching process in step S2 is related to the selection of the sampling interval.

5. The method for fast matching of large viewing angle images of an elliptical target according to claim 1, characterized in that: The precise matching process in step S2 is specifically as follows: First, all the correct matching point pairs obtained in the coarse matching stage are used to solve the transformation matrix between the actual captured image and the reference image coordinate system; The homography matrix is ​​then used to transform the actual captured image to obtain a resampled image, further eliminating image deformation.

6. The method for fast matching of large viewing angle images of an elliptical target according to claim 5, characterized in that: The transformation matrix is ​​a homography matrix.

7. The method for fast matching of large viewing angle images of an elliptical target according to claim 6, characterized in that: The homography matrix is ​​obtained using the RANSAC algorithm.

8. The method for fast matching of large viewing angle images of an elliptical target according to claim 1, characterized in that: The selection of the threshold value K in step S2 determines the number of improved SIFT matches and the computational efficiency of the accelerated PSIFT algorithm.

Images