A method, storage medium and device for detecting mismatched points of RTK images

Abstract

The present invention discloses a method, storage medium, and device for detecting mismatched points in RTK images, relating to the field of RTK image measurement technology. The key points of the technical solution are as follows: first, performing preliminary matching of feature points, retaining feature points less than a threshold value T among the preliminary matched feature points, and treating the remaining as mismatched points; determining whether the camera posture change is less than the threshold value T; calculating the position increment of the retained feature points on the image; calculating the motion direction and motion distance of each retained feature point on the image based on the position increment; calculating the mean and standard deviation of the motion direction and motion distance of the retained feature points on the image; determining the consistency of the motion direction and motion distance of each feature point with the mean; and using the RANSAC algorithm to detect and eliminate mismatched points among the retained feature points. The present invention utilizes the principle of consistency of the motion direction and motion distance of feature points in adjacent images to detect and eliminate mismatched points, thereby improving image measurement accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of RTK image measurement, and more particularly to a method, storage medium and device for detecting mismatched points of RTK images. Background Art

[0002] In the actual application of RTK image measurement, the photos taken by users may involve various harsh scenes, such as areas with few targets or drastic changes in illumination between adjacent images. In these scenes, accurately matching feature points between images is a difficult task. Poor feature point matching will directly affect the optimization accuracy of the camera pose, and ultimately the accuracy of RTK image measurement. For image feature point matching in harsh scenes, brute force matching or fast nearest neighbor matching is usually used. The matched feature point results are poor and there are a large number of mismatches. You can first use Lowe's algorithm to eliminate a large number of mismatches, and then use the RANSAC algorithm to eliminate a small number of mismatches. However, even with the above two methods, a small number of mismatches still exist. When the total number of matched feature points in the camera pose is small, these mismatches will seriously affect the camera pose accuracy, resulting in poor measurement results.

[0003] Therefore, a new solution needs to be proposed to solve this problem. Summary of the Invention

[0004] In response to the shortcomings of the existing technology, the purpose of the present invention is to provide a method, storage medium and device for detecting mismatched points in RTK images. When the camera posture does not change much, the principle of consistency in the movement direction and movement distance of feature points in adjacent images is used to detect and eliminate mismatched points. To a certain extent, the impact on camera posture estimation can be reduced, and the image measurement accuracy can be effectively improved.

[0005] The above technical objectives of the present invention are achieved through the following technical solutions: A method for detecting mismatched points in RTK images, comprising the following steps:

[0006] S1. Perform a preliminary match on the feature points of two adjacent images, set a threshold, use Lowe's algorithm to detect the feature points smaller than the threshold among the preliminary matched feature points, and retain them. The rest are treated as mismatched points and are removed.

[0007] S2, determine whether the camera posture change is less than the threshold, if so, proceed to the next step;

[0008] S3. After completing step S2, calculate the position increment of the feature points retained in step S1 on two adjacent images;

[0009] S4, calculating the movement direction and movement distance of each retained feature point in two adjacent images according to the position increment;

[0010] S5. Calculate the mean and standard deviation of the motion direction and motion distance of all retained feature points on the image;

[0011] S6. Compare the movement direction and movement distance of each retained feature point with the mean. If the consistency judgment formula is satisfied, the feature point is retained; otherwise, it is considered a mismatched point and is removed.

[0012] S7. Use the RANSAC algorithm to detect and eliminate mismatched points among the feature points retained in step S6.

[0013] In one embodiment, in step S1, the specific method for performing preliminary matching on feature points of two adjacent images is: first extracting feature points and calculating descriptors of the two adjacent images, and then performing preliminary matching on the feature points using a brute force matcher.

[0014] In one embodiment, the threshold is based on Lowe's algorithm and is determined by the ratio of the distance of the closest matching feature point to the distance of the second closest matching feature point.

[0015] In one embodiment, if the effect of removing mismatched points in step S6 is not good, steps S2 to S6 are repeated.

[0016] In one embodiment, the calculation formula for the position increment of the feature points retained in step S1 on two adjacent images is as follows:

[0017]

[0018] Among them, dx and dy represent the pixel coordinate increments of the feature points on the x-axis and y-axis of two adjacent images, respectively, and p x,k and p y,k Respectively represent the x-axis and y-axis coordinates of the feature point on the k-th frame image; p x,k+1 and p y,k+1 They represent the x-axis and y-axis coordinates of the feature point on the k+1th frame image respectively.

[0019] In one embodiment, the calculation formula for the motion direction of the retained feature points in two adjacent images is as follows:

[0020]

[0021] The calculation formula for the motion distance of the retained feature points in two adjacent images is as follows:

[0022]

[0023] Where θ represents the direction of movement and d represents the distance of movement.

[0024] In one embodiment, the calculation formulas for the mean and standard deviation of the motion direction and motion distance of the retained feature points on the image are as follows:

[0025]

[0026] in, and Respectively represent the mean of the motion direction and motion distance of the retained feature points on the image, σ θ and σ d They represent the standard deviation of the motion direction and motion distance of the retained feature points on the image.

[0027] In one embodiment, the consistency determination formula is:

[0028]

[0029] A computer-readable storage medium includes a program for detecting mismatched points of RTK images. When the program is executed by a processor, any of the methods for detecting mismatched points of RTK images is implemented.

[0030] A computer device includes a processor, a memory, a communication interface, and a communication bus. The processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store a program for detecting mismatched points in RTK images. The program enables the processor to perform operations corresponding to any of the methods for detecting mismatched points in RTK images.

[0031] In summary, the present invention has the following beneficial effects: the present invention first uses Lowe's algorithm to detect and eliminate most of the mismatched points, and then combines the camera posture change and the position increment of the retained feature points on the adjacent images to further analyze the movement direction and movement distance of the retained feature points, and retains the feature points that conform to the overall movement trend through the consistency judgment formula, ensuring that when the camera posture changes little, a small number of mismatched points contained in these feature points can be eliminated, and finally uses the RANSAC algorithm to detect and eliminate a very small number of mismatched points in the feature points retained after consistency judgment. Through meticulous feature point analysis and screening, the accuracy of feature point matching is significantly improved, the camera posture optimization accuracy is enhanced, and it can adapt to various harsh scenes, reduce the influence of mismatched points on the measurement results, and effectively improve the measurement accuracy of the image. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0033] The embodiment of the present application provides a method for detecting mismatched points in RTK images, comprising the following steps:

[0034] S1. Perform a preliminary match on the feature points of two adjacent images, set a threshold, and use Lowe's algorithm to detect the feature points smaller than the threshold among the preliminary matched feature points. These points are retained and the rest are treated as mismatched points and removed. A large number of mismatched points can be removed in this process.

[0035] S2, determine whether the camera posture change is less than the threshold, if so, proceed to the next step;

[0036] S3. After completing step S2, calculate the position increment of the feature points retained in step S1 on two adjacent images;

[0037] S4, calculating the movement direction and movement distance of each retained feature point in two adjacent images according to the position increment;

[0038] S5. Calculate the mean and standard deviation of the motion direction and motion distance of all retained feature points on the image. If the condition of step S2 is met, the motion direction and motion distance of all retained feature points should be roughly consistent. Therefore, calculate their mean and standard deviation as the consistency detection standard;

[0039] S6. Compare the movement direction and movement distance of each retained feature point with the mean. If the consistency judgment formula is satisfied, the feature point is retained; otherwise, it is considered a mismatched point and is removed.

[0040] S7. Use the RANSAC algorithm to detect and eliminate mismatched points among the feature points retained in step S6.

[0041] Specifically, in step S2, the relative pose change of the camera is estimated by calculating the basic matrix between two adjacent images. The basic matrix can be solved by the eight-point algorithm or the RANSAC algorithm. After obtaining the basic matrix, the rotation matrix and the translation vector of the camera are calculated, and then it is determined whether the change in the camera pose is less than a preset threshold. In this embodiment, the norm of the rotation matrix and the modulus of the translation vector are set as the measure of the pose change. If it is less than T, it is considered that the camera pose change is not large, and the method of the present invention can continue to be used. If it is greater than T, the method of the present invention cannot be used.

[0042] In the above method, Lowe's algorithm is first used to detect and eliminate most of the mismatched points. Then, the movement direction and distance of the retained feature points are further analyzed in combination with the camera posture change and the position increment of the retained feature points on the adjacent images. The feature points that conform to the overall movement trend are retained through the consistency judgment formula, ensuring that a small number of mismatched points contained in these feature points can be eliminated when the camera posture changes little. Finally, the RANSAC algorithm is used to detect and eliminate a very small number of mismatched points among the feature points retained after consistency judgment. Through meticulous feature point analysis and screening, the accuracy of feature point matching is significantly improved, the camera posture optimization accuracy is enhanced, and it can adapt to various harsh scenes, reduce the impact of mismatched points on the measurement results, and effectively improve the measurement accuracy of the image.

[0043] In this embodiment, in step S1, the specific method for performing preliminary matching on the feature points of two adjacent images is: firstly extracting feature points and calculating descriptors of the two adjacent images, and then performing preliminary matching on the feature points using a brute force matcher.

[0044] In this embodiment, the threshold is based on Lowe's algorithm and is determined by the ratio of the distance of the closest matching feature point to the distance of the second closest matching feature point.

[0045] In this embodiment, if the effect of eliminating the mismatched points in step S6 is not good, for example, if the proportion of mismatched points in the retained feature points is still high, steps S2 to S6 are repeated.

[0046] In this embodiment, the specific method of incrementing the position of the feature point retained in step S1 on two adjacent images is to traverse all matched retained feature points and use the pixel coordinates of the feature point on the second image minus the pixel coordinates on the first image, i.e., the position increment, which is calculated as follows:

[0047]

[0048] Among them, dx and dy represent the pixel coordinate increments of the feature points on the x-axis and y-axis of two adjacent images, respectively, and p x,k and p y,k Respectively represent the x-axis and y-axis coordinates of the feature point on the k-th frame image; p x,k+1 and p y,k+1 They represent the x-axis and y-axis coordinates of the feature point on the k+1th frame image respectively.

[0049] In this embodiment, the calculation formula for the motion direction of the retained feature points in two adjacent images is as follows:

[0050]

[0051] The calculation formula for the motion distance of the retained feature points in two adjacent images is as follows:

[0052]

[0053] Where θ represents the direction of movement and d represents the distance of movement.

[0054] In this embodiment, the calculation formulas for the mean and standard deviation of the motion direction and motion distance of the retained feature points on the image are as follows:

[0055]

[0056] in, and Respectively represent the mean of the motion direction and motion distance of the retained feature points on the image, σ θ and σ d They represent the standard deviation of the motion direction and motion distance of the retained feature points on the image.

[0057] In this embodiment, the consistency judgment formula is:

[0058]

[0059] In step S6, all retained feature points are traversed, and the consistency formula is used as the detection standard for mismatched points. For each retained feature point, its movement direction and movement distance are compared with the mean. If the difference between the movement direction and movement distance of a feature point and the mean is less than or equal to 2 times the standard deviation (this is an empirical value and can be adjusted according to actual conditions), the feature point is considered to be matched correctly and is retained; otherwise, the point is considered to be a mismatched point and is eliminated.

[0060] The present invention also discloses a computer-readable storage medium, which includes a program for detecting mismatched points of RTK images. When the program for detecting mismatched points of RTK images is executed by a processor, any of the methods for detecting mismatched points of RTK images is implemented.

[0061] The present invention also discloses a computer device, including a processor, a memory, a communication interface and a communication bus. The processor, memory and communication interface communicate with each other through the communication bus. The memory is used to store a program for detecting mismatched points of RTK images. The program for detecting mismatched points of RTK images enables the processor to perform operations corresponding to any of the operations for detecting mismatched points of RTK images.

[0062] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting mismatched points in RTK images, characterized by: The steps include: S1. Perform a preliminary match on the feature points of two adjacent images, set a threshold, use Lowe's algorithm to detect the feature points smaller than the threshold among the preliminary matched feature points, and retain them. The rest are treated as mismatched points and are removed. S2, determine whether the camera posture change is less than the threshold, if so, proceed to the next step; S3. After completing step S2, calculate the position increment of the feature points retained in step S1 on two adjacent images; S4, calculating the movement direction and movement distance of each retained feature point in two adjacent images according to the position increment; S5. Calculate the mean and standard deviation of the motion direction and motion distance of all retained feature points on the image; S6. Compare the movement direction and movement distance of each retained feature point with the mean. If the consistency judgment formula is satisfied, the feature point is retained; otherwise, it is considered a mismatched point and is removed. S7. Use the RANSAC algorithm to detect and eliminate mismatched points among the feature points retained in step S6.

2. The method for detecting mismatched points in RTK images according to claim 1, wherein: In step S1, the specific method for performing preliminary matching on the feature points of two adjacent images is: firstly extracting the feature points and calculating the descriptors of the two adjacent images, and then using a brute force matcher to perform preliminary matching on the feature points.

3. The method for detecting mismatched points in RTK images according to claim 1, wherein: The threshold is determined based on Lowe's algorithm and is determined by the ratio of the distance of the closest matching feature point to the distance of the second closest matching feature point.

4. The method for detecting mismatched points in RTK images according to claim 1, wherein: If the effect of removing the mismatched points in step S6 is not good, then steps S2 to S6 are repeated.

5. The method for detecting mismatched points in RTK images according to claim 1, wherein: The calculation formula for the position increment of the feature points retained in step S1 on two adjacent images is as follows: Among them, dx and dy represent the pixel coordinate increments of the feature points on the x-axis and y-axis of two adjacent images, respectively, and p x,k and p y,k Respectively represent the x-axis and y-axis coordinates of the feature point on the k-th frame image; p x,k+1 and p y,k+1 They represent the x-axis and y-axis coordinates of the feature point on the k+1th frame image respectively.

6. The method for detecting mismatched points in RTK images according to claim 1, wherein: The calculation formula for the motion direction of the retained feature points in two adjacent images is as follows: The calculation formula for the motion distance of the retained feature points in two adjacent images is as follows: Where θ represents the direction of movement and d represents the distance of movement.

7. The method for detecting mismatched points in RTK images according to claim 1, wherein: The calculation formulas for the mean and standard deviation of the motion direction and motion distance of the retained feature points on the image are as follows: in, and Respectively represent the mean of the motion direction and motion distance of the retained feature points on the image, σ θ and σ d They represent the standard deviation of the motion direction and motion distance of the retained feature points on the image.

8. The method for detecting mismatched points in RTK images according to claim 1, wherein: The consistency judgment formula is:

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a program for detecting mismatched points of RTK images. When the program for detecting mismatched points of RTK images is executed by a processor, the method for detecting mismatched points of RTK images as described in any one of claims 1 to 8 is implemented.

10. A computer device, characterized in that: The invention comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus, the memory is used to store a program of a method for detecting mismatched points of RTK images, and the program of a method for detecting mismatched points of RTK images enables the processor to perform operations corresponding to the method for detecting mismatched points of RTK images according to any one of claims 1 to 8.

Images