RTK image mismatching point detection method, storage medium and device

Abstract

The invention discloses an RTK image mismatching point detection method, a storage medium and an RTK image mismatching point detection device, and relates to the technical field of RTK image measurement, and the technical scheme is characterized in that the method comprises the following steps: carrying out the preliminary matching of feature points, keeping the feature points smaller than a threshold T in the preliminarily matched feature points, and taking the rest as mismatching points; judging whether the camera posture change is smaller than a threshold T; calculating the position increment of the reserved feature points on the image; calculating the motion direction and the motion distance of each reserved feature point on the image according to the position increment; calculating the mean value and the standard deviation of the motion direction and the motion distance of the reserved feature points on the image; the consistency between the motion direction and the motion distance of each feature point and the mean value is judged; and detecting and eliminating mismatching points in the reserved feature points by using an RANSAC (Random Sample Consensus) algorithm. According to the method, the principle that the motion direction and the motion distance of the feature points in the adjacent images are consistent is utilized, mismatching points are detected and eliminated, and the measurement precision of the images is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of RTK image measurement, and more specifically, to a method, storage medium and device for detecting mismatched points of RTK images. Background Art

[0002] In the actual application of RTK image measurement, the photos taken by users may involve various harsh scenes, such as areas with few targets and drastic changes in the illumination of adjacent images. In these scenes, accurate matching of feature points between images is a difficult task. When the feature point matching is poor, it will directly affect the optimization accuracy of the camera posture, and ultimately affect the accuracy of RTK image measurement. For image feature point matching in harsh scenes, brute force matching or fast nearest neighbor matching is usually used. The matching feature point results are poor and there are a large number of mismatches. You can first use Lowe's algorithm to remove a large number of mismatch points, and then use the RANSAC algorithm to remove a small number of mismatch points. However, there are still a small number of mismatch points through the above two methods. When the total number of matching feature points of the camera posture is small, these mismatch points will seriously affect the accuracy of the camera posture, resulting in poor measurement results.

[0003] Therefore, a new solution needs to be proposed to solve this problem. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a method, storage medium and device for detecting mismatched points of RTK images. When the camera posture does not change much, the principle of consistency of the movement direction and movement distance of feature points in adjacent images is used to detect and eliminate mismatched points, which can reduce the impact on camera posture estimation to a certain extent and effectively improve the image measurement accuracy.

[0005] The above technical objectives of the present invention are achieved through the following technical solutions: A method for detecting mismatched points of RTK images, comprising the following steps:

[0006] S1. Perform a preliminary match on the feature points of two adjacent images, set a threshold T, use Lowe's algorithm to detect the feature points smaller than the threshold T in the preliminary matching feature points, and retain them, and remove the rest as mismatched points;

[0007] S2, determine whether the camera posture change is less than the threshold T, if it is less than the threshold T, proceed to the next step;

[0008] S3, after completing step S2, calculating the position increment of the feature points retained in step S1 on two adjacent images;

[0009] S4, calculating the movement direction and movement distance of each retained feature point on two adjacent images according to the position increment;

[0010] S5, calculating the mean and standard deviation of the movement direction and movement distance of all retained feature points on the image;

[0011] S6, comparing the movement direction and movement distance of each retained feature point with the mean value, if the consistency judgment formula is satisfied, the feature point is retained, otherwise it is considered as a mismatch point and is removed;

[0012] S7. Use the RANSAC algorithm to detect and remove mismatched points among the feature points retained in step S6.

[0013] In one of the embodiments, in step S1, the specific method for performing preliminary matching on the feature points of two adjacent images is: firstly extracting the feature points and calculating the descriptors of the two adjacent images, and then using a brute force matcher to perform preliminary matching on the feature points.

[0014] In one embodiment, the threshold T is determined based on Lowe's algorithm by the ratio of the distance of the closest matching feature point to the distance of the second closest matching feature point.

[0015] In one embodiment, if the effect of removing the mismatched points in step S6 is not good, steps S2 to S6 are repeated.

[0016] In one embodiment, the calculation formula for the position increment of the feature points retained in step S1 on two adjacent images is as follows:

[0017]

[0018] Among them, dx and dy represent the pixel coordinate increments of the feature points on the x-axis and y-axis of two adjacent images, respectively. x,k and p y,k They represent the x-axis and y-axis coordinates of the feature point on the k-th frame image; p x,k+1 and p y,k+1 They respectively represent the x-axis and y-axis coordinates of the feature point in the k+1th frame image.

[0019] In one embodiment, the calculation formula for the movement direction of the retained feature points in two adjacent images is as follows:

[0020]

[0021] The calculation formula for the motion distance of the retained feature points in two adjacent images is as follows:

[0022]

[0023] Among them, θ represents the direction of movement; d represents the distance of movement.

[0024] In one embodiment, the calculation formulas for the mean and standard deviation of the movement direction and movement distance of the retained feature points on the image are as follows:

[0025]

[0026] in, and Respectively represent the mean of the motion direction and motion distance of the retained feature points on the image, σ θ and σ d They respectively represent the standard deviation of the motion direction and motion distance of the retained feature points on the image.

[0027] In one embodiment, the consistency judgment formula is:

[0028]

[0029] A computer-readable storage medium includes a method program for detecting mismatched points of RTK images. When the method program for detecting mismatched points of RTK images is executed by a processor, any of the methods for detecting mismatched points of RTK images described above is implemented.

[0030] A computer device includes a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus, the memory is used to store a program for a method for detecting mismatched points of RTK images, and the program for detecting mismatched points of RTK images enables the processor to perform operations corresponding to any of the methods for detecting mismatched points of RTK images.

[0031] In summary, the present invention has the following beneficial effects: the present invention first uses Lowe's algorithm to detect and eliminate most of the mismatching points, and then combines the camera posture change and the position increment of the retained feature points on the adjacent images to further analyze the movement direction and movement distance of the retained feature points, and retains the feature points that meet the overall movement trend through the consistency judgment formula, ensuring that when the camera posture does not change much, a small number of mismatching points contained in these feature points can be eliminated, and finally uses the RANSAC algorithm to detect and eliminate a very small number of mismatching points in the feature points retained after consistency judgment. Through meticulous feature point analysis and screening, the accuracy of feature point matching is significantly improved, the camera posture optimization accuracy is enhanced, and it can adapt to various harsh scenes, reduce the influence of mismatching points on the measurement results, and effectively improve the measurement accuracy of the image. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present invention will be described clearly and completely below in combination with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The embodiment of the present application provides a method for detecting mismatched points of an RTK image, comprising the following steps:

[0034] S1. Perform a preliminary match on the feature points of two adjacent images, set a threshold T, use Lowe's algorithm to detect the feature points smaller than the threshold T in the preliminary matching feature points, and retain them. The rest are treated as mismatched points and removed. A large number of mismatched points can be removed in this process.

[0035] S2, determine whether the camera posture change is less than the threshold T, if it is less than the threshold T, proceed to the next step;

[0036] S3, after completing step S2, calculating the position increment of the feature points retained in step S1 on two adjacent images;

[0037] S4, calculating the movement direction and movement distance of each retained feature point on two adjacent images according to the position increment;

[0038] S5, calculating the mean and standard deviation of the movement direction and movement distance of all the retained feature points on the image. Under the condition of step S2, the movement direction and movement distance of all the retained feature points should be roughly consistent, so the mean and standard deviation are calculated as the consistency detection standard;

[0039] S6, comparing the movement direction and movement distance of each retained feature point with the mean value, if the consistency judgment formula is satisfied, the feature point is retained, otherwise it is considered as a mismatch point and is removed;

[0040] S7. Use the RANSAC algorithm to detect and remove mismatched points among the feature points retained in step S6.

[0041] Specifically, in step S2, the relative posture change of the camera is estimated by calculating the basic matrix between two adjacent images. The basic matrix can be solved by the eight-point algorithm or the RANSAC algorithm. After obtaining the basic matrix, the rotation matrix and the translation vector of the camera are calculated, and then it is determined whether the change of the camera posture is less than a preset threshold value T. In this embodiment, the norm of the rotation matrix and the modulus of the translation vector are set as the measure of the posture change. If it is less than T, it is considered that the camera posture change is not large, and the method of the present invention can continue to be used. If it is greater than T, the method of the present invention cannot be used.

[0042] In the above method, Lowe's algorithm is first used to detect and eliminate most of the mismatched points. Then, the movement direction and distance of the retained feature points are further analyzed in combination with the camera posture change and the position increment of the retained feature points on the adjacent images. The feature points that meet the overall movement trend are retained through the consistency judgment formula, ensuring that a small number of mismatched points contained in these feature points can be eliminated when the camera posture does not change much. Finally, the RANSAC algorithm is used to detect and eliminate a very small number of mismatched points in the feature points retained after consistency judgment. Through meticulous feature point analysis and screening, the accuracy of feature point matching is significantly improved, the camera posture optimization accuracy is enhanced, and it can adapt to various harsh scenes, reduce the impact of mismatched points on measurement results, and effectively improve the measurement accuracy of images.

[0043] In this embodiment, in step S1, the specific method for performing preliminary matching on the feature points of two adjacent images is: firstly extracting the feature points and calculating the descriptors of the two adjacent images, and then using a brute force matcher to perform preliminary matching on the feature points.

[0044] In this embodiment, the threshold T is based on Lowe's algorithm and is determined by the ratio of the distance of the closest matching feature point to the distance of the second closest matching feature point.

[0045] In this embodiment, if the effect of removing the mismatched points in step S6 is not good, for example, the proportion of mismatched points in the retained feature points is still high, then steps S2 to S6 are repeated.

[0046] In this embodiment, the specific method of calculating the position increment of the feature points retained in step S1 on two adjacent images is: traverse all matched retained feature points, and use the pixel coordinates of the feature points on the second image minus the pixel coordinates on the first image, that is, the position increment, and the calculation formula is as follows:

[0047]

[0048] Among them, dx and dy represent the pixel coordinate increments of the feature points on the x-axis and y-axis of two adjacent images, respectively. x,k and p y,k They represent the x-axis and y-axis coordinates of the feature point on the k-th frame image; p x,k+1 and p y,k+1 They respectively represent the x-axis and y-axis coordinates of the feature point in the k+1th frame image.

[0049] In this embodiment, the calculation formula for the movement direction of the retained feature points in two adjacent images is as follows:

[0050]

[0051] The calculation formula for the motion distance of the retained feature points in two adjacent images is as follows:

[0052]

[0053] Among them, θ represents the direction of movement; d represents the distance of movement.

[0054] In this embodiment, the calculation formulas for the mean and standard deviation of the movement direction and movement distance of the retained feature points on the image are as follows:

[0055]

[0056] in, and Respectively represent the mean of the motion direction and motion distance of the retained feature points on the image, σ θ and σ d They respectively represent the standard deviation of the motion direction and motion distance of the retained feature points on the image.

[0057] In this embodiment, the consistency judgment formula is:

[0058]

[0059] In step S6, all retained feature points are traversed, and the consistency formula is used as the detection standard for mismatched points. For each retained feature point, its movement direction and movement distance are compared with the mean. If the difference between the movement direction and movement distance of a feature point and the mean is less than or equal to 2 times the standard deviation (this is an empirical value and can be adjusted according to actual conditions), the feature point is considered to be matched correctly and the point is retained; otherwise, the point is considered to be a mismatched point and is eliminated.

[0060] The present invention also discloses a computer-readable storage medium, which includes a program for detecting mismatched points of RTK images. When the program for detecting mismatched points of RTK images is executed by a processor, any of the methods for detecting mismatched points of RTK images is implemented.

[0061] The present invention also discloses a computer device, including a processor, a memory, a communication interface and a communication bus. The processor, the memory and the communication interface communicate with each other through the communication bus. The memory is used to store a program for detecting mismatched points of RTK images. The program for detecting mismatched points of RTK images enables the processor to perform operations corresponding to any of the methods for detecting mismatched points of RTK images.

[0062] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for detecting mismatched points of RTK images, characterized in that: The steps include: S1. Perform a preliminary match on the feature points of two adjacent images, set a threshold T, use Lowe's algorithm to detect the feature points smaller than the threshold T in the preliminary matching feature points, and retain them, and remove the rest as mismatched points; S2, determine whether the camera posture change is less than the threshold T, if it is less than the threshold T, proceed to the next step; S3, after completing step S2, calculating the position increment of the feature points retained in step S1 on two adjacent images; S4, calculating the movement direction and movement distance of each retained feature point on two adjacent images according to the position increment; S5, calculating the mean and standard deviation of the movement direction and movement distance of all retained feature points on the image; S6, comparing the movement direction and movement distance of each retained feature point with the mean value, if the consistency judgment formula is satisfied, the feature point is retained, otherwise it is considered as a mismatch point and is removed; S7. Use the RANSAC algorithm to detect and remove mismatched points among the feature points retained in step S6.

2. The method for detecting mismatched points of RTK images according to claim 1, characterized in that: In step S1, the specific method for performing preliminary matching on the feature points of two adjacent images is: firstly extracting the feature points and calculating the descriptors of the two adjacent images, and then using a brute force matcher to perform preliminary matching on the feature points.

3. The method for detecting mismatched points of RTK images according to claim 1, characterized in that: The threshold T is based on Lowe's algorithm and is determined by the ratio of the distance of the closest matching feature point to the distance of the second closest matching feature point.

4. The method for detecting mismatched points of RTK images according to claim 1, characterized in that: If the effect of removing the mismatched points in step S6 is not good, steps S2 to S6 are repeated.

5. The method for detecting mismatched points of RTK images according to claim 1, characterized in that: The calculation formula for the position increment of the feature points retained in step S1 on two adjacent images is as follows: Among them, dx and dy represent the pixel coordinate increments of the feature points on the x-axis and y-axis of two adjacent images, respectively. x,k and p y,k They represent the x-axis and y-axis coordinates of the feature point on the k-th frame image; p x,k+1 and p y,k+1 They respectively represent the x-axis and y-axis coordinates of the feature point in the k+1th frame image.

6. The method for detecting mismatched points of RTK images according to claim 1, characterized in that: The calculation formula for the motion direction of the retained feature points in two adjacent images is as follows: The calculation formula for the motion distance of the retained feature points in two adjacent images is as follows: Among them, θ represents the direction of movement; d represents the distance of movement.

7. The method for detecting mismatched points of RTK images according to claim 1, characterized in that: The calculation formulas for the mean and standard deviation of the motion direction and motion distance of the retained feature points on the image are as follows: in, and Respectively represent the mean of the motion direction and motion distance of the retained feature points on the image, σ θ and σ d They respectively represent the standard deviation of the motion direction and motion distance of the retained feature points on the image.

8. The method for detecting mismatched points of RTK images according to claim 1, characterized in that: The consistency judgment formula is:

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a method program for detecting mismatched points of RTK images. When the method program for detecting mismatched points of RTK images is executed by a processor, the method for detecting mismatched points of RTK images as described in any one of claims 1 to 8 is implemented.

10. A computer device, characterized in that: The invention comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus, the memory is used to store a program of a method for detecting mismatched points of RTK images, and the program of a method for detecting mismatched points of RTK images enables the processor to perform operations corresponding to the method for detecting mismatched points of RTK images as described in any one of claims 1 to 8.

Images