Consistency search method, integrated navigation system and GNSS receiver

Abstract

The present invention discloses a consistency search method, an integrated navigation system and a GNSS receiver, wherein the consistency search method comprises: obtaining a plurality of two-dimensional images of a target area, each of which comprises position information; for a feature point of a target image in the two-dimensional image, searching for the best matching point on the epipolar line of the feature point in each two-dimensional image; obtaining a depth estimation value of a landmark point corresponding to the feature point by using an estimation algorithm; determining a depth range of the landmark point by using the depth estimation value; obtaining a search range by using the epipolar line of the feature point in each two-dimensional image by using the depth range; and performing an optimal value search by using a search method within the search range to obtain the optimal depth information of the landmark point. The present invention can improve the accuracy and reliability of landmark point position estimation in photogrammetry.

Description

Technical Field

[0001] The present invention relates to the field of integrated navigation systems, and in particular to a consistency search method, an integrated navigation system and a GNSS receiver. Background Art

[0002] After decades of development, GNSS-based engineering surveying technology has evolved from centering measurement to the tilt measurement that is widely used today. In recent years, the release of the integrated navigation system has provided a new direction for engineering surveying technology. Based on the original hardware, the integrated navigation system adds a camera, which transforms the traditional method of measuring the point to be measured at close range through the tip of the rod into a method of long-distance point selection measurement through photos in the notebook, which further improves the efficiency of engineering measurement. For the sake of convenience, in the following text, "photogrammetry" will be used to refer to the above-mentioned camera-based GNSS engineering surveying technology.

[0003] The above-mentioned photogrammetry technology is mainly divided into two parts: camera pose estimation and landmark point position estimation. Among them, camera pose estimation uses a fusion navigation algorithm of GNSS, INS and vision to provide the camera with high-precision pose information as input for subsequent landmark point position estimation. Landmark point position estimation means that after the user selects the point of interest (feature point) through the handbook, a series of photos and their high-precision poses, point matching algorithms between photos, and estimation algorithms are used to estimate the landmark points. It should be noted that the point of interest (feature point) here refers to the 2D point in the photo, and the landmark point refers to the corresponding point in the 3D space.

[0004] The integrated navigation system in the prior art has the defect of low accuracy in estimating the position of landmark points in photogrammetry. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the defect of low landmark point position estimation accuracy in photogrammetry of the combined navigation system in the prior art, and to provide a consistency search method, an integrated navigation system and a GNSS receiver that can improve the landmark point position estimation accuracy and measurement reliability in photogrammetry.

[0006] The present invention solves the above technical problems through the following technical solutions:

[0007] A consistency search method for photogrammetric non-Gaussian depth estimation, used in an integrated navigation system, is characterized in that the consistency search method comprises:

[0008] Acquire a plurality of two-dimensional images of a target area, each of the two-dimensional images including position information;

[0009] For a feature point of a target image in the two-dimensional image, searching for the best matching point on the epipolar line of the feature point in each two-dimensional image;

[0010] Obtaining a depth estimation value of a landmark point corresponding to the feature point by using an estimation algorithm;

[0011] Determine the depth range of the landmark using the depth estimate;

[0012] Acquire a search range using the epipolar lines of the feature points in each two-dimensional image within the depth range;

[0013] An optimal value is searched within the search range using a search method to obtain optimal depth information of the landmark point.

[0014] Preferably, the obtaining the depth estimation value of the landmark point corresponding to the feature point by using an estimation algorithm includes:

[0015] The estimation algorithm for obtaining the depth estimation value of the landmark point using the reprojection error, the residual expression of the reprojection error is: ,in, and are the rotation and translation of the carrier system relative to the camera system, Represents the three-dimensional position of the landmark point in the world coordinate system. is the camera projection function, is the two-dimensional position of the feature point (corresponding to the above 3D landmark point) in the pixel coordinate system, is the rotation matrix of the carrier coordinate system relative to the world coordinate system at time k, is the position of the carrier in the world coordinate system at time k.

[0016] Preferably, the depth estimation value is ,in, is the depth estimate, is the true depth value, is the estimation error, which follows a uniform distribution. The preset error range.

[0017] Preferably, the consistency search method comprises:

[0018] A minimization problem of depth estimation is constructed using the reprojection error and the pixel descriptor error, and the optimal depth information of the landmark point is obtained using the minimization problem.

[0019] Preferably, the minimization problem is ,in, is the descriptor error of the best matching pixel on the epipolar line of the k-frame image, is the reprojection error of the k-frame image, and the method of obtaining the optimal depth information of the landmark point by using the minimization problem includes:

[0020] A search method is used to find the optimal depth information for the minimization problem.

[0021] Preferably, the searching for optimal depth information using a search method for the minimization problem includes:

[0022] The pixel search of the minimization problem is transformed into a depth search, and the minimization problem is ,in, is the depth search value.

[0023] Preferably, the consistency search method comprises:

[0024] The search range of the depth search is ,in, is the search step length, and the value of the search step length is obtained through the accuracy index of the integrated navigation system.

[0025] Preferably, the consistency search method comprises:

[0026] Determine whether a depth value that satisfies the minimum cumulative descriptor error appears, and if so, select the depth value that satisfies the minimum cumulative descriptor error as the optimal depth information;

[0027] The coordinates of the landmark points are obtained using the optimal depth information and pose information.

[0028] The present invention also provides an integrated navigation system, which is characterized in that the integrated navigation system is used to implement the above-mentioned consistency search method for photogrammetry-oriented non-Gaussian depth estimation.

[0029] The present invention also provides a GNSS receiver, which is characterized in that the GNSS receiver is used in the integrated navigation system as described above.

[0030] On the basis of being in accordance with the common sense in the art, the above-mentioned preferred conditions can be arbitrarily combined to obtain the preferred embodiments of the present invention.

[0031] The positive and progressive effects of the present invention are:

[0032] It can improve the accuracy of landmark position estimation and measurement reliability in photogrammetry. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 The figure is a schematic diagram of the principle of feature point matching in the prior art.

[0034] Figure 2 This is a schematic diagram of the principle of feature point matching in Example 1 of the present invention.

[0035] Figure 3 This is a flow chart of the consistency search method according to Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0037] Example 1

[0038] This embodiment provides an integrated navigation system, which includes a GNSS receiver INS module and a vision module. In other embodiments, the integrated navigation system is a receiver including a GNSS module, an INS module and a vision module.

[0039] The integrated navigation system is used for:

[0040] Acquire a plurality of two-dimensional images of a target area, each of the two-dimensional images including position information;

[0041] For a feature point of a target image in the two-dimensional image, searching for the best matching point on the epipolar line of the feature point in each two-dimensional image;

[0042] Obtaining a depth estimation value of a landmark point corresponding to the feature point by using an estimation algorithm;

[0043] Determine the depth range of the landmark using the depth estimate;

[0044] Acquire a search range using the epipolar lines of the feature points in each two-dimensional image within the depth range;

[0045] An optimal value is searched within the search range using a search method to obtain optimal depth information of the landmark point.

[0046] The location estimation of landmark points means that after the user selects a point of interest (feature point) through a handbook, the landmark points are estimated by using a series of photos and their high-precision poses, point matching algorithms between photos, and estimation algorithms.

[0047] The matching algorithm refers to finding the feature points corresponding to the landmark points in the pictures other than the user-selected point pictures. In the present invention, this process is implemented by epipolar constraints, such as Figure 1As shown. When the depth of the landmark point is unknown and the relative position of the camera is known, the point of the left image (known point image 100) will map a line in the right image (to-be-matched image 101), which is called the epipolar line 102. The point to be matched will fall on the epipolar line, but it is necessary to search for the specific point on the epipolar line. Since similar features are inevitable in the image, and around the correct feature, due to factors such as pixel blur, the search algorithm often considers that multiple adjacent pixels are legal matches. Therefore, in this case, directly selecting the "best match" (i.e., the matching point with the highest score) may not necessarily result in a correct result. This is the problem that the present invention seeks to solve.

[0048] The estimation algorithm in the landmark point position estimation is implemented by reprojection error. Specifically, the integrated navigation system is used for:

[0049] The estimation algorithm for obtaining the depth estimation value of the landmark point using the reprojection error, the residual expression of the reprojection error is: ,in, and are the rotation and translation of the carrier system relative to the camera system, Represents the three-dimensional position of the landmark point in the world coordinate system. is the camera projection function, is the two-dimensional position of the feature point (corresponding to the above 3D landmark point) in the pixel coordinate system, is the rotation matrix of the carrier coordinate system relative to the world coordinate system at time k, is the position of the carrier in the world coordinate system at time k.

[0050] By constructing multiple residuals mentioned above through the points selected by the user and the matching points of multiple images, the three-dimensional position of the landmark point can be estimated.

[0051] The above reprojection error expression is the same as Figure 1 The polar constraints shown have the same physical meaning. In other words, they both express the same constraint relationship.

[0052] During the optimization iteration process, the depth value of the landmark point will gradually converge and be reflected on the epipolar line, which is expressed as its projection on the epipolar line gradually moving. Finally, the iterator converges to the "optimal value", which means the estimation is complete, but it should be noted that the optimal value is based on the assumption that the above matching result is the "optimal match". Therefore, when the match is incorrect, the estimated value will also be incorrect. Generally speaking, for errors at the decimeter level or even the meter level, simple fault detection and troubleshooting algorithms can be used to identify and correct them, but for depth errors within decimeters (5~10cm), it is not easy to identify. Therefore, a more reliable algorithm is needed to ensure the accuracy of matching and estimation.

[0053] As mentioned above, since there are several similar pixels around the best matching pixel in the epipolar line, it is difficult for the matching algorithm to find the so-called "best match", so that the estimator cannot reach the optimal estimate. In essence, this problem is caused by the fact that the matching error caused by the epipolar line search does not obey the Gaussian distribution. For most of the optimal estimation algorithms commonly used in engineering, there is a basic assumption that the measurement noise obeys the Gaussian distribution. Under this premise, as long as the measurement is sufficient (that is, the landmark point is visible in multiple pictures and is successfully matched), and the measurement noise is not particularly large (within a few pixels), its estimated value can always converge to a reasonable error range. However, for the epipolar line search, its noise obviously does not obey the Gaussian distribution, but approximately obeys the uniform distribution, that is, the probability of its error being any value is equal, because on the epipolar line, pixels with similar characteristics to the target pixel may appear at any position of the epipolar line.

[0054] To solve this problem, the conventional solution is to derive the optimal estimation algorithm based on the Bayesian formula again, changing the Gaussian distribution assumption to a uniform distribution, so that the estimator can reach the optimal value. However, the assertion discussed earlier that "the polar line search error approximately obeys the uniform distribution" is not rigorous, which makes it difficult to reach the optimal value even if the problem is modeled using a uniform distribution. This approach was first proposed in the paper "SVO: Fast semi-direct monocular visual odometry", but practice has shown that although its effect can be improved to a certain extent, it still cannot achieve the stability and reliability required by engineering measurement.

[0055] The consistency search algorithm of this embodiment can solve this problem from a new dimension. This embodiment combines the extreme line search with the optimal estimation to construct a unified algorithm, so that the entire system forms a large loop to ensure the consistency of the search and estimation.

[0056] First, a rough estimate of the system is made through the traditional polar line search + optimal estimation method. Since the fault detection and elimination algorithm is used in the optimal estimation, the rough estimate can ensure that the depth error is at the decimeter level.

[0057] After this estimation, the depth estimation value is ,in, is the depth estimate, is the true depth value, is the estimation error, which follows a uniform distribution. The preset error range. The value can be set empirically, such as 20 cm.

[0058] The integrated navigation system is used for:

[0059] A minimization problem of depth estimation is constructed using the reprojection error and the pixel descriptor error, and the optimal depth information of the landmark point is obtained using the minimization problem.

[0060] Since a depth range is determined, when performing epipolar line search, the corresponding epipolar line length will also be shortened to a smaller range 103, such as Figure 2 Next, the present invention will improve the depth estimation accuracy by searching rather than estimating. In order to improve the search accuracy, the present invention expands the search cost from reprojection error to reprojection error + pixel descriptor error. The minimization problem is: ,in, is the descriptor error of the best matching pixel on the epipolar line of the k-frame image, is the reprojection error of the k-frame image, and the integrated navigation system is used for:

[0061] A search method is used to find the optimal depth information for the minimization problem.

[0062] The integrated navigation system is used to transform the pixel search of the minimization problem into a depth search. The minimization problem is: ,in, is the depth search value.

[0063] The combined navigation system is used for: the search range of the depth search is ,in, is the search step length, and the value of the search step length is obtained through the accuracy index of the integrated navigation system.

[0064] The integrated navigation system is used to: determine whether a depth value that satisfies a minimum cumulative descriptor error appears, and if so, select the depth value that satisfies the minimum cumulative descriptor error as the optimal depth information;

[0065] The coordinates of the landmark points are obtained using the optimal depth information and pose information.

[0066] See also Figure 3 , using the above combined navigation system, this embodiment also provides a consistency search method for photogrammetry non-Gaussian depth estimation, including:

[0067] Step S100, obtaining a plurality of two-dimensional images of a target area, each of which includes position information;

[0068] Step S101, for a feature point of a target image in the two-dimensional image, searching for the best matching point on the epipolar line of the feature point in each two-dimensional image;

[0069] Step S102, using an estimation algorithm to obtain a depth estimation value of a landmark point corresponding to the feature point;

[0070] Step S103, determining the depth range of the landmark point using the depth estimation value;

[0071] Step S104, using the epipolar lines of the feature points in each two-dimensional image in the depth range to obtain a search range;

[0072] Step S105: searching for an optimal value within the search range using a search method to obtain optimal depth information of the landmark point.

[0073] In step S102, an estimation algorithm is used to obtain the depth estimation value of the landmark point using the reprojection error. The residual expression of the reprojection error is: ,in, and are the rotation and translation of the carrier system relative to the camera system, Represents the three-dimensional position of the landmark point in the world coordinate system. is the projection function of the camera.

[0074] The depth estimate is ,in, is the depth estimate, is the true depth value, is the estimation error, which follows a uniform distribution. The preset error range.

[0075] In step S105, specifically, a minimization problem of the depth estimation value is constructed using the reprojection error and the pixel descriptor error, and the optimal depth information of the landmark point is obtained using the minimization problem.

[0076] The minimization problem is ,in, is the descriptor error of the best matching pixel on the epipolar line of the k-frame image, is the reprojection error of the k-frame image, and the method of obtaining the optimal depth information of the landmark point by using the minimization problem includes:

[0077] A search method is used to find the optimal depth information for the minimization problem.

[0078] Wherein, the searching for the optimal depth information for the minimization problem by using a search method includes:

[0079] The pixel search of the minimization problem is transformed into a depth search, and the minimization problem is ,in, is the depth search value.

[0080] The consistency search method comprises:

[0081] The search range of the depth search is ,in, is the search step length, and the value of the search step length is obtained through the accuracy index of the integrated navigation system.

[0082] The consistency search method comprises:

[0083] Determine whether a depth value that satisfies the minimum cumulative descriptor error appears, and if so, select the depth value that satisfies the minimum cumulative descriptor error as the optimal depth information;

[0084] The coordinates of the landmark points are obtained using the optimal depth information and pose information.

[0085] Although the specific embodiments of the present invention are described above, those skilled in the art should understand that these are only examples, and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A consistency search method for non-Gaussian depth estimation based on photogrammetry, used in an integrated navigation system, characterized in that: The consistency search method comprises: Acquire a plurality of two-dimensional images of a target area, each of the two-dimensional images including position information; For a feature point of a target image in the two-dimensional image, searching for the best matching point on the epipolar line of the feature point in each two-dimensional image; Obtaining a depth estimation value of a landmark point corresponding to the feature point by using an estimation algorithm; Determine the depth range of the landmark using the depth estimate; Acquire a search range using the epipolar lines of the feature points in each two-dimensional image within the depth range; Using a search method to search for an optimal value within the search range to obtain optimal depth information of the landmark point; The step of obtaining the depth estimation value of the landmark point corresponding to the feature point by using an estimation algorithm includes: The estimation algorithm for obtaining the depth estimation value of the landmark point using the reprojection error, the residual expression of the reprojection error is: ,in, and are the rotation and translation of the carrier system relative to the camera system, Represents the three-dimensional position of the landmark point in the world coordinate system. is the camera projection function, is the two-dimensional position of the feature point in the pixel coordinate system, is the rotation matrix of the carrier coordinate system relative to the world coordinate system at time k, is the position of the carrier in the world coordinate system at time k.

2. The consistency search method for photogrammetric non-Gaussian depth estimation according to claim 1, characterized in that: The depth estimate is ,in, is the depth estimate, is the true depth value, is the estimation error, which follows a uniform distribution. The preset error range.

3. The consistency search method for photogrammetric non-Gaussian depth estimation according to claim 2, characterized in that: The consistency search method comprises: A minimization problem of depth estimation is constructed using the reprojection error and the pixel descriptor error, and the optimal depth information of the landmark point is obtained using the minimization problem.

4. The consistency search method for photogrammetric non-Gaussian depth estimation according to claim 3, characterized in that: The minimization problem is ,in, is the descriptor error of the best matching pixel on the epipolar line of the k-frame image, is the reprojection error of the k-frame image, and the method of obtaining the optimal depth information of the landmark point by using the minimization problem includes: A search method is used to find the optimal depth information for the minimization problem.

5. The consistency search method for photogrammetric non-Gaussian depth estimation according to claim 4, characterized in that: The searching of the optimal depth information for the minimization problem using a search method includes: The pixel search of the minimization problem is transformed into a depth search, and the minimization problem is ,in, is the depth search value.

6. The consistency search method for photogrammetric non-Gaussian depth estimation according to claim 5, characterized in that: The consistency search method comprises: The search range of the depth search is ,in, is the search step length, and the value of the search step length is obtained through the accuracy index of the integrated navigation system.

7. The consistency search method for photogrammetric non-Gaussian depth estimation according to claim 5, characterized in that: The consistency search method comprises: Determine whether a depth value that satisfies the minimum cumulative descriptor error appears, and if so, select the depth value that satisfies the minimum cumulative descriptor error as the optimal depth information; The coordinates of the landmark points are obtained using the optimal depth information and pose information.

8. An integrated navigation system, characterized in that: The combined navigation system is used to implement the consistency search method for photogrammetry non-Gaussian depth estimation as described in any one of claims 1 to 7.

9. A GNSS receiver, characterized in that: The GNSS receiver is used in the integrated navigation system as claimed in claim 8.

Images