A global fundamental matrix estimation method based on interior point updating in planar motion

Abstract

The present invention is a global fundamental matrix estimation method based on interior point update in plane motion. In order to solve the correspondence problem of feature points in scene images taken by a camera from different perspectives in plane motion and improve the reconstruction accuracy of feature points on the surface of an object, the present invention incorporates coplanar constraints into the solution of the fundamental matrix to reduce the number of parameters to be solved in the fundamental matrix; introduces an interior point update matrix according to a set epipolar geometric distance threshold, eliminates potential mismatching points and obtains a reliable initial value of the fundamental matrix estimation; on this basis, the four-point method is used to iteratively solve the fundamental matrix, and the obtained fundamental matrix satisfies the rank constraint at the same time, and the fundamental matrix estimation method has good robustness to noise and mismatching points; in the attitude measurement based on feature points, the maximum angle error of the satellite attitude measurement is less than 0.273°, which improves the reconstruction accuracy of the feature points and has good practical application prospects in multi-view three-dimensional reconstruction.

Description

Technical Field

[0001] The invention relates to the technical field of plane motion, and is a global fundamental matrix estimation method based on interior point updating in plane motion. Background Art

[0002] In computer vision and robotics, estimating the fundamental matrix based on the matching relationship of feature points in an image is an important problem in epipolar geometry. With the rapid development of computer vision technology, the demand and application of three-dimensional models in fields such as computer graphics and virtual reality are growing rapidly. Among them, the three-dimensional reconstruction technology using photographed objects or scene images has received widespread attention. Figure 3 In 3D reconstruction, the motion structure recovery algorithm uses the matching relationship of feature points based on a series of input images to restore the position, direction and scene structure of the camera at different times. The matching relationship of feature points in the image and the relative pose information between cameras directly affect the accuracy of 3D reconstruction of the object. In robotics, accurate estimation of the fundamental matrix is ​​the premise and basis of visual navigation. The accurately estimated fundamental matrix can establish the matching relationship of feature points in the image, and the relative pose information between cameras is contained in the fundamental matrix. Therefore, studying a high-precision and robust fundamental matrix estimation method is of great significance for 3D reconstruction of objects and navigation and planning of robots.

[0003] The estimation accuracy of the fundamental matrix is ​​mainly related to the extraction accuracy of the feature points and the matching relationship of the feature points. Among them, the positioning error of the feature points is usually caused by the noise in the image; when the feature points are not matched accurately, only a few mismatched points will seriously affect the calculation accuracy of the fundamental matrix. In order to solve the above problems, a large number of fundamental matrix estimation methods have emerged in recent years, mainly including linear methods, iterative methods and robust methods.

[0004] The linear method mainly includes the seven-point method, the eight-point method, the improved eight-point method, etc. This method estimates the fundamental matrix by solving a set of linear equations using least squares and singular value decomposition. The linear method is usually more efficient. When the feature point extraction and matching are accurate enough, an accurate fundamental matrix estimation result can be obtained. However, when there are abnormalities in the matching relationship of the feature points, the accuracy of the linear method will be seriously affected. The iterative method is mainly divided into two methods: based on minimizing the epipolar geometric distance and based on the gradient. Compared with the linear method, the iterative method improves the solution accuracy of the fundamental matrix and effectively reduces the influence of noise. However, this method has high computational complexity and is also not suitable for situations with many mismatched points. Robust methods have become the main research direction because of their advantages such as eliminating mismatched points and strong anti-noise ability. M-estimators, the least median square method (LMedS) and the random sample consensus method (RANSAC) are currently the most effective robust methods. In order to obtain reliable results, the initial matching points are usually screened before calculation, and the matching points with smaller errors are selected as the inlier set, and then the points in the inlier set are used to estimate the fundamental matrix. Among them, the M-estimators method reduces the impact of mismatched points by assigning different weight coefficients to each point. However, this method has high requirements on the initial value of the basic matrix. The LMedS method uses the median of the distance from the point to the corresponding polar line to optimize the basic matrix, but this method is very time-consuming. The RANSAC algorithm calculates the basic matrix by iteratively selecting internal points. It has the advantages of high accuracy and strong anti-noise ability. It is the most widely used robust method. However, when the proportion of mismatched points is high, the efficiency of this method becomes low.

[0005] In order to further improve the robustness and computational efficiency of the fundamental matrix estimation, many scholars have made some improvements to the RANSAC algorithm. Hartley et al. used epipolar geometric relationships to identify inliers and mismatched points, but there are cases where mismatched points cannot be completely identified and the elimination of mismatched points depends on the estimation accuracy of the fundamental matrix. The randomized RANSAC algorithm first checks some points in the inlier set. If the points pass the test, the remaining points are globally tested, which effectively reduces the amount of calculation. Different constraints are used to iteratively solve the fundamental matrix. Xiao et al. proposed a fundamental matrix estimation method based on inlier set optimization (ISSO). This method effectively eliminates some mismatched points by introducing guided sampling and local optimization algorithms. These methods have alleviated the shortcomings of the RANSAC algorithm to a certain extent, but there is still the problem that the accuracy of the fundamental matrix solution deteriorates as the proportion of mismatched points increases. Yan et al. proposed a fundamental matrix estimation method based on epipolar geometric errors (EGEC). This method eliminates mismatched points in the process of solving the fundamental matrix, improves the computational efficiency of the fundamental matrix, but the computational accuracy needs to be improved. He et al. introduced the research results of robots in visual navigation and positioning in recent years and gave the expression of the basic matrix in planar motion.

[0006] Based on the above analysis, it can be found that the existing fundamental matrix estimation methods mainly solve the problem of estimating the fundamental matrix under general motion. Among them, the linear method and the iterative method have poor robustness and are not suitable for situations with many mismatched points. Although the RANSAC algorithm can eliminate mismatched points, when the proportion of mismatched points is high, the calculation results are unstable and the number of iterations of the algorithm increases exponentially with the increase in the number of required inliers. When the camera moves on a plane, the estimation accuracy of the fundamental matrix estimation method under general motion decreases and the number of required inliers is large. Summary of the invention

[0007] In order to improve the solution accuracy of the fundamental matrix, the present invention provides a global fundamental matrix estimation method based on interior point update in planar motion. The present invention provides the following technical solutions:

[0008] A method for estimating a global fundamental matrix based on interior point updating in planar motion comprises the following steps:

[0009] Step 1: Incorporate the coplanar constraint into the solution of the basic matrix to reduce the number of parameters to be solved in the basic matrix;

[0010] Step 2: The set epipolar geometric distance threshold is introduced into the inlier update matrix to eliminate potential mismatching points and obtain a reliable initial value for the basic matrix estimation;

[0011] Step 3: Use the four-point method to iteratively solve the basic matrix, and the obtained basic matrix satisfies the rank constraint;

[0012] Step 4: Extend the epipolar geometric relationship in binocular vision to the three-view image, and globally optimize the basic matrix within the group by minimizing the coordinate deviation between the epipolar intersection and the feature points with the same name in each group of images.

[0013] Preferably, the step 1 is specifically:

[0014] Step 1.1: When the mobile robot moves on the horizontal plane, set the camera coordinate system to coincide with the robot coordinate system. The camera moves from position 1 to position 2, and the rotation angle around the y axis is The translation direction is θ;

[0015] Determine the rotation and translation relationship between the camera coordinate system at position 2 and the camera coordinate system at position 1:

[0016]

[0017] E=t×R (2)

[0018] Where, E is the eigenmatrix;

[0019] Step 1.2: Substitute equation (1) into equation (2):

[0020]

[0021] Assume that the intrinsic parameter of the camera is K, and the intrinsic parameter remains unchanged during the movement of the camera:

[0022]

[0023] Among them, f x With f y are the equivalent focal lengths in the X-axis and Y-axis directions in the image pixel coordinate system, respectively; x0 and y0 are the coordinates of the intersection of the optical axis and the image plane; the parameter s is the non-perpendicular factor between the X and Y coordinate axes;

[0024] Step 1.3: According to the relationship between the fundamental matrix and the eigenmatrix:

[0025]

[0026] d=1 / f x f y

[0027] f1=-d cosθ

[0028] f2=dy0 cosθ

[0029]

[0030]

[0031]

[0032] f6=df x sinθ+dx0 cosθ

[0033]

[0034] The fundamental matrix satisfies:

[0035] det(F)=0

[0036] det(F+F T )=0 (6)

[0037] Mf=0 (7)

[0038] in: f=[f1,f2,f3,f4,f5,f6,f7] T .

[0039] Preferably, the step 2 is specifically:

[0040] Step 2.1: Create a matrix L to represent the noise-free matrix M, and then use equation (8) to eliminate mismatched points while solving L;

[0041]

[0042] rank(L)=rank(M) -1 (8)

[0043] Where W = diag(w1,w2,…,w n ) is an n×n interior point update matrix;

[0044] Step 2.2: Since the matrix L satisfies Lf = 0, equation (8) can be transformed into the minimization problem of equation (9):

[0045]

[0046] All points are correct matching points, so the W matrix is ​​initially an n×n identity matrix, ξ=Inf;

[0047] Step 2.3: Before the iteration begins, based on the spatial consistency of the camera in planar motion, the similarity of the coordinates of the feature points with the same name in the image is used to eliminate obvious mismatching points and update W for the first time; solve equation (9) and replace M TWM performs singular value decomposition, f is the eigenvector corresponding to the minimum singular value; when the set iteration stop condition ε max >ξ does not hold, then use formula (10) to update the matrix W and ξ until the calculation result meets the set stop condition and substitute the obtained basic matrix and interior point set as initial values ​​into the four-point method:

[0048]

[0049] Where: i is the defined epipolar geometric distance; ε max =Q 25％ (ε 1, …ε n )

[0050] is the lowest quartile of the epipolar geometric distance;

[0051]

[0052] Preferably, if the i-th pair of matching points is a correct matching point, then w is set i =1, otherwise set w i =0.

[0053] Preferably, the step 3 is specifically:

[0054] The average value of the polar line geometric distance obtained by the initial value of the basic matrix is ​​used as the threshold. The four-point method is used to perform multiple repeated operations in the obtained interior point set, and the model containing the largest number of interior points is selected as the estimated basic matrix. The obtained basic matrix is ​​stable and satisfies the rank constraint at the same time.

[0055] Preferably, four pairs of matching points are randomly selected from the interior point set and introduced into formula (7):

[0056] Af=0 (12)

[0057] in:

[0058]

[0059] It is known that f1′, f2′, f3′ are vectors in the right null space of matrix A, so f is expressed as follows:

[0060] f=af1′+bf2′+cf3′ (13)

[0061] Convert the vector to the basic matrix form and set c = 1:

[0062] F=aF1′+bF2′+F3′ (14)

[0063] Among them, a and b are the parameters to be solved;

[0064] Combining equation (6) and equation (14):

[0065] C[a 3 ,a 2 b,ab 2 ,b 3 ,a 2 ,ab,b 2 ,a,b,1] T =0 (15)

[0066] in: is f1′ , f2′ , The parameter matrix composed of the elements in f3′;

[0067] The minimum problem automatic solver is used to solve the parameters, and finally the parameters are substituted into equation (14) to complete the estimation of the basic matrix.

[0068] Preferably, the step 4 is specifically:

[0069] The three adjacent frames of the collected images are set as a group, and the basic matrix F between the first frame image and the middle frame image is calculated respectively. 12 The basic matrix F between the intermediate frame image and the third frame image 32 ; The feature points m in the first frame image and the third frame image i ,m′ i Use the estimated basic matrix to map to the two epipolar lines l′ in the intermediate frame image 1i =F 12 m i , l′ 3i =F 32 m′ i , let the intersection point of the polar lines be p i ; Convert the optimization problem of the basic matrix into the minimization problem of formula (16):

[0070]

[0071] The present invention has the following beneficial effects:

[0072] According to the motion characteristics of the mobile robot itself, the present invention reduces the complexity of the basic matrix, combines the advantages of the linear method and the robust method, and involves all matching points in the initial model calculation of the basic matrix. The basic matrix estimation four-point method used uses the properties of the basic matrix itself to solve in each iteration, and at the same time solves the problem that the basic matrix solution may fall into the local optimum due to the random selection of internal points during the iteration process, avoiding the error uncertainty introduced by the use of artificially set thresholds. And according to the polar line geometric relationship in the three views, the basic matrix in each group of images is globally optimized within the group, which improves the solution accuracy of the basic matrix. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 is the epipolar geometry in binocular vision;

[0074] Figure 2 is the camera motion on the plane;

[0075] Figure 3 It is the polar line geometric relationship in three views;

[0076] Figure 4 is the epipolar geometric distance under different noises and different proportions of mismatched points;

[0077] Figure 5 is the distance between the feature point and the intersection point of the epipolar line under different noises and different proportions of mismatched points;

[0078] Figure 6 is the epipolar geometric relationship in the image;

[0079] Figure 7 is the intersection point of the feature point and the epipolar line in the image;

[0080] Figure 8 3D reconstruction of satellite feature points;

[0081] Fig. 9 is the dynamic angle error. DETAILED DESCRIPTION

[0082] The present invention is described in detail below in conjunction with specific embodiments. Specific embodiment one:

[0084] according to Figures 1 to 9As shown, in order to solve the problem of the correspondence between feature points in scene images taken from different perspectives by the camera in plane motion and to improve the reconstruction accuracy of feature points on the surface of the object, the present invention incorporates the coplanar constraint into the solution of the basic matrix to reduce the number of parameters to be solved in the basic matrix; and introduces the internal point update matrix according to the set epipolar geometric distance threshold to eliminate potential mismatching points and obtain a reliable initial value of the basic matrix estimation; on this basis, the four-point method is used to iteratively solve the basic matrix, and the obtained basic matrix can simultaneously meet the rank constraint; in addition, the epipolar geometric relationship in binocular vision is extended to three views, and the basic matrix is ​​globally optimized within the group by minimizing the coordinate deviation between the epipolar intersection and the feature point of the same name in each group of images. Experimental results show that the basic matrix estimation method of the present invention has good robustness to noise and mismatching points; in the attitude measurement based on feature points, the maximum angle error of the satellite attitude measurement is less than 0.273°, which improves the reconstruction accuracy of the feature points and is effective in multi-view. Figure 3 It has good practical application prospects in dimensional reconstruction.

[0085] The specific optimization technical solution adopted by the present invention to solve the above technical problems is: the present invention relates to a global fundamental matrix estimation method based on interior point updating in planar motion.

[0086] The present invention relates to a global fundamental matrix estimation method based on interior point updating in planar motion, specifically:

[0087] A method for estimating a global fundamental matrix based on interior point updating in planar motion comprises the following steps:

[0088] Step 1: Incorporate the coplanarity constraint into the solution of the basic matrix to reduce the number of parameters to be solved in the basic matrix; analyze the basic matrix between the two views in the plane motion, and the simplified basic matrix includes the constraint of coplanarity of the plane motion. Figure 2 As shown, when the mobile robot moves on a horizontal plane, the camera coordinate system is assumed to coincide with the robot coordinate system.

[0089] Preferably, the step 1 is specifically:

[0090] Step 1.1: When the mobile robot moves on the horizontal plane, set the camera coordinate system to coincide with the robot coordinate system. The camera moves from position 1 to position 2, and the rotation angle around the y axis is The translation direction is θ;

[0091] Determine the rotation and translation relationship between the camera coordinate system at position 2 and the camera coordinate system at position 1:

[0092]

[0093] E=t×R (2)

[0094] Where, E is the eigenmatrix;

[0095] Step 1.2: Substitute equation (1) into equation (2):

[0096]

[0097] Assume that the intrinsic parameter of the camera is K, and the intrinsic parameter remains unchanged during the movement of the camera:

[0098]

[0099] Among them, f x With f y are the equivalent focal lengths in the X-axis and Y-axis directions in the image pixel coordinate system, respectively; x0 and y0 are the coordinates of the intersection of the optical axis and the image plane; the parameter s is the non-perpendicular factor between the X and Y coordinate axes;

[0100] Step 1.3: According to the relationship between the fundamental matrix and the eigenmatrix:

[0101]

[0102] d=1f x f y

[0103] f1=-d cosθ

[0104] f2=dy0 cosθ

[0105]

[0106]

[0107]

[0108] f6=df x sinθ+dx0 cosθ

[0109]

[0110] The fundamental matrix satisfies:

[0111] det(F)=0

[0112] det(F+F T )=0 (6)

[0113] Mf=0 (7)

[0114] in: f=[f1,f2,f3,f4,f5,f6,f7] T .

[0115] Step 2: The set epipolar geometric distance threshold is introduced into the inlier update matrix to eliminate potential mismatching points and obtain a reliable initial value for the basic matrix estimation;

[0116] Preferably, the step 2 is specifically:

[0117] Step 2.1: Create a matrix L to represent the noise-free matrix M, and then use equation (8) to eliminate mismatched points while solving L;

[0118]

[0119] rank(L)=rank(M)-1 (8)

[0120] Where W = diag(w1,w2,…,w n ) is an n×n interior point update matrix;

[0121] Step 2.2: Since the matrix L satisfies Lf = 0, equation (8) can be transformed into the minimization problem of equation (9):

[0122]

[0123] All points are correct matching points, so the W matrix is ​​initially an n×n identity matrix, ξ=Inf;

[0124] Step 2.3: Before the iteration begins, based on the spatial consistency of the camera in planar motion, the similarity of the coordinates of the feature points with the same name in the image is used to eliminate obvious mismatching points and update W for the first time; solve equation (9) and replace M T WM performs singular value decomposition, f is the eigenvector corresponding to the minimum singular value; when the set iteration stop condition ε max >ξ does not hold, then use formula (10) to update the matrix W and ξ until the calculation result meets the set stop condition and substitute the obtained basic matrix and interior point set as initial values ​​into the four-point method:

[0125]

[0126] Where: i is the defined epipolar geometric distance; ε max =Q 25％ (ε1,…ε n )

[0127] is the lowest quartile of the epipolar geometric distance;

[0128]

[0129] If the i-th pair of matching points is the correct matching point, set wi =1, otherwise set w i =0.

[0130] Step 3: Use the four-point method to iteratively solve the basic matrix, and the obtained basic matrix satisfies the rank constraint;

[0131] The step 3 is specifically as follows:

[0132] In the RANSAC algorithm, an artificially set threshold is usually used to iteratively update the model, and the solved basic matrix does not satisfy the rank constraint of 2. Therefore, the last singular value in the matrix needs to be forced to 0 to make the basic matrix satisfy the rank constraint.

[0133] First, the average value of the epipolar geometric distance obtained using the initial value of the basic matrix is ​​used as the threshold, and the four-point method is used to perform multiple repeated operations in the obtained inlier set. Finally, the model with the largest number of inliers is selected as the estimated basic matrix. The obtained basic matrix is ​​stable and satisfies the rank constraint at the same time.

[0134] The average value of the polar line geometric distance obtained by the initial value of the basic matrix is ​​used as the threshold. The four-point method is used to perform multiple repeated operations in the obtained interior point set, and the model containing the largest number of interior points is selected as the estimated basic matrix. The obtained basic matrix is ​​stable and satisfies the rank constraint at the same time.

[0135] Preferably, four pairs of matching points are randomly selected from the interior point set and introduced into formula (7):

[0136] Af=0 (12)

[0137] in:

[0138]

[0139] It is known that f1′, f2′, f3′ are vectors in the right null space of matrix A, so f is expressed as follows:

[0140] f=af1′+bf2′+cf3′ (13)

[0141] Convert the vector to the basic matrix form and set c = 1:

[0142] F=aF1′+bF2′+F3′ (14)

[0143] Among them, a and b are the parameters to be solved;

[0144] Combining equation (6) and equation (14):

[0145] C[a 3 ,a 2 b,ab2 ,b 3 ,a 2 ,ab,b 2 ,a,b,1] T =0 (15)

[0146] in: is a parameter matrix consisting of the elements in f1′, f2′, f3′;

[0147] The minimum problem automatic solver is used to solve the parameters, and finally the parameters are substituted into equation (14) to complete the estimation of the basic matrix.

[0148] Step 4: Extend the epipolar geometric relationship in binocular vision to the three-view image, and globally optimize the basic matrix within the group by minimizing the coordinate deviation between the epipolar intersection and the feature points with the same name in each group of images.

[0149] The step 4 is specifically as follows:

[0150] The three adjacent frames of the collected images are set as a group, and the basic matrix F between the first frame image and the middle frame image is calculated respectively. 12 The basic matrix F between the intermediate frame image and the third frame image 32 ; The feature points m in the first frame image and the third frame image i ,m′ i Use the estimated basic matrix to map to the two epipolar lines l′ in the intermediate frame image 1i =F 12 m i , l′ 3i =F 32 m′ i , let the intersection point of the polar lines be p i ; Convert the optimization problem of the basic matrix into the minimization problem of formula (16):

[0151] Specific embodiment 2:

[0153] like Figure 1 As shown, the space point P wi The projection point on the image planes I and I′ is m i and m′ i O1 and O2 are the optical centers of the camera, and the line between them is the baseline. The baseline intersects the image plane at two points, e1 and e2, which are called poles. wi The plane formed by the optical centers O1 and O2 of the camera is defined as the polar plane Π, which intersects the image plane at the straight lines l and l' respectively. The straight line l' is point m i The corresponding epipolar line on the image plane I′, and m′ iis located on this line; similarly, the line l is the point m' i The corresponding epipolar line on the image plane I. This constraint relationship is called the epipolar geometric relationship between images.

[0154] Let the space point P wi The imaging point on the image plane is:

[0155] m i =[u i v i 1] T

[0156] m′ i =[u′ i v′ i 1] T (1)

[0157] According to the camera's epipolar geometry, we can get:

[0158] m′ i T Fm i =0 (2)

[0159] Where F is the basis matrix.

[0160] Let the basic matrix be:

[0161]

[0162] Combined formula (1), (2), (3):

[0163] Uf=0 (4)

[0164] in:

[0165]

[0166] f=[F 11 F 12 F 13 F 21 F 22 F 23 F 31 F 32 1] T

[0167] Finally, the solution of the basic matrix is ​​transformed into a least squares problem of min||Uf|| under the constraint of ||f||=1. Due to the influence of noise and incorrect matching relationships in actual measurements, the basic matrix cannot be directly obtained by solving a set of linear equations. The basic matrix estimation method based on RANSAC is widely used.

[0168] First, eight pairs of matching points are randomly selected from the matching point set and the basic matrix F is solved using formula (4); then the epipolar geometric distances of all matching points are calculated using the basic matrix F, and the points whose epipolar geometric distances are less than the set threshold are identified as inliers; after repeated experiments, the estimation result of the basic matrix is ​​obtained. The accuracy of the basic matrix estimation method based on the RANSAC algorithm is more dependent on the proportion of inliers. When the proportion of inliers in the matching point set is low, it is difficult for the RANSAC algorithm to accurately find the inlier set used to estimate the basic matrix, and the feature points selected by random sampling may be unevenly distributed, which will also affect the accuracy and stability of the solution of the basic matrix.

[0169] Experiments were conducted using real images of different scenes in the Middlebury dataset and simulated data containing different levels of Gaussian noise and mismatched points, and the method of the present invention was compared with the ISSO algorithm, the EGEC algorithm, and the RANSAC algorithm. The average value of the epipolar geometric distance and the average value of the distance between the epipolar intersection and the feature point were used as the standard for measuring the accuracy of the basic matrix calculation.

[0170] In the simulation experiment, 300 pairs of matching points were simulated and generated according to the distribution law of feature points in the Midd image of the Middlebury dataset. The positioning error of feature points and the mismatch relationship of feature points were simulated by adding Gaussian noise with a mean of 0 and a standard deviation of σ and different proportions of mismatched points. Each set of experimental data was tested independently 100 times and the average value of all experimental data was selected as the final result.

[0171] Figure 4 The relationship between the average value of the epipolar geometric distance obtained by different fundamental matrix estimation methods and the noise is shown when the proportion of mismatched points is 0. As the noise becomes stronger, the accuracy of all fundamental matrix estimation methods decreases linearly. The accuracy of the RANSAC method is basically the same as that of the EGEC method, and the method proposed in the present invention has good results under different noise levels. Figure 4 It is verified that when there is no noise in the matching points, compared with the RANSAC algorithm, the other three algorithms are more robust to mismatched points; in the method of the present invention, the average value of the epipolar geometric distance is basically independent of the proportion of mismatched points.

[0172] Ideally, Figure 3 The intersection of the epipolar lines should coincide with the feature points of the same name in the image. Figure 5 The relationship between the average value of the distance between the intersection of the epipolar lines and the feature points and the noise and mismatch point ratios is shown when the mismatch point ratio is 0 and there is no noise in the matching points. Figure 5It can be seen that under the same conditions, the average value of the distance between the intersection of the epipolar lines and the feature points obtained by the improved basic matrix estimation method is smaller. It can be seen that the anti-interference performance of the algorithm is better than other algorithms, and it can adapt to different degrees of image noise and uncertainty of mismatched points.

[0173] In order to simulate the actual situation, 10% of mismatched points were added to the simulation data, and Gaussian noise with a mean of 0 and a standard deviation of 0 to 2 pixels was added. Tables 1 and 2 show the average values ​​of the epipolar geometric distance and the average values ​​of the distances between the epipolar intersection and the feature points under different conditions. As the noise level increases, the errors of all estimation methods gradually increase. When the proportion of mismatched points is 10%, the influence of noise is greater than that of mismatched points in the method of the present invention, the ISSO method, and the EGEC method. In addition, the proposed method has higher accuracy when both mismatched points and noise exist in the data set.

[0174] Table 1 Epipolar geometric distance (pixels)

[0175]

[0176]

[0177] Table 2 Distance between feature points and epipolar intersections (pixels)

[0178]

[0179] In the real experiment, we added a set of satellite images and used the proposed method, ISSO algorithm and EGEC algorithm to estimate the fundamental matrix. Figure 6 The '*' in the image is the feature point in the first two frames. Then the estimated basic matrix is ​​used to restore the epipolar lines in the image. It can be seen from the figure that the feature points basically fall on the corresponding epipolar lines, showing the precise epipolar geometric relationship. Table 3 gives the average epipolar geometric distances in the four sets of actual images.

[0180] Table 3 Epipolar geometry distance (pixels)

[0181]

[0182]

[0183] It can be seen from Table 3 that the improved fundamental matrix estimation method has a smaller error in the epipolar geometric distance obtained in the actual image, and the obtained fundamental matrix has a higher accuracy. Figure 7 The 'o' in the image is the feature point in the collected intermediate frame, and the '+' is the solved epipolar intersection point. Figure 7It can be seen that the intersection of the epipolar lines and the feature points basically coincide. Table 4 gives the average distances between the intersection of the epipolar lines and the feature points in four sets of real images. The results show that the algorithm we proposed is more accurate than other traditional methods.

[0184] Table 4 Distance between feature points and epipolar intersections (pixels)

[0185]

[0186] In order to further verify the effect of applying the basic matrix estimation method of the present invention in improving the reconstruction accuracy of feature points, the satellite is taken as the reconstruction target, and static experiments and dynamic experiments are designed to verify the reconstruction accuracy of the feature points by calculating the rotation angle of the satellite. In the experiment, the satellite is placed on a turntable with a control accuracy of 0.010°. First, the camera is controlled to collect images of the satellite at ten different angles. During the reconstruction process, the camera coordinate system in the first frame image is defined as the world coordinate system, and the basic matrices estimated by the estimation method of the present invention and the ISSO method are used to perform three-dimensional reconstruction of the feature points on the satellite surface, and a satellite feature point library is established. Figure 8 The relative position relationship between the reconstruction results of the satellite surface feature points obtained by the estimation method of the present invention and the camera coordinate system when the camera collects the image is shown.

[0187] After that, the turntable is controlled to rotate within the range of 0° to 10°, and the image is collected every 2°. The rotation angle of the satellite is calculated using the perspective n-point algorithm using the collected images and the satellite feature point library established under different basic matrix estimation methods. The measured static angle error is shown in Table 5. It can be seen from Table 5 that the angle error increases with the increase of the rotation angle. In the estimation method of the present invention, the maximum angle error is 0.104°, and the accuracy is better than the result of the ISSO method.

[0188] Table 5 Static rotation angle error

[0189]

[0190] Then, the motion trajectory of the turntable is controlled to be a cosine curve from 0° to 10°, and the average running speed of the turntable is 1° / s. Fig. 9 The dynamic angle error measured by the satellite in one motion cycle is shown. In the method of the present invention, the dynamic angle error of the satellite is within the range of 0.273°. The static and dynamic experimental results show that the proposed basic matrix estimation method improves the reconstruction accuracy of feature points.

[0191] The present invention combines the elimination of mismatched points with the estimation of the fundamental matrix, providing a stable and reliable initial value and an inlier set of the fundamental matrix; then the rank constraint is incorporated into the estimation of the fundamental matrix and a global optimization function of the fundamental matrix under the constraints of three views is constructed. Experimental results show that when the camera moves on a plane, the fundamental matrix estimation method of the present invention is more effective than the traditional method, and improves the reconstruction accuracy of the feature points, which is of great significance for the study of high-precision non-cooperative target posture measurement problems and robot visual navigation and positioning problems.

[0192] The above is only a preferred implementation of a method for estimating a global fundamental matrix based on interior point updates in a plane motion. The protection scope of a method for estimating a global fundamental matrix based on interior point updates in a plane motion is not limited to the above embodiments. All technical solutions under this idea belong to the protection scope of the present invention. It should be pointed out that for those skilled in the art, several improvements and changes without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A method for estimating a global fundamental matrix based on interior point updating in planar motion, characterized by: The following steps are involved: Step 1: Incorporate the coplanar constraint into the solution of the basic matrix to reduce the number of parameters to be solved in the basic matrix; Step 2: The set epipolar geometric distance threshold is introduced into the inlier update matrix to eliminate potential mismatching points and obtain a reliable initial value of the basic matrix estimate; Step 3: Use the four-point method to iteratively solve the basic matrix, and the obtained basic matrix satisfies the rank constraint; Step 4: Extend the epipolar geometric relationship in binocular vision to three-view images, and globally optimize the basic matrix within the group by minimizing the coordinate deviation between the epipolar intersection and the feature points with the same name in each group of images; The step 4 is specifically as follows: The three adjacent frames of the collected images are set as a group, and the basic matrix F between the first frame image and the middle frame image is calculated respectively. 12 The basic matrix F between the intermediate frame image and the third frame image 32 ; The feature points m in the first frame image and the third frame image i ,m i 'Use the estimated basic matrix to map to two epipolar lines l1' in the intermediate frame image i =F 12 m i , l3′ i =F 32 m i ′, let the intersection point of the polar lines be p i ; Convert the optimization problem of the basic matrix into the minimization problem of formula (16):

2. The method for estimating a global fundamental matrix based on interior point updating in a planar motion according to claim 1, characterized in that: The step 1 is specifically as follows: Step 1.1: When the mobile robot moves on the horizontal plane, set the camera coordinate system to coincide with the robot coordinate system, and the camera moves from position 1 to position 2. The rotation angle around the y-axis is φ, and the translation direction is θ; Determine the rotation and translation relationship between the camera coordinate system at position 2 and the camera coordinate system at position 1: E=t×R (2) Where, E is the eigenmatrix; Step 1.2: Substitute equation (1) into equation (2): Assume that the intrinsic parameter of the camera is K, and the intrinsic parameter remains unchanged during the movement of the camera: Among them, f x With f y are the equivalent focal lengths in the X-axis and Y-axis directions in the image pixel coordinate system, respectively; x0 and y0 are the coordinates of the intersection of the optical axis and the image plane; parameter s is the non-perpendicular factor between the X and Y coordinate axes; Step 1.3: According to the relationship between the fundamental matrix and the eigenmatrix: d=1 / f x f y f1=-dcosθ f2=dy0 cosθ f6=df x sinθ+dx0 cosθ The fundamental matrix satisfies: det(F)=0 it(F+F T )=0 (6) Mf=0(7) in: f=[f1,f2,f3,f4,f5,f6,f7] T .

3. The method for estimating a global fundamental matrix based on interior point updating in a planar motion according to claim 2, characterized in that: The step 2 is specifically as follows: Step 2.1: Create a matrix L to represent the noise-free matrix M, and then use equation (8) to eliminate mismatched points while solving L; Where W = diag(w1,w2,…,w n ) is an n×n interior point update matrix; Step 2.2: Since the matrix L satisfies Lf = 0, transform equation (8) into the minimization problem of equation (9): All points are correct matching points, so the W matrix is ​​initially an n×n identity matrix, ξ=Inf; Step 2.3: Before the iteration begins, based on the spatial consistency of the camera in planar motion, the similarity of the coordinates of the feature points with the same name in the image is used to eliminate obvious mismatching points and update W for the first time; solve equation (9) and replace M T WM performs singular value decomposition, f is the eigenvector corresponding to the minimum singular value; when the set iteration stop condition ε max >ξ does not hold, then use formula (10) to update the matrix W and ξ until the calculation result meets the set stop condition and substitute the obtained basic matrix and interior point set as initial values ​​into the four-point method: Where: i is the defined epipolar geometric distance; ε max =Q 25％ (ε1,…ε n ) is the lowest quartile of the epipolar geometric distance; 4. The method for estimating a global fundamental matrix based on interior point updating in a plane motion according to claim 3, characterized in that: If the matching point i is the correct matching point, set w i =1, otherwise set w i =0.

5. The method for estimating a global fundamental matrix based on interior point updating in a planar motion according to claim 4, characterized in that: The step 3 is specifically as follows: The average value of the polar line geometric distance obtained by the initial value of the basic matrix is ​​used as the threshold. The four-point method is used to perform multiple repeated operations in the obtained interior point set, and the model containing the largest number of interior points is selected as the estimated basic matrix. The obtained basic matrix is ​​stable and satisfies the rank constraint at the same time.

6. The method for estimating a global fundamental matrix based on interior point updating in a planar motion according to claim 5, characterized in that: Randomly select four pairs of matching points from the interior point set and bring them into equation (7): Af=0 (12) in: It is known that f1′, f2′, f3′ are vectors in the right null space of matrix A, so f is expressed as follows: f=af1′+bf2′+cf3′ (13) Convert the vector to the basic matrix form and set c = 1: F=aF1′+bF2′+F3′ (14) Among them, a and b are the parameters to be solved; Combining equation (6) and equation (14): C[a 3 ,a 2 b,ab 2 ,b 3 ,a 2 ,ab,b 2 ,a,b,1] T =0 (15) in: is a parameter matrix consisting of the elements in f1′, f2′, f3′; The minimum problem automatic solver is used to solve the parameters, and finally the parameters are substituted into equation (14) to complete the estimation of the basic matrix.

Images