A method for calibrating a binocular camera system without a calibration object

Abstract

A kind of non-calibration object binocular camera system correction method, comprising: a group of images of relatively static scene is photographed by binocular camera, respectively to left camera image and right camera image feature extraction, feature matching, obtain the candidate feature matching pair of two cameras, the candidate feature matching pair is filtered by epipolar geometry constraint, obtain the effective feature matching pair and essential matrix satisfying requirement, according to effective feature matching pair, essential matrix and known binocular camera internal parameter and distortion coefficient, binocular relationship is calculated, and the external parameter conversion relationship of binocular camera coordinate system is obtained.The method can realize the binocular camera system correction under the condition of non-calibration board or calibration object, without professional equipment environment and complex manual operation, also without adjusting parameter, the external parameter conversion relationship of binocular camera system of binocular camera coordinate system can be quickly and accurately established.

Description

Technical Field

[0001] This invention relates to the field of machine vision, and in particular to a calibration method for a calibration-free binocular camera system. Background Technology

[0002] In existing technologies, the extrinsic parameter transformation relationship of the binocular camera coordinate system is established through calibration. Given the intrinsic parameters of the two cameras, a calibration board or calibration object composed of feature points with known positional relationships is typically used. This board or object is translated and rotated within the shared field of view of the binocular cameras, and multiple sets of images of the calibration board or calibration object with different positions and orientations are captured. Pixel recognition of the known feature points is performed on all captured images, and the extrinsic parameter transformation relationship of the two-camera coordinate system is calibrated and solved by combining the known physical positional relationships of the feature points.

[0003] Common calibration plates include checkerboard calibration plates, dot or ring calibration plates.

[0004] The shortcomings of this approach are:

[0005] (1) Calibration plates or calibration objects are relatively specialized precision measuring tools. Because the physical positional relationship of the feature point lattice needs to be known, there are usually high requirements for manufacturing accuracy and preservation. If the calibration plate or calibration object is not manufactured accurately, or if the physical positional relationship of the feature point lattice changes due to deformation in the later stage, it will affect the final calibration accuracy.

[0006] (2) The calibration process of existing binocular camera systems usually requires professionals to carry out the process in a specific laboratory or production environment in an automated or semi-automated environment. Since the calibration plate needs to be moved to take multiple images during the process, it places high demands on the operators and the environment.

[0007] (3) Existing calibration methods using feature extraction and feature matching have extremely high environmental requirements and cannot be applied to common environments without adjusting parameters. Because similar features may appear in real-world environments, mismatches may occur, resulting in the inability to correctly calculate the binocular transformation relationship; the parameters of existing feature matching validity screening algorithms (such as RANSAC) also need to be adjusted according to the actual situation. Summary of the Invention

[0008] This disclosure provides a calibration method for a binocular camera system without a calibration plate or calibration object. It can quickly and accurately establish the external parameter transformation relationship of the dual-camera coordinate system of the binocular camera system without professional equipment environment and complicated manual operation.

[0009] The calibration method for a calibration-free binocular camera system disclosed herein mainly includes the following steps:

[0010] S1, acquire a set of images of a relatively still scene captured by the binocular camera, and obtain the left camera image and the right camera image;

[0011] S2, apply feature extraction algorithms to the left and right camera images respectively to obtain the features in the two camera images;

[0012] S3, perform feature matching on the features of the left camera image and the right camera image to obtain a candidate set of feature matching pairs between the two cameras;

[0013] S4, perform epipolar geometry constraint filtering on the candidate set of feature matching pairs of the two cameras to obtain the effective feature matching pairs and essence matrix of the two cameras that meet the epipolar geometry requirements;

[0014] S5, determine whether the number of feature matching pairs contained in PairMatches meets the requirement:

[0015] If the requirements are not met, acquire a new set of images of the relatively still scene taken with the binocular camera and repeat the above steps.

[0016] If the requirements are met, the binocular relationship is calculated based on the effective feature matching of PairMatches, the essential matrix, and the known binocular relationship to be calibrated, along with the intrinsic parameters and distortion coefficients of the two cameras, to obtain the extrinsic parameter transformation relationship of the two camera coordinate system [R|T], where R is the rotation matrix and T is the translation matrix.

[0017] Compared with the prior art, the beneficial effects of this disclosure are: (1) It can quickly and accurately establish the external parameter transformation relationship of the dual-camera coordinate system of the binocular camera system without professional equipment environment and complex manual operation; (2) It adopts the feature matching pair filtering method based on epipolar geometric constraints. In the case of similar feature interference in the environment leading to feature mismatch, the algorithm uses the constraint relationship to filter the mismatch, which ensures the accuracy and robustness of the binocular relationship calculation in subsequent steps. Attached Figure Description

[0018] The above and other objects, features and advantages of this disclosure will become more apparent from the more detailed description of exemplary embodiments of this disclosure taken in conjunction with the accompanying drawings, in which the same reference numerals generally represent the same components.

[0019] Figure 1 This is an overall flowchart of an exemplary embodiment according to the present disclosure;

[0020] Figure 2 This is a flowchart illustrating the process of applying epipolar geometric constraints to the candidate set for feature matching between two cameras. Detailed Implementation

[0021] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0022] This disclosure provides a method for calibrating a binocular camera system without a calibration plate or calibration object. In one exemplary embodiment, the intrinsic parameters and distortion coefficients of the binocular relationship to be calibrated are known. The intrinsic parameter matrix of the left camera is M_L, the intrinsic parameter matrix of the right camera is M_R, the distortion coefficient of the left camera is D_L, and the distortion coefficient of the right camera is D_R, wherein:

[0023] Both M_L and M_R are 3x3 matrices;

[0024] The representations of D_L and D_R are related to the camera distortion model. This embodiment uses only one distortion model as an example (but this should not limit the scope of protection of this disclosure). Let D_L and D_R both be represented as 1*12 vectors.

[0025] A set of images of a relatively static scene are captured by a binocular camera. The calibration algorithm of the binocular camera system without calibration plate described in this disclosure is used to obtain the external parameter transformation relationship [R|T] of the dual-camera coordinate system, where R is a 3*3 rotation matrix and T is a 3*1 translation vector.

[0026] The overall process is as follows: Figure 1 As shown, the main steps include:

[0027] 1. Take a set of images of a relatively static scene using a binocular camera.

[0028] The scene can be kept relatively still while the two cameras capture images separately; or the two cameras can be triggered simultaneously to capture images of the scene. The resulting image from the left camera is called LeftImage, and the image from the right camera is called RightImage.

[0029] 2. Apply feature extraction algorithms to the left camera image (LeftImage) and the right camera image (RightImage) respectively to obtain the features in the two camera images.

[0030] In this algorithm, the features of the left camera image are represented as LeftFeatures, and the features of the right camera image are represented as RightFeatures. No restrictions are placed on the feature extraction algorithm, as long as it can effectively extract features, such as SIFT, SURF, ORB, and deep learning feature extraction algorithms.

[0031] 3. Perform feature matching on the left camera image features LeftFeatures and the right camera image features RightFeatures to obtain candidate feature matching pairs PutativePairMatches for the two cameras.

[0032] The feature matching algorithm is not limited, such as Brute force matching, Approximate Nearest Neighbors matching, etc.

[0033] 4. Perform epipolar geometry constraint filtering on the candidate feature matching pairs PutativePairMatches of the two cameras, to obtain valid feature matching pairs PairMatches of the two cameras and the essential matrix EssentialMatrix that meet the epipolar geometry requirements.

[0034] The epipolar geometry constraint filtering method performed on the candidate feature matching pairs of the two cameras is as attached Figure 2 shown as follows:

[0035] (1) Randomly sample NSample candidate feature matching pair samples PairMatchesSample from the candidate feature matching pair set PutativePairMatches.

[0036] Wherein, the candidate feature matching pair set PutativePairMatches contains Num feature matching pairs, the candidate feature matching pair sample set PairMatchesSample contains NSample feature matching pairs, which needs to satisfy 0<NSample<Num. NSample can be 5 or 8, and in this embodiment, NSample=5 is taken as an example for the subsequent method description.

[0037] (2) Calculate and output the essential matrix sample set EssentialMatrixSamples[m] by using the candidate feature matching pair sample set PairMatchesSample. The specific calculation method is as follows:

[0038] 1) Assume that the NSample (here 5) feature matching pairs in the candidate feature matching pair sample set PairMatchesSample are as follows:

[0039] {(x1,y1),(x1',y1')}, which indicates that (x1,y1) and (x1',y1') are a set of feature point matching pairs, the same applies hereinafter; {(x2,y2),(x2',y2')},

[0040] {(x3,y3),(x3',y3')},

[0041] {(x4,y4),(x4′,y4′)}

[0042] {(x5,y5),(x5′,y5′)}

[0043] Where (x1, y1), ..., (x5, y5) are the normalized image coordinates of the five feature points in the left camera, and (x1′, y1′), ..., (x5′, y5′) are the normalized image coordinates of the five feature points in the right camera.

[0044] 2) Suppose that each essential matrix solution EssensialMatrixSamples[i], i = 0, 1, ..., m-1, in the essential matrix sample set EssensialMatrixSamples[m] can be expressed as:

[0045]

[0046] Since the essential matrix EssensialMatrixSamples[i] must satisfy:

[0047]

[0048] When NSample = 5, we can obtain:

[0049] U 5*9 e=0

[0050] in,

[0051]

[0052] e = [E 11i E 21i E 31i E 12i E 22i E 32i E 13i E 23i E 33i ] T ;

[0053] 3) Solve for U 5*9 When e=0, we can obtain m essential matrix solutions: EssensialMatrixSamples[i], i=0,1,...,m-1. When NSample=5, it is equivalent to solving 9 unknowns with 5 equations, which will result in 4 fundamental solutions, i.e., m=4.

[0054] (3) Output EssensialMatrixSamples[i] for each essential matrix sample, i = 0, 1, ..., m-1. Sort the matching pairs according to the residual order, and then add the matching pairs to the inner point set combination in order of the smallest residual. Iterate and calculate the deviation corresponding to different inner point set combinations, update the essential matrix sample with the smallest deviation as the best essential matrix sample BestEssensialMatrixSample, and update the corresponding inner point set combination as the output inner point set Inliers.

[0055] The specific method is as follows:

[0056] Suppose that among the Num candidate feature matching pairs PutativePairMatches, the set of remaining feature matching pairs after removing the NSample candidate feature matching pairs PairMatchesSample is called the feature matching pair candidate complement PairMatchesResiduals. Then the number of matching pairs in the feature matching pair candidate complement is NResidual = Num - NSample.

[0057] For each essential matrix sample, output EssensialMatrixSamples[i], i = 0, 1, ..., m-1:

[0058] a) Calculate the residual ResidualError[p], p = 0, 1, ..., NResidual-1 for each matching pair {(x_residual[p], y_residual[p]), (x_residual′[p], y_residual′[p])} in the candidate complement set of feature matching pairs;

[0059] The residual calculation method is as follows:

[0060] First, calculate the fundamental matrix FundamentalMatrixSample[i] corresponding to EssensialMatrixSamples[i]:

[0061] FundamentalMatrix[i] = (M_R -1 ) T ·EssensialMatrix[i]·M_L -1

[0062] Using the intrinsic parameter matrix M_L of the left camera, the intrinsic parameter matrix M_R of the right camera, the distortion coefficients D_L of the left camera, and the distortion coefficients D_R of the right camera, the coordinates of point c are obtained respectively. p (x_residual[p], y_residual[p]) and cp ′(x_residual′[p],y_residual′[p]) The coordinates of the points after distortion correction are (x_residual_undistort[p],y_residual_undistort[p]), (x_residual_undistort′[p],y_residual_undistort′[p]);

[0063] Calculate the polar line epLine[p]:

[0064]

[0065] The residual is then expressed as:

[0066]

[0067] b) Sort the matching pairs in the candidate complement set of feature matching pairs according to the calculated residuals, and exclude matching pairs whose residuals are greater than the residual threshold residualErrorThresh (e.g., can be set to 2) to obtain the filtered and sorted candidate complement set of feature matching pairs.

[0068] c) Using the candidate samples of feature matching pairs as the initial inlier set, the matching pairs in the sorted candidate complement set of feature matching pairs are added to the initial inlier set in order of increasing residual, resulting in different combinations of inlier sets. The error corresponding to each combination of inlier sets is calculated one by one. If the currently calculated error is the lowest in history, the essential matrix sample at this time is updated to the best essential matrix sample BestEssensialMatrixSample, and the corresponding combination of inlier sets is updated as the output inlier set Inliers.

[0069] The error corresponding to each combination of interior point sets is calculated as follows:

[0070] Suppose the number of matching pairs in the interior set combination is k (k = NSample + 1, ..., Num), then

[0071]

[0072] in,

[0073]

[0074] Where D is the length of the image diagonal, A is the area of ​​the image, and e kThe k-th smallest residual is calculated from the EssentialMatrixSamples[i] output as the essential matrix sample. This is the residual corresponding to the last pair of matching pairs added to the inner point set combination in the above process when the number of matching pairs in the inner point set combination is k.

[0075] (4) The above steps (1), (2), and (3) are considered as one iteration. Determine whether the maximum number of iterations has been reached (e.g., it can be set to 10000). If not, continue to the next iteration.

[0076] (5) If the maximum number of iterations is reached, the BestEssensialMatrixSample is taken as the EssensialMatrix, and the corresponding output inliers is taken as the effective feature matching pair PairMatches for two cameras that meet the epipolar geometry requirements.

[0077] In this step, the representation of the camera's intrinsic parameters and the distortion model should not be restricted; distortion models such as pinhole camera distortion models and fisheye camera distortion models can be used.

[0078] 5. Determine whether the number of feature matching pairs PairedNum in PairMatches meets the requirement (e.g., PairedNum is greater than 10). If it does not meet the requirement, retake a set of images of the relatively static scene using the binocular camera and repeat all the above steps.

[0079] 6. If the number of feature matching pairs in the effective feature matching pairs PairMatches meets the requirements, calculate the binocular relationship based on the effective feature matching pairs PairMatches, the essential matrix, and the known intrinsic parameters and distortion coefficients of the binocular relationship to be calibrated, and obtain the extrinsic parameter transformation relationship of the binocular coordinate system [R|T], where R is a 3*3 rotation matrix, T is a 3*1 translation vector, the known intrinsic parameter matrix of the left camera is M_L, the intrinsic parameter matrix of the right camera is M_R, the distortion coefficient of the left camera is D_L, and the distortion coefficient of the right camera is D_R.

[0080] The method for calculating the binocular relationship based on effective feature matching pairs, and the known intrinsic parameters and distortion coefficients of the dual cameras for the binocular relationship to be calibrated, is as follows:

[0081] (1) The essential matrix is ​​calculated to obtain four possible solutions for the transformation relationship of the dual-camera coordinate system extrinsic parameters: [R1|T1], [R2|T2], [R3|T3], [R4|T4].

[0082] The specific method is as follows:

[0083] 1) Perform SVD decomposition on the essential matrix:

[0084] EssensialMatrix=U0DV0 T

[0085] Where U0 and V0 are both 3x3 matrices,

[0086] 2) Obtain matrices U and V from U0 and V0. Both U and V are 3x3 matrices. The specific method is as follows:

[0087] Let U0 = [u1 u2 u3], Where u1, u2, and u3 are 3x1 column vectors, and v1, v2, and v3 are 1x3 row vectors, then:

[0088]

[0089]

[0090] Here, det() represents solving for the determinant.

[0091] 3) Order

[0092] The four possible solutions are represented as follows:

[0093]

[0094]

[0095]

[0096]

[0097] Where U[:,3] represents the 3rd column of matrix U.

[0098] (2) The positive depth of field constraints are checked on the four possible solutions [R1|T1], [R2|T2], [R3|T3], and [R4|T4] respectively. The number of points that satisfy the positive depth of field constraints is as follows:

[0099] Num_Cheirality_Condition1, Num_Cheirality_Condition2,

[0100] Num_Cheirality_Condition3, Num_Cheirality_Condition4.

[0101] Taking the calculation of the number of points Num_Cheirality_Condition1 that satisfy the positive depth-of-field constraint using the solution [R1|T1] as an example, the specific method is introduced as follows:

[0102] set up Where r1, r2, and r3 are 1*3 row vectors.

[0103] Using the possible solutions [R1|T1] of the dual-camera coordinate system extrinsic parameter transformation relationship, the intrinsic parameter matrix M_L of the left camera, the intrinsic parameter matrix M_R of the right camera, the distortion coefficients D_L of the left camera, and the distortion coefficients D_R of the right camera, 3D coordinate reconstruction is performed on all matching pairs in the effective feature matching pairs PairMatches, respectively, to obtain the 3D coordinates P[id](x_3d[id],y_3d[id],z_3d[id]), id=0,……,PairedNum-1 corresponding to all effective feature matching pairs; the number of pairs that simultaneously satisfy z_3d[id]>0 and r3(P[id]-T1)>0 is counted, which is Num_Cheirality_Condition1:

[0104]

[0105] (3) Select the solution corresponding to the maximum number of points that satisfy the positive depth of field constraint as the dual-camera coordinate system extrinsic parameter transformation relationship [R|T] output.

[0106] In solving binocular relationships, the conditions for filtering the four solutions should not be restricted; in principle, the optimal solution can be found by using these conditions.

[0107] The above technical solutions are merely exemplary embodiments of the present invention. For those skilled in the art, based on the application methods and principles disclosed in the present invention, it is easy to make various types of improvements or modifications, and not limited to the methods described in the specific embodiments of the present invention. Therefore, the methods described above are only preferred and not restrictive.

Claims

1. A calibration method for a calibration-free binocular camera system, comprising the following steps: S1, acquire a set of images of a relatively still scene captured by a binocular camera, and obtain the left camera image and the right camera image; S2, apply feature extraction algorithms to the left and right camera images respectively to obtain the features in the two camera images; S3, perform feature matching on the features of the left camera image and the right camera image to obtain a candidate set of feature matching pairs between the two cameras; S4, perform epipolar geometry constraint filtering on the candidate set of feature matching pairs of the two cameras to obtain the effective feature matching pairs and essence matrix of the two cameras that meet the epipolar geometry requirements; S5, determine whether the number of feature matching pairs contained in PairMatches meets the requirement: S51, if the requirements are not met, then obtain another set of images of a relatively static scene taken with a binocular camera, and repeat the above steps to obtain valid feature matching pairs and essence matrix. S52, if the requirements are met, then based on the effective feature matching, perform binocular relationship calculation on PairMatches, the essential matrix, and the known binocular relationship to be calibrated for the dual cameras' intrinsic parameters and distortion coefficients, to obtain the dual camera coordinate system extrinsic parameter transformation relationship [R|T], where R is the rotation matrix and T is the translation matrix.

2. The method according to claim 1, characterized in that, The specific method of step S1 includes: Keep the scene relatively still and use two cameras to take pictures separately, or use two cameras to trigger simultaneously to capture scene images.

3. The method according to claim 1, characterized in that, The SURF feature extraction method is used in step S2.

4. The method according to claim 1, characterized in that, The nearest neighbor feature matching algorithm is used in step S3.

5. The method according to claim 1, characterized in that, The method for filtering the candidate set by epipolar geometry constraints based on feature matching between the two cameras in step S4 specifically includes the following steps: S41, randomly sample N feature matching pairs from the candidate set PutativePairMatches containing N0 feature matching pairs to form the feature matching pair candidate sample set PairMatchesSample. S42, using feature matching, calculate the essential matrix sample set EssensialMatrixSamples[m] from the candidate sample set PairMatchesSample; S43. For each EssensialMatrixSamples[i], i = 0, 1, ..., m-1, sort the matching pairs according to the residual order, and then add the matching pairs to the inner point set combination in order of the smallest residual. Iterate and calculate the deviation corresponding to different inner point set combinations, update the EssensialMatrixSample with the smallest deviation as the BestEssensialMatrixSample, and update the corresponding inner point set combination as the output inner point set Inliers. S44, taking the above steps S41-S43 as one iteration, determine whether the maximum number of iterations has been reached. If not, continue to the next iteration. If the maximum number of iterations is reached, the BestEssensialMatrixSample is taken as the EssensialMatrix, and the corresponding output inliers are taken as the effective feature matching pairs between the two cameras that satisfy the epipolar geometry requirements.

6. The method according to claim 5, characterized in that, The specific method of step S42 includes: Suppose that there are N feature matching pairs in the candidate sample set PairMatchesSample as follows: {(x1,y1),(x1′,y1′)},…,{(x N ,yN),(x N ′,y N ′)} Where (x1,y1),...,(x N ,y N ) are the normalized image coordinates of N feature points in the left camera, (x1′, y1′), ..., (x N ′,y N ′) are the normalized image coordinates of N feature points in the right camera; Let each essential matrix solution EssensialMatrixSamples[i], i = 0, 1, ..., m-1, in the essential matrix sample set EssensialMatrixSamples[m] be represented as Since the essential matrix EssensialMatrixSamples[i] must satisfy: Then we can obtain: U N*9 e=0 in, and=[And 11i ,AND 21i ,AND 31i ,AND 12i ,AND 22i ,AND 32i ,AND 13i ,AND 23i ,AND 33i ] T ; Solve for U N*9 When e=0, we can obtain m essential matrix solutions EssensialMatrixSamples[i], i=0,1,……,m-1.

7. The method according to claim 6, characterized in that, The specific method of step S43 includes: Let the number of matching pairs in the candidate set PutativePairMatches be N0. The set of remaining feature matching pairs after removing N candidate samples is called the candidate complement set PairMatchesResiduals. Then the number of matching pairs in the candidate complement set is NR = N0 - N. For each essential matrix sample, output EssensialMatrixSamples[i], i = 0, 1, ..., m-1: a) Calculate the residual ResidualError[p], p=0,1,……,NR-1 for each matching pair {(x_residual[p],y_residual[p]), (x_residual′[p],y_residual′[p])} in the candidate complement set of feature matching pairs; where the residual calculation method includes: Calculate the fundamental matrix FundamentalMatrixSample[i] corresponding to EssensialMatrixSamples[i]: FundamentalMatrix[i]=(M_R -1 ) T ·EssentialMatrix[i]·M_L -1 ; Using the known intrinsic parameter matrix M_L of the left camera, the intrinsic parameter matrix M_R of the right camera, the distortion coefficients D_L of the left camera, and the distortion coefficients D_R of the right camera, the coordinate points are obtained respectively. c p(x_residual[p], y_residual[p]) and c p '(x_residual′[p],y_residual′[p]) The coordinates of the points after distortion correction (x_residual_undistort[p],y_residual_undistort[p]),(x_residual_undistort′[p],y_residual_undistort′[p]); Calculate the polar line epLine[p]: The residual is then expressed as: b) Sort the matching pairs in the candidate complement of feature matching pairs according to the calculated residuals, and exclude matching pairs whose residuals are greater than the residual threshold to obtain the sorted candidate complement of feature matching pairs. c) Using the candidate samples of feature matching pairs as the initial inlier set, the matching pairs in the sorted candidate supplement set of feature matching pairs are added to the initial inlier set in order of increasing residual, resulting in different combinations of inlier sets. The error corresponding to each combination of inlier sets is calculated one by one. If the currently calculated error is the lowest in history, the essential matrix sample at this time is updated to the best essential matrix sample BestEssensialMatrixSample, and the corresponding combination of inlier sets is updated as the output inlier set Inliers. The method for calculating the deviation Error corresponding to each combination of interior point sets includes: Suppose the number of matching pairs in the interior set combination is k (k = NSample + 1, ..., Num), then in, Where D is the length of the image diagonal, A is the area of ​​the image, and e k The k-th smallest residual is calculated from the EssentialMatrixSamples[i] output as the essential matrix sample. This is the residual corresponding to the last pair of matching pairs added to the inner point set combination in the above process when the number of matching pairs in the inner point set combination is k.

8. The method according to claim 7, characterized in that, Step S52 specifically includes: The essential matrix is ​​decomposed by SVD to obtain four possible solutions for the transformation relationship of the extrinsic parameters of the dual-camera coordinate system: [R1|T1], [R2|T2], [R3|T3], [R4|T4]. The positive depth constraint is checked on the four possible solutions [R1|T1], [R2|T2], [R3|T3], and [R4|T4] respectively, and the number of points that satisfy the positive depth constraint is obtained respectively. The solution corresponding to the maximum number of points satisfying the positive depth of field constraint is selected as the output of the dual-camera coordinate system extrinsic parameter transformation relationship [R|T].

9. The method according to claim 8, characterized in that, In step S52, the specific method for obtaining four possible solutions includes: Perform SVD decomposition on the essential matrix: EssentialMatrix=U0DV0 T Where U0 and V0 are both 3x3 matrices, Let U0 = [u1 u2 u3], Where u1, u2, and u3 are 3x1 column vectors, and v1, v2, and v3 are 1x3 row vectors; then: Where det() represents solving the determinant; make The four possible solutions are represented as follows: Where U[:,3] represents the 3rd column of matrix U.

10. The method according to claim 9, wherein in step S52, the method for performing a positive depth-of-field constraint verification on the possible solutions [R1|T1] obtained by the solution to obtain the number of points satisfying the positive depth-of-field constraint includes: set up Where r1, r2, and r3 are 1*3 row vectors; Let the intrinsic parameter matrix of the left camera be M_L, the intrinsic parameter matrix of the right camera be M_R, the distortion coefficients of the left camera be D_L, and the distortion coefficients of the right camera be D_R. Using the possible solution [R1|T1], we perform 3D coordinate reconstruction on all matching pairs in the effective feature matching pairs PairMatches in turn, and obtain the 3D coordinates P[id](x_3d[id],y_3d[id],z_3d[id]) corresponding to all effective feature matching pairs, id=0,……,PairedNum-1; The number of points that simultaneously satisfy z_3d[id]>0 and r3(P[id]-T1)>0 is the number of points in the possible solution [R1|T1] that satisfy the positive depth constraint.

Images