Imaging parameter calibration method of large-view-field binocular stereoscopic vision system assisted by moving camera

Abstract

A method for calibrating imaging parameters of a large-view-field binocular stereoscopic vision system assisted by a moving camera comprises the following steps: taking a left camera and / or a right camera in a binocular system as an auxiliary camera, moving the auxiliary camera to a plurality of view angles to carry out first image acquisition, and resetting the auxiliary camera to the binocular system to carry out secondary image acquisition. After at least three view angle images are obtained through two times of image acquisition, one-time joint calibration of internal parameters and external parameters of all view angle cameras is achieved through the multi-view angle self-calibration technology, and finally binocular system imaging parameters are extracted. According to the method, world coordinates of space points do not need to be known, dependence of a high-precision calibration object is avoided, the method is suitable for a large-view-field scene, the method has the advantages of being easy and convenient to operate, high in calibration precision, high in robustness and the like, and the portability and practicability of a binocular stereoscopic vision system in large-view-field measurement are remarkably improved.

Description

TECHNICAL FIELD

[0001] The application relates to a kind of binocular vision field technologies, in particular to a kind of dynamic camera auxiliary large field of view binocular stereo vision system imaging parameter calibration method. BACKGROUND

[0002] Binocular stereo vision system is imaged from different angles of view to the same scene by two cameras, and the high-precision three-dimensional reconstruction is realized by using the principle of triangulation, combining feature point matching in different angle of view images and accurately calibrated imaging parameters (including equivalent focal length, principal point coordinates, distortion coefficient and other internal parameters and rotation matrix, translation vector and other external parameters). The existing binocular stereo vision system imaging parameter calibration technology still has the following problems: 1) the calibration method of known space point world coordinates can calibrate all imaging parameters at a time, but is limited by high manufacturing cost and low manufacturing precision of calibration object, and is not suitable for large field calibration; 2) the calibration method of known part of imaging parameters does not need large field calibration object with known space coordinates, but needs to use the internal parameter calibration result under small field for large field external parameter calibration, which has the problem of low calibration accuracy. SUMMARY

[0003] The application proposes a kind of dynamic camera auxiliary large field of view binocular stereo vision system imaging parameter calibration method to solve the above problems in the prior art, which can realize the one-time calibration of all imaging internal and external parameters by using multi-view geometric constraints without knowing the world coordinates of space points, and effectively solve the portable calibration problem of binocular stereo vision system in large field measurement.

[0004] The application is realized by the following technical solutions:

[0005] The application relates to a kind of dynamic camera auxiliary large field of view binocular stereo vision system imaging parameter calibration method, which moves the left and / or right camera in the binocular system as an auxiliary camera to multiple angles of view for the first image acquisition, then resets the auxiliary camera to the binocular system for the second image acquisition, obtains at least three angle of view images after two image acquisitions, realizes the one-time joint calibration of all angle of view camera internal and external parameters by using multi-view self-calibration technology, and finally extracts the binocular system imaging parameters, which specifically includes:

[0006] Step 1, constructing a binocular stereo vision system, wherein the two cameras are left and right cameras, and the left and right cameras are adjusted to ensure that the left and right cameras can clearly image the measured scene and the measured scene is in the common field of view of the left and right cameras.

[0007] Preferably, the left and right cameras are of the same model, and the imaging lenses are of the same model.

[0008] Preferably, the image acquisition parameters include: image resolution, lens focal length / image distance, exposure time, acquisition frame rate, gain, etc. The adjusted image acquisition parameters are the final parameter settings for the left and right cameras and will not be changed again.

[0009] Preferably, the spatial pose of the left and right cameras is the final layout scheme of the binocular stereo vision system. In the final layout scheme, the optical axis angle between the left and right cameras is between 5° and 30°.

[0010] Preferably, the scene being tested should include rich feature information, such as a speckle pattern, which remains stationary during the calibration process.

[0011] Step 2: Construct auxiliary cameras and their imaging perspectives: Select the left and right cameras, or both, from the binocular stereo vision system as auxiliary cameras. By adjusting the different poses and perspectives of the auxiliary cameras, perform multiple sets of clear images of the scene under test. This is the initial image acquisition.

[0012] The auxiliary camera is designated as follows: when the left camera is selected, it is designated as the left auxiliary camera; when the right camera is selected, it is designated as the right auxiliary camera; when both the left and right cameras are selected, they are designated as the left and right auxiliary cameras respectively. When the left auxiliary camera has m imaging angles (m≥0) and the right auxiliary camera has n imaging angles (n≥0), m+n≥1 must be satisfied.

[0013] Preferably, the different perspectives should ensure that the left auxiliary camera or (and) the right auxiliary camera have a common field of view for the image of the scene under test.

[0014] Preferably, the angle between the optical axis of the left auxiliary camera and / or the right auxiliary camera and the adjacent viewing angle optical axis is between 5° and 10°.

[0015] Step 3, Auxiliary Camera Reset: Reassemble the left and right cameras into a binocular stereo vision system using the original spatial pose parameters from Step 1, and acquire images of the scene under test.

[0016] Step 4: Using a multi-view self-calibration algorithm, based on the m+n+2 (≥3) view images obtained from the initial and secondary image acquisitions, perform a one-time calibration of all extrinsic parameters. This includes:

[0017] 4.1 The virtual camera numbers corresponding to the m views of the left auxiliary camera are respectively The virtual camera numbers corresponding to the n views of the right auxiliary camera are respectively... The left camera's number is The right camera's number is , of which: numbered Virtual camera's internal parameters and left camera The internal references are the same, numbered as Virtual camera's built-in inputs and right camera The internal parameters are the same; all virtual cameras, left camera, and right camera are sorted from left to right as follows: .

[0018] 4.2 Estimation Initial values ​​of intrinsic parameters: set Initial value vector of intrinsic parameters All are equal, i.e., equivalent focal length , Principal point coordinates , distortion coefficient , The initial values ​​are all equal, where: This is the ratio of the lens focal length to the periodic size of the pixel in the x-direction. It is the ratio of the lens focal length to the periodic size of the pixel in the y-direction; Half the horizontal resolution of the image. Half the vertical resolution of the image; and It is 0.

[0019] 4.3 Construction Feature point matching relationships in images captured from different viewpoints: obtained using inter-viewpoint matching techniques. Pixel coordinates of matching feature points in a captured image .

[0020] The aforementioned viewpoint matching technology employs, but is not limited to, three-dimensional digital image correlation viewpoint matching technology, scale-invariant feature matching algorithm (SIFT), and accelerated robust feature algorithm (SURF).

[0021] 4.4 Estimation Initial values ​​of extrinsic parameters: Constructing the epipolar geometry equations Solve for the fundamental matrix F; according to Solve for the essential matrix E; according to ,estimate The initial value of the external parameter, i.e. .

[0022] 4.5 Estimating Matching Feature Points The corresponding initial world coordinates are: the initial intrinsic parameters estimated in step 4.2. Step 4.3 Matching feature point coordinates and the initial values ​​of the external parameters estimated in step 4.4 Based on the principle of binocular triangulation, the matching feature points are estimated. The corresponding initial world coordinates.

[0023] 4.6 Estimating New Perspectives Initial values ​​of extrinsic parameters: Initial values ​​of intrinsic parameters estimated in step 4.2. Step 4.3 Matching feature point coordinates And the initial world coordinates estimated in step 4.5 are used to solve for the new perspective using monocular pose estimation techniques. Initial values ​​of external parameters .

[0024] The monocular pose estimation technique described herein employs, but is not limited to, multi-point perspective imaging (PNP) technology.

[0025] 4.7 Reconstruct and update the world coordinates of feature points from multiple perspectives: Based on the initial intrinsic parameters estimated in step 4.2 Step 4.3 Matching feature point coordinates Step 4.4 Estimated initial values ​​of external parameters and the initial values ​​of the external parameters estimated in step 4.6 Based on the principle of multi-view triangulation, the matching feature points are estimated. The corresponding initial world coordinates.

[0026] 4.8 Update the intrinsic parameters, extrinsic parameters, and world coordinates of feature points of the viewpoint, specifically as follows: ,in: For imaging intrinsic and extrinsic parameters, For the world coordinates of the feature point, For reprojection error, The scaling factor is used. Using nonlinear optimization of beam adjustment, with the goal of minimizing reprojection error, the optimized imaging intrinsic and extrinsic parameters are obtained. and optimized feature point world coordinates .

[0027] In the aforementioned beam adjustment nonlinear optimization process, the following settings are made: The internal references are the same. The internal references are the same.

[0028] 4.9 Repeat steps 4.6-4.8 to obtain the result. The optimal imaging intrinsic and extrinsic parameters are determined, and the final optimized intrinsic parameters are denoted as... , External references are recorded as , , … , , , , … All of the above external parameters are based on The viewpoint coordinate system is used as a reference.

[0029] Step 5: Obtain imaging parameters of the binocular stereo vision system: Using the left camera as the reference coordinate system, transform the extrinsic parameters of the right camera calibrated in Step 4 to the reference coordinate system, specifically as follows: , .

[0030] Step 6: Calibrate the scaling factor: Use the binocular stereo vision system after imaging parameter calibration to measure the geometric object (size b) of objective size in space, and obtain the measured size of the geometric object as a; then the scaling factor is a / b.

[0031] Technical effect

[0032] This invention utilizes a self-calibration technology for a binocular stereo vision system with an auxiliary camera. This technology enables the one-time calibration of all imaging intrinsic and extrinsic parameters of a binocular stereo vision system without requiring known world coordinates of spatial points. The calibration process is simple and cost-effective. Therefore, this invention is particularly suitable for calibration scenarios with a large field of view. Attached Figure Description

[0033] Figure 1 This is a flowchart of the invention;

[0034] Figure 2 This is a schematic diagram of an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of speckle images captured by the left auxiliary camera, left camera, right camera, and right auxiliary camera in the embodiment.

[0036] Figure 4 This is the checkerboard image required by the MATLAB calibration toolbox. Detailed Implementation

[0037] like Figure 1 As shown in this embodiment, a method for calibrating imaging parameters of a large field-of-view binocular stereo vision system with a moving camera assistance is provided, including:

[0038] S1. Constructing a binocular stereo vision system: Both the left and right cameras are Daheng MER2-160-227U3M cameras, each equipped with a 4mm Computar lens. The imaging parameters of the left and right cameras were adjusted as follows: resolution 1440 pixels × 1280 pixels, frame rate 1 fps, exposure time 1 ms, and gain set to 0. The optical axis angle between the left and right cameras was approximately 10°. The scene under test was a speckle cylinder. Both cameras provided clear images of the speckle scene, and the scene was within the common field of view of both cameras. Figure 2 As shown.

[0039] S2. Constructing Auxiliary Cameras and Their Imaging Viewpoints: Select the left camera from S1 as the auxiliary camera, designated as the left auxiliary camera; select the right camera from S1 as the auxiliary camera, designated as the right auxiliary camera. The left auxiliary camera has an imaging viewpoint of 1 (i.e., m=1) and is placed to the left of the left camera, with an angle of approximately 10° between its optical axes. The right auxiliary camera has an imaging viewpoint of 1 (i.e., n=1) and is placed to the right of the right camera, with an angle of approximately 10° between its optical axes. The left and right auxiliary cameras image the speckle scene. The acquired images are shown below. Figure 3 As shown.

[0040] S3. Auxiliary Camera Reset: Adjust the left auxiliary camera back to the position described in step S1, and adjust the right auxiliary camera back to the position described in step S1, then acquire images. The acquired images are as follows: Figure 3 As shown.

[0041] S4. Multi-view self-calibration: The left auxiliary camera, left camera, right camera, and right auxiliary camera are numbered from left to right as follows: .in The internal references are the same. The internal references are the same.

[0042] Assumption The initial values ​​of the intrinsic parameters are all: equivalent focal length is The principal point coordinates are , distortion coefficient .

[0043] Using the SIFT algorithm, matching Figure 3 Feature points in speckle scene images taken from different perspectives.

[0044] By further combining techniques such as epipolar geometry, PNP, binocular stereo reconstruction, multi-view stereo reconstruction, and bundle adjustment, the intrinsic and extrinsic parameters of the left auxiliary camera, left camera, right camera, and right auxiliary camera were calibrated all at once. The calibration results are shown in Tables 1 and 2.

[0045] Table 1. Results of Intrinsic Parameter Calibration for Multi-View Imaging

[0046] Table 2. Results of extrinsic parameter calibration for multi-view imaging (in terms of...) (The reference coordinate system is used; the rotation matrix is ​​expressed in Euler angles)

[0047] S5. Obtain the imaging parameters of the binocular stereo vision system: Using the left camera as the reference coordinate system, transform the extrinsic parameters of the right camera calibrated in S4 to the reference coordinate system. The imaging parameters of the binocular stereo vision system are shown in Tables 3 and 4.

[0048] Table 3. Intrinsic parameter calibration results of binocular stereo vision system

[0049] Table 4. Calibration results of imaging extrinsic parameters of the binocular stereo vision system (with the left camera as the reference coordinate system; rotation matrix is ​​represented by quaternions)

[0050] S6. Calibration of scale factor: Using the calibrated binocular stereo vision system, a ruler with a length of 20cm in space is reconstructed. The reconstruction result is 1.0749cm, so the scale factor is 18.6064. The final calibration results of the binocular stereo vision system are shown in Table 5.

[0051] Table 5. Final calibration results of imaging extrinsics for the binocular stereo vision system (with the left camera as the reference coordinate system; rotation matrices are represented by quaternions).

[0052] Table 6 shows the calibration results of the binocular stereo vision system imaging parameters in this embodiment and the calibration results of the checkerboard-based MATLAB calibration toolbox. The calibration results are comparable. The calibration results show that this method can complete the high-precision calibration of the intrinsic and extrinsic parameters of the binocular stereo vision system imaging in one step without the need for high-precision calibration of the target.

[0053] Table 6. Comparison of results from this method and the MATLAB calibration toolbox (rotation matrices are expressed in Euler angles)

[0054] Through practical experiments, this method, under specific environmental settings where the left camera moves once and the right camera moves once in a binocular stereo vision system, can simultaneously calibrate the intrinsic and extrinsic parameters of the binocular stereo vision system after acquiring speckle scene data from four different perspectives. Using the Zhang's calibration method in MATLAB as a reference, the parameter calibration error is within 10%, and the calibration error for individual parameters is within 5%. The calibration results are shown in Table 6. In summary, this method employs a moving camera-assisted technique to achieve simultaneous calibration of the intrinsic and extrinsic parameters of a binocular stereo vision system without requiring known world coordinates of spatial points. During the calibration process, the consistency constraint between the intrinsic parameters of the auxiliary camera and the left (right) camera is considered, significantly improving the calibration accuracy of the binocular stereo vision system.

[0055] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A method for calibrating imaging parameters of a large field-of-view binocular stereo vision system assisted by a moving camera, characterized in that, By using the left and / or right cameras in the binocular system as auxiliary cameras and moving them to multiple viewpoints for initial image acquisition, and then resetting the auxiliary cameras to the binocular system for secondary image acquisition, at least three viewpoint images are obtained through the two image acquisitions. Then, using multi-view self-calibration technology, the intrinsic and extrinsic parameters of all viewpoint cameras are calibrated simultaneously, and finally, the imaging parameters of the binocular system are extracted.

2. The method for calibrating imaging parameters of a large field-of-view binocular stereo vision system with a moving camera assisted according to claim 1, characterized in that, The binocular stereo vision system includes two cameras, namely the left and right cameras. By adjusting the image acquisition parameters of the left and right cameras and the spatial pose of the left and right cameras, it is ensured that the left and right cameras can clearly image the scene under test and that the scene under test is within the common field of view of the left and right cameras.

3. The method for calibrating imaging parameters of a large field-of-view binocular stereo vision system with a moving camera assisted according to claim 1, characterized in that, The first image acquisition is achieved by selecting the left and right cameras or both of them in the binocular stereo vision system as auxiliary cameras, and by adjusting the different poses and different angles of the auxiliary cameras, multiple sets of clear images are taken of the scene under test, which is the first image acquisition. The auxiliary camera, when the left camera is selected, is designated as the left auxiliary camera; When the right camera is selected, it will be designated as the right auxiliary camera; When selecting a left camera and a right camera, they are designated as the left auxiliary camera and the right auxiliary camera, respectively. When the left auxiliary camera has m imaging angles (m≥0) and the right auxiliary camera has n imaging angles (n≥0), the condition m+n≥1 must be met.

4. The method for calibrating imaging parameters of a large field-of-view binocular stereo vision system with a moving camera assisted according to claim 1, characterized in that, The secondary image acquisition involves using the left and right cameras to form a binocular stereo vision system again according to the original spatial pose parameters, and then acquiring images of the scene under test.

5. The method for calibrating imaging parameters of a large field-of-view binocular stereo vision system with a moving camera assisted according to claim 1, characterized in that, The aforementioned multi-view self-calibration technology specifically includes: 4.1 The virtual camera numbers corresponding to the m views of the left auxiliary camera are respectively The virtual camera numbers corresponding to the n views of the right auxiliary camera are respectively... The left camera's number is The right camera's number is , of which: numbered Virtual camera's internal parameters and left camera The internal references are the same, numbered as Virtual camera's built-in inputs and right camera The internal parameters are the same; all virtual cameras, left camera, and right camera are sorted from left to right as follows: ; 4.2 Estimation Initial values ​​of intrinsic parameters: set Initial value vector of intrinsic parameters All are equal, i.e., equivalent focal length , Principal point coordinates , distortion coefficient , The initial values ​​are all equal, where: This is the ratio of the lens focal length to the periodic size of the pixel in the x-direction. It is the ratio of the lens focal length to the periodic size of the pixel in the y-direction; Half the horizontal resolution of the image. Half the vertical resolution of the image; and =0, 4.3 Construction Feature point matching relationships in images captured from different viewpoints: obtained using inter-viewpoint matching techniques. Pixel coordinates of matching feature points in a captured image , 4.4 Estimation Initial values ​​of extrinsic parameters: Constructing the polar geometry equations Solve for the fundamental matrix F; according to Solve for the essential matrix E; according to ,estimate The initial value of the external parameter, i.e. , 4.5 Estimating Matching Feature Points The corresponding initial world coordinates are: the initial intrinsic parameters estimated in step 4.

2. Step 4.3 Matching feature point coordinates and the initial values ​​of the external parameters estimated in step 4.4 Based on the principle of binocular triangulation, the matching feature points are estimated. The corresponding initial world coordinates, 4.6 Estimating New Perspectives Initial values ​​of extrinsic parameters: Initial values ​​of intrinsic parameters estimated in step 4.

2. Step 4.3 Matching feature point coordinates And the initial world coordinates estimated in step 4.5 are used to solve for the new perspective using monocular pose estimation techniques. Initial values ​​of external parameters , 4.7 Reconstruct and update the world coordinates of feature points from multiple perspectives: Based on the initial intrinsic parameters estimated in step 4.2 Step 4.3 Matching feature point coordinates Step 4.4 Estimated initial values ​​of external parameters and the initial values ​​of the external parameters estimated in step 4.6 Based on the principle of multi-view triangulation, the matching feature points are estimated. The corresponding initial world coordinates, 4.8 Update the intrinsic parameters, extrinsic parameters, and world coordinates of feature points of the viewpoint, specifically as follows: ,in: For imaging intrinsic and extrinsic parameters, For the world coordinates of the feature point, For reprojection error, Using the scaling factor, nonlinear optimization of beam adjustment is employed, with the goal of minimizing reprojection error, to obtain the optimized imaging intrinsic and extrinsic parameters. and optimized feature point world coordinates , 4.9 Repeat steps 4.6-4.8 to obtain... The optimal imaging intrinsic and extrinsic parameters are determined, and the final optimized intrinsic parameters are denoted as... , External references are recorded as , , … , , , , … The external parameters are all based on The viewpoint coordinate system is used as a reference.

6. The method for calibrating imaging parameters of a large field-of-view binocular stereo vision system assisted by a moving camera according to claim 5, characterized in that, The aforementioned multi-view self-calibration technology uses the left camera as the reference coordinate system and transforms the extrinsic parameters of the already calibrated right camera to the reference coordinate system, specifically as follows: , .

7. The method for calibrating imaging parameters of a large field-of-view binocular stereo vision system with a moving camera assisted according to claim 1, characterized in that, Using a binocular stereo vision system with calibrated imaging parameters, a geometric object of objective size in space is measured, and the measured size of the geometric object is obtained as a; then the scaling factor is calibrated as a / b, where b is the objective size of the geometric object.

Images