Binocular camera external parameter correction method and device, equipment and storage medium

Abstract

The application relates to the field of image processing, and provides a binocular camera external parameter correction method and device based on deep learning feature matching, equipment and a storage medium. The method comprises the following steps: acquiring left and right original images containing a preset calibration board which are synchronously collected by a binocular camera; processing the left and right original images, extracting a calibration board corner point and acquiring a scene feature matching point; performing dynamic consistency evaluation and screening on the scene feature matching point by using epipolar geometry constraints to obtain an optimized feature matching point set; based on the optimized feature matching point set, using a rotation matrix obtained in calibration board pose estimation as a prior constraint of a rotation parameter, and using a fixed baseline length of the binocular camera as a length constraint of a translation parameter, a nonlinear optimization model is constructed by fusing the prior constraint and the length constraint; and by minimizing the three-dimensional re-projection error of the optimized feature matching point set, the rotation matrix and the translation vector of the binocular camera are jointly optimized and calculated, and the correction of the external parameter matrix is completed.

Claims

1. A method for extrinsic parameter correction of a binocular camera, characterized in that, The method includes: Acquire the original left and right images, including the preset calibration plate, simultaneously captured by the binocular cameras; The original left and right images are processed to extract the calibration board corner points and obtain scene feature matching points; The epipolar matrix is ​​calculated based on the extracted calibration board corner points, and the epipolar matrix is ​​used to perform dynamic consistency evaluation and screening of the scene feature matching points to obtain an optimized feature matching point set. Based on the optimized feature matching point set, the rotation matrix obtained in the calibration board pose estimation is used as a prior constraint for the rotation parameter, and the baseline length of the fixed binocular camera is used as a module length constraint for the translation parameter to construct a nonlinear optimization model that integrates the prior constraint and the module length constraint. The nonlinear optimization model is solved by minimizing the 3D reprojection error of the optimized feature matching point set, and the rotation matrix and translation vector of the binocular camera are jointly optimized to complete the correction of the extrinsic parameter matrix.

2. The binocular camera extrinsic parameter correction method according to claim 1, characterized in that, The method of using the rotation matrix obtained in the calibration board pose estimation as a prior constraint for the rotation parameters includes: parameterizing the rotation matrix as a unit quaternion or rotation vector; adding a regularization term about the parameterized unit quaternion or rotation vector in the nonlinear optimization model, wherein the regularization objective is the centered calibration board pose rotation matrix, and the regularization weight is dynamically adjusted according to the confidence level extracted from the calibration board corner points; The method further includes: using the translation vector obtained in the calibration plate pose estimation to determine the optimized initial value of the translation parameter.

3. The binocular camera extrinsic parameter correction method according to claim 1, characterized in that, The method of calculating the epipolar matrix based on the extracted calibration board corner points includes: using the correspondence between the calibration board corner points in the left and right images, as well as the known physical dimensions and three-dimensional structure of the calibration board, to calculate the fundamental matrix or the essential matrix, wherein the epipolar matrix is ​​defined by the fundamental matrix or the essential matrix.

4. The binocular camera extrinsic parameter correction method according to claim 3, characterized in that, The step of using the epipolar matrix to perform dynamic consistency evaluation and filtering of the scene feature matching points to obtain an optimized feature matching point set includes: For each pair of scene feature matching points, calculate the geometric error that satisfies the epipolar matrix; The consistency error threshold is dynamically determined based on the geometric error distribution of all scene feature matching point pairs. Matching point pairs with geometric errors greater than the consistency error threshold are discarded as mismatched point pairs, while matching point pairs with geometric errors less than or equal to the consistency error threshold are retained to form the optimized feature matching point set.

5. The binocular camera extrinsic parameter correction method according to claim 1, characterized in that, The modulus constraint, which uses the baseline length of the fixed binocular camera as the translation parameter, includes: The magnitude of the translation vector is read as a fixed value from the factory calibration parameters of the binocular camera or the last valid calibration result; During the optimization process, the translation vector t is parameterized as follows: ,in, and The direction angle parameter to be optimized is... This is the fixed value.

6. The binocular camera extrinsic parameter correction method according to claim 1, characterized in that, After the extrinsic parameter matrix is ​​corrected, if the size information of the calibration plate in the image is available, the method further includes the following scale recovery process: The scale factor is calculated based on the proportional relationship between the known physical dimensions of the calibration plate and the three-dimensional dimensions of the calibration plate reconstructed from the corrected extrinsic parameters. The optimized translation vector is scaled using the scale factor to restore the true scale of the stereo camera's extrinsic parameters.

7. The binocular camera extrinsic parameter correction method according to claim 6, characterized in that, If valid calibration plate size information cannot be obtained from the current image, the scale recovery process includes: Read the translation vector magnitude stored in the factory calibration file or historical calibration file of the stereo camera and use it as the true scale; The optimized, unscaled translation vector is normalized and then multiplied by the true scale to obtain a translation vector with the true scale.

8. A binocular camera extrinsic parameter correction device, characterized in that, The device includes: The acquisition module is used to acquire the original left and right images, including the preset calibration plate, synchronously acquired by the binocular camera; The extraction module is used to process the left and right original images, extract the corner points of the calibration board, and obtain scene feature matching points; The optimization module is used to calculate the epipolar matrix based on the extracted calibration board corner points, and to use the epipolar matrix to perform dynamic consistency evaluation and screening of the scene feature matching points to obtain an optimized feature matching point set. The modeling module is used to construct a nonlinear optimization model that integrates the prior constraints and the module length constraints based on the optimized feature matching point set, using the rotation matrix obtained in the calibration plate pose estimation as the prior constraint for the rotation parameter, and the baseline length of the fixed binocular camera as the translation parameter for the module length constraint. The correction module is used to solve the nonlinear optimization model. By minimizing the 3D reprojection error of the optimized feature matching point set, it jointly optimizes and calculates the rotation matrix and translation vector of the binocular camera to complete the correction of the extrinsic parameter matrix.

9. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images