Binocular stereoscopic vision correction method, system and device and storage medium

Abstract

The invention relates to the field of camera calibration in image analysis, in particular to a binocular stereoscopic vision correction method, system and device and a storage medium. In order to realize high-precision binocular correction, firstly, a distortion degree objective function is constructed based on a calibration plate deformation curve and a corresponding distortion evaluation straight line, and distortion parameters are acquired to realize binocular camera image distortion removal; then, acquiring a camera internal reference matrix according to the distorted image of the calibration plate at different distances from the camera, and realizing binocular camera internal reference calibration; thirdly, constructing a binocular external parameter objective function based on the internal parameter matrix of the binocular camera, and obtaining a rotation matrix and a translation vector for realizing external parameter calibration of the binocular camera; and finally, a camera alignment matrix is calculated according to the binocular translation vector, a binocular camera correction rotation matrix is obtained, high-precision stereo correction of the binocular camera is achieved, when the method is used for obtaining images by the camera in a large-size industrial part scanning scene, an ideal image set can be obtained, and the accuracy of subsequent three-dimensional data reconstruction can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration in image analysis, and in particular to a binocular stereo vision correction method, system, device and storage medium. Background Art

[0002] Binocular stereo correction can accurately match the corresponding points of the two images obtained by the left and right cameras, reduce parallax calculation errors, and facilitate the subsequent accurate acquisition of the three-dimensional coordinate information of the object and improve the accuracy of three-dimensional reconstruction. For example, in industrial parts scanning, the part size information can be accurately obtained.

[0003] Existing binocular correction algorithms, such as those based on deep learning, require a large amount of training data and a complex network structure, have a long processing time, are inefficient, and are affected by training samples. They can only handle binocular correction problems in existing scenarios. When faced with new scenarios, the binocular correction quality is poor and the accuracy is low. Summary of the invention

[0004] The purpose of the present invention is to provide a binocular stereo vision correction method, system, device and storage medium suitable for reconstructing three-dimensional data in a large-size industrial part scanning scene.

[0005] The technical solution of the present invention is as follows: A binocular stereoscopic vision correction method includes the following operations: S1, the first camera and the second camera respectively obtain the calibration plate image to obtain the respective distorted images containing deformation curves; based on the data of each deformation curve pair and the corresponding straight line pair data before deformation in the respective distorted images, obtain the distortion feature points of each deformation curve pair; take the average of the positions of all the distortion feature points as the respective distortion center points; based on the first and last coordinates of each deformation curve in the respective distorted images, obtain the respective corresponding distortion evaluation lines; based on the sum of the areas formed by all the deformation curves and the corresponding distortion evaluation lines, construct the respective distortion degree objective functions; perform gradient descent processing on the respective distortion degree objective functions to obtain the respective distortion parameters; based on the respective distortion center points and the respective distortion parameters, perform distortion correction on the images obtained by the first camera and the second camera respectively; S2, obtaining dedistorted images of the calibration plate at different distances from the first camera and the second camera, respectively, to obtain a plurality of calibration images of each; based on the pixel length and width of the plurality of calibration images and the movement distance of the calibration plate, obtaining the horizontal focal length and vertical focal length of each camera; based on the horizontal focal length and vertical focal length of each camera, obtaining the first camera internal parameter matrix and the second camera internal parameter matrix, and performing internal parameter calibration on the first camera and the second camera respectively; S3. Based on the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix, a binocular extrinsic parameter objective function is constructed; the binocular extrinsic parameter objective function is solved by a nonlinear least square method to obtain a binocular rotation matrix and a binocular translation vector; based on the extrinsic parameter transformation formula between the first camera and the second camera, the binocular rotation matrix and the binocular translation vector are respectively decomposed to obtain a first camera rotation matrix, a second camera rotation matrix, a first camera translation vector and a second camera translation vector; based on the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector, the first camera and the second camera are extrinsically calibrated; S4. Based on the binocular translation vector, obtain a camera alignment matrix; multiply the camera alignment matrix with the first camera rotation matrix and the second camera rotation matrix respectively to obtain a first correction rotation matrix and a second correction rotation matrix; based on the first correction rotation matrix and the second correction rotation matrix, calibrate the first camera and the second camera respectively to make the optical axes of the first camera and the second camera parallel.

[0006] The method for obtaining the distortion feature points of the current deformation curve pair in S1 is specifically as follows: respectively obtain the central symmetric points of the first curve and the second curve in the current deformation curve pair to obtain the first central symmetric point and the second central symmetric point; respectively draw perpendicular lines through the first central symmetric point and the second central symmetric point to the first straight line and the second straight line of the current straight line pair before deformation to obtain the first perpendicular line and the second perpendicular line; and use the intersection position of the first perpendicular line and the second perpendicular line as the distortion feature point of the current deformation curve pair.

[0007] The distortion objective function in S1 is as follows: , is the distortion value, m For the m A deformation curve, M is the total number of deformation curves, For the m The deformation curve n-1 curve points to the distortion evaluation line I The vertical distance on For the m The deformation curve n curve points to the distortion evaluation line I The vertical distance on For the m The deformation curve n-1 The curve point and n The connection between the curve points is projected onto the distortion evaluation line I The distance on N For the mThe total number of curve points on the deformation curve; the vertical distance from the curve point to the corresponding distortion evaluation line is obtained based on the coordinates of the curve point; the distortion parameter is obtained based on the coordinates of the curve point and the coordinates of the corresponding point on the straight line before deformation.

[0008] The operations for obtaining the horizontal focal length and vertical focal length of the camera in S2 are specifically as follows: the camera obtains an image when the calibration plate is at a first position to obtain a first calibration image; the pixel width and length of the first calibration image are obtained to obtain a first pixel width and length; the camera obtains an image when the calibration plate is at a second position to obtain a second calibration image; the pixel width and length of the second calibration image are obtained to obtain a second pixel width and length; based on the first pixel width, the second pixel width and the movement distance of the calibration plate, a target-camera distance parameter is obtained; based on the target-camera distance parameter, the first pixel width and length, the calibration plate length and the calibration plate width, the horizontal focal length and vertical focal length of the camera are obtained.

[0009] The binocular external parameter objective function in S3 is as follows: , is the binocular extrinsic parameter objective function value, The first camera view j The transpose of the homogeneous coordinate matrix of the points, For the second camera view j The transpose of the homogeneous coordinate matrix of the points, is the inverse matrix of the transposed intrinsic parameter matrix of the first camera, is the inverse matrix of the second camera intrinsic parameter matrix, is the binocular rotation matrix, is the binocular translation vector, J is the total number of points.

[0010] The external parameter transformation formula between the first camera and the second camera in S3 is as follows: , , , t They are binocular rotation matrix and binocular translation vector respectively. , , , They are the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector, and the second camera translation vector respectively.

[0011] The operation of obtaining the camera alignment matrix in S4 is specifically as follows: taking the ratio of the binocular translation vector to the binocular translation vector modulus as the first alignment sub-matrix; obtaining the second alignment sub-matrix based on the x-axis translation vector and the y-axis translation vector in the binocular translation vector; taking the product of the first alignment sub-matrix and the second alignment sub-matrix as the third alignment sub-matrix; the first alignment sub-matrix, the second alignment sub-matrix and the third alignment sub-matrix form a camera alignment matrix.

[0012] A binocular stereoscopic vision correction system, used to implement the above binocular stereoscopic vision correction method, comprising: The distortion parameter generation and dedistortion module is used for the first camera and the second camera to respectively obtain the calibration plate images to obtain the respective distorted images containing deformation curves; based on the data of each deformation curve pair and the corresponding straight line pair data before deformation in the respective distorted images, the distortion feature points of each deformation curve pair are obtained; the average of the positions of all the distortion feature points is used as the respective distortion center points; based on the first and last coordinates of each deformation curve in the respective distorted images, the corresponding distortion evaluation lines are obtained; based on the sum of the areas formed by all the deformation curves and the corresponding distortion evaluation lines, the respective distortion degree objective functions are constructed; the respective distortion degree objective functions are processed by the gradient descent method to obtain the respective distortion parameters; based on the respective distortion center points and the respective distortion parameters, the images obtained by the first camera and the second camera are respectively subjected to distortion correction; The camera intrinsic parameter matrix generation and intrinsic parameter calibration module is used to obtain dedistorted images of calibration plates at different distances from the first camera and the second camera, respectively, to obtain a plurality of calibration images of each camera; based on the pixel length and width of the respective calibration images and the movement distance of the calibration plate, to obtain the horizontal focal length and vertical focal length of each camera; based on the horizontal focal length and vertical focal length of each camera, to obtain the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix, and to perform intrinsic parameter calibration on the first camera and the second camera respectively; The extrinsic parameter data generation and extrinsic parameter calibration module is used for the first camera and the second camera to respectively perform the operations of the camera intrinsic parameter matrix generation and the intrinsic parameter calibration module to obtain the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix; based on the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix, a binocular extrinsic parameter objective function is constructed; the binocular extrinsic parameter objective function is solved by using the nonlinear least square method to obtain the binocular rotation matrix and the binocular translation vector; based on the extrinsic parameter transformation formula between the first camera and the second camera, the binocular rotation matrix and the binocular translation vector are respectively decomposed and processed to obtain the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector; based on the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector, the first camera and the second camera are extrinsic parameter calibrated; The rotation matrix generation and camera correction module is used to obtain the camera alignment matrix based on the binocular translation vector; the camera alignment matrix is ​​multiplied with the first camera rotation matrix and the second camera rotation matrix respectively to obtain the first correction rotation matrix and the second correction rotation matrix; based on the first correction rotation matrix and the second correction rotation matrix, the first camera and the second camera are corrected respectively to make the optical axes of the first camera and the second camera parallel.

[0013] A binocular stereoscopic vision correction device comprises a processor and a memory, wherein the processor implements the above binocular stereoscopic vision correction method when executing a computer program stored in the memory.

[0014] A computer-readable storage medium is used to store a computer program, wherein the computer program implements the above-mentioned binocular stereoscopic vision correction method when executed by a processor.

[0015] The beneficial effects of the present invention are: The present invention provides a binocular stereo vision correction method. First, based on the obtained deformation curve pair data and the corresponding undeformed straight line pair data on the calibration plate, the distortion center point is obtained, and based on the sum of the areas formed by all the deformation curves and the corresponding distortion evaluation lines, a distortion degree objective function is constructed. Based on the distortion degree objective function, the distortion parameters are obtained, and combined with the distortion center point, the distortion correction of the image acquired by the binocular camera is realized; then, according to the dedistorted image of the calibration plate at different distances from the camera, the horizontal focal length and the vertical focal length of the camera are obtained, and based on the horizontal focal length and the vertical focal length of the camera, the camera intrinsic parameter matrix is ​​obtained; based on the camera intrinsic parameter matrix, the linear calibration of the binocular camera intrinsic parameters is realized; then, after obtaining the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix in the binocular camera, the distortion degree objective function is constructed. A binocular extrinsic parameter objective function is established, and the nonlinear least squares method is used to solve the objective function to obtain the binocular rotation matrix and the binocular translation vector. After decomposition, the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector are obtained, and the orientation relationship between the first camera and the second camera is obtained to realize the extrinsic parameter calibration of the binocular camera; finally, the camera alignment matrix is ​​calculated according to the binocular translation vector, and the first correction rotation matrix and the second correction rotation matrix are obtained to realize the stereo correction of the first camera and the second camera, so that the optical axes of the first camera and the second camera are parallel, and high-precision binocular stereo vision correction is realized. When the camera acquires images in large-size industrial parts scanning scenarios, an ideal stereo image set can be obtained, which can improve the accuracy of subsequent three-dimensional data reconstruction. DETAILED DESCRIPTION

[0016] This embodiment is suitable for stereo correction of a binocular camera in a large-size industrial parts scanning scenario. The left camera and the right camera (i.e., the first camera and the second camera) in the binocular camera are used to acquire the target image, and the calibration plate is used as a reference target to provide calibration reference data.

[0017] This embodiment provides a binocular stereoscopic vision correction method, including the following operations: S1, the first camera and the second camera respectively obtain the calibration plate image to obtain the respective distorted images containing deformation curves; based on the data of each deformation curve pair and the corresponding straight line pair data before deformation in the respective distorted images, obtain the distortion feature points of each deformation curve pair; take the average of the positions of all the distortion feature points as the respective distortion center points; based on the first and last coordinates of each deformation curve in the respective distorted images, obtain the respective corresponding distortion evaluation lines; based on the sum of the areas formed by all the deformation curves and the corresponding distortion evaluation lines, construct the respective distortion degree objective functions; perform gradient descent processing on the respective distortion degree objective functions to obtain the respective distortion parameters; based on the distortion center points, radial distortion parameters and tangential distortion parameters of the first camera and the second camera, perform distortion correction on the images obtained by the respective cameras; S2, obtaining dedistorted images of the calibration plate at different distances from the first camera and the second camera, respectively, to obtain a plurality of calibration images of each; based on the pixel length and width of the plurality of calibration images and the movement distance of the calibration plate, obtaining the horizontal focal length and vertical focal length of each camera; based on the horizontal focal length and vertical focal length of each camera, obtaining the first camera internal parameter matrix and the second camera internal parameter matrix, and performing internal parameter calibration on the first camera and the second camera respectively; S3. Based on the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix, a binocular extrinsic parameter objective function is constructed; the binocular extrinsic parameter objective function is solved by a nonlinear least square method to obtain a binocular rotation matrix and a binocular translation vector; based on the extrinsic parameter transformation formula between the first camera and the second camera, the binocular rotation matrix and the binocular translation vector are respectively decomposed to obtain a first camera rotation matrix, a second camera rotation matrix, a first camera translation vector and a second camera translation vector; based on the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector, the first camera and the second camera are extrinsically calibrated; S4. Based on the binocular translation vector, obtain a camera alignment matrix; multiply the camera alignment matrix with the first camera rotation matrix and the second camera rotation matrix respectively to obtain a first correction rotation matrix and a second correction rotation matrix; based on the first correction rotation matrix and the second correction rotation matrix, calibrate the first camera and the second camera respectively to make the optical axes of the first camera and the second camera parallel.

[0018] S1, the first camera and the second camera respectively obtain the calibration plate image to obtain the distorted image containing the deformation curve; based on the data of each deformation curve pair and the corresponding straight line pair data before deformation in the distorted image, the distortion feature point of each deformation curve pair is obtained; the mean value of the position of all the distortion feature points is used as the distortion center point; based on the first and last coordinates of each deformation curve in the distorted image, the corresponding distortion evaluation line is obtained; based on the sum of the areas formed by all the deformation curves and the corresponding distortion evaluation line, the distortion degree objective function is constructed; the distortion degree objective function is processed by the gradient descent method until convergence to obtain the distortion parameters; based on the distortion center point and the distortion parameters, the images obtained by the first camera and the second camera are respectively subjected to distortion correction.

[0019] Based on the deformation curve pair data and the corresponding straight line pair data before deformation in the calibration plate image, the distortion center point is obtained; and based on the sum of the areas formed by all deformation curves and the corresponding distortion evaluation lines, a distortion degree objective function is constructed; based on the distortion degree objective function, the distortion parameters are obtained and combined with the distortion center point to realize the distortion correction of the images obtained by the first camera and the second camera respectively.

[0020] The method for obtaining the above-mentioned distortion center point and distortion parameters (including radial distortion parameters and tangential distortion parameters) is specifically as follows. Since the methods for obtaining the distortion center point and distortion parameters (including radial distortion parameters and tangential distortion parameters) of the first camera and the second camera are similar, in order to save space, camera is used to replace the first camera or the second camera below.

[0021] First, a camera is used to obtain an image of a calibration plate containing several straight lines. Due to the presence of the lens in the camera lens, the propagation of light changes during the imaging process, and the image is distorted, causing the straight lines in the actual environment to become curves in the image, thus obtaining a distorted image containing a deformation curve.

[0022] Then, based on each deformed curve pair data and the corresponding undeformed straight line pair data in the distorted image, the distortion feature points of each deformed curve pair are obtained. The two straight lines in the undeformed straight line pair intersect, and the two deformed curves form a curve pair. After each undeformed straight line is distorted, the deformation curve data obtained includes but is not limited to fitting with a Gaussian function.

[0023] The method for obtaining the distortion feature points of the current deformation curve pair is as follows: according to the deformation curve pair data and the corresponding undeformed straight line pair data, all deformation curves and undeformed straight lines are plotted in the same mathematical calculation model (such as an Excel icon), and the central symmetric points of the first curve and the second curve in the current deformation curve pair are obtained respectively, and the first central symmetric point and the second central symmetric point are obtained. In this embodiment, a Gaussian curve is used to fit the position point relationship in the deformation curve, and the symmetric point of the Gaussian curve is the central symmetric point of the deformation curve. Through the first central symmetric point and the second central symmetric point, perpendicular lines corresponding to the first straight line and the second straight line of the current undeformed straight line pair are drawn respectively, that is, through the first central symmetric point, a perpendicular line to the first straight line corresponding to the first curve is drawn, and through the second central symmetric point, a perpendicular line to the second straight line corresponding to the second curve is drawn, and the first perpendicular line and the second perpendicular line are obtained; the intersection of the first perpendicular line and the second perpendicular line is used as the distortion feature point of the current deformation curve pair.

[0024] Next, in order to reduce the fitting error, the mean of all distorted feature point positions is taken as the distortion center point.

[0025] Then, based on the first and last coordinates of each deformation curve, the corresponding distortion evaluation line is obtained, that is, the straight line connecting the first and last coordinates of the deformation curve is the distortion evaluation line.

[0026] Finally, based on the sum of the areas formed by all deformation curves and the corresponding distortion evaluation lines, a distortion degree objective function is constructed; the distortion degree objective function is processed by the gradient descent method to obtain the distortion parameters.

[0027] Specifically, the distortion degree objective function is as follows: , is the distortion value, is the first radial distortion parameter , the second radial distortion parameter , the first tangential distortion parameter , the second tangential distortion parameter The objective function is m For the m A deformation curve, M is the total number of deformation curves, For the m The deformation curve n-1 curve points to the distortion evaluation line I The vertical distance on For the m The deformation curve n curve points to the distortion evaluation line I The vertical distance on For the m The deformation curven-1 The curve point and n The connection between the curve points is projected onto the distortion evaluation line I The distance on N For the m The total number of curve points on the deformation curve.

[0028] The vertical distance from the curve point to the corresponding distortion evaluation line is obtained based on the coordinates of the curve point, e.g. It is based on n The curve point is m The coordinate values ​​on the deformation curve ( )get.

[0029] The distortion parameters (including radial distortion parameters and tangential distortion parameters) are obtained based on the coordinates of the curve point and the coordinates of the corresponding points on the straight line before deformation.

[0030] Among them, the radial distortion parameter, the tangential distortion parameter, the coordinates of the curve point and the coordinates of the corresponding point on the straight line before deformation satisfy the following relationship: , , , , , are the horizontal and vertical coordinates of the curve points, , They are , The horizontal and vertical coordinates of the corresponding points on the straight line before deformation, , are the first radial distortion parameter and the second radial distortion parameter respectively, , are the first tangential distortion parameter and the second tangential distortion parameter respectively, .

[0031] Based on the relationship between the radial distortion parameters, tangential distortion parameters and the coordinates of the curve points and the coordinates of the corresponding points on the straight line before deformation, as well as the relationship between the coordinates of the curve points and the distortion degree objective function, the relationship between the distortion degree objective function and the radial distortion parameters and the tangential distortion parameters is established. The distortion degree objective function is processed by the gradient descent method until convergence to obtain the radial distortion parameters and the tangential distortion parameters.

[0032] S2. Obtain dedistorted images of the calibration plate at different distances from the first camera and the second camera to obtain several calibration images of each camera; obtain the horizontal focal length and vertical focal length of each camera based on the pixel length and width of the calibration images and the movement distance of the calibration plate; obtain the intrinsic parameter matrix of the first camera and the intrinsic parameter matrix of the second camera based on the horizontal focal length and vertical focal length of each camera, and perform intrinsic parameter calibration on the first camera and the second camera respectively.

[0033] According to the dedistorted images of the calibration plate at different distances from the camera, the horizontal focal length and vertical focal length of the camera are obtained, and based on the horizontal focal length and vertical focal length of the camera, the camera intrinsic parameter matrix is ​​obtained; based on the camera intrinsic parameter matrix, the linear calibration of the camera intrinsic parameters is realized.

[0034] The method for obtaining the camera intrinsic parameter matrix is ​​specifically as follows. Since the methods for obtaining the camera intrinsic parameter matrices of the first camera and the second camera are similar, in order to save space, the term "camera" is still used to replace the first camera or the second camera.

[0035] First, the distance between the calibration plate and the camera is changed by moving the calibration plate, and dedistorted images of the calibration plate at different distances from the camera are obtained to obtain several calibration images.

[0036] Then, based on the pixel length and width of several calibration images, the calibration plate is moved to obtain the horizontal focal length and vertical focal length of the camera.

[0037] The operations for obtaining the horizontal focal length and the vertical focal length of the camera are specifically as follows: the camera acquires an image when the calibration plate is at a first position to obtain a first calibration image; the pixel width and length of the first calibration image are acquired to obtain a first pixel width and length; the camera acquires an image when the calibration plate is at a second position to obtain a second calibration image; the pixel width and length of the second calibration image are acquired to obtain a second pixel width and length; based on the first pixel width, the second pixel width and the movement distance of the calibration plate, a target-to-camera distance parameter is acquired; based on the target-to-camera distance parameter, the first pixel width and length (or the second pixel width and length), the calibration plate length and the calibration plate width, the horizontal focal length and the vertical focal length of the camera are acquired.

[0038] The above operation of obtaining the distance parameter between the target and the camera can be achieved through the following formula: , is the distance parameter between the target and the camera, is the moving distance of the calibration plate, , They are the first pixel width and the second pixel width respectively.

[0039] The above operation of obtaining the horizontal focal length and vertical focal length of the camera can be achieved by the following formula: , , , are the horizontal focal length and vertical focal length of the camera respectively. is the first pixel width or the second pixel width, is the first pixel length or the second pixel length, , are the length and width of the calibration plate respectively.

[0040] Finally, the camera intrinsic parameter matrix is ​​obtained based on the camera horizontal focal length and vertical focal length.

[0041] Camera intrinsic matrix The expression is as follows: , , are the horizontal and vertical coordinates of the distortion center point respectively.

[0042] S3. Construct a binocular extrinsic parameter objective function based on the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix; solve the binocular extrinsic parameter objective function using the nonlinear least squares method to obtain the binocular rotation matrix and the binocular translation vector; based on the extrinsic parameter transformation formula between the first camera and the second camera, decompose the binocular rotation matrix and the binocular translation vector respectively to obtain the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector; perform extrinsic parameter calibration on the first camera and the second camera based on the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector.

[0043] After obtaining the intrinsic parameter matrix of the first camera and the intrinsic parameter matrix of the second camera in the binocular camera, the binocular extrinsic parameter objective function is constructed, and the objective function is solved by the nonlinear least squares method to obtain the binocular rotation matrix and the binocular translation vector. After decomposition, the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector are obtained to obtain the orientation relationship between the first camera and the second camera.

[0044] First, the left camera and the right camera (ie, the first camera and the second camera) in the binocular camera respectively perform the operation of S2 to obtain the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix.

[0045] Then, based on the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix, a binocular extrinsic parameter objective function is constructed; the nonlinear least squares method is used to solve the binocular extrinsic parameter objective function to obtain the binocular rotation matrix and the binocular translation vector.

[0046] The above binocular external parameter objective function is as follows: , is the binocular extrinsic parameter objective function value, The first camera view j The transpose of the homogeneous coordinate matrix of the points, For the second camera view j The transpose of the homogeneous coordinate matrix of the points, is the inverse matrix of the transposed intrinsic parameter matrix of the first camera, is the inverse matrix of the second camera intrinsic parameter matrix, is the binocular rotation matrix, is the binocular translation vector, J is the total number of points.

[0047] Next, based on the external parameter transformation formula between the first camera and the second camera, the binocular rotation matrix and the binocular translation vector are decomposed (which can be achieved by the singular value decomposition method) to obtain the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector.

[0048] The external parameter transformation formula between the first camera and the second camera is as follows: , , , t They are binocular rotation matrix and binocular translation vector respectively. , , , They are the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector, and the second camera translation vector respectively.

[0049] Finally, the camera is calibrated externally based on the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector.

[0050] S4. Based on the binocular translation vector, obtain a camera alignment matrix; multiply the camera alignment matrix with the first camera rotation matrix and the second camera rotation matrix respectively to obtain a first correction rotation matrix and a second correction rotation matrix; based on the first correction rotation matrix and the second correction rotation matrix, calibrate the first camera and the second camera respectively to make the optical axes of the first camera and the second camera parallel.

[0051] The camera alignment matrix is ​​calculated according to the binocular translation vector, and the first correction rotation matrix and the second correction rotation matrix are obtained to implement stereo correction of the first camera and the second camera so that the optical axes of the first camera and the second camera are parallel.

[0052] First, based on the binocular translation vector, obtain the camera alignment matrix. Specifically, the ratio of the binocular translation vector to the binocular translation vector modulus is used as the first alignment sub-matrix; based on the binocular translation vector x Axis translation vector and y The axis translation vector is used to obtain the second alignment sub-matrix; the product of the first alignment sub-matrix and the second alignment sub-matrix is ​​used as the third alignment sub-matrix; the first alignment sub-matrix, the second alignment sub-matrix and the third alignment sub-matrix form a camera alignment matrix.

[0053] Then, the camera alignment matrix is ​​multiplied by the first camera rotation matrix and the second camera rotation matrix respectively to obtain a first correction rotation matrix and a second correction rotation matrix.

[0054] Next, based on the first correction rotation matrix and the second correction rotation matrix, the first camera and the second camera are corrected respectively, so that the optical axes of the first camera and the second camera are parallel, coplanar, and aligned in rows.

[0055] Finally, the left and right cameras (the first camera and the second camera) can achieve binocular correction by adjusting their respective overall rotation matrices. When the camera acquires images in large-size industrial parts scanning scenarios, an ideal stereo image set can be obtained, which can improve the accuracy of subsequent three-dimensional data reconstruction.

[0056] This embodiment further provides a binocular stereoscopic vision correction system, which is used to implement the above binocular stereoscopic vision correction method, including: The distortion parameter generation and dedistortion module is used for the first camera and the second camera to respectively obtain the calibration plate images to obtain the respective distorted images containing deformation curves; based on the data of each deformation curve pair and the corresponding straight line pair data before deformation in the respective distorted images, the distortion feature points of each deformation curve pair are obtained; the average of the positions of all the distortion feature points is used as the respective distortion center points; based on the first and last coordinates of each deformation curve in the respective distorted images, the corresponding distortion evaluation lines are obtained; based on the sum of the areas formed by all the deformation curves and the corresponding distortion evaluation lines, the respective distortion degree objective functions are constructed; the respective distortion degree objective functions are processed by the gradient descent method to obtain the respective distortion parameters; based on the respective distortion center points and the respective distortion parameters, the images obtained by the first camera and the second camera are respectively subjected to distortion correction; The camera intrinsic parameter matrix generation and intrinsic parameter calibration module is used to obtain dedistorted images of calibration plates at different distances from the first camera and the second camera, respectively, to obtain a plurality of calibration images of each camera; based on the pixel length and width of the respective calibration images and the movement distance of the calibration plate, to obtain the horizontal focal length and vertical focal length of each camera; based on the horizontal focal length and vertical focal length of each camera, to obtain the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix, and to perform intrinsic parameter calibration on the first camera and the second camera respectively; The extrinsic parameter data generation and extrinsic parameter calibration module is used for the first camera and the second camera to respectively perform the operations of the camera intrinsic parameter matrix generation and the intrinsic parameter calibration module to obtain the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix; based on the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix, a binocular extrinsic parameter objective function is constructed; the binocular extrinsic parameter objective function is solved by using the nonlinear least square method to obtain the binocular rotation matrix and the binocular translation vector; based on the extrinsic parameter transformation formula between the first camera and the second camera, the binocular rotation matrix and the binocular translation vector are respectively decomposed and processed to obtain the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector; based on the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector, the first camera and the second camera are extrinsic parameter calibrated; The rotation matrix generation and camera correction module is used to obtain the camera alignment matrix based on the binocular translation vector; the camera alignment matrix is ​​multiplied with the first camera rotation matrix and the second camera rotation matrix respectively to obtain the first correction rotation matrix and the second correction rotation matrix; based on the first correction rotation matrix and the second correction rotation matrix, the first camera and the second camera are corrected respectively to make the optical axes of the first camera and the second camera parallel.

[0057] This embodiment also provides a binocular stereoscopic vision correction device, including a processor and a memory, wherein the processor implements the above-mentioned binocular stereoscopic vision correction method when executing a computer program stored in the memory.

[0058] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program implements the above-mentioned binocular stereoscopic vision correction method when executed by a processor.

[0059] The present embodiment provides a binocular stereo vision correction method, firstly, based on the obtained deformation curve pair data and the corresponding undeformed straight line pair data on the calibration plate, the distortion center point is obtained, and based on the sum of the areas formed by all the deformation curves and the corresponding distortion evaluation lines, a distortion degree objective function is constructed, based on the distortion degree objective function, the distortion parameters are obtained, and combined with the distortion center point, the distortion correction of the image acquired by the binocular camera is realized; then, according to the dedistorted image of the calibration plate at different distances from the camera, the horizontal focal length and the vertical focal length of the camera are obtained, and based on the horizontal focal length and the vertical focal length of the camera, the camera intrinsic parameter matrix is ​​obtained; based on the camera intrinsic parameter matrix, the linear calibration of the binocular camera intrinsic parameters is realized; then, after obtaining the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix in the binocular camera, the distortion degree objective function is constructed. A binocular extrinsic parameter objective function is established, and the nonlinear least squares method is used to solve the objective function to obtain the binocular rotation matrix and the binocular translation vector. After decomposition, the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector are obtained, and the orientation relationship between the first camera and the second camera is obtained to realize the extrinsic parameter calibration of the binocular camera; finally, the camera alignment matrix is ​​calculated according to the binocular translation vector, and the first correction rotation matrix and the second correction rotation matrix are obtained to realize the stereo correction of the first camera and the second camera, so that the optical axes of the first camera and the second camera are parallel, and high-precision binocular stereo vision correction is realized. When the camera acquires images in large-size industrial parts scanning scenarios, an ideal stereo image set can be obtained, which can improve the accuracy of subsequent three-dimensional data reconstruction.

Claims

1. A binocular stereoscopic vision correction method, characterized in that: The following operations are included: S1, the first camera and the second camera respectively acquire the calibration plate image to obtain the distorted image containing the deformation curve; based on the data of each deformation curve pair and the corresponding straight line pair data before deformation in each distorted image, obtain the distortion feature point of each deformation curve pair; take the average value of the positions of all the distortion feature points as the distortion center point; Based on the first and last coordinates of each deformation curve in the respective distorted images, the corresponding distortion evaluation lines are obtained; based on the sum of the areas formed by all the deformation curves and the corresponding distortion evaluation lines, the respective distortion degree objective functions are constructed; the respective distortion degree objective functions are processed by the gradient descent method to obtain the respective distortion parameters; Based on respective distortion center points and respective distortion parameters, respectively performing distortion correction on images acquired by the first camera and the second camera; S2, obtaining dedistorted images of the calibration plate at different distances from the first camera and the second camera, respectively, to obtain a plurality of calibration images of each; based on the pixel length and width of the plurality of calibration images and the movement distance of the calibration plate, obtaining the horizontal focal length and vertical focal length of each camera; based on the horizontal focal length and vertical focal length of each camera, obtaining the first camera internal parameter matrix and the second camera internal parameter matrix, and performing internal parameter calibration on the first camera and the second camera respectively; S3. Based on the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix, a binocular extrinsic parameter objective function is constructed; the binocular extrinsic parameter objective function is solved by a nonlinear least square method to obtain a binocular rotation matrix and a binocular translation vector; based on the extrinsic parameter transformation formula between the first camera and the second camera, the binocular rotation matrix and the binocular translation vector are respectively decomposed to obtain a first camera rotation matrix, a second camera rotation matrix, a first camera translation vector and a second camera translation vector; based on the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector, the first camera and the second camera are extrinsically calibrated; S4. Based on the binocular translation vector, obtain a camera alignment matrix; multiply the camera alignment matrix with the first camera rotation matrix and the second camera rotation matrix respectively to obtain a first correction rotation matrix and a second correction rotation matrix; based on the first correction rotation matrix and the second correction rotation matrix, calibrate the first camera and the second camera respectively to make the optical axes of the first camera and the second camera parallel.

2. The binocular stereoscopic vision correction method according to claim 1, characterized in that: In S1, the method for obtaining the distortion feature points of the current deformation curve pair is specifically as follows: The central symmetric points of the first curve and the second curve in the current deformed curve pair are respectively obtained to obtain the first central symmetric point and the second central symmetric point; perpendicular lines corresponding to the first straight line and the second straight line of the current straight line pair before deformation are respectively drawn through the first central symmetric point and the second central symmetric point to obtain the first perpendicular line and the second perpendicular line; the intersection position of the first perpendicular line and the second perpendicular line is used as the distortion feature point of the current deformed curve pair.

3. The binocular stereoscopic vision correction method according to claim 1, characterized in that: In S1, the distortion degree objective function is as follows: ， is the distortion value, m For the m A deformation curve, M is the total number of deformation curves, For the m The deformation curve n- 1 curve points to the distortion evaluation line I The vertical distance on For the m The deformation curve n curve points to the distortion evaluation line I The vertical distance on For the m The deformation curve n-1 The curve point and n The connection between the curve points is projected onto the distortion evaluation line I The distance on N For the m The total number of curve points on the deformation curve; The vertical distance from the curve point to the corresponding distortion evaluation line is obtained based on the coordinates of the curve point; the distortion parameter is obtained based on the coordinates of the curve point and the coordinates of the corresponding point on the line before deformation.

4. The binocular stereoscopic vision correction method according to claim 1, characterized in that: In S2, the operation of obtaining the horizontal focal length and the vertical focal length of the camera is specifically as follows: The camera acquires an image of the calibration plate at the first position to obtain a first calibration image; the pixel width and length of the first calibration image are acquired to obtain a first pixel width and length; The camera acquires an image of the calibration plate at the second position to obtain a second calibration image; and acquires a pixel width and length of the second calibration image to obtain a second pixel width and length. Based on the first pixel width, the second pixel width and the movement distance of the calibration plate, a distance parameter between the target and the camera is obtained; The horizontal focal length and vertical focal length of the camera are obtained based on the distance parameters between the target and the camera, the first pixel width and length, the calibration plate length and the calibration plate width.

5. The binocular stereoscopic vision correction method according to claim 1, characterized in that: In S3, the binocular external parameter objective function is as follows: ， is the binocular extrinsic parameter objective function value, The first camera view j The transpose of the homogeneous coordinate matrix of the points, For the second camera view j The transpose of the homogeneous coordinate matrix of the points, is the inverse matrix of the transposed intrinsic parameter matrix of the first camera, is the inverse matrix of the second camera intrinsic parameter matrix, is the binocular rotation matrix, is the binocular translation vector, J is the total number of points.

6. The binocular stereoscopic vision correction method according to claim 1, characterized in that: In S3, the external parameter transformation formula between the first camera and the second camera is as follows: ， ， , t They are binocular rotation matrix and binocular translation vector respectively. , , , They are the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector, and the second camera translation vector respectively.

7. The binocular stereoscopic vision correction method according to claim 1, characterized in that: In S4, the operation of obtaining the camera alignment matrix is ​​specifically as follows: The ratio of the binocular translation vector to the binocular translation vector modulus is used as the first alignment sub-matrix; Based on the x-axis translation vector and the y-axis translation vector in the binocular translation vector, a second alignment submatrix is ​​obtained; The product of the first alignment submatrix and the second alignment submatrix is ​​used as the third alignment submatrix; The first alignment sub-matrix, the second alignment sub-matrix and the third alignment sub-matrix form a camera alignment matrix.

8. A binocular stereoscopic vision correction system, used to implement the binocular stereoscopic vision correction method according to claim 1, characterized in that: include: The distortion parameter generation and dedistortion module is used for the first camera and the second camera to respectively acquire the calibration plate images to obtain respective distorted images containing deformation curves; Based on the data of each deformation curve pair and the corresponding straight line pair before deformation in the respective distorted images, the distortion feature points of each deformation curve pair are obtained; the average of the positions of all the distortion feature points is used as the respective distortion center points; based on the first and last coordinates of each deformation curve in the respective distorted images, the corresponding distortion evaluation lines are obtained; based on the sum of the areas formed by all the deformation curves and the corresponding distortion evaluation lines, the respective distortion degree objective functions are constructed; the respective distortion degree objective functions are processed by the gradient descent method until convergence to obtain the respective distortion parameters; based on the respective distortion center points and the respective distortion parameters, the images obtained by the first camera and the second camera are respectively subjected to distortion correction; The camera intrinsic parameter matrix generation and intrinsic parameter calibration module is used to obtain dedistorted images of calibration plates at different distances from the first camera and the second camera, respectively, to obtain a plurality of calibration images of each camera; based on the pixel length and width of the respective calibration images and the movement distance of the calibration plate, to obtain the horizontal focal length and vertical focal length of each camera; based on the horizontal focal length and vertical focal length of each camera, to obtain the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix, and to perform intrinsic parameter calibration on the first camera and the second camera respectively; The extrinsic parameter data generation and extrinsic parameter calibration module is used for the first camera and the second camera to respectively perform the operations of the camera intrinsic parameter matrix generation and the intrinsic parameter calibration module to obtain the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix; based on the first camera intrinsic parameter matrix and the second camera intrinsic parameter matrix, a binocular extrinsic parameter objective function is constructed; the binocular extrinsic parameter objective function is solved by using the nonlinear least square method to obtain the binocular rotation matrix and the binocular translation vector; based on the extrinsic parameter transformation formula between the first camera and the second camera, the binocular rotation matrix and the binocular translation vector are respectively decomposed and processed to obtain the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector; based on the first camera rotation matrix, the second camera rotation matrix, the first camera translation vector and the second camera translation vector, the first camera and the second camera are extrinsic parameter calibrated; The rotation matrix generation and camera correction module is used to obtain the camera alignment matrix based on the binocular translation vector; the camera alignment matrix is ​​multiplied with the first camera rotation matrix and the second camera rotation matrix respectively to obtain the first correction rotation matrix and the second correction rotation matrix; based on the first correction rotation matrix and the second correction rotation matrix, the first camera and the second camera are corrected respectively to make the optical axes of the first camera and the second camera parallel.

9. A binocular stereoscopic vision correction device, characterized in that: It comprises a processor and a memory, wherein when the processor executes the computer program stored in the memory, it implements a binocular stereoscopic vision correction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, a binocular stereoscopic vision correction method as described in any one of claims 1 to 7 is implemented.

Images