Binocular camera parameter calibration method, device and equipment and storage medium

Abstract

The embodiment of the invention provides a binocular camera parameter calibration method and device, equipment and a storage medium, and relates to the technical field of image processing. The specific implementation mode is as follows: acquiring a first image and a second image shot by a binocular camera for a reference object; performing parameter calibration processing on the basis of a parameter group obtained through calibration in the initial mode to obtain an optimized calibration result corresponding to the initial mode; and determining a camera parameter calibration result of the binocular camera based on the obtained optimized calibration result. By applying the scheme provided by the embodiment of the invention, the calibration accuracy of the camera parameters of the camera can be effectively improved.

Description

Technical Field

[0001] The present application relates to the technical field of image processing, and particularly to a method, apparatus, device, and storage medium for calibrating parameters of a binocular camera. Background Art

[0002] A binocular camera is a stereo vision system composed of two cameras. The binocular camera can capture images of two perspectives of the same scene through the two cameras, and then, through image processing technology, stitch the two images together to create an image with a wider viewing range. Before stitching the two images, it is necessary to perform distortion correction on each image, and the distortion correction effect of the image will affect the stitching effect of the image.

[0003] The distortion correction of an image depends on the internal parameters and distortion coefficients of the camera. The more accurate the calibration of the internal parameters and distortion coefficients of the camera, the better the distortion correction effect of the image. Usually, after each binocular camera is produced, workers manually calibrate the camera parameters. However, due to factors such as improper operation methods of workers during manual calibration, the calibration of camera parameters is inaccurate.

[0004] Therefore, how to improve the calibration accuracy of camera parameters has become an urgent problem to be solved. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a method, apparatus, device, and storage medium for calibrating parameters of a binocular camera to improve the calibration accuracy of camera parameters. The specific technical solutions are as follows:

[0006] In the first aspect of the present application, a method for calibrating parameters of a binocular camera is provided. The method includes:

[0007] Obtain a first image and a second image captured by the binocular camera for a reference object; wherein, each camera of the binocular camera has a parameter set including camera internal parameters and distortion coefficients calibrated by an initial method;

[0008] Based on the parameter set calibrated by the initial method, perform parameter calibration processing to obtain an optimized calibration result corresponding to the initial method;

[0009] Based on the obtained optimized calibration result, determine the calibration result of the camera parameters of the binocular camera;

[0010] Wherein, the parameter calibration processing includes:

[0011] Based on the current parameter set of each camera, perform distortion correction on the captured first image and second image; and use the corrected first image and the corrected second image to calculate the external parameters of the binocular camera as the current external parameters;

[0012] Calculate an error value based on the current external parameters; wherein, the error value is used to characterize the degree of difference between the first position and the second position of a specified feature point of the reference object, the first position is the position obtained by projecting the specified feature point in any one of the corrected first image and the corrected second image onto the other image according to the current external parameters, and the second position is the position of the specified feature point in the other image;

[0013] In response to the current error value being less than a preset threshold, use the current parameter group and the current external parameters as the optimized calibration result corresponding to the initial method; otherwise, adjust the camera internal parameters in the reference parameter group of each camera to obtain the current parameter group of each camera, and return to perform the step of distortion correction on the first image and the second image obtained by shooting based on the current parameter group of each camera; the determination method of the reference parameter group of each camera includes determining the parameter group calibrated by the initial method as the initial reference parameter group.

[0014] Optionally, the adjusting the camera internal parameters in the reference parameter group of each camera to obtain the current parameter group of each camera includes:

[0015] Select a target coefficient to be utilized, and adjust the camera internal parameters in the reference parameter group of each camera based on the current target step size and the target coefficient; wherein, the target step size is a degree value used to characterize the allowable adjustment degree, and the target coefficient is used to characterize the influence degree on the size and direction of the target step size during parameter adjustment. Optionally, the number of the initial methods is multiple;

[0016] Performing parameter calibration processing on the parameter group calibrated based on the initial method to obtain the optimized calibration result corresponding to the initial method includes:

[0017] For each initial method, perform parameter calibration processing on the parameter group calibrated by the initial method to obtain the optimized calibration result corresponding to the initial method;

[0018] The determining the camera parameter calibration result of the binocular camera based on the obtained optimized calibration result includes:

[0019] Select, from the obtained optimized calibration results, the optimized calibration result corresponding to the initial method with the smallest corresponding target loss value as the camera parameter calibration result of the binocular camera;

[0020] Wherein, the target loss value corresponding to any initial method is the last loss value obtained by performing parameter calibration processing on the parameter group calibrated by the initial method;

[0021] Among them, the multiple initial methods include:

[0022] An initial method that is for any camera under the target camera model and belongs to laboratory calibration, and an initial method that is for the binocular camera itself and belongs to factory line calibration;

[0023] Among them, the target camera model is the camera model to which the binocular camera belongs.

[0024] Optionally, the parameter calibration process further includes:

[0025] In response to meeting the step size update condition, perform a reduction process on the current target step size to obtain a new current target step size; where meeting the step size update condition includes: a first condition indicating that the number of calculations for the loss value reaches a predetermined number or a second condition indicating that the loss value calculated most recently is less than a preset intermediate threshold; where the intermediate threshold is greater than the preset threshold;

[0026] The method for determining the reference parameter group of each camera further includes:

[0027] In response to meeting the step size update condition, determine the reference loss value among the calculated loss values, determine the current parameter group existing in the calculation process of the reference loss value to obtain a backup parameter group, and determine the target coefficient based on which the camera internal parameters in the backup parameter group are updated; where the method for determining the reference loss value includes: when the step size update condition includes the first condition, determine the smallest loss value obtained during the predetermined number of calculations for the loss value as the reference loss value, or when the step size update condition includes the second condition, determine the loss value calculated most recently as the reference loss value;

[0028] Based on the determined target coefficient and the current target step size before performing the reduction process, adjust the current reference parameter group of each camera as the new reference parameter group.

[0029] Optionally, the adjusting the camera internal parameters in the reference parameter group of each camera based on the current target step size and the target coefficient includes:

[0030] Based on the current target step size and multiple target coefficients, adjust the first parameter and the second parameter in the camera internal parameters of the reference parameter group of each camera.

[0031] Each target coefficient is used to adjust one of the first parameter and the second parameter, and different target coefficients are used to adjust different parameters. The first parameter represents the pixel size of the focal length in the horizontal direction, and the second parameter is used to represent the pixel size of the focal length in the vertical direction.

[0032] Optionally, the number of the reference objects is multiple and the reference objects are preset pictures, and the specified feature points are the corner points of the preset pictures;

[0033] Performing distortion correction on the first image and the second image obtained by shooting based on the current parameter group of each camera includes:

[0034] Determining each corner point in the first image obtained by shooting, and performing distortion correction on each determined corner point according to the current parameter group of the camera used for shooting the first image to obtain a corrected first image;

[0035] Determining each corner point in the second image obtained by shooting, and performing distortion correction on each determined corner point according to the current parameter group of the camera used for shooting the second image to obtain a corrected second image;

[0036] Calculating the external parameters of the binocular camera by using the corrected first image and the corrected second image as the current external parameters includes:

[0037] Determining each corner point in the corrected first image to obtain a first corner point set, and determining each corner point in the corrected second image to obtain a second corner point set;

[0038] Calculating the external parameters of the binocular camera based on the positions of the corner points in the first corner point set and the positions of the matching corner points in the second corner point set.

[0039] Optionally, the selection of the target coefficient to be used includes:

[0040] Selecting the coefficient to be used from the set coefficient value range according to the set selection method as the target coefficient.

[0041] In the second aspect of the present application, a parameter calibration device for a binocular camera is provided, and the device includes:

[0042] An acquisition module, configured to acquire a first image and a second image obtained by a binocular camera shooting a reference object; wherein, each camera of the binocular camera has a parameter group obtained by initial calibration and including camera internal parameters and distortion coefficients;

[0043] A calibration module, configured to perform parameter calibration processing based on the parameter group obtained by the initial calibration to obtain an optimized calibration result corresponding to the initial method;

[0044] A determination module, configured to determine the camera parameter calibration result of the binocular camera based on the obtained optimized calibration result;

[0045] Wherein, the parameter calibration processing includes:

[0046] Based on the current parameter set of each camera, perform distortion correction on the first image and the second image obtained by shooting; and, use the corrected first image and the corrected second image to calculate the external parameters of the binocular camera as the current external parameters;

[0047] Based on the current external parameters, calculate an error value; wherein, the error value is used to characterize the degree of difference between the first position and the second position of the specified feature point of the reference object, the first position is the position obtained by projecting the position of the specified feature point in any one of the corrected first image and the corrected second image onto the other image according to the current external parameters, and the second position is the position of the specified feature point in the other image;

[0048] In response to the current error value being less than a preset threshold, use the current parameter set and the current external parameters as the optimized calibration result corresponding to the initial method; otherwise, adjust the camera internal parameters in the reference parameter set of each camera to obtain the current parameter set of each camera, and return to the step of performing distortion correction on the first image and the second image obtained by shooting based on the current parameter set of each camera; the determination method of the reference parameter set of each camera includes determining the parameter set calibrated by the initial method as the initial reference parameter set.

[0049] Optionally, adjusting the camera internal parameters in the reference parameter set of each camera to obtain the current parameter set of each camera includes:

[0050] Select a target coefficient to be utilized, and adjust the camera internal parameters in the reference parameter set of each camera based on the current target step size and the target coefficient; wherein, the target step size is a degree value used to characterize the allowable adjustment degree, and the target coefficient is used to characterize the influence degree on the size and direction of the target step size during parameter adjustment.

[0051] Optionally, the number of the initial methods is multiple; the calibration module includes:

[0052] A calibration unit for performing parameter calibration processing on the parameter set calibrated by each initial method to obtain the optimized calibration result corresponding to the initial method;

[0053] The determination module includes:

[0054] A selection unit for selecting, from the obtained optimized calibration results, the optimized calibration result corresponding to the initial method with the smallest corresponding target loss value as the camera parameter calibration result of the binocular camera;

[0055] Among them, the target loss value corresponding to any initial method is the last loss value obtained by performing parameter calibration processing on the parameter group calibrated based on this initial method;

[0056] Among them, the multiple initial methods include:

[0057] The initial method that belongs to laboratory calibration for any camera under the target camera model, and the initial method that belongs to factory line calibration for the binocular camera itself;

[0058] Among them, the target camera model is the camera model to which the binocular camera belongs.

[0059] Optionally, the parameter calibration processing further includes:

[0060] In response to meeting the step size update condition, perform a reduction process on the current target step size to obtain a new current target step size; where meeting the step size update condition includes: the first condition indicating that the number of calculations for the loss value reaches a predetermined number or the second condition indicating that the loss value calculated most recently is less than a preset intermediate threshold; where the intermediate threshold is greater than the preset threshold;

[0061] The determination method of the reference parameter group for each camera further includes:

[0062] In response to meeting the step size update condition, determine the reference loss value among the calculated loss values, determine the current parameter group existing in the calculation process of the reference loss value to obtain a backup parameter group, and determine the target coefficient based on which the camera internal parameters in the backup parameter group are updated; where the determination method of the reference loss value includes: when the step size update condition includes the first condition, determine the smallest loss value obtained during the calculation of the predetermined number of times for the loss value as the reference loss value, or when the step size update condition includes the second condition, determine the loss value calculated most recently as the reference loss value;

[0063] Based on the determined target coefficient and the current target step size before performing the reduction process, adjust the current reference parameter group for each camera as the new reference parameter group.

[0064] The adjustment of the camera internal parameters in the reference parameter group for each camera based on the current target step size and the target coefficient includes:

[0065] Based on the current target step size and multiple target coefficients, adjust the first parameter and the second parameter in the camera internal parameters of the reference parameter group for each camera.

[0066] Each target coefficient is used to adjust one of the first parameter and the second parameter. Different target coefficients are used to adjust different parameters. The first parameter represents the pixel size of the focal length in the horizontal direction, and the second parameter is used to represent the pixel size of the focal length in the vertical direction. Optionally, the number of the reference objects is multiple and the reference objects are preset pictures, and the specified feature points are the corner points of the preset pictures;

[0067] Performing distortion correction on the first image and the second image obtained by shooting based on the current parameter group of each camera includes:

[0068] Determine each corner point in the first image obtained by shooting, and perform distortion correction on each determined corner point according to the current parameter group of the camera used to shoot the first image to obtain the corrected first image;

[0069] Determine each corner point in the second image obtained by shooting, and perform distortion correction on each determined corner point according to the current parameter group of the camera used to shoot the second image to obtain the corrected second image;

[0070] Calculating the external parameters of the binocular camera by using the corrected first image and the corrected second image as the current external parameters includes:

[0071] Determine each corner point in the corrected first image to obtain a first corner point set, and determine each corner point in the corrected second image to obtain a second corner point set;

[0072] Calculate the external parameters of the binocular camera based on the positions of the corner points in the first corner point set and the positions of the matching corner points in the second corner point set.

[0073] Optionally, the selection of the target coefficient to be used includes:

[0074] Select a coefficient to be used from the set coefficient value range according to the set selection method as the target coefficient.

[0075] In the third aspect of the present application, an electronic device is provided, including:

[0076] A memory for storing a computer program;

[0077] A processor for implementing the parameter calibration method of the binocular camera as described in any one of the above when executing the program stored in the memory.

[0078] In the fourth aspect of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the parameter calibration method of the binocular camera as described in any one of the above is implemented.

[0079] Advantages of the embodiments of the present application:

[0080] The solution of the present application can perform distortion correction on the first image and the second image captured by the binocular camera for the reference object through the current parameter group of the binocular camera, calculate the current external parameters of the binocular camera by using the corrected first image and the corrected second image, calculate the error value based on the current external parameters. When the error value is not less than the preset threshold, adjust the camera internal parameters in the reference parameter group of each camera to obtain the current parameter group of each camera, and re-perform distortion correction on the first image and the second image captured by the binocular camera for the reference object based on the current parameter group, calculate the current external parameters of the binocular camera by using the corrected first image and the corrected second image, calculate the error value based on the current external parameters, until the calculated error value is less than the preset threshold, then use the current parameter group and the current external parameters as the optimized calibration result corresponding to the initial method. Therefore, the solution of the present application can continuously iterate and update the parameter group and external parameters of each camera, and calculate the loss value of the binocular camera based on the updated parameter group and external parameters of each camera, so as to determine relatively accurate camera parameters based on the loss value. It can be seen that the solution of the present application can improve the calibration accuracy of the camera parameters of the camera.

[0081] Of course, it is not necessary for any product or method implementing the present application to achieve all the above advantages simultaneously. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other embodiments can also be obtained based on these drawings.

[0083] Figure 1 It is a schematic flowchart of a method for calibrating parameters of a binocular camera provided by an embodiment of the present application;

[0084] Figure 2a It is a picture taken by one camera of an exemplary binocular camera provided by the present application;

[0085] Figure 2b It is a picture taken by the other camera of an exemplary binocular camera provided by the present application

[0086] Figure 3a It is a picture obtained by splicing the parameter group and external parameters calibrated by any initial method provided by an embodiment of the present application;

[0087] Figure 3bA picture obtained by stitching using the parameter set determined by the parameter calibration method of the binocular camera of the present application;

[0088] Figure 4 A schematic diagram provided by an embodiment of the present application for determining each corner point in the first image and each matching corner point in the second image;

[0089] Figure 5 A schematic structural diagram of a parameter calibration device for a binocular camera provided by an embodiment of the present application;

[0090] Figure 6 A schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0091] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art based on the present application belong to the scope of protection of the present application.

[0092] Next, the professional terms involved in the present application will be explained.

[0093] Camera internal parameters: A series of parameters describing the projection attributes of the camera, usually including the focal length (f), the optical axis point (coordinate of the principal point, usually represented by (cx, cy)), and the pixel scale factor (s), etc. These parameters are mainly used to map three-dimensional world points to two-dimensional pixel coordinates, or vice versa, to resolve two-dimensional pixel coordinates back to three-dimensional world points. Among them, the camera internal parameters are a 3*3 matrix.

[0094] Distortion coefficient: A parameter describing the distortion of the camera lens.

[0095] Next, the camera internal parameters and the distortion coefficient will be specifically introduced.

[0096] The camera internal parameter matrix M, that is, the camera internal parameters, can be characterized as:

[0097]

[0098] In the camera intrinsic matrix M, f is the focal length of the camera, dX and dY are the physical lengths of the first pixel in the X and Y directions on the camera sensor plate respectively. Exemplarily, the camera sensor plate can be CMOS (Complementary Metal - Oxide - Semiconductor Sensor, a kind of sensor), CCD (Charge - Coupled Device Sensor), etc. Assume the size of the camera sensor plate is: width 5376μm, height 3024μm, and the final imaging size is 1280 * 720. The width of each pixel unit in each physical imaging plane is 5376 / 1280, that is 4.2 pixels / μm, and the height is 3024 / 720, that is 4.2 pixels / μm.

[0099] c x and c y are the coordinates of the camera principal point, that is, the coordinates of the center of the camera sensor plate in the pixel coordinate system. θ represents the angle between the horizontal side and the vertical side of the camera sensor plate (90° means no error). Therefore, the camera intrinsic matrix M can also be expressed in the following form:

[0100]

[0101] where, fx is the pixel size of the focal length in the horizontal direction, fy is the pixel size of the focal length in the vertical direction, cx is the abscissa of the principal point (i.e., the abscissa of the image center), and cy is the ordinate of the principal point (i.e., the ordinate of the image center).

[0102] The distortion coefficient D can be characterized as:

[0103] D = [k1 k2 p1 p2 k3];

[0104] where, k1, k2, and k3 are radial distortion parameters. The physical meaning is the expansion / contraction from the center to the edge. Radial distortion can be regarded as a change in the coordinate point along the length direction, that is, the points farther from the principal point have a change in the distance length. k1: the first - order radial distortion coefficient, usually mainly corrects the barrel distortion or pincushion distortion in the central area of the image. k2: the second - order radial distortion coefficient, usually corrects the distortion in a larger radius area. k3: the third - order radial distortion coefficient, usually used for fish - eye lenses or wide - angle lenses.

[0105] p1 and p2 are tangential distortion parameters. The physical meaning of tangential distortion is the shear deformation caused by the tilt of the lens and the sensor. p1 is the horizontal tangential distortion; p2 is the vertical tangential distortion.

[0106] Specifically, the distortion of the camera is corrected by the camera intrinsic matrix and the distortion coefficient, and the camera intrinsic matrix M and the distortion coefficient D are the results of camera calibration.

[0107] To understand the meaning of the distortion coefficient, let any point P(x, y) on the pixel coordinate system have corresponding polar coordinates (r, θ2), where r is the distance from point P to the origin of the coordinate system, and θ2 represents the angle with the horizontal axis. Usually, the relationship of distortion is fitted by a polynomial, and k1, k2, k3, p1, and p2 are the polynomial coefficients for fitting the distortion. This polynomial is artificially designed.

[0108] The formula for correcting radial distortion is:

[0109] x 1distorted = x(1 + k1r 2 + k2r 4 + k3r 6 );

[0110] y 1distorted = y(1 + k1r 2 + k2r 4 + k3r 6 );

[0111] Among them, x 1distorted and y 1distorted are the values of the abscissa and ordinate formed after radial distortion occurs to point P.

[0112] Tangential distortion can be understood as the coordinate point changing along the tangent direction, that is, the horizontal angle θ2 changes. The formula for correcting tangential distortion is:

[0113] x 2distorted = x + 2p1xy + p2(r 2 + 2x 2 );

[0114] x 2distorted = y + p1(r 2 + 2y 2 )+ 2p2xy;

[0115] Among them, x 2distorted and y 2distorted are the values of the abscissa and ordinate formed after tangential distortion occurs to point P.

[0116] The formulas for correcting radial distortion and correcting tangential distortion are both for points in the camera coordinate system. Therefore, for any point (x distorted , y distorted ) in the camera coordinate system, the correct position (x, y) on the corresponding pixel plane (i.e., the pixel coordinate system) can be found through 5 distortion coefficients (k1 k2 p1 p2 k3). Among them, the correction formula can be expressed as:

[0117] x distorted = x(1 + k1r2 +k2r 4 +k3r 6 )+2p1xy+p2(r 2 +2x 2 );

[0118] y distorted =y(1+k1r 2 +k2r 4 +k3r 6 )+p1(r 2 +2y 2 )+2p2xy。

[0119] External camera parameters: Parameters that describe the position and orientation of the camera in the world coordinate system. In the stitching algorithm, the homography matrix can be used as the external parameters of the binocular camera.

[0120] Homography matrix: Also known as the homogeneous transformation matrix or projection matrix, it is a linear mapping in two-dimensional or three-dimensional space and is usually used to describe the projective transformation between two two-dimensional planes, such as the geometric transformation from one image to another. In principle, this transformation can describe any projective transformation between planes.

[0121] Image undistortion: That is, image correction, which is the process of eliminating the image distortion caused by the camera lens distortion.

[0122] In the technical solution of this application, operations such as the acquisition, storage, use, processing, transmission, provision, and disclosure of the user's personal information are all carried out under the condition of obtaining the user's authorization.

[0123] In order to improve the calibration accuracy of the camera parameters of the camera, the embodiments of this application provide a method, device, electronic device, and storage medium for calibrating the parameters of a binocular camera; the execution subject of the solution of this application can be a binocular camera or an electronic device with computing capabilities, such as a computer, server, mobile phone, etc. The method includes:

[0124] Obtain a first image and a second image captured by the binocular camera for a reference object; wherein, each camera of the binocular camera has a parameter set including camera internal parameters and distortion coefficients calibrated by an initial method;

[0125] Based on the parameter set calibrated by the initial method, perform parameter calibration processing to obtain the optimized calibration result corresponding to the initial method;

[0126] Based on the obtained optimized calibration result, determine the camera parameter calibration result of the binocular camera;

[0127] Among them, the parameter calibration processing includes:

[0128] Based on the current parameter set of each camera, perform distortion correction on the first image and the second image obtained by shooting; and, use the corrected first image and the corrected second image to calculate the external parameters of the binocular camera as the current external parameters;

[0129] Based on the current external parameters, calculate an error value; wherein, the error value is used to characterize the degree of difference between the first position and the second position of the specified feature point of the reference object, the first position is the position obtained by projecting the position of the specified feature point in any one of the corrected first image and the corrected second image into the other image according to the current external parameters, and the second position is the position of the specified feature point in the other image;

[0130] In response to the current error value being less than a preset threshold, use the current parameter set and the current external parameters as the optimized calibration result corresponding to the initial method; otherwise, adjust the camera internal parameters in the reference parameter set of each camera to obtain the current parameter set of each camera, and return to the step of performing distortion correction on the first image and the second image obtained by shooting based on the current parameter set of each camera; the determination method of the reference parameter set of each camera includes determining the parameter set calibrated by the initial method as the initial reference parameter set.

[0131] The solution of the present application can perform distortion correction on the first image and the second image obtained by the binocular camera shooting the reference object through the current parameter set of the binocular camera, use the corrected first image and the corrected second image to calculate the current external parameters of the binocular camera, calculate the error value based on the current external parameters, when the error value is not less than the preset threshold, adjust the camera internal parameters in the reference parameter set of each camera to obtain the current parameter set of each camera, and re-perform distortion correction on the first image and the second image obtained by the binocular camera shooting the reference object based on the current parameter set, use the corrected first image and the corrected second image to calculate the current external parameters of the binocular camera, calculate the error value based on the current external parameters, until the calculated error value is less than the preset threshold, then use the current parameter set and the current external parameters as the optimized calibration result corresponding to the initial method. Therefore, the solution of the present application can continuously iterate and update the parameter set and external parameters of each camera, and calculate the loss value of the binocular camera based on the updated parameter set and external parameters of each camera, so as to determine relatively accurate camera parameters based on the loss value. It can be seen that the solution of the present application can improve the calibration accuracy of the camera parameters of the camera.

[0132] Next, in conjunction with the accompanying drawings, a method for calibrating the parameters of a binocular camera provided by an embodiment of the present application will be introduced.

[0133] Figure 1The flowchart of a method for calibrating parameters of a binocular camera provided by this application. As Figure 1 shown, the method includes:

[0134] S101, obtaining a first image and a second image captured by the binocular camera for a reference object; wherein, the first image and the second image are images captured by different cameras of the binocular camera; each camera has a parameter set including camera internal parameters and distortion coefficients calibrated by an initial method.

[0135] In this application, the two cameras of the binocular camera to be calibrated can be arranged vertically, horizontally, or in other directions. This application does not limit the arrangement of the two cameras of the binocular camera. Additionally, the number of reference objects in the shooting scene can be one or more. When the number of reference objects is one, the two cameras shoot the one reference object to obtain the first image and the second image; when the number of reference objects is multiple, the two cameras shoot the multiple reference objects to obtain the first image and the second image.

[0136] The first image and the second image can be images captured by different cameras of the binocular camera for the same reference object at the same time. The reference object can be pre-specified, such as a picture or other object, etc. This application does not limit the reference object.

[0137] Exemplarily, Figure 2a and Figure 2b are schematic diagrams of the first image and the second image captured by a binocular camera of this application. In this scene, the two cameras of this binocular camera are arranged in an up-and-down layout, Figure 2a and Figure 2b both have reference objects, and the reference objects are specifically various pictures. Figure 2a and Figure 2b In, 001 refers to each reference object.

[0138] Exemplarily, the number of the initial methods is one, that is, there is one initial method. The parameter set including camera internal parameters and distortion coefficients calibrated by this one initial method can be applied to the calibration method of this application; for example, this initial method can include: an initial method belonging to laboratory calibration performed on any camera under the target camera model;

[0139] Or, an initial method belonging to factory line calibration performed on the binocular camera itself;

[0140] Wherein, the target camera model is the camera model to which the binocular camera belongs

[0141] Exemplarily, the number of the initial methods is multiple, that is, there are multiple initial methods, and the parameter groups including the camera internal parameters and distortion coefficients calibrated by each initial method are all applied to the calibration method of the present application; for example, the multiple initial methods include:

[0142] The initial method that belongs to laboratory calibration and is performed on any camera under the target camera model, and the initial method that belongs to factory line calibration and is performed on the binocular camera itself;

[0143] Wherein, the target camera model is the camera model to which the binocular camera belongs.

[0144] The initial method that belongs to laboratory calibration and is performed on any camera under the target camera model means that in the laboratory, by professional staff, using professional methods, the calibration is performed on any camera under the target camera model. It can be understood that in this method, for the any camera used during calibration, the calibrated parameters are very accurate. This method uses the parameters calibrated for the any camera as the parameters of each camera under the target camera model. However, even for each camera belonging to the same camera model, there may be certain differences in the true parameters of each camera.

[0145] The initial method that belongs to factory line calibration and is performed on the binocular camera itself means that after the factory shipment, by on-site staff, such as production workers, the calibration operation is performed on each binocular camera to obtain the parameters of the binocular camera. In this method, calibration can be performed for each camera. However, due to factors such as the large workload of the staff, improper operation techniques of the staff, and interference in the calibration environment, the calibration of the camera parameters is inaccurate.

[0146] As can be seen from the above, the camera parameters calibrated by the above various initial methods may all have errors.

[0147] The methods used by the above laboratory and factory staff for calibrating parameters are the same. The difference lies in the environment and operation techniques, resulting in more accurate parameters for laboratory calibration and larger errors for the parameters calibrated by factory staff.

[0148] The method used for calibrating parameters can be the Zhang Zhengyou calibration method, also known as the checkerboard calibration method. Next, the calibration method will be specifically introduced. This calibration method is for the calibration process of any one camera in the binocular camera.

[0149] Step 1: Use the checkerboard as the calibration object, and set the world coordinates of the checkerboard corner points as the plane coordinate system;

[0150] When collecting data, multiple checkerboard images at different angles can be taken to improve the accuracy.

[0151] Step 2: Calculate the homography matrix based on the correspondence between image coordinates and world coordinates to describe the projection transformation from world coordinates to image coordinates.

[0152] Step 3: Decompose the homography matrix and combine it with the orthogonality constraint of the rotation matrix to derive the camera intrinsic parameter matrix (including focal length (fx, fy), principal point (cx, cy), distortion coefficients and extrinsic parameters.

[0153] The homography matrix can be calculated by the least squares method, and multiple sets of equations can be solved together to obtain the initial values of the internal parameters.

[0154] Through the above steps, the calibrated camera parameters for each camera in the binocular camera can be obtained.

[0155] The above is only an exemplary method of calibrating parameters provided in the present application. Any method that can be used to calibrate parameters in laboratories and factories, that is, any initial method, can be applied to the present application.

[0156] The parameter group involved in this embodiment may be a parameter group obtained by calibration using any initial method and including camera intrinsic parameters and distortion coefficients.

[0157] S102, performing parameter calibration processing based on the parameter group obtained by calibration in the initial method to obtain an optimized calibration result corresponding to the initial method;

[0158] For each initial method, a parameter calibration process can be performed based on a parameter group obtained by calibrating the initial method to obtain an optimized calibration result corresponding to the initial method.

[0159] The parameter calibration process includes steps A1 to A3:

[0160] Step A1, based on the current parameter group of each camera, performing distortion correction on the first image and the second image obtained by shooting; and using the corrected first image and the corrected second image, calculating the external parameters of the binocular camera as the current external parameters;

[0161] When performing distortion correction on the first image and the second image obtained by shooting, for the first image, the camera intrinsic parameters and distortion coefficients in the parameter group of the camera that shot the first image can be used to execute a specified distortion correction algorithm to perform distortion correction, and for the second image, the camera intrinsic parameters and distortion coefficients in the parameter group of the camera that shot the second image can be used to execute a specified distortion correction algorithm to perform distortion correction. Any algorithm that uses the parameters of the parameter group to perform distortion correction on an image can be applied to this application.

[0162] In this application, the homography matrix of the binocular camera can be used as the external parameters of the binocular camera.

[0163] Specifically, based on the positions of the same feature points of the reference object in the corrected first image and the corrected second image, the mapping relationship can be calculated, that is, the homography matrix is calculated. The calculated homography matrix is the external parameter of the binocular camera.

[0164] Step A2, calculate the error value based on the current external parameters; wherein, the error value is used to characterize the degree of difference between the first position and the second position of the specified feature point of the reference object, the first position is the position obtained by projecting the position of the specified feature point in any one of the corrected first image and the corrected second image into the other image according to the current external parameters, and the second position is the position of the specified feature point in the other image;

[0165] The specified feature points of the reference object can be preset. For example, when the reference object is a picture, it can be the specified corner points of the specified picture; and the number of specified feature points can be one or more.

[0166] The external parameters of this application can characterize the projective transformation between the first image and the second image. Thus, based on the current external parameters, it can be determined that the position of the specified feature point in any one of the corrected first image and the corrected second image is projected into the other image according to the current external parameters to obtain the first position, and the second position refers to the position of the specified feature point itself in the other image.

[0167] Exemplarily, in one implementation, the error value can be calculated in the following manner:

[0168] Wherein, is the position of the specified feature point in any one of the corrected first image and the corrected second image, is the second position of the specified feature point, and H is the external parameter; represents the first position; loss is the error value.

[0169] The error value calculated based on the current external parameters in this application can be the error value corresponding to a single specified feature point or the error value obtained by summing the error values corresponding to multiple specified feature points.

[0170] Step A3, in response to the current error value being less than a preset threshold, use the current parameter set and the current external parameters as the optimized calibration result corresponding to the initial method; otherwise, adjust the camera internal parameters in the reference parameter set of each camera to obtain the current parameter set of each camera, and return to perform the step of distortion correction on the first image and the second image obtained by shooting based on the current parameter set of each camera; the determination method of the reference parameter set of each camera includes determining the parameter set calibrated by the initial method as the initial reference parameter set.

[0171] The specific magnitude of the error value can be set according to the actual scenario, and the present application does not limit this. If the error value is less than the preset threshold, it indicates that the current parameter set and the current external parameters are relatively accurate and can be used as the camera parameter calibration result.

[0172] Among them, there can be various specific implementation manners for adjusting the camera internal parameters in the reference parameter set of each camera to obtain the current parameter set of each camera.

[0173] Exemplarily, in one implementation manner, the adjustment of the camera internal parameters in the reference parameter set of each camera to obtain the current parameter set of each camera includes:

[0174] Select the adjustment amount to be utilized from the selectable range of the predetermined representation of the adjustment amount, and based on the adjustment amount to be utilized, adjust the camera internal parameters in the reference parameter set of each camera, thereby obtaining the current parameter set of each camera.

[0175] Exemplarily, in one implementation manner, the adjustment of the camera internal parameters in the reference parameter set of each camera to obtain the current parameter set of each camera may include:

[0176] Select the target coefficient to be utilized, and based on the current target step size and the target coefficient, adjust the camera internal parameters in the reference parameter set of each camera; wherein, the target step size is used to represent the degree value of the allowable adjustment degree, and the target coefficient is used to represent the influence degree on the magnitude and direction of the target step size during parameter adjustment.

[0177] That is, if the error value is not less than the preset threshold, the target coefficient to be utilized can be selected. Based on the current target step size and the target coefficient, the camera internal parameters in the reference parameter group of each camera are adjusted to obtain the current parameter group of each camera, and the steps of performing distortion correction on the first image and the second image obtained by shooting are returned based on the current parameter group of each camera. Thus, based on the adjusted camera internal parameters, the first image and the second image can be re-distortion corrected, and then the external parameters of the binocular camera are calculated using the corrected first image and the corrected second image as the current external parameters. Based on the current external parameters, the error value is calculated; until the error value is less than the preset threshold.

[0178] When adjusting the camera internal parameters in the reference parameter group of each camera, only fx and fy in the camera internal parameters can be adjusted.

[0179] And when adjusting each camera of the binocular camera, the target step sizes utilized can be the same or different, and the selected target coefficients can also be the same or different. Optionally, the selection of the target coefficient to be utilized may include:

[0180] Select the coefficient to be utilized from the set coefficient value range according to the set selection method as the target coefficient.

[0181] In one implementation, the coefficient value range can be preset according to the specific scenario.

[0182] In one implementation, the coefficient value range can be set according to the step size. For example, if the previous step size is 10, then in the next iteration, the coefficient value range can be between [-10, 10]. If the previous one is 5, then in the next iteration, the coefficient value range can be between [-5, 5]. In this implementation, the initial coefficient value range can be preset.

[0183] The target coefficient can be randomly selected from the coefficient value range or can be selected from the coefficient value range in sequence.

[0184] The coefficient ranges corresponding to the two cameras of the binocular camera can be the same or different.

[0185] Optionally, the adjustment of the camera internal parameters in the reference parameter group of each camera based on the current target step size and the target coefficient includes:

[0186] Based on the current target step size and multiple target coefficients, adjust the first parameter and the second parameter in the camera internal parameters of the reference parameter group of each camera;

[0187] Each target coefficient is used to adjust one of the first parameter and the second parameter, and different target coefficients are used to adjust different parameters. The first parameter represents the pixel size of the focal length in the horizontal direction, and the second parameter represents the pixel size of the focal length in the vertical direction. In the above embodiments, the physical meanings of the various parameters in the camera internal parameters are introduced. It can be understood that the coordinates of the image center are usually relatively stable. Therefore, in order to improve efficiency, only the first parameter and the second parameter in the camera internal parameters can be adjusted. The first parameter is the fx, and the second parameter is the fy.

[0188] The target step size has an initial value, and the initial value of the target step size can be selected according to the actual scenario. For example, it can be set to 10. The initial value of the target step size can also be determined according to the pixel size of the reference object in the image. For example, if the size of a reference object is 60 pixels, taking one-third of 60 pixels, i.e., 20 pixels, or one-fifth of 60 pixels, i.e., 12 pixels, as the initial value of the target step size is acceptable.

[0189] In this embodiment, both the first parameter and the second parameter have corresponding target coefficients. When selecting the target coefficient each time, the target coefficient of each of the first parameter and the second parameter can be selected, that is, the target coefficient of the first parameter and the target coefficient of the second parameter are updated simultaneously, or only one of the target coefficient of the first parameter and the target coefficient of the second parameter can be selected, that is, only one of the target coefficient of the first parameter and the target coefficient of the second parameter is updated, and the other target coefficient remains unchanged.

[0190] Specifically, the fx and fy in the camera internal parameters in the current parameter group of each camera can be adjusted according to the following formula: fx′ = fx + i * step, fy′ = fy + j * step.

[0191] Among them, fx′ and fy′ belong to the adjusted camera internal parameters, and fx and fy belong to the camera internal parameters in the reference parameter group. i and j are the target step sizes, and j is the target step size.

[0192] Optionally, the parameter calibration process further includes:

[0193] In response to meeting the step size update condition, the current target step size is reduced to obtain a new current target step size; where meeting the step size update condition includes: a first condition indicating that the number of calculations of the loss value reaches a predetermined number or a second condition indicating that the loss value calculated most recently is less than a preset intermediate threshold; where the intermediate threshold is greater than the preset threshold. In one implementation, the predetermined number can be preset.

[0194] In one implementation, the predetermined number of times is related to the target coefficient. The predetermined number of times is determined based on the number of selections that each target coefficient can make within the set coefficient value range. For example, if fx and fy in the camera internal parameters are adjusted, and there are 20 target coefficients for each of fx and fy within the value range, and only one of the target coefficients of fx and the target coefficient of fy is updated each time, the predetermined number of times can be 20×20 = 400 times.

[0195] When the current loss value is less than a preset intermediate threshold, the current target step size can be reduced to obtain a new current target step size, and then the new target step size is used for parameter calibration processing. This method does not require performing the processing for the predetermined number of times for each step size, and can effectively improve the processing efficiency.

[0196] When reducing the current target step size, the amount reduced each time can be preset. The amount of reduction of the current target step size can consider both accuracy and efficiency. The step size of the next iteration needs to cover the entire coefficient value range of the previous iteration. Taking the previous step size as 10 as an example, in the next iteration, if the coefficient value range is between [-10, 10], and the iteration step size is selected as 1, then 20 tests are required; if the step size is 2, 10 tests are required. This is a consideration of efficiency, that is, the smaller the selected step size, the lower the efficiency. However, the smaller the step size, the closer the final result is to the local optimal external parameters, and the higher the accuracy. For example, if the local optimal result is at 5.1, then with a step size of 1, the final result will be 5, and with a step size of 2, the final result will be 6. Therefore, when reducing the current target step size, the amount can consider both accuracy and efficiency.

[0197] Correspondingly, when the target step size is updated, the method for determining the reference parameter group of each camera further includes:

[0198] In response to meeting the step size update condition, determine the reference loss value in the calculated loss values, determine the current parameter group existing in the calculation process of the reference loss value to obtain a backup parameter group, and determine the target coefficient based on which the camera internal parameters in the backup parameter group are updated; wherein, the method for determining the reference loss value includes: when the step size update condition includes the first condition, determine the minimum loss value obtained during the calculation for the predetermined number of times of the loss value as the reference loss value, or when the step size update condition includes the second condition, determine the most recently calculated loss value as the reference loss value;

[0199] Based on the determined target coefficient and the current target step size before performing the reduction processing, adjust the current reference parameter group of each camera as the new reference parameter group.

[0200] Specifically, when the step size update condition is the first condition, if the predetermined number of times is N times, then among the N times of loss value calculation processes before meeting the step size update condition, the smallest loss value is used as the reference loss value, and the target coefficient in the parameter group used to calculate this reference loss value is determined. Then, using the determined target coefficient and the current target step size before performing the reduction process, the current reference parameter group of each camera is adjusted as the new reference parameter group.

[0201] In a specific embodiment, when the determination method for meeting the step size update condition is the first condition and the predetermined number of times is 400 times, then in the first four hundred loops before meeting the step size update condition, i′ and j′ can be found such that loss is minimized, and then fx2 = fx1 + i′ * step, fy2 = fy1 + j′ * step; where fx1 is the parameter fx in the reference parameter group before meeting the step size update condition, fy1 is the parameter fy in the reference parameter group before meeting the step size update condition, fx2 is the parameter fx in the updated reference parameter group, and fy2 is the parameter fy in the updated reference parameter group. Step is the target step size before meeting the step size update condition.

[0202] S103. Based on the obtained optimized calibration result, determine the camera parameter calibration result of the binocular camera.

[0203] When the number of initial methods is one, the obtained optimized calibration result can be directly used as the camera parameter calibration result of the binocular camera.

[0204] When the number of initial methods is multiple, the parameter groups calibrated by each initial method can be determined, and the size of the loss value obtained in the last parameter calibration process can be determined. The optimized calibration result obtained by the initial method with the smallest loss value among them is used as the camera parameter calibration result of the binocular camera. The specific content of this embodiment is specifically introduced in the following embodiments.

[0205] The camera parameter calibration result obtained by the solution of the present application can effectively optimize the effect of the camera stitching algorithm, and the optimization effect is as follows Figure 3b , it can be seen that the stitching seams on both sides of the image are effectively reduced.

[0206] Exemplarily, Figure 3a is the effect obtained by stitching using the parameter group and external parameters calibrated by any initial method, while Figure 3b the effect obtained by stitching using the camera parameter calibration result obtained by the solution of the present application. It can be seen that Figure 3b the stitching gap in Figure 3a is effectively reduced compared to Figure 3a and Figure 3bAmong them, 001 refers to each reference object.

[0207] For the solution of this application, the first image and the second image captured by the binocular camera for the reference object can be distortion-corrected through the current parameter group of the binocular camera. Using the corrected first image and the corrected second image, calculate the current external parameters of the binocular camera. Based on the current external parameters, calculate the error value. When the error value is not less than the preset threshold, adjust the camera internal parameters in the reference parameter group of each camera to obtain the current parameter group of each camera, and re-distortion-correct the first image and the second image captured by the binocular camera for the reference object based on the current parameter group. Using the corrected first image and the corrected second image, calculate the current external parameters of the binocular camera. Based on the current external parameters, calculate the error value until the calculated error value is less than the preset threshold, then use the current parameter group and the current external parameters as the optimized calibration result corresponding to the initial method. Therefore, the solution of this application can continuously iterate and update the parameter group and external parameters of each camera, and calculate the loss value of the binocular camera based on the updated parameter group and external parameters of each camera, so as to determine relatively accurate camera parameters based on the loss value. It can be seen that the solution of this application can improve the calibration accuracy of the camera parameters of the camera.

[0208] Optionally, the number of the initial methods is multiple; the parameter calibration process is performed on the parameter group calibrated based on the initial method to obtain the optimized calibration result corresponding to the initial method, including:

[0209] For each initial method, perform parameter calibration processing on the parameter group calibrated based on this initial method to obtain the optimized calibration result corresponding to this initial method.

[0210] For different binocular cameras, the difference between the parameters determined by each initial method and the actual parameters of this binocular camera can be different. Therefore, for different binocular cameras, when using the parameter groups calibrated by different initial methods to perform the above parameter calibration method, the obtained calibration results may also be different. Therefore, in order to obtain a more accurate camera parameter calibration result, for each camera, the camera parameter calibration result of this camera can be calculated using the parameters determined by each initial method, and a more accurate calibration result can be selected from them as the final calibration result.

[0211] Correspondingly, the determination of the camera parameter calibration result of the binocular camera based on the obtained optimized calibration result includes:

[0212] Select the optimized calibration result corresponding to the initial method with the smallest target loss value from the obtained optimized calibration results as the camera parameter calibration result of the binocular camera;

[0213] Among them, the target loss value corresponding to any initial method is the last loss value obtained by performing parameter calibration processing based on the parameter group calibrated according to this initial method.

[0214] For each initial method, a loss value less than the preset threshold corresponding to this initial method is obtained. Among them, the optimized calibration result corresponding to the initial method with the smallest loss value is the most accurate calibration result. Therefore, the optimized calibration result corresponding to the initial method with the smallest corresponding target loss value can be selected as the camera parameter calibration result of the binocular camera.

[0215] It can be seen that the solution of this embodiment can perform parameter calibration processing for each initial method, determine the optimized calibration result corresponding to this initial method, and thus, based on the last loss value obtained by performing parameter calibration processing for each initial method, determine the most accurate camera parameter calibration result of the binocular camera from the optimized calibration results corresponding to each initial method. Therefore, the method of this embodiment can further improve the calibration accuracy of the camera parameters of the camera.

[0216] Optionally, the number of the reference objects is multiple and the reference objects are preset pictures, and the specified feature points are the corner points of the preset pictures;

[0217] In one implementation manner, the performing distortion correction on the captured first image and second image based on the current parameter group of each camera includes:

[0218] Determine the corner points in the captured first image, and perform distortion correction on the determined corner points according to the current parameter group of the camera used to capture the first image to obtain the corrected first image;

[0219] Determine the corner points in the captured second image, and perform distortion correction on the determined corner points according to the current parameter group of the camera used to capture the second image to obtain the corrected second image;

[0220] The preset picture may specifically be a picture of a QR code or a checkerboard picture, and the present application does not limit this.

[0221] The specific manner of undistortion can be completed through the function P new = undistort(P, M, D); where undistort is a function in the OpenCV library (OpenCV is an open-source computer vision and machine learning software library), mainly used to correct the distortion of images. P is the corner point to be undistorted, M is the camera internal parameter, D is the distortion coefficient, and P new is the distorted corner point.

[0222] Therefore, by using the above function, distortion correction can be performed for each corner point in the first image to obtain the corrected first image, and distortion correction can be performed for each corner point in the second image to obtain the corrected second image.

[0223] In one implementation, performing distortion correction on the first and second images obtained by shooting based on the current parameter set of each camera includes:

[0224] For each of the first and second images, traverse each pixel in the image;

[0225] According to the current parameter set of the camera used to shoot the first image, correct each pixel in the first image; according to the current parameter set of the camera used to shoot the second image, correct each pixel in the second image;

[0226] Store the pixel values after correction of each pixel in the first image in a new image to obtain the first image after distortion correction; store the pixel values after correction of each pixel in the second image in a new image to obtain the second image after distortion correction.

[0227] Calculating the external parameters of the binocular camera by using the corrected first image and the corrected second image as the current external parameters includes:

[0228] Determine the respective corner points in the corrected first image to obtain a first set of corner points, and determine the respective corner points in the corrected second image to obtain a second set of corner points;

[0229] Calculate the external parameters of the binocular camera based on the positions of the corner points in the first set of corner points and the positions of the matching corner points in the second set of corner points.

[0230] The external parameters of the binocular camera can be calculated by the function H = Ransac(P1, P2).

[0231] Wherein, the P1 is the first set of corner points; P2 is the second set of corner points. H is the calculated homography matrix, that is, the external parameters of the binocular camera.

[0232] Exemplarily, Figure 4 It is a schematic diagram for determining the respective corner points in the first image and the matching corner points in the second image. Figure 4 In it, 001 refers to each reference object.

[0233] The order of the corner points in the first set of corner points matches the order of the corner points in the second set of corner points. For example, the second corner point in the first set of corner points and the second corner point in the second set of corner points are the corner points of the same reference object.

[0234] It can be seen that the solution of this embodiment can effectively determine the current external parameters of each camera of the binocular camera.

[0235] Corresponding to the above parameter calibration method of the binocular camera, the present application provides a parameter calibration device for the binocular camera. Figure 5 FIG. is a schematic structural diagram of a parameter calibration device for a binocular camera provided by an embodiment of the present application; the device includes:

[0236] An acquisition module 501, configured to acquire a first image and a second image obtained by the binocular camera shooting a reference object; wherein, each camera of the binocular camera has a parameter group including camera internal parameters and distortion coefficients calibrated by an initial method.

[0237] A calibration module 502, configured to perform parameter calibration processing based on the parameter group calibrated by the initial method to obtain an optimized calibration result corresponding to the initial method.

[0238] A determination module 503, configured to determine the camera parameter calibration result of the binocular camera based on the obtained optimized calibration result.

[0239] Wherein, the parameter calibration processing includes:

[0240] Based on the current parameter group of each camera, perform distortion correction on the obtained first image and second image; and use the corrected first image and the corrected second image to calculate the external parameters of the binocular camera as the current external parameters.

[0241] Based on the current external parameters, calculate an error value; wherein, the error value is used to characterize the difference degree between the first position and the second position of the specified feature point of the reference object, the first position is the position obtained by projecting the position of the specified feature point in any one of the corrected first image and the corrected second image onto the other image according to the current external parameters, and the second position is the position of the specified feature point in the other image.

[0242] In response to the current error value being less than a preset threshold, use the current parameter group and the current external parameters as the optimized calibration result corresponding to the initial method; otherwise, adjust the camera internal parameters in the reference parameter group of each camera to obtain the current parameter group of each camera, and return to the step of performing distortion correction on the obtained first image and second image based on the current parameter group of each camera; wherein, the determination method of the reference parameter group of each camera includes determining the parameter group calibrated by the initial method as the initial reference parameter group.

[0243] The solution of this application can perform distortion correction on the first image and the second image captured by the binocular camera for the reference object through the current parameter group of the binocular camera, calculate the current external parameters of the binocular camera by using the corrected first image and the corrected second image, calculate the error value based on the current external parameters, and when the error value is not less than the preset threshold, adjust the camera internal parameters in the reference parameter group of each camera to obtain the current parameter group of each camera, and re-perform distortion correction on the first image and the second image captured by the binocular camera for the reference object based on the current parameter group, calculate the current external parameters of the binocular camera by using the corrected first image and the corrected second image, calculate the error value based on the current external parameters, until the calculated error value is less than the preset threshold, then use the current parameter group and the current external parameters as the optimized calibration result corresponding to the initial method. Therefore, the solution of this application can continuously iterate and update the parameter group and external parameters of each camera, and calculate the loss value of the binocular camera based on the updated parameter group and external parameters of each camera, so as to determine relatively accurate camera parameters based on the loss value. It can be seen that the solution of this application can improve the calibration accuracy of the camera parameters of the camera.

[0244] Optionally, adjusting the camera internal parameters in the reference parameter group of each camera to obtain the current parameter group of each camera includes:

[0245] Select the target coefficient to be used, and adjust the camera internal parameters in the reference parameter group of each camera based on the current target step size and the target coefficient; wherein, the target step size is a degree value used to represent the allowable adjustment degree, and the target coefficient is used to represent the influence degree on the size and direction of the target step size during parameter adjustment. Optionally, the number of the initial methods is multiple;

[0246] The calibration module includes:

[0247] The calibration unit is used to perform parameter calibration processing on the parameter group calibrated based on each initial method for each initial method to obtain the optimized calibration result corresponding to this initial method;

[0248] The determination module includes:

[0249] The selection unit is used to select the optimized calibration result corresponding to the initial method with the smallest target loss value from the obtained optimized calibration results as the camera parameter calibration result of the binocular camera;

[0250] Wherein, the target loss value corresponding to any initial method is the last loss value obtained by performing parameter calibration processing on the parameter group calibrated based on this initial method.

[0251] Optionally, the multiple initial methods include:

[0252] The initial method for any camera under the target camera model, which belongs to laboratory calibration, and the initial method for the binocular camera itself, which belongs to factory line calibration;

[0253] Among them, the target camera model is the camera model to which the binocular camera belongs.

[0254] Optionally, the parameter calibration process further includes:

[0255] In response to meeting the step size update condition, perform a reduction process on the current target step size to obtain a new current target step size; where meeting the step size update condition includes: a first condition indicating that the number of calculations of the loss value reaches a predetermined number or a second condition indicating that the loss value calculated most recently is less than a preset intermediate threshold; where the intermediate threshold is greater than the preset threshold;

[0256] The method for determining the reference parameter group of each camera further includes:

[0257] In response to meeting the step size update condition, determine the reference loss value among the calculated loss values, determine the current parameter group existing in the calculation process of the reference loss value to obtain a backup parameter group, and determine the target coefficient based on which the camera internal parameters in the backup parameter group are updated; where the method for determining the reference loss value includes: when the step size update condition includes the first condition, determine the smallest loss value obtained during the predetermined number of calculations of the loss value as the reference loss value, or when the step size update condition includes the second condition, determine the loss value calculated most recently as the reference loss value;

[0258] Based on the determined target coefficient and the current target step size before performing the reduction process, adjust the current reference parameter group of each camera as the new reference parameter group.

[0259] The adjustment of the camera internal parameters in the reference parameter group of each camera based on the current target step size and the target coefficient includes:

[0260] Based on the current target step size and multiple target coefficients, adjust the first parameter and the second parameter in the camera internal parameters of the reference parameter group of each camera;

[0261] Each target coefficient is used to adjust one of the first parameter and the second parameter, and different target coefficients are used to adjust different parameters. The first parameter represents the pixel size of the focal length in the horizontal direction, and the second parameter is used to represent the pixel size of the focal length in the vertical direction. Optionally, the number of reference objects is multiple and the reference objects are preset pictures, and the specified feature points are the corner points of the preset pictures;

[0262] Performing distortion correction on the first image and the second image obtained by shooting based on the current parameter set of each camera includes:

[0263] Determine each corner point in the first image obtained by shooting, and perform distortion correction on each determined corner point according to the current parameter set of the camera used to shoot the first image to obtain the corrected first image;

[0264] Determine each corner point in the second image obtained by shooting, and perform distortion correction on each determined corner point according to the current parameter set of the camera used to shoot the second image to obtain the corrected second image;

[0265] Using the corrected first image and the corrected second image to calculate the external parameters of the binocular camera as the current external parameters includes:

[0266] Determine each corner point in the corrected first image to obtain a first corner point set, and determine each corner point in the corrected second image to obtain a second corner point set;

[0267] Calculate the external parameters of the binocular camera based on the positions of the corner points in the first corner point set and the positions of the matching corner points in the second corner point set.

[0268] Optionally, the selection of the target coefficient to be used includes:

[0269] Select the coefficient to be used from the set coefficient value range according to the set selection method as the target coefficient.

[0270] An embodiment of the present application also provides an electronic device, as Figure 6 shown, including:

[0271] A memory 601 for storing a computer program;

[0272] A processor 602, when executing the program stored on the memory 601, implements the parameter calibration method of the binocular camera described in any one of the above.

[0273] And the above electronic device may further include a communication bus and / or a communication interface, and the processor 602, the communication interface, and the memory 601 complete communication with each other through the communication bus.

[0274] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0275] The communication interface is used for communication between the above electronic device and other devices.

[0276] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0277] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0278] In another embodiment provided by the present application, there is also provided a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above-mentioned parameter calibration methods for binocular cameras are implemented.

[0279] In another embodiment provided by the present application, there is also provided a computer program product containing instructions, which when running on a computer, causes the computer to execute any of the parameter calibration methods for binocular cameras in the above embodiments.

[0280] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a solid-state disk (SSD), etc.

[0281] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including", or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article, or device that includes a series of elements includes not only those elements but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article, or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article, or device that includes the element.

[0282] Each embodiment in this specification is described in a related manner. The same or similar parts between the embodiments can be referred to each other, and the differences between each embodiment and other embodiments are emphasized.

[0283] The above are only the preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A parameter calibration method for a binocular camera, characterized in that The method includes: Obtaining a first image and a second image captured by a binocular camera for a reference object; wherein, each camera of the binocular camera has a parameter set including camera internal parameters and distortion coefficients calibrated by an initial method; Performing parameter calibration processing based on the parameter set calibrated by the initial method to obtain an optimized calibration result corresponding to the initial method; Determining a camera parameter calibration result of the binocular camera based on the obtained optimized calibration result; Wherein, the parameter calibration processing includes: Performing distortion correction on the captured first image and second image based on the current parameter set of each camera; and calculating the external parameters of the binocular camera using the corrected first image and the corrected second image as the current external parameters; Calculating an error value based on the current external parameters; wherein, the error value is used to characterize the degree of difference between a first position and a second position of a specified feature point of the reference object, the first position is the position obtained by projecting the position of the specified feature point in any one of the corrected first image and the corrected second image into the other image according to the current external parameters, and the second position is the position of the specified feature point in the other image; In response to the current error value being less than a preset threshold, taking the current parameter set and the current external parameters as the optimized calibration result corresponding to the initial method; otherwise, adjusting the camera internal parameters in the reference parameter set of each camera to obtain the current parameter set of each camera, and returning to the step of performing distortion correction on the captured first image and second image based on the current parameter set of each camera; the determination method of the reference parameter set of each camera includes determining the parameter set calibrated by the initial method as the initial reference parameter set.

2. The method according to claim 1, characterized in that, The adjusting the camera internal parameters in the reference parameter set of each camera to obtain the current parameter set of each camera includes: Selecting a target coefficient to be utilized, and adjusting the camera internal parameters in the reference parameter set of each camera based on the current target step size and the target coefficient; wherein, the target step size is a value used to characterize the degree of allowable adjustment, and the target coefficient is used to characterize the influence degree on the magnitude and direction of the target step size during parameter adjustment.

3. The method according to claim 1 or 2, characterized in that, The number of the initial methods is multiple; Performing parameter calibration processing based on the parameter set calibrated by the initial method to obtain an optimized calibration result corresponding to the initial method, including: For each initial method, performing parameter calibration processing based on the parameter set calibrated by the initial method to obtain an optimized calibration result corresponding to the initial method; The determining the camera parameter calibration result of the binocular camera based on the obtained optimized calibration result includes: Selecting the optimized calibration result corresponding to the initial method with the smallest corresponding target loss value from the obtained optimized calibration results as the camera parameter calibration result of the binocular camera; Wherein, the target loss value corresponding to any initial method is the last loss value obtained by performing parameter calibration processing based on the parameter set calibrated by the initial method. Among them, the multiple initial methods include: An initial method belonging to laboratory calibration performed on any camera under the target camera model, and an initial method belonging to factory line calibration performed on the binocular camera itself; Among them, the target camera model is the camera model to which the binocular camera belongs.

4. The method according to claim 2, characterized in that, The parameter calibration process further includes: In response to satisfying the step size update condition, perform a reduction process on the current target step size to obtain a new current target step size; where satisfying the step size update condition includes: a first condition indicating that the number of calculations of the loss value reaches a predetermined number or a second condition indicating that the loss value calculated most recently is less than a preset intermediate threshold; where the intermediate threshold is greater than the preset threshold; The method for determining the reference parameter group of each camera further includes: In response to satisfying the step size update condition, determine the reference loss value among the calculated loss values, determine the current parameter group existing in the calculation process of the reference loss value, obtain a backup parameter group, and determine the target coefficient based on which the camera internal parameters in the backup parameter group are updated; where the method for determining the reference loss value includes: when the step size update condition includes the first condition, determine the minimum loss value obtained during the predetermined number of calculations of the loss value as the reference loss value, or when the step size update condition includes the second condition, determine the loss value calculated most recently as the reference loss value; Based on the determined target coefficient and the current target step size before performing the reduction process, adjust the current reference parameter group of each camera as the new reference parameter group.

5. The method according to claim 2, characterized in that, The adjusting the camera internal parameters in the reference parameter group of each camera based on the current target step size and the target coefficient includes: Based on the current target step size and multiple target coefficients, adjust the first parameter and the second parameter in the camera internal parameters of the reference parameter group of each camera; Each target coefficient is used to adjust one of the first parameter and the second parameter, and different target coefficients are used to adjust different parameters. The first parameter represents the pixel size of the focal length in the horizontal direction, and the second parameter is used to represent the pixel size of the focal length in the vertical direction.

6. The method according to any one of claims 1-2, characterized in that, The number of the reference objects is multiple and the reference objects are preset pictures, and the specified feature points are the corner points of the preset pictures; The performing distortion correction on the captured first image and second image based on the current parameter group of each camera includes: Determine the corner points in the captured first image, and perform distortion correction on the determined corner points according to the current parameter group of the camera used to capture the first image to obtain the corrected first image; Determine the corner points in the captured second image, and perform distortion correction on the determined corner points according to the current parameter group of the camera used to capture the second image to obtain the corrected second image; The calculating the external parameters of the binocular camera by using the corrected first image and the corrected second image as the current external parameters includes: Determine each corner point in the corrected first image to obtain a first set of corner points, and determine each corner point in the corrected second image to obtain a second set of corner points; Based on the positions of the corner points in the first set of corner points and the positions of the matching corner points in the second set of corner points, calculate the external parameters of the binocular camera.

7. The method according to claim 2, wherein The selecting the target coefficient to be utilized includes: Select, from a set coefficient value range, according to a set selection method, the coefficient to be utilized as the target coefficient.

8. A parameter calibration device for a binocular camera, characterized in that, The apparatus includes: An acquisition module, configured to acquire a first image and a second image captured by the binocular camera for a reference object; wherein, each camera of the binocular camera has a parameter set obtained by initial calibration, including the camera internal parameters and distortion coefficients; A calibration module, configured to perform parameter calibration processing based on the parameter set obtained by the initial calibration to obtain an optimized calibration result corresponding to the initial method; A determination module, configured to determine the camera parameter calibration result of the binocular camera based on the obtained optimized calibration result; Wherein, the parameter calibration processing includes: Based on the current parameter set of each camera, perform distortion correction on the captured first image and second image; and, use the corrected first image and the corrected second image to calculate the external parameters of the binocular camera as the current external parameters; Based on the current external parameters, calculate an error value; wherein, the error value is used to characterize the degree of difference between a first position and a second position of a specified feature point of the reference object, the first position is the position obtained by projecting the position of the specified feature point in any one of the corrected first image and the corrected second image into the other image according to the current external parameters, and the second position is the position of the specified feature point in the other image; In response to the current error value being less than a preset threshold, use the current parameter set and the current external parameters as the optimized calibration result corresponding to the initial method, otherwise, adjust the camera internal parameters in the reference parameter set of each camera to obtain the current parameter set of each camera, and return to the step of performing distortion correction on the captured first image and second image based on the current parameter set of each camera; the determination method of the reference parameter set of each camera includes determining the parameter set obtained by the initial calibration as the initial reference parameter set.

9. An electronic device, characterized in that, Includes: A memory, configured to store a computer program; A processor, configured to implement the method according to any one of claims 1-7 when executing the program stored on the memory.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1-7 is implemented.

Images