Binocular camera calibration method and device and electronic equipment

Abstract

The invention relates to the technical field of camera calibration, and provides a binocular camera calibration method, device and equipment and electronic equipment, and the binocular camera calibration method comprises the steps: recording a circle center coordinate of a circular mark of a calibration plate; the binocular camera shoots a plurality of calibration images at different angles for the calibration plate; using a canny edge extraction algorithm to obtain a circle center coordinate of the circular mark of the calibration image; calculating initial values of calibration parameters of the binocular camera; and optimizing camera calibration parameters by adopting a beam adjustment method. Compared with a traditional binocular camera calibration method, the technical scheme of the invention improves the accuracy of binocular camera calibration parameters.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a binocular camera calibration method, device and electronic equipment. Background Art

[0002] With the rapid development of computer vision technology, stereo vision has been widely used as an important three-dimensional perception method. The binocular stereo vision system simulates the principle of human binocular vision, uses two cameras to shoot the same scene from different perspectives, and then calculates the depth information of the scene through image processing algorithms.

[0003] The primary task of a binocular stereo vision system is to obtain the corresponding information of an object in the real three-dimensional world through the captured image information. Therefore, it is particularly important to establish a geometric model in the process of mapping the object from the three-dimensional world to the camera imaging plane. The most critical part of this process is to obtain the intrinsic and extrinsic parameters of the camera. Camera calibration is to obtain these parameters, thereby improving the accuracy and robustness of three-dimensional point cloud reconstruction. Summary of the invention

[0004] The object of the present invention is to provide a binocular camera calibration method, device and electronic equipment, aiming to improve the accuracy of binocular camera calibration parameters in a binocular stereo vision system.

[0005] In order to achieve the above object, the solution adopted by the present invention to solve the technical problem is:

[0006] In a first aspect, the present invention provides a binocular camera calibration method, the method comprising the following steps:

[0007] Acquire multiple images of a calibration plate at different angles, where multiple circular marks with the same radius are evenly arranged in rows and columns on the calibration plate;

[0008] According to the calibration plate, taking the center of the circular mark in the upper left corner as the origin, record the center coordinates of each circular mark in the world coordinate system;

[0009] Based on the Canny edge extraction algorithm, edge extraction is performed on the circular mark of the calibration plate image to obtain the coordinates of the center of the circle in the pixel coordinate system;

[0010] According to the determined center coordinates of the circular mark, substitute into the homography matrix equation to solve and obtain the initial values ​​of the camera parameters;

[0011] The camera extrinsic parameters are optimized for the initial values ​​of the camera parameters based on the diagonal constraints.

[0012] Further, the edge extraction of the circular mark of the calibration plate image is performed based on the Canny edge extraction algorithm to obtain the coordinates of the center of the circle in the pixel coordinate system, including:

[0013] Preprocessing the acquired calibration plate image based on a Gaussian filtering algorithm;

[0014] Calculate the gradient value of each pixel on the calibration plate image to obtain a gradient image corresponding to the calibration plate image;

[0015] According to the gradient image, the image edge amplitude and angle are calculated, and non-maximum suppression is performed along the gradient direction;

[0016] A double threshold algorithm is used on the gradient image to detect and connect edges.

[0017] Further, according to the gradient image, the image edge amplitude and angle are calculated, and non-maximum suppression is performed along the gradient direction, including:

[0018] The gradient image is subjected to calculation using Soble horizontal and vertical operators and Gaussian convolution kernel to obtain the amplitude and angle of the image gradient;

[0019] The amplitude is non-maximum suppressed according to the image gradient angle. At each point, the center of the field is compared with the pixel along its corresponding gradient direction. If the central pixel is the maximum value, it is retained, otherwise the center is set to 0.

[0020] Further, the gradient image is subjected to edge detection and edge connection using a dual threshold algorithm, comprising:

[0021] Select strong and weak thresholds, set the points with gradient amplitudes below the weak threshold to 0, and retain those greater than the strong threshold and mark them as 255;

[0022] For points whose gradient amplitude is greater than the weak threshold but less than the high threshold, it is determined whether there is a point greater than the strong threshold in its neighborhood. If so, it is retained and set to 255; if not, it is discarded and set to 0.

[0023] Furthermore, according to the determined center coordinates of the circular mark, the homography matrix equation is substituted to solve the initial values ​​of the camera parameters, including:

[0024] According to the correspondence between the object's coordinate points in the three-dimensional world and the imaging plane, the homography matrix equation is established;

[0025] According to the center coordinates of each circular mark in the world coordinate system and the center coordinates in the pixel coordinate system, the initial values ​​of the internal and external parameters of the binocular camera are solved.

[0026] Furthermore, the initial values ​​of the camera parameters are optimized using a bundle adjustment method, including:

[0027] Establishing an overall optimization equation according to the initial values ​​of the internal and external parameters of the binocular camera;

[0028] According to the overall optimization equation, the reprojection error is minimized.

[0029] In a second aspect, the present invention provides a binocular camera calibration device, the camera calibration device comprising:

[0030] An image acquisition module is used to acquire multiple calibration plate images at different angles;

[0031] An image processing module is used to extract the edges of each circular mark of the calibration plate image and obtain the coordinates of the center of the circle in a pixel coordinate system;

[0032] The data operation module is used to substitute the center coordinates of the circular mark into the homography matrix equation to solve and obtain the initial values ​​of the camera parameters.

[0033] In a third aspect, the present invention provides an electronic device, the electronic device comprising a memory and a processor;

[0034] The memory is used to store computer programs;

[0035] The processor is used to execute the computer program and implement the steps of the binocular camera calibration method as described above when executing the computer program.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the binocular camera calibration method as described above are implemented.

[0037] Beneficial effects:

[0038] The present invention provides a binocular camera calibration method, device and storage medium. The camera calibration method obtains a calibration plate image through a binocular camera; uses a canny edge extraction algorithm to obtain the coordinates of the center of a circle in the calibration image; calculates the initial values ​​of the binocular camera calibration parameters; and uses a bundle adjustment method to optimize the camera calibration parameters. The technical solution of the present invention effectively improves the calibration accuracy and efficiency of the binocular camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of a binocular camera calibration method provided by an embodiment of the present invention;

[0040] Figure 2 A binocular camera is provided in an embodiment of the present invention to capture a calibration plate image;

[0041] Figure 3 is a schematic block diagram of a binocular camera calibration device provided by an embodiment of the present invention;

[0042] Figure 4is a schematic block diagram of a binocular camera electronic device structure provided by an embodiment of the present invention; DETAILED DESCRIPTION

[0043] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. It will be appreciated that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. In addition, the embodiments described are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the art without creative work are within the scope of protection of the present invention.

[0044] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0045] The technical solution of the present invention and some implementation methods of the present invention are described in detail below with specific embodiments. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0046] Figure 1 The present invention provides a flow chart of a binocular camera calibration method. Figure 1 As shown, the camera calibration method includes steps S101 to S105.

[0047] S101, a binocular camera photographs a calibration plate to obtain multiple calibration images at different angles, wherein a plurality of circular marks with the same radius are evenly arranged in rows and columns on the calibration plate.

[0048] like Figure 2 As shown, the background of the calibration plate is white and the circular mark is black. In order to obtain the coordinates of each circular mark on the calibration plate in the world coordinate system, usually, the radius of the circular marks is the same and the intervals between the circular marks are also the same.

[0049] S102, according to the calibration plate, define the calibration plate to be located in the world coordinate system Z w = 0, with the center of the circle marked in the upper left corner as the origin, the line where the circle is located is X w Axis, the column is Y w Axis, records the center coordinates of each circular mark in the world coordinate system.

[0050] For example, suppose there are i (i=0, 1, 2......, n-1, n, and n>0) rows and j (j=0, 1, 2......, n-1, n, and n>0) columns of i×j circular marks on the calibration plate, the radius of the circular mark is r, and the interval between the circular marks is d. The coordinate formula of the center of the circular mark in the i-th row and j-th column can be expressed as:

[0051] P ij (X w ,Y w ,Z w )=((j-1)×(2r+d),(i-1)×(2r+d),Z w )

[0052] Since the calibration plate is defined to be located in the world coordinate system Z w = 0, that is, Z can be omitted in the above formula w The axes are represented as:

[0053] P ij (X w ,Y w )=((j-1)×(2r+d),(i-1)×(2r+d))

[0054] S103 , based on the Canny edge extraction algorithm, edge extraction is performed on the circular mark of the calibration plate image to obtain the coordinates of the center of the circle in a pixel coordinate system.

[0055] Specifically, in the edge extraction process, the obtained calibration plate image is first preprocessed using a Gaussian filtering algorithm, and the two-dimensional Gaussian formula can be expressed as:

[0056]

[0057] Secondly, the gradient value of each pixel point on the calibration plate image is calculated to obtain a gradient image corresponding to the calibration plate image.

[0058] Finally, according to the gradient image, the image edge amplitude and angle are calculated, and non-maximum suppression is performed along the gradient direction.

[0059] The gradient image is calculated using the Soble horizontal and vertical operators and the Gaussian convolution kernel to obtain the amplitude and angle of the image gradient. The calculation formulas for the gradient amplitude G and the gradient direction θ are as follows:

[0060]

[0061]

[0062] Among them, G x and G yare the products of the Soble horizontal and vertical operators and the Gaussian convolution kernel, respectively.

[0063] Correspondingly, the direction of the gradient is generally always perpendicular to the boundary, and the directions of the gradient are classified into vertical, horizontal, and two diagonal directions, i.e., 0 degrees, 45 degrees, 90 degrees, and 135 degrees.

[0064] Furthermore, non-maximum suppression technology is applied to filter out non-edge pixels. For each pixel, the center of the field is compared with the pixel along its corresponding gradient direction. If the central pixel is the maximum value, it is retained, otherwise the center is set to 0.

[0065] Specifically, the gradient direction is first approximated to one of the following values, including 0, 45, 90, 135, 180, 225, 270, and 315, which represent up, down, left, right, and 45-degree directions.

[0066] Secondly, compare the gradient strength of the pixel point with the pixels in the positive and negative directions of its gradient. If the gradient strength of the pixel point is the largest, it is retained; otherwise, it is suppressed, that is, set to 0.

[0067] Finally, after non-maximum suppression, there are still many noise points in the image. The dual threshold algorithm is used to detect and connect the edges. That is, by selecting strong and weak thresholds, the points with gradient amplitudes lower than the weak threshold are set to 0, and those greater than the strong threshold are retained and marked as 255.

[0068] Correspondingly, for points whose gradient amplitude is greater than the weak threshold but less than the high threshold, it is determined whether there is a point greater than the strong threshold in its neighborhood. If so, it is retained and set to 255; if not, it is discarded and set to 0.

[0069] Furthermore, the coordinates of the center of the circle in the pixel coordinate system can be obtained by the following formula:

[0070]

[0071] Among them, m pq The definition is as follows:

[0072]

[0073] S104 , according to the determined center coordinates of the circular mark, substitute them into the homography matrix equation to solve and obtain the initial values ​​of the camera parameters.

[0074] Specifically, the coordinate mapping relationship between the pixel coordinate system and the world coordinate system is:

[0075]

[0076] Among them, u and v represent the coordinates in the pixel coordinate system, s represents the scale factor, and f x 、fy , u0, v0, γ represent the five camera internal parameters, R, t represents the camera external parameters, X w , Y w , Z w (Assume that the calibration chessboard is located in the world coordinate system Z w = 0) represents the coordinates in the world coordinate system.

[0077] Further, let the homography matrix H be:

[0078]

[0079] Among them, M is the internal parameter matrix:

[0080]

[0081] Furthermore, using the constraint relationship between the rotation vectors, the homography matrix H is transformed into three column vectors:

[0082] H=[h1 h2 h3]=sM[r1 r2 t]

[0083] Furthermore, because the rotation vectors are mutually orthogonal in construction, that is, r1 and r2 are mutually orthogonal, the two definitions of orthogonality are used to derive two constraints provided by each homography matrix, namely, the dot product of the rotation vectors is 0 and the lengths of the rotation vectors are equal.

[0084] Furthermore, the two constraints can finally be transformed into the following form:

[0085]

[0086] In order to simplify the expression, the i-th column vector h of the 3×3 homography matrix H = [h1, h2, h3] is defined as i =[h i1 ,h i2 ,h i3 ] T ,make:

[0087]

[0088] Furthermore, the center coordinates of each circular mark obtained in S102 and S103 are substituted into the homography matrix constraint equation to solve and obtain the initial values ​​of the camera parameters.

[0089] S105, optimizing the initial values ​​of the camera parameters by using a bundle adjustment method.

[0090] Specifically, the coordinate mapping relationship between the pixel coordinate system and the world coordinate system can be expressed in the form of Lie algebra:

[0091]

[0092] Where ξ is the rotation vector, ξ^ is the matrix expression of the rotation vector, and s i is the scale factor, u i is the pixel coordinate, K is the camera calibration internal parameter, exp(ξ^) is the camera external parameter, P w is the spatial coordinate, then the overall optimization equation is:

[0093]

[0094] Furthermore, the internal parameters, external parameters and spatial coordinates in the optimization equation are taken as optimization objects, and the reprojection error is minimized to obtain an optimized camera calibration parameter.

[0095] like Figure 3 As shown, an embodiment of the present invention further provides a binocular camera calibration device, including: an image acquisition module 1001, an image processing module 1002, and a data calculation module 1003.

[0096] The image acquisition module 1001 is used to acquire multiple calibration plate images at different angles.

[0097] The image processing module 1002 is used to perform high-speed filtering and edge extraction on each circular mark of the calibration plate image to obtain the coordinates of the center of the circle in the pixel coordinate system.

[0098] The data operation module 1003 is used to substitute the center coordinates of the circular mark into the homography constraint matrix equation and the optimization equation to solve the camera calibration parameters.

[0099] An embodiment of the present invention further provides a readable storage medium, characterized in that the readable storage medium stores a program or instruction, and the program or instruction is implemented when executed by a processor.

[0100] like Figure 4 As shown, an embodiment of the present invention further provides a binocular camera calibration electronic device, comprising: a processor, a memory, and data stored in the memory and operable on the processor, wherein the memory, the processor, and the storage medium where the computer program is located can be connected through a system bus, and when the program or instructions in the storage medium are executed by the processor, the steps of the binocular camera calibration method as described above are implemented.

[0101] The technical solution of the above-mentioned binocular camera calibration device includes all the technical solutions of the above-mentioned embodiments, and therefore can at least achieve all the technical effects of the above-mentioned embodiments, which will not be repeated here.

[0102] The preferred specific embodiments of the present invention are described in detail above, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A binocular camera calibration method, characterized in that: include: Acquire multiple images of a calibration plate at different angles, where multiple circular marks with the same radius are evenly arranged in rows and columns on the calibration plate; According to the calibration plate, taking the center of the circular mark in the upper left corner as the origin, record the center coordinates of each circular mark in the world coordinate system; Based on the Canny edge extraction algorithm, edge extraction is performed on the circular mark of the calibration plate image to obtain the coordinates of the center of the circle in the pixel coordinate system; According to the determined center coordinates of the circular mark, the initial values ​​of the camera calibration parameters are obtained by substituting the equation of the homography matrix constraint condition into the equation; The bundle adjustment method is used to optimize the initial values ​​of the camera calibration parameters.

2. A binocular camera calibration method as claimed in claim 1, characterized in that: The method of extracting the edge of the circular mark of the calibration plate image based on the Canny edge extraction algorithm to obtain the coordinates of the center of the circle in the pixel coordinate system includes: Preprocessing the acquired calibration plate image based on a Gaussian filtering algorithm; Calculate the gradient value of each pixel on the calibration plate image to obtain a gradient image corresponding to the calibration plate image; According to the acquired gradient image, the image edge amplitude and angle are calculated, and non-maximum suppression is performed along the gradient direction; A double threshold algorithm is used on the gradient image to detect and connect edges.

3. A binocular camera calibration method as claimed in claim 2, characterized in that: According to the gradient image, the image edge amplitude and angle are calculated, and non-maximum suppression is performed along the gradient direction, including: The gradient image is subjected to calculation using Soble horizontal and vertical operators and Gaussian convolution kernel to obtain the amplitude and angle of the image gradient; The amplitude is non-maximum suppressed according to the image gradient angle, the maximum value of the gradient intensity at each pixel is retained, and other values ​​are filtered out to make the blurred boundary clear.

4. A binocular camera calibration method as claimed in claim 2, characterized in that: The gradient image is subjected to edge detection and connection using a double threshold algorithm, including: Select strong and weak thresholds, mark points with gradient amplitudes lower than the weak threshold as weak edge points, and points with gradient amplitudes greater than the strong threshold as strong edge points; For points whose gradient amplitude is greater than the weak threshold but less than the high threshold, it is determined whether there are points greater than the strong threshold in its neighborhood. If so, it is retained and marked as a strong edge point. If not, it is discarded and marked as a weak edge point.

5. A binocular camera calibration method as claimed in claim 1, characterized in that: According to the determined center coordinates of the circular mark, substitute the homography constraint matrix equation to solve the initial values ​​of the camera parameters, including: According to the correspondence between the object's coordinate points in the three-dimensional world and the imaging plane, the homography matrix equation is established; According to the center coordinates of each circular mark in the world coordinate system and the center coordinates in the pixel coordinate system, the internal and external parameters of the binocular camera are solved.

6. A binocular camera calibration method as claimed in claim 1, characterized in that: The initial values ​​of the camera parameters are optimized using a bundle adjustment method, including: Establishing an overall optimization equation according to the initial values ​​of the internal and external parameters of the binocular camera; According to the overall optimization equation, the reprojection error is minimized.

7. A binocular camera calibration device, characterized in that: include: An image acquisition module is used to acquire multiple calibration plate images at different angles; An image processing module is used to extract the edges of each circular mark of the calibration plate image and obtain the coordinates of the center of the circle in a pixel coordinate system; The data operation module is used to substitute the center coordinates of the circular mark into the homography constraint matrix equation and the optimization equation to solve the camera calibration parameters.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program or instruction, and when the program or instruction is executed by the processor, the steps of the binocular camera calibration method as described in any one of claims 1 to 6 are implemented.

9. A binocular camera calibration electronic device, the camera calibration device comprising: A processor, a memory, and data stored in the memory and operable on the processor, wherein the memory, the processor, and a readable storage medium where the computer program is located can be connected via a system bus, and when the data, program, or instruction is executed by the processor, the steps of the binocular camera calibration method as described in any one of claims 1 to 6 are implemented.

Images