Binocular camera calibration method and system, computer equipment and storage medium

Abstract

The invention provides a binocular camera calibration method and system, computer equipment and a storage medium, and belongs to the field of three-dimensional vision measurement, and the method comprises the steps: constructing a calibration plate, arranging circular marks in array distribution on the calibration plate, embedding a randomly distributed Gaussian speckle pattern in the center of each circular mark, and enabling the center of each circular mark to be a control point; shooting calibration plate images at different angles by using a binocular camera, and obtaining two-dimensional coordinate initial values of a plurality of control points on the calibration plate images; based on the Gaussian speckle pattern, optimizing the two-dimensional coordinate initial values of the plurality of control points by using a speckle matching method; and constructing a homography matrix H through the optimized two-dimensional coordinates of the plurality of control points and the optimized three-dimensional coordinates of the control points, and estimating internal parameters of the camera and external parameters of the camera relative to the calibration plate by using the homography matrix. According to the method, by introducing the speckle matching technology, the influence of eccentric errors is remarkably reduced, the control point matching precision is greatly improved, and therefore the accuracy of high-precision three-dimensional calibration is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional vision measurement, and in particular relates to a binocular camera calibration method, system, computer equipment and storage medium. Background Art

[0002] High-precision 3D vision measurement is widely used in industrial fields such as quality inspection and automated manufacturing. Binocular camera calibration is a key step in ensuring the accuracy of spatial data in 3D vision measurement. By providing accurate spatial information, the binocular vision system can better evaluate geometric features and surface defects, thereby improving the reliability and efficiency of quality control. Since the accuracy of 3D reconstruction is highly dependent on the accuracy of calibration, improving the accuracy of calibration is crucial to the consistency of measurement results in industrial applications.

[0003] The key to high-precision calibration of a binocular system is to establish an accurate correspondence between 3D control points and 2D image points. Common methods usually use a flat calibration plate with a specific pattern, such as a checkerboard or circular control points. The checkerboard is widely used because of its simple implementation, but its edge intersection detection method is sensitive to noise. Circular control points can provide higher sub-pixel accuracy and are more robust to noise, so they are more suitable for high-precision calibration tasks.

[0004] However, circular control points have eccentricity errors, that is, the geometric center obtained by ellipse fitting may deviate from the true projection of the physical center of the control point. This error can lead to: (1) inaccurate correspondence between control points in different images; (2) deviations between the true projection of the 3D control point and the image detection point. In addition, errors in the manufacture of the calibration plate can further aggravate these problems and affect the calibration accuracy.

[0005] Several studies have proposed various solutions to the eccentricity error problem. For example, there are methods that generate unbiased estimates of linear transformations through conic transformations; there are also methods that use constraints in multi-view geometry to optimize the projection of the center of the circle. However, these methods fail to completely solve the distortion problem caused by nonlinear transformations, and some methods rely on iterative optimization, are sensitive to initial values, and are prone to convergence problems. In addition, these methods usually assume that the coordinates of the three-dimensional control points are completely accurate, ignoring the impact of the manufacturing errors of the calibration plate. Summary of the invention

[0006] In order to solve the above problems, the present invention provides a binocular camera calibration method. The method proposes a speckle-enhanced circular calibration plate, combines digital image correlation technology and an optimization algorithm based on bundle adjustment, embeds Gaussian speckle patterns in circular control points, uses speckle matching technology to improve the accuracy of control point matching, and compensates for the calibration plate manufacturing error and eccentricity error through a bundle adjustment algorithm, guiding the algorithm to converge more robustly, thereby significantly improving the accuracy and reliability of calibration and meeting the needs of high-precision three-dimensional visual measurement.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] A binocular camera calibration method, comprising:

[0009] Constructing a calibration plate, on which circular marks distributed in an array are arranged, a randomly distributed Gaussian speckle pattern is embedded in the center of each circular mark, and the center of each circular mark is a control point;

[0010] Use a binocular camera to capture images of the calibration plate at different angles, and obtain the initial two-dimensional coordinate values ​​of multiple control points on the calibration plate image; Based on the Gaussian speckle pattern, use the speckle matching method to optimize the initial two-dimensional coordinate values ​​of multiple control points;

[0011] The homography matrix H is constructed by optimizing the two-dimensional coordinates of multiple control points and the three-dimensional coordinates of the control points, and the homography matrix is ​​used to estimate the camera intrinsic parameters and the extrinsic parameters of the camera relative to the calibration plate.

[0012] Preferably, several circular marks on the calibration plate are set as coding points for decoding, and the remaining circular marks are set as non-coding points; the radius of the coding point is larger than the radius of the non-coding point.

[0013] Preferably, before obtaining the initial values ​​of the two-dimensional coordinates of the plurality of control points on the calibration plate image, the method further includes:

[0014] The sub-pixel edge detection algorithm is used to extract the edge coordinates of the circular mark on the calibration plate image, and the center of the circular mark is estimated as the control point by fitting the edge coordinates of the ellipse.

[0015] Based on the control points and several coded points, the non-coded points are decoded according to a predefined coding scheme, and a unique identifier is assigned to each control point;

[0016] Matching is performed based on the control points with coded points and assigned unique identifiers, and the initial values ​​of the two-dimensional coordinates of multiple control points on the calibration plate image are calculated based on the matching results.

[0017] Preferably, the camera internal parameters include focal length f x 、f y and the principal point coordinates (c x , cy ), the external parameters of the camera relative to the calibration plate include the rotation matrix R and the translation vector t. The relative pose is calculated by the external parameters of the camera relative to the calibration plate, as follows:

[0018]

[0019] Among them, R l , R r Respectively represent the rotation matrix of the world coordinate system to the left and right camera coordinate systems, t l ,t r Respectively represent the translation vector from the world coordinate system to the left and right camera coordinate systems.

[0020] Preferably, it also includes using a bundle adjustment algorithm to optimize the camera parameters and the three-dimensional coordinates of the control points to minimize the reprojection error, and the optimization objective function is as follows:

[0021]

[0022] In the formula, the value with * indicates the value obtained after optimization; the intrinsic parameters of the left camera and the right camera are expressed as Where K l and K r are the intrinsic parameter matrices of the left and right cameras respectively, and the distortion coefficient is represented as a vector D = [k1 k2 p1 p2 k3] T , where k1, k2, k3 are radial distortion coefficients, p1, p2 are tangential distortion coefficients; the distortion coefficient vector D l is the distortion coefficient of the left camera, D r D for the right camera r ; Each of these represents the external parameter of the i-th image of the left camera, describing the position of the calibration plate relative to the left camera, and N represents the number of images taken;

[0023] M i represents the number of control points identified on the i-th calibration plate; T = [R | t], represents the external parameter matrix; T * represents the best external parameter matrix obtained after optimization, P represents the coordinate set of the three-dimensional control points, P * represents the set of three-dimensional control point coordinates obtained after optimization, represents the two-dimensional image coordinates of the jth control point on the i-th image from the left perspective, represents the two-dimensional image coordinates of the jth control point on the i-th image from the right perspective, D l represents the distortion coefficient vector of the left camera, P j represents the jth three-dimensional control point, D rRepresents the distortion coefficient vector of the right camera.

[0024] Preferably, the camera is further calibrated, and the scale of the scale calibration includes two real known distances d i The optimization objective of scale correction is defined as follows:

[0025]

[0026] Where s is the scale factor to be optimized, N is the number of images taken, and the function φ(·) represents the triangulation process, which is used to calculate the 3D coordinates from the pixel coordinates in the left and right images. and are the pixel coordinates of the two markers on the ruler in the left camera image, and and are the pixel coordinates of the two marker points on the ruler in the right camera image.

[0027] The present invention also proposes a binocular camera calibration system, comprising:

[0028] A calibration plate, wherein the calibration plate is provided with circular marks distributed in an array, a randomly distributed Gaussian speckle pattern is embedded in the center of each circular mark, and the center of each circular mark is a control point;

[0029] The coordinate acquisition module is used to use a binocular camera to capture calibration plate images at different angles and obtain the initial two-dimensional coordinate values ​​of multiple control points on the calibration plate image; based on the Gaussian speckle pattern, the initial two-dimensional coordinate values ​​of multiple control points are optimized using a speckle matching method;

[0030] The parameter calibration module is used to construct the homography matrix H through the optimized two-dimensional coordinates of multiple control points and the three-dimensional coordinates of the control points, and use the homography matrix to estimate the camera intrinsic parameters and the external parameters of the camera relative to the calibration plate.

[0031] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any one of the steps in the binocular camera calibration method.

[0032] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is loaded by a processor, it can execute any one of the steps in the binocular camera calibration method.

[0033] The binocular camera calibration method provided by the present invention has the following beneficial effects:

[0034] The present invention proposes a novel speckle enhanced circular calibration plate, which directly embeds the speckle pattern into the circular mark. Since the speckle pattern itself has certain grayscale characteristics and texture information, the speckle pattern can still maintain good recognizability in a relatively complex or uneven lighting environment. Therefore, by setting the speckle pattern on the circular mark, the flexibility of the circular mark is retained, and a robust initialization is provided for speckle matching. The introduction of the speckle matching technology significantly reduces the influence of the eccentricity error, greatly improves the accuracy of the control point matching, and thus makes the two-dimensional coordinates optimized by the speckle matching method based on the Gaussian speckle pattern more accurate. The internal and external parameters of the camera can be accurately estimated through the optimized two-dimensional coordinates and three-dimensional coordinates, thereby improving the accuracy of high-precision stereo calibration. At the same time, the generation of the speckle pattern is relatively easy, and it can be directly projected on the surface of the object or printed on materials such as paper, without the need for complex installation and fixing devices. The use of a calibration plate with a speckle pattern makes the stereo calibration independent of the high-precision calibration target, thereby reducing the dependence on expensive equipment, reducing the cost and complexity of the system calibration, and meeting the needs of high-precision three-dimensional visual measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0036] Figure 1 This is a flow chart of a binocular camera calibration method according to Embodiment 1 of the present invention;

[0037] Figure 2 Schematic diagram of a binocular camera calibration method according to Embodiment 1 of the present invention;

[0038] Figure 3 This is the structural diagram of the calibration board. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0040] Example 1

[0041] The present invention provides a binocular camera calibration method to solve the challenge caused by eccentricity error in circular marker detection. By embedding Gaussian speckle patterns into circular markers, the speckle matching technology is used to improve the matching accuracy of markers between different images. At the same time, in order to solve the matching error between control point projection and detection, the present invention adopts the bundled adjustment technology to jointly optimize the camera parameters and 3D control point coordinates, thereby compensating for eccentricity error and manufacturing defects.

[0042] Specific as Figure 1 and Figure 2 As shown, the stereo calibration method includes the following steps:

[0043] Step 1: Construct a speckle enhancement calibration plate to form circular control points based on speckle enhancement.

[0044] In order to solve the problem that traditional circular control points are prone to eccentricity errors, the present invention designs a new pattern that embeds a Gaussian speckle pattern in the center of the circular control point. This speckle-enhanced circular control point provides an innovative solution for the precise detection of two-dimensional coordinates during the calibration process. By using Gaussian speckle pattern matching technology, this method significantly improves the accuracy of control point detection. Figure 3 As shown, the calibration plate proposed by the present invention is provided with circular marks consisting of concentric circle grids distributed in an N array, and the center of each circular mark is embedded with a randomly distributed Gaussian speckle pattern, which increases the feature details and uniqueness. Among them, nine circular marks with larger radius are used as encoding points for decoding. In this embodiment, the calibration plate is composed of 11×17 concentric circle grids, and the center of each circular mark is a control point.

[0045] Step 2: Eccentricity-aware control point matching based on speckle matching.

[0046] The eccentricity error may lead to deviations in the matching of control points in different images. To solve this problem, the present invention proposes a control point matching method based on eccentricity perception, combined with the speckle enhancement calibration plate proposed in step 1, firstly generates the initial values ​​of the two-dimensional coordinates of the control points by ellipse fitting, and then optimizes the initial values ​​by Gaussian speckle pattern matching, thereby improving the accuracy and robustness of control point detection.

[0047] Specifically, the present invention uses a binocular camera to shoot calibration plate images at different angles, adopts a sub-pixel edge detection algorithm to extract edge coordinates of a circular mark on the calibration plate image, and estimates the center of the circular mark as a control point by ellipse fitting edge coordinates; based on the control points and a number of coded points, non-coded points are decoded according to a predefined coding scheme, and a unique identifier is assigned to each control point; matching is performed according to control points with coded points and assigned unique identifiers, and the initial values ​​of the two-dimensional coordinates of multiple control points on the calibration plate image are calculated according to the matching results; based on the Gaussian speckle pattern, the initial values ​​of the two-dimensional coordinates of multiple control points are optimized by using a speckle matching method.

[0048] Next, control point matching is further explained.

[0049] (1) Use a binocular camera to capture images of the calibration plate at different angles and perform circle center detection for initializing speckle matching. In order to provide accurate initial conditions for the speckle matching process, the center of the circular control point is first detected.

[0050] The process improves detection accuracy through the following steps: First, a sub-pixel edge detection algorithm is used to extract the edge coordinates of the circular mark on the calibration plate image, and the center of the circular mark is estimated by fitting the edge coordinates of the ellipse. Specifically, after the image is Gaussian filtered to smooth the noise, the gradient amplitude and direction are calculated to provide key information for edge detection. Subsequently, the edge points are accurately located through non-maximum suppression and edge tracking based on hysteresis threshold, and a quadratic function is fitted around each edge pixel to achieve sub-pixel precision positioning. Once the center of the ellipse is determined, the present invention decodes the circular control points according to a predefined coding scheme and assigns a unique identifier to each control point to ensure its consistent matching in multiple views. After decoding is completed, the homography transformation between the reference image and the target image is calculated using the identified control points. This transformation provides an initial estimate of the geometric relationship between the two images, laying the foundation for the subsequent calculation of the deformation function.

[0051] (2) Speckle matching. In the speckle matching process, the first image of the left camera is designated as the reference image. The coordinates of the center of the ellipse of the circular control point in the reference image are used as the key points to be matched, and the coordinates of the center of the ellipse in the target image are used as the initial values ​​for matching. After the initial correspondence is established, the matching process is further optimized by the ICGN algorithm. The algorithm iteratively minimizes the grayscale difference of the small area of ​​the control point in the reference image and the target image, gradually updates the deformation parameters, and achieves sub-pixel precise matching. After completing the speckle matching, the corresponding points in the target image are precisely located and accurately matched with the points in the reference image to obtain the optimized two-dimensional coordinates.

[0052] Step 3: Initial estimates of calibration parameters.

[0053] In stereo camera calibration, the initial estimation of intrinsic and extrinsic parameters is crucial because nonlinear optimization algorithms are highly sensitive to initial values. Accurate initial estimation can not only accelerate optimization convergence, but also avoid falling into local minima, thereby improving calibration accuracy and efficiency. Conversely, inaccurate initial values ​​may cause the optimization to be affected by noise and distortion, resulting in suboptimal results.

[0054] The present invention constructs a homography matrix H through the optimized two-dimensional coordinates of multiple control points and the three-dimensional coordinates of the control points, and uses the homography matrix to estimate the camera intrinsic parameters and the camera extrinsic parameters relative to the calibration plate. The coordinate system of the three-dimensional coordinates is the calibration plate plane coordinate system, the origin is the upper left corner of the calibration plate plane, and the actual physical spacing of the control points on the calibration plate can be set by itself, so the three-dimensional coordinates are known.

[0055] Specifically, the calibration process is as follows: Figure 1 As shown in step 3, first estimate the intrinsic parameters and extrinsic parameters of each monocular camera respectively, and then estimate the relative pose parameters between cameras.

[0056] (1) Initial estimation of monocular camera parameters. For each monocular camera, the homography matrix H can be calculated from the 2D and 3D coordinates of the control points in the multi-view images of the calibration plate. The homography matrix can be used to preliminarily estimate the camera intrinsic parameters, including the focal length f. x 、f y and the principal point coordinates (c x ,c y ).

[0057] (2) Initial estimation of binocular camera extrinsic parameters. After the initial estimation of the monocular camera parameters, the extrinsic parameters of the camera relative to the calibration plate are estimated, and the relative pose between the two cameras is determined by the camera extrinsic parameters. The relative pose is represented by the rotation matrix R and the translation vector t, that is, the relative pose can be calculated by the extrinsic parameters of the left and right cameras estimated in the multi-view calibration target, and the formula is as follows:

[0058]

[0059] Among them, R l , R r Respectively represent the rotation matrix of the world coordinate system to the left and right camera coordinate systems, t l ,t r Respectively represent the translation vector from the world coordinate system to the left and right camera coordinate systems.

[0060] Due to noise and calibration errors, the relative poses obtained under different views may be different. To obtain a robust estimate, the rotation matrix is ​​first converted to a rotation vector, the average value is calculated, and then converted back to a rotation matrix to ensure orthogonality; the translation vector is directly averaged. This method effectively reduces the impact of noise and inconsistency and provides a stable relative pose estimate.

[0061] These initial estimates provide a reliable starting point for the subsequent dual-objective calibration step, ensuring that the optimization process can proceed smoothly based on good initial values.

[0062] Step 4: Use the bundle adjustment algorithm to optimize the camera parameters and control points, and perform scale correction on the camera.

[0063] The initial camera parameter estimation assumes perfect accuracy of the 3D control points and their 2D projections, but these points may be affected by factors such as eccentricity errors and manufacturing defects. To address these issues, Figure 1 As shown in step 4, the present invention uses bundle adjustment to optimize the camera parameters and 3D control points by minimizing the reprojection error. That is, after the initial estimation is completed, the intrinsic parameters, extrinsic parameters and distortion coefficients are further optimized by minimizing the reprojection error. The reprojection error refers to the difference between the observed image point and the point projected onto the image plane by the 3D world coordinates, and its optimization objective function is defined as:

[0064]

[0065] In the above formula, the value with an asterisk indicates the value obtained after optimization; K represents the intrinsic parameter matrix of the camera, and D represents the distortion coefficient. represents the external parameter set, p i,j represents the observed two-dimensional coordinates of the jth point in the i-th image, P j is the corresponding 3D world point. The function π(·) models the process of projecting a 3D point onto the image plane through the camera’s intrinsic parameters, extrinsic parameters, and distortion. The optimization goal is to minimize the difference between the observed image point and the projected point by adjusting the intrinsic parameters, distortion coefficients, and extrinsic parameters.

[0066] (1) Adjust the bundle of three-dimensional points. The optimization objective function of the bundle adjustment is as follows:

[0067]

[0068] The optimization function minimizes the reprojection error by adjusting the camera parameters and 3D control points. In the formula, the values ​​marked with * represent the values ​​obtained after optimization; the intrinsic parameters of the left and right cameras are expressed as Where K l and K r are the intrinsic parameter matrices of the left camera and the right camera respectively, and the distortion coefficient is represented as a vector D = [k1k2p1p2k3] T , where k1, k2, k3 are radial distortion coefficients, p1, p2 are tangential distortion coefficients; the distortion coefficient vector D l is the distortion coefficient of the left camera, D rD for the right camera r ; Each of these represents the external parameter of the i-th image of the left camera, describing the position of the calibration plate relative to the left camera, and N represents the number of images taken;

[0069] M i represents the number of control points identified on the i-th calibration plate; T = [R | t], represents the external parameter matrix; T * represents the best external parameter matrix obtained after optimization, P represents the coordinate set of the three-dimensional control points, P * represents the set of three-dimensional control point coordinates obtained after optimization, represents the two-dimensional image coordinates of the jth control point on the i-th image from the left perspective, represents the two-dimensional image coordinates of the jth control point on the i-th image from the right perspective, D l represents the distortion coefficient vector of the left camera, P j represents the jth three-dimensional control point, D r Represents the distortion coefficient vector of the right camera.

[0070] By compensating for the manufacturing error and eccentricity error of the calibration plate through the bundled adjustment algorithm, high-precision stereo system calibration can be achieved without relying on high-precision calibration targets, thereby reducing dependence on expensive equipment, reducing the cost and complexity of system calibration, and meeting the needs of high-precision three-dimensional visual measurement.

[0071] (2) Accurate scale correction. The bundle adjustment algorithm can accurately determine the intrinsic parameters and relative rotation, but there is ambiguity in the relative translation. The estimated translation parameters lack accurate scale information and their scale factors are unknown. To solve this problem, the present invention introduces scale factor correction to ensure that the camera calibration results are consistent with the size of the real world, thereby improving the calibration accuracy.

[0072] The present invention introduces a calibrated scale, such as Figure 1 As shown in step 4 in , the ruler includes two marking points with known precise distances. The present invention captures images of the ruler from different positions and angles to perform scale correction. The optimization target of scale correction is defined as follows:

[0073]

[0074] In the above optimization formula, s is the scale factor to be optimized, N is the number of images taken, and the function φ(·) represents the triangulation process used to calculate the three-dimensional coordinates from the pixel coordinates in the left and right images. Specifically, and are the pixel coordinates of the two markers on the ruler in the left camera image, and and are the pixel coordinates of the two marker points on the ruler in the right camera image.

[0075] The optimization goal is to minimize the distance calculated after scaling and the actual known distance d i The difference between them can be used to achieve accurate scale correction.

[0076] Simulation experiment: The performance of the proposed method in the dual-target calibration system was verified experimentally. Specifically, the performance of the classic Zhang Zhengyou calibration method and the proposed method were compared by generating five sets of calibration plate simulation data. The performance of each method was evaluated based on two key indicators: reprojection error and control point reconstruction accuracy. The quantitative results in Tables 1 and 2 show that the proposed method has significantly improved performance compared with the Zhang Zhengyou calibration method.

[0077] Table 1 Comparison of RMS reprojection errors of Zhang Zhengyou's calibration method and the method proposed in this invention on different simulation data sets (unit: pixel)

[0078]

[0079] Table 2 Comparison of the average reconstruction error of control points between Zhang Zhengyou's calibration method and the method proposed in this invention on different simulation data sets (unit: mm)

[0080]

[0081] Compared with the prior art, the binocular camera calibration method proposed in this invention has the following advantages:

[0082] 1. Reduce the impact of eccentricity error of circular control points:

[0083] Traditional circular calibration plates are easily affected by eccentricity errors during the calibration process, resulting in a decrease in the accuracy of control point matching. The present invention significantly reduces the impact of eccentricity errors by introducing speckle matching technology, thereby improving the calibration accuracy. Digital image correlation technology is used to improve the accuracy of control point matching, among which speckle matching has been shown to have a significant effect in establishing accurate correspondences. Although previous studies have explored the method of applying DIC in camera calibration by synthesizing Gaussian speckle patterns, the uniqueness of the method of the present invention is that the Gaussian speckle pattern is directly embedded in the circular marker. This combination retains the flexibility of the circular marker and provides a robust initialization for speckle matching. The influence of eccentricity errors is significantly reduced through speckle matching, thereby significantly improving the overall accuracy. This improvement effectively improves the control point matching accuracy, thereby improving the overall accuracy of high-precision stereo calibration.

[0084] 2. Improve calibration accuracy and robustness:

[0085] The speckle-enhanced circular calibration plate designed in the present invention is combined with speckle matching technology, which not only improves the accuracy of control point matching, but also enhances the robustness of the calibration system in complex environments, and is suitable for high-precision stereo calibration needs.

[0086] 3. Reduce dependence on high-precision equipment:

[0087] The prior art usually needs to rely on high-precision and expensive calibration targets or equipment to achieve high-precision calibration. The present invention, through innovative calibration plate design and stereo calibration algorithm, can achieve accurate calibration without the need for high-precision calibration targets, and can achieve accurate stereo system calibration without relying on high-precision calibration targets, thereby reducing dependence on expensive equipment, thereby effectively reducing equipment costs and usage thresholds.

[0088] 4. Wider applicability:

[0089] The calibration plate and algorithm of the present invention are not only applicable to traditional stereo calibration scenarios, but can also maintain high efficiency and stability in a variety of complex environments and different stereo vision systems, and have broader application prospects.

[0090] In summary, the present invention overcomes the limitations of the prior art through innovative calibration plate design and algorithm proposal, and provides a stereo calibration solution with higher accuracy, lower cost and greater applicability.

[0091] Based on the same inventive concept, the present invention also proposes a binocular camera calibration system, which includes:

[0092] A calibration plate, wherein the calibration plate is provided with circular marks distributed in an array, a randomly distributed Gaussian speckle pattern is embedded in the center of each circular mark, and the center of each circular mark is a control point;

[0093] The coordinate acquisition module is used to use a binocular camera to capture calibration plate images at different angles and obtain the initial two-dimensional coordinate values ​​of multiple control points on the calibration plate image; based on the Gaussian speckle pattern, the initial two-dimensional coordinate values ​​of multiple control points are optimized using a speckle matching method;

[0094] The parameter calibration module is used to construct the homography matrix H through the optimized two-dimensional coordinates of multiple control points and the three-dimensional coordinates of the control points, and use the homography matrix to estimate the camera intrinsic parameters and the external parameters of the camera relative to the calibration plate. Each module in the above-mentioned binocular camera calibration system can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0095] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the embodiment of the binocular camera calibration method. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0096] Furthermore, the present invention also provides a non-temporary computer-readable storage medium containing instructions, and a computer program is stored on the storage medium. For example, a memory containing instructions, the above instructions can be executed by a processor of a computer device to complete the above method. For example, a non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, and an optical data storage device. When the computer program is executed by the processor, the steps in the embodiment of the binocular camera calibration method can be implemented. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0097] It will be appreciated by those skilled in the art that embodiments of the present invention may provide methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0098] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0099] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0100] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0101] It should be pointed out that the specific implementation methods described above can enable those skilled in the art to understand the invention more comprehensively, but do not limit the invention in any way. Therefore, although the invention has been described in detail in this specification and embodiments, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved. Any simple change or equivalent replacement of the technical solution that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belongs to the protection scope of the present invention.

Claims

1. A binocular camera calibration method, characterized in that: include: Constructing a calibration plate, on which circular marks distributed in an array are arranged, a randomly distributed Gaussian speckle pattern is embedded in the center of each circular mark, and the center of each circular mark is a control point; Use a binocular camera to capture images of the calibration plate at different angles, and obtain the initial two-dimensional coordinate values ​​of multiple control points on the calibration plate image; Based on the Gaussian speckle pattern, use the speckle matching method to optimize the initial two-dimensional coordinate values ​​of multiple control points; The homography matrix H is constructed by optimizing the two-dimensional coordinates of multiple control points and the three-dimensional coordinates of the control points, and the homography matrix is ​​used to estimate the camera intrinsic parameters and the extrinsic parameters of the camera relative to the calibration plate.

2. The binocular camera calibration method according to claim 1, characterized in that: Several circular marks on the calibration plate are set as coding points for decoding, and the remaining circular marks are set as non-coding points; the radius of the coding point is greater than the radius of the non-coding point.

3. The binocular camera calibration method according to claim 2, characterized in that: Before obtaining the initial values ​​of the two-dimensional coordinates of multiple control points on the calibration plate image, it also includes: The sub-pixel edge detection algorithm is used to extract the edge coordinates of the circular mark on the calibration plate image, and the center of the circular mark is estimated as the control point by fitting the edge coordinates of the ellipse. Based on the control points and several coded points, the non-coded points are decoded according to a predefined coding scheme, and a unique identifier is assigned to each control point; Matching is performed based on the control points with coded points and assigned unique identifiers, and the initial values ​​of the two-dimensional coordinates of multiple control points on the calibration plate image are calculated based on the matching results.

4. The binocular camera calibration method according to claim 1, characterized in that: The camera internal parameters include focal length f x 、f y and the principal point coordinates (c x ,c y ), the external parameters of the camera relative to the calibration plate include the rotation matrix R and the translation vector t. The relative pose is calculated by the external parameters of the camera relative to the calibration plate, as follows: Among them, R l , R r Respectively represent the rotation matrix of the world coordinate system to the left and right camera coordinate systems, t l ,t r Respectively represent the translation vector from the world coordinate system to the left and right camera coordinate systems.

5. The binocular camera calibration method according to claim 4, characterized in that: It also includes using the bundle adjustment algorithm to optimize the camera parameters and the three-dimensional coordinates of the control points to minimize the reprojection error. The optimization objective function is as follows: In the formula, the value with * indicates the value obtained after optimization; the intrinsic parameters of the left camera and the right camera are expressed as Where K l and K r are the intrinsic parameter matrices of the left and right cameras respectively, and the distortion coefficient is represented as a vector D = [k1 k2 p1 p2 k3] T , where k1, k2, k3 are radial distortion coefficients, p1, p2 are tangential distortion coefficients; the distortion coefficient vector D l is the distortion coefficient of the left camera, D r D for the right camera r ; Each of these represents the external parameter of the i-th image of the left camera, describing the position of the calibration plate relative to the left camera, and N represents the number of images taken; M i represents the number of control points identified on the ith calibration plate; T = [R|t], represents the external parameter matrix; T * represents the best external parameter matrix obtained after optimization, P represents the coordinate set of the three-dimensional control points, P * represents the set of three-dimensional control point coordinates obtained after optimization, represents the two-dimensional image coordinates of the jth control point on the i-th image from the left perspective, represents the two-dimensional image coordinates of the jth control point on the i-th image from the right perspective, D l represents the distortion coefficient vector of the left camera, P j represents the jth three-dimensional control point, D r Represents the distortion coefficient vector of the right camera.

6. The binocular camera calibration method according to claim 5, characterized in that: It also includes scale calibration of the camera. The scale calibration ruler contains two real known distances d i The optimization objective of scale correction is defined as follows: Where s is the scale factor to be optimized, N is the number of images taken, and the function φ(·) represents the triangulation process, which is used to calculate the 3D coordinates from the pixel coordinates in the left and right images. and are the pixel coordinates of the two markers on the ruler in the left camera image, and and are the pixel coordinates of the two marker points on the ruler in the right camera image.

7. A binocular camera calibration system, characterized in that: include: A calibration plate, wherein the calibration plate is provided with circular marks distributed in an array, a randomly distributed Gaussian speckle pattern is embedded in the center of each circular mark, and the center of each circular mark is a control point; The coordinate acquisition module is used to use a binocular camera to capture calibration plate images at different angles and obtain the initial two-dimensional coordinate values ​​of multiple control points on the calibration plate image; based on the Gaussian speckle pattern, the initial two-dimensional coordinate values ​​of multiple control points are optimized using a speckle matching method; The parameter calibration module is used to construct the homography matrix H through the optimized two-dimensional coordinates of multiple control points and the three-dimensional coordinates of the control points, and use the homography matrix to estimate the camera intrinsic parameters and the external parameters of the camera relative to the calibration plate.

8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is loaded into a processor, it can execute the steps of the method according to any one of claims 1 to 6.

Images