Camera calibration method and system, and computer-readable storage medium

Abstract

A camera calibration method and system, and a computer-readable storage medium, comprising: obtaining a calibration plate image captured by a camera and the world coordinates of the spatial points corresponding to all pixels in the calibration plate image; calculating a homography matrix; calculating the camera's intrinsic parameters based on the constraint relationship between the homography matrix and the camera's intrinsic parameters; calculating the camera coordinates of the spatial points corresponding to the pixels based on the obtained intrinsic parameters; determining whether all spatial points are coplanar; if so, moving the origin of the world coordinate system to the center of gravity of the spatial points, and calculating the camera's extrinsic parameters based on the transformation relationship between the world coordinate system and the camera coordinate system after the movement; if not, solving the camera's extrinsic parameters based on the correspondence between the camera coordinates and the world coordinates of the spatial points; and globally optimizing the camera's intrinsic parameters, extrinsic parameters, and distortion coefficients. This method can adapt to situations where the spatial points corresponding to the pixels are coplanar or non-coplanar, and can improve the accuracy of camera calibration, obtaining more accurate intrinsic parameters, extrinsic parameters, and distortion coefficients.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a camera calibration method and system, and a computer-readable storage medium. Background Art

[0002] In image measurement and machine vision applications, determining the three-dimensional geometric position of a point on the surface of a spatial object requires establishing a geometric model of the camera image. This involves determining the correspondence between the three-dimensional geometric position of a point on the surface and its corresponding point in the image. Once the image coordinates of the image captured by the camera are obtained, the corresponding three-dimensional spatial coordinates can be inferred based on the geometric model of the camera image. The parameters in the geometric model are the camera parameters, and the process of determining these parameters is called camera calibration. Camera parameter calibration is a critical step. The accuracy of the calibration results and the stability of the calibration algorithm directly impact the accuracy of the camera's results. Therefore, effective camera calibration is a prerequisite for subsequent work. Improved camera calibration methods are necessary to improve calibration accuracy and obtain more accurate camera parameters. Camera calibration is often performed using calibration plates, which are widely used in machine vision, image measurement, photogrammetry, and 3D reconstruction. By capturing an image of a calibration plate with a fixed-pitch pattern array, the camera's geometric model can be calculated using the calibration algorithm, resulting in highly accurate measurement and reconstruction results. At present, calibration plates with checkerboard patterns or solid circle array patterns are usually used for camera calibration. The checkerboard calibration plate obtains feature points by locating the checkerboard corners, and the circle array calibration plate obtains feature points by locating the centers of the circles. After determining the coordinates of the feature points and their correspondence with the world coordinates, subsequent calibration work can be carried out. Summary of the Invention

[0003] The present invention provides a camera calibration method and system, and a computer-readable storage medium, aiming to improve the accuracy of camera calibration and obtain more accurate camera intrinsic parameters, extrinsic parameters, and distortion coefficients.

[0004] According to the first aspect, an embodiment provides a camera calibration method, including:

[0005] Get the calibration plate image taken by the camera;

[0006] Obtain the world coordinates of the spatial points corresponding to all pixels in the calibration plate image, and obtain the feature points in the calibration plate image, as well as the image coordinates of the feature points;

[0007] Calculate the homography matrix H according to the image coordinates of the feature points and the world coordinates of the corresponding spatial points a ;

[0008] According to the constraint relationship between the homography matrix and the camera intrinsic parameters, the homography matrix H is used a Calculate the camera's intrinsic parameters;

[0009] Calculate the camera coordinates of the spatial points corresponding to all pixels based on the calculated intrinsic parameters;

[0010] According to the covariance of the world coordinates of the spatial points corresponding to all pixels in the calibration plate image, determine whether the spatial points are coplanar. If they are coplanar, move the origin of the world coordinate system to the center of gravity of the spatial point. According to the camera coordinates of the spatial point corresponding to the pixel point and the world coordinates after the shift, calculate the rotation matrix R of the world coordinate system relative to the camera coordinate system after the shift c and the translation vector t c , according to the rotation matrix R of the world coordinate system relative to the camera coordinate system after the movement c and the translation vector t c Calculate the camera's extrinsic parameters; if they are not coplanar, solve the camera's extrinsic parameters based on the correspondence between the camera coordinates and the world coordinates of the spatial point corresponding to the pixel point;

[0011] According to the preset objective function, the camera's distortion coefficient and the calculated internal and external parameters are globally optimized to obtain the camera's final internal and external parameters and distortion coefficient.

[0012] In one embodiment, the intrinsic parameters of the camera include the equivalent focal length f x and f y ; The method of calculating the camera's intrinsic parameters using the homography matrix according to the constraint relationship between the homography matrix and the camera's intrinsic parameters includes:

[0013] According to the following formula, the homography matrix H a Calculate the homography matrix H b :

[0014] H0=h0-h6c x , H1=h1-h7c x , H2=h2-h8c x ,

[0015] H3=h3-h6c y , H4=h4-h7c y , H5=h5-h8c y ,

[0016] H6=h6, H7=h7, H8=h8,

[0017] Where (c x ,c y ) are the coordinates of the camera's principal optical axis point, h0, h1, h2, h3, h4, h5, h6, h7, and h8 are the homography matrices Ha The elements in H0, H1, H2, H3, H4, H5, H6, H7 and H8 are homography matrices H b The elements in

[0018] According to the following constraints, the homography matrix H b Calculate the camera's equivalent focal length f x and f y :

[0019]

[0020] In one embodiment, determining whether the spatial points are coplanar based on the covariance of the world coordinates of the spatial points corresponding to all pixels in the calibration plate image includes:

[0021] Perform singular value decomposition on the covariance matrix of the world coordinates of the spatial points corresponding to all pixels, and arrange the decomposed eigenvalues ​​from large to small as λ1, λ2, and λ3;

[0022] If λ3 / λ1 is less than a set threshold, the spatial points are judged to be coplanar; otherwise, the spatial points are judged to be non-coplanar.

[0023] In one embodiment, the external parameters of the camera include the rotation matrix R of the world coordinate system relative to the camera coordinate system and the translation vector t; the origin of the mobile world coordinate system is moved to the center of gravity of the spatial point, and the rotation matrix R of the world coordinate system relative to the camera coordinate system after the movement is calculated based on the camera coordinates of the spatial point corresponding to the pixel point and the world coordinates after the movement. c and the translation vector t c , according to the rotation matrix R of the world coordinate system relative to the camera coordinate system after the movement c and the translation vector t c Calculate the camera's external parameters, including:

[0024] The rotation matrix R is calculated based on the eigenvector obtained by performing singular value decomposition on the covariance matrix of the world coordinates of the spatial points corresponding to all pixels. v =[v1,v2,v1×v2], where v1 and v2 are the eigenvectors corresponding to eigenvalues ​​λ1 and λ2 respectively;

[0025] The translation vector t is calculated according to the following formula c :

[0026]

[0027] Where (x, y, z) is the world coordinate of the space point corresponding to the pixel point, and is the translation vector tc The elements in

[0028] Move the origin of the world coordinate system to the center of gravity of the space point, and calculate the rotation matrix R according to the following formula c :

[0029]

[0030] Where (x c ,y c ,z c ) is the camera coordinate of the spatial point corresponding to the pixel point, and (x′, y′, z′) is the world coordinate of the spatial point corresponding to the pixel point after moving the origin of the world coordinate system to the center of gravity of the spatial point;

[0031] The rotation matrix R and translation vector t are calculated according to the following formula:

[0032] t=H b t c , R=H b R c .

[0033] In one embodiment, the camera's extrinsic parameters include a rotation matrix R of a world coordinate system relative to a camera coordinate system and a translation vector t; and solving the camera's extrinsic parameters based on the correspondence between the camera coordinates and the world coordinates of a spatial point corresponding to a pixel point includes:

[0034] Calculate the parameters a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a 10 and a 11 :

[0035]

[0036] Where λ is the scale parameter, (x c ,y c ,z c ) is the camera coordinate of the space point corresponding to the pixel point, (x, y, z) is the world coordinate of the space point corresponding to the pixel point, and the rotation matrix Translation vector

[0037] In one embodiment, the camera distortion coefficient and the calculated intrinsic and extrinsic parameters are globally optimized according to a preset objective function to obtain the final intrinsic and extrinsic parameters and distortion coefficient of the camera, including:

[0038] The initial value of the distortion coefficient is set in advance, and the calculated internal and external parameters are used as the initial values ​​of the internal and external parameters. The optimal solution is iterated according to the following objective function to obtain the final internal and external parameters and distortion coefficient of the camera:

[0039]

[0040] where n m is the number of feature points in the calibration plate image, n0 is the number of calibration plate images taken by the camera, q jk is the image coordinate of the jth feature point in the kth calibration plate image, p j is the world coordinate of the spatial point corresponding to the feature point, e k represents the external parameter obtained based on the k-th calibration plate image, f k represents the internal parameter obtained from the kth calibration plate image, K represents the distortion coefficient, φ(p j ,e k ,f k ,K) represents the world coordinate p j The image coordinates are obtained by transforming the obtained intrinsic parameters, extrinsic parameters and distortion coefficients.

[0041] In one embodiment, according to the formula Q l+1 =Q l +δ iteratively solves the optimal solution, where Q l It represents the vector composed of the intrinsic parameters, extrinsic parameters and distortion coefficients of the camera at the lth iteration. δ is determined by the formula Jδ = ε, where ε is the q corresponding to all feature points of all calibration plate images at the current iteration. jk -φ(p j ,e k ,r k ,K) values, the matrix J is specifically

[0042]

[0043] Among them J e Represents the partial derivative of the image coordinates of the feature point with respect to the external parameter, specifically

[0044]

[0045] Where (u, v) is the image coordinate of the feature point, r = [r x ,r y ,r z ] is the equivalent rotation axis based on the rotation matrix R, t=[t x ,t y ,t z ] is the translation vector;

[0046] J e,k Represents the partial derivative of the image coordinates of the feature point in the k-th calibration plate image with respect to the external parameter;

[0047] J kIt represents the partial derivative of the image coordinates of the feature point in the k-th calibration plate image with respect to the intrinsic parameters and distortion coefficients, specifically:

[0048]

[0049] where f x and f y is the equivalent focal length, (c x ,c y ) are the coordinates of the camera's principal optical axis point, and k0, k1, k2, k3, k4, k5, k6, and k7 are distortion coefficients.

[0050] According to a second aspect, an embodiment provides a camera calibration system, including:

[0051] The calibration plate image acquisition module is used to obtain the calibration plate image taken by the camera;

[0052] The image and world coordinate acquisition module is used to obtain the world coordinates of the spatial points corresponding to all pixels in the calibration plate image, and obtain the feature points in the calibration plate image and the image coordinates of the feature points;

[0053] The homography matrix calculation module is used to calculate the homography matrix H based on the image coordinates of the feature points and the world coordinates of the corresponding spatial points. a ;

[0054] The intrinsic parameter calculation module is used to use the homography matrix H according to the constraint relationship between the homography matrix and the camera intrinsic parameters. a Calculate the camera's intrinsic parameters;

[0055] The camera coordinate acquisition module is used to calculate the camera coordinates of the spatial points corresponding to all pixel points based on the calculated intrinsic parameters;

[0056] The external parameter calculation module is used to determine whether the spatial points are coplanar based on the covariance of the world coordinates of the spatial points corresponding to all pixels in the calibration plate image. If they are coplanar, the origin of the world coordinate system is moved to the center of gravity of the spatial point. Based on the camera coordinates of the spatial point corresponding to the pixel point and the world coordinates after the movement, the rotation matrix R of the world coordinate system relative to the camera coordinate system is calculated. c and the translation vector t c , according to the rotation matrix R of the world coordinate system relative to the camera coordinate system after the movement c and the translation vector t c Calculate the camera's extrinsic parameters; if they are not coplanar, solve the camera's extrinsic parameters based on the correspondence between the camera coordinates and the world coordinates of the spatial point corresponding to the pixel point;

[0057] The global optimization module is used to perform global optimization on the camera's distortion coefficient and the calculated intrinsic and extrinsic parameters according to a preset objective function to obtain the camera's final intrinsic and extrinsic parameters and distortion coefficient.

[0058] In one embodiment, the intrinsic parameters of the camera include the equivalent focal length f x and f y ; The internal parameter calculation module is specifically used for:

[0059] According to the following formula, the homography matrix H a Calculate the homography matrix H b :

[0060] H0=h0-h6c x , H1=h1-h7c x , H2=h2-h8c x ,

[0061] H3=h3-h6c y , H4=h4-h7c y , H5=h5-h8c y ,

[0062] H6=h6, H7=h7, H8=h8,

[0063] Where (c x ,c y ) are the coordinates of the camera's principal optical axis point, h0, h1, h2, h3, h4, h5, h6, h7, and h8 are the homography matrices H a The elements in H0, H1, H2, H3, H4, H5, H6, H7 and H8 are homography matrices H b The elements in

[0064] According to the following constraints, the homography matrix H b Calculate the camera's equivalent focal length f x and f y :

[0065]

[0066] According to a third aspect, an embodiment provides a computer-readable storage medium, on which a program is stored. The program can be executed by a processor to implement the camera calibration method described in the first aspect.

[0067] According to the camera calibration method and system and computer-readable storage medium of the above-mentioned embodiment, the method first obtains the calibration plate image taken by the camera, then obtains the world coordinates of the spatial points corresponding to all pixels in the calibration plate image, and obtains the feature points in the calibration plate image, as well as the image coordinates of the feature points, calculates the homography matrix according to the image coordinates of the feature points and the world coordinates of the corresponding spatial points, calculates the intrinsic parameters of the camera using the homography matrix according to the constraint relationship between the homography matrix and the intrinsic parameters of the camera, and then calculates the camera coordinates of the spatial points corresponding to all pixels according to the calculated intrinsic parameters, determines whether the spatial points are coplanar according to the covariance of the world coordinates of the spatial points corresponding to all pixels in the calibration plate image, and if they are coplanar, moves the origin of the world coordinate system to the center of gravity of the spatial points, and calculates the rotation matrix R of the world coordinate system relative to the camera coordinate system after the movement. c and the translation vector t c Calculate the camera's extrinsic parameters; if they are not coplanar, solve the camera's extrinsic parameters based on the correspondence between the camera coordinates and the world coordinates of the spatial point corresponding to the pixel point. Finally, according to a preset objective function, globally optimize the camera's distortion coefficients and the calculated intrinsic and extrinsic parameters to obtain the camera's final intrinsic and extrinsic parameters and distortion coefficients. This method can adapt to situations where the spatial points corresponding to the pixel points are coplanar or non-coplanar. Different methods are used to solve the extrinsic parameters for coplanar and non-coplanar situations. The calculated intrinsic and extrinsic parameters and initialized distortion coefficients are globally optimized according to the preset objective function to construct a globally optimal analysis method, thereby improving the accuracy of camera calibration and obtaining more accurate camera intrinsic and extrinsic parameters and distortion coefficients. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 Schematic diagram of the transformation of each coordinate system in the pinhole camera model;

[0069] Figure 2 is a flowchart of a camera calibration method according to an embodiment;

[0070] Figure 3 A schematic diagram of a checkerboard calibration plate according to an embodiment;

[0071] Figure 4 is a schematic diagram of a circular array calibration plate according to an embodiment;

[0072] Figure 5 A schematic structural diagram of a camera calibration system according to an embodiment. DETAILED DESCRIPTION

[0073] The present invention will be further described in detail below by means of specific embodiments in conjunction with the accompanying drawings. Similar elements in different embodiments are numbered with associated similar elements. In the following embodiments, many detailed descriptions are provided to enable the present application to be better understood. However, those skilled in the art will readily appreciate that some of the features may be omitted in different circumstances, or may be replaced by other elements, materials, or methods. In some cases, some operations related to the present application are not shown or described in the specification. This is to avoid the core portion of the present application being overwhelmed by excessive descriptions, and for those skilled in the art, it is not necessary to describe these related operations in detail. They will fully understand the related operations based on the description in the specification and the general technical knowledge in the art.

[0074] In addition, the features, operations, or characteristics described in the specification may be combined in any appropriate manner to form various embodiments. Furthermore, the steps or actions in the method description may be reordered or adjusted in a manner readily apparent to those skilled in the art. Therefore, the various sequences in the specification and drawings are provided solely for the purpose of clearly describing a particular embodiment and are not intended to be mandatory, unless otherwise specified.

[0075] The serial numbers assigned to components herein, such as "first," "second," etc., are used solely to distinguish the objects being described and do not convey any sequential or technical meaning. References to "connection" and "coupling" herein, unless otherwise specified, include both direct and indirect connections (couplings).

[0076] In high-precision measurement scenarios, barrel distortion or pincushion distortion may occur due to lens design or installation. To improve measurement accuracy, distortion correction is required. Distortion occurs during the transformation of the image plane from the lens to the camera. Therefore, it is necessary to obtain the camera's intrinsic parameter information to perform a correction operation and process the distortion. The distortion of a camera lens is usually a nonlinear distortion. The linear model of the camera cannot accurately describe the imaging geometry of the camera. This nonlinear distortion needs to be corrected. Currently, commonly used methods include optimization calculations based on fixed distortion models (such as tangential distortion and radial distortion) and intelligent algorithms without fixed distortion models. These algorithms cannot achieve both computational accuracy and efficiency when performing global optimization.

[0077] The purpose of camera calibration is to obtain the camera's intrinsic parameters, extrinsic parameters, and distortion coefficients. Currently, camera calibration is mainly calculated based on the Zhang Zhengyou calibration method, which mainly includes the following calculation steps:

[0078] (1) Obtain the homography matrix based on the image coordinates of the feature points in the calibration plate and the world coordinates of the corresponding spatial points; for a point P in space,w , if the pixel point q is obtained after the camera imaging i , then the space point P is called w is pixel q i The corresponding spatial point, pixel q i is a spatial point P w The corresponding pixel point;

[0079] (2) Decomposing the homography matrix and calculating the initial parameters of the intrinsic and / or extrinsic parameters;

[0080] (3) The LM (Levenberg-Marquardt) algorithm is used to perform nonlinear optimization on the initial parameters, and the internal parameters, external parameters and distortion coefficients are iteratively calculated to obtain the final calibration results.

[0081] In order to more clearly understand the technical solution of the present invention, the coordinate transformation relationship in the camera imaging process is introduced below. Figure 1 As shown in Figure 2, the projection transformation relationship of each coordinate system in the imaging process of the camera can be expressed by the pinhole camera model. The point (i.e., spatial point) P in the world coordinate system (WCS) is w Project the point P on the imaging plane through the lens projection center to obtain point P w The image coordinates q projected onto the imaging plane i , you need to first convert it to the Camera Coordinate System (CCS). The x- and y-axes of the camera coordinate system are parallel to the u- and v-axes of the image, respectively. The z-axis is perpendicular to the imaging plane and is oriented so that all points in front of the camera have positive z-coordinates. The u-axis of the image is horizontal, and the v-axis is vertical. Figure 1 Medium x c axis, y c axis and z c The axes represent the x-axis, y-axis, and z-axis of the camera coordinate system. The transformation from the world coordinate system to the camera coordinate system can be expressed as c = c H w p w To express, where p c =(x c ,y c ,z c ) T is the coordinate in the camera coordinate system, p w =(x w ,y w ,z w ) T is the coordinate in the world coordinate system,c H w It can be represented by the rotation matrix R and the translation vector t.

[0082] Expressed in matrix form:

[0083]

[0084] remember Therefore, we can get:

[0085] x c =R0x w +R1y w +R2z w +t0

[0086] y c =R3x w +R4y w +R5z w +t1. (1)

[0087] z c =R6x w +R7y w +R8z w +t2

[0088] After converting the world coordinate system to the camera coordinate system, it needs to be converted to the image plane coordinate system. This is a process of converting 3D coordinates to 2D coordinates. This transformation can be expressed as:

[0089]

[0090] Expressed in matrix form:

[0091]

[0092] Where f represents the focal length of the camera lens, (x I ,y I ) T Represents the coordinates in the image plane coordinate system.

[0093] Combining formula (1), we can get:

[0094]

[0095] After projection onto the imaging plane, the lens distortion will result in coordinates q c =(x I ,y I ) T changes, so that the distorted coordinate p is formed on the imaging plane d =(x d ,y d )T , this change can be modeled solely on the imaging plane, meaning that no three-dimensional information is required. Distortion includes radial distortion and tangential distortion, which can be described by the following model:

[0096]

[0097] in k0, k1, k2, k3, k4, k5, k6, and k7 are distortion coefficients, including the radial distortion coefficient and the tangential distortion coefficient. They can be expressed in matrix form as follows:

[0098]

[0099] The matrix [K] represents the distortion transformation performed according to the distortion coefficients k0, k1, k2, k3, k4, k5, k6, and k7 according to the above model.

[0100] According to the above model, when the distortion coefficient and the distorted coordinate p are known, d =(x d ,y d ) T In the case of , the Newton method can be used to solve the undistorted coordinate q c =(x I ,y I ) T .

[0101] Finally, the image plane coordinate system is converted to the image coordinate system (ICS), which can be expressed as follows:

[0102]

[0103] Expressed in matrix form:

[0104]

[0105] where s x and s y are the pixel sizes of the camera in the horizontal and vertical directions, respectively. (c x ,c y ) is the principal optical axis of the camera, usually the center of the image.

[0106] Therefore, the entire transformation above can be expressed as follows if the distortion is not considered:

[0107]

[0108] This is the mathematical model on which camera calibration is based. is the intrinsic parameter part of the camera, and the rotation matrix R and translation vector t are the extrinsic parameters.

[0109] Based on the above model, the present invention provides a camera calibration method, please refer to Figure 2 In one embodiment, the method includes steps 110 to 190, which are described in detail below.

[0110] Step 110: Acquire the calibration plate image captured by the camera.

[0111] The calibration plate can be a checkerboard calibration plate (such as Figure 3 As shown), circular array calibration plate (as shown Figure 4 When calibrating, the camera can be placed in multiple positions (position is also the position and angle of the camera relative to the calibration plate) based on experience, and the calibration plate is photographed in each position to obtain multiple different calibration plate images for calibration.

[0112] Step 120: Obtain the world coordinates of the spatial points corresponding to all the pixels in the calibration plate image, and obtain the feature points in the calibration plate image and the image coordinates of the feature points.

[0113] For the calibration plate image, it is obtained by imaging the calibration plate, so the spatial point corresponding to the pixel point in the calibration plate image is the point on the calibration plate. The world coordinate system is a spatial coordinate system, and the world coordinate system can be constructed according to the parameter information of the calibration plate to obtain the world coordinates of the spatial point. The parameter information of the calibration plate may include the size of the calibration plate, the size of the checkerboard, the radius of the circular feature point, the spacing between the feature points, etc. For the checkerboard calibration plate, the feature point is the corner point of the checkerboard. For the circular array calibration plate, the feature point is the center of gravity of the circular feature point in the circular array. The circular feature point is the circular pattern on the circular array calibration plate. The acquisition of the world coordinates of the spatial point corresponding to the pixel point, the feature point in the calibration plate image, and the image coordinates of the feature point can be achieved through existing technologies and will not be elaborated here.

[0114] Step 130: Calculate the homography matrix H based on the image coordinates of the feature points and the world coordinates of the corresponding spatial points a It can be understood that the image coordinates of multiple feature points and the world coordinates of the corresponding space points can be used to calculate the homography matrix using the least squares method, which is recorded as H a .

[0115] Step 140: Based on the constraint relationship between the homography matrix and the camera intrinsic parameters, use the homography matrix H a Calculate the camera's intrinsic parameters.

[0116] Let the equivalent focal length f x =f / s x , f y=f / s y , then the internal reference part can be expressed as When establishing the world coordinate system, it is usually assumed that the points on the calibration plate are located on the plane z = 0, so the rotation and translation in the z direction can be ignored. Therefore, the external parameter part can be expressed as Where r1, r2, r3, r4, r5 and r6 are the elements of the rotation matrix, t x and t y are the x and y components of the translation vector respectively.

[0117] The homography matrix H a The elements in are represented by h0, h1, h2, h3, h4, h5, h6, h7 and h8, then According to the relationship between the homography matrix and the internal and external parameters, we can get:

[0118]

[0119] So we can get h0=f x r1+c x r3, h1 = f x r4+c x r6,h2=f x t x +c x , h3=f y r2+c y r3, h4 = f y r5+c y r6,h5=f y t y +c y , h6=r3, h7=r6, h8=1.

[0120] If the origin of the image coordinate system is set to the image center, the resulting homography matrix H b for:

[0121]

[0122] Therefore, we can get H0=f x r1, h1 = f x r4,H2=f x t x , H3=f y r2,H4=f y r5, H5 = f y t y , H6=r3,H7=r6,H8=1. Therefore, the homography matrix H can be obtained according to the following formula a Calculate the homography matrix H b :

[0123] H0=h0-h6c x , H1=h1-h7c x , H2=h2-h8c x ,

[0124] H3=h3-h6c y , H4=h4-h7c y , H5=h5-h8c y ,

[0125] H6=h6, H7=h7, H8=h8,

[0126] The coordinates of the camera's principal optical axis point (c x ,c y ) can be found in the camera manual.

[0127]

[0128] The homography matrix H can be obtained by constraining the unit orthogonality of each vector in the rotation matrix b The constraint relationship between the camera intrinsic parameters, specifically, can be established based on the unit orthogonality:

[0129]

[0130] Substituting formula (2) into the equation yields the constraint relationship:

[0131]

[0132] According to the above constraints, the homography matrix H can be obtained b Calculate the equivalent focal length f x and f y .

[0133] Step 150: Calculate the camera coordinates of the spatial points corresponding to all pixel points based on the calculated intrinsic parameters.

[0134] From the above coordinate transformation relationship in the camera imaging process, it can be seen that according to the camera intrinsic parameters, the image coordinates of the pixel point can be used to obtain the corresponding coordinates in the camera coordinate system, that is, the camera coordinates of the spatial point corresponding to the pixel point.

[0135] Step 160: Determine whether the spatial points are coplanar based on the covariance of the world coordinates of the spatial points corresponding to all pixel points in the calibration plate image. If they are coplanar, execute step 170; if not, execute step 180.

[0136] Since there may be errors when establishing the world coordinate system, the world coordinates of the obtained spatial points are not coplanar. Therefore, the present invention first determines whether the spatial points corresponding to the pixel points are coplanar before calculating the extrinsic parameters. It can adapt to the situations where the spatial points corresponding to the pixel points are coplanar and non-coplanar, and adopts different methods to solve the extrinsic parameters for the coplanar and non-coplanar situations, which is beneficial to improve the accuracy of extrinsic parameter estimation.

[0137] In one embodiment, whether the pixels are coplanar or not can be determined based on the covariance matrix of the world coordinates of the spatial points corresponding to all pixels. The covariance matrix can be expressed as:

[0138]

[0139] Where (x i ,y i ,z i ) is the world coordinate of the space point corresponding to the i-th pixel, c x 、c y 、c z They represent the average values ​​of the x-coordinate, y-coordinate, and z-coordinate of the world coordinates of the spatial points corresponding to all pixels, and n represents the number of pixels.

[0140] To determine whether the spatial points are coplanar, specifically, the covariance matrix can be subjected to singular value decomposition (SVD), and three eigenvalues ​​can be obtained, which are arranged from large to small as λ1, λ2, and λ3. The corresponding eigenvectors are recorded as v1, v2, and v3. If λ3 / λ1 is less than the set threshold, the spatial points are judged to be coplanar, otherwise the spatial points are judged to be non-coplanar.

[0141] Step 170: Move the origin of the world coordinate system to the center of gravity of the spatial point, and calculate the rotation matrix R of the world coordinate system relative to the camera coordinate system based on the camera coordinates of the spatial point corresponding to the pixel point and the world coordinates after the movement. c and the translation vector t c , according to the rotation matrix R of the world coordinate system relative to the camera coordinate system after the movement c and the translation vector t c Calculate the camera's extrinsic parameters.

[0142] In one embodiment, a rotation matrix R can be first calculated based on the eigenvectors obtained by performing singular value decomposition on the covariance matrix. v =[v1,v2,v1×v2], and then calculate the translation vector t according to the following formula c :

[0143]

[0144] Where (x, y, z) is the world coordinate of the space point corresponding to the pixel point, ,Right now and is the translation vector t c The elements in are the translation vectors t c The x-component, y-component, and z-component of .

[0145] Then move the origin of the world coordinate system to the center of gravity of the space point and calculate the rotation matrix R according to the following formula c :

[0146]

[0147] Where (x c ,y c ,z c ) is the camera coordinate of the spatial point corresponding to the pixel point, and (x′, y′, z′) is the world coordinate of the spatial point corresponding to the pixel point after moving the origin of the world coordinate system to the center of gravity of the spatial point. The center of gravity of the spatial point can be the average value of the above coordinates c x 、c y 、c z The coordinate points (c x ,c y ,c z ).

[0148] Finally, the rotation matrix R and translation vector t are calculated according to the following formula:

[0149] t=H b t c , R=H b R c .

[0150] Step 180: Solve the camera's extrinsic parameters based on the correspondence between the camera coordinates and the world coordinates of the spatial point corresponding to the pixel point.

[0151] For a pixel point, the camera coordinates and world coordinates of its corresponding space point have been obtained from the above steps, and a coordinate pair of camera coordinates and world coordinates can be obtained. Using multiple such coordinate pairs, the rotation matrix R and translation vector t can be calculated.

[0152] In one embodiment, the parameters a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a 10 and a 11 :

[0153]

[0154] Then the rotation matrix Translation vector Where λ is the scale parameter, (x c ,y c ,z c ) and (x, y, z) are the camera coordinates and world coordinates of the space point corresponding to the same pixel point. It should be noted that the scale parameter λ does not necessarily need to be pre-set or solved for a specific value. As long as the parameters a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a 10 and a 11 That's it.

[0155] The present invention provides a method for solving the above equation. Specifically, after expanding the above equation, it can be obtained:

[0156] λx c =a0x+a1y+a2z+a3

[0157] λy c =a4x+a5y+a6z+a7,

[0158] λz c =a8x+a9y+a 10 z+a 11

[0159] Substituting the third line into the first two lines yields:

[0160] a0xz c +a1yz c +a2zz c +a3z c -a8xx c -a9yx c -a 10 zx c -a 11 x c =0

[0161] a4xz c +a5yz c +a6zz c +a7z c -a8xy c -a9yy c -a 10 zy c -a 11 y c =0,

[0162] Written in matrix form:

[0163]

[0164] Where X=[a0,a1,a2,a3,a4,a5,a6,a7,a8,a9,a 10 ,a 11 ] T , the matrix on the left of X is the coefficient matrix, denoted as S. If the calculation is based on N pixels, the coefficient matrix S is a 2N×12 matrix. T S is decomposed by SVD, and the eigenvector corresponding to the minimum eigenvalue obtained is the solution of X.

[0165] Step 190: According to a preset objective function, the camera distortion coefficient and the calculated intrinsic parameters and extrinsic parameters are globally optimized to obtain the final intrinsic parameters, extrinsic parameters and distortion coefficient of the camera.

[0166] This step primarily performs global optimization of the intrinsic and extrinsic parameters, as well as the distortion coefficients, while also performing nonlinear optimization of the distortion coefficients to achieve distortion correction. Based on a preset objective function, global optimization iterates the initial values ​​of the intrinsic and extrinsic parameters, as well as the distortion coefficients, to find the optimal solution. Iterations cease when a preset stopping condition is reached, resulting in the camera's final intrinsic and extrinsic parameters, as well as the distortion coefficients. The initial values ​​of the intrinsic and extrinsic parameters can be those calculated in the previous steps, while the initial values ​​of the distortion coefficients can be pre-set based on experience.

[0167] In one embodiment, the optimal solution can be iteratively solved according to the following objective function to obtain the final intrinsic parameters, extrinsic parameters and distortion coefficients of the camera:

[0168]

[0169] where n m is the number of feature points in the calibration plate image, n0 is the number of calibration plate images taken by the camera, where different calibration plate images are taken when the camera and / or calibration plate are in different positions, q jk is the image coordinate of the jth feature point in the kth calibration plate image, p j is the world coordinate of the spatial point corresponding to the feature point, e k represents the external parameter obtained based on the k-th calibration plate image, f k represents the internal parameter obtained from the kth calibration plate image, K represents the distortion coefficient, φ(p j ,e k ,f k ,K) represents the world coordinate p j The image coordinates are obtained by transformation based on the obtained intrinsic parameters, extrinsic parameters and distortion coefficients. As can be seen from the above, this includes the process of transforming the world coordinates to the camera coordinates using the extrinsic parameters, and then transforming the camera coordinates to the image coordinates using the intrinsic parameters and distortion coefficients.

[0170] In one embodiment, the LM algorithm can be used for iterative calculation, and the update of the parameters during the iteration process can be expressed as Q l+1 =Q l +δ, where Q l It represents the vector composed of the intrinsic parameters, extrinsic parameters and distortion coefficients of the camera at the lth iteration. δ is determined by the formula Jδ = ε, where ε is the q corresponding to all feature points of all calibration plate images at the current iteration. jk -φ(p j ,e k ,r k ,K) values, the matrix J can be specifically

[0171]

[0172] Among them J e Represents the partial derivative of the image coordinates of the feature point with respect to the external parameter, specifically

[0173]

[0174] Where (u, v) is the image coordinate of the feature point, r = [r x ,r y ,r z ] is the equivalent rotation axis based on the rotation matrix R, t=[t x ,t y ,t z ] is the translation vector. The rotation matrix R can be represented by the equivalent rotation axis r and the equivalent axis angle θ. Therefore, the equivalent rotation axis r and the equivalent axis angle θ can be regarded as external parameters. Here, the partial derivatives of the rotation matrix R are represented by the partial derivatives with respect to the equivalent rotation axis r.

[0175] J e,k It represents the partial derivative of the image coordinates of the feature point in the k-th calibration plate image with respect to the external parameter.

[0176] J k It represents the partial derivative of the image coordinates of the feature point in the k-th calibration plate image with respect to the intrinsic parameters and distortion coefficients, specifically:

[0177]

[0178] It can be seen from formula (3) that the matrix J used in the method of updating the camera parameters of the present invention is a sparse matrix, thereby improving the calculation speed.

[0179] The calculation method of the partial derivatives of each part is explained below.

[0180] For the intrinsic parameter part, the partial derivative of the image coordinates of the feature point with respect to the equivalent focal length is:

[0181]

[0182] The partial derivative of the image coordinates of the feature point with respect to the camera's principal optical axis is:

[0183]

[0184] The partial derivatives of the image coordinates of the feature points with respect to the distortion coefficient can be calculated in several parts. The partial derivatives with respect to k0 and k1 are:

[0185]

[0186] The partial derivatives with respect to k2 and k3 are:

[0187]

[0188] The partial derivative with respect to k4 is:

[0189]

[0190] The partial derivative with respect to k5 is:

[0191]

[0192] The partial derivative with respect to k6 is:

[0193]

[0194] The partial derivative with respect to k7 is:

[0195]

[0196] For the external parameter part, the partial derivative of the image coordinates of the feature point with respect to the translation vector is:

[0197]

[0198]

[0199]

[0200] in

[0201]

[0202] d1=1+k0(x c 2 +y c 2 )+k1(x c 2 +y c 2 ) 2 +k4(x c2 +y c 2 ) 3 ,

[0203]

[0204] make We can get:

[0205]

[0206] Then the partial derivative of d1 with respect to the translation vector is:

[0207]

[0208] The partial derivative of d2 with respect to the translation vector is:

[0209]

[0210] The partial derivative of the image coordinates of the feature point with respect to the equivalent rotation axis is:

[0211]

[0212] in

[0213]

[0214]

[0215] The above calculation process can be expressed as follows using the chain rule:

[0216]

[0217]

[0218]

[0219]

[0220] Where m=(u,v), c=(c x ,c y ), f=(f x ,f y ), K represents the distortion coefficient.

[0221] At this point, the solution to the matrix J is completed.

[0222] Based on the above camera calibration method, the present invention also provides a camera calibration system, please refer to Figure 5In one embodiment, the system includes a calibration plate image acquisition module 1, an image and world coordinate acquisition module 2, a homography matrix calculation module 3, an intrinsic parameter calculation module 4, a camera coordinate acquisition module 5, an extrinsic parameter calculation module 6 and a global optimization module 7, which are described below respectively.

[0223] The calibration plate image acquisition module 1 is used to acquire the calibration plate image captured by the camera.

[0224] The image and world coordinate acquisition module 2 is used to obtain the world coordinates of the spatial points corresponding to all pixels in the calibration plate image, and obtain the feature points in the calibration plate image and the image coordinates of the feature points. The acquisition of the world coordinates of the spatial points corresponding to the pixels, the feature points in the calibration plate image, and the image coordinates of the feature points can be achieved using existing technologies and will not be elaborated here.

[0225] The homography matrix calculation module 3 is used to calculate the homography matrix H according to the image coordinates of the feature points and the world coordinates of the corresponding space points. a .

[0226] The intrinsic parameter calculation module 4 is used to use the homography matrix H according to the constraint relationship between the homography matrix and the camera intrinsic parameters. a Calculate the camera's intrinsic parameters. In one embodiment, the intrinsic parameter calculation module 4 is specifically used to:

[0227] According to the following formula, the homography matrix H a Calculate the homography matrix H b :

[0228] H0=h0-h6c x , H1=h1-h7c x , H2=h2-h8c x ,

[0229] H3=h3-h6c y , H4=h4-h7c y , H5=h5-h8c y ,

[0230] H6=h6, H7=h7, H8=h8,

[0231] Then according to the following constraint relationship, the homography matrix H b Calculate the camera's equivalent focal length f x and f y :

[0232]

[0233] The camera coordinate acquisition module 5 is used to calculate the camera coordinates of the spatial points corresponding to all pixel points based on the calculated intrinsic parameters.

[0234] The external parameter calculation module 6 is used to determine whether the spatial points are coplanar based on the covariance of the world coordinates of the spatial points corresponding to all the pixels in the calibration plate image. If they are coplanar, the origin of the world coordinate system is moved to the center of gravity of the spatial point, and the rotation matrix R of the world coordinate system relative to the camera coordinate system is calculated based on the camera coordinates of the spatial point corresponding to the pixel and the world coordinates after the movement. c and the translation vector t c , according to the rotation matrix R of the world coordinate system relative to the camera coordinate system after the movement c and the translation vector t c Calculate the camera's extrinsic parameters; if they are not coplanar, solve the camera's extrinsic parameters based on the correspondence between the camera coordinates and the world coordinates of the spatial point corresponding to the pixel point.

[0235] In one embodiment, the external parameter calculation module 6 is specifically used to perform singular value decomposition on the covariance matrix of the world coordinates of the spatial points corresponding to all pixel points, and arrange the eigenvalues ​​obtained by the decomposition from large to small as λ1, λ2, λ3. If λ3 / λ1 is less than the set threshold, it is judged that the spatial points are coplanar, otherwise it is judged that the spatial points are not coplanar.

[0236] In one embodiment, if the spatial points corresponding to all pixels in the calibration plate image are coplanar, the external parameter calculation module 6 is specifically used to calculate the rotation matrix R according to the eigenvectors obtained by performing singular value decomposition on the covariance matrix of the world coordinates of the spatial points corresponding to all pixels. v =[v1,v2,v1×v2], the translation vector t is calculated according to the following formula c :

[0237]

[0238] Move the origin of the world coordinate system to the center of gravity of the space point and calculate the rotation matrix R according to the following formula c :

[0239]

[0240] The rotation matrix R and translation vector t are calculated according to the following formula:

[0241] t=H b t c , R=H b R c .

[0242] In one embodiment, if the spatial points corresponding to all pixels in the calibration plate image are not coplanar, the external parameter calculation module 6 is specifically used to calculate the parameters a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10 according to the following equations: 10 and a11 :

[0243]

[0244] Then the rotation matrix Translation vector

[0245] The global optimization module 7 is used to perform global optimization on the camera's distortion coefficient and the calculated internal and external parameters according to a preset objective function to obtain the camera's final internal and external parameters and distortion coefficient.

[0246] In one embodiment, the global optimization module 7 is specifically configured to:

[0247] The initial value of the distortion coefficient is set in advance, and the calculated internal and external parameters are used as the initial values ​​of the internal and external parameters. The optimal solution is iterated according to the following objective function to obtain the final internal and external parameters and distortion coefficient of the camera:

[0248]

[0249] where n m is the number of feature points in the calibration plate image, n0 is the number of calibration plate images taken by the camera, where different calibration plate images are taken when the camera and / or calibration plate are in different positions, q jk is the image coordinate of the jth feature point in the kth calibration plate image, p j is the world coordinate of the spatial point corresponding to the feature point, e k represents the external parameter obtained based on the k-th calibration plate image, f k represents the internal parameter obtained from the kth calibration plate image, K represents the distortion coefficient, φ(p j ,e k ,f k ,K) represents the image coordinates obtained by transforming the world coordinates pj according to the obtained intrinsic parameters, extrinsic parameters and distortion coefficients. From the above, we can see that this includes the process of transforming the world coordinates to the camera coordinates using the extrinsic parameters, and then transforming the camera coordinates to the image coordinates using the intrinsic parameters and distortion coefficients.

[0250] In one embodiment, the global optimization module 7 is further configured to: l+1 =Q l +δ iteratively solves the optimal solution, where Q l It represents the vector composed of the intrinsic parameters, extrinsic parameters and distortion coefficients of the camera at the lth iteration. δ is determined by the formula Jδ = ε, where ε is the q corresponding to all feature points of all calibration plate images at the current iteration. jk -φ(p j ,e k ,r k,K) values, the matrix J is specifically

[0251]

[0252] Among them J e Represents the partial derivative of the image coordinates of the feature point with respect to the external parameter, specifically

[0253]

[0254] Where (u, v) is the image coordinate of the feature point, r = [r x ,r y ,r z ] is the equivalent rotation axis based on the rotation matrix R, t=[t x ,t y ,t z ] is the translation vector.

[0255] Then J e,k Represents the partial derivative of the image coordinates of the feature point in the k-th calibration plate image with respect to the extrinsic parameter.

[0256] J k It represents the partial derivative of the image coordinates of the feature point in the k-th calibration plate image with respect to the intrinsic parameters and distortion coefficients, specifically:

[0257]

[0258] For the calculation method of each partial derivative, please refer to the corresponding content about the camera calibration method above, which will not be repeated here.

[0259] According to the camera calibration method and system of the above embodiment, only one calibration plate image is required when estimating the intrinsic parameters, and the intrinsic parameter estimation value can be calculated directly based on the calibration plate image, while the existing technology requires more than one calibration plate image. Therefore, the camera calibration method and system of the present invention reduces the requirement for the number of images and improves efficiency. In addition, the camera calibration method and system of the present invention can adapt to the situations where the spatial points corresponding to the pixel points are coplanar or non-coplanar, and adopts different methods to solve the extrinsic parameters for the coplanar and non-coplanar situations. The calculated intrinsic parameters, extrinsic parameters and initialized distortion coefficients are globally optimized according to a preset objective function to construct a globally optimal analysis method. During the optimization, the distortion coefficients of radial distortion and tangential distortion can be optimized together, which can eliminate not only mirror distortion but also tangential distortion. In summary, the technical solution of the present invention can improve the accuracy of camera calibration and obtain more accurate camera intrinsic parameters, extrinsic parameters and distortion coefficients.

[0260] Those skilled in the art will appreciate that all or part of the functions of the various methods in the above embodiments can be implemented by hardware or by computer program. When all or part of the functions in the above embodiments are implemented by computer program, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, disk, optical disk, hard disk, etc., and the program is executed by a computer to implement the above functions. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above functions can be implemented. In addition, when all or part of the functions in the above embodiments are implemented by computer program, the program can also be stored in a storage medium such as a server, another computer, disk, optical disk, flash disk or mobile hard disk, and saved in the memory of the local device by downloading or copying, or the system of the local device is updated. When the program in the memory is executed by the processor, all or part of the functions in the above embodiments can be implemented.

[0261] The above examples are used to illustrate the present invention, which are only used to help understand the present invention and are not intended to limit the present invention. Those skilled in the art can make several simple deductions, modifications or substitutions based on the concept of the present invention.

Claims

1. A camera calibration method, characterized in that: include: Get the calibration plate image taken by the camera; Obtain the world coordinates of the spatial points corresponding to all pixels in the calibration plate image, and obtain the feature points in the calibration plate image, as well as the image coordinates of the feature points; Calculate the homography matrix H according to the image coordinates of the feature points and the world coordinates of the corresponding spatial points a ; According to the constraint relationship between the homography matrix and the camera intrinsic parameters, the homography matrix H is used a Calculate the camera's intrinsic parameters; Calculate the camera coordinates of the spatial points corresponding to all pixels based on the calculated intrinsic parameters; According to the covariance of the world coordinates of the spatial points corresponding to all pixels in the calibration plate image, determine whether the spatial points are coplanar. If they are coplanar, move the origin of the world coordinate system to the center of gravity of the spatial point. According to the camera coordinates of the spatial point corresponding to the pixel point and the world coordinates after the shift, calculate the rotation matrix R of the world coordinate system relative to the camera coordinate system after the shift c and the translation vector t c , according to the rotation matrix R of the world coordinate system relative to the camera coordinate system after the movement c and the translation vector t c Calculate the camera's extrinsic parameters; if they are not coplanar, solve the camera's extrinsic parameters based on the correspondence between the camera coordinates and the world coordinates of the spatial point corresponding to the pixel point; According to the preset objective function, the camera's distortion coefficient and the calculated internal and external parameters are globally optimized to obtain the camera's final internal and external parameters and distortion coefficient.

2. The camera calibration method according to claim 1, wherein: The intrinsic parameters of the camera include the equivalent focal length f x and f y ; The method of calculating the camera's intrinsic parameters using the homography matrix according to the constraint relationship between the homography matrix and the camera's intrinsic parameters includes: According to the following formula, the homography matrix H a Calculate the homography matrix H b : H0=h0-h6c x ,H1=h1-h7c x ,H2=h2-h8c x , H3=h3-h6c y ,H4=h4-h7c y ,H5=h5-h8c y , H6=h6, H7=h7, H8=h8, Where (c x ,c y ) are the coordinates of the camera's principal optical axis point, h0, h1, h2, h3, h4, h5, h6, h7, and h8 are the homography matrices H a The elements in H0, H1, H2, H3, H4, H5, H6, H7 and H8 are homography matrices H b The elements in According to the following constraints, the homography matrix H b Calculate the camera's equivalent focal length f x and f y :

3. The camera calibration method according to claim 1, wherein: The determining whether the spatial points are coplanar according to the covariance of the world coordinates of the spatial points corresponding to all the pixels in the calibration plate image includes: Perform singular value decomposition on the covariance matrix of the world coordinates of the spatial points corresponding to all pixels, and arrange the decomposed eigenvalues ​​from large to small as λ1, λ2, and λ3; If λ3 / λ1 is less than a set threshold, the spatial points are judged to be coplanar; otherwise, the spatial points are judged to be non-coplanar.

4. The camera calibration method according to claim 3, wherein: The external parameters of the camera include the rotation matrix R of the world coordinate system relative to the camera coordinate system and the translation vector t; the origin of the mobile world coordinate system is moved to the center of gravity of the spatial point, and the rotation matrix R of the world coordinate system relative to the camera coordinate system after the movement is calculated based on the camera coordinates of the spatial point corresponding to the pixel point and the world coordinates after the movement. c and the translation vector t c , according to the rotation matrix R of the world coordinate system relative to the camera coordinate system after the movement c and the translation vector t c Calculate the camera's external parameters, including: The rotation matrix R is calculated based on the eigenvector obtained by performing singular value decomposition on the covariance matrix of the world coordinates of the spatial points corresponding to all pixels. v =[v1,v2,v1×v2], where v1 and v2 are the eigenvectors corresponding to eigenvalues ​​λ1 and λ2 respectively; The translation vector t is calculated according to the following formula c : Where (x, y, z) is the world coordinate of the space point corresponding to the pixel point, and is the translation vector t c The elements in Move the origin of the world coordinate system to the center of gravity of the space point, and calculate the rotation matrix R according to the following formula c : Where (x c ,y c ,z c ) is the camera coordinate of the spatial point corresponding to the pixel point, and (x′, y′, z′) is the world coordinate of the spatial point corresponding to the pixel point after moving the origin of the world coordinate system to the center of gravity of the spatial point; The rotation matrix R and translation vector t are calculated according to the following formula: t=H b t c ,R=H b R c 。 5. The camera calibration method according to claim 1, wherein: The camera's extrinsic parameters include a rotation matrix R of the world coordinate system relative to the camera coordinate system and a translation vector t; solving the camera's extrinsic parameters based on the correspondence between the camera coordinates and the world coordinates of the spatial point corresponding to the pixel point includes: Calculate the parameters a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a 10 and a 11 : Where λ is the scale parameter, (x c ,y c ,z c ) is the camera coordinate of the space point corresponding to the pixel point, (x, y, z) is the world coordinate of the space point corresponding to the pixel point, and the rotation matrix Translation vector 6. The camera calibration method according to any one of claims 1 to 5, wherein: The camera distortion coefficient and the calculated internal and external parameters are globally optimized according to the preset objective function to obtain the final internal and external parameters and distortion coefficient of the camera, including: The initial value of the distortion coefficient is set in advance, and the calculated internal and external parameters are used as the initial values ​​of the internal and external parameters. The optimal solution is iterated according to the following objective function to obtain the final internal and external parameters and distortion coefficient of the camera: where n m is the number of feature points in the calibration plate image, n0 is the number of calibration plate images taken by the camera, q jk is the image coordinate of the jth feature point in the kth calibration plate image, p j is the world coordinate of the spatial point corresponding to the feature point, e k represents the external parameter obtained based on the k-th calibration plate image, f k represents the internal parameter obtained from the kth calibration plate image, K represents the distortion coefficient, φ(p j ,e k ,f k ,K) represents the world coordinate p j The image coordinates are obtained by transforming the obtained intrinsic parameters, extrinsic parameters and distortion coefficients.

7. The camera calibration method according to claim 6, wherein: According to the formula Q l+1 =Q l +δ iteratively solves the optimal solution, where Q l It represents the vector composed of the intrinsic parameters, extrinsic parameters and distortion coefficients of the camera at the lth iteration. δ is determined by the formula Jδ = ε, where ε is the q corresponding to all feature points of all calibration plate images at the current iteration. jk -φ(p j ,e k ,r k ,K) values, the matrix J is specifically Among them J e Represents the partial derivative of the image coordinates of the feature point with respect to the external parameter, specifically Where (u, v) is the image coordinate of the feature point, r = [r x ,r y ,r z ] is the equivalent rotation axis based on the rotation matrix R, t=[t x ,t y ,t z ] is the translation vector; J e,k Represents the partial derivative of the image coordinates of the feature point in the k-th calibration plate image with respect to the external parameter; J k It represents the partial derivative of the image coordinates of the feature point in the k-th calibration plate image with respect to the intrinsic parameters and distortion coefficients, specifically: where f x and f y is the equivalent focal length, (c x ,c y ) are the coordinates of the camera's principal optical axis point, and k0, k1, k2, k3, k4, k5, k6, and k7 are distortion coefficients.

8. A camera calibration system, characterized in that: include: The calibration plate image acquisition module is used to obtain the calibration plate image taken by the camera; The image and world coordinate acquisition module is used to obtain the world coordinates of the spatial points corresponding to all pixels in the calibration plate image, and obtain the feature points in the calibration plate image and the image coordinates of the feature points; Homography matrix calculation module, used to calculate the homography matrix H based on the image coordinates of the feature points and the world coordinates of the corresponding space points a ; The intrinsic parameter calculation module is used to use the homography matrix H according to the constraint relationship between the homography matrix and the camera intrinsic parameters. a Calculate the camera's intrinsic parameters; The camera coordinate acquisition module is used to calculate the camera coordinates of the spatial points corresponding to all pixel points based on the calculated intrinsic parameters; The external parameter calculation module is used to determine whether the spatial points are coplanar based on the covariance of the world coordinates of the spatial points corresponding to all pixels in the calibration plate image. If they are coplanar, the origin of the world coordinate system is moved to the center of gravity of the spatial point. Based on the camera coordinates of the spatial point corresponding to the pixel point and the world coordinates after the movement, the rotation matrix R of the world coordinate system relative to the camera coordinate system is calculated. c and the translation vector t c , according to the rotation matrix R of the world coordinate system relative to the camera coordinate system after the movement c and the translation vector t c Calculate the camera's extrinsic parameters; if they are not coplanar, solve the camera's extrinsic parameters based on the correspondence between the camera coordinates and the world coordinates of the spatial point corresponding to the pixel point; The global optimization module is used to perform global optimization on the camera's distortion coefficient and the calculated intrinsic and extrinsic parameters according to a preset objective function to obtain the camera's final intrinsic and extrinsic parameters and distortion coefficient.

9. The camera calibration system according to claim 8, wherein: The intrinsic parameters of the camera include the equivalent focal length f x and f y ; The internal parameter calculation module is specifically used for: According to the following formula, the homography matrix H a Calculate the homography matrix H b : H0=h0-h6c x ,H1=h1-h7c x ,H2=h2-h8c x , H3=h3-h6c y ,H4=h4-h7c y ,H5=h5-h8c y , H6=h6, H7=h7, H8=h8, Where (c x ,c y ) are the coordinates of the camera's principal optical axis point, h0, h1, h2, h3, h4, h5, h6, h7, and h8 are the homography matrices H a The elements in H0, H1, H2, H3, H4, H5, H6, H7 and H8 are homography matrices H b The elements in According to the following constraints, the homography matrix H b Calculate the camera's equivalent focal length f x and f y :

10. A computer-readable storage medium, characterized in that The medium stores a program, which can be executed by a processor to implement the camera calibration method according to any one of claims 1 to 7.

Images