Camera calibration plate and calibration method thereof

Abstract

The present invention belongs to the field of computer vision, and specifically is a camera calibration plate and a calibration method thereof. It comprises: a plane plate and dot marker points and annular marker points distributed in an array on the plane plate; the dot marker points and annular marker points are coplanar points; the camera calibration plate is divided into a core area and an extension area according to the marker point distribution area; wherein the core area, an array composed of dot marker points and annular marker points, is used to be set within the camera field of view during the calibration process; the extension area, an array composed of dot marker points, is used to cover the camera field of view during the calibration process, and some marker points in the extension area are allowed to exceed the camera field of view when the camera calibration image is collected. The calibration plate provided by the present invention can make the marker points fill the camera field of view as much as possible during the camera calibration process, thereby improving the accuracy of camera calibration.

Description

Technical Field

[0001] The invention belongs to the field of computer vision, in particular to a camera calibration plate and a calibration method thereof. Background Art

[0002] Calibration plates are used to calibrate camera intrinsic and extrinsic parameters in applications such as machine vision measurement and 3D reconstruction. When calibrating a camera, the larger the area ratio of the envelope formed by all the landmarks (or corner points) observed by the camera to the imaging range, the higher the camera calibration accuracy will be. Currently, the more commonly used calibration plates include checkerboard calibration plates, HALCON calibration plates, CharuCo calibration plates, etc. Among them, the checkerboard calibration plate is the calibration plate mainly used in the OpenCV open source library. It relies on the odd and even numbers of the corner points of the rows and columns to locate the imaging rotation angle. During the calibration process, the entire calibration plate must be within the camera's field of view, resulting in a considerable area within the camera's field of view not being covered by the calibration plate, so the calibration effect is poor; on the other hand, the corner point detection used by the checkerboard calibration plate recognition algorithm has very high requirements on imaging quality. When the ambient light is bright or the camera exposure time is long, it may cause the corner point recognition to fail or the detection accuracy to decrease; the HALCON calibration plate is a calibration plate composed of a 7*7 dot marker array distribution. It relies on the missing corner of the inner frame in the upper left corner to locate the imaging rotation angle. Like the checkerboard calibration plate, the entire calibration plate must be within the camera's field of view during the calibration process. Similarly, the problem of leaving too much blank space within the field of view cannot be avoided; CharuCo The calibration plate is a new type of calibration plate that has emerged in recent years. It is a combination of a chessboard and a QR code, and has the function of local encoding. In theory, it can correctly identify some corner points that are beyond the camera's display range, so that the camera's field of view can be covered as much as possible. It is also supported by the OpenCV open source library. However, the pattern of the CharuCo calibration plate is relatively complex, which is not conducive to machining. In addition, like the chessboard calibration plate, it also has the problem of high image quality requirements. Summary of the invention

[0003] The purpose of the present invention is to provide a plane calibration plate composed of dot marking points and ring marking points to overcome the defect that the above-mentioned existing calibration plate cannot improve the calibration accuracy of camera intrinsic parameters and distortion coefficients.

[0004] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a camera calibration plate, comprising: a plane plate and dot marking points and ring marking points arrayed on the plane plate;

[0005] The dot marking point and the ring marking point are coplanar points;

[0006] The camera calibration plate is divided into a core area and an extension area according to the distribution area of ​​the landmark points;

[0007] The core area is an array of dot markers and ring markers, and is used to be set within the camera field of view during the calibration process;

[0008] The extension area is an array of dot marking points, which is used to cover the camera field of view during the calibration process. When the camera calibration image is collected, some marking points in the extension area are allowed to exceed the camera field of view.

[0009] The core area is an array of 4 rows and 5 columns; the first row of the core area is, from left to right, a circle mark point, a circle mark point, a circle mark point, and a circle mark point, and the three circles from left to right are circle mark point No. 1, circle mark point No. 2, and circle mark point No. 3;

[0010] The two ends of the last row in the core area are circular marking points, namely circular marking point No. 4 and circular marking point No. 5, and the point between the two circular marking points is a dot marking point;

[0011] The marking points in the remaining rows of the core area are all round dot marking points.

[0012] The diameter of the dot mark point is equal to the outer diameter of the ring mark point, and the inner diameter of the ring mark point is equal to half of the outer diameter of the ring mark point; the inner circle and outer circle of the ring mark point are concentric circles;

[0013] The center row spacing and center column spacing of the dot marking points and the ring marking points are both twice the diameter of the dot marking points.

[0014] The dot mark points and the ring mark points are used as target areas for image processing, and the rest of the calibration plate and the inner circle area of ​​the ring mark points are used as background areas.

[0015] A camera calibration method comprises the following steps:

[0016] 1) Camera calibration, including calibration plate coordinate system calibration, calibration plate plane coordinate system calibration, image coordinate system calibration and camera coordinate system calibration;

[0017] During the camera calibration image acquisition process, the calibration plate is placed in front of the camera lens as required;

[0018] 2) Collect no less than 4 calibration plate images. For the collected calibration plate images, the calibration plate recognition first performs binary segmentation on the image, performs region growth on the imaging area of ​​the marker point, judges the area after the growth, and identifies the dot marker points and the ring marker points in the image;

[0019] 3) Traverse and search for all the identified circular marking points. For each circular marking point found, find the other two circular marking points with the closest Euclidean distance to the circular marking point. According to whether the combination of the three circular marking points meets the set combination, further identify other circular marking points and dot marking points in the core area. According to the recognition situation, determine whether the core area of ​​the calibration plate is successfully recognized;

[0020] 4) If the core area of ​​the calibration plate is successfully identified, the homography mapping relationship and the two-dimensional coordinates of the dot markers in the extension area in the calibration plate plane coordinate system are used to match the identified dot markers in the image, and the dot markers and ring markers in the core area and the dot markers in the extension area that are successfully matched are used to calculate the homography of the calibration plate plane coordinate system relative to the camera image coordinate system again, and then try to continue matching the dot markers that have not been matched in the image until no more dot markers can be matched, and the identification of the extension area of ​​the calibration plate is completed;

[0021] 5) After the calibration plate recognition in each image is completed, the recognition result of the calibration plate is a set of two-dimensional coordinates and imaging coordinates of the marker points in the calibration plate plane coordinate system. Each image corresponds to a set of rotation and translation change relationships (R i |t i ), all recognition results are used as input variables of nonlinear optimization for optimization and solution, and finally the camera calibration parameters are obtained, that is, the camera calibration is completed.

[0022] The step 1) is specifically:

[0023] (1) Definition of the calibration plate coordinate system: The geometric center of the calibration plate surface is defined as the origin of the calibration plate coordinate system; the direction from the No. 1 ring mark point to the No. 3 ring mark point is defined as the positive direction of the calibration plate coordinate system X axis; the direction from the No. 1 ring mark point to the No. 4 ring mark point is defined as the positive direction of the calibration plate coordinate system Y axis; the Z axis direction of the calibration plate coordinate system satisfies the right-hand rule with the X axis and Y axis directions;

[0024] (2) Definition of the calibration plate plane coordinate system: The origin of the calibration plate plane coordinate system is the origin of the calibration plate coordinate system, and the X-axis and Y-axis of the calibration plate plane coordinate system are the same as the X-axis and Y-axis of the calibration plate coordinate system;

[0025] (3) Definition of the image coordinate system: The origin of the image coordinate system is located in the upper left corner of the image, the positive direction of the u-axis is horizontally to the right, and the positive direction of the v-axis is vertically downward;

[0026] (4) Definition of the camera coordinate system: The origin of the camera coordinate system is defined as the position of the lens optical center, the positive direction of the X axis is the same as the positive direction of the u axis of the image coordinate system, the positive direction of the Y axis is the same as the positive direction of the v axis of the image coordinate system, and the Z axis direction satisfies the right-hand rule with the X axis and Y axis directions;

[0027] After determining each coordinate system, the angle between the positive direction of the Z axis of the calibration plate coordinate system and the positive direction of the Z axis of the camera coordinate system during image acquisition is set to be no less than 20° and no more than 40°, and each calibration plate image is imaged to ensure the integrity of the core area, that is, the 20 markers in the 4 rows and 5 columns of the core area appear completely in the image and cannot intersect with the image boundary.

[0028] In step 2), the region growth is performed on the imaging region of the marker point, and the region where the growth is completed becomes the pending region, and the pending region is judged, specifically:

[0029] If the length-to-width ratio of the bounding rectangle of the pending area is less than the set threshold, the ratio of the area of ​​the pending area to the area of ​​the bounding rectangle of the pending area is greater than the set threshold, and there is a hole with an area greater than the set threshold at the center of the pending area, then the image area is judged to be the area where the circular mark point is located;

[0030] If the length-to-width ratio of the circumscribed rectangle of the pending area is less than the set threshold, the ratio of the area of ​​the pending area to the area of ​​the circumscribed rectangle of the pending area is greater than the set threshold, and the pending area is not the area where the circular mark point is located, then the pending area is determined to be the area where the circular mark point is located;

[0031] The undetermined area that does not satisfy the imaging feature constraints of the ring mark point nor the imaging feature constraints of the dot mark point will no longer be used for the next step of target recognition.

[0032] In step 3), the combination of settings includes:

[0033] Case 1: There is one collinear circle mark between the No. 1 circle mark and the No. 2 circle mark, and there are two collinear circle mark between the No. 1 circle mark and the No. 4 circle mark. The No. 1 circle mark, the No. 2 circle mark, and the No. 4 circle mark satisfy the clockwise order. In the triangle formed by the No. 1 circle mark, the No. 2 circle mark and the No. 4 circle mark on the calibration plate, the No. 1 circle mark corresponds to the right-angle vertex.

[0034] Case 2: There is one collinear circle mark between the No. 2 circle mark and the No. 3 circle mark, and there are two collinear circle mark between the No. 3 circle mark and the No. 5 circle mark. The No. 3 circle mark, the No. 2 circle mark, and the No. 5 circle mark satisfy the counterclockwise order. In the triangle formed by the No. 2 circle mark, the No. 3 circle mark and the No. 5 circle mark on the calibration plate, the No. 3 circle mark corresponds to the right-angle vertex.

[0035] Case 3: There are 2 collinear circular mark points between the No. 1 circular mark point and the No. 4 circular mark point, and there are 3 collinear circular mark points between the No. 4 circular mark point and the No. 5 circular mark point. The No. 4 circular mark point, the No. 1 circular mark point, and the No. 5 circular mark point satisfy the clockwise order. In the triangle formed by the No. 1 circular mark point, the No. 4 circular mark point and the No. 5 circular mark point on the calibration plate, the No. 4 circular mark point corresponds to the right-angle vertex.

[0036] Case 4: There are 2 collinear circle mark points between the No. 3 circle mark point and the No. 5 circle mark point, and there are 3 collinear circle mark points between the No. 4 circle mark point and the No. 5 circle mark point. The No. 5 circle mark point, the No. 3 circle mark point, and the No. 4 circle mark point satisfy the counterclockwise order. In the triangle formed by the No. 3 circle mark point, the No. 4 circle mark point and the No. 5 circle mark point on the calibration plate, the No. 5 circle mark point corresponds to the right-angle vertex.

[0037] In step 3), it is determined whether the core area of ​​the calibration plate is successfully identified, specifically:

[0038] For the three circular markers found, first calculate the vertex angle of each circular marker in the triangle, mark the circular marker with the largest vertex angle as V, and then determine the number of collinear circular markers between V and the other two circular markers;

[0039] If the number of collinear circle mark points is 1 and 2 respectively, it is preliminarily determined to be case 1 or case 2, in which the circle mark point with the number of collinear circle mark points between V is 1 is marked as S, and the other circle mark point is marked as L; if the number of collinear circle mark points is 2 and 3 respectively, it is preliminarily determined to be case 3 or case 4, in which the circle mark point with the number of collinear circle mark points between V is 2 is marked as S, and the other circle mark point is marked as L;

[0040] If, the preliminary judgment is case 1 or case 2, further, if VSL is sorted clockwise, it is judged as case 1; if VSL is sorted counterclockwise, it is judged as case 2;

[0041] If, the preliminary judgment is case 3 or case 4, further, if VSL is sorted clockwise, it is judged as case 3; if VSL is sorted counterclockwise, it is judged as case 4;

[0042] If the three circular ring landmarks found satisfy any of the above combinations, the imaging coordinates of the identified circular ring landmarks and collinear circular dot landmarks, as well as their two-dimensional coordinates in the calibration plate plane coordinate system, are used to calculate the homography of the calibration plate plane coordinate system relative to the camera image coordinate system. The mapping relationship of the homography and the two-dimensional coordinates of the core area circular ring landmarks and circular dot landmarks in the calibration plate plane coordinate system are used to match other circular ring landmarks and circular dot landmarks identified in the image. If the number of matched landmarks is greater than the set threshold, the core area is successfully identified, otherwise continue to search for other three circular ring landmark combinations.

[0043] In step 5), the camera is calibrated using a pinhole model and a lens distortion model:

[0044] The image of the calibration plate collected by the camera at each relative position corresponds to a rotation and translation change relationship (R|t). Each imaging landmark point on the calibration plate in each image corresponds to two nonlinear constraint equations, namely:

[0045] u ij =f(R i ,t i ,fu,fv,u0,v0,k1,k2,k3,p1,p2,X ij ,Y ij ,Z ij )

[0046] v ij =g(R i ,t i ,fu,fv,u0,v0,k1,k2,k3,p1,p2,X ij ,Y ij ,Z ij )

[0047] Among them, u ij and v ij represents the imaging coordinates corresponding to the jth control point in the i-th image collected during the camera calibration process; X ij , Y ij and Z ij represents the three-dimensional coordinates of the jth control point in the i-th image collected during the camera calibration process in the calibration plate coordinate system, where Z ij Set to 0; (R i |t i ) represents the rotation and translation change relationship of the calibration plate coordinate system of the i-th image collected during the camera calibration process relative to the camera coordinate system; fu and fv are the focal ratios in pixels, u0 and v0 are reference points, fu, fv, u0, v0 are camera intrinsic parameters, k1, k2, k3 are radial distortion coefficients, and p1, p2 are tangential distortion coefficients;

[0048] The camera calibration is completed based on the optimal solution of the camera intrinsic parameters, radial distortion coefficient, and tangential distortion coefficient.

[0049] The present invention has the following beneficial effects and advantages:

[0050] 1. The calibration plate provided by the present invention can make the marking points fill the camera field of view as much as possible during the camera calibration process, thereby improving the accuracy of camera calibration.

[0051] 2. The present invention meets the needs of calibrating the relative posture relationship of binocular (or multi-ocular) cameras, that is, as long as the core area on the calibration plate is within the common field of view of the binocular (or multi-ocular) camera, the area of ​​the extended area can be increased according to actual needs to ensure that there are enough landmark points within the field of view of each camera, thereby improving the calibration accuracy of the binocular (or multi-ocular) camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A schematic diagram of the structure of the present invention;

[0053] Among them, 1 is the roundabout marking point No. 1, 2 is the roundabout marking point No. 2, 3 is the roundabout marking point No. 3, 4 is the roundabout marking point No. 4, and 5 is the roundabout marking point No. 5. DETAILED DESCRIPTION

[0054] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0055] like Figure 1 As shown, the present invention provides a plane calibration plate composed of circular dot marking points and circular ring marking points, the marking points are coplanar points, and the marking points are distributed in an array on the calibration plate. According to the different distribution areas of the marking points, the calibration plate is divided into a core area and an extension area. The core area is mainly used for identification and positioning of the calibration plate; the extension area mainly provides a larger imaging area and increases the number of constraint equations for solving parameters of camera calibration. During the calibration process, it is necessary to ensure that the core area on the calibration plate is within the field of view of the camera, and the extension area fills the field of view of the camera as much as possible. Some marking points in the extension area may exceed the field of view.

[0056] The calibration plate can be divided into target area and background area according to the recognition content. The dot mark points and the ring mark points are the target area, and the rest of the area is the background area. The inner circle of the ring mark point also belongs to the background area. The target area is set to black and the background area is set to white.

[0057] The ring landmarks are composed of concentric circles;

[0058] The inner diameter is equal to half the outer diameter;

[0059] The outer diameter of the ring marking point is equal to the diameter of the dot marking point;

[0060] The diameter of the dot mark point is equal to the outer diameter of the ring mark point, and the inner diameter of the ring mark point is equal to half of the outer diameter of the ring mark point; the inner circle and outer circle of the ring mark point are concentric circles;

[0061] The center row spacing and center column spacing of the dot marking points and the ring marking points are both twice the diameter of the dot marking points.

[0062] The core area is a pattern consisting of 5 circular marking points and 15 dot marking points arranged in 4 rows and 5 columns, where:

[0063] The first row from left to right is marked as a circle, a dot, a circle, a dot, and a circle. The three circles from left to right are marked as circle 1, circle 2, and circle 3.

[0064] The second line contains 5 dot feature points;

[0065] The third line contains 5 dot feature points;

[0066] The fourth row from left to right is marked as a ring, a dot, a dot, a dot, and a ring. From left to right, the two rings are marked as ring 4 and ring 5.

[0067] The epitaxial area is composed of circular dot markers distributed in an array;

[0068] In order to further illustrate the recognition method of the calibration plate imaging, the coordinate system related to the camera calibration process is first defined.

[0069] A camera calibration method comprises the following steps:

[0070] 1) Camera calibration, including calibration plate coordinate system calibration, calibration plate plane coordinate system calibration, image coordinate system calibration and camera coordinate system calibration;

[0071] During the camera calibration image acquisition process, the calibration plate is placed in front of the camera lens as required;

[0072] 2) Collect no less than 4 calibration plate images. For the collected calibration plate images, the calibration plate recognition first performs binary segmentation on the image, performs region growth on the imaging area of ​​the marker point, judges the area after the growth, and identifies the dot marker points and the ring marker points in the image;

[0073] 3) Traverse and search for all the identified circular marking points. For each circular marking point found, find the other two circular marking points with the closest Euclidean distance to the circular marking point. According to whether the combination of the three circular marking points meets the set combination, further identify other circular marking points and dot marking points in the core area. According to the recognition situation, determine whether the core area of ​​the calibration plate is successfully recognized;

[0074] 4) If the core area of ​​the calibration plate is successfully identified, the homography mapping relationship and the two-dimensional coordinates of the dot markers in the extension area in the calibration plate plane coordinate system are used to match the identified dot markers in the image, and the dot markers and ring markers in the core area and the dot markers in the extension area that are successfully matched are used to calculate the homography of the calibration plate plane coordinate system relative to the camera image coordinate system again, and then try to continue matching the dot markers that have not been matched in the image until no more dot markers can be matched, and the identification of the extension area of ​​the calibration plate is completed;

[0075] 5) After the calibration plate recognition in each image is completed, the recognition result of the calibration plate is a set of two-dimensional coordinates and imaging coordinates of the landmark points in the plane coordinate system of the calibration plate. Each image corresponds to a set of rotation and translation change relationships (Ri|ti). All recognition results are used as input variables of nonlinear optimization for optimization and solution, and finally the camera calibration parameters are obtained, that is, the camera calibration is completed.

[0076] In step 1), the calibration plate coordinate system: the geometric center position of the calibration plate surface is defined as the origin of the calibration plate coordinate system; the direction from the No. 1 ring mark point to the No. 3 ring mark point is defined as the positive direction of the X axis of the calibration plate coordinate system; the direction from the No. 1 ring mark point to the No. 4 ring mark point is defined as the positive direction of the Y axis of the calibration plate coordinate system; the Z axis direction of the calibration plate coordinate system satisfies the right-hand rule with the X axis and Y axis directions;

[0077] Calibration plate plane coordinate system: The origin of the coordinate system is the origin of the calibration plate coordinate system, and the X-axis and Y-axis of the coordinate system are the same as the X-axis and Y-axis of the calibration plate coordinate system;

[0078] Image coordinate system: The origin is located in the upper left corner of the image, the positive direction of the u-axis is horizontally to the right, and the positive direction of the v-axis is vertically downward;

[0079] Camera coordinate system: The origin is defined as the position of the lens optical center, the positive direction of the X-axis is the same as the positive direction of the u-axis of the image coordinate system, the positive direction of the Y-axis is the same as the positive direction of the v-axis of the image coordinate system, and the Z-axis direction satisfies the right-hand rule with the X-axis and Y-axis directions;

[0080] Camera calibration uses the pinhole model and lens distortion model;

[0081] In formula 2, (X, Y, Z) is the coordinate of the marker point in the calibration plate coordinate system. Since the present invention uses a plane calibration plate, the Z component of the marker point in the calibration plate coordinate system can be set to 0; (R|t) is the rotation and translation matrix of the calibration plate relative to the camera coordinate system; (X, Y, Z) is transformed into the coordinates (x, y, z) in the camera coordinate system through the (R|t) rotation and translation transformation, where the z of the marker point in the camera coordinate system is not equal to 0;

[0082] In formula 3, fu and fv are focal ratios in pixels, u0 and v0 are reference points, fu, fv, u0, v0 are collectively referred to as camera intrinsic parameters; k1, k2 and k3 are radial distortion coefficients of the camera lens, p1 and p2 are tangential distortion coefficients of the camera lens, k1, k2, k3 and p1, p2 are collectively referred to as distortion coefficients; u and v are imaging coordinates corresponding to the marker points (including distortion);

[0083]

[0084] x′=x / z

[0085] y′=y / z

[0086] r 2 = x′ 2 +y′ 2

[0087] x″=x′·(1+k1·r 2 +k2·r 4 +k3·r 6 )+2·p1·x′·y′+p2·(r 2 +2·x′ 2 )

[0088] y″=y′·(1+k1·r 2 +k2·r 4 +k3·r 6 )+p1·(r 2 +2·y′ 2 )+2·p2·x′·y′

[0089] u=fu·x″+u0

[0090] v=fv·y″+v0 (3)

[0091] The image of the calibration plate collected by the camera at each relative position corresponds to a rotation and translation change relationship (R|t), and each imaging landmark point on the calibration plate in each image corresponds to two constraint equations;

[0092] The following uses Formula 4 as a more general description of Formula 2 and Formula 3, where u ij and vij Represents the imaging coordinates corresponding to the jth control point in the i-th image collected during the camera calibration process;

[0093] X ij , Y ij and Z ij represents the three-dimensional coordinates of the jth control point in the i-th image collected during the camera calibration process in the calibration plate coordinate system, where Z ij Can be set to 0;

[0094] u ij =f(R i ,t i ,fu,fv,u0,v0,k1,k2,k3,p1,p2,X ij ,Y ij ,Z ij )

[0095] v ij =g(R i ,t i ,fu,fv,u0,v0,k1,k2,k3,p1,p2,X ij ,Y ij ,Z ij ) (4)

[0096] (R i |t i ) represents the rotation and translation relationship between the calibration plate coordinate system of the i-th image collected during the camera calibration process and the camera coordinate system;

[0097] fu, fv, u0, v0 are camera internal parameters and k1, k2, k3, p1, p2 are distortion coefficients;

[0098] The purpose of camera calibration in the present invention is to determine the above camera internal parameters fu, fv, u0, v0 and distortion coefficients k1, k2, k3, p1, p2;

[0099] In formula 4, a correctly identified landmark corresponds to two constraint equations; assuming that a total of N images (N is not less than 4) are collected during a calibration process, the number of landmarks that can be correctly identified in each image is M i (subscript i corresponds to the image number), then a total of nonlinear constraint equations, where X ij , Y ij and Z ij and u ij and v ij is the input variable of nonlinear optimization, (R i |t i), fu, fv, u0, v0 and k1, k2, k3, p1, p2 are the target variables of nonlinear optimization;

[0100] The present invention uses the LM algorithm to complete the nonlinear optimization solution. The following only describes the identification method of the calibration plate imaging, that is, the input variables of the nonlinear optimization process;

[0101] During the camera calibration process, the calibration plate is placed in front of the camera lens during image acquisition. The angle between the positive Z-axis of the calibration plate coordinate system and the positive Z-axis of the camera coordinate system is required to be no less than 20 degrees and no more than 40 degrees. In addition, the core area integrity should be ensured for each calibration plate image, that is, the core area of ​​4 rows and 5 columns, a total of 20 markers, must appear completely in the image and must not intersect with the image boundary.

[0102] A total of no less than 4 calibration plate images are collected. For the collected calibration plate images, the calibration plate recognition first identifies the dot markers and ring markers in the image, and then identifies the core area of ​​the calibration plate according to the combination of the dot markers and ring markers. Finally, according to the recognition results of the core area, the imaging markers in the extension area are further searched.

[0103] In order to identify the landmarks, the image is first segmented into two values, and then region growing is performed on the landmark imaging area, and the region where the growth ends is judged:

[0104] If the aspect ratio of the imaging circumscribed rectangle is less than the set threshold, the ratio of the imaging area of ​​the marker point to the area of ​​the circumscribed rectangle of the region is greater than the set threshold, and there is a hole with an area greater than the set threshold at the imaging center of the marker point, then the image region is judged to be the area where the circular marker point is located;

[0105] If the aspect ratio of the imaging circumscribed rectangle is less than the set threshold, the ratio of the imaging area of ​​the marker point to the area of ​​the circumscribed rectangle of the region is greater than the set threshold, and the marker point is not a circular marker point, then the image region is determined to be the region where the circular marker point is located;

[0106] The area that does not satisfy the imaging feature constraints of the ring mark point nor the imaging feature constraints of the dot mark point is no longer used as a candidate area for the next step of target recognition;

[0107] The circular signs and dot signs found above are placed in a queue array respectively. The circular sign point queue array is traversed to find all the recognized circular signs. Then, for each circular sign point found, the other two circular signs with the closest Euclidean distance to the circular sign point are found. Then, it is determined whether the above three circular signs meet one of the following four combinations:

[0108] Case 1. The No. 1 circular marking point, the No. 2 circular marking point and the No. 4 circular marking point as described in claim 3 are characterized in that there is one collinear circular marking point between the No. 1 circular marking point and the No. 2 circular marking point, there are two collinear circular marking points between the No. 1 circular marking point and the No. 4 circular marking point, and the No. 1 circular marking point, the No. 2 circular marking point and the No. 4 circular marking point satisfy the clockwise order;

[0109] Case 2. The No. 2 circular marking point, the No. 3 circular marking point and the No. 5 circular marking point as described in claim 3 are characterized in that there is one collinear circular marking point between the No. 2 circular marking point and the No. 3 circular marking point, there are two collinear circular marking points between the No. 3 circular marking point and the No. 5 circular marking point, and the No. 2 circular marking point, the No. 3 circular marking point and the No. 5 circular marking point satisfy the clockwise order;

[0110] Case 3. The No. 1 circular marking point, the No. 4 circular marking point and the No. 5 circular marking point as described in claim 3 are characterized in that there are 2 collinear circular marking points between the No. 1 circular marking point and the No. 4 circular marking point, there are 3 collinear circular marking points between the No. 4 circular marking point and the No. 5 circular marking point, and the No. 1 circular marking point, the No. 4 circular marking point and the No. 5 circular marking point satisfy the counterclockwise order;

[0111] Case 4. The No. 3 circle mark point, the No. 4 circle mark point and the No. 5 circle mark point as described in claim 3 are characterized in that there are 2 collinear circle mark points between the No. 3 circle mark point and the No. 5 circle mark point, there are 3 collinear circle mark points between the No. 4 circle mark point and the No. 5 circle mark point, and the No. 3 circle mark point, the No. 4 circle mark point, and the No. 5 circle mark point satisfy the counterclockwise order;

[0112] Determine whether the core area of ​​the calibration plate is successfully identified, specifically:

[0113] For the three circular markers found, first calculate the vertex angle of each circular marker in the triangle, mark the circular marker with the largest vertex angle as V, and then determine the number of collinear circular markers between V and the other two circular markers;

[0114] If the number of collinear circle mark points is 1 and 2 respectively, it is preliminarily determined to be case 1 or case 2, in which the circle mark point with the number of collinear circle mark points between V is 1 is marked as S, and the other circle mark point is marked as L; if the number of collinear circle mark points is 2 and 3 respectively, it is preliminarily determined to be case 3 or case 4, in which the circle mark point with the number of collinear circle mark points between V is 2 is marked as S, and the other circle mark point is marked as L;

[0115] If, the preliminary judgment is case 1 or case 2, further, if VSL is sorted clockwise, it is judged as case 1; if VSL is sorted counterclockwise, it is judged as case 2;

[0116] If, the preliminary judgment is case 3 or case 4, further, if VSL is sorted clockwise, it is judged as case 3; if VSL is sorted counterclockwise, it is judged as case 4;

[0117] If the three circular ring landmarks found satisfy any of the above combinations, the imaging coordinates of the identified circular ring landmarks and collinear circular dot landmarks, as well as their two-dimensional coordinates in the calibration plate plane coordinate system, are used to calculate the homography of the calibration plate plane coordinate system relative to the camera image coordinate system. The mapping relationship of the homography and the two-dimensional coordinates of the core area circular ring landmarks and circular dot landmarks in the calibration plate plane coordinate system are used to match other circular ring landmarks and circular dot landmarks identified in the image. If the number of matched landmarks is greater than the set threshold, the core area is successfully identified, otherwise continue to search for other three circular ring landmark combinations.

[0118] Among them, the homography matrix mapping relationship and the homography matrix mapping method are as follows:

[0119]

[0120]

[0121] in, Represents the homogeneous coordinates of the i-th marker point identified in the k-th calibration plate acquisition image in the calibration plate plane coordinate system; represents the homogeneous coordinates of the i-th landmark point identified in the k-th calibration plate captured image in the image coordinate system; H represents the homography matrix.

[0122] If the three circular marker points found meet one of the four combinations, the two-dimensional coordinates of the corresponding marker points in the calibration plate plane coordinate system and the corresponding imaging coordinates are used to calculate the homography matrix of the calibration plate plane coordinate system relative to the camera image coordinate system, and the mapping relationship of the homography matrix and the two-dimensional coordinates of other marker points in the calibration plate plane coordinate system are used to match the imaging recognition results corresponding to other circular marker points and dot marker points in the core area of ​​the calibration plate in the image. If the number of matched marker points is greater than the set threshold, the core area is successfully identified, otherwise continue to search for other three circular marker point combinations;

[0123] If the three circular marking points found do not satisfy any of the four combinations, continue to search for other three circular marking point combinations;

[0124] If the above core area recognition is successful, the imaging recognition results corresponding to the circular dot markers in the extension area of ​​the calibration plate in the image are matched using the mapping relationship of the homography matrix and the two-dimensional coordinates of other markers in the calibration plate plane coordinate system. The homography matrix of the calibration plate plane coordinate system relative to the camera image coordinate system is calculated again using the successfully matched markers. Then, the imaging recognition results corresponding to the circular dot markers in the extension area of ​​the calibration plate in the image are tried again until no more markers can be matched.

[0125] At this point, the recognition of the calibration plate in each image is completed. The recognition result of the calibration plate is a set of two-dimensional coordinates and imaging coordinates of the landmark points in the plane coordinate system of the calibration plate. Each image corresponds to a set of rotation and translation change relationships (R i |t i ), the calibration plate recognition results in all the images to be recognized are used as the input variables of the nonlinear optimization solution process to complete the optimization solution, and finally the camera calibration results fu, fv, u0, v0 and k1, k2, k3, p1, p2 are obtained;

[0126] Example:

[0127] Assuming that the camera's field of view is 50°*50° and the optimal working distance is about 700mm, it can be calculated that at a distance of 700mm, the camera's field of view coverage is 652mm*652mm. Therefore, in order to calibrate the camera at a distance of 700mm and to make the calibration plate fill the camera's field of view as much as possible, the size of the calibration plate should be equivalent to the camera's field of view coverage. Therefore, the specific design example of the calibration plate according to the solution of the present invention is as follows:

[0128] The calibration plate consists of 20 rows and 23 columns of marking points;

[0129] 1) The diameter of the dot mark is 15mm, the outer diameter of the ring mark is 15mm, and the inner diameter of the ring mark is 7.5mm;

[0130] 2) The center row spacing of the marking points is 30mm, and the column spacing is 30mm;

[0131] 3) The first mark point in the upper left corner of the core area is located at the 9th row and 10th column of the entire calibration plate. The area inside the dotted box in the figure is the core area;

[0132] 4) The core area consists of 4 rows and 5 columns of landmarks, of which the first row contains 3 circular landmarks, located in the first, third and fifth columns respectively; the fourth row contains 2 circular landmarks, located in the first and fifth columns respectively, and the remaining landmarks in the core area are dot landmarks;

[0133] 5) The extension area consists only of dot markers;

[0134] 6) In order to obtain a relatively high imaging quality, the target area is black and the background area is white;

[0135] 7) During the camera calibration process, the calibration plate is placed in front of the camera lens during image acquisition. The angle between the positive Z-axis of the calibration plate coordinate system and the positive Z-axis of the camera coordinate system is required to be no less than 20 degrees and no more than 40 degrees. In addition, the core area integrity should be ensured for each calibration plate image, that is, the core area of ​​4 rows and 5 columns, a total of 20 markers, must appear completely in the image and must not intersect with the image boundary.

[0136] 8) Collect no less than 4 calibration plate images in total. For the collected calibration plate images, the calibration plate recognition first performs binary segmentation on the image, then identifies the dot markers and ring markers in the image by region growing, and then identifies the core area of ​​the calibration plate according to the combination of the dot markers and ring markers. Finally, further search for imaging markers in the extension area according to the identification results of the core area;

[0137] 9) The recognition results of each calibration plate (the two-dimensional coordinates of the marker points in the plane coordinate system of the calibration plate and the corresponding imaging coordinates) are used as the input parameters of the nonlinear optimization, and then the LM nonlinear optimization algorithm is used to calculate the optimization target variables fu, fv, u0, v0 and k1, k2, k3, p1, p2 to complete the camera calibration.

[0138] The specific implementation modes of the present invention are described above in conjunction with the accompanying drawings, but these descriptions cannot be understood as limiting the scope of the present invention. The protection scope of the present invention is defined by the attached claims, and any changes based on the claims of the present invention are within the protection scope of the present invention.

Claims

1. A camera calibration board, comprising: A plane plate and dot marker points and annular marker points arrayed on the plane plate; the dot marker points and annular marker points are coplanar points; the camera calibration plate is divided into a core area and an extension area according to the marker point distribution area; the core area, an array composed of dot marker points and annular marker points, is used to be set within the camera field of view during the calibration process; the extension area, an array composed of dot marker points, is used to cover the camera field of view during the calibration process, and some marker points in the extension area are allowed to exceed the camera field of view when the camera calibration image is collected; characterized in that its calibration method comprises the following steps: The core area is an array of 4 rows and 5 columns; the first row of the core area is, from left to right, a circle mark point, a circle mark point, a circle mark point, and a circle mark point, and the three circles from left to right are circle mark point No. 1, circle mark point No. 2, and circle mark point No. 3; The two ends of the last row in the core area are circular marking points, namely circular marking point No. 4 and circular marking point No. 5, and the point between the two circular marking points is a dot marking point; The marking points in the remaining rows of the core area are all dot marking points; 1) Camera calibration, including calibration of the calibration plate coordinate system, calibration of the calibration plate plane coordinate system, image coordinate system calibration and camera coordinate system calibration; during the camera calibration image acquisition process, the calibration plate is placed in front of the camera lens as required; (1) Definition of the calibration plate coordinate system: The geometric center position of the calibration plate surface is defined as the origin of the calibration plate coordinate system; the direction from the No. 1 ring mark point to the No. 3 ring mark point is defined as the positive direction of the calibration plate coordinate system X axis; the direction from the No. 1 ring mark point to the No. 4 ring mark point is defined as the positive direction of the calibration plate coordinate system Y axis; the Z axis direction of the calibration plate coordinate system satisfies the right-hand rule with the X axis and Y axis directions; (2) Definition of the calibration plate plane coordinate system: The origin of the calibration plate plane coordinate system is the origin of the calibration plate coordinate system, and the X-axis and Y-axis of the calibration plate plane coordinate system are the same as the X-axis and Y-axis of the calibration plate coordinate system; (3) Definition of the image coordinate system: The origin of the image coordinate system is located in the upper left corner of the image, the positive direction of the u-axis is horizontally to the right, and the positive direction of the v-axis is vertically downward; (4) Definition of the camera coordinate system: The origin of the camera coordinate system is defined as the position of the lens optical center, the positive direction of the X-axis is the same as the positive direction of the u-axis of the image coordinate system, the positive direction of the Y-axis is the same as the positive direction of the v-axis of the image coordinate system, and the Z-axis direction satisfies the right-hand rule with the X-axis and Y-axis directions; After determining each coordinate system, set the angle between the positive Z-axis direction of the calibration plate coordinate system and the positive Z-axis direction of the camera coordinate system during image acquisition to be no less than 20° and no more than 40°, and ensure the integrity of the core area for each calibration plate image, that is, the 20 markers in the 4 rows and 5 columns of the core area appear completely in the image and cannot intersect with the image boundary; 2) Collect no less than 4 calibration plate images. For the collected calibration plate images, the calibration plate recognition first performs binary segmentation on the image, performs region growth on the imaging area of ​​the marker point, judges the area after the growth, and identifies the dot marker points and the ring marker points in the image; 3) Traverse and search for all the identified circular marking points. For each circular marking point found, find the other two circular marking points with the closest Euclidean distance to the circular marking point. According to whether the combination of the three circular marking points meets the set combination, further identify other circular marking points and dot marking points in the core area. According to the recognition situation, determine whether the core area of ​​the calibration plate is successfully recognized; 4) If the core area of ​​the calibration plate is successfully identified, the homography mapping relationship and the two-dimensional coordinates of the dot markers in the extension area in the calibration plate plane coordinate system are used to match the identified dot markers in the image, and the dot markers and ring markers in the core area and the dot markers in the extension area that are successfully matched are used to calculate the homography of the calibration plate plane coordinate system relative to the camera image coordinate system again, and then try to continue matching the dot markers that have not been matched in the image until no more dot markers can be matched, and the identification of the extension area of ​​the calibration plate is completed; 5) After the calibration plate recognition in each image is completed, the recognition result of the calibration plate is a set of two-dimensional coordinates and imaging coordinates of the marker points in the calibration plate plane coordinate system. Each image corresponds to a set of rotation and translation change relationships (R i |t i ), all recognition results are used as input variables of nonlinear optimization for optimization and solution, and finally the camera calibration parameters are obtained, that is, the camera calibration is completed.

2. The camera calibration plate according to claim 1, characterized in that: The diameter of the dot mark point is equal to the outer diameter of the ring mark point, and the inner diameter of the ring mark point is equal to half of the outer diameter of the ring mark point; the inner circle and outer circle of the ring mark point are concentric circles; The center row spacing and center column spacing of the dot marking points and the ring marking points are both twice the diameter of the dot marking points.

3. The camera calibration plate according to claim 1, characterized in that: The dot mark points and the ring mark points are used as target areas for image processing, and the rest of the calibration plate and the inner circle area of ​​the ring mark points are used as background areas.

4. The camera calibration plate according to claim 1, characterized in that: In step 2), the region growth is performed on the imaging region of the marker point, and the region where the growth is completed becomes the pending region, and the pending region is judged, specifically: If the length-to-width ratio of the bounding rectangle of the pending area is less than the set threshold, the ratio of the area of ​​the pending area to the area of ​​the bounding rectangle of the pending area is greater than the set threshold, and there is a hole with an area greater than the set threshold at the center of the pending area, then the image area is judged to be the area where the circular mark point is located; If the length-to-width ratio of the circumscribed rectangle of the pending area is less than the set threshold, the ratio of the area of ​​the pending area to the area of ​​the circumscribed rectangle of the pending area is greater than the set threshold, and the pending area is not the area where the circular mark point is located, then the pending area is determined to be the area where the circular mark point is located; The undetermined area that does not satisfy the imaging feature constraints of the ring mark point nor the imaging feature constraints of the dot mark point will no longer be used for the next step of target recognition.

5. The camera calibration plate according to claim 1, characterized in that: In step 3), the combination of settings includes: Case 1: There is one collinear circular mark between the No. 1 circular mark and the No. 2 circular mark, and there are two collinear circular mark between the No. 1 circular mark and the No. 4 circular mark. The No. 1 circular mark, the No. 2 circular mark, and the No. 4 circular mark satisfy the clockwise order. In the triangle formed by the No. 1 circular mark, the No. 2 circular mark and the No. 4 circular mark on the calibration plate, the No. 1 circular mark corresponds to the right-angle vertex. Case 2: There is one collinear circle mark between the No. 2 circle mark and the No. 3 circle mark, and there are two collinear circle mark between the No. 3 circle mark and the No. 5 circle mark. The No. 3 circle mark, the No. 2 circle mark, and the No. 5 circle mark satisfy the counterclockwise order. In the triangle formed by the No. 2 circle mark, the No. 3 circle mark and the No. 5 circle mark on the calibration plate, the No. 3 circle mark corresponds to the right-angle vertex. Case 3: There are 2 collinear circular mark points between the No. 1 circular mark point and the No. 4 circular mark point, and there are 3 collinear circular mark points between the No. 4 circular mark point and the No. 5 circular mark point. The No. 4 circular mark point, the No. 1 circular mark point, and the No. 5 circular mark point satisfy the clockwise order. In the triangle formed by the No. 1 circular mark point, the No. 4 circular mark point and the No. 5 circular mark point on the calibration plate, the No. 4 circular mark point corresponds to the right-angle vertex. Case 4: There are 2 collinear circle mark points between the No. 3 circle mark point and the No. 5 circle mark point, and there are 3 collinear circle mark points between the No. 4 circle mark point and the No. 5 circle mark point. The No. 5 circle mark point, the No. 3 circle mark point, and the No. 4 circle mark point satisfy the counterclockwise order. In the triangle formed by the No. 3 circle mark point, the No. 4 circle mark point and the No. 5 circle mark point on the calibration plate, the No. 5 circle mark point corresponds to the right-angle vertex.

6. The camera calibration plate calibration method according to claim 1, characterized in that: In step 3), it is determined whether the core area of ​​the calibration plate is successfully identified, specifically: For the three circular markers found, first calculate the vertex angle of each circular marker in the triangle, mark the circular marker with the largest vertex angle as V, and then determine the number of collinear circular markers between V and the other two circular markers; If the number of collinear circle mark points is 1 and 2 respectively, it is preliminarily determined to be case 1 or case 2, in which the circle mark point with the number of collinear circle mark points between V is 1 is marked as S, and the other circle mark point is marked as L; if the number of collinear circle mark points is 2 and 3 respectively, it is preliminarily determined to be case 3 or case 4, in which the circle mark point with the number of collinear circle mark points between V is 2 is marked as S, and the other circle mark point is marked as L; If, the preliminary judgment is case 1 or case 2, further, if VSL is sorted clockwise, it is judged as case 1; if VSL is sorted counterclockwise, it is judged as case 2; If, the preliminary judgment is case 3 or case 4, further, if VSL is sorted clockwise, it is judged as case 3; if VSL is sorted counterclockwise, it is judged as case 4; If the three circular ring landmarks found satisfy any of the above combinations, the imaging coordinates of the identified circular ring landmarks and collinear circular dot landmarks, as well as their two-dimensional coordinates in the calibration plate plane coordinate system, are used to calculate the homography of the calibration plate plane coordinate system relative to the camera image coordinate system. The mapping relationship of the homography and the two-dimensional coordinates of the core area circular ring landmarks and circular dot landmarks in the calibration plate plane coordinate system are used to match other circular ring landmarks and circular dot landmarks identified in the image. If the number of matched landmarks is greater than the set threshold, the core area is successfully identified, otherwise continue to search for other three circular ring landmark combinations.

7. The camera calibration plate calibration method according to claim 1, characterized in that: In step 5), the camera is calibrated using a pinhole model and a lens distortion model: The image of the calibration plate collected by the camera at each relative position corresponds to a rotation and translation change relationship (R|t). Each imaging landmark point on the calibration plate in each image corresponds to two nonlinear constraint equations, namely: u ij =f(R i ,t i ,fu,fv,u0,v0,k1,k2,k3,p1,p2,X ij ,Y ij ,Z ij ) v ij =g(R i ,t i ,fu,fv,u0,v0,k1,k2,k3,p1,p2,X ij ,Y ij ,Z ij ) Among them, u ij and v ij represents the imaging coordinates corresponding to the jth control point in the i-th image collected during the camera calibration process; X ij , Y ij and Z ij represents the three-dimensional coordinates of the jth control point in the i-th image collected during the camera calibration process in the calibration plate coordinate system, where Z ij Set to 0; (R i |t i ) represents the rotation and translation change relationship of the calibration plate coordinate system of the i-th image collected during the camera calibration process relative to the camera coordinate system; fu and fv are focal ratios in pixels, u0 and v0 are reference points, fu, fv, u0, v0 are camera intrinsic parameters, k1, k2, k3 are radial distortion coefficients, and p1, p2 are tangential distortion coefficients; The camera calibration is completed based on the optimal solution of the camera intrinsic parameters, radial distortion coefficient, and tangential distortion coefficient.

Images