Camera calibration verification method, storage medium and electronic device

Abstract

The present invention relates to a camera calibration verification method, a storage medium and an electronic device, wherein the camera calibration verification method comprises the following steps: S10, shooting an image, identifying feature points of a calibration pattern in the image, and calculating the external parameters of the camera in combination with parameters of the calibration pattern, camera internal parameters and distortion coefficients; S20, obtaining a verification pattern for camera verification, identifying a first feature point and a second feature point in the verification pattern, and calculating the world coordinates of the first feature point according to the first feature point and the second feature point; S30, comparing the calculated world coordinates of the first feature point with the measured world coordinates to obtain an accuracy value; S40, if the accuracy value meets the requirements, confirming that the calibration is successful and outputting the external parameters as the final external parameters, otherwise outputting an evaluation result including the accuracy value. The camera calibration verification method of the present invention can automatically perform the step of verifying the accuracy immediately after calibrating the external parameters, which is efficient and fast, and saves manpower and time.

Description

Technical Field

[0001] The present invention relates to camera calibration and verification, and in particular to a camera calibration and verification method, a storage medium and an electronic device. Background Art

[0002] The on-board camera is the core sensor in the field of automotive driver assistance systems (ADAS) and autonomous driving. Most application scenarios require the calibration of the camera's internal and external parameters. Usually, the internal parameters (such as camera focal length, camera eccentricity, and camera distortion coefficient) are calibrated on the camera production line. In the vehicle factory, after the camera is installed, the camera's external parameters (installation position and installation posture) are calibrated. The verification of the external parameter calibration results is processed offline. By selecting some points on a specific restricted plane (such as the ground), the calibration results are used to calculate the distance, and then the accuracy is compared with the measured distance to evaluate the accuracy. This calibration method is more labor-intensive and time-consuming, so sampling verification is generally used to verify the accuracy of certain camera parameters, and accuracy verification is not performed on each camera. As a result, after leaving the factory, some functions may still have poor effects due to insufficient camera parameter accuracy. Summary of the invention

[0003] The purpose of the present invention is to provide a flexible and efficient camera calibration and verification method, storage medium and electronic device.

[0004] A camera calibration verification method comprises the steps of:

[0005] S10, capturing an image through a camera, wherein the captured image includes a calibration pattern for camera calibration, identifying feature points of the calibration pattern, and calculating an external parameter of the camera by combining parameters of the calibration pattern, an internal parameter of the camera, and a distortion coefficient;

[0006] S20, obtaining a verification pattern for camera verification, identifying a first feature point and a second feature point in the verification pattern, and calculating the world coordinates of the first feature point according to the first feature point and the second feature point;

[0007] S30, comparing the calculated world coordinates of the first feature point with the measured world coordinates to obtain an accuracy value;

[0008] S40, if the accuracy value meets the requirement, confirm that the calibration is successful and output the external parameter as the final external parameter, otherwise output the evaluation result including the accuracy value.

[0009] As an embodiment, in step S10, a movable calibration plate and verification plate are utilized; the calibration plate is printed or sprayed or pasted or displayed with the calibration pattern, and a fixing part for convenient hanging and fixing can be fixed on the other side; the verification plate is printed or sprayed or pasted or displayed with the verification pattern, and a fixing part for convenient hanging and fixing can be fixed on the other side.

[0010] As an implementation manner, the calibration plate and the verification plate are also printed, sprayed, pasted or displayed with serial number information that can distinguish the calibration plate and the verification plate.

[0011] As an implementation manner, the calibration pattern and the verification pattern are two-dimensional codes, and the serial number information includes the two-dimensional code.

[0012] As an implementation manner, in the step S10, the image captured by the camera includes both the calibration pattern and the verification pattern, and the calibration pattern and the verification pattern in the image are determined by the serial number information.

[0013] As an implementation manner, in step S20, the method for calculating the world coordinates of the first feature point includes:

[0014] S21, performing distortion correction on the image coordinates of the first feature point and the second feature point according to the intrinsic parameters of the camera to obtain corrected image coordinates (xUndis1, yUndis1) and (xUndis2, yUndis2);

[0015] S22, let the world coordinates of the first feature point be (X 1 , Y 1 , Z 1 ), the world coordinates of the second feature point are (X 2 , Y 2 , Z 2 ), ΔW c Indicates (X 1 , Y 1 , Z 1 ) and (X 2 , Y 2 , Z 2 )’s world coordinate difference, vector I 1 =(xUndis1, yUndis1, 1) T , I 2 =(xUndis2, yUndis2, 1) T , the rotation matrix of the external parameter is R, the translation vector of the external parameter is T, the internal parameter matrix is ​​A, and the matrix H = A*R, then the distance S between the verification plate and the optical center of the camera is calculated by formula (1):

[0016]

[0017] in

[0018] x 1 =(1,0,0)*(H -1 *I 1 ),

[0019] x 2 =(1,0,0)*(H -1 *I 2 );

[0020] S23, calculate the world coordinates of the first feature point by formula (2): 1 , Y 1 , Z 1 ):

[0021]

[0022] As an implementation manner, the physical plane where the verification pattern is located is perpendicular to the horizontal axis of the world coordinate system, and the distance S is the horizontal distance between the physical plane where the verification pattern is located and the optical center of the camera.

[0023] A storage medium includes instructions, wherein the instructions are used to implement the above-mentioned camera calibration and verification method.

[0024] An electronic device comprises a processor and the above-mentioned storage medium, wherein the electronic device calls instructions of the storage medium through the processor to implement the above-mentioned camera calibration and verification method.

[0025] The camera calibration and verification method of the present invention can automatically perform the step of verifying accuracy immediately after calibrating the external parameters, which is efficient and fast, saving manpower and time. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 Schematic diagram of a verification pattern of a verification board in one embodiment.

[0027] Figure 2 A schematic diagram of an image captured by a camera in a camera calibration and verification method according to an embodiment. DETAILED DESCRIPTION

[0028] The camera calibration and verification method of the present invention will be further described in detail below in conjunction with specific embodiments and drawings.

[0029] In the calibration and verification method of the present invention, a movable calibration plate and verification plate are used. The calibration plate and verification plate are preferably rectangular or square plates, one side of which is printed or sprayed or pasted or displayed with a pattern with preset feature points (calibration pattern on the calibration plate, verification pattern on the verification plate), and the other side can be fixed with a fixing for easy hanging and fixing, the size of which can be determined according to the size of the place where the calibration and verification are implemented, the distance between the camera and the calibration plate and the verification plate, as long as the captured image is easy to identify the calibration pattern and verification pattern on both. The placement of the calibration plate and the verification plate can be determined according to the installation position of the camera. For example, when calibrating and verifying the camera arranged on the front insurance, the calibration plate and the verification plate can be hung vertically in front of the front of the car; if the camera on the exterior rearview mirror or the fender is calibrated and verified, the calibration plate and the verification plate can be hung horizontally on one side of the side of the car body. Preferably, the calibration plate and the verification plate are either placed horizontally or vertically, which is convenient for presetting the actual position of the calibration plate and the verification plate. The car manufacturer can flexibly use a bracket or wall to fix the calibration plate and verification plate in a suitable position in a suitable place outdoors or indoors, and then perform camera calibration and verification work, which is very flexible and efficient. The calibration pattern and verification pattern of the calibration plate and verification plate should include at least two feature points to facilitate the rapid positioning of the range, size and angle of the calibration pattern and verification pattern. In this embodiment, please refer to Figure 1 As shown, the calibration pattern and the verification pattern both include a two-dimensional code, preferably at least one matrix two-dimensional code, and the two-dimensional codes on the two boards may be the same or different. Two corner points of the four corners of the matrix two-dimensional code may be used as feature points to be identified, and the middle two-dimensional code information may contain the id information of each two-dimensional code. In order to improve the efficiency of calibration and verification, the calibration plate and the verification plate also include serial number information that can distinguish the calibration plate and the verification plate. The serial number information may be the id information of the two-dimensional code constructed in a preset order, and the calibration plate and the verification plate may also be distinguished by setting an area of ​​interest. In this way, in the calibration and verification method, the calibration plate and the verification plate may be distinguished by the serial number information. When multiple calibration plates and verification plates are provided, it is also possible to distinguish which calibration plate and verification plate are photographed, and then determine the orientation of the calibration plate and the verification plate, so as to achieve one site, and all vehicle-mounted cameras can be calibrated and verified in one operation.

[0030] In addition, the positions of the calibration plate and verification plate are preset according to the theoretical installation position of the camera, the vehicle body parameters, and the size of the calibration field, so that the image captured by the camera includes both the image of the calibration plate and the image of the verification plate. The actual image captured can be used as a reference. Figure 2 As shown, calibration and verification are automatically completed at one time, which is efficient and fast.

[0031] The calibration and verification method of the present invention is used to determine the external parameters of a camera, and the instructions for implementing the method are stored in a storage medium. The storage medium can be fixed in the electronic system of the car, or it can be a pluggable storage medium. The user can call the instructions of the storage medium through a processor of an electronic device to implement the above-mentioned camera calibration and verification method. For example, when the calibration and verification method of the present invention is used to calibrate and verify the external parameters of a vehicle-mounted camera, the electronic device can be a vehicle-mounted host or an external control host connected to the vehicle-mounted host, which runs the calibration and verification method of the present invention by calling instructions to calibrate and verify the external parameters of all vehicle-mounted cameras.

[0032] During the camera calibration process, the following coordinate systems are used: (1) World coordinate system, which is a coordinate system used to describe the camera position coordinates. For example, a world coordinate system is established using a calibration plate or a vehicle coordinate system. In this embodiment, the world coordinate system is established using the vehicle body. The longitudinal axis of the vehicle body is the Xw axis, with the forward direction being positive, the transverse axis of the vehicle body is the Yw axis, with the left side being positive, and the Zw axis is vertically upward, satisfying the right-hand coordinate system; (2) Camera coordinate system, with the optical center as the origin and the optical axis as the Zc axis. Xc and Yc are parallel to the x-axis and y-axis of the imaging plane, respectively; (3) Image coordinate system, a 2D coordinate system of the imaging plane. The origin is the intersection of the optical axis and the imaging plane. The x-axis and y-axis are parallel to the Xc axis and Yc axis of the camera coordinate system, and the optical center is the midpoint of the image; (4) Pixel coordinate system, which finally presents the image in front of us. The upper left corner is the origin, and the two axes are parallel to the image coordinate system, respectively.

[0033] The camera calibration and verification method of the present invention is described below with a specific embodiment, and the method includes steps S10 to S50.

[0034] Among them, step S10, capture an image through a camera, identify the feature points of the calibration pattern of the calibration plate in the captured image, and calculate the external parameters of the camera in combination with the parameters of the actual calibration plate, the camera internal parameters and the distortion coefficient. This step specifically determines the calibration pattern used for calibration through the serial number information in the image, then identifies the feature points of the calibration pattern in the image, confirms the correspondence between the feature points in the image and the feature points of the calibration pattern on the actual calibration plate, determines the mapping relationship between the image coordinates of the feature points and the world coordinate points, and then calculates the camera external parameters based on the camera internal parameters (focal length and eccentricity) and the distortion coefficient. The optional algorithm for calculating the camera external parameters is relatively mature, such as but not limited to Zhang Zhengyou's camera calibration calculation method, which will not be repeated here.

[0035] Step S20, determine the verification pattern for accuracy verification through the serial number information in the image, and identify the first feature point C1 and the second feature point C2 (two corner points, reference Figure 1), and calculate the world coordinates (X) of the first feature point C1 according to the first feature point C1 and the second feature point C2 1 , Y 1 , Z 1 ). The first feature point C1 is, for example but not limited to, the lower left corner of the matrix two-dimensional code in the upper left corner of the verification pattern, and the second feature point C2 is, for example but not limited to, the upper right corner of the matrix two-dimensional code. The world coordinates (X) of the first feature point C1 are calculated based on the first feature point C1 and the second feature point C2. 1 , Y 1 , Z 1 ) may include:

[0036] S21, determining that the image coordinates of the first feature point C1 and the second feature point C2 in the image coordinate system are C1(xdis1, ydis1) and C2(xdis2, ydis2), and performing distortion correction on the first feature point C1 and the second feature point C2 according to the camera internal parameters (focal length and eccentricity parameters) and the distortion coefficient, and obtaining the corrected image coordinates as (xUndis1, yUndis1) and (xUndis2, yUndis2);

[0037] S22, let the coordinates of the first feature point C1 in the world coordinate system be (X 1 , Y 1 , Z 1 ), the world coordinates of the second feature point C2 are (X 2 , Y 2 , Z 2 ), ΔWc represents (X 1 , Y 1 , Z 1 ) and (X 2 , Y 2 , Z 2 ) (ΔWc can be directly obtained based on the verification pattern parameters), vector I 1 =(xUndis1, yUndis1, 1) T , I 2 =(xUndis2, yUndis2, 1) T , the rotation matrix of the external parameter is R, the translation vector of the external parameter is T, the internal parameter matrix is ​​A, and the matrix H = A*R, then the distance S between the verification plate and the optical center of the camera can be calculated by formula (1):

[0038]

[0039] in

[0040] x 1 =(1,0,0)*(H- 1* I1 ), x 2 =(1,0,0)*(H -1 *I 2 ), in this embodiment, the verification board is perpendicular to the Xw axis of the world coordinate system, so the distance S is the Xw axial distance between the verification board and the optical center of the camera. When the verification board is placed horizontally, the distance S is also the distance from the optical center of the camera to the verification board (Xw axial).

[0041] S23, calculate the world coordinates of the first feature point C1 by formula (2): 1 , Y 1 , Z 1 ):

[0042]

[0043] Step S40, comparing the calculated world coordinates (X 1 , Y 1 , Z 1 ) and the measured world coordinates (X 1 ′,Y 1 ′,Z 1 ′), and get the accuracy value. In this example, the world coordinates are established based on the vehicle body, and the measured world coordinates (X 1 ′,Y 1 ′,Z 1 ') can be measured in advance using a measuring tool such as a laser rangefinder. The accuracy value can be the absolute value of the difference between the two or the ratio of the absolute value of the difference to the measured value.

[0044] S50, if the accuracy value meets the requirement, confirm that the calibration is successful and output the external parameter as the final external parameter, otherwise output the evaluation result including the accuracy value. In this embodiment, the calibration is confirmed to be successful when the accuracy value is less than or equal to 5%.

[0045] In this way, the car is at one station, and the camera only needs to take an image once to complete the calibration and verification steps directly, which is efficient and fast. By arranging multiple calibration plates and verification plates, the car is at one station, and each camera takes an image once to complete the calibration and verification work of all cameras.

[0046] In addition, since each matrix QR code has its own ID, in actual use, the calibration pattern and verification pattern can be reused. For a certain camera, one or several QR codes in the calibration pattern are used for calibration, and for another camera, the QR code in the calibration pattern can be used for verification. If reuse is not required, the calibration pattern and verification pattern can also be a black and white checkerboard or other patterns that are easy to identify feature points.

[0047] In the above embodiment, only the first feature point and the second feature point in the verification pattern are identified during verification. It can be understood that more feature points can be identified during verification, and the world coordinates of multiple feature points can be calculated. The calculated world coordinates of the multiple feature points are compared with the measured world coordinates of these feature points to obtain an accuracy mean, so that the accuracy calculation is more accurate.

[0048] In addition, both the calibration pattern and the verification pattern may include multiple QR code information, such as Figure 1 and Figure 2 The calibration pattern and verification pattern in the embodiment include nine matrix two-dimensional codes, and the ID information of the nine matrix two-dimensional codes can be arranged in sequence to form the serial number information of the calibration plate and the verification plate. Figure 2 The pattern close to the camera can be used as a calibration pattern, and the pattern far away from the camera can be used as a verification pattern.

[0049] It is understandable that although the above-mentioned calibration board and verification board include the word "board" in their names, they can actually be plate-like or sheet-like objects such as displays, wooden boards with patterns, plastic boards, etc., or they can also be flexible materials that can be rolled up when not in use.

[0050] In the description of the present invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0051] Although the present invention is described in conjunction with the above specific embodiments, it is obvious that those skilled in the art can make many substitutions, modifications and changes according to the above content. Therefore, all such substitutions, improvements and changes are included in the spirit and scope of the appended claims.

Claims

1. A camera calibration verification method, characterized in that: Includes steps: S10, capturing an image through a camera, wherein the captured image includes a calibration pattern for camera calibration, identifying feature points of the calibration pattern, and calculating an external parameter of the camera by combining parameters of the calibration pattern, an internal parameter of the camera, and a distortion coefficient; S20, obtaining a verification pattern of a verification plate for camera verification, identifying a first feature point and a second feature point in the verification pattern, and calculating the world coordinates of the first feature point according to the first feature point and the second feature point; S30, comparing the calculated world coordinates of the first feature point with the measured world coordinates to obtain an accuracy value; S40, if the accuracy value meets the requirement, confirm that the calibration is successful and output the external parameter as the final external parameter, otherwise output the evaluation result including the accuracy value; In step S20, the method for calculating the world coordinates of the first feature point includes: S21, performing distortion correction on the image coordinates of the first feature point and the second feature point according to the intrinsic parameters of the camera to obtain corrected image coordinates (xUndis1, yUndis1) and (xUndis2, yUndis2); S22, assuming that the world coordinates of the first feature point are (X1, Y1, Z1), the world coordinates of the second feature point are (X2, Y2, Z2), ΔW C Represents the difference in world coordinates between (X1, Y1, Z1) and (X2, Y2, Z2), vector I1 = (xUndis1, yUndis1, 1) T ,I2=(yUndis2,yUndis2,1) T , the rotation matrix of the external parameter is R, the translation vector of the external parameter is T, the internal parameter matrix is ​​A, and the matrix H = A*R, then the distance S between the verification plate and the optical center of the camera is calculated by formula (1): in x1=(1,0,0)*(H -1 *I1), x2=(1,0,0)*(H -1 *I2); S23, calculate the world coordinates of the first feature point as (X1, Y1, Z1) using formula (2):

2. The camera calibration verification method according to claim 1, characterized in that: When the camera takes an image, a movable calibration plate and verification plate are used; the calibration plate is printed or sprayed or pasted or displayed with the calibration pattern, and a fixing piece for convenient hanging and fixing can be fixed on the other side; the verification plate is printed or sprayed or pasted or displayed with the verification pattern, and a fixing piece for convenient hanging and fixing can be fixed on the other side.

3. The camera calibration verification method according to claim 2, characterized in that: The calibration plate and the verification plate are also printed, sprayed, pasted or displayed with serial number information that can distinguish the calibration plate and the verification plate.

4. The camera calibration verification method according to claim 3, characterized in that: The calibration pattern and verification pattern are two-dimensional codes, and the serial number information includes the two-dimensional code.

5. The camera calibration verification method according to claim 3, characterized in that: In the step S10, the image captured by the camera includes both the calibration pattern and the verification pattern, and the calibration pattern and the verification pattern in the image are determined by the serial number information.

6. The camera calibration verification method according to claim 1, characterized in that: The physical plane where the verification pattern is located is perpendicular to the horizontal axis of the world coordinate system, and the distance S is the horizontal distance between the physical plane where the verification pattern is located and the optical center of the camera.

7. A storage medium, characterized in that: The storage medium includes instructions, and the instructions are used to implement the camera calibration and verification method as described in any one of claims 1 to 6.

8. An electronic device, characterized in that: The electronic device comprises a processor and a storage medium as claimed in claim 7, and the electronic device calls instructions of the storage medium through the processor to implement the camera calibration and verification method as claimed in any one of claims 1 to 6.

Images