Vehicle-mounted fisheye camera internal reference rapid calibration method and system

Abstract

The invention belongs to the technical field of vehicle-mounted camera calibration, and particularly relates to a vehicle-mounted fisheye camera internal reference rapid calibration method and system, and the method comprises the steps: obtaining a factory fixed internal reference, a distortion coefficient, an error range of the internal reference, and an angular point coordinate of a standard calibration cloth of a camera; according to the distortion coefficient, assuming the internal reference of the camera, and carrying out distortion correction on the angular point coordinates of the calibration cloth shot by the camera; calculating a homography matrix between the corrected angular point coordinates and the angular point coordinates of the standard calibration cloth; performing backstepping calculation by using the homography matrix to obtain backstepping calibration cloth corner point coordinates; calculating a standard deviation between the angular point coordinates of the backstepping calibration cloth and the angular point coordinates of the standard calibration cloth; and through polling calculation, the corresponding internal reference value is optimal when the standard deviation is minimum. According to the invention, the calibration cloth is used to carry out primary calibration on the internal reference of the camera, and a more accurate internal reference value is obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle-mounted camera calibration, and in particular relates to a method and system for rapid calibration of internal parameters of a vehicle-mounted fisheye camera. Background Art

[0002] In order to obtain a wider viewing angle, the on-board surround view camera generally uses a fisheye wide-angle lens. However, due to production process and other issues, the position of the optical center of the fisheye camera often shifts during the production process and is generally not at the center of the camera output image. Figure 1 shown.

[0003] Camera manufacturers often provide a fixed camera internal parameter value. If this fixed internal parameter value is used for calibration, the result is often unsatisfactory. Figure 2 As shown, it can be seen that the checkerboard stitching has obvious uneven stitching.

[0004] Since the optical center of the camera has errors when it leaves the factory, if the camera internal parameters are not optimized and corrected, the distortion correction results may have errors. The traditional internal parameters are calculated through Zhang Zhengdingyou's chessboard calibration method, which requires multiple chessboard photos at different positions and angles to calculate the camera internal parameters. However, this method has limitations and cannot be applied to product offline. Summary of the invention

[0005] Purpose of the invention: Due to the production process problems of fisheye cameras, there will be a certain error in the optical center, and this error will also have a certain impact on the subsequent external parameter calibration, and then affect the result of the entire calibration. The present invention provides a method for rapid calibration of the internal parameters of a vehicle-mounted fisheye camera. By using a calibration cloth, the internal parameters of the camera are calibrated once to obtain more accurate internal parameter values.

[0006] Technical solution: The method for rapid calibration of the internal parameters of a vehicle-mounted fisheye camera of the present invention comprises the following steps: S1. Obtain the fixed internal parameters and distortion coefficients of the camera when it leaves the factory, as well as the coordinates of the corner points of the standard calibration cloth arranged; S2. Get the error range of the camera's factory internal parameters and , is the error range of pixel scale, The error range of the coordinates at the sight focus; S3, obtaining the coordinates of the corner points of the calibration cloth by shooting with a camera; S4. Assume that the camera internal parameters are in the following form: , in, is the coordinate of the camera's sight focus in the image coordinate system, is the pixel scale, assuming that the camera internal parameters The initial value is , The initial value of remains unchanged; S5. According to the distortion coefficient, the assumed camera internal parameters, and the coordinates of the calibrated corner points obtained by the camera, the OpenCV library function fisheye::undistortPoints() is used to perform distortion correction to obtain the corrected corner point coordinates. S6, calculating the homography matrix between the corrected corner point coordinates and the standard calibration cloth corner point coordinates; S7, performing reverse calculation based on the corrected corner point coordinates and the homography matrix to obtain reverse calibration corner point coordinates; S8, calculating the standard deviation between the coordinates of the inversely deduced calibration corner points and the coordinates of the standard calibration corner points; S9, repeat S5-S8, and calculate by polling so that the coordinates of the sight focus are Increase to , the process corresponds to the minimum standard deviation , which is the optimal internal parameter ; S10, assuming that the camera internal parameters The initial value is , The initial value remains unchanged, and S5-S8 are repeated. The pixel scale is calculated by polling. Increase to , the process corresponds to the minimum standard deviation , which is the optimal internal parameter ; S11, the S9 What you get with the S10 The combined output is used as the optimal internal parameter.

[0007] To further improve the above technical solution, the calibration cloth consists of four chessboard grids and four rectangular blocks, the four chessboard grids are distributed in front, rear and on both sides of the vehicle, and the four rectangular blocks are distributed at the four corners of the vehicle and aligned with the chessboard grids in the horizontal and vertical directions; the corner points of the calibration cloth include the corner points of the chessboard grids and the four corner points of the rectangular blocks.

[0008] Furthermore, the camera captures and obtains the coordinates of the corner points of the calibration cloth, including: capturing an image of the calibration cloth through a camera, and extracting the coordinates of the corner points of the calibration cloth in the captured image in an image coordinate system (eye_point) using OpenCV.

[0009] Furthermore, the distortion correction is performed using the fisheye::undistortPoints() function provided by OpenCV. The input parameters of the function include: assumed camera intrinsics (intrisic), distortion coefficients (coeffs) and the coordinates of the calibrated corner points (eye_point) obtained by the camera. The function output is the corrected corner point coordinates (out_point_line).

[0010] Furthermore, the findHomography() function in OpenCV is used to match the coordinates of the corrected corner points (out_point_line) with the known coordinates of the corner points of the standard calibration cloth (pts_board) to obtain the homography matrix (m_find_homography_vec) used to describe the perspective transformation between the image coordinate system and the standard calibration cloth coordinate system for subsequent inverse calculations.

[0011] Furthermore, the homography matrix (m_find_homography_vec) is applied to the corrected corner point coordinates (out_point_line), and each point of the corrected corner point coordinates (out_point_line) is linearly combined, and reverse mapping is performed to obtain the corresponding coordinates in the standard calibration cloth coordinate system. The correctness of the reverse mapping result is ensured by normalization operation, and the inverse calibration cloth corner point coordinates are obtained and saved in (homo_get_point).

[0012] Furthermore, the standard deviation between the corner point coordinates of the inverse calibration cloth (homo_get_point) and the corner point coordinates of the standard calibration cloth (pts_board) includes: traversing all corner point pairs between the corner point coordinates of the inverse calibration cloth (homo_get_point) and the corner point coordinates of the standard calibration cloth (pts_board), calculating the square difference between each corner point pair and accumulating them, and evaluating the accuracy of the currently assumed camera intrinsic parameters through the returned error standard deviation.

[0013] The system for implementing the above-mentioned method for rapid calibration of internal parameters of a vehicle-mounted fisheye camera comprises: The camera parameter acquisition module is used to obtain the camera's factory fixed internal parameters, distortion coefficients, and error range of the internal parameters. and ; A corner point coordinate extraction module is used to obtain the calibration cloth image captured by the camera and extract the calibration cloth corner point coordinates in the calibration cloth image; Assume that the camera internal parameter setting module is in the following form: . in, is the camera's sight focus position in the image coordinate system. is the pixel scale, and the setting rule is: Assume that the initial value of the camera internal parameter is , unchanged, the scale is 1, so that the coordinates of the focus of sight are Increase to , camera internal parameters The initial value is , The initial value of remains unchanged, and the scale is 1, so that the pixel scale is Increase to , output the assumed camera internal parameters according to the set rules; The coordinate correction module is used to obtain the assumed camera internal parameters and distortion coefficients, perform distortion correction on the extracted calibration corner point coordinates, and obtain the corrected corner point coordinates; A homography matrix calculation module is used to calculate the homography matrix between the corrected corner point coordinates and the standard calibration corner point coordinates; A corner point calculation module is used to perform reverse calculation on the corrected corner point coordinates according to the homography matrix to obtain the coordinates of the reverse calibration corner points; A standard deviation calculation module is used to calculate the standard deviation between the coordinates of the inverse calibration corner point and the coordinates of the standard calibration corner point; The polling calculation module is used to poll the assumed camera intrinsic parameter value output by the assumed camera intrinsic parameter setting module, and obtain the standard deviation based on the assumed camera intrinsic parameter value using the coordinate correction module, the homography matrix calculation module, the corner point calculation module, and the standard deviation calculation module. When the standard deviation is minimum, the optimal intrinsic parameter value is determined.

[0014] Beneficial effect: Compared with the prior art, the advantages of the present invention are: This article provides a method for calibrating the internal parameters of the camera when the car is off the production line. Based on the factory parameters of the camera, firstly, the coordinates of the calibration cloth are obtained by taking a photo of the calibration cloth chessboard through the camera, and the coordinates after the distortion are obtained by anti-distortion, and then the homography matrix is ​​calculated with the coordinates of the standard calibration cloth. The obtained homography matrix is ​​used to obtain the coordinates of the new calibration cloth by comparing it with the above-obtained coordinates of the distortion. After obtaining the calibration cloth coordinates, it is compared with the standard calibration cloth coordinates to obtain the minimum error, and finally the optimal solution is obtained.

[0015] The main advantages of this method are that the calibration time is short, accurate internal parameters can be obtained quickly, the original calibration site can be directly used, and no other calibration materials need to be added, which saves costs. In addition, the internal parameters of the calibrated camera are more accurate, which plays an important role in the subsequent external parameter calibration, making the image stitching more accurate and less ghosting in the stitching area. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is the effect picture taken by the fisheye camera; Figure 2 This is the effect diagram of calibration using the camera's factory internal parameters; Figure 3 It is the layout diagram of the standard calibration cloth; Figure 4 is a flow chart of the method provided by the present invention; Figure 5 It is an effect diagram of calibration after the internal parameter optimization performed by the present invention. DETAILED DESCRIPTION

[0017] The technical solution of the present invention is described in detail below with reference to the accompanying drawings, but the protection scope of the present invention is not limited to the embodiments.

[0018] Embodiment 1: The present invention is based on the factory parameters of the camera as a benchmark. First, the fisheye coordinates of the calibration cloth are obtained by using OpenCV image recognition, and the coordinates after distortion are obtained by anti-distortion. Then, the homography matrix is ​​calculated with the coordinates of the standard calibration cloth. The obtained homography matrix is ​​used to obtain the coordinates of the new calibration cloth by comparing them with the coordinates of the distortion obtained above. After obtaining the calibration cloth coordinates, they are compared with the standard calibration cloth coordinates to obtain the minimum error and finally obtain the optimal solution. Figure 3 As shown, the calibration cloth consists of four chessboard grids and four rectangular blocks. The four chessboard grids are distributed in front, rear and on both sides of the vehicle, and the four rectangular blocks are distributed at the four corners of the vehicle and are aligned with the chessboard grids in the horizontal and vertical directions. The corner points of the calibration cloth include the corner points of the chessboard grids and the four corner points of the rectangular blocks.

[0019] like Figure 4 The method for rapid calibration of the internal parameters of a vehicle-mounted fisheye camera shown in the figure comprises the following steps: S1. Obtain the fixed internal parameters, distortion coefficients and corner coordinates of the standard calibration cloth of the camera. S2, obtaining the error range of the camera's factory internal parameters; S3, obtaining the coordinates of the corner points of the calibration cloth captured by the camera; S4. Assume that the camera internal parameters are in the following form: , in, The image coordinate system is the focus position of the camera. Assuming that the camera internal parameters are The initial value is taken as the lower limit value according to the error range of the internal parameter in S2. is the pixel scale, assuming that the camera internal parameters The initial value is determined based on the value in the fixed internal parameter; S5. According to the distortion coefficient, the assumed camera internal parameters, and the coordinates of the corner points of the calibration cloth obtained by the camera, the OpenCV library function fisheye::undistortPoints() is used to perform distortion correction to obtain the corrected corner point coordinates. S6, calculating the homography matrix between the corrected corner point coordinates and the corner point coordinates of the standard calibration cloth; S7, performing reverse calculation based on the corrected corner point coordinates and the homography matrix to obtain reverse calibration corner point coordinates; S8, calculating the standard deviation between the coordinates of the corner points of the inverse calibration cloth and the coordinates of the corner points of the standard calibration cloth; S9, repeat S4-S9, through polling calculation, so that the coordinates of the sight focus are Increase to , the process corresponds to the minimum standard deviation , which is the optimal internal parameter ; S10, assuming that the camera internal parameters The initial value is , The initial value remains unchanged, and S5-S8 are repeated. The pixel scale is calculated by polling. Increase to , the process corresponds to the minimum standard deviation , which is the optimal internal parameter ; S11, the S9 What you get with the S10 The combined output is used as the optimal internal parameter.

[0020] like Figure 5 As shown, this is a diagram showing the effect of calibration after internal parameter optimization according to the present invention.

[0021] Embodiment 2: A system for implementing the method for rapid calibration of internal parameters of a vehicle-mounted fisheye camera in Embodiment 1, comprising: The camera parameter acquisition module is used to obtain the camera's factory fixed internal parameters, distortion coefficients, and error range of the internal parameters; A corner point coordinate extraction module is used to obtain the fisheye image taken by the camera and extract the corner point coordinates of the calibration cloth in the fisheye image; The fisheye image coordinate correction module is used to obtain the camera internal parameters and distortion coefficients, perform distortion correction on the extracted calibration corner point coordinates, and obtain the corrected corner point coordinates. It is assumed that the camera internal parameters are in the following form: , in, The image coordinate system is the focus position of the camera. Assuming that the camera internal parameters are The initial value is taken as the lower limit according to the error range of the internal parameter. is the pixel scale, assuming that the camera internal parameters The initial value is determined based on the value in the fixed internal parameter; A homography matrix calculation module is used to calculate the homography matrix between the corrected corner point coordinates and the standard calibration corner point coordinates; A corner point calculation module is used to perform reverse calculation on the corrected corner point coordinates according to the homography matrix to obtain the coordinates of the reverse calibration corner points; A standard deviation calculation module is used to calculate the standard deviation between the coordinates of the inverse calibration corner point and the coordinates of the standard calibration corner point; The polling calculation module is used to adjust the camera internal parameter value according to the standard deviation calculation result to obtain the optimal internal parameter value.

[0022] Embodiment 3: The coordinates of the corner points of the calibration cloth are obtained by photographing with a camera, and distortion correction is performed on the coordinates of the corner points.

[0023] 1. Distortion Correction First, call the OpenCV function fisheye::undistortPoints() to correct the distortion of the corner points on the captured fisheye image. Some of the code is as follows: std::vector <cv::point2f>out_point_line; fisheye::undistortPoints(eye_point, out_point_line, intrisic,coeffs); In the above code: eye_point is the original corner point coordinates obtained by the camera. These points are usually stretched or distorted due to the distortion of the fisheye lens, resulting in inaccurate positions; out_point_line stores the coordinates of the corner points after distortion correction; intrisic and coeffs are the camera's intrinsic parameters (including focal length and optical center) and lens distortion coefficients, respectively. These parameters are used to describe the characteristics of the fisheye lens and help correct distortion.

[0024] Through the fisheye::undistortPoints() function, the original corner point coordinates are mapped to a corrected coordinate system to eliminate the effects of distortion.

[0025] 2. Coordinate scale transformation In the corrected coordinate system, the x and y coordinates will be scaled and transformed based on the camera's intrinsic parameters (normal_fx, normal_fy, normal_cx, normal_cy).

[0026] , normal_fx, normal_fy: Represents the scale factor of the focal length, used to adjust the scale of the coordinates.

[0027] normal_cx, normal_cy: represents the coordinates of the optical center position, used to translate the origin of the coordinate system to the actual optical center position of the camera.

[0028] This transformation adjusts the original distortion-corrected coordinates to coordinates that conform to the standard pixel coordinate system. The purpose of this processing is to be able to compare and verify with the known corner point coordinates of the standard calibration cloth.

[0029] The above results in the obtained coordinate points after distortion correction.

[0030] 3. Calculate the homography matrix Next, the findHomography() function is used to calculate the homography matrix between the two sets of coordinates using the coordinates of the corner points after distortion correction and the corner points of the standard calibration cloth.

[0031] homography_camera = findHomography(out_point_line, pts_board); m_find_homography_vec = homography_camera.reshape(1, 1); findHomography(): This function calculates a homography matrix, which describes the perspective transformation from one coordinate system to another. The two coordinate systems here are the coordinates of the corner points after distortion correction and the coordinates of the standard corner points on the calibration cloth.

[0032] homography_camera: stores the calculated homography matrix.

[0033] m_find_homography_vec: The homography matrix is ​​expanded into a one-dimensional vector for subsequent coordinate transformation.

[0034] The homography matrix reflects the geometric relationship between the camera's distortion correction result and the actual calibration cloth.

[0035] The obtained homography matrix is ​​used to inversely calculate the coordinate points of the calculated calibration cloth based on the obtained distortion correction points.

[0036] 4. Coordinate inversion and calculation Through the calculated homography matrix, the distortion-corrected points are further reverse-mapped to the coordinate system of the standard calibration cloth to verify whether the position of each point is accurate.

[0037] , Reverse transformation: According to the elements in the homography matrix, the corrected corner point coordinates are converted back to the corresponding coordinates on the calibration cloth.

[0038] point_z: It is the normalization factor of the projection transformation to ensure the accuracy of the perspective effect of the coordinates.

[0039] tmp_point.x, tmp_point.y: The coordinate points of the calibration cloth obtained by inverse calculation.

[0040] 5. Calculation of standard deviation and optimization The accuracy of the internal parameters is evaluated by calculating the distance difference between the calibration cloth coordinate points obtained by inversion and the actual coordinates of the standard calibration cloth. The final result is to verify the effect of distortion correction and homography matrix calculation by comparing the sum of square differences (standard deviation) of all points.

[0041] Then, the sum of the square differences between the points obtained above and the coordinate points of the standard calibration cloth is obtained.

[0042] , Square error calculation: The error of each point is obtained by calculating the difference (distance_x and distance_y) between the corner point obtained by inversion and the corner point of the standard calibration cloth.

[0043] Return standard deviation: Returns the sum of squared errors as a metric for evaluating model optimization.

[0044] Finally, we get its standard deviation. Repeat the above steps, compare each parameter of the internal reference, and finally get the optimal solution.

[0045] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes in form and details may be made without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A method for rapid calibration of internal parameters of a vehicle-mounted fisheye camera, characterized in that: The steps include: S1. Obtain the fixed internal parameters and distortion coefficients of the camera when it leaves the factory, as well as the coordinates of the corner points of the standard calibration cloth arranged; S2. Get the error range of the camera's factory internal parameters and , is the error range of pixel scale, The error range of the coordinates at the sight focus; S3, obtaining the coordinates of the corner points of the calibration cloth by shooting with a camera; S4. Assume that the camera internal parameters are in the following form: , in, is the coordinate of the camera's sight focus in the image coordinate system, is the pixel scale; S5. According to the distortion coefficient, the assumed camera internal parameters, and the coordinates of the calibrated corner points obtained by the camera, the OpenCV library function fisheye::undistortPoints() is used to perform distortion correction to obtain the corrected corner point coordinates. S6, calculating the homography matrix between the corrected corner point coordinates and the standard calibration cloth corner point coordinates; S7, performing reverse calculation based on the corrected corner point coordinates and the homography matrix to obtain reverse calibration corner point coordinates; S8, calculating the standard deviation between the coordinates of the inversely deduced calibration corner points and the coordinates of the standard calibration corner points; S9, assuming that the camera internal parameters The initial value is , The initial value remains unchanged, and S5-S8 are repeated. Through polling calculation, the coordinates of the sight focus are changed from Increase to , the process corresponds to the minimum standard deviation , which is the optimal internal parameter ; S10, assuming that the camera internal parameters The initial value is , The initial value remains unchanged, and S5-S8 are repeated. The pixel scale is calculated by polling. Increase to , the process corresponds to the minimum standard deviation , which is the optimal internal parameter ; S11, the S9 What you get with the S10 The combined output is used as the optimal internal parameter.

2. The method for rapid calibration of internal parameters of a vehicle-mounted fisheye camera according to claim 1, characterized in that: The calibration cloth consists of four chessboard grids and four rectangular blocks. The four chessboard grids are distributed in front, rear and on both sides of the vehicle, and the four rectangular blocks are distributed at the four corners of the vehicle and are aligned with the chessboard grids in the horizontal and vertical directions. The corner points of the calibration cloth include the corner points of the chessboard grids and the four corner points of the rectangular blocks.

3. The method for rapid calibration of internal parameters of a vehicle-mounted fisheye camera according to claim 2, characterized in that: The camera captures and obtains the coordinates of the corner points of the calibration cloth, including: capturing an image of the calibration cloth through a camera, and using OpenCV to extract the coordinates (eye_point) of the corner points of the calibration cloth in the captured image in the image coordinate system.

4. The method for rapid calibration of internal parameters of a vehicle-mounted fisheye camera according to claim 3 is characterized in that: The distortion correction is performed using the fisheye::undistortPoints() function provided by OpenCV. The input parameters of the function include: assumed camera intrinsics (intrisic), distortion coefficients (coeffs), and the coordinates of the calibrated corner points (eye_point) obtained by the camera. The function output is the corrected corner point coordinates (out_point_line).

5. The method for rapid calibration of internal parameters of a vehicle-mounted fisheye camera according to claim 4, characterized in that: Use the findHomography() function in OpenCV to match the corrected corner point coordinates (out_point_line) with the known standard calibration cloth corner point coordinates (pts_board) to obtain the homography matrix (m_find_homography_vec) used to describe the perspective transformation between the image coordinate system and the standard calibration cloth coordinate system for subsequent inverse calculations.

6. The method for rapid calibration of internal parameters of a vehicle-mounted fisheye camera according to claim 5, characterized in that: Apply the homography matrix (m_find_homography_vec) to the corrected corner point coordinates (out_point_line), perform linear combination on each point of the corrected corner point coordinates (out_point_line), reverse map to obtain the corresponding coordinates in the standard calibration cloth coordinate system, and ensure the correctness of the reverse mapping result through normalization operation, and obtain the inverse calibration cloth corner point coordinates and save them in (homo_get_point).

7. The method for rapid calibration of internal parameters of a vehicle-mounted fisheye camera according to claim 5, characterized in that: The standard deviation between the corner point coordinates of the inverse calibration cloth (homo_get_point) and the corner point coordinates of the standard calibration cloth (pts_board) includes: traversing all corner point pairs between the corner point coordinates of the inverse calibration cloth (homo_get_point) and the corner point coordinates of the standard calibration cloth (pts_board), calculating the square difference between each corner point pair and accumulating them, and evaluating the accuracy of the currently assumed camera intrinsic parameters through the returned error standard deviation.

8. A system for implementing the method for rapid calibration of internal parameters of a vehicle-mounted fisheye camera according to claim 1, characterized in that: The system comprises: The camera parameter acquisition module is used to obtain the camera's factory fixed internal parameters, distortion coefficients, and error range of the internal parameters. and ; A corner point coordinate extraction module is used to obtain the calibration cloth image captured by the camera and extract the calibration cloth corner point coordinates in the calibration cloth image; Assume that the camera internal parameter setting module is in the following form: , in, is the camera's sight focus position in the image coordinate system. is the pixel scale, and the setting rule is: Assume that the initial value of the camera internal parameter is , unchanged, the scale is 1, so that the coordinates of the focus of sight are Increase to , camera internal parameters The initial value is , The initial value of remains unchanged, and the scale is 1, so that the pixel scale is Increase to , output the assumed camera internal parameters according to the set rules; The coordinate correction module is used to obtain the assumed camera internal parameters and distortion coefficients, perform distortion correction on the extracted calibration corner point coordinates, and obtain the corrected corner point coordinates; A homography matrix calculation module is used to calculate the homography matrix between the corrected corner point coordinates and the standard calibration corner point coordinates; A corner point calculation module is used to perform reverse calculation on the corrected corner point coordinates according to the homography matrix to obtain the coordinates of the reversely calibrated corner points; A standard deviation calculation module is used to calculate the standard deviation between the coordinates of the inverse calibration corner point and the coordinates of the standard calibration corner point; The polling calculation module is used to poll the assumed camera intrinsic parameter value output by the assumed camera intrinsic parameter setting module, and obtain the standard deviation based on the assumed camera intrinsic parameter value using the coordinate correction module, the homography matrix calculation module, the corner point calculation module, and the standard deviation calculation module. When the standard deviation is minimum, the optimal intrinsic parameter value is determined.

Images