Camera calibration method, calibration system and storage medium

Abstract

The invention provides a camera calibration method, a calibration system and a storage medium. The camera calibration method comprises the following steps: adjusting an image acquisition state in real time based on illumination intensity and a reference object to obtain a plurality of reference object images so as to obtain a distortion coefficient of a target camera, and performing distortion removal correction on the reference object images according to the distortion coefficient to obtain an initial calibration parameter; further calculating a first projection error according to the initial calibration parameter, and taking the initial calibration parameter as a target calibration parameter when the first projection error is within a preset error range; and otherwise, obtaining the target calibration parameter based on the cloud calibration data and the initial calibration parameter. According to the method, high-quality reference object images can be obtained under different illumination conditions, the calibration accuracy can be objectively evaluated through the first projection error, cloud calibration data is adopted for optimization, and the accuracy of calibration parameters is improved; meanwhile, the whole process is automatically carried out, and the calibration efficiency and precision are improved.

Description

Technical Field

[0001] The present application relates to the field of camera calibration technology, and in particular to a camera calibration method, a calibration system and a storage medium. Background Art

[0002] Existing vehicles are integrating more and more cameras to achieve assisted driving functions; however, correctly calibrating camera parameters for each specific vehicle is a time-consuming and labor-intensive process; current calibration methods usually require specialized equipment and skilled technicians to manually adjust settings for each vehicle, which is costly and difficult to ensure that each vehicle is optimally configured. Summary of the invention

[0003] In order to solve the above-mentioned technical problems, the present application provides a camera calibration method, a calibration system and a storage medium that can automatically calibrate and produce better calibration results.

[0004] Specifically, the present application provides a camera calibration method, comprising the following steps:

[0005] S100: adjusting the image acquisition state in real time based on the light intensity and the reference object to obtain a plurality of reference object images.

[0006] S200: Obtaining a distortion coefficient of a target camera according to the reference object image, and performing de-distortion correction on the reference object image according to the distortion coefficient to obtain initial calibration parameters.

[0007] And S300: calculating a first reprojection error according to the initial calibration parameters, and if the first reprojection error is within a preset error range, using the initial calibration parameters as target calibration parameters; otherwise obtaining target calibration parameters based on the cloud calibration data and the initial calibration parameters.

[0008] In the above technical solution, by adjusting the image acquisition state in real time based on the light intensity, it can be ensured that high-quality reference object images can be obtained under different lighting conditions, so as to reduce the calibration error caused by lighting changes; more accurate initial calibration parameters can be obtained through de-distortion correction; the accuracy of the calibration can be objectively evaluated by calculating the first re-projection error. If the error is within the preset error range, the accuracy and reliability of the calibration parameters can be ensured. If it is not within the preset error range, the accuracy of the calibration parameters is improved based on external data; among them, by optimizing through cloud calibration data, more samples and higher quality data can be used for adjustment, thereby improving the accuracy of the calibration parameters.

[0009] In addition, the entire process is highly automated. Through dynamic adjustment and real-time calculation, the calibration efficiency and accuracy can be greatly improved and manual intervention can be reduced.

[0010] Furthermore, the real-time adjustment of the image acquisition state in step S100 includes:

[0011] After the reference object is placed at the calibration location, the light intensity of the calibration location and the visibility of the reference object are collected in real time to adjust the image acquisition state based on the light intensity and visibility; wherein the image acquisition state includes at least the image acquisition frame rate and the image exposure time.

[0012] In the above technical solution, the image acquisition frame rate and image exposure time are adjusted based on the real-time acquired light intensity, which can ensure that clear and consistent images can be obtained under different lighting conditions, thereby improving the accuracy of the calibration results; by adjusting the exposure time, overexposure or underexposure can be avoided, and the image contrast and detail clarity can be guaranteed, thereby improving the quality of calibration; through real-time adjustment, the image quality can be optimized in time during the acquisition process, reducing the number of times that recalibration is required due to lighting problems, thereby improving the efficiency of calibration.

[0013] In addition, by automatically adjusting the image acquisition status, the reliance on manual adjustment is reduced and the degree of automation of the calibration process is improved; this real-time adaptive adjustment can improve the robustness of the calibration system and ensure that a high calibration accuracy can be maintained in different environments and conditions.

[0014] Furthermore, the visibility of the reference objects is collected, including:

[0015] Acquire a reference object image to obtain visibility judgment information; wherein the visibility judgment information at least includes the brightness and contrast of the reference object image, and the boundary, position and direction of the reference object in the reference object image.

[0016] Visibility is acquired based on the visibility judgment information.

[0017] In the above technical solution, obtaining the brightness and contrast information of the image is helpful to evaluate the clarity of the reference object in the image. Changes in brightness and contrast directly affect the visibility of the reference object. Therefore, these two parameters can be used to fully understand the visibility of the reference object; determining the boundary, position and direction of the reference object can help determine its actual performance in the image. The clarity of the boundary and the centrality of the position will affect the visibility, while the correctness of the direction ensures that the reference object is not blocked or distorted in the field of view.

[0018] The above, taking into account brightness, contrast, boundaries, position and direction, can provide a more accurate visibility assessment. By automatically analyzing these visibility judgment information, image acquisition parameters such as exposure time and frame rate can be automatically adjusted to ensure high-quality images under various conditions, further improving the accuracy of calibration.

[0019] Furthermore, the step S200 of obtaining the distortion coefficient of the target camera includes:

[0020] Corresponding feature corner points are acquired based on each reference object image, so as to calculate a homography matrix based on the feature corner points.

[0021] The internal parameters and external parameters of the target camera are obtained by combining the size of the reference object and the homography matrix; wherein the internal parameters include at least focal length, principal point coordinates and distortion coefficient, and the external parameters include at least rotation matrix and translation vector.

[0022] In the above technical solution, the precise extraction of feature corner points and the calculation of the homography matrix can accurately describe the geometric deformation of the image, and then calculate the distortion coefficient, thereby achieving high-precision distortion correction; and by accurately calculating the homography matrix and combining it with size information, the errors caused by measurement errors and inaccurate model assumptions can be reduced, thereby improving the accuracy of the final calibration results.

[0023] Furthermore, the de-distortion correction in step S200 includes:

[0024] The pixel coordinates in the reference object image are converted to a normalized coordinate system based on the focal length, the principal point coordinates and the distortion coefficient to obtain normalized coordinates; wherein the pixel coordinates are in the image coordinate system.

[0025] A first calibration parameter is obtained based on the distortion coefficient and the normalized coordinate, and the first calibration parameter is converted into the image coordinate system to obtain a second calibration parameter.

[0026] The second calibration parameters are interpolated to obtain initial calibration parameters.

[0027] In the above technical solution, converting the pixel coordinates to a normalized coordinate system helps to remove the geometric deformation caused by camera distortion, making the calculation more accurate; by converting these parameters back to the image coordinate system, more accurate calibration parameters can be obtained to improve the geometric accuracy of the final image; interpolating the second calibration parameters can fill in the gaps or deficiencies that may exist in the calculation process, further improving the calibration accuracy.

[0028] Further, the calculating of the first reprojection error in step S300 includes:

[0029] The three-dimensional coordinates and the two-dimensional coordinates corresponding to the reference object image are obtained based on the initial calibration parameters, an intrinsic parameter matrix is ​​constructed based on the focal length and the principal point coordinates, and an extrinsic parameter matrix is ​​constructed based on the rotation matrix and the translation vector.

[0030] The distortion coordinates are obtained according to the distortion coefficients and the two-dimensional coordinates, and the reprojection points are obtained according to the three-dimensional coordinates, the intrinsic parameter matrix and the extrinsic parameter matrix.

[0031] Point-by-point reprojection errors are calculated according to the three-dimensional coordinates and the reprojection points, so as to calculate a first reprojection error based on the point-by-point reprojection errors.

[0032] In the above technical solution, by obtaining the three-dimensional and two-dimensional coordinates of the reference object image based on the initial calibration parameters, the accuracy of the camera model can be evaluated in detail; by constructing the intrinsic parameter matrix and the extrinsic parameter matrix, the optical characteristics and spatial position of the camera can be described in detail, thereby providing the necessary information for the subsequent reprojection error calculation. This helps to improve the accuracy of the error calculation; by obtaining the distortion coordinates based on the distortion coefficient and the two-dimensional coordinates, the image distortion can be accurately corrected, thereby improving the accuracy of the reprojection points; calculating the point-by-point reprojection error can refine the error analysis and understand the error situation of each reference point, thereby optimizing the calibration parameters more accurately; by understanding the errors in the calibration process, measures can be taken to reduce the impact of the errors.

[0033] Furthermore, the step S300 of acquiring target calibration parameters based on the cloud calibration data and the initial calibration parameters includes:

[0034] S301: Obtain cloud calibration data, and adjust the initial calibration parameters based on the cloud calibration data to obtain third calibration parameters.

[0035] S302: Optimize the third calibration parameter using a preset optimization algorithm to obtain a fourth calibration parameter, and calculate a second reprojection error based on the target calibration parameter.

[0036] S303: Determine whether the second re-projection error is within a preset error range, and if so, use the fourth calibration parameter as the target calibration parameter; otherwise, return to step S301, and use the fourth calibration parameter as the initial calibration parameter for iterative optimization.

[0037] In the above technical solution, by obtaining cloud calibration data, calibration can be performed in combination with a large amount of existing data to improve the accuracy of calibration parameters. These data usually come from calibration under a variety of different conditions, which can provide a wide range of references and improve calibration results; by adjusting the initial calibration parameters based on cloud data, the errors caused by the initial parameters can be reduced; by further optimizing the adjusted parameters using a preset optimization algorithm, the calibration results can be refined and the accuracy of the calibration parameters can be improved; calculating the second projection error based on the optimized fourth calibration parameter helps to evaluate the accuracy of the optimization result; if the second projection error is not within the preset error range, the system will automatically return for further adjustment and optimization. This automated iterative process can continuously improve the calibration accuracy until the preset standard is reached.

[0038] Furthermore, after obtaining the target calibration parameters, it also includes:

[0039] The captured image of the target camera is obtained based on the target calibration parameters, and the captured image is displayed through the vehicle computer interface so that the user can provide feedback on the calibration based on the captured image.

[0040] The camera calibration method is optimized according to the calibration opinion.

[0041] In the above technical solution, by displaying the captured images on the vehicle interface and collecting the user's calibration opinions, feedback from actual use can be directly obtained. The user's actual experience and opinions can help discover problems that may be omitted or not considered in the calibration, thereby further optimizing the calibration method.

[0042] Based on the same concept, the present application also provides a camera calibration system, including:

[0043] The acquisition module is used to adjust the image acquisition state in real time based on the light intensity and the reference object to obtain multiple reference object images.

[0044] The correction module is used to obtain the distortion coefficient of the target camera according to the reference object image, and perform de-distortion correction on the reference object image according to the distortion coefficient to obtain initial calibration parameters.

[0045] and an optimization module, for calculating a first reprojection error according to the initial calibration parameters, and if the first reprojection error is within a preset error range, using the initial calibration parameters as target calibration parameters; otherwise, obtaining target calibration parameters based on cloud calibration data and the initial calibration parameters.

[0046] In the above technical solution, the image acquisition state is automatically adjusted to adapt to different lighting conditions and reference objects, so as to obtain high-quality reference object images. This automated image acquisition reduces manual intervention and improves acquisition efficiency and consistency. By obtaining the distortion coefficient of the reference object image and performing de-distortion correction, the initial calibration parameters can be accurately obtained. This precise initial calibration provides a reliable basis for subsequent optimization. The first re-projection error is calculated according to the initial calibration parameters, and the preliminary results of the calibration can be accurately evaluated. If the error is within the preset error range, the initial calibration parameters are accurate enough, avoiding unnecessary further optimization. If the first re-projection error is not within the preset error range, the system will use the cloud calibration data and the initial calibration parameters for further optimization. This method combines big data and diversified calibration data, which helps to improve the accuracy and adaptability of calibration.

[0047] In addition, through automated collection, correction and optimization processes, the user's operation complexity is reduced and the overall calibration experience is improved.

[0048] Based on the same concept, the present application also provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the camera calibration method when running.

[0049] Compared with the prior art, the beneficial effects of this application are:

[0050] The present application first adjusts the image acquisition state in real time based on the light intensity and the reference object to obtain multiple reference object images; then obtains the distortion coefficient of the target camera according to the reference object image, and de-distorts the reference object image according to the distortion coefficient to obtain initial calibration parameters; further calculates the first reprojection error according to the initial calibration parameters, and when the first reprojection error is within a preset error range, uses the initial calibration parameters as the target calibration parameters; otherwise, obtains the target calibration parameters based on the cloud calibration data and the initial calibration parameters.

[0051] This application can ensure that high-quality reference images can be obtained under different lighting conditions, and the calibration accuracy can be objectively evaluated through the first projection error. It also uses cloud-based calibration data for optimization to improve the accuracy of the calibration parameters. At the same time, the entire process is automated, which improves the calibration efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flowchart of the camera calibration method described in Example 1.

[0053] Figure 2 This is a framework diagram of the camera calibration system described in Example 3. DETAILED DESCRIPTION

[0054] The following is a further detailed description of a camera calibration method, calibration system and storage medium of the present application in conjunction with specific embodiments and drawings.

[0055] Embodiment 1:

[0056] See also Figure 1 , the present application provides a camera calibration method, comprising the following steps:

[0057] S100: adjusting the image acquisition state in real time based on the light intensity and the reference object to obtain a plurality of reference object images.

[0058] In this embodiment, when the target vehicle is parked or undergoing regular maintenance, the vehicle interface will prompt the user to place a calibration target. This calibration target, i.e., a reference object, can be temporarily placed within the field of view of the target camera; wherein the reference object, such as a checkerboard pattern, is composed of black and white or colorful square grids.

[0059] Furthermore, the real-time adjustment of the image acquisition state in step S100 includes:

[0060] After the reference object is placed at the calibration location, the light intensity of the calibration location and the visibility of the reference object are collected in real time to adjust the image acquisition state based on the light intensity and visibility; wherein the image acquisition state includes at least the image acquisition frame rate and the image exposure time.

[0061] In this embodiment, a plurality of images containing reference objects are captured by a camera as reference object images; wherein the reference objects are required to appear at different positions and angles to cover the entire field of view.

[0062] In the above technical solution, the image acquisition frame rate and image exposure time are adjusted based on the real-time acquired light intensity, which can ensure that clear and consistent images can be obtained under different lighting conditions, thereby improving the accuracy of the calibration results; by adjusting the exposure time, overexposure or underexposure can be avoided, and the image contrast and detail clarity can be guaranteed, thereby improving the quality of calibration; through real-time adjustment, the image quality can be optimized in time during the acquisition process, reducing the number of times that recalibration is required due to lighting problems, thereby improving the efficiency of calibration.

[0063] In addition, by automatically adjusting the image acquisition status, the reliance on manual adjustment is reduced and the degree of automation of the calibration process is improved; this real-time adaptive adjustment can improve the robustness of the calibration system and ensure that a high calibration accuracy can be maintained in different environments and conditions.

[0064] Furthermore, the visibility of the reference objects is collected, including:

[0065] Acquire a reference object image to obtain visibility judgment information; wherein the visibility judgment information at least includes the brightness and contrast of the reference object image, and the boundary, position and direction of the reference object in the reference object image.

[0066] Visibility is acquired based on the visibility judgment information.

[0067] In this embodiment, the frame rate and exposure time of image acquisition are dynamically adjusted according to the ambient light intensity and the visibility of the reference object. Specifically:

[0068] Image analysis: By analyzing the brightness and contrast in the reference object image, it is determined whether the reference object is clearly visible. For example, when the brightness and contrast are both greater than their corresponding thresholds, it is determined that the current reference object is clear and there is no need to move it.

[0069] Edge detection: Use edge detection algorithms (such as Canny edge detection) to identify the boundaries of the reference object to determine its visibility; based on the results of edge detection, the boundary information of the reference object is extracted, and whether the boundary is clear and complete is analyzed to determine whether the reference object is clearly visible.

[0070] Feature matching: Use feature point matching technology to determine the position and orientation of the reference object.

[0071] Based on the results of feature point matching, analyze whether the position and direction of the reference object in the image are as expected; if the position or direction of the reference object is incorrect, prompt the user to adjust the shooting angle or the position of the reference object through the vehicle interface.

[0072] The feature point matching technology is, for example, SIFT (Scale-Invariant Feature Transform) or SURF (Speeded-Up Robust Features).

[0073] In addition, depth information such as distance may be obtained to further assist in determining the visibility of the reference object.

[0074] Among them, the distance information of the reference object is obtained through devices such as depth cameras or laser rangefinders, and combined with the depth information, it is determined whether the actual distance of the reference object is within the expected range. Too far or too close a distance may cause image blur or undetectable feature points.

[0075] Furthermore, it also includes:

[0076] A light-sensitive sensor or other sensor is used to measure the ambient light intensity, and the exposure time and gain of the camera are adjusted according to the light intensity to adapt to shooting under different light conditions.

[0077] When the lighting conditions are good and the reference object is highly visible, a higher frame rate can be used to capture more motion details; when the lighting conditions are poor, the frame rate is reduced to reduce noise and allow a longer exposure time to increase image brightness.

[0078] For example: if the light level is below a certain threshold (e.g. 10 lux), you may need to increase the exposure time to improve image brightness while reducing the frame rate to avoid overexposure and motion blur; for example, you can increase the exposure time from 1 / 60 second to 1 / 30 second and reduce the frame rate from 60fps to 30fps.

[0079] If the light level is high (e.g., above 1000 lux), the exposure time can be reduced to avoid overexposure, and the frame rate can be increased to capture more details; for example, the exposure time can be reduced from 1 / 60 second to 1 / 120 second, and the frame rate can be increased from 30fps to 60fps.

[0080] Under medium light intensity, a balanced exposure time and frame rate can be maintained according to actual needs, such as 1 / 60 second and 30fps.

[0081] In the above technical solution, obtaining the brightness and contrast information of the image is helpful to evaluate the clarity of the reference object in the image. Changes in brightness and contrast directly affect the visibility of the reference object. Therefore, these two parameters can be used to fully understand the visibility of the reference object; determining the boundary, position and direction of the reference object can help determine its actual performance in the image. The clarity of the boundary and the centrality of the position will affect the visibility, while the correctness of the direction ensures that the reference object is not blocked or distorted in the field of view.

[0082] The above, taking into account brightness, contrast, boundaries, position and direction, can provide a more accurate visibility assessment. By automatically analyzing these visibility judgment information, image acquisition parameters such as exposure time and frame rate can be automatically adjusted to ensure high-quality images under various conditions, further improving the accuracy of calibration.

[0083] After a plurality of reference object images are acquired, step S200 may be executed.

[0084] S200: Obtaining a distortion coefficient of a target camera according to the reference object image, and performing de-distortion correction on the reference object image according to the distortion coefficient to obtain initial calibration parameters.

[0085] Furthermore, the step S200 of obtaining the distortion coefficient of the target camera includes:

[0086] Corresponding feature corner points are acquired based on each reference object image, so as to calculate a homography matrix based on the feature corner points.

[0087] The internal parameters and external parameters of the target camera are obtained by combining the size of the reference object and the homography matrix; wherein the internal parameters include at least focal length, principal point coordinates and distortion coefficient, and the external parameters include at least rotation matrix and translation vector.

[0088] In this embodiment, the characteristic corner points and the corner points of the chessboard are detected in each reference object image; for each image, the homography matrix H is calculated using the detected characteristic corner points, which represents the mapping relationship from the reference object plane to the image plane.

[0089] Furthermore, the homography matrices of multiple reference object images are used, combined with the known size of the reference object, to solve the intrinsic parameters (K) and extrinsic parameters (R, T) by nonlinear optimization methods (such as Levenberg-Marquardt algorithm, i.e., nonlinear least squares method), where K includes the focal length f, the principal point coordinates c x 、c y and distortion coefficients, R is the rotation matrix, and T is the translation vector.

[0090] In the above technical solution, the precise extraction of feature corner points and the calculation of the homography matrix can accurately describe the geometric deformation of the image, and then calculate the distortion coefficient, thereby achieving high-precision distortion correction; and by accurately calculating the homography matrix and combining it with size information, the errors caused by measurement errors and inaccurate model assumptions can be reduced, thereby improving the accuracy of the final calibration results.

[0091] Furthermore, the de-distortion correction in step S200 includes:

[0092] The pixel coordinates in the reference object image are converted to a normalized coordinate system based on the focal length, the principal point coordinates and the distortion coefficient to obtain normalized coordinates; wherein the pixel coordinates are in the image coordinate system.

[0093] In this embodiment, the main purpose of the de-distortion correction is to eliminate or reduce the geometric distortion generated by the camera when capturing an image, so as to improve the image quality. The distortion generally includes radial distortion (barrel distortion and pincushion distortion), tangential distortion, etc.

[0094] The internal parameters of the camera, including focal length, principal point coordinates and distortion coefficient, are calculated through Zhang Zhengyou calibration method and other methods.

[0095] The pixel coordinates (x, y) in the reference object image are converted to normalized coordinates (u, v) in the normalized coordinate system, that is, the effects of focal length and principal point coordinates are removed.

[0096] Where u = xc x fv=yc y fu,v=fx-c x =fy-c y; f is the focal length, (c x ,c y ) are the principal point coordinates.

[0097] A first calibration parameter is obtained based on the distortion coefficient and the normalized coordinate, and the first calibration parameter is converted into the image coordinate system to obtain a second calibration parameter.

[0098] In this embodiment, the obtained distortion coefficient is used to perform distortion correction on the normalized coordinates, specifically:

[0099] r=u 2 +v 2

[0100] u d =u(1+k1r 2 +k2r 4 +…)+2p1uv+p2(r 2 +2u 2 )

[0101] v d =v(1+k1r 2 +k2r 4 +…)+2p2uv+p1(r 2 +2v 2 )

[0102] Among them, u d and v d is the normalized coordinate after correction, r is the distance from the normalized coordinate to the origin, k1, k2 are the radial distortion coefficients, and p1, p2 are the tangential distortion coefficients.

[0103] Furthermore, the normalized coordinates after distortion correction are converted back to the image coordinate system. Specifically:

[0104] x d =fu d +c x

[0105] y d =fv d +c y

[0106] The second calibration parameters are interpolated to obtain initial calibration parameters.

[0107] In this embodiment, since the coordinates after distortion correction may not be integer pixel positions, it is necessary to use methods such as bilinear interpolation or nearest neighbor interpolation to map the corrected coordinates to the original image to generate an undistorted image.

[0108] In the above technical solution, converting the pixel coordinates to a normalized coordinate system helps to remove the geometric deformation caused by camera distortion, making the calculation more accurate; by converting these parameters back to the image coordinate system, more accurate calibration parameters can be obtained to improve the geometric accuracy of the final image; interpolating the second calibration parameters can fill in the gaps or deficiencies that may exist in the calculation process, further improving the calibration accuracy.

[0109] After the initial calibration parameters are obtained, step S300 may be executed.

[0110] S300: Calculate a first reprojection error according to the initial calibration parameters. If the first reprojection error is within a preset error range, use the initial calibration parameters as target calibration parameters; otherwise, obtain target calibration parameters based on cloud calibration data and the initial calibration parameters.

[0111] Further, the calculating of the first reprojection error in step S300 includes:

[0112] The three-dimensional coordinates and the two-dimensional coordinates corresponding to the reference object image are obtained based on the initial calibration parameters, an intrinsic parameter matrix is ​​constructed based on the focal length and the principal point coordinates, and an extrinsic parameter matrix is ​​constructed based on the rotation matrix and the translation vector.

[0113] In this embodiment, the three-dimensional coordinates X of the reference object image in the world coordinate system are obtained. i (i=1,2,…,n) and their two-dimensional coordinates x on the image i .

[0114] Constructing the internal parameter matrix And the external parameter matrix [R|t]; where R is the rotation matrix and t is the translation matrix.

[0115] The distortion coordinates are obtained according to the distortion coefficients and the two-dimensional coordinates, and the reprojection points are obtained according to the three-dimensional coordinates, the intrinsic parameter matrix and the extrinsic parameter matrix.

[0116] In this embodiment, the preset distortion model is used to calculate the distortion coordinates (x distorted ,y distorted ).

[0117]

[0118] x distorted =x(1+k1r 2 +k2r 4 )

[0119] y distorted =y(1+k1r 2 +k2r 4 )

[0120] Among them, k1 and k2 are radial distortion coefficients respectively.

[0121] Furthermore, for each chessboard corner point X i , use the internal parameter matrix K and the external parameter matrix [R|t] to reproject and get the reprojected point x distorted,i .

[0122] x distorted,i =K·[R|t]·X i

[0123] Since the result of the reprojection is a homogeneous coordinate, it needs to be converted into a two-dimensional coordinate, specifically:

[0124]

[0125] Point-by-point reprojection errors are calculated according to the three-dimensional coordinates and the reprojection points, so as to calculate a first reprojection error based on the point-by-point reprojection errors.

[0126] In this embodiment, the point-by-point reprojection error is calculated according to the three-dimensional coordinates and the reprojection point, specifically:

[0127] e i =||X i -x distorted,i ||

[0128] The first reprojection error can be obtained by calculating the mean or root mean square error of all point reprojection errors, such as:

[0129]

[0130] In the above technical solution, by obtaining the three-dimensional and two-dimensional coordinates of the reference object image based on the initial calibration parameters, the accuracy of the camera model can be evaluated in detail; by constructing the intrinsic parameter matrix and the extrinsic parameter matrix, the optical characteristics and spatial position of the camera can be described in detail, thereby providing the necessary information for the subsequent reprojection error calculation. This helps to improve the accuracy of the error calculation; by obtaining the distortion coordinates based on the distortion coefficient and the two-dimensional coordinates, the image distortion can be accurately corrected, thereby improving the accuracy of the reprojection points; calculating the point-by-point reprojection error can refine the error analysis and understand the error situation of each reference point, thereby optimizing the calibration parameters more accurately; by understanding the errors in the calibration process, measures can be taken to reduce the impact of the errors.

[0131] Further, if the first reprojection error is within a preset error range, the calibration is considered successful, and the initial calibration parameters are used as target calibration parameters; otherwise, the target calibration parameters are obtained based on the cloud calibration data and the initial calibration parameters.

[0132] Furthermore, the step S300 of acquiring target calibration parameters based on the cloud calibration data and the initial calibration parameters includes:

[0133] S301: Obtain cloud calibration data, and adjust the initial calibration parameters based on the cloud calibration data to obtain third calibration parameters.

[0134] In this embodiment, algorithm parameters that are continuously optimized based on global vehicle calibration data (i.e., the cloud calibration data) are obtained from a cloud calibration platform. These parameters may include but are not limited to preset calibration templates (such as calibrated standard patterns or reference object images), optimized algorithm configurations, etc.

[0135] A calibration adjustment model is constructed using the calibration template and algorithm configuration in the cloud data, and the initial calibration parameters obtained from the previous step are loaded and input into the calibration adjustment model to generate new calibration parameters, namely the third calibration parameters.

[0136] S302: Optimize the third calibration parameter using a preset optimization algorithm to obtain a fourth calibration parameter, and calculate a second reprojection error based on the target calibration parameter.

[0137] In this embodiment, the preset optimization algorithm is, for example, the Levenberg-Marquardt algorithm, for performing nonlinear least squares optimization to obtain the fourth calibration parameter.

[0138] Then, the second reprojection error is calculated using the same method as that for calculating the first reprojection error, which will not be described in detail here.

[0139] S303: Determine whether the second re-projection error is within a preset error range, and if so, use the fourth calibration parameter as a target calibration parameter; otherwise, return to step S301, and use the fourth calibration parameter as an initial calibration parameter for iterative optimization.

[0140] In the above technical solution, by obtaining cloud calibration data, calibration can be performed in combination with a large amount of existing data to improve the accuracy of calibration parameters. These data usually come from calibration under a variety of different conditions, which can provide a wide range of references and improve calibration results; by adjusting the initial calibration parameters based on cloud data, the errors caused by the initial parameters can be reduced; by further optimizing the adjusted parameters using a preset optimization algorithm, the calibration results can be refined and the accuracy of the calibration parameters can be improved; calculating the second projection error based on the optimized fourth calibration parameter helps to evaluate the accuracy of the optimization result; if the second projection error is not within the preset error range, the system will automatically return for further adjustment and optimization. This automated iterative process can continuously improve the calibration accuracy until the preset standard is reached.

[0141] Embodiment 2:

[0142] The present application also provides a method for optimizing a camera calibration method.

[0143] That is, after obtaining the target calibration parameters, it also includes:

[0144] The captured image of the target camera is obtained based on the target calibration parameters, and the captured image is displayed through the vehicle computer interface so that the user can provide feedback on the calibration based on the captured image.

[0145] In this embodiment, the user can mark the problem area or calibration error in the captured image and make comments or suggestions on the accuracy of the calibration, such as image distortion, distortion and other issues.

[0146] The camera calibration system collects user feedback data, including the specific location of the calibration error and the user's opinions, analyzes the user feedback data, finds out the problems in the calibration process, and optimizes the camera calibration method based on the feedback results.

[0147] The camera calibration method is optimized according to the calibration opinion.

[0148] In the above technical solution, by displaying the captured images on the vehicle interface and collecting the user's calibration opinions, feedback from actual use can be directly obtained. The user's actual experience and opinions can help discover problems that may be omitted or not considered in the calibration, thereby further optimizing the calibration method.

[0149] Embodiment three:

[0150] See also Figure 2 , the present application also provides a camera calibration system, comprising:

[0151] The acquisition module is used to adjust the image acquisition state in real time based on the light intensity and the reference object to obtain multiple reference object images.

[0152] The correction module is used to obtain the distortion coefficient of the target camera according to the reference object image, and perform de-distortion correction on the reference object image according to the distortion coefficient to obtain initial calibration parameters.

[0153] and an optimization module, for calculating a first reprojection error according to the initial calibration parameters, and if the first reprojection error is within a preset error range, using the initial calibration parameters as target calibration parameters; otherwise, obtaining target calibration parameters based on cloud calibration data and the initial calibration parameters.

[0154] In the above technical solution, the image acquisition state is automatically adjusted to adapt to different lighting conditions and reference objects, so as to obtain high-quality reference object images. This automated image acquisition reduces manual intervention and improves acquisition efficiency and consistency. By obtaining the distortion coefficient of the reference object image and performing de-distortion correction, the initial calibration parameters can be accurately obtained. This precise initial calibration provides a reliable basis for subsequent optimization. The first re-projection error is calculated according to the initial calibration parameters, and the preliminary results of the calibration can be accurately evaluated. If the error is within the preset error range, the initial calibration parameters are accurate enough, avoiding unnecessary further optimization. If the first re-projection error is not within the preset error range, the system will use the cloud calibration data and the initial calibration parameters for further optimization. This method combines big data and diversified calibration data, which helps to improve the accuracy and adaptability of calibration.

[0155] In addition, through automated collection, correction and optimization processes, the user's operation complexity is reduced and the overall calibration experience is improved.

[0156] Embodiment 4:

[0157] The present application also provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the camera calibration method when running.

[0158] In this embodiment, the storage medium stores several computer programs for enabling a device to execute all or part of the steps of the method described in each embodiment of the present application.

[0159] The medium may include a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and other media that can store program codes.

[0160] In summary, the present application provides a camera calibration method, a calibration system and a storage medium; first, the image acquisition state is adjusted in real time based on the light intensity and the reference object to obtain multiple reference object images; then, the distortion coefficient of the target camera is obtained according to the reference object image, and the reference object image is dedistorted and corrected according to the distortion coefficient to obtain the initial calibration parameters; further, the first reprojection error is calculated according to the initial calibration parameters, and when the first reprojection error is within the preset error range, the initial calibration parameters are used as the target calibration parameters; otherwise, the target calibration parameters are obtained based on the cloud calibration data and the initial calibration parameters. The present application can ensure that high-quality reference object images can be obtained under different lighting conditions, and the accuracy of the calibration can be objectively evaluated through the first reprojection error, and the cloud calibration data is used for optimization to improve the accuracy of the calibration parameters; at the same time, the entire process is automated, which improves the calibration efficiency and accuracy.

[0161] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application to this. Those of ordinary skill in the art may make various changes and modifications therein without departing from the scope and spirit of the present application. All these changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0162] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0163] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0164] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some modules according to the embodiments of the present application. The application can also be implemented as a device program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0165] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0166] Although the description of the present application is carried out in conjunction with the above specific embodiments, it is obvious that those skilled in the art can make many substitutions, modifications and changes based on the above content. Therefore, all such substitutions, improvements and changes are included in the spirit and scope of the appended claims.

Claims

1. A camera calibration method, characterized in that: The following steps are involved: S100: adjusting the image acquisition state in real time based on the light intensity and the reference object to obtain multiple reference object images; S200: Acquire a distortion coefficient of a target camera according to the reference object image, and perform dedistortion correction on the reference object image according to the distortion coefficient to acquire an initial calibration parameter; as well as S300: Calculating a first reprojection error according to the initial calibration parameter, and if the first reprojection error is within a preset error range, using the initial calibration parameter as a target calibration parameter; Otherwise, the target calibration parameters are obtained based on the cloud calibration data and the initial calibration parameters.

2. The camera calibration method according to claim 1, characterized in that: The real-time adjustment of the image acquisition state in step S100 includes: After the reference object is placed at the calibration location, the light intensity of the calibration location and the visibility of the reference object are collected in real time to adjust the image acquisition state based on the light intensity and visibility; wherein the image acquisition state includes at least the image acquisition frame rate and the image exposure time.

3. The camera calibration method according to claim 2, characterized in that: Collect visibility of reference objects, including: Acquire a reference object image to obtain visibility judgment information; wherein the visibility judgment information at least includes brightness and contrast of the reference object image, and a boundary, position and direction of the reference object in the reference object image; Visibility is acquired based on the visibility judgment information.

4. The camera calibration method according to claim 1, characterized in that: The step S200 of obtaining the distortion coefficient of the target camera includes: Acquire corresponding feature corner points based on each reference object image, so as to calculate a homography matrix based on the feature corner points; The internal parameters and external parameters of the target camera are obtained by combining the size of the reference object and the homography matrix; wherein the internal parameters include at least focal length, principal point coordinates and distortion coefficient, and the external parameters include at least rotation matrix and translation vector.

5. The camera calibration method according to claim 4, characterized in that: The de-distortion correction in step S200 includes: Converting the pixel coordinates in the reference object image to a normalized coordinate system based on the focal length, the principal point coordinates and the distortion coefficient to obtain normalized coordinates; wherein the pixel coordinates are in the image coordinate system; Acquire a first calibration parameter based on the distortion coefficient and the normalized coordinate, and convert the first calibration parameter into the image coordinate system to acquire a second calibration parameter; The second calibration parameters are interpolated to obtain initial calibration parameters.

6. The camera calibration method according to claim 5, characterized in that: The calculating of the first reprojection error in step S300 includes: Acquire the three-dimensional coordinates and the two-dimensional coordinates corresponding to the reference object image based on the initial calibration parameters, construct an intrinsic parameter matrix based on the focal length and the principal point coordinates, and construct an extrinsic parameter matrix based on the rotation matrix and the translation vector; Obtaining distortion coordinates according to the distortion coefficients and the two-dimensional coordinates, and obtaining reprojection points according to the three-dimensional coordinates, the intrinsic parameter matrix and the extrinsic parameter matrix; Point-by-point reprojection errors are calculated according to the three-dimensional coordinates and the reprojection points, so as to calculate a first reprojection error based on the point-by-point reprojection errors.

7. The camera calibration method according to claim 6, characterized in that: The step S300 of acquiring target calibration parameters based on the cloud calibration data and the initial calibration parameters includes: S301: Obtain cloud calibration data, and adjust the initial calibration parameters based on the cloud calibration data to obtain third calibration parameters; S302: optimizing the third calibration parameter using a preset optimization algorithm to obtain a fourth calibration parameter, and calculating a second reprojection error based on the target calibration parameter; S303: Determine whether the second re-projection error is within a preset error range, and if so, use the fourth calibration parameter as a target calibration parameter; otherwise, return to step S301, and use the fourth calibration parameter as an initial calibration parameter for iterative optimization.

8. The camera calibration method according to claim 1, characterized in that: After the target calibration parameters are obtained, the method further includes: obtaining a captured image of the target camera based on the target calibration parameters, and displaying the captured image through a vehicle computer interface so that the user can provide feedback on the calibration opinions based on the captured image; The camera calibration method as described in any one of claims 1 to 7 is optimized according to the calibration opinion.

9. A calibration system using the camera calibration method according to any one of claims 1 to 8, characterized in that: include: An acquisition module, used for adjusting the image acquisition state in real time based on the light intensity and the reference object to obtain multiple reference object images; A correction module, used to obtain a distortion coefficient of a target camera according to the reference object image, and to perform dedistortion correction on the reference object image according to the distortion coefficient to obtain an initial calibration parameter; as well as an optimization module, configured to calculate a first reprojection error according to the initial calibration parameters, and if the first reprojection error is within a preset error range, use the initial calibration parameters as target calibration parameters; Otherwise, the target calibration parameters are obtained based on the cloud calibration data and the initial calibration parameters.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the camera calibration method according to any one of claims 1 to 8 when running.

Images