Vehicle surround view camera calibration method, system and device and computer medium

Abstract

The invention discloses a calibration method, system and device for a vehicle around-view camera and a computer medium, and relates to the technical field of image processing, and the method comprises the steps: collecting a calibration plate image obtained after a calibration plate is shot by the around-view camera on a vehicle; projecting the calibration plate image to a world coordinate system to obtain a distorted image; selecting an initial angular point in the distorted image according to the calibration plate; projecting the initial angular point to a fisheye image coordinate system of a look-around camera to obtain a target angular point; according to the target angular point, generating an initial external parameter of the look-around camera; in the calibration plate image, extracting overlapped corner point parameters in the overlapped shooting area; and based on the initial external parameter, generating a target external parameter which enables the error between the overlapping angular point parameters to be minimum, and taking the target external parameter as the external parameter of the surround-view camera. According to the method, the extraction precision and robustness of the angular points are improved, the continuity and smoothness of the images shot by the surround view camera are improved, splicing gaps and dislocation phenomena are reduced, and the calibration result of the vehicle surround view camera is more accurate.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and more specifically, to a vehicle surround view camera calibration method, system, device and computer medium. Background Art

[0002] With the development of autonomous driving technology and Advanced Driving Assistance Systems (ADAS), surround view monitoring (AVM) systems have become widely used in smart cars as a means of providing a panoramic image of the vehicle's surroundings. AVM systems typically consist of fisheye cameras installed on the front, rear, left, and right sides of the vehicle. These cameras capture images and stitch them together to create a bird's-eye view (BEV) of the vehicle's surroundings, helping the driver more intuitively understand the vehicle's surroundings.

[0003] To generate seamless BEV images, extrinsic calibration between surround-view cameras is crucial for system accuracy. This extrinsic calibration also refers to position and pose calibration. For example, with image stitching quality as the optimization goal, camera extrinsic parameters are adjusted to align overlapping images. However, due to a lack of reasonable initial values ​​or reliance on image feature matching, convergence stability and accuracy are often difficult to guarantee.

[0004] In summary, how to accurately calibrate a vehicle's surround view camera is a problem that currently needs to be solved urgently by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a vehicle surround view camera calibration method, which can, to a certain extent, solve the technical problem of how to accurately calibrate a vehicle surround view camera. This application also provides a vehicle surround view camera marking system, electronic device, and computer-readable storage medium.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] A vehicle surround view camera calibration method, comprising:

[0008] Collect the calibration plate image obtained by taking a picture of the calibration plate by the surround view camera on the vehicle;

[0009] Projecting the calibration plate image to the world coordinate system to obtain a dedistorted image;

[0010] Selecting initial corner points in the dedistorted image according to the calibration plate;

[0011] Projecting the initial corner point to the fisheye image coordinate system of the surround-view camera to obtain the target corner point;

[0012] Generating initial external parameters of the surround-view camera according to the target corner point;

[0013] Extracting overlapping corner point parameters in the overlapping shooting area from the calibration plate image;

[0014] A target extrinsic parameter that minimizes an error between the overlapping corner point parameters is generated based on the initial extrinsic parameter, and the target extrinsic parameter is used as an extrinsic parameter of the surround-view camera.

[0015] In an exemplary embodiment, projecting the calibration plate image into a world coordinate system to obtain a dedistorted image includes:

[0016] Obtaining the internal parameter data of the surround view camera and setting the external parameters;

[0017] Projecting the calibration plate image to normalized image coordinates according to the intrinsic reference data to obtain a calibration plate normalized image;

[0018] Extracting the ray direction of the pixel point in the normalized image of the calibration plate in the camera coordinate system;

[0019] Transforming the ray direction according to the set extrinsic parameter to obtain the direction vector of the pixel point in the normalized image of the calibration plate in the world coordinate system;

[0020] Determining a camera center position of the surround-view camera in the world coordinate system according to the set external parameters;

[0021] The normalized image of the calibration plate is transformed based on the camera center position and the direction vector to obtain a dedistorted image.

[0022] In an exemplary embodiment, projecting the initial corner point onto the fisheye image coordinate system of the surround view camera to obtain the target corner point includes:

[0023] Obtaining the internal parameter data of the surround view camera and setting the external parameters;

[0024] Converting the initial corner point to the camera coordinate system according to the set external parameters to obtain the camera coordinate system corner point;

[0025] Projecting the camera coordinate system corner point to the normalized image coordinate system to obtain the normalized camera corner point;

[0026] The normalized camera corner points are converted to the fisheye image coordinate system of the surround-view camera according to the intrinsic reference data to obtain target corner points.

[0027] In an exemplary embodiment, generating target extrinsic parameters that minimize the error between the overlapping corner point parameters based on the initial extrinsic parameters, and using the target extrinsic parameters as the extrinsic parameters of the surround-view camera, includes:

[0028] Generating conversion information of the camera coordinate system of the surround-view camera relative to the vehicle coordinate system according to the initial external parameters;

[0029] According to the conversion information, the overlapping corner point parameters are converted into a bird's-eye view coordinate system to obtain bird's-eye view corner point parameters;

[0030] Generating target extrinsic parameters that minimize the errors between the corner point parameters of the bird's-eye view image;

[0031] The target external parameter is used as an external parameter of the camera coordinate system of the surround view camera relative to the vehicle coordinate system.

[0032] In an exemplary embodiment, converting the overlapping corner point parameters to a bird's-eye view coordinate system according to the conversion information to obtain the bird's-eye view corner point parameters includes:

[0033] Obtaining internal reference data of the surround view camera;

[0034] Projecting the overlapping corner point parameters to normalized image coordinates according to the intrinsic reference data to obtain normalized overlapping corner point parameters;

[0035] generating a depth scale factor based on translation information in the transformation information;

[0036] Back-projecting the normalized overlapping corner point parameters according to the depth scale factor to obtain bird's-eye view plane coordinates;

[0037] The plane coordinates of the bird's-eye view are transformed according to the transformation information to obtain bird's-eye view corner point parameters.

[0038] In an exemplary embodiment, generating a target extrinsic parameter that minimizes an error between parameters of corner points of the bird's-eye view includes:

[0039] generating distance values ​​between corner point parameters of the bird's-eye view;

[0040] Using the distance value as the error between the corner point parameters of the bird's-eye view;

[0041] Generating an external parameter optimization model that minimizes the error;

[0042] Generate target extrinsic parameters according to the extrinsic parameter optimization model.

[0043] In an exemplary embodiment, generating conversion information of the camera coordinate system of the surround view camera relative to the vehicle coordinate system based on the initial extrinsic parameters includes:

[0044] Obtaining position information of the vehicle relative to the calibration plate;

[0045] generating rotation and translation information of the world coordinate system relative to the vehicle coordinate system based on the position information;

[0046] Transformation information of the camera coordinate system of the surround-view camera relative to the vehicle coordinate system is generated according to the initial external parameters and the rotation and translation information.

[0047] A vehicle surround view camera calibration system, comprising:

[0048] An image acquisition module is used to acquire an image of the calibration plate obtained by photographing the calibration plate with a surround-view camera on the vehicle;

[0049] A dedistortion module, configured to project the calibration plate image into a world coordinate system to obtain a dedistorted image;

[0050] An initial corner point selection module, configured to select initial corner points in the dedistorted image according to the calibration plate;

[0051] A corner point projection module is used to project the initial corner point into the fisheye image coordinate system of the surround view camera to obtain a target corner point;

[0052] An initial extrinsic parameter generation module, configured to generate initial extrinsic parameters of the surround-view camera based on the target corner points;

[0053] An overlapping corner point selection module is used to extract overlapping corner point parameters in the overlapping shooting area from the calibration plate image;

[0054] A target extrinsic parameter generation module is configured to generate target extrinsic parameters that minimize the error between the overlapping corner point parameters based on the initial extrinsic parameters, and use the target extrinsic parameters as the extrinsic parameters of the surround-view camera.

[0055] An electronic device, comprising:

[0056] memory for storing computer programs;

[0057] A processor is configured to implement the steps of any of the above-mentioned vehicle surround view camera calibration methods when executing the computer program.

[0058] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned vehicle surround view camera calibration methods.

[0059] The present application provides a vehicle surround-view camera calibration method, which includes collecting a calibration plate image obtained by photographing a calibration plate with a surround-view camera on a vehicle; projecting the calibration plate image into a world coordinate system to obtain a dedistorted image; selecting initial corner points in the dedistorted image based on the calibration plate; projecting the initial corner points into the fisheye image coordinate system of the surround-view camera to obtain target corner points; generating initial extrinsic parameters of the surround-view camera based on the target corner points; extracting overlapping corner point parameters in the overlapping shooting area from the calibration plate image; generating target extrinsic parameters based on the initial extrinsic parameters that minimize the error between the overlapping corner point parameters, and using the target extrinsic parameters as the extrinsic parameters of the surround-view camera. In this application, the calibration plate image is projected onto the world coordinate system for dedistortion to select initial corner points. This avoids the problem of unstable corner point extraction caused by strong edge distortion in fisheye images, improves the accuracy and robustness of initial corner point extraction, and then accurately generates the initial extrinsic parameters of the surround-view camera using the initial corner points. Furthermore, the corner point parameters of the overlapping area of ​​adjacent surround-view cameras can be optimized to minimize errors in the overlapping area, improve the coherence and smoothness of the images captured by the surround-view cameras, reduce splicing gaps and misalignment, and make the calibration results of the vehicle surround-view camera more accurate. The vehicle surround-view camera calibration system, electronic device, and computer-readable storage medium provided in this application also solve corresponding technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0061] Figure 1 A flowchart of a vehicle surround view camera marking method provided in an embodiment of the present application;

[0062] Figure 2 A schematic structural diagram of a vehicle surround view camera marking system provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0064] Figure 4 Another structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0066] See also Figure 1 , Figure 1 A flowchart of a vehicle surround view camera marking method provided in an embodiment of the present application.

[0067] A vehicle surround view camera calibration method provided in an embodiment of the present application may include the following steps:

[0068] Step S101: capturing an image of the calibration plate obtained by photographing the calibration plate with a surround view camera on a vehicle.

[0069] In practical applications, a calibration plate is required to calibrate the surround-view camera on a vehicle. Specifically, an image of the calibration plate is captured by the surround-view camera on the vehicle, so that the calibration plate image can be analyzed to obtain the calibration result of the surround-view camera.

[0070] It should be noted that the information of the calibration board and the surround-view camera can be flexibly determined according to the application scenario. For example, the calibration board can be a checkerboard calibration board, and the surround-view camera can be four surround-view cameras on the front, rear, left, and right of the vehicle.

[0071] Step S102: Project the calibration plate image into the world coordinate system to obtain a dedistorted image.

[0072] Step S103: Selecting initial corner points in the dedistorted image according to the calibration plate.

[0073] Step S104: Projecting the initial corner point to the fisheye image coordinate system of the surround view camera to obtain the target corner point.

[0074] In actual applications, due to the influence of the complex external environment, the edge areas of the calibration plate captured by each surround-view camera will be relatively blurred. In addition, the calibration plate image captured by the surround-view camera is a fisheye image, and fisheye images are most severely distorted in the edge areas. It is difficult to directly extract corner points on the original calibration plate image for surround-view camera calibration. To avoid this situation and obtain more accurate corner points for calibrating the surround-view camera, the calibration plate image can be first projected into the world coordinate system (WCS) for dedistortion to obtain a dedistorted image. The world coordinate system refers to a unified reference coordinate system that describes all geometric entities in the vehicle environment. This entity can be the calibration plate, the ground, etc.; then, based on the calibration plate, initial corner points are selected in the dedistorted image. For example, the checkerboard corner points in the calibration plate target are extracted from the dedistorted image using a corner detection algorithm such as Harris. Finally, the initial corner points are projected into the fisheye image coordinate system of the surround-view camera to obtain the target corner points, so that the target corner points can be used to solve the PnP (Perspective-n-Point) extrinsic parameter of the surround-view camera.

[0075] In an exemplary embodiment, the calibration plate image is projected onto the world coordinate system to obtain a dedistorted image. Image projection can be performed with the help of a normalized plane and based on the distance between pixel points, that is, intrinsic parameter data of the surround-view camera and extrinsic parameters can be obtained, and the extrinsic parameters can be set arbitrarily by the user; the calibration plate image is projected onto the normalized image coordinates according to the intrinsic parameter data to obtain a calibration plate normalized image; the ray direction of the pixel point in the calibration plate normalized image in the camera coordinate system is extracted; the ray direction is transformed according to the set extrinsic parameter to obtain the direction vector of the pixel point in the calibration plate normalized image in the world coordinate system; the camera center position of the surround-view camera in the world coordinate system is determined according to the set extrinsic parameter; the calibration plate normalized image is converted based on the camera center position and the direction vector to obtain a dedistorted image.

[0076] In order to understand the dedistortion process, it is assumed that the pixels in the calibration plate image are Indicates that the pixel points in the normalized image of the calibration plate It can be expressed as , Represents the inverse of the internal parameter matrix corresponding to the internal parameter data; The ray direction in the camera coordinate system is recorded as , then the direction vector It can be expressed as , Indicates the external parameters of the camera coordinate system projected onto the world coordinate system, the camera center position Expressed as , Represents the translation information in the set external parameter; the three-dimensional space point corresponding to the pixel point can be expressed as the intersection of the ray along the direction and the ground plane Z=0, and the world point is set to , obtained by ray-plane intersection :in, is the intersection parameter, by setting The third dimension (Z value) is 0 and we get , Indicates the coordinate value of the pixel point in the calibration plate image when the Z value is zero. Represents the direction vector when the Z value is zero; ultimately, the pixel point Projected point on the ground in the world coordinate system for In other words, this formula can be used to transform the normalized image of the calibration plate based on the camera center position and direction vector to obtain the dedistorted image.

[0077] In an exemplary embodiment, the process of projecting the initial corner point to the fisheye image coordinate system of the surround-view camera to obtain the target corner point can also be carried out with the help of a normalized plane, that is, the intrinsic parameter data of the surround-view camera and the set extrinsic parameters can be obtained; the initial corner point is converted to the camera coordinate system according to the set extrinsic parameters to obtain the camera coordinate system corner point; the camera coordinate system corner point is projected to the normalized image coordinates to obtain the normalized camera corner point; the normalized camera corner point is converted to the fisheye image coordinate system of the surround-view camera according to the intrinsic parameter data to obtain the target corner point. The fisheye image coordinate system refers to a coordinate system established with the fisheye image taken by the surround-view camera.

[0078] In order to understand the back-projection process of corner points, it is assumed that the initial corner point set is express, represents the total number of initial corner points, Indicates the number of the initial corner point; then the camera coordinate system corner point It can be expressed as ; Normalized camera corners It can be expressed as ; Target corner It can be expressed as .

[0079] Step S105: Generate initial external parameters of the surround view camera based on the target corner points.

[0080] In practical applications, after obtaining the target corner points, we can first use the target corner points to preliminarily obtain the initial extrinsic parameters of the surround view camera.

[0081] In an exemplary embodiment, a 3D-2D point pair can be constructed based on the target corner points to solve the initial external parameters. For example, the extracted target corner points can be screened and sorted to ensure that their order is consistent with the predefined calibration plate structure to obtain a set of image coordinate points. At the same time, since the spatial structure of the calibration plate is known, each corner point can be mapped to an actual coordinate point in the world coordinate system. Through the above one-to-one correspondence of 3D-2D points , the PnP algorithm is used to solve the initial external parameters of the surround view camera.

[0082] The goal of PnP is to estimate the camera's rotation matrix R and translation matrix t so that the following projection error is minimized:

[0083] ;

[0084] in, represents the Euclidean distance, Indicates that the 3D point is passed through the internal reference data Projected onto the image plane, ; After solving, the initial external parameters of each surround camera can be obtained , Indicates the number of the surround-view camera. This initial extrinsic parameter is the extrinsic parameter of the surround-view camera relative to the world coordinate system and is used for subsequent stitching optimization.

[0085] Step S106: extracting the overlapping corner point parameters in the overlapping shooting area from the calibration plate image.

[0086] Step S107: generating target extrinsic parameters that minimize the error between overlapping corner point parameters based on the initial extrinsic parameters, and using the target extrinsic parameters as extrinsic parameters of the surround view camera.

[0087] In practical applications, the initial extrinsic parameters obtained based on the target corner points do not utilize the constraints of the overlapping areas between different surround-view cameras, which will result in poor stitching of the images captured by the surround-view cameras. To further improve the panoramic stitching effect of the images captured by the surround-view cameras, the initial extrinsic parameters can be optimized based on the constraints of the overlapping areas. That is, the overlapping corner point parameters in the overlapping shooting area can be extracted from the calibration plate image, and the target extrinsic parameters that minimize the error between the overlapping corner point parameters can be generated based on the initial extrinsic parameters, and the target extrinsic parameters can be used as the extrinsic parameters of the surround-view cameras. In this way, the initial extrinsic parameters obtained by preliminary estimation can be used to further improve the stitching accuracy and consistency of the surround-view images through the degree of overlap in the overlapping area of ​​adjacent surround-view cameras.

[0088] In an exemplary embodiment, in a process of generating target extrinsics based on initial extrinsics that minimizes the error between overlapping corner point parameters and using the target extrinsics as the extrinsics of a surround-view camera, since the surround-view camera is mounted on a vehicle coordinate system (VCS), the initial extrinsics can be optimized using the vehicle coordinate system (VCS) as a transit point, and the parameters can be processed using IPM (Inverse Perspective Mapping). Specifically, transformation information of the surround-view camera's camera coordinate system relative to the vehicle coordinate system is generated based on the initial extrinsics. The vehicle coordinate system is a three-dimensional coordinate system established with the vehicle's geometric center as its origin and can follow the right-hand rule, with the X-axis pointing forward of the vehicle, the Y-axis pointing to the left of the vehicle, and the Z-axis pointing upward of the vehicle. Based on the transformation information, the overlapping corner point parameters are transformed into a bird's-eye view (BEV) coordinate system to obtain BEV corner point parameters. Target extrinsics that minimize the error between the BEV corner point parameters are generated, and the target extrinsics are used as the extrinsics of the surround-view camera's camera coordinate system relative to the vehicle coordinate system.

[0089] In a specific application scenario, in the process of converting the overlapping corner point parameters to the bird's-eye view coordinate system according to the transformation information to obtain the bird's-eye view corner point parameters, the intrinsic parameter data of the surround view camera can be obtained; the overlapping corner point parameters are projected to the normalized image coordinates according to the intrinsic parameter data to obtain the normalized overlapping corner point parameters; the depth scale factor is generated based on the translation information in the transformation information; the normalized overlapping corner point parameters are back-projected according to the depth scale factor to obtain the bird's-eye view plane coordinates; the bird's-eye view plane coordinates are transformed according to the transformation information to obtain the bird's-eye view corner point parameters.

[0090] In order to understand the generation process of bird's-eye view corner point parameters, it is assumed that the target corner point is represented as , then the normalized overlapping corner parameter The formula Generate; Bird's-eye view plane coordinates The formula can be generate, Depth scale factor; Bird's-eye view corner parameter The formula can be generate.

[0091] In a specific application scenario, in the process of generating the target external parameter that minimizes the error between the corner point parameters of the bird's-eye view, the distance value between the corner point parameters of the bird's-eye view can be generated. This distance value can be determined according to actual needs, such as the Euclidean distance value; the distance value is used as the error between the corner point parameters of the bird's-eye view; an external parameter optimization model that minimizes the error is generated, assuming that the distance value is the Euclidean distance value, the corner point parameters are the corner point coordinates, and the set of all overlapping surround view camera pairs is , No. A surround view camera and There is an overlapping area between the surround view cameras, and the bird's-eye view corner parameters in this overlapping area are expressed as … , represents the number of overlapping corner points, then the external parameter optimization model can be expressed as , Indicates the The target extrinsic parameters of the camera coordinate system of the surround view camera relative to the vehicle coordinate system, Indicates the The target extrinsic parameters are the camera coordinate system of each surround-view camera relative to the vehicle coordinate system; the target extrinsic parameters are generated according to the extrinsic parameter optimization model, that is, the target extrinsic parameters can be generated by solving the extrinsic parameter optimization model, such as generating the target extrinsic parameters by iteratively solving the extrinsic parameter optimization model through a nonlinear least squares algorithm.

[0092] In specific application scenarios, in the process of generating the conversion information of the camera coordinate system of the surround view camera relative to the vehicle coordinate system based on the initial external parameters, the position information of the vehicle relative to the calibration plate can be obtained; based on the position information, the rotation and translation information of the world coordinate system relative to the vehicle coordinate system is generated. ; Based on the initial external parameters and rotation and translation information, generate the transformation information of the camera coordinate system of the surround view camera relative to the vehicle coordinate system, for example, it can be obtained by the formula To process the initial external parameters and rotation and translation information, generate the transformation information of the camera coordinate system of the surround view camera relative to the vehicle coordinate system ,in, Represents a homogeneous matrix consisting of a rotation matrix and a translation matrix.

[0093] The present application provides a vehicle surround-view camera calibration method, which includes collecting a calibration plate image obtained by photographing a calibration plate with a surround-view camera on a vehicle; projecting the calibration plate image into a world coordinate system to obtain a dedistorted image; selecting initial corner points in the dedistorted image based on the calibration plate; projecting the initial corner points into the fisheye image coordinate system of the surround-view camera to obtain target corner points; generating initial extrinsic parameters of the surround-view camera based on the target corner points; extracting overlapping corner point parameters in the overlapping shooting area from the calibration plate image; generating target extrinsic parameters based on the initial extrinsic parameters that minimize the error between the overlapping corner point parameters, and using the target extrinsic parameters as the extrinsic parameters of the surround-view camera. In this application, the calibration plate image is projected into the world coordinate system for dedistortion to select the initial corner points, which avoids the problem of unstable corner point extraction caused by strong edge distortion of the fisheye image, improves the extraction accuracy and robustness of the initial corner points, and then the initial external parameters of the surround-view camera can be accurately generated through the initial corner points; and the corner point parameters of the overlapping area of ​​adjacent surround-view cameras can be used for optimization to minimize the error in the overlapping area, thereby improving the continuity and smoothness of the images taken by the surround-view camera, reducing splicing gaps and misalignment, and making the calibration results of the vehicle surround-view camera more accurate.

[0094] See also Figure 2 , Figure 2 A schematic structural diagram of a vehicle surround view camera marking system provided in an embodiment of the present application.

[0095] An embodiment of the present application provides a vehicle surround view camera calibration system, which may include:

[0096] The image acquisition module 101 is used to acquire an image of the calibration plate obtained by photographing the calibration plate with a surround view camera on the vehicle;

[0097] The dedistortion module 102 is used to project the calibration plate image into the world coordinate system to obtain a dedistorted image;

[0098] An initial corner point selection module 103 is used to select initial corner points in the dedistorted image according to a calibration plate;

[0099] A corner point projection module 104 is used to project the initial corner point to the fisheye image coordinate system of the surround view camera to obtain the target corner point;

[0100] An initial external parameter generation module 105 is used to generate initial external parameters of the surround view camera based on the target corner points;

[0101] The overlapping corner point selection module 106 is used to extract the overlapping corner point parameters in the overlapping shooting area in the calibration plate image;

[0102] The target extrinsic parameter generation module 107 is configured to generate target extrinsic parameters that minimize the error between overlapping corner point parameters based on the initial extrinsic parameters, and use the target extrinsic parameters as extrinsic parameters for the surround view camera.

[0103] In an embodiment of the present application, a vehicle surround view camera calibration system is provided, wherein a dedistortion module may include:

[0104] A first acquisition unit, configured to acquire internal parameter data of the surround view camera and set external parameters;

[0105] A first normalization unit, configured to project the calibration plate image to normalized image coordinates according to the intrinsic reference data to obtain a normalized image of the calibration plate;

[0106] A first extraction unit is used to extract the ray direction of the pixel point in the normalized image of the calibration plate in the camera coordinate system;

[0107] A first transformation unit is used to transform the ray direction according to the set external parameter to obtain the direction vector of the pixel point in the normalized image of the calibration plate in the world coordinate system;

[0108] A first determining unit is used to determine a camera center position of the surround view camera in a world coordinate system according to set external parameters;

[0109] The first conversion unit is used to convert the normalized image of the calibration plate based on the camera center position and direction vector to obtain a dedistorted image.

[0110] In an embodiment of the present application, a vehicle surround view camera calibration system is provided, wherein a corner projection module may include:

[0111] A second acquisition unit is used to obtain the internal parameter data of the surround view camera and set the external parameters;

[0112] The second conversion unit is used to convert the initial corner point into the camera coordinate system according to the set external parameters to obtain the camera coordinate system corner point;

[0113] A first projection unit, configured to project the camera coordinate system corner point to the normalized image coordinate system to obtain a normalized camera corner point;

[0114] The third conversion unit is used to convert the normalized camera corner point into the fisheye image coordinate system of the surround view camera according to the intrinsic reference data to obtain the target corner point.

[0115] In an embodiment of the present application, a vehicle surround view camera calibration system is provided, wherein a target extrinsic parameter generation module may include:

[0116] A first generating unit, configured to generate conversion information of a camera coordinate system of a surround view camera relative to a vehicle coordinate system based on initial external parameters;

[0117] a fourth conversion unit, configured to convert the overlapping corner point parameters into a bird's-eye view coordinate system according to the conversion information to obtain bird's-eye view corner point parameters;

[0118] The second generating unit is used to generate target extrinsic parameters that minimize the error between the corner point parameters of the bird's-eye view;

[0119] The first setting unit is configured to use the target external parameter as an external parameter of a camera coordinate system of the surround view camera relative to a vehicle coordinate system.

[0120] An embodiment of the present application provides a vehicle surround view camera calibration system, wherein a fourth conversion unit is configured to: obtain intrinsic parameter data of the surround view camera; project overlapping corner point parameters to normalized image coordinates based on the intrinsic parameter data to obtain normalized overlapping corner point parameters; generate a depth scale factor based on translation information in the conversion information; back-project the normalized overlapping corner point parameters based on the depth scale factor to obtain bird's-eye view plane coordinates; and convert the bird's-eye view plane coordinates based on the conversion information to obtain bird's-eye view corner point parameters.

[0121] An embodiment of the present application provides a vehicle surround view camera calibration system, wherein the second generation unit is used to: generate a distance value between the corner point parameters of a bird's-eye view; use the distance value as the error between the corner point parameters of the bird's-eye view; generate an extrinsic parameter optimization model that minimizes the error; and generate a target extrinsic parameter based on the extrinsic parameter optimization model.

[0122] An embodiment of the present application provides a vehicle surround view camera calibration system, in which the first generation unit can be used to: obtain position information of the vehicle relative to the calibration plate; generate rotation and translation information of the world coordinate system relative to the vehicle coordinate system based on the position information; and generate transformation information of the camera coordinate system of the surround view camera relative to the vehicle coordinate system based on the initial external parameters and the rotation and translation information.

[0123] This application also provides an electronic device and a computer-readable storage medium, both of which have the corresponding effects of the vehicle surround view camera calibration method provided in the embodiment of this application. Figure 3 , Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0124] An electronic device provided in an embodiment of the present application includes a memory 201 and a processor 202. The memory 201 stores a computer program, and when the processor 202 executes the computer program, the steps of the vehicle surround view camera calibration method described in any of the above embodiments are implemented.

[0125] See also Figure 4Another electronic device provided in an embodiment of the present application may further include: an input port 203 connected to the processor 202 for transmitting commands inputted from the outside to the processor 202; a display unit 204 connected to the processor 202 for displaying the processing results of the processor 202 to the outside world; and a communication module 205 connected to the processor 202 for enabling communication between the electronic device and the outside world. The display unit 204 may be a display panel, a laser scanning display, etc. The communication method adopted by the communication module 205 includes but is not limited to Mobile High-Definition Link (MHL), Universal Serial Bus (USB), High-Definition Multimedia Interface (HDMI), wireless connection: Wireless Fidelity (WiFi), Bluetooth communication technology, Bluetooth low energy communication technology, and communication technology based on IEEE802.11s.

[0126] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the vehicle surround view camera calibration method described in any of the above embodiments are implemented.

[0127] The computer-readable storage medium involved in this application includes random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs (Compact Disc Read-Only Memory), or any other form of storage medium known in the technical field.

[0128] For the description of the relevant portions of the vehicle surround-view camera calibration system, electronic device, and computer-readable storage medium provided in the embodiments of this application, please refer to the detailed description of the corresponding portions in the vehicle surround-view camera calibration method provided in the embodiments of this application, and will not be repeated here. In addition, the portions of the above-mentioned technical solutions provided in the embodiments of this application that are consistent with the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0129] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0130] The above description of the disclosed embodiments will enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vehicle surround view camera calibration method, characterized in that: include: Collect the calibration plate image obtained by taking a picture of the calibration plate by the surround view camera on the vehicle; Projecting the calibration plate image to the world coordinate system to obtain a dedistorted image; Selecting initial corner points in the dedistorted image according to the calibration plate; Projecting the initial corner point to the fisheye image coordinate system of the surround-view camera to obtain the target corner point; Generating initial external parameters of the surround-view camera according to the target corner point; Extracting overlapping corner point parameters in the overlapping shooting area from the calibration plate image; A target extrinsic parameter that minimizes an error between the overlapping corner point parameters is generated based on the initial extrinsic parameter, and the target extrinsic parameter is used as an extrinsic parameter of the surround-view camera.

2. The method according to claim 1, characterized in that The step of projecting the calibration plate image onto a world coordinate system to obtain a dedistorted image includes: Obtaining the internal parameter data of the surround view camera and setting the external parameters; Projecting the calibration plate image to normalized image coordinates according to the intrinsic reference data to obtain a calibration plate normalized image; Extracting the ray direction of the pixel point in the normalized image of the calibration plate in the camera coordinate system; Transforming the ray direction according to the set extrinsic parameter to obtain the direction vector of the pixel point in the normalized image of the calibration plate in the world coordinate system; Determining a camera center position of the surround-view camera in the world coordinate system according to the set external parameters; The normalized image of the calibration plate is transformed based on the camera center position and the direction vector to obtain a dedistorted image.

3. The method according to claim 1, characterized in that The projecting the initial corner point to the fisheye image coordinate system of the surround-view camera to obtain the target corner point includes: Obtaining the internal parameter data of the surround view camera and setting the external parameters; Converting the initial corner point to the camera coordinate system according to the set external parameters to obtain the camera coordinate system corner point; Projecting the camera coordinate system corner point to the normalized image coordinate system to obtain the normalized camera corner point; The normalized camera corner points are converted to the fisheye image coordinate system of the surround-view camera according to the intrinsic reference data to obtain target corner points.

4. The method according to claim 1, wherein Generating target extrinsic parameters that minimize the error between the overlapping corner point parameters based on the initial extrinsic parameters, and using the target extrinsic parameters as the extrinsic parameters of the surround-view camera, includes: Generating conversion information of the camera coordinate system of the surround-view camera relative to the vehicle coordinate system according to the initial external parameters; According to the conversion information, the overlapping corner point parameters are converted into a bird's-eye view coordinate system to obtain bird's-eye view corner point parameters; Generating target extrinsic parameters that minimize the errors between the corner point parameters of the bird's-eye view image; The target external parameter is used as an external parameter of the camera coordinate system of the surround view camera relative to the vehicle coordinate system.

5. The method according to claim 4, characterized in that The step of converting the overlapping corner point parameters into a bird's-eye view coordinate system according to the conversion information to obtain the bird's-eye view corner point parameters includes: Obtaining internal reference data of the surround view camera; Projecting the overlapping corner point parameters to normalized image coordinates according to the intrinsic reference data to obtain normalized overlapping corner point parameters; generating a depth scale factor based on translation information in the transformation information; Back-projecting the normalized overlapping corner point parameters according to the depth scale factor to obtain bird's-eye view plane coordinates; The plane coordinates of the bird's-eye view are transformed according to the transformation information to obtain bird's-eye view corner point parameters.

6. The method according to claim 4, characterized in that Generating target extrinsic parameters that minimize the error between the corner point parameters of the bird's-eye view image includes: generating distance values ​​between corner point parameters of the bird's-eye view; Using the distance value as the error between the corner point parameters of the bird's-eye view; Generating an external parameter optimization model that minimizes the error; Generate target extrinsic parameters according to the extrinsic parameter optimization model.

7. The method according to claim 4, characterized in that Generating conversion information of the camera coordinate system of the surround view camera relative to the vehicle coordinate system based on the initial external parameters includes: Obtaining position information of the vehicle relative to the calibration plate; generating rotation and translation information of the world coordinate system relative to the vehicle coordinate system based on the position information; Transformation information of the camera coordinate system of the surround-view camera relative to the vehicle coordinate system is generated according to the initial external parameters and the rotation and translation information.

8. A vehicle surround view camera calibration system, characterized in that: include: An image acquisition module is used to acquire an image of the calibration plate obtained by photographing the calibration plate with a surround-view camera on the vehicle; A dedistortion module, configured to project the calibration plate image into a world coordinate system to obtain a dedistorted image; An initial corner point selection module, configured to select initial corner points in the dedistorted image according to the calibration plate; A corner point projection module is used to project the initial corner point into the fisheye image coordinate system of the surround view camera to obtain a target corner point; An initial extrinsic parameter generation module, configured to generate initial extrinsic parameters of the surround-view camera based on the target corner points; An overlapping corner point selection module is used to extract overlapping corner point parameters in the overlapping shooting area from the calibration plate image; A target extrinsic parameter generation module is configured to generate target extrinsic parameters that minimize the error between the overlapping corner point parameters based on the initial extrinsic parameters, and use the target extrinsic parameters as the extrinsic parameters of the surround-view camera.

9. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the vehicle surround view camera calibration method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the vehicle surround view camera calibration method according to any one of claims 1 to 7.

Images