Camera extrinsic parameter calibration method, device, equipment and storage medium

Abstract

The application provides a camera extrinsic parameter calibration method, device, equipment and storage medium. In one example, the method comprises: acquiring an image collected by a vehicle-mounted surround-view camera; for a front / rear camera in the vehicle-mounted surround-view camera, determining an angle extrinsic parameter of the front / rear camera according to a straight line equation of a lane line in a de-distorted image, a width constraint of the lane line, and an installation position parameter of the front / rear camera; correcting the angle extrinsic parameter of a side camera according to a geometric feature of the lane line in an image collected by the side camera; and converting the images collected by each vehicle-mounted surround-view camera into a BEV view according to the optimized extrinsic parameter, and optimizing the angle extrinsic parameter of the side camera in a manner of aligning key texture feature points in a common view area of the side camera and the front / rear camera, to obtain an accurate angle extrinsic parameter of the side camera. The method can realize camera extrinsic parameter calibration in a natural scene, and can improve the accuracy of the angle extrinsic parameter of the side camera.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, device and storage medium for calibrating camera extrinsic parameters. Background Technology

[0002] The in-vehicle 360-degree surround view system generates a panoramic bird's-eye view image in real time using four wide-angle cameras around the vehicle, providing intuitive visual assistance to the driver in low-speed driving scenarios. As ADAS (Advanced Driver Assistance Systems) technology evolves towards higher levels of autonomous driving, the market demand for surround view systems, as the core unit of environmental perception, continues to rise, gradually covering more and more vehicle models.

[0003] In the engineering implementation of surround view systems, multi-camera extrinsic parameter calibration is a core step in panoramic stitching. The accuracy of the calibration parameters directly affects the geometric consistency of image fusion, and its error tolerance needs to be controlled at the sub-pixel level. Therefore, calibration accuracy directly determines the reliability and user experience of the surround view system. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, device and storage medium for calibrating camera extrinsic parameters.

[0005] Specifically, this application is implemented through the following technical solution:

[0006] According to a first aspect of the embodiments of this application, a camera extrinsic parameter calibration method is provided, comprising:

[0007] Acquire images captured by the vehicle-mounted surround view camera;

[0008] For the front-view / rear-view cameras in the vehicle surround view camera system, the angular extrinsic parameters of the front-view / rear-view cameras are determined based on the straight line equations of the lane lines in the distortion-free image, the lane line width constraints, and the installation position parameters of the front-view / rear-view cameras. The distortion-free image is obtained by distorting the images acquired by the front-view / rear-view cameras based on their intrinsic parameters. The lane line width constraints include ensuring that the widths of different lane lines are consistent when projected onto the vehicle's world coordinate system.

[0009] Based on the angular extrinsic parameters of the front / rear view cameras, the installation position parameters of the front / rear view cameras, and the installation position parameters of the side view cameras in the vehicle surround view cameras, the lane lines in the images acquired by the side view cameras are geometrically corrected to determine the angular extrinsic parameters of the side view cameras.

[0010] Based on the installation position parameters and angular extrinsic parameters of each target camera, the images acquired by each vehicle-mounted surround view camera are converted into BEV views. The angular extrinsic parameters of the side view camera are optimized by aligning key texture feature points in the common viewing area of ​​the side view camera and the front / rear view camera, so as to obtain the accurate angular extrinsic parameters of the side view camera.

[0011] According to a second aspect of the embodiments of this application, a camera extrinsic parameter calibration device is provided, comprising:

[0012] The acquisition unit is used to acquire images captured by the vehicle-mounted surround view camera;

[0013] The first determining unit is configured to determine the angular extrinsic parameters of the front / rear view cameras in the vehicle surround view camera based on the straight line equations of lane lines in the distorted image, the width constraints of lane lines, and the installation position parameters of the front / rear view cameras; wherein, the distorted image is obtained by distorting the image acquired by the front / rear view cameras based on the intrinsic parameters of the front / rear view cameras, and the lane line width constraints include the consistency of the width of different lane lines when the lane lines are projected onto the vehicle's world coordinate system;

[0014] The second determining unit is used to perform geometric feature correction on the lane lines in the image acquired by the side-view camera based on the angular extrinsic parameters of the front-view / rear-view camera, the installation position parameters of the front-view / rear-view camera, and the installation position parameters of the side-view camera in the vehicle surround view camera, and to determine the angular extrinsic parameters of the side-view camera.

[0015] The optimization unit is used to convert the images acquired by each vehicle-mounted surround view camera into a bird's-eye view of the BEV based on the installation position parameters and angular extrinsic parameters of each target camera, and to optimize the angular extrinsic parameters of the side view camera by aligning key texture feature points in the common viewing area of ​​the side view camera and the front / rear view camera, so as to obtain the accurate angular extrinsic parameters of the side view camera.

[0016] According to a third aspect of the present application, an electronic device is provided, including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor being configured to execute the machine-executable instructions to implement the method provided in the first aspect.

[0017] According to a fourth aspect of the embodiments of this application, a machine-readable storage medium is provided, wherein machine-executable instructions are stored therein, and when the machine-executable instructions are executed by a processor, the method provided in the first aspect is implemented.

[0018] The technical solution provided in this application can bring at least the following beneficial effects:

[0019] By acquiring images from the vehicle-mounted surround-view cameras, for the front / rear-view cameras, the angular extrinsic parameters of the front / rear-view cameras are determined based on the straight-line equations of lane lines in the distortion-free images, lane line width constraints, and the installation position parameters of the front / rear-view cameras. Then, based on the angular extrinsic parameters of the front / rear-view cameras, the installation position parameters of the front / rear-view cameras, the installation position parameters of the side-view cameras in the vehicle-mounted surround-view cameras, and the geometric features of lane lines in the images acquired by the side-view cameras, the angular extrinsic parameters of the side-view cameras are corrected to determine the final angular extrinsic parameters of the side-view cameras. Based on the installation position parameters and angular extrinsic parameters of each target camera, the images acquired by each vehicle-mounted surround-view camera are converted into BEV views. By aligning key texture feature points in the shared field of view of the side-view camera and the front / rear-view camera, the angular extrinsic parameters of the side-view camera are optimized to obtain accurate angular extrinsic parameters. This achieves camera extrinsic parameter calibration in natural scenes without the need for a calibration site, reducing labor and site costs and improving the practicality of the solution. In addition, based on the lane line calibration solution, the extrinsic parameter angles of the side-view camera are optimized, making the final calibrated extrinsic parameters more accurate. Attached Figure Description

[0020] Figure 1 This is a schematic flowchart illustrating a camera extrinsic parameter calibration method according to an exemplary embodiment of this application;

[0021] Figure 2 This is a schematic diagram of a vanishing point in a distortion-free image of a forward-looking camera, as illustrated in an exemplary embodiment of this application.

[0022] Figure 3 This is a schematic diagram of the straight lines on the sides of two lane lines in a distortion-free image of a forward-looking camera, as illustrated in an exemplary embodiment of this application.

[0023] Figure 4 This is a schematic diagram illustrating a calibration environment as shown in an exemplary embodiment of this application;

[0024] Figure 5 This is a schematic diagram illustrating the structure of a camera extrinsic parameter calibration system according to an exemplary embodiment of this application;

[0025] Figure 6 This is a schematic diagram illustrating a camera capturing an image, as shown in an exemplary embodiment of this application.

[0026] Figure 7 This is a schematic diagram illustrating a data processing flow according to an exemplary embodiment of this application;

[0027] Figure 8 This is a schematic diagram illustrating a detailed external parameter calibration process according to an exemplary embodiment of this application;

[0028] Figure 9This is a schematic diagram illustrating a coordinate system according to an exemplary embodiment of this application;

[0029] Figure 10 This is a schematic diagram illustrating the shared viewing area of ​​a front-view camera, a left-view camera, and a rear-view camera, as shown in an exemplary embodiment of this application.

[0030] Figure 11 This is a schematic diagram illustrating a scene of feature point alignment between a front / rear view camera and a side view camera, as shown in an exemplary embodiment of this application.

[0031] Figure 12 This is a schematic diagram illustrating a matching point pair between a rear view and a left view, as shown in an exemplary embodiment of this application.

[0032] Figure 13 This is a schematic diagram showing the comparison of stitching effects before and after optimization of the extrinsic parameters of the side-view camera angle, as illustrated in an exemplary embodiment of this application.

[0033] Figure 14 This is a schematic diagram illustrating the structure of a camera extrinsic parameter calibration device according to an exemplary embodiment of this application;

[0034] Figure 15 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0035] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, some technical terms involved in the embodiments of this application will be explained below.

[0036] 1. Lane vanishing point (or simply vanishing point): This is the point where lane lines gradually disappear in the field of vision. It represents the intersection of these lines in perspective projection when they are parallel to the horizon and extend into the distance. Due to the curvature of the Earth and changes in viewing angle, lane lines gradually converge to a single point from near to far, which is usually located in the distance of the field of vision.

[0037] To make the above-mentioned objectives, features and advantages of the embodiments of this application more apparent and understandable, the technical solutions of the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0038] It should be noted that the sequence number of each step in the embodiments of this application does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0039] Please see Figure 1 This is a flowchart illustrating a camera extrinsic parameter calibration method provided in an embodiment of this application. Figure 1 As shown, the camera extrinsic parameter calibration method may include the following steps:

[0040] Step S100: Acquire images captured by the vehicle-mounted surround view camera.

[0041] For example, images of the external scene can be captured using an in-vehicle surround-view camera.

[0042] For example, images of the external scene can be captured by an onboard surround-view camera when the vehicle is in a natural scene (such as driving on a road).

[0043] Step S110: For the front / rear view cameras in the vehicle surround view camera, determine the extrinsic angle parameters of the front / rear view cameras based on the straight line equation of the lane lines in the distorted image, the width constraint of the lane lines, and the installation position parameters of the front / rear view cameras; wherein, the distorted image is obtained by distorting the image acquired by the front / rear view cameras based on the intrinsic parameters of the front / rear view cameras, and the lane line width constraint includes the fact that the width of different lane lines is consistent when the lane lines are projected onto the vehicle's world coordinate system.

[0044] For example, for a vehicle-mounted surround-view camera (which may be called a front-view camera) installed at the front of the vehicle body (or a vehicle-mounted surround-view camera installed at the rear of the vehicle body (which may be called a rear-view camera)), the angular extrinsic parameters of the vehicle-mounted surround-view camera can be calibrated based on the geometric features of the lane lines in the images captured by the vehicle-mounted surround-view camera.

[0045] For ease of description and understanding, the following text uses the calibration of the extrinsic angle of the foreseeable camera as an example; the calibration of the extrinsic angle of the rearseeable camera can be obtained in the same way.

[0046] In this embodiment of the application, during the process of calibrating the extrinsic parameters of the forward-looking camera, the image acquired by the designated target camera can be distorted based on the intrinsic parameters of the forward-looking camera to obtain a distorted image, and the straight line equation of the lane line in the distorted image can be determined.

[0047] For example, the straight line equation of a lane line may include the straight line equation of the side of the lane line.

[0048] In one example, edge point extraction and line fitting of straight lines can be performed in the distorted image to obtain the equation of the straight line in the distorted image (the equation of the straight line on the side of the lane line).

[0049] For example, methods for extracting edge points may include, but are not limited to, the Canny operator, the Sobel operator, or the Laplace operator.

[0050] For example, line fitting methods may include, but are not limited to, least squares method, gradient descent method, etc.

[0051] In another example, a deep learning network can be used to extract lines and determine the equations of lines in the distorted image.

[0052] For example, the angular extrinsic parameters of the forward-looking camera can be determined based on the straight line equation of the lane lines in the distorted image of the forward-looking camera, the width constraints of the lane lines, and the installation position parameters of the forward-looking camera.

[0053] For example, the installation location parameters of the vehicle-mounted surround view camera can be determined based on the installation location of the vehicle-mounted surround view camera on the vehicle.

[0054] For example, the external position parameters of the vehicle-mounted surround view camera may include w, h, and z; wherein w and h can be determined based on the length and width of the vehicle body (which can be denoted as H and W), and the installation position of the vehicle-mounted surround view camera on the vehicle. The height of the vehicle-mounted surround view camera can be determined by measuring its height.

[0055] In one example, determining the angular extrinsic parameters of the front / rear view camera based on the straight line equation of the lane lines in the distorted image, the lane line width constraints, and the mounting position parameters of the front / rear view camera may include:

[0056] Based on the installation position parameters of the front / rear view cameras and lane width constraints, determine the roll angle of the front / rear view cameras; and,

[0057] The coordinates of the vanishing point in the distorted image are determined based on the straight line equation of the lane lines in the distorted image.

[0058] Based on the roll angle of the forward / rear-view camera and the coordinates of the vanishing point, determine the pitch and yaw angles of the forward / rear-view camera.

[0059] For example, the intersection points between the lines can be obtained based on the equations of the straight lines on the sides of multiple lane lines in the distortion-free image of the forward-looking camera, and the coordinates of the vanishing point in the distortion-free image can be obtained by averaging the multiple intersection points.

[0060] For example, a schematic diagram of the vanishing point in the distortion-free image of a forward-looking camera can be shown as follows: Figure 2 As shown.

[0061] For example, in order to determine the vanishing point coordinates, the distorted image may include at least two lane lines.

[0062] For example, a schematic diagram of the straight lines on the sides of the two lane lines in the distortion-free image of the forward-looking camera can be found in [reference needed]. Figure 3 .like Figure 3 As shown, the intersection point of any two lines out of the four lines—the two sides of lane line 0 and the two sides of lane line 1—can be determined. Figure 3 The image contains four intersection points, and the average of these four intersection points is used to obtain the coordinates of the vanishing point in the distorted image.

[0063] In addition, considering that the width of lane lines is fixed in real-world scenarios, for example, the national standard lane line width is 15cm, meaning that different lane lines have the same width.

[0064] Therefore, when the lane lines in the distortion map are projected onto the vehicle world coordinate system, the width of different lane lines (which can be characterized by the distance between the two sides of the vehicle line) is consistent.

[0065] For example, consistent width of different lane lines can include different lane lines having the same width, or having tolerable deviations.

[0066] Accordingly, given the equation of the straight line on the side of the lane line in the distorted image, the roll angle (which can be denoted as roll) of the forward-looking camera can be determined based on the mounting position parameters of the forward-looking camera and the lane line width constraint.

[0067] For example, the value of roll can be optimized based on a numerical iterative algorithm, so that the width of different lane lines is consistent when the lane lines in the distortion map are projected onto the vehicle's world coordinate system.

[0068] For example, given the roll angle and vanishing point coordinates of the forward-looking camera, the pitch angle (which can be denoted as pitch) and yaw angle (which can be denoted as yaw) of the forward-looking camera can be determined based on the roll angle and vanishing point coordinates of the forward-looking camera. The specific implementation process can be explained in the following text with specific examples.

[0069] At this point, the extrinsic parameters of the forward-looking camera, including roll, pitch, and yaw, have been determined.

[0070] Step S120: Based on the angular extrinsic parameters of the front / rear view cameras, the installation position parameters of the front / rear view cameras, the installation position parameters of the side view cameras in the vehicle surround view cameras, and the geometric features of the lane lines in the images acquired by the side view cameras, the angular extrinsic parameters of the side view cameras are corrected to determine the angular extrinsic parameters of the side view cameras.

[0071] For example, the side-view camera in a vehicle surround view camera may include a vehicle surround view camera installed on the left side of the vehicle body, or a vehicle surround view camera installed on the right side of the vehicle body.

[0072] For example, when the angular extrinsic parameters of the front-view / rear-view cameras are determined in the manner described above, the angular extrinsic parameters of the side-view cameras can be corrected based on the angular extrinsic parameters of the front-view / rear-view cameras, the installation position parameters of the front-view / rear-view cameras, the installation position parameters of the side-view cameras in the vehicle surround-view cameras, and the geometric features of the lane lines in the images acquired by the side-view cameras, so as to determine the angular extrinsic parameters of the side-view cameras.

[0073] For example, the above-mentioned correction of the angular extrinsic parameters of the side-view camera may include, but is not limited to: when the (lane line) is projected onto the vehicle's world coordinate system, aligning the lane line in the image captured by the side-view camera with the lane line in the image captured by the front / rear-view camera, making the slope of the lane line in the image captured by the side-view camera 0 (i.e., the lane line is parallel to the vehicle body, allowing for tolerable errors), and making the two sides of the lane line parallel, etc.

[0074] For example, aligning lane lines in images captured by side-view cameras with lane lines in images captured by front / rear-view cameras can include: for the same lane line, if the center line of the lane line in the image captured by the side-view camera is projected onto the vehicle's world coordinate system, then the center line of the lane line in the image captured by the front / rear-view camera is projected onto the vehicle's world coordinate system, and the two are collinear (allowing for tolerable errors).

[0075] Step S130: Based on the installation position parameters and angular extrinsic parameters of each vehicle-mounted surround view camera, convert the images acquired by each vehicle-mounted surround view camera into BEV views, and optimize the angular extrinsic parameters of the side-view camera by aligning key texture feature points in the common viewing area of ​​the side-view camera and the front / rear-view camera to obtain the accurate angular extrinsic parameters of the side-view camera.

[0076] In this embodiment of the application, in order to achieve 360-degree surround view, the side-view camera usually shares a common viewing area with the front-view / rear-view camera, that is, the field of view coverage overlaps.

[0077] To determine more accurate angular extrinsic parameters for the side-view camera, the extrinsic parameters of the side-view camera can be updated based on key texture feature points in the shared viewing area of ​​the side-view camera and the front / rear-view camera.

[0078] For example, key texture feature points may include feature points obtained by extracting feature points (such as corner points) from key texture regions of the road surface.

[0079] For example, key textured areas on the roadside may include, but are not limited to, lane lines, road arrows, or other road traffic signs.

[0080] For example, the alignment of key texture feature points within the common field of view of adjacent vehicle surround view cameras in the BEV (Bird Eye View) view can include: in the BEV view of adjacent vehicle surround view cameras, two key texture feature points corresponding to the same physical location within the common field of view will coincide when projected onto the vehicle world coordinate system (allowing for tolerable errors).

[0081] For example, in the BEV view of the left-view camera and the BEV view of the front-view camera, two key texture feature points that correspond to the same physical location and are within the common viewing area (of the left-view camera and the front-view camera) will overlap when projected onto the vehicle world coordinate system.

[0082] It can be seen that, in Figure 1 In the illustrated method, images are acquired from the vehicle-mounted surround-view camera. For the front / rear-view cameras in the vehicle-mounted surround-view camera system, the angular extrinsic parameters of the front / rear-view cameras are determined based on the straight-line equations of lane lines in the distortion-free image, the width constraints of lane lines, and the installation position parameters of the front / rear-view cameras. Then, based on the angular extrinsic parameters of the front / rear-view cameras, the installation position parameters of the front / rear-view cameras, the installation position parameters of the side-view cameras in the vehicle-mounted surround-view camera system, and the geometric features of lane lines in the images acquired by the side-view cameras, the angular extrinsic parameters of the side-view cameras are corrected to determine the final angular extrinsic parameters of the side-view cameras. Furthermore, based on the installation position parameters and angular extrinsic parameters of each target camera, the images collected by each vehicle-mounted surround-view camera are converted into BEV views. By aligning key texture feature points in the shared field of view of the side-view camera and the front / rear-view camera, the angular extrinsic parameters of the side-view camera are optimized to obtain accurate angular extrinsic parameters of the side-view camera. This achieves camera extrinsic parameter calibration in natural scenes without the need to deploy calibration sites, reducing labor and site costs and improving the practicality of the solution. In addition, based on the lane line calibration solution, the extrinsic parameter angle of the side-view camera is optimized, making the final calibrated extrinsic parameters more accurate.

[0083] In some embodiments, the above-mentioned correction of the angular extrinsic parameters of the side-view camera based on the angular extrinsic parameters of the front / rear-view camera, the installation position parameters of the front / rear-view camera, the installation position parameters of the side-view camera in the vehicle surround view camera, and the geometric features of lane lines in the image acquired by the side-view camera, to determine the angular extrinsic parameters of the side-view camera, may include:

[0084] Adjusting the pitch angle of the side-view camera will align the lane lines in the image captured by the side-view camera with the lane lines in the image captured by the front / rear-view camera when projected onto the vehicle's world coordinate system. This pitch angle will be determined as the initial target pitch angle of the side-view camera.

[0085] Adjusting the roll angle of the side-view camera will result in a roll angle with a lane line slope of 0 in the image captured by the side-view camera when projected onto the vehicle's world coordinate system. This roll angle will be determined as the initial target roll angle of the side-view camera.

[0086] Adjusting the yaw angle of the side-view camera will make the yaw angle of the two sides of the lane line in the image captured by the side-view camera parallel when projected onto the vehicle's world coordinate system, and this angle will be determined as the initial target yaw angle of the side-view camera.

[0087] For example, lane lines in the images captured by the vehicle surround view camera can be projected onto the vehicle's world coordinate system based on the installation position parameters of the vehicle surround view camera and the angular extrinsic parameters of the vehicle surround view camera.

[0088] For example, during the process of geometric feature correction of lane lines in images acquired by side-view cameras, different parameters (pitch angle, roll angle, or yaw angle) in the angular extrinsic parameters of the side-view camera can be adjusted to ensure that the corresponding geometric features of lane lines in images acquired by side-view cameras meet the requirements.

[0089] For example, the initial target pitch angle of the side-view camera can be determined by adjusting the pitch angle of the side-view camera so that the lane lines in the image captured by the side-view camera are aligned with the lane lines in the image captured by the front / rear-view camera when projected onto the vehicle's world coordinate system.

[0090] For example, lane alignment may include alignment of the center lines of lane lines (such as collinearity).

[0091] The initial target roll angle of the side-view camera can be determined by adjusting the roll angle of the side-view camera so that the slope of the lane line in the image acquired by the side-view camera is 0 when projected onto the vehicle's world coordinate system (allowing for tolerable errors).

[0092] The initial target yaw angle of the side-view camera can be determined by adjusting the yaw angle of the side-view camera, which makes the two sides of the lane line in the image captured by the side-view camera parallel when projected onto the vehicle's world coordinate system.

[0093] In one example, for any angular extrinsic parameter of a side-view camera, that angular extrinsic parameter can be adjusted in the following way:

[0094] Determine the adjustment range of the external parameters for this angle;

[0095] Within this adjustment range, the external parameters of the angle are adjusted using the dichotomy method.

[0096] For example, in order to improve the efficiency of geometric feature correction of lane lines in images acquired by side-view cameras, that is, to improve the efficiency of coarse calibration of the angle extrinsic parameters of side-view cameras, in the process of geometric feature correction of lane lines in images acquired by side-view cameras by adjusting the angle extrinsic parameters of side-view cameras in the manner described above, the adjustment range of each angle extrinsic parameter can be determined separately, and the angle extrinsic parameter can be adjusted using the bisection method within the determined adjustment range.

[0097] For example, the adjustment range of the angular extrinsic parameter can be determined based on the slope of the lane lines in the image captured by the side-view camera when projected onto the vehicle's world coordinate system.

[0098] In some embodiments, the optimization of the angular extrinsic parameters of the side-view camera by aligning key texture feature points in the common viewing area of ​​the side-view camera and the front / rear-view camera to obtain the accurate angular extrinsic parameters of the side-view camera may include:

[0099] Corner point extraction is performed on the key texture regions of the road surface within the shared field of view of the side-view camera and the front / rear-view camera to determine the key texture feature points within the shared field of view of the side-view camera and the front / rear-view camera;

[0100] Feature point matching is performed on key texture feature points in images captured by the side-view camera and key texture feature points in images captured by the front-view / rear-view camera to determine key texture feature matching point pairs;

[0101] Based on the images captured by the front / rear view cameras, the key texture feature points in the key texture feature matching point pairs are aligned in the vehicle body world coordinate system to determine the precise target roll angle of the side view camera.

[0102] For example, after determining the angular extrinsic parameters of the front-view / rear-view cameras and coarsely calibrating the angular extrinsic parameters of the side-view cameras, the images captured by the front-view / rear-view cameras and the side-view cameras can be converted into BEV views based on the angular extrinsic parameters of the front-view / rear-view cameras and the current angular extrinsic parameters of the side-view cameras, respectively.

[0103] Take the front-view camera and the left-view camera as examples.

[0104] Based on the BEV view of the front-view camera and the BEV view of the left-view camera, corner points can be extracted from the key texture areas of the road surface within the shared view area of ​​the front-view camera and the left-view camera to determine the key texture feature points within the shared view area of ​​the front-view camera and the left-view camera.

[0105] For example, corner extraction algorithms may include, but are not limited to, SIFT (Scale-Invariant Feature Transform) algorithms or deep learning methods.

[0106] For the key texture feature points extracted from the shared viewing area of ​​the front-view camera and the left-view camera, key texture feature matching point pairs can be determined by feature point matching.

[0107] For example, feature point matching methods may include, but are not limited to: ORB matching method (a feature detection and description algorithm) or deep learning methods such as superpoint and superglue.

[0108] Furthermore, based on the image captured by the forward-looking camera, the key texture feature points in the key texture feature matching point pair can be aligned in the vehicle body world coordinate system to determine the precise target roll angle of the left-looking camera.

[0109] For example, aligning key texture feature points in a key texture feature matching point pair may include: when projected onto the vehicle body world coordinate system, the two key texture feature points in the key texture feature matching point pair coincide (allowing for tolerable errors).

[0110] It should be noted that, in order to improve the accuracy of the angular extrinsic parameters of the left-view camera, when optimizing the angular extrinsic parameters of the left-view camera in the manner described above, the key texture feature points in the shared viewing area of ​​the left-view camera and the rear-view camera can also be taken into account and processed.

[0111] Furthermore, the optimization of the extrinsic parameters of the right-view camera can be achieved similarly.

[0112] In one example, using the images captured by the front / rear view cameras as a reference, and aligning the key texture feature points in the key texture feature matching point pairs in the vehicle body world coordinate system to determine the precise target roll angle of the side view camera, this can include:

[0113] During the alignment of key texture feature points in the key texture feature matching point pair, the pitch angle and yaw angle in the angular extrinsic parameters of the side-view camera are fixed, and the roll angle in the angular extrinsic parameters of the side-view camera is optimized by minimizing the reprojection error of the key texture feature matching point, so as to determine the precise target roll angle of the side-view camera.

[0114] For example, during the alignment of key texture feature points in a key texture feature matching point pair, the key texture feature matching points can be projected onto the world coordinate system based on the angular extrinsic parameters of the side-view camera, and the Euclidean distance between the key texture feature points in the key texture feature matching point pair can be calculated. Based on the Euclidean distance between the key texture feature points in the key texture feature matching pair, the reprojection error of the key texture feature matching point pair can be determined.

[0115] In the above process, the pitch angle and yaw angle in the angular extrinsic parameters of the side-view camera can be fixed, and the roll angle of the side-view camera can be adjusted to minimize the reprojection error of the key texture feature matching points. The roll angle that minimizes the reprojection error of the key texture feature matching points is determined as the precise target roll angle of the side-view camera.

[0116] In one example, after determining the precise target roll angle of the side-view camera as described above, it may also include:

[0117] The precise target roll angle of the fixed side-view camera is determined by optimizing the pitch and yaw angles in the angular extrinsic parameters of the side-view camera based on the lane line geometry.

[0118] For example, after determining the precise target roll angle of the side-view camera in the manner described above, the pitch angle and yaw angle of the side-view camera can be optimized by performing geometric feature correction on the lane lines in the image acquired by the side-view camera while keeping the precise target roll angle of the side-view camera fixed.

[0119] For example, with a fixed precise target roll angle for the side-view camera, the initial target pitch angle of the side-view camera can be adjusted to determine the pitch angle at which the lane lines in the image captured by the side-view camera are aligned with the lane lines in the image captured by the front / rear-view camera when projected onto the vehicle's world coordinate system.

[0120] For example, with a fixed precise target roll angle for the side-view camera, the precise target yaw angle of the side-view camera can be determined by adjusting the initial target yaw angle of the side-view camera, so that the two sides of the lane line in the image captured by the side-view camera are parallel when projected onto the vehicle's world coordinate system.

[0121] To enable those skilled in the art to better understand the technical solutions provided in the embodiments of this application, the technical solutions provided in the embodiments of this application are described below with reference to specific examples.

[0122] This embodiment provides a method for extrinsic parameter calibration of a vehicle-mounted surround-view camera. The extrinsic parameter calibration is performed using lane line information on the road while the vehicle is traveling, achieving extrinsic parameter calibration based on a natural scene. A schematic diagram of the calibration environment can be shown below. Figure 4 As shown. Compared to extrinsic parameter calibration schemes based on calibration cloths, this approach can reduce deployment costs and improve the practicality of the solution.

[0123] For example, in the implementation scheme of extrinsic parameter calibration of vehicle surround view camera, initial extrinsic parameter information is required, that is, the camera installation location information (i.e., installation location parameters). For example, the installation location parameters of vehicle surround view camera can be determined by measurement.

[0124] Please see Figure 5In this embodiment, the vehicle-mounted surround-view camera extrinsic parameter calibration system may include: an image acquisition unit, a data transmission unit, a data processing unit, a vehicle-mounted surround-view panoramic image generation unit, and an image display unit. Wherein:

[0125] The image acquisition unit may include vehicle-mounted cameras (i.e., vehicle-mounted surround view cameras) distributed in four or more locations on the front, rear, left, and right sides of the vehicle body, and all vehicle-mounted cameras have completed internal parameter calibration.

[0126] For example, taking a vehicle with four surround-view cameras as an example, a schematic diagram of the images captured by the vehicle surround-view cameras can be found here. Figure 6 .

[0127] in, Figure 6 From left to right, the images are: the front view captured by the front camera, the left view captured by the left camera, the right view captured by the right camera, and the rear view captured by the rear camera.

[0128] The data transmission unit acquires image data and camera intrinsic parameter information acquired by the image acquisition unit and transmits them to the data processing unit.

[0129] The data processing unit is a key processing unit for calibrating the external parameters of the vehicle surround view system. A flowchart of the processing procedure can be found here. Figure 7 .

[0130] The vehicle surround view panoramic image generation unit generates lookup tables between the vehicle surround view panoramic image and the four views (front, left, right, and rear) based on the extrinsic parameters calculated by the data processing unit and the camera intrinsic parameters. It also generates fusion weight tables between the front and left views, the left and rear views, the rear and right views, and the right and front views. Based on the original images from the four view cameras and the lookup tables, the vehicle surround view panoramic image is obtained.

[0131] The image display unit presents a seamless panoramic image to the user after the extrinsic parameters have been calibrated.

[0132] The following describes the implementation process of extrinsic parameter calibration for vehicle-mounted surround-view cameras.

[0133] like Figure 7 As shown, the process for calibrating the extrinsic parameters of an onboard surround-view camera may include the following steps:

[0134] S1. Calibration of external parameters for front and rear camera angles.

[0135] S2, coarse calibration of external parameters for left and right camera angles.

[0136] S3: Precise optimization of external parameters for left and right camera angles.

[0137] It should be noted that in this embodiment, the initial position parameters of the four cameras can be calculated based on the known vehicle body length and width (H and W). Next, camera extrinsic parameter calibration is performed according to the data processing flow. A detailed extrinsic parameter calibration flowchart is shown below. Figure 8 As shown.

[0138] In one example, the initial position parameters can be as shown in Table 1 (taking the vehicle surround view cameras as being installed at the middle positions of the front / rear / left / right sides as an example):

[0139] Table 1

[0140] camera h w z Front camera -H / 2 0 Actual measured height Left camera 0 -W / 2 Actual measured height Right camera 0 W / 2 Actual measured height Rear camera H / 2 0 Actual measured height

[0141] S1. Calibration of external parameters for front and rear camera angles.

[0142] S1.1. Based on the principle of equal lane width and vanishing point, determine the extrinsic angle parameters of the forward-looking camera.

[0143] S1.2. Based on the principle of equal lane width and vanishing point, determine the extrinsic angle parameters of the rear-view camera.

[0144] Taking the calibration of the extrinsic parameters of the front-view camera as an example, the calibration of the extrinsic parameters of the rear-view camera can be obtained in the same way.

[0145] For example, distortion correction can be performed on the front view based on the input front view and the intrinsic parameters of the front camera to obtain a distortion-free image. Edge point extraction and line fitting of straight lines are performed in the distortion-free image to obtain the equations of the straight lines in the distortion-free image. A schematic diagram can be shown below. Figure 3 As shown.

[0146] For example, edge point extraction methods can use the Canny operator, Sobel operator, Laplacian operator, etc., while line fitting can use the least squares method, gradient descent method, etc.

[0147] It should be noted that deep learning networks can also be used to extract lines, and this application does not limit this implementation.

[0148] The intersection points between the fitted lines can be obtained, and the vanishing point is obtained by averaging multiple intersection points. A schematic diagram of this can be shown below. Figure 2 and Figure 3 As shown.

[0149] For example, a single view (front view or rear view) contains at least two lane lines, that is, two pairs of straight line equations.

[0150] The extrinsic parameters of the forward-looking camera can be calibrated based on the vanishing point and the straight lane line fitted in the distortion-free image.

[0151] The following describes the procedure for calibrating the extrinsic parameters of the forward-looking camera.

[0152] First Figure 9 The coordinate system shown illustrates the transformation between the vehicle body's world coordinate system Ow-XwYwZw and the camera coordinate system Oc-XcYcZ.

[0153] like Figure 9 As shown, 0c-XcYcZc is the camera coordinate system (front or rear camera), 0w-XwYwZw is the vehicle world coordinate system, and 0cw-XcwYcwZcw is also a world coordinate system (auxiliary world coordinate system), which is a coordinate system with the intersection of the camera optical axis and the ground as the origin.

[0154] The vehicle's world coordinate system Ow-XwYwZw can be converted to the camera coordinate system Oc-XcYcZ in the following way:

[0155] 1) Rotate around Zw by yaw+π / 2 (the angle is positive and rotates clockwise) so that the direction of Ow-XwYwZw is consistent with the coordinate system Ocw-XcwYcwZcw;

[0156] 2) Rotate -pitch along the Xw axis to align Yw with the Zc direction;

[0157] 3) Rotate -roll along the Yw axis so that Xw is aligned with the Xc direction and Zw is aligned with the -Yc direction;

[0158] 4) Rotate -90 degrees along the Xw axis, so that Zw and Zc are in the same direction, and Yw and Yc are in the same direction;

[0159] 5) Translate the coordinate system by T so that the coordinate system Ow-XwYwZw completely coincides with the camera coordinate system Oc-XcYcZc.

[0160] For example, this can be expressed as a formula as follows:

[0161]

[0162]

[0163] Among them, R 33 Let T represent a 3x3 rotation matrix, and T represent a 3x1 translation vector (also called a translation matrix) (or simply T). 31 ), where pitch, yaw, and roll represent the camera's pitch, yaw, and roll angles, respectively, and (cam_x, cam_y, cam_z) represents the camera's mounting position in the vehicle's world coordinate system.

[0164] The transformation relationship between points on the distortion map and the world coordinate system is shown in equation (3). Therefore, according to equations (1)-(3), and Z w=0, and the homography matrix H between the vehicle body world coordinate system and the camera distortion correction map can be derived, as shown in equation (4).

[0165]

[0166] Where fx, fy, cx, and cy represent the camera intrinsic parameters, focal length and principal point, and r represents the rotation matrix R. 33 The value in the vector T, t represents the translation vector. 31 The values ​​in the image, (u, v), represent pixel coordinates in the image, (X... w Y w ) represents the coordinates in the vehicle's world coordinate system.

[0167] Suppose that there are two lane lines in the distortion-free image of the front view, with the sides corresponding to lines l1 to l4 respectively.

[0168] In the world coordinate system O of the vehicle body, the straight lines l1-l4 are... w Below and X w When the coordinate axes are parallel, the line is parallel to the Y-axis. cw If the included angle between the axes is the yaw angle, then a point on the line can be represented as: (Y cw *tan(yaw), Y cw The vanishing point coordinates on the distortion-free map are when Y = 0. cw The values ​​of u0 and v0 when ->∞. Combining equations (1)-(4), taking the limit, we get:

[0169]

[0170] Based on formula (5), given the roll angle (roll), vanishing point coordinates (u0, v0), and camera intrinsic parameters (fx, fy, cx, cy), the camera's pitch and yaw angles can be calculated as follows:

[0171]

[0172] Since the vanishing point is the intersection of lines l1-l4 in the front view, and the vanishing point is known, the camera's rotation angle is only related to the roll angle. Therefore, we only need to solve for the unknown variable roll to complete the calibration of the extrinsic parameters (pitch, yaw, roll). Based on the above formula derivation, we can obtain the relationship between roll and pitch, yaw.

[0173] For example, the value of roll can be optimized based on a numerical iterative algorithm, so that... Figure 7 The widths of straight lines 0 and 1 in the vehicle's world coordinate system are equal (the widths of standard lane lines are equal).

[0174] The pitch and yaw are calculated according to equation (6), thus completing the calibration of the external parameters of the forward-looking camera angle.

[0175] S2, coarse calibration of external parameters for left and right camera angles.

[0176] S2.1. Obtain the coarse angular extrinsic parameters of the left-view camera by correcting the lane line geometry features.

[0177] S2.2. Obtain the coarse angular extrinsic parameters of the right-view camera by correcting the lane line geometric features.

[0178] Taking the coarse calibration of the extrinsic parameters of the left-view camera as an example, the coarse calibration of the extrinsic parameters of the right-view camera can be obtained in the same way.

[0179] For example, based on the center line of the lane line extracted from the left view, adjust the pitch so that when projected into the vehicle world coordinate system, the center line of the same lane line in the left view is aligned with the center line of the same lane line in the front view (or rear view); adjust the roll so that when projected into the vehicle world coordinate system, the lane line in the left view is vertical (slope is 0); adjust the yaw so that when projected into the vehicle world coordinate system, the two sides of the lane line in the left view are parallel and different lane lines are parallel.

[0180] For example, for any angular extrinsic parameter, the adjustment method may include obtaining the adjustment range of the angular extrinsic parameter, and adjusting the angular extrinsic parameter within the adjustment range using a bisection method.

[0181] S3: Precise optimization of external parameters for left and right camera angles.

[0182] Taking the precise optimization of the extrinsic parameters of the left-view camera as an example, the precise optimization of the extrinsic parameters of the right-view camera can be obtained in the same way.

[0183] For example, given the coarse angular extrinsic parameters of the left-view camera, the left view, front view, and rear view can be converted into BEV views, respectively, and these views can be defined within the shared viewing areas of the left-view and front-view cameras, and the shared viewing areas of the left-view and rear-view cameras, for example... Figure 10 The area circled in the front view and the area circled on the right side of the left view are the same viewing area; the area circled in the rear view and the area circled on the left side of the left view are the same viewing area. Figure 10 The view from left to right is the front view, left view and back view. The key feature points (key texture feature points) of the common viewing area are obtained by the corner point extraction algorithm, and then the matching point pairs (key texture feature matching point pairs) are obtained by the corner point matching algorithm.

[0184] Using the front view (or rear view) as a reference, the coordinates of feature points are transformed from the BEV view coordinate system to the vehicle body world coordinate system. The feature points in the left view are aligned with the corresponding points in the front and rear views in the world coordinate system using a bisection method.

[0185] by Figure 11 Taking the scenario shown as an example, assume the black dots are the reference lane line feature points in the front / rear views, and the white dots are the lane line feature points to be registered in the left / right views. During the roll correction process, the above lane line feature points are required to be aligned longitudinally with respect to the vehicle (corresponding to the lateral alignment in the above figure). Then, the sum of the coordinate differences between the reference point and the point to be registered has two possibilities: ① The x-coordinates of the point to be registered are all greater than the reference point, as shown in the above figure. In this case, the sum of the x-coordinate differences is greater than 0 (y1+y2>0, i.e., the black dot is to the left of the white dot); ② The x-coordinates of the point to be registered are all less than the reference point. In this case, the sum of the x-coordinate differences is less than 0 (y1+y2<0, i.e., the black dot is to the right of the white dot). By finding the iteration intervals roll left and roll right by the x-coordinate sums being of opposite signs, assuming roll left corresponds to (y1+y2>0) and roll right corresponds to (y1+y2<0), the binary iteration interval of the roll angle is determined. Within this interval, the roll angle is iterated to minimize the absolute value of y1+y2.

[0186] For example, such as Figure 12 The corner points connected by the white line are the matching point pairs between the rear view and the left view.

[0187] In this process, pitch and yaw can be fixed first, and the roll of the left-view camera can be optimized by minimizing the reprojection error of the matching point. Then, roll can be fixed, and pitch and yaw can be optimized based on the lane line geometry.

[0188] For example, pitch can be optimized by aligning the lane lines in the left view with the center line of the lane lines in the front (or rear) view when projected onto the vehicle's world coordinate system; yaw can be optimized by making the lane lines in the left view parallel.

[0189] The above method can complete the process of optimizing the extrinsic parameters of the left and right views. Compared with the coarse angle extrinsic parameters of S2, the optimized extrinsic parameters will have a smaller misalignment in the final stitched image, thus providing users with the best panoramic stitching effect.

[0190] Figure 13 This compares the stitching effect of the extrinsic angle parameters obtained from S2 and S3 on a chessboard-like surface. For example... Figure 13 As shown, compared to Figure 13 The splicing effect diagram on the left side of the middle. Figure 13 In the splicing effect diagram on the right, the misalignment in the upper left area is reduced.

[0191] The method provided in this application has been described above. The apparatus provided in this application is described below:

[0192] Please see Figure 14 This is a schematic diagram of the structure of a camera extrinsic parameter calibration device provided in an embodiment of this application, as shown below. Figure 14 As shown, the camera extrinsic calibration device may include:

[0193] The acquisition unit is used to acquire images captured by the vehicle-mounted surround view camera;

[0194] The first determining unit is configured to determine the angular extrinsic parameters of the front / rear view cameras in the vehicle surround view camera based on the straight line equations of lane lines in the distorted image, the width constraints of lane lines, and the installation position parameters of the front / rear view cameras; wherein, the distorted image is obtained by distorting the image acquired by the front / rear view cameras based on the intrinsic parameters of the front / rear view cameras, and the lane line width constraints include the consistency of the width of different lane lines when the lane lines are projected onto the vehicle's world coordinate system;

[0195] The second determining unit is used to correct the angle extrinsic parameters of the side-view camera based on the angle extrinsic parameters of the front-view / rear-view camera, the installation position parameters of the front-view / rear-view camera, the installation position parameters of the side-view camera in the vehicle surround view camera, and the geometric features of the lane lines in the image acquired by the side-view camera, and to determine the angle extrinsic parameters of the side-view camera.

[0196] The optimization unit is used to convert the images acquired by each vehicle-mounted surround view camera into BEV views based on the installation position parameters and angular extrinsic parameters of each target camera, and to optimize the angular extrinsic parameters of the side view camera by aligning key texture feature points in the common viewing area of ​​the side view camera and the front / rear view camera, so as to obtain the accurate angular extrinsic parameters of the side view camera.

[0197] For example, the specific processing flow of the acquisition unit, the first determination unit, the second determination unit, and the optimization unit for camera extrinsic parameter calibration can be found in the relevant descriptions in the above embodiments, and will not be repeated here in the embodiments of this application.

[0198] This application provides an electronic device including a processor and a memory, wherein the memory stores machine-executable instructions that can be executed by the processor, and the processor executes the machine-executable instructions to implement the camera extrinsic calibration method described above.

[0199] Please see Figure 15This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device may include a processor 1501 and a memory 1502 storing machine-executable instructions. The processor 1501 and the memory 1502 can communicate via a system bus 1503. Furthermore, by reading and executing the machine-executable instructions in the memory 1502 corresponding to the camera extrinsic calibration logic, the processor 1501 can execute the camera extrinsic calibration method described above.

[0200] The memory 1502 mentioned in this document can be any electronic, magnetic, optical, or other physical storage device that can contain or store information such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0201] In some embodiments, a machine-readable storage medium, such as Figure 15 The memory 1502 in the device stores machine-executable instructions, which, when executed by a processor, implement the camera extrinsic calibration method described above. For example, the storage medium may be ROM, RAM, CD-ROM, magnetic tape, floppy disk, or optical data storage device.

Claims

1. A method for calibrating camera extrinsic parameters, characterized in that, include: Acquire images captured by the vehicle-mounted surround view camera; For the front-view / rear-view cameras in the vehicle surround view camera system, the angular extrinsic parameters of the front-view / rear-view cameras are determined based on the straight line equations of the lane lines in the distortion-free image, the lane line width constraints, and the installation position parameters of the front-view / rear-view cameras. The distortion-free image is obtained by distorting the images acquired by the front-view / rear-view cameras based on their intrinsic parameters. The lane line width constraints include ensuring that the widths of different lane lines are consistent when projected onto the vehicle's world coordinate system. Based on the angular extrinsic parameters of the front / rear view camera, the installation position parameters of the front / rear view camera, the installation position parameters of the side view camera in the vehicle surround view camera, and the geometric features of the lane lines in the image acquired by the side view camera, the angular extrinsic parameters of the side view camera are corrected to determine the angular extrinsic parameters of the side view camera. Based on the installation position parameters and angular extrinsic parameters of each target camera, the images acquired by each vehicle-mounted surround view camera are converted into a bird's-eye view of the BEV. The angular extrinsic parameters of the side-view camera are optimized by aligning key texture feature points in the common viewing area of ​​the side-view camera and the front / rear-view camera to obtain the accurate angular extrinsic parameters of the side-view camera. The step of correcting the extrinsic angle parameters of the side-view camera based on the angular extrinsic parameters of the front / rear-view camera, the installation position parameters of the front / rear-view camera, the installation position parameters of the side-view camera in the vehicle surround view camera, and the geometric features of lane lines in the image acquired by the side-view camera, to determine the extrinsic angle parameters of the side-view camera, includes: Adjusting the pitch angle of the side-view camera will align the lane lines in the image captured by the side-view camera with the lane lines in the image captured by the front / rear-view camera when projected onto the vehicle's world coordinate system. This pitch angle will be determined as the initial target pitch angle of the side-view camera. Adjusting the roll angle of the side-view camera will result in a roll angle with a lane line slope of 0 in the image captured by the side-view camera when projected onto the vehicle's world coordinate system. This roll angle will be determined as the initial target roll angle of the side-view camera. Adjusting the yaw angle of the side-view camera will make the yaw angle at which the two sides of the lane line in the image captured by the side-view camera are parallel when projected onto the vehicle's world coordinate system, and this angle will be determined as the initial target yaw angle of the side-view camera. For any angular extrinsic parameter of the side-view camera, the angular extrinsic parameter of the side-view camera is adjusted in the following manner: Determine the adjustment range of the external parameters for this angle; Within this adjustment range, the external parameters of the angle are adjusted using the dichotomy method.

2. The method according to claim 1, characterized in that, The determination of the angular extrinsic parameters of the front / rear view cameras based on the straight line equations of the lane lines in the distortion-free image, the lane line width constraints, and the installation position parameters of the front / rear view cameras includes: Based on the installation position parameters of the front / rear view cameras and the lane width constraints, the roll angle of the front / rear view cameras is determined; and, Based on the straight line equation of the lane lines in the distorted image, determine the coordinates of the vanishing point in the distorted image; Based on the roll angle of the forward / rear-view camera and the coordinates of the vanishing point, the pitch angle and yaw angle of the forward / rear-view camera are determined.

3. The method according to claim 1, characterized in that, The method of optimizing the angular extrinsic parameters of the side-view camera by aligning key texture feature points in the shared field of view of the side-view camera and the front / rear-view camera to obtain the precise angular extrinsic parameters of the side-view camera includes: Corner point extraction is performed on the key texture regions of the road surface within the shared field of view of the side-view camera and the front / rear-view camera to determine the key texture feature points within the shared field of view of the side-view camera and the front / rear-view camera. Feature point matching is performed on the key texture feature points in the image acquired by the side-view camera and the key texture feature points in the image acquired by the front-view / rear-view camera to determine key texture feature matching point pairs; Using the images captured by the front / rear view cameras as a reference, the key texture feature points in the key texture feature matching point pairs are aligned in the vehicle body world coordinate system to determine the precise target roll angle of the side view camera.

4. The method according to claim 3, characterized in that, The step of aligning key texture feature points in key texture feature matching point pairs in the vehicle body world coordinate system, based on the images captured by the front / rear view cameras, to determine the precise target roll angle of the side view camera includes: During the alignment of key texture feature points in the key texture feature matching point pair, the pitch angle and yaw angle in the angular extrinsic parameters of the side-view camera are fixed, and the roll angle in the angular extrinsic parameters of the side-view camera is optimized by minimizing the reprojection error of the key texture feature matching points, so as to determine the precise target roll angle of the side-view camera.

5. The method according to claim 3, characterized in that, After determining the precise target roll angle of the side-view camera, the method further includes: The precise target roll angle of the side-view camera is fixed, and the pitch angle and yaw angle in the angular extrinsic parameters of the side-view camera are optimized based on the lane line geometry to determine the precise target pitch angle and precise target yaw angle of the side-view camera.

6. A camera extrinsic parameter calibration device, characterized in that, include: The acquisition unit is used to acquire images captured by the vehicle-mounted surround view camera; The first determining unit is configured to determine the angular extrinsic parameters of the front / rear view cameras in the vehicle surround view camera based on the straight line equations of lane lines in the distorted image, the width constraints of lane lines, and the installation position parameters of the front / rear view cameras; wherein, the distorted image is obtained by distorting the image acquired by the front / rear view cameras based on the intrinsic parameters of the front / rear view cameras, and the lane line width constraints include the consistency of the width of different lane lines when the lane lines are projected onto the vehicle's world coordinate system; The second determining unit is used to perform geometric feature correction on the lane lines in the image acquired by the side-view camera based on the angular extrinsic parameters of the front-view / rear-view camera, the installation position parameters of the front-view / rear-view camera, and the installation position parameters of the side-view camera in the vehicle surround view camera, and to determine the angular extrinsic parameters of the side-view camera. The optimization unit is used to convert the images acquired by each vehicle surround view camera into a bird's-eye view of the BEV based on the installation position parameters and angular extrinsic parameters of each target camera, and to optimize the angular extrinsic parameters of the side view camera by aligning key texture feature points in the common viewing area of ​​the side view camera and the front / rear view camera, so as to obtain the accurate angular extrinsic parameters of the side view camera. Specifically, the second determining unit is used to adjust the pitch angle of the side-view camera, determining the initial target pitch angle of the side-view camera as the pitch angle that aligns the lane lines in the image captured by the side-view camera with those in the image captured by the front / rear-view camera when projected onto the vehicle's world coordinate system; adjusting the roll angle of the side-view camera, determining the initial target roll angle of the side-view camera as the roll angle that makes the slope of the lane lines in the image captured by the side-view camera zero when projected onto the vehicle's world coordinate system; and adjusting the yaw angle of the side-view camera, determining the initial target yaw angle of the side-view camera as the yaw angle that makes the two sides of the lane lines in the image captured by the side-view camera parallel when projected onto the vehicle's world coordinate system. Specifically, for any angular extrinsic parameter of the side-view camera, the angular extrinsic parameter of the side-view camera is adjusted in the following manner: Determine the adjustment range of the external parameters for this angle; Within this adjustment range, the external parameters of the angle are adjusted using the dichotomy method.

7. An electronic device, characterized in that, The method includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor, the processor executing the machine-executable instructions to implement the method as described in any one of claims 1-5.

8. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-executable instructions, which, when executed by a processor, implement the method as described in any one of claims 1-5.

Images