An external parameter calibration method, device and equipment

Abstract

The application provides an external parameter calibration method, device and equipment. The method comprises the following steps: obtaining four initial image straight lines and a reference image straight line based on a target image, the reference image straight line corresponding to four initial intersection coordinates of the four initial image straight lines, and the four initial image straight lines corresponding to a reference intersection coordinate; iteratively optimizing a roll angle based on four physical straight lines corresponding to the four initial image straight lines in a vehicle body world coordinate system, to obtain a target roll angle; determining a target pitch angle and a target yaw angle based on the reference intersection coordinate and the target roll angle; determining a target translation parameter based on the target roll angle, the target pitch angle, the target yaw angle, the four initial intersection coordinates and physical coordinates corresponding to the four initial intersection coordinates in the vehicle body world coordinate system; and calibrating a camera external parameter for a surround-view camera, wherein the camera external parameter comprises the target roll angle, the target pitch angle, the target yaw angle and the target translation parameter. Through the application, the camera external parameter can be effectively calibrated.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an external parameter calibration method, device and equipment. BACKGROUND

[0002] With the continuous development of image technology, vehicle-mounted panoramic systems have also been widely applied. A vehicle-mounted panoramic system includes multiple cameras installed on a vehicle, such as a camera located at the front of the vehicle, a camera located at the rear of the vehicle, a camera located at the left side of the vehicle, and a camera located at the right side of the vehicle. During the driving of the vehicle, these cameras capture images in real time, and based on multiple images captured by the multiple cameras, the vehicle-mounted panoramic system can stitch the multiple images into a panoramic image, i.e., a 360-degree surround view image. The vehicle-mounted panoramic system can provide the driver with the situation around the vehicle based on the panoramic image, and provide visual assistance for the driver's operation.

[0003] In order to enable the vehicle-mounted panoramic system to generate a panoramic image, the camera extrinsic parameters of the cameras installed on the vehicle need to be calibrated before the vehicle is shipped. For example, in order to determine the mutual relationship between the three-dimensional geometric position of a point on the surface of a space object and the corresponding point in the image, a geometric model of imaging needs to be established, and the parameters of the geometric model are the camera extrinsic parameters of the camera. The process of solving the camera extrinsic parameters of the camera is called calibration of the camera extrinsic parameters, and the calibration of the camera extrinsic parameters is a very critical link.

[0004] However, there is no effective way to calibrate the camera extrinsic parameters in the related art, and the camera extrinsic parameters cannot be accurately calibrated, resulting in poor imaging effect of the panoramic image and poor user experience. SUMMARY

[0005] The present application provides an external parameter calibration method, a vehicle to be calibrated includes a surround view camera, a calibration cloth exists around the vehicle to be calibrated, and the calibration cloth includes 4 parallel lines and a perpendicular line of the 4 parallel lines, comprising:

[0006] Based on the acquired target image, 4 initial image straight lines corresponding to the 4 parallel lines and a reference image straight line corresponding to the perpendicular line are obtained; wherein the reference image straight line corresponds to 4 initial intersection point coordinates of the 4 initial image straight lines, and the 4 initial image straight lines correspond to reference intersection point coordinates;

[0007] Based on 4 physical straight lines corresponding to the 4 initial image straight lines in the vehicle body world coordinate system, the roll angle of the surround view camera is iteratively optimized to obtain a target roll angle; based on the reference intersection point coordinates and the target roll angle, a target pitch angle and a target yaw angle of the surround view camera are determined;

[0008] determine a target roll angle, a target pitch angle, and a target yaw angle of the surround-view camera based on the target translation parameter and the target rotation parameter;

[0009] calibrate camera extrinsic parameters for the surround-view camera, wherein the camera extrinsic parameters include the target roll angle, the target pitch angle, the target yaw angle, and the target translation parameter.

[0010] The application provides an external parameter calibration device, a vehicle to be calibrated includes a surround-view camera, a calibration cloth exists around the vehicle to be calibrated, and the calibration cloth includes four parallel lines and four perpendicular lines of the four parallel lines, and the device includes:

[0011] An acquisition module is configured to acquire four initial image straight lines corresponding to the four parallel lines and a reference image straight line corresponding to the perpendicular line based on a target image acquired, wherein the reference image straight line corresponds to four initial intersection coordinates of the four initial image straight lines, and the four initial image straight lines correspond to reference intersection coordinates;

[0012] A processing module is configured to perform iterative optimization on a roll angle of the surround-view camera based on four physical straight lines corresponding to the four initial image straight lines in a vehicle body world coordinate system, to obtain a target roll angle.

[0013] A determination module is configured to determine a target pitch angle and a target yaw angle of the surround-view camera based on the reference intersection coordinates and the target roll angle, and determine a target translation parameter based on the target roll angle, the target pitch angle, the target yaw angle, the four initial intersection coordinates, and physical coordinates corresponding to the four initial intersection coordinates in the vehicle body world coordinate system, wherein the target translation parameter represents an installation position of the surround-view camera in the vehicle body world coordinate system.

[0014] A calibration module is configured to calibrate camera extrinsic parameters for the surround-view camera, wherein the camera extrinsic parameters include the target roll angle, the target pitch angle, the target yaw angle, and the target translation parameter.

[0015] The application provides an electronic device, which includes a processor and a machine readable storage medium, the machine readable storage medium stores machine executable instructions that can be executed by the processor, and the processor is configured to execute the machine executable instructions to implement the external parameter calibration method of the above examples.

[0016] From the above technical solutions, in the embodiment of the present application, the calibration cloth is deployed around the vehicle to be calibrated (such as laying two aligned calibration cloths on both sides of the vehicle body), and the calibration cloth includes four parallel lines and a perpendicular line of the four parallel lines. In this way, the camera extrinsic parameter calibration is realized through the four parallel lines and the perpendicular line of the calibration cloth. The camera extrinsic parameter calibration can be completed based on the parallel line information and the straight line intersection information, the camera extrinsic parameter can be accurately calculated, the automatic calibration of the camera extrinsic parameter is realized, the camera extrinsic parameter can be effectively calibrated, the camera extrinsic parameter can be accurately calibrated, the imaging effect of the panoramic image is improved, and the user experience is improved. The line feature can be used for extrinsic parameter calibration, which can better adapt to complex environments and improve the calibration accuracy. The line feature is more robust than the point feature, and can better adapt to strong light, weak light, and partial area occlusion. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the description of the embodiments of the present application or the prior art will be briefly introduced. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0018] Figure 1 is a flowchart of the extrinsic parameter calibration method in an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of a calibration site in an embodiment of the present application;

[0020] Figure 3 is a flowchart of the extrinsic parameter calibration method in an embodiment of the present application;

[0021] Figure 4A is a schematic diagram of an original image / fish-eye image in an embodiment of the present application;

[0022] Figures 4B-4E is a schematic diagram of a de-distorted image in an embodiment of the present application;

[0023] Figures 5A-5D is a schematic diagram of edge point extraction and straight line fitting in an embodiment of the present application;

[0024] Figure 5E is a schematic diagram of straight line intersection in an embodiment of the present application;

[0025] Figure 6A is a schematic diagram of a coordinate system in an embodiment of the present application;

[0026] Figure 6Bis a calibration flowchart for a rotation parameter in an embodiment of the present application;

[0027] Figure 6C is a calibration flowchart for a translation parameter in an embodiment of the present application;

[0028] Figure 7 is a structural schematic diagram of an external parameter calibration device in an embodiment of the present application;

[0029] Figure 8 is a structural schematic diagram of an external parameter calibration device in an embodiment of the present application;

[0030] Figure 9 is a hardware structural diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The terminology used in the embodiments of the present application is merely for the purpose of describing particular embodiments and is not intended to be limiting of the present application. As used in the present application and the claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0032] It should be understood that although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order of the information. These terms are used merely to distinguish one type of information from another. For example, a first information can be termed a second information, and similarly, a second information can be termed a first information, without departing from the scope of the present application. Furthermore, the word "if" can be interpreted as meaning "when" or "upon" or "in response to determining," depending on the context.

[0033] An external parameter calibration method is provided in the embodiments of the present application. The vehicle to be calibrated includes surround-view cameras (such as a surround-view camera on the left side of the vehicle to be calibrated, a surround-view camera on the right side of the vehicle to be calibrated, a surround-view camera in front of the vehicle to be calibrated, and a surround-view camera behind the vehicle to be calibrated), and there are calibration cloths (a calibration cloth on the left side of the vehicle to be calibrated and a calibration cloth on the right side of the vehicle to be calibrated) around the vehicle to be calibrated, and the calibration cloth can include four parallel lines (i.e. parallel straight lines) and perpendicular lines of the four parallel lines.

[0034] Referring to Figure 1 FIG. 1 shows a flowchart of the external parameter calibration method. The method can include:

[0035] In step 101, four initial image straight lines corresponding to four parallel lines and a reference image straight line corresponding to a vertical line are obtained based on the obtained target image; the reference image straight line corresponds to four initial intersection coordinates of the four initial image straight lines, and the four initial image straight lines correspond to reference intersection coordinates.

[0036] In step 102, the roll angle of the surround-view camera is iteratively optimized based on four physical straight lines corresponding to the four initial image straight lines in the vehicle body world coordinate system, to obtain a target roll angle; and the target pitch angle and the target yaw angle of the surround-view camera are determined based on the reference intersection coordinates and the target roll angle.

[0037] In step 103, a target translation parameter is determined based on the target roll angle, the target pitch angle, the target yaw angle, the four initial intersection coordinates, and the physical coordinates corresponding to the four initial intersection coordinates in the vehicle body world coordinate system, wherein the target translation parameter represents the installation position of the surround-view camera in the vehicle body world coordinate system.

[0038] In step 104, the camera extrinsic parameters of the surround-view camera are calibrated, wherein the camera extrinsic parameters can include the target roll angle, the target pitch angle, the target yaw angle, and the target translation parameter.

[0039] For example, on the calibration board, the distance between the two parallel lines on the first side is equal to the distance between the two parallel lines on the second side. Based on this, the target roll angle of the surround-view camera can be obtained by iteratively optimizing the four physical straight lines corresponding to the four initial image straight lines in the vehicle body world coordinate system, which can include but is not limited to: obtaining an initial roll angle, generating a homography matrix based on the initial roll angle, and converting the four initial image straight lines to four physical straight lines in the vehicle body world coordinate system based on the homography matrix. A first distance between the two physical straight lines on the first side is determined, and a second distance between the two physical straight lines on the second side is determined. The first side can be the left side, and the second side can be the right side, or the first side can be the upper side, and the second side can be the lower side. If the difference between the first distance and the second distance is less than a threshold value, the initial roll angle is taken as the target roll angle. If the difference between the first distance and the second distance is greater than or equal to the threshold value, the initial roll angle is iteratively optimized to obtain an iteratively optimized initial roll angle; and the operation of generating a homography matrix based on the initial roll angle is performed based on the iteratively optimized initial roll angle.

[0040] For example, generating the homography matrix based on the initial roll angle can include, but is not limited to, determining an initial pitch angle and an initial yaw angle of the surround-view camera based on the reference intersection coordinates, camera intrinsic parameters of the surround-view camera, and the initial roll angle. Determining initial translation parameters based on a length, a width, and a height of the vehicle to be calibrated, the initial translation parameters can include an initial horizontal parameter, an initial vertical parameter, and an initial vertical parameter; wherein the initial horizontal parameter is determined based on the length, the initial vertical parameter is determined based on the width, and the initial vertical parameter is determined based on the height. Generating the homography matrix based on the initial roll angle, the initial pitch angle, the initial yaw angle, the initial horizontal parameter, the initial vertical parameter, and the initial vertical parameter.

[0041] For example, determining the target pitch angle and the target yaw angle based on the reference intersection coordinates and the target roll angle can include, but is not limited to, determining the target pitch angle and the target yaw angle by using the following expression: wherein pitch can represent the target pitch angle, yaw can represent the target yaw angle, roll can represent the target roll angle, (u0, v0) can represent the reference intersection coordinates, and (fx, fy, cx, cy) can represent the camera intrinsic parameters.

[0042] For example, determining the target translation parameters based on the target roll angle, the target pitch angle, the target yaw angle, the four initial intersection coordinates, and the physical coordinates corresponding to the four initial intersection coordinates in the vehicle body world coordinate system can include, but is not limited to, determining a rotation matrix based on the target roll angle, the target pitch angle, and the target yaw angle; selecting at least two initial intersection coordinates from the four initial intersection coordinates as candidate intersection coordinates; and determining the target translation parameters based on the rotation matrix, the camera intrinsic parameters of the surround-view camera, the candidate intersection coordinates, and the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system. The target translation parameters can include target horizontal parameters, target vertical parameters, and target vertical parameters.

[0043] For example, determining the target translation parameters based on the rotation matrix, the camera intrinsic parameters of the surround-view camera, the candidate intersection coordinates, and the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system can include, but is not limited to, determining the target horizontal parameters, the target vertical parameters, and the target vertical parameters by using the following expression: 00 fx-R 10 cx+R 10 x′)h+(-R 01 fx-R 11 cx+R 11 x′)w+(-R 02 fx-R 12 cx+R 12 x′)z=xx′R 10+yx' R 11 -x(R 00 fx+R 10 cx)-y(R 01 fx+R 11 cx); (R 20 fy-R 10 cy+R 10 y')h+(R 21 fy-R 11 cy+R 11 y')w+(R 22 fy-R 12 cy+R 12 y')z=xy' R 10 +yy' R 11 -x(R 10 cy-R 20 fy)-y(R 11 cy-R 21 fy); wherein R 00 -R 22 represents a parameter in a rotation matrix, (fx, fy, cx, cy) represents camera intrinsic parameters, (x', y') represents candidate intersection coordinates, (x, y) represents physical coordinates corresponding to the candidate intersection coordinates in a vehicle world coordinate system, h represents a target horizontal parameter, w represents a target vertical parameter, and z represents a target vertical parameter.

[0044] For example, if the surround camera is located on the left side of the vehicle to be calibrated, the calibration cloth is located on the left side of the vehicle to be calibrated, and if the surround camera is located on the right side of the vehicle to be calibrated, the calibration cloth is located on the right side of the vehicle to be calibrated; wherein the four parallel lines are straight lines parallel to the vehicle width direction of the vehicle to be calibrated, and the perpendicular lines of the four parallel lines are straight lines perpendicular to the vehicle width direction of the vehicle to be calibrated; if the surround camera is located in front of the vehicle to be calibrated, the calibration cloth is located on the left side and / or right side of the vehicle to be calibrated, and if the surround camera is located behind the vehicle to be calibrated, the calibration cloth is located on the left side and / or right side of the vehicle to be calibrated; the four parallel lines are straight lines parallel to the vehicle length direction of the vehicle to be calibrated, and the perpendicular lines of the four parallel lines are straight lines perpendicular to the vehicle length direction of the vehicle to be calibrated.

[0045] It can be seen from the above technical solutions that, in the embodiments of the present application, the calibration cloth is deployed around the vehicle to be calibrated (for example, two aligned calibration cloths are laid on both sides of the vehicle body), and the calibration cloth includes four parallel lines and a vertical line of the four parallel lines. In this way, the camera extrinsic parameter calibration is realized through the four parallel lines and the vertical line of the calibration cloth, the camera extrinsic parameter calibration can be completed based on the parallel line information and the straight line intersection information, the camera extrinsic parameter can be accurately calculated, thereby realizing the automatic calibration of the camera extrinsic parameter, effectively calibrating the camera extrinsic parameter, accurately calibrating the camera extrinsic parameter, improving the imaging effect of the panoramic image, and improving the user experience. The extrinsic parameter calibration can be performed using line features, which can better adapt to complex environments and improve the calibration accuracy. The extrinsic parameter calibration is performed through line features, which is more robust than point features and can better adapt to strong light, weak light, and partial area occlusion.

[0046] The technical solutions of the embodiments of the present application are described below in combination with specific application scenarios.

[0047] The vehicle-mounted panoramic system can include multiple cameras installed on the vehicle, such as a camera located at the front of the vehicle, a camera located at the rear of the vehicle, a camera located at the left side of the vehicle, and a camera located at the right side of the vehicle. During the driving of the vehicle, these cameras capture images in real time, and the vehicle-mounted panoramic system can stitch these images into a panoramic image, i.e., a 360-degree surround view image. In order to be able to generate a panoramic image, the camera extrinsic parameters of the cameras installed on the vehicle need to be calibrated before the vehicle is shipped.

[0048] In order to calibrate the camera extrinsic parameters of the cameras, four calibration boards containing feature points can be placed in the common view area of the cameras, and the extrinsic parameter calibration is completed based on the position information of the feature points. However, since this scheme uses feature points for extrinsic parameter calibration, in the case of overexposure, overdarkness, corner occlusion, etc., the positioning of the feature points will be affected, thereby reducing the calibration accuracy. In addition, this scheme requires the four calibration boards to be placed in alignment, and the operation of accurately placing the calibration boards on site is complex, which affects the calibration efficiency.

[0049] In view of the above finding, a kind of extrinsic parameter calibration method is provided in the embodiments of the present application to realize the extrinsic parameter calibration of surround view camera. Two aligned calibration cloths are laid on both sides of the vehicle body, the straight line information in the calibration cloth is detected, the parallelism between straight lines and the intersection pixel coordinates of straight lines are used, and the calibration of camera extrinsic parameter is completed in combination with known calibration cloth world coordinates. Since only two calibration cloths are used, the deployment cost can be reduced, and line features are more robust than point features, which can better adapt to strong light, weak light, and partial area occlusion.

[0050] The surround view camera can be a camera located in front of the vehicle, a camera located in the rear of the vehicle, a camera located on the left side of the vehicle, a camera located on the right side of the vehicle, and the like, which are collectively referred to as surround view cameras. For example, the surround view camera can be a fisheye camera, or other types of cameras.

[0051] Exemplarily, a vehicle requiring calibration of camera extrinsic parameters is referred to as a vehicle to be calibrated, and the camera parameters of the surround view cameras on the vehicle to be calibrated need to be calibrated before the vehicle to be calibrated is shipped, and the camera parameters can include camera intrinsic parameters (i.e., internal parameters) and camera extrinsic parameters (i.e., external parameters).

[0052] The camera intrinsic parameters consist of (cx, cy, fx, fy, s0, s1, s2, s3) and the like, (cx, cy) is the principal point coordinates, (fx, fy) is the camera focal length, (s0, s1, s2, s3) is the camera distortion coefficient, the camera intrinsic parameters are parameters related to the characteristics of the camera itself, and the calibration process of the camera intrinsic parameters is not limited in the embodiment. In subsequent embodiments, the camera intrinsic parameters are taken as an example consisting of (cx, cy, fx, fy) and the like.

[0053] The camera extrinsic parameters are used to represent the conversion relationship between the world coordinate system and the camera coordinate system. The camera extrinsic parameters consist of (roll, pitch, yaw, Tx, Ty, Tz) and the like, (roll, pitch, yaw) are three-axis rotation parameters, roll can represent the roll angle, pitch can represent the pitch angle, and yaw can represent the yaw angle. (Tx, Ty, Tz) are three-axis translation parameters, Tx represents the horizontal parameter, i.e., the translation parameter value of the x-axis, and the horizontal parameter is hereinafter referred to as h. Ty represents the vertical parameter, i.e., the translation parameter value of the y-axis, and the vertical parameter is hereinafter referred to as w. Tz represents the vertical parameter, i.e., the translation parameter value of the z-axis, and the vertical parameter is hereinafter referred to as z.

[0054] Exemplarily, in order to calibrate the camera extrinsic parameters (such as the above 6 parameters), two alignment calibration cloths need to be laid on both sides of the vehicle body, as shown in FIG. 1, which is a schematic diagram of a calibration site. Two calibration cloths are placed in alignment, and the size of the calibration cloth is known. Figure 2

[0055] As shown in FIG. 2, the two sides of the vehicle to be calibrated have calibration cloths, and the vehicle to be calibrated is parked in parallel to the calibration cloths. For example, there is a calibration cloth 1 on the left side of the vehicle to be calibrated, and there is a calibration cloth 2 on the right side of the vehicle to be calibrated. The calibration cloth 1 and the calibration cloth 2 are placed in alignment, and the shapes of the calibration cloth 1 and the calibration cloth 2 are the same. Figure 2

[0056] As shown in FIG. 3, the two sides of the vehicle to be calibrated have calibration cloths, and the vehicle to be calibrated is parked in parallel to the calibration cloths. For example, there is a calibration cloth 1 on the left side of the vehicle to be calibrated, and there is a calibration cloth 2 on the right side of the vehicle to be calibrated. The calibration cloth 1 and the calibration cloth 2 are placed in alignment, and the shapes of the calibration cloth 1 and the calibration cloth 2 are the same. Figure 2 ​​As shown, taking the calibration cloth 1 as an example, the calibration cloth 1 is composed of five black and white rectangular areas, such as black rectangular area a1, white rectangular area a2, black rectangular area a3, white rectangular area a4, and black rectangular area a5. Alternatively, the sequence is white rectangular area, black rectangular area, white rectangular area, black rectangular area, and white rectangular area.

[0057] On this basis, the straight line intersecting the black rectangular area a1 and the white rectangular area a2 can be recorded as a vertical straight line b1, the straight line intersecting the white rectangular area a2 and the black rectangular area a3 can be recorded as a vertical straight line b2, the straight line intersecting the black rectangular area a3 and the white rectangular area a4 can be recorded as a vertical straight line b3, and the straight line intersecting the white rectangular area a4 and the black rectangular area a5 can be recorded as a vertical straight line b4.

[0058] The calibration cloth 1 further includes four horizontal straight lines perpendicular to the vertical straight lines. The first horizontal straight line is recorded as a horizontal straight line c1, the horizontal straight line c1 sequentially passes through the five black and white rectangular areas, the second horizontal straight line is recorded as a horizontal straight line c2, and the horizontal straight line c1 and the horizontal straight line c2 form a white rectangular area (or a black rectangular area). The third horizontal straight line is recorded as a horizontal straight line c3, and the fourth horizontal straight line is recorded as a horizontal straight line c4. The horizontal straight line c3 and the horizontal straight line c4 form a white rectangular area (or a black rectangular area).

[0059] Similarly, the calibration cloth 2 has the same shape as the calibration cloth 1, and the calibration cloth 2 also has four vertical straight lines and four horizontal straight lines, which are recorded as a vertical straight line b1', a vertical straight line b2', a vertical straight line b3', a vertical straight line b4', a horizontal straight line c1', a horizontal straight line c2', a horizontal straight line c3', and a horizontal straight line c4'.

[0060] The distance between the vertical straight line b2 and the vertical straight line b1 is equal to the distance between the vertical straight line b4 and the vertical straight line b3, the distance between the vertical straight line b2' and the vertical straight line b1' is equal to the distance between the vertical straight line b4' and the vertical straight line b3', the distance between the horizontal straight line c2 and the horizontal straight line c1 is equal to the distance between the horizontal straight line c4 and the horizontal straight line c3, and the distance between the horizontal straight line c2' and the horizontal straight line c1' is equal to the distance between the horizontal straight line c4' and the horizontal straight line c3'.

[0061] For example, if the surround camera is located on the left side of the vehicle to be calibrated, the external parameter calibration of the surround camera is completed based on the calibration cloth 1 on the left side, and the external parameter calibration is completed based on the information of the horizontal straight line c1, the horizontal straight line c2, the horizontal straight line c3, the horizontal straight line c4, and the vertical straight line b1 (or other vertical straight lines). In subsequent embodiments, the horizontal straight line c1, the horizontal straight line c2, the horizontal straight line c3, and the horizontal straight line c4 can be regarded as four parallel lines, and the vertical straight line b1 can be regarded as a vertical line of the four parallel lines.

[0062] For example, if the surround camera is located on the right side of the vehicle to be calibrated, the external parameter calibration of the surround camera is completed based on the right side calibration cloth 2, and the external parameter calibration is completed based on the information of the horizontal straight line c1', the horizontal straight line c2', the horizontal straight line c3', the horizontal straight line c4', and the vertical straight line b1' (or other vertical straight lines). In subsequent embodiments, the horizontal straight line c1', the horizontal straight line c2', the horizontal straight line c3', and the horizontal straight line c4' can be regarded as four parallel lines, and the vertical straight line b1' can be regarded as a vertical line of the four parallel lines.

[0063] As can be seen from the above, for the surround camera located on the left side of the vehicle to be calibrated and the surround camera located on the right side of the vehicle to be calibrated, the four parallel lines are straight lines parallel to the vehicle width direction of the vehicle to be calibrated, and the vertical line of the four parallel lines is a straight line perpendicular to the vehicle width direction of the vehicle to be calibrated.

[0064] For example, if the surround camera is located on the right side of the vehicle to be calibrated, the external parameter calibration of the surround camera is completed based on the right side calibration cloth 2, and the external parameter calibration is completed based on the information of the horizontal straight line c1', the horizontal straight line c2', the horizontal straight line c3', the horizontal straight line c4', and the vertical straight line b1' (or other vertical straight lines). In subsequent embodiments, the horizontal straight line c1', the horizontal straight line c2', the horizontal straight line c3', and the horizontal straight line c4' can be regarded as four parallel lines, and the vertical straight line b1' can be regarded as a vertical line of the four parallel lines.

[0065] For example, based on the vertical straight line b1, the vertical straight line b3, the vertical straight line b1', the vertical straight line b3', and the horizontal straight line c1, in subsequent embodiments, the vertical straight line b1, the vertical straight line b3, the vertical straight line b1', and the vertical straight line b3' can be regarded as four parallel lines, and the horizontal straight line c1 can be regarded as a vertical line of the four parallel lines.

[0066] As can be seen from the above, for the surround camera located on the left side of the vehicle to be calibrated and the surround camera located on the right side of the vehicle to be calibrated, the four parallel lines are straight lines parallel to the vehicle width direction of the vehicle to be calibrated, and the vertical line of the four parallel lines is a straight line perpendicular to the vehicle width direction of the vehicle to be calibrated.

[0067] In the above application scenarios, an external parameter calibration method is provided in the embodiments of the present application, as shown in Figure 3 The method can include the following steps:

[0068] In step 301, an initial image of the calibration cloth is captured by the surround-view camera.

[0069] If the surround-view camera is a fisheye camera, the initial image can be a fisheye image. As shown in Figure 4A From left to right, they are: a front-view initial image captured by the surround-view camera in front of the vehicle to be calibrated, a left-view initial image captured by the surround-view camera on the left side of the vehicle to be calibrated, a right-view initial image captured by the surround-view camera on the right side of the vehicle to be calibrated, and a rear-view initial image captured by the surround-view camera behind the vehicle to be calibrated.

[0070] In step 302, a distortion correction is performed on the initial image to obtain a distortion-corrected image, and a target image is generated based on the distortion-corrected image. The target image is the distortion-corrected image, or the target image is obtained by processing the distortion-corrected image.

[0071] For example, the initial image can be distortion-corrected based on the camera intrinsic parameters of the surround-view camera to obtain a distortion-corrected image. For example, as shown in Figure 4B is a distortion-corrected image obtained by performing distortion correction on the front-view initial image, as shown in Figure 4C is a distortion-corrected image obtained by performing distortion correction on the left-view initial image, as shown in Figure 4D is a distortion-corrected image obtained by performing distortion correction on the right-view initial image, as shown in Figure 4E is a distortion-corrected image obtained by performing distortion correction on the rear-view initial image.

[0072] In step 303, four initial image straight lines corresponding to four parallel lines and a reference image straight line corresponding to a perpendicular line are obtained based on the target image. For example, the reference image straight line and the four initial image straight lines can correspond to four initial intersection coordinates, and the four initial image straight lines can correspond to a reference intersection coordinate.

[0073] For example, the calibration cloth can include four parallel lines of the surround-view camera and a perpendicular line of the four parallel lines. For example, when the surround-view camera is located on the left side of the vehicle to be calibrated, the four parallel lines are horizontal straight lines c1, c2, c3, and c4, and the perpendicular line is a vertical straight line b1.

[0074] After the target image is obtained, straight line edge point extraction and straight line fitting can be performed in the target image to obtain image straight lines in the target image (in order to distinguish, the straight lines in the target image are referred to as image straight lines), that is, four initial image straight lines corresponding to the four pairs of parallel lines can be obtained, and a reference image straight line corresponding to the vertical line is obtained, that is, five image straight lines are obtained through straight line edge point extraction and straight line fitting.

[0075] Referring to Figure 5A Fig. 4 shows an example of performing straight line edge point extraction and straight line fitting in the front view de-distorted image to obtain five image straight lines, Figure 5A which are shown in Fig. 4. Referring to Figure 5B Fig. 5 shows an example of performing straight line edge point extraction and straight line fitting in the left view de-distorted image to obtain five image straight lines, Figure 5B which are shown in Fig. 5. Referring to Figure 5C Fig. 6 shows an example of performing straight line edge point extraction and straight line fitting in the right view de-distorted image to obtain five image straight lines, Figure 5C which are shown in Fig. 6. Referring to Figure 5D Fig. 7 shows an example of performing straight line edge point extraction and straight line fitting in the back view de-distorted image to obtain five image straight lines, Figure 5D which are shown in Fig. 7.

[0076] For example, when the four initial image straight lines and the reference image straight line are obtained, the reference image straight line intersects each initial image straight line, that is, the reference image straight line corresponds to four initial intersection coordinates of the four initial image straight lines. Although the four parallel lines in the world coordinate system are in parallel relationship, in the de-distorted image, the four initial image straight lines intersect at a point, which is referred to as a reference intersection coordinate.

[0077] For example, taking the front view de-distorted image as an example, referring to Figure 5E Fig. 8 shows a schematic diagram of straight line intersection. The four initial image straight lines are l1, l2, l3 and l4, and the reference image straight line is l5. Obviously, from Figure 5E it can be seen that the four initial intersection coordinates of the reference image straight line and the four initial image straight lines are p2, p3, p4 and p5. In addition, the four initial image straight lines intersect at p1, that is, the reference intersection coordinate is p1.

[0078] For example, when straight line edge point extraction is performed in the target image, a gradient change point can be extracted, which can be a black-white intersection point. For example, a canny operator, a sobel operator, a laplace operator and the like can be used to extract edge points in the target image, and the mode is not limited.

[0079] For example, when performing linear fitting, a large number of edge points can be fitted into a straight line, for example, a straight line can be fitted by using a least square method, a gradient descent method, etc., and no limitation is made to this manner.

[0080] For example, as shown in FIG. 4, four initial image straight lines are obtained, and a reference image straight line is obtained. Figure 5E As shown, the four initial image straight lines correspond to four physical straight lines in the real world coordinate system, the four physical straight lines are parallel to each other, and the interval of the left two physical straight lines is equal to the interval of the right two physical straight lines. The reference image straight line corresponds to one physical straight line in the real world coordinate system, and the physical straight line is perpendicular to the four physical straight lines, which is a characteristic of the calibration cloth.

[0081] In step 304, based on the four initial image straight lines, the reference intersection coordinates, the four initial intersection coordinates, and the physical coordinates corresponding to the four initial intersection coordinates in the vehicle body world coordinate system, the camera extrinsic parameters are determined.

[0082] For example, after obtaining the four initial image straight lines, the reference image straight line, the four initial intersection coordinates, and the reference intersection coordinates, the camera extrinsic parameters can be determined based on this information. For example, the camera extrinsic parameters can include rotation parameters (roll, pitch, yaw) and translation parameters (h, w, z), and the calibration process of the rotation parameters (roll, pitch, yaw) and the translation parameters (h, w, z) is described below.

[0083] For example, as shown in FIG. 5, the four initial image straight lines, the reference image straight line, the four initial intersection coordinates, and the reference intersection coordinates are obtained. Figure 6A As shown, a schematic diagram of the coordinate systems involved in the calibration process of the camera extrinsic parameters is shown. Oc-XcYcZc is the camera coordinate system, that is, the coordinate system of the surround view camera. Ow-XwYwZw is the vehicle body world coordinate system, that is, the coordinate system with the center of the vehicle to be calibrated as the origin. Ocw-XcwYcwZcw is the world coordinate system (auxiliary world coordinate system), which is the coordinate system with the intersection of the camera optical axis and the ground as the origin.

[0084] When calibrating the camera extrinsic parameters, the camera extrinsic parameters need to convert the vehicle body world coordinate system Ow-XwYwZw into the camera coordinate system Oc-XcYcZc, and the following conditions need to be met: 1. Rotate yaw+pi / 2 around Zw to make the direction of Ow-XwYwZw consistent with the coordinate system Ocw-XcwYcwZcw. 2. Rotate -pitch along the Xw axis to make Yw consistent with the direction of Zc. 3. Rotate -roll along the Yw axis to make Xw consistent with the direction of Xc, and Zw consistent with the direction of -Yc. 4. Rotate -90 degrees along the Xw axis to make Zw consistent with the direction of Zc, and Yw consistent with the direction of Yc. 5. Coordinate system translation T, so that the coordinate system Ow-XwYwZw and the camera coordinate system Oc-XcYcZc completely coincide.

[0085] To make the above conditions hold, the transformation relation can be found in expressions (1) and (2):

[0086]

[0087]

[0088] In the above expression, R 33 This represents a 3x3 rotation matrix, determined based on rotation parameters (roll, pitch, yaw). T represents a 3x1 translation matrix, determined based on translation parameters (h, w, z). For example, pitch, yaw, and roll represent the pitch, yaw, and roll angles, respectively, and (cam_x, cam_y, cam_z) represents the installation position of the surround-view camera in the vehicle's world coordinate system.

[0089] For example, the transformation relationship between coordinate points on the target image (such as the coordinates of the reference intersection point, the coordinates of the four initial intersection points, etc.) and the world coordinate system (such as the world coordinate system of the vehicle body) can be seen in expression (3). Based on expressions (1), (2) and (3), and Zw = 0, the homography matrix H between the world coordinate system and the camera coordinate system can be derived, as shown in expression (4).

[0090]

[0091]

[0092] In expressions (3) and (4), (cx, cy) are the principal point coordinates, and (fx, fy) are the camera focal lengths; these are camera intrinsic parameters. r represents the rotation matrix R. 33 The value in, such as r 00 This represents the value in the first row and first column, r. 01 This represents the value in the 1st row and 2nd column, r. 10 This represents the value in the 2nd row and 1st column. t represents the value in the translation parameter T, such as t0 for the 1st row, t1 for the 2nd row, and t2 for the 3rd row. (u,v) represents the coordinates of the target image, and (Xw,Yw) represents the coordinates in the world coordinate system.

[0093] Exemplarily, since the four initial image straight lines (l1-l4) are parallel to the Xw coordinate axis in the world coordinate system Ow, the angle between the straight line and the Ycw axis is the yaw angle yaw, and the point on the straight line can be represented as (Ycw*tan(yaw), Ycw, 0). For the reference intersection point coordinates on the target image, it is the value of (u0, v0) when Ycw->∞. Based on the above principle, combined with expression (1), expression (2), expression (3), expression (4), after taking the limit value, the following expression (5) can be obtained.

[0094]

[0095] Based on expression (5), in the case where the roll angle roll, the reference intersection point coordinates (u0, v0) and the camera intrinsic parameters (fx, fy, cx, cy) are known, the pitch angle and the yaw angle can be solved, as shown in expression (6).

[0096]

[0097] Obviously, since the reference intersection point coordinates (u0, v0) are the intersection points of the four initial image straight lines (l1-l4) and are known values, the pitch angle is only related to the roll angle roll, and the yaw angle is only related to the roll angle roll, so only one unknown quantity roll needs to be solved to complete the calibration of the external parameters (pitch, yaw, roll).

[0098] Based on the above principle, in the embodiment, the rotation parameters (roll, pitch, yaw) can be calibrated, as shown in Figure 6B The process includes:

[0099] Step 601, acquiring an initial roll angle.

[0100] For example, a certain fixed value (such as 0, 10, etc.) can be used as the initial roll angle roll.

[0101] Step 602, determining initial translation parameters based on the length, width and height of the vehicle to be calibrated, the initial translation parameters can include an initial horizontal parameter h, an initial vertical parameter w and an initial vertical parameter z.

[0102] Exemplarily, in addition to the rotation parameters (roll, pitch, yaw), the camera external parameters also include three translation parameters (h, w, z), and the translation parameters (h, w, z) are irrelevant to the constraint condition that the left and right parallel lines are equal in width, so when solving the rotation parameters (roll, pitch, yaw), the translation parameters can be set arbitrarily, and the set translation parameters are used as the initial horizontal parameter h, the initial vertical parameter w and the initial vertical parameter z.

[0103] For example, the initial horizontal parameter h can be determined based on the vehicle length, the initial vertical parameter w can be determined based on the vehicle width, and the initial vertical parameter z can be determined based on the vehicle height. On this basis, for the surround view camera in front of the vehicle to be calibrated, the horizontal coordinate in the vehicle body world coordinate system is approximately the negative value of 1 / 2 of the vehicle length, the vertical coordinate is approximately 0, and the vertical coordinate is approximately the vehicle height. Therefore, the initial horizontal parameter h is the negative value of 1 / 2 of the vehicle length, the initial vertical parameter w is 0, and the initial vertical parameter z is the vehicle height. Alternatively, for the surround view camera behind the vehicle to be calibrated, the horizontal coordinate in the vehicle body world coordinate system is approximately 1 / 2 of the vehicle length, the vertical coordinate is approximately 0, and the vertical coordinate is approximately the vehicle height. Therefore, the initial horizontal parameter h is 1 / 2 of the vehicle length, the initial vertical parameter w is 0, and the initial vertical parameter z is the vehicle height. Alternatively, for the surround view camera on the left side of the vehicle to be calibrated, the horizontal coordinate in the vehicle body world coordinate system is approximately 0, the vertical coordinate is approximately the negative value of 1 / 2 of the vehicle width, and the vertical coordinate is approximately the vehicle height. Therefore, the initial horizontal parameter h is 0, the initial vertical parameter w is the negative value of 1 / 2 of the vehicle width, and the initial vertical parameter z is the vehicle height. Alternatively, for the surround view camera on the right side of the vehicle to be calibrated, the horizontal coordinate in the vehicle body world coordinate system is approximately 0, the vertical coordinate is approximately 1 / 2 of the vehicle width, and the vertical coordinate is approximately the vehicle height. Therefore, the initial horizontal parameter h is 0, the initial vertical parameter w is 1 / 2 of the vehicle width, and the initial vertical parameter z is the vehicle height.

[0104] In step 603, the initial pitch angle and the initial yaw angle of the surround view camera are determined based on the reference intersection coordinates, the camera intrinsic parameters of the surround view camera, and the initial roll angle.

[0105] For example, the reference intersection coordinates (u0, v0), the camera intrinsic parameters (fx, fy, cx, cy), and the initial roll angle roll can be substituted into expression (6) to obtain the initial pitch angle pitch and the initial yaw angle yaw.

[0106] In step 604, a homography matrix is generated based on the initial roll angle, the initial pitch angle, the initial yaw angle, the initial horizontal parameter, the initial vertical parameter, and the initial vertical parameter.

[0107] For example, as shown in expression (3) and expression (4), the homography matrix H can be determined based on the rotation matrix R 33 and the translation matrix T 31 , and after substituting the initial roll angle roll, the initial pitch angle pitch, and the initial yaw angle yaw into expression (1), the rotation matrix R 33 , and after substituting the initial horizontal parameter h, the initial vertical parameter w, and the initial vertical parameter z into expression (2), the translation matrix T 31Based on the initial roll angle roll, the initial pitch angle pitch, the initial yaw angle yaw, the initial horizontal parameter h, the initial vertical parameter w, and the initial vertical parameter z, a homography matrix H can be generated.

[0108] At step 605, the 4 initial image lines are converted into 4 physical lines in the vehicle body world coordinate system based on the homography matrix. For example, the homography matrix is used to represent the conversion relationship between the image coordinate system and the vehicle body world coordinate system, and thus, for the initial image line in the image coordinate system, the initial image line can be converted into a physical line in the vehicle body world coordinate system, and the conversion process is not limited. For the 4 initial image lines, the 4 initial image lines can correspond to 4 physical lines in the vehicle body world coordinate system.

[0109] Based on the 4 physical lines in the vehicle body world coordinate system, the first distance of the 2 physical lines on the first side and the second distance of the 2 physical lines on the second side can be determined. For the front surround-view camera or the rear surround-view camera, the first side can be the left side and the second side can be the right side, i.e., the first distance of the 2 physical lines on the left side and the second distance of the 2 physical lines on the right side are determined. For the left surround-view camera or the right surround-view camera, the first side can be the upper side and the second side can be the lower side, i.e., the first distance of the 2 physical lines on the upper side and the second distance of the 2 physical lines on the lower side are determined.

[0110] At step 606, it is determined whether the difference between the first distance and the second distance is less than a threshold value.

[0111] If not, step 607 can be performed, and if yes, step 608 can be performed.

[0112] For example, in the actual scene of the calibration cloth, the distance between the 2 lines on the first side is equal to the distance between the 2 lines on the second side (e.g., the left and right parallel lines are equal in width, or the upper and lower parallel lines are equal in width), and based on the constraint condition of the equal width of the parallel lines (i.e., the difference between the two distances is 0), it can be determined whether the difference between the first distance and the second distance is less than a threshold value (which can be slightly greater than 0). If yes, it indicates that the constraint condition of the equal width of the parallel lines can be met when the initial roll angle is used, and the initial roll angle at this time is accurate. If not, it indicates that the constraint condition of the equal width of the parallel lines is not met when the initial roll angle is used, and the initial roll angle at this time is not accurate.

[0113] At step 607, the initial roll angle is iteratively optimized to obtain an iteratively optimized initial roll angle, and based on the iteratively optimized initial roll angle, step 603 is returned to be executed, i.e., based on the reference intersection coordinates, the camera intrinsic parameters, and the iteratively optimized initial roll angle, the initial pitch angle and the initial yaw angle are determined.

[0114] For example, the iterative optimization of the initial roll angle refers to optimizing the initial roll angle by using a constraint of equal parallel line width, i.e., by adjusting the initial roll angle to make the difference between the first distance and the second distance smaller and smaller until the difference between the first distance and the second distance is less than a threshold value. In this embodiment, the iterative optimization process is not limited, and it is only required that the difference between the first distance and the second distance can be made smaller.

[0115] Step 608: Taking the initial roll angle as a target roll angle.

[0116] Step 609: Determining a target pitch angle and a target yaw angle of the surround-view camera based on the reference intersection coordinates and the target roll angle. For example, the target pitch angle and the target yaw angle of the surround-view camera are determined based on the reference intersection coordinates, camera intrinsic parameters of the surround-view camera, and the target roll angle.

[0117] For example, the target pitch angle and the target yaw angle can be determined by using expression (6), in which pitch can represent the target pitch angle, yaw can represent the target yaw angle, roll can represent the target roll angle, (u0, v0) can represent the reference intersection coordinates, and (fx, fy, cx, cy) can represent the camera intrinsic parameters.

[0118] As described above, in the case where the reference intersection coordinates (u0, v0) are known and the camera intrinsic parameters (fx, fy, cx, cy) are known, the rotation parameters (roll, pitch, yaw) are only related to roll, and thus, only one unknown quantity roll needs to be solved to obtain pitch and yaw, so as to complete the calibration of the rotation parameters (roll, pitch, yaw).

[0119] After the calibration of the rotation parameters (roll, pitch, yaw) is completed, the translation parameters (h, w, z) also need to be calibrated. As shown in FIG. 6, a calibration process for the translation parameters (h, w, z) is shown. Figure 6C

[0120] Step 611: Determining a rotation matrix based on the target roll angle, the target pitch angle, and the target yaw angle.

[0121] For example, after the calibration of the rotation parameters (roll, pitch, yaw) is completed, the target roll angle roll, the target pitch angle pitch, and the target yaw angle yaw are substituted into expression (1) to obtain the rotation matrix R 33 .

[0122] Step 612: Selecting at least two initial intersection coordinates from the four initial intersection coordinates as candidate intersection coordinates. For example, any two initial intersection coordinates can be selected as the candidate intersection coordinates. ​

[0123] Step 613, determining a target translation extrinsic parameter based on the rotation matrix, camera intrinsic parameter of the surround-view camera, the candidate intersection coordinate, and a physical coordinate corresponding to the candidate intersection coordinate in the vehicle body world coordinate system. The target translation extrinsic parameter can include a target horizontal parameter h, a target vertical parameter w, and a target vertical parameter z.

[0124] For example, in expression (4), the homography matrix H between the world coordinate system and the camera coordinate system has been derived, and then the conversion relationship between the corresponding feature point coordinates in the two coordinate systems can be derived based on the homography matrix H, which can be seen from expression (7):

[0125]

[0126]

[0127] Based on expression (1), expression (2), expression (3), expression (4), expression (7), and expression (8), the relationship between the translation parameter hwz and the feature points can be derived, which can be seen from expression (9) and expression (10), which is the relationship between the translation parameter hwz and the feature points.

[0128] (-R 00 fx-R 10 cx+R 10 x′)h+(-R 01 fx-R 11 cx+R 11 x′)w+(-R 02 fx-R 12 cx+R 12 x′)z=

[0129] xx′R 10 +yx′R 11 -x(R 00 fx+R 10 cx)-y(R 01 fx+R 11 cx)expression (9)

[0130] (R 20 fy-R 10 cy+R 10 y′)h+(R 21 fy-R 11 cy+R 11 y′)w+(R 22 fy-R 12 cy+R 12 y′)z=

[0131] xy′R10 +yy′R 11 -x(R 10 cy-R 20 fy)-y(R 11 cy-R 21 fy) Expression (10)

[0132] From Expression (9) and Expression (10), a pair of feature points (i.e. candidate intersection coordinates, physical coordinates corresponding to candidate intersection coordinates in the vehicle body world coordinate system) can form two linear equations, therefore, only two pairs of feature points are needed to construct a non-homogeneous linear equation set containing four equations. Obviously, based on two pairs of feature points, the translation parameters hwz can be obtained by solving the non-homogeneous linear equation set through the least square method.

[0133] In summary, the rotation matrix, the camera intrinsic parameters of the surround-view camera, the candidate intersection coordinates, and the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system can be substituted into Expression (9) and Expression (10) to obtain the target horizontal parameter h, the target vertical parameter w, and the target vertical parameter z.

[0134] In the above expressions, R 00 -R 22 represents the parameters in the rotation matrix, such as R 00 represents the value of the first row and the first column, R 01 represents the value of the first row and the second column, R 02 represents the value of the first row and the third column, R 10 represents the value of the second row and the first column, and so on. (fx, fy, cx, cy) represents the camera intrinsic parameters, (x’, y’) represents the candidate intersection coordinates, (x, y) represents the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system, h represents the target horizontal parameter, w represents the target vertical parameter, and z represents the target vertical parameter.

[0135] Regarding the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system, the physical coordinates of each point on the calibration cloth can be obtained in advance, i.e. the physical coordinates of the intersection points of the vertical line and the four parallel lines can be obtained, therefore, the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system can be directly determined.

[0136] At this point, the acquisition of the rotation parameters (roll, pitch, yaw) and the translation parameters (h, w, z) is completed, and the rotation parameters (roll, pitch, yaw) and the translation parameters (h, w, z) are calibrated for the surround-view camera.

[0137] In a possible implementation, the rotation parameters (roll, pitch, yaw) and the translation parameters (h, w, z) of the left surround-view camera, the rotation parameters (roll, pitch, yaw) and the translation parameters (h, w, z) of the right surround-view camera, the rotation parameters (roll, pitch, yaw) and the translation parameters (h, w, z) of the front surround-view camera, and the rotation parameters (roll, pitch, yaw) and the translation parameters (h, w, z) of the rear surround-view camera can be calibrated in the above manner.

[0138] As can be seen from the above technical solutions, in the embodiments of the present application, the camera extrinsic parameter calibration can be completed based on the parallel line information and the straight line intersection information, and the scheme is suitable for the extrinsic parameter calibration of the front, left, right, and rear four cameras, and the camera extrinsic parameters can be accurately calculated, so that the automatic calibration of the camera extrinsic parameters is realized, the camera extrinsic parameters can be effectively calibrated, the camera extrinsic parameters can be accurately calibrated, the imaging effect of the panoramic image is improved, and the user experience is improved. The extrinsic parameter calibration can be performed by using line features, which can better adapt to complex environments and improve the calibration accuracy. The extrinsic parameter calibration is performed by using line features, which is more robust than point features and can better adapt to strong light, weak light, and partial area occlusion. Only two calibration cloths need to be aligned and placed, and the actual deployment cost is low.

[0139] Based on the same application concept as the above method, the embodiments of the present application also propose an extrinsic parameter calibration device, as shown in Figure 7 The extrinsic parameter calibration device can include an image acquisition unit, a data transmission unit, a data processing unit, a vehicle-mounted surround-view panoramic image generation unit, and an image display unit.

[0140] For example, the image acquisition unit is composed of vehicle-mounted cameras (i.e., cameras) distributed at the front, rear, left, and right of the vehicle body or more, and the intrinsic parameters of the vehicle-mounted cameras have been calibrated.

[0141] The data processing unit is a key unit for calibrating the extrinsic parameters of the vehicle-mounted cameras, and the data processing unit is used for calibrating the extrinsic parameters of the vehicle-mounted cameras. The specific calibration method can be referred to the above method, which will not be described here.

[0142] The vehicle-mounted surround-view panoramic image generation unit generates a lookup table between the vehicle-mounted surround-view panoramic image and the front, left, right, and rear four views according to the extrinsic parameter results calculated by the data processing unit and the camera intrinsic parameters, and generates a fusion weight table between the front view and the left view, the left view and the rear view, the rear view and the right view, and the right view and the front view. According to the original images of the four view cameras and the lookup table, the vehicle-mounted surround-view panoramic image is obtained.

[0143] An image display unit presents a panoramic image without seams to a user after the external parameters are calibrated.

[0144] Based on the same application concept as the above method, an external parameter calibration device is proposed in the embodiments of the present application. The vehicle to be calibrated includes a surround-view camera. There is a calibration cloth around the vehicle to be calibrated, and the calibration cloth includes four parallel lines and four vertical lines of the four parallel lines. As shown in Figure 8 The device can include:

[0145] An acquisition module 81 is configured to acquire four initial image straight lines corresponding to the four parallel lines and a reference image straight line corresponding to the vertical line based on the acquired target image. The reference image straight line corresponds to four initial intersection point coordinates, and the four initial image straight lines correspond to a reference intersection point coordinate.

[0146] A processing module 82 is configured to perform iterative optimization on a roll angle of the surround-view camera based on four physical straight lines corresponding to the four initial image straight lines in a vehicle body world coordinate system, to obtain a target roll angle.

[0147] A determination module 83 is configured to determine a target pitch angle and a target yaw angle of the surround-view camera based on the reference intersection point coordinate and the target roll angle, and determine a target translation parameter based on the target roll angle, the target pitch angle, the target yaw angle, the four initial intersection point coordinates, and physical coordinates corresponding to the four initial intersection point coordinates in the vehicle body world coordinate system. The target translation parameter represents an installation position of the surround-view camera in the vehicle body world coordinate system.

[0148] A calibration module 84 is configured to calibrate camera external parameters for the surround-view camera. The camera external parameters include the target roll angle, the target pitch angle, the target yaw angle, and the target translation parameter.

[0149] Exemplarily, on the calibration cloth, the interval of the two parallel lines on the first side is equal to the interval of the two parallel lines on the second side; the processing module 82 is specifically configured to: obtain an initial roll angle based on the four initial image straight lines, generate a homography matrix based on the initial roll angle, and convert the four initial image straight lines into four physical straight lines in the vehicle world coordinate system based on the homography matrix, when performing the iterative optimization on the roll angle of the surround-view camera based on the four initial image straight lines corresponding to the four physical straight lines in the vehicle world coordinate system; determine a first interval of the two physical straight lines on the first side and a second interval of the two physical straight lines on the second side, the first side being the left side and the second side being the right side, or the first side being the upper side and the second side being the lower side; if the difference between the first interval and the second interval is less than a threshold value, the initial roll angle is taken as the target roll angle; if the difference between the first interval and the second interval is greater than or equal to the threshold value, the initial roll angle is iteratively optimized to obtain an iteratively optimized initial roll angle; and based on the iteratively optimized initial roll angle, the homography matrix is generated based on the initial roll angle.

[0150] Exemplarily, the processing module 82 is specifically configured to: determine an initial pitch angle and an initial yaw angle of the surround-view camera based on the reference intersection coordinates, camera intrinsic parameters and the initial roll angle, when generating the homography matrix based on the initial roll angle; determine initial translation parameters based on the length, width and height of the vehicle to be calibrated, the initial translation parameters including an initial horizontal parameter, an initial vertical parameter and an initial vertical parameter, the initial horizontal parameter being determined based on the length, the initial vertical parameter being determined based on the width, and the initial vertical parameter being determined based on the height; and generate the homography matrix based on the initial roll angle, the initial pitch angle, the initial yaw angle, the initial horizontal parameter, the initial vertical parameter and the initial vertical parameter.

[0151] Exemplarily, the determination module 83 is specifically configured to: determine a target pitch angle and a target yaw angle of the surround-view camera based on the reference intersection coordinates and the target roll angle, when determining the target pitch angle and the target yaw angle of the surround-view camera based on the reference intersection coordinates and the target roll angle.

[0152] The target pitch angle and the target yaw angle are determined by using the following expression:

[0153]

[0154] Wherein, pitch represents the target pitch angle, yaw represents the target yaw angle, roll represents the target roll angle, (u0, v0) represents the reference intersection coordinates, and (fx, fy, cx, cy) represents the camera intrinsic parameters.

[0155] For example, the determining module 83 is specifically configured to determine the target translation parameters based on the target roll angle, the target pitch angle, the target yaw angle, the four initial intersection coordinates, and the physical coordinates corresponding to the four initial intersection coordinates in the vehicle body world coordinate system, by determining a rotation matrix based on the target roll angle, the target pitch angle, and the target yaw angle, selecting at least two initial intersection coordinates from the four initial intersection coordinates as candidate intersection coordinates, and determining target translation extrinsic parameters based on the rotation matrix, the camera intrinsic parameters, the candidate intersection coordinates, and the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system, wherein the target translation extrinsic parameters include target horizontal parameters, target vertical parameters, and target vertical parameters.

[0156] For example, the determining module 83 is specifically configured to determine the target translation parameters based on the rotation matrix, the camera intrinsic parameters, the candidate intersection coordinates, and the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system, by determining the target horizontal parameters, the target vertical parameters, and the target vertical parameters by using the following expressions: (-R 00 fx-R 10 cx+R 10 x′)h+(-R 01 fx-R 11 cx+R 11 x′)w+(-R 02 fx-

[0157] R 12 cx+R 12 x′)z=xx′R 10 +yx′R 11 -x(R 00 fx+R 10 cx)-y(R 01 fx+R 11 cx);

[0158] (R 20 fy-R 10 cy+R 10 y′)h+(R 21 fy-R 11 cy+R 11 y′)w+(R 22 fy-R 12 cy+

[0159] R 12 y′)z=xy′R 10 +yy′R 11 -x(R 10 cy-R 20 fy)-y(R 11 cy-R 21 fy);

[0160] wherein R 00 -R 22 denotes the parameters in the rotation matrix, (fx, fy, cx, cy) denotes the camera intrinsic parameters, (x', y') denotes the candidate intersection coordinates, (x, y) denotes the physical coordinates corresponding to the candidate intersection coordinates in the vehicle world coordinate system, h denotes the target horizontal parameter, w denotes the target vertical parameter, and z denotes the target vertical parameter.

[0161] For example, if the surround-view camera is located on the left side of the vehicle to be calibrated, the calibration cloth is located on the left side of the vehicle to be calibrated, and if the surround-view camera is located on the right side of the vehicle to be calibrated, the calibration cloth is located on the right side of the vehicle to be calibrated; wherein the four parallel lines are straight lines parallel to the vehicle width direction of the vehicle to be calibrated, and the perpendicular lines of the four parallel lines are straight lines perpendicular to the vehicle width direction.

[0162] If the surround-view camera is located in front of the vehicle to be calibrated, the calibration cloth is located on the left side and / or right side of the vehicle to be calibrated, and if the surround-view camera is located behind the vehicle to be calibrated, the calibration cloth is located on the left side and / or right side of the vehicle to be calibrated; wherein the four parallel lines are straight lines parallel to the vehicle length direction of the vehicle to be calibrated, and the perpendicular lines of the four parallel lines are straight lines perpendicular to the vehicle length direction.

[0163] Based on the same application concept as the above method, an electronic device is provided in the embodiments of the present application, as shown in Figure 9 The electronic device includes a processor 91 and a machine-readable storage medium 92, the machine-readable storage medium 92 stores machine-executable instructions that can be executed by the processor 91; the processor 91 is configured to execute the machine-executable instructions to implement the external parameter calibration method disclosed in the above examples of the present application.

[0164] Based on the same application concept as the above method, the embodiments of the present application also provide a machine-readable storage medium, the machine-readable storage medium stores a plurality of computer instructions, and the computer instructions can implement the external parameter calibration method disclosed in the above examples of the present application when executed by a processor.

[0165] The machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device, and can contain or store information such as executable instructions, data, etc. For example, the machine-readable storage medium can be a RAM (Random Access Memory), a volatile memory, a non-volatile memory, a flash memory, a storage drive (such as a hard disk drive), a solid state disk, any type of storage disk (such as an optical disk, a DVD, etc.), or similar storage medium, or a combination thereof.

[0166] The systems, apparatuses, modules or units disclosed in the above embodiments can be implemented by a computer or an entity, or by a product with certain functions. A typical implementation device is a computer, and the specific form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0167] For the convenience of description, the above apparatuses are described in various units by functions respectively. Of course, the functions of the units can be implemented in one or more software and / or hardware in the implementation of the present application.

[0168] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0169] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device implemented in accordance with the flowcharts and / or block diagrams. Figure One The device that implements the function specified in one flow or multiple flows and / or blocks. Figure One The device that implements the function specified in one block or multiple blocks.

[0170] Moreover, these computer program instructions can also be stored in a computer-readable memory capable of directing the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction devices that implement the flowcharts and / or block diagrams. Figure One The device that implements the function specified in one flow or multiple flows and / or blocks. Figure One The device that implements the function specified in one block or multiple blocks.

[0171] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, thus the instructions executed on the computer or other programmable data processing devices provide processes for implementing the functions specified in the flow Figure One one flow or multiple flows and / or blocks Figure One one block or multiple blocks.

[0172] The above only describes the embodiments of the present application and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A method for calibrating an extrinsic parameter, characterized in that, The to-be-calibrated vehicle comprises a surround view camera, a calibration cloth exists around the to-be-calibrated vehicle, and the calibration cloth comprises four parallel lines and four vertical lines of the four parallel lines, comprising: Based on the obtained target image, four initial image straight lines corresponding to the four parallel lines and a reference image straight line corresponding to the vertical line are obtained; wherein the reference image straight line corresponds to four initial intersection coordinates of the four initial image straight lines, and the four initial image straight lines correspond to reference intersection coordinates; Based on the four physical straight lines corresponding to the four initial image straight lines in the vehicle body world coordinate system, the roll angle of the surround view camera is iteratively optimized to obtain a target roll angle; based on the reference intersection coordinates and the target roll angle, a target pitch angle and a target yaw angle of the surround view camera are determined; Based on the target roll angle, the target pitch angle, the target yaw angle, the four initial intersection coordinates, and the physical coordinates corresponding to the four initial intersection coordinates in the vehicle body world coordinate system, a target translation parameter is determined, the target translation parameter representing the installation position of the surround view camera in the vehicle body world coordinate system; The camera extrinsic parameters of the surround view camera are calibrated; wherein the camera extrinsic parameters comprise the target roll angle, the target pitch angle, the target yaw angle and the target translation parameter.

2. The method of claim 1, wherein, On the calibration cloth, the distance between the two parallel lines on the first side is equal to the distance between the two parallel lines on the second side; The iterative optimization of the roll angle of the surround view camera based on the four physical straight lines corresponding to the four initial image straight lines in the vehicle body world coordinate system to obtain the target roll angle comprises: An initial roll angle is obtained, a homography matrix is generated based on the initial roll angle, and the four initial image straight lines are converted into four physical straight lines in the vehicle body world coordinate system based on the homography matrix; A first distance between the two physical straight lines on the first side is determined, and a second distance between the two physical straight lines on the second side is determined, the first side being the left side and the second side being the right side, or the first side being the upper side and the second side being the lower side; If the difference between the first distance and the second distance is less than a threshold value, the initial roll angle is taken as the target roll angle; If the difference between the first distance and the second distance is greater than or equal to the threshold value, the initial roll angle is iteratively optimized to obtain an iteratively optimized initial roll angle; based on the iteratively optimized initial roll angle, the operation of generating a homography matrix based on the initial roll angle is returned.

3. The method of claim 2, wherein: The generation of the homography matrix based on the initial roll angle comprises: Based on the reference intersection coordinates, the camera intrinsic parameters of the surround view camera and the initial roll angle, an initial pitch angle and an initial yaw angle of the surround view camera are determined; Based on the length, width and height of the to-be-calibrated vehicle, an initial translation parameter is determined, the initial translation parameter comprising an initial horizontal parameter, an initial vertical parameter and an initial vertical parameter; wherein the initial horizontal parameter is determined based on the length, the initial vertical parameter is determined based on the width, and the initial vertical parameter is determined based on the height. generate the homography matrix based on the initial roll angle, the initial pitch angle, the initial yaw angle, the initial horizontal parameter, the initial vertical parameter, and the initial upright parameter.

4. The method of claim 1, wherein, The target pitch angle and the target yaw angle of the surround-view camera are determined based on the reference intersection coordinates and the target roll angle, including: The target pitch angle and the target yaw angle are determined using the following expressions: wherein pitch represents the target pitch angle, yaw represents the target yaw angle, roll represents the target roll angle, (u0, v0) represents the reference intersection coordinates, and (fx, fy, cx, cy) represents the camera intrinsic parameters.

5. The method of claim 1, wherein, The target translation parameter is determined based on the target roll angle, the target pitch angle, the target yaw angle, the four initial intersection coordinates, and the physical coordinates corresponding to the four initial intersection coordinates in the vehicle body world coordinate system, including: A rotation matrix is determined based on the target roll angle, the target pitch angle, and the target yaw angle. At least two initial intersection coordinates are selected from the four initial intersection coordinates as candidate intersection coordinates. The target translation parameter is determined based on the rotation matrix, the camera intrinsic parameters of the surround-view camera, the candidate intersection coordinates, and the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system, and the target translation parameter includes a target horizontal parameter, a target vertical parameter, and a target upright parameter.

6. The method of claim 5, wherein, The target translation parameter is determined based on the rotation matrix, the camera intrinsic parameters of the surround-view camera, the candidate intersection coordinates, and the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system, including: The target horizontal parameter, the target vertical parameter, and the target upright parameter are determined using the following expressions: (-R 00 fx-R 10 cx+R 10 x′)h+(-R 01 fx-R 11 cx+R 11 x′)w+(-R 02 fx- R 12 cx+R 12 x′)z=xx′R 10 +yx′R 11 -x(R 00 fx+R 10 cx)-y(R 01 fx+R 11 cx); (R 20 fy-R 10 cy+R 10 y′)h+(R 21 fy-R 11 cy+R 11 y′)w+(R 22 fy-R 12 cy+ R 12 y′)z = xy′R 10 + yy′R 11 - x(R 10 cy-R 20 fy) - y(R 11 cy-R 21 fy); wherein R 00 - R 22 denotes a parameter in the rotation matrix, (fx, fy, cx, cy) denotes camera intrinsic parameters, (x', y') denotes the candidate intersection coordinates, (x, y) denotes the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system, h denotes a target horizontal parameter, w denotes a target vertical parameter, and z denotes a target vertical parameter.

7. The method of any one of claims 1-6, wherein, if the surround-view camera is located on the left side of the vehicle to be calibrated, the calibration cloth is located on the left side of the vehicle to be calibrated, and if the surround-view camera is located on the right side of the vehicle to be calibrated, the calibration cloth is located on the right side of the vehicle to be calibrated; wherein the four parallel lines are straight lines parallel to the vehicle width direction of the vehicle to be calibrated, and the perpendicular lines of the four parallel lines are straight lines perpendicular to the vehicle width direction. if the surround-view camera is located in front of the vehicle to be calibrated, the calibration cloth is located on the left side and / or the right side of the vehicle to be calibrated, and if the surround-view camera is located behind the vehicle to be calibrated, the calibration cloth is located on the left side and / or the right side of the vehicle to be calibrated; wherein the four parallel lines are straight lines parallel to the vehicle length direction of the vehicle to be calibrated, and the perpendicular lines of the four parallel lines are straight lines perpendicular to the vehicle length direction.

8. An external parameter calibration device, characterized in that The vehicle to be calibrated includes a surround-view camera, and a calibration cloth exists around the vehicle to be calibrated, and the calibration cloth includes four parallel lines and four perpendicular lines of the parallel lines, including: An acquisition module is configured to acquire four initial image straight lines corresponding to the four parallel lines and a reference image straight line corresponding to the perpendicular line based on a target image that has been acquired; wherein the reference image straight line corresponds to four initial intersection coordinates, and the four initial image straight lines correspond to reference intersection coordinates. The processing module is configured to perform iterative optimization on the roll angle of the surround-view camera based on the four initial image straight lines corresponding to four physical straight lines in the vehicle world coordinate system, to obtain a target roll angle. The determining module is configured to determine a target pitch angle and a target yaw angle of the surround-view camera based on the reference intersection coordinates and the target roll angle, and determine a target translation parameter based on the target roll angle, the target pitch angle, the target yaw angle, the four initial intersection coordinates, and the physical coordinates corresponding to the four initial intersection coordinates in the vehicle world coordinate system. The calibration module is configured to calibrate camera extrinsic parameters for the surround-view camera, wherein the camera extrinsic parameters include the target roll angle, the target pitch angle, the target yaw angle, and the target translation parameter.

9. The apparatus of claim 8, In some embodiments, In some embodiments, In some embodiments, the first side of the two parallel lines on the calibration board has an equal distance to the second side of the two parallel lines. The processing module is configured to perform iterative optimization on the roll angle of the surround-view camera based on the four initial image straight lines corresponding to four physical straight lines in the vehicle world coordinate system, to obtain a target roll angle. Specifically, the processing module is configured to obtain an initial roll angle, generate a homography matrix based on the initial roll angle, and convert the four initial image straight lines into four physical straight lines in the vehicle world coordinate system based on the homography matrix. The processing module is further configured to determine a first distance between the two physical straight lines on the first side, and determine a second distance between the two physical straight lines on the second side. The first side is the left side, and the second side is the right side, or the first side is the upper side, and the second side is the lower side. If the difference between the first distance and the second distance is less than a threshold value, the initial roll angle is taken as the target roll angle. If the difference between the first distance and the second distance is greater than or equal to the threshold value, the initial roll angle is iteratively optimized to obtain an iteratively optimized initial roll angle. The operation of generating a homography matrix based on the initial roll angle is performed again based on the iteratively optimized initial roll angle. In some embodiments, the processing module is configured to generate a homography matrix based on the initial roll angle. Specifically, the processing module is configured to determine an initial pitch angle and an initial yaw angle of the surround-view camera based on the reference intersection coordinates, camera intrinsic parameters, and the initial roll angle. The processing module is further configured to determine an initial translation parameter based on the length, width, and height of the vehicle to be calibrated. The initial translation parameter includes an initial horizontal parameter, an initial vertical parameter, and an initial vertical parameter. The initial horizontal parameter is determined based on the length of the vehicle, the initial vertical parameter is determined based on the width of the vehicle, and the initial vertical parameter is determined based on the height of the vehicle. The processing module is further configured to generate the homography matrix based on the initial roll angle, the initial pitch angle, the initial yaw angle, the initial horizontal parameter, the initial vertical parameter, and the initial vertical parameter. In some embodiments, the determining module is configured to determine a target pitch angle and a target yaw angle of the surround-view camera based on the reference intersection coordinates and the target roll angle. Specifically, the determining module is configured to determine the target pitch angle and the target yaw angle based on the following expressions: In some embodiments, the determining module is configured to determine a target pitch angle and a target yaw angle of the surround-view camera based on the reference intersection coordinates and the target roll angle. Specifically, the determining module is configured to determine the target pitch angle and the target yaw angle based on the following expressions: Wherein, pitch represents the target pitch angle, yaw represents the target yaw angle, roll represents the target roll angle, (u0, v0) represents the reference intersection coordinates, (fx, fy, cx, cy) represents the camera intrinsic parameters; Wherein, the determining module is configured to determine the target translation parameters based on the target roll angle, the target pitch angle, the target yaw angle, the four initial intersection coordinates, and the physical coordinates corresponding to the four initial intersection coordinates in the vehicle body world coordinate system, specifically by: determining a rotation matrix based on the target roll angle, the target pitch angle, and the target yaw angle; selecting at least two initial intersection coordinates from the four initial intersection coordinates as candidate intersection coordinates; determining target translation extrinsic parameters based on the rotation matrix, the camera intrinsic parameters, the candidate intersection coordinates, and the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system, wherein the target translation extrinsic parameters include target horizontal parameters, target vertical parameters, and target vertical parameters; Wherein, the determining module is configured to determine the target translation parameters based on the rotation matrix, the camera intrinsic parameters, the candidate intersection coordinates, and the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system, specifically by: determining the target horizontal parameters, the target vertical parameters, and the target vertical parameters using the following expression: (-R 00 fx-R 10 cx+R 10 x')h+(-R 01 fx-R 11 cx+R 11 x')w+(-R 02 fx- R 12 cx+R 12 x′)z=xx′R 10 +yx′R 11 -x(R 00 fx+R 10 cx)-y(R 01 fx+R 11 cx); (R 20 fy-R 10 cy+R 10 y')h+(R 21 fy-R 11 cy+R 11 y')w+(R 22 fy-R 12 cy+ R 12 y′)z = xy′R 10 + yy′R 11 - x(R 10 cy-R 20 fy)- y(R 11 cy-R 21 fy); wherein R 00 - R 22 denotes a parameter in the rotation matrix, (fx, fy, cx, cy) denotes camera intrinsic parameters, (x', y') denotes the candidate intersection coordinates, (x, y) denotes the physical coordinates corresponding to the candidate intersection coordinates in the vehicle body world coordinate system, h denotes a target horizontal parameter, w denotes a target vertical parameter, and z denotes a target vertical parameter; Wherein, if the surround-view camera is located on the left side of the vehicle to be calibrated, the calibration cloth is located on the left side of the vehicle to be calibrated, and if the surround-view camera is located on the right side of the vehicle to be calibrated, the calibration cloth is located on the right side of the vehicle to be calibrated; wherein the four parallel lines are straight lines parallel to the vehicle width direction of the vehicle to be calibrated, and the perpendicular line of the four parallel lines is a straight line perpendicular to the vehicle width direction. If the surround-view camera is located in front of the vehicle to be calibrated, the calibration cloth is located on the left side and / or right side of the vehicle to be calibrated, and if the surround-view camera is located behind the vehicle to be calibrated, the calibration cloth is located on the left side and / or right side of the vehicle to be calibrated; wherein the four parallel lines are straight lines parallel to the vehicle length direction of the vehicle to be calibrated, and the perpendicular line of the four parallel lines is a straight line perpendicular to the vehicle length direction.

10. An electronic device, comprising: Comprising: a processor and a machine readable storage medium storing machine executable instructions executable by the processor; the processor is configured to execute the machine executable instructions to implement the method of any one of claims 1-7.

Images