Methods, devices and electronic equipment for calibrating camera extrinsic parameters

Abstract

This application discloses a method, apparatus, and electronic device for calibrating camera extrinsic parameters. The method includes: acquiring road perception information transmitted by roadside equipment, the road perception information including first angle information of an autonomous vehicle relative to lane lines, and the roadside equipment being at least one from a preset set of roadside equipment; acquiring a first road image captured by the camera of the autonomous vehicle, and performing lane line detection on the first road image to obtain a first lane line detection result, the first lane line detection result including first lane line edge detection results; determining a first camera extrinsic parameter correction value for the autonomous vehicle based on the road perception information and the first lane line detection result; and determining a calibration result for the camera extrinsic parameters based on the first camera extrinsic parameter correction value. This application corrects the camera extrinsic parameters of the autonomous vehicle by using a pre-established and regularly maintained set of roadside equipment, eliminating the need for calibration in a fixed calibration workshop or external calibration site, thus improving calibration efficiency and accuracy.

Description

Technical Field

[0001] This application relates to the field of camera calibration technology, and in particular to a method, apparatus and electronic device for calibrating camera extrinsic parameters. Background Technology

[0002] In the camera suite of autonomous vehicles, the forward-looking monocular camera, which serves as the "eyes" of the autonomous vehicle, includes multiple perception tasks, such as lane line recognition, traffic light recognition, and object recognition. The recognition accuracy also has a direct impact on the implementation of downstream modules such as lane line positioning and traffic light-based driving strategies.

[0003] With the development of autonomous driving technology and hardware, traditional outdoor manual calibration methods, such as the Zhang Zhengyou calibration method, are gradually being replaced by automated calibration methods. Simultaneous calibration of multiple sensors, including monocular cameras, is carried out in calibration workshops that include calibration platforms, fixed-position calibration boards, lighting equipment, etc.

[0004] However, the above calibration scheme has at least the following technical problems:

[0005] 1) Although calibration platforms are convenient and efficient, they are very expensive. Autonomous vehicles often operate in multiple cities, and the cost of going to a fixed calibration workshop each time or establishing a calibration workshop in each operating city is very high.

[0006] 2) Although outdoor manual calibration has low hardware costs, it has high labor costs. It requires dedicated calibration personnel to collect data on-site for calibration, and it cannot meet the needs of real-time correction when external parameters need to be calibrated. Summary of the Invention

[0007] This application provides a method, apparatus, and electronic device for calibrating camera extrinsic parameters, so as to improve the calibration efficiency and accuracy of camera extrinsic parameters.

[0008] The embodiments of this application adopt the following technical solutions:

[0009] In a first aspect, embodiments of this application provide a method for calibrating camera extrinsic parameters, wherein the method includes:

[0010] Acquire road perception information sent by roadside equipment, the road perception information including the first angle information of the autonomous vehicle relative to the lane line, and the roadside equipment is at least one of a preset set of roadside equipment;

[0011] A first road image captured by the camera of an autonomous vehicle is acquired, and lane line detection is performed on the first road image to obtain a first lane line detection result, which includes a first lane line edge detection result.

[0012] The first camera extrinsic parameter correction value of the autonomous vehicle is determined based on the road perception information and the first lane line detection result.

[0013] The calibration result of the camera extrinsic parameters is determined based on the first camera extrinsic parameter correction value.

[0014] Optionally, the preset roadside equipment set is obtained in the following manner:

[0015] Acquire road images captured by cameras of roadside devices within a preset area, and perform lane line detection on the road images captured by the cameras of the roadside devices to obtain a second lane line detection result. The second lane line detection result includes lane line type and second lane line edge detection result.

[0016] The corner detection results of the second lane line are determined based on the edge detection results of the second lane line.

[0017] If the lane line type is a dashed lane line, the deviation of the second lane line edge detection result is less than a first preset deviation threshold, and the deviation of the second lane line corner detection result is less than a second preset deviation threshold, then the roadside device is added to the preset roadside device set.

[0018] Optionally, the first camera extrinsic parameter correction value includes a heading angle correction value, and determining the first camera extrinsic parameter correction value for the autonomous vehicle based on the road perception information and the first lane detection result includes:

[0019] The first lane line edge detection result is subjected to inverse perspective transformation to obtain the second included angle information of the autonomous vehicle relative to the lane line;

[0020] The heading angle correction value is determined based on the first angle information of the autonomous vehicle relative to the lane line and the second angle information of the autonomous vehicle relative to the lane line.

[0021] Optionally, the first camera extrinsic parameter correction value includes a heading angle correction value and a pitch angle correction value, and determining the first camera extrinsic parameter correction value of the autonomous vehicle based on the road perception information and the first lane detection result includes:

[0022] Based on the heading angle correction value, the first lane line edge detection result is subjected to inverse perspective transformation processing;

[0023] Based on the results of the inverse perspective transformation, straight line fitting is performed on the lane line edges to obtain multiple straight line fitting equations for the lane line edges.

[0024] Based on the straight line fitting equations of multiple lane line edges, determine the straight line slope error between multiple lane line edges;

[0025] The pitch angle correction value is determined based on the straight slope error between the edges of multiple lane lines.

[0026] Optionally, the road perception information further includes the absolute position of the lane line corner points detected by the roadside equipment, and the first camera extrinsic parameter correction values ​​include heading angle correction values, pitch angle correction values, and roll angle correction values. Determining the first camera extrinsic parameter correction values ​​for the autonomous vehicle based on the road perception information and the first lane line detection results includes:

[0027] The detection results of the first lane line corner points are determined based on the first lane line edge detection results.

[0028] The absolute position of the lane corner detected by the autonomous vehicle is determined based on the first lane corner detection result and the camera parameters of the autonomous vehicle.

[0029] Based on the heading angle correction value and the pitch angle correction value, the roll angle correction value is determined according to the absolute position of the lane line corner point detected by the autonomous vehicle and the absolute position of the lane line corner point detected by the roadside equipment.

[0030] Optionally, the first camera extrinsic parameter correction value includes a first camera extrinsic parameter correction value obtained based on multiple roadside devices, and the determination of the camera extrinsic parameter calibration result based on the first camera extrinsic parameter correction value includes:

[0031] The first camera extrinsic parameter correction values ​​obtained from multiple roadside devices are fused to obtain the fused camera extrinsic parameter correction values.

[0032] The calibration result of the camera extrinsic parameters is determined based on the fused camera extrinsic parameter correction values.

[0033] Optionally, determining the calibration result of the camera extrinsic parameters based on the first camera extrinsic parameter correction value includes:

[0034] Obtain the second camera extrinsic parameter correction value, which is obtained based on other roadside devices outside the preset roadside device set;

[0035] Determine the error between the first camera extrinsic parameter correction value and the second camera extrinsic parameter correction value;

[0036] The first camera extrinsic parameter correction value is verified based on the error between the first camera extrinsic parameter correction value and the second camera extrinsic parameter correction value to obtain a first verification result;

[0037] The calibration result of the camera extrinsic parameters is determined based on the first verification result.

[0038] Optionally, determining the calibration result of the camera extrinsic parameters based on the first camera extrinsic parameter correction value includes:

[0039] Acquire a second road image captured by the camera of the autonomous vehicle, and perform lane line detection on the second road image to obtain a third lane line detection result;

[0040] The corresponding high-precision map data is obtained based on the current location of the autonomous vehicle, and the lane lines in the high-precision map data are projected onto the second road image to obtain the lane line projection result.

[0041] The correction value of the first camera extrinsic parameter is verified based on the third lane line detection result and the lane line projection result to obtain the second verification result.

[0042] The calibration result of the camera extrinsic parameters is determined based on the second verification result.

[0043] Secondly, embodiments of this application also provide a camera extrinsic parameter calibration device, wherein the device includes:

[0044] An acquisition unit is used to acquire road perception information sent by roadside equipment, the road perception information including the first angle information of the autonomous vehicle relative to the lane line, and the roadside equipment is at least one of a preset set of roadside equipment.

[0045] The lane detection unit is used to acquire a first road image captured by the camera of the autonomous vehicle, and to perform lane detection on the first road image to obtain a first lane detection result, which includes a first lane edge detection result.

[0046] The first determining unit is used to determine the first camera extrinsic parameter correction value of the autonomous vehicle based on the road perception information and the first lane line detection result.

[0047] The second determining unit is used to determine the calibration result of the camera extrinsic parameters based on the first camera extrinsic parameter correction value.

[0048] Thirdly, embodiments of this application also provide an electronic device, including:

[0049] Processor; and

[0050] A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform any of the methods described above.

[0051] Fourthly, embodiments of this application also provide a computer-readable storage medium that stores one or more programs, which, when executed by an electronic device including multiple applications, cause the electronic device to perform any of the methods described above.

[0052] The above-mentioned at least one technical solution adopted in the embodiments of this application can achieve the following beneficial effects: The camera extrinsic parameter calibration method of the embodiments of this application first acquires road perception information sent by roadside equipment, the road perception information including the first angle information of the autonomous vehicle relative to the lane line, and the roadside equipment is at least one in a preset set of roadside equipment; then, it acquires a first road image captured by the camera of the autonomous vehicle, and performs lane line detection on the first road image to obtain a first lane line detection result, the first lane line detection result including the first lane line edge detection result; then, it determines the first camera extrinsic parameter correction value of the autonomous vehicle based on the road perception information and the first lane line detection result; finally, it determines the camera extrinsic parameter calibration result based on the first camera extrinsic parameter correction value. The camera extrinsic parameter calibration method of the embodiments of this application corrects the camera extrinsic parameters of the autonomous vehicle through a pre-established and regularly maintained set of roadside equipment, without the need to use a fixed calibration workshop or external calibration site for calibration, thus improving the calibration efficiency and accuracy of camera extrinsic parameters. Attached Figure Description

[0053] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0054] Figure 1 This is a flowchart illustrating a method for calibrating camera extrinsic parameters according to an embodiment of this application.

[0055] Figure 2 This is a schematic diagram of the structure of a camera extrinsic parameter calibration device according to an embodiment of this application;

[0056] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0058] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0059] This application provides a method for calibrating camera extrinsic parameters, such as... Figure 1 The diagram shows a flowchart of a camera extrinsic parameter calibration method according to an embodiment of this application. The method includes at least the following steps S110 to S140:

[0060] Step S110: Obtain road perception information sent by roadside equipment, the road perception information including the first angle information of the autonomous vehicle relative to the lane line, and the roadside equipment is at least one of a preset set of roadside equipment.

[0061] The camera extrinsic parameter calibration method in this application is mainly used to calibrate the camera extrinsic parameters of autonomous vehicles. Here, the camera extrinsic parameters mainly refer to the attitude extrinsic parameters in the extrinsic parameter transformation matrix from the camera to the vehicle body of the autonomous vehicle. After the initial calibration of the extrinsic and intrinsic parameters of the autonomous vehicle, or if the camera position changes, i.e., the rigid body connection between the camera and the vehicle body changes, one or more roadside devices from the preset roadside device set can be selected to calibrate the camera attitude extrinsic parameters.

[0062] When calibrating the camera's attitude extrinsic parameters, it is necessary to first obtain the road perception information sent by the roadside equipment. Here, the roadside equipment can be understood as the roadside equipment corresponding to the current parking position of the autonomous vehicle, that is, the autonomous vehicle is currently within the perception range of the roadside equipment. The roadside equipment can detect road information within its perception range, including vehicle targets and road surface markings such as lane lines, and thus calculate the first angle information of the autonomous vehicle relative to the lane line.

[0063] The roadside equipment in this application embodiment is at least one of a preset roadside equipment set. The preset roadside equipment set can be understood as a set containing unique identifiers and absolute positions of roadside equipment selected based on certain screening conditions. The roadside equipment in the preset roadside equipment set can be roadside equipment that meets the screening conditions such as flat road surface within the perception range, clear and parallel dashed lane lines, and accurate identification of lane lines. This can provide reliable support for the correction of camera extrinsic parameters of autonomous vehicles, thereby solving the problem of high cost and low efficiency caused by existing calibration schemes that rely on fixed calibration workshops or external calibration sites for calibration.

[0064] Step S120: Obtain a first road image captured by the camera of the autonomous vehicle, and perform lane line detection on the first road image to obtain a first lane line detection result, which includes a first lane line edge detection result.

[0065] This application embodiment also requires acquiring a first road image captured by the camera of the autonomous vehicle, and then using a lane line detection algorithm to detect lane lines in the first road image. Since a major feature of lane lines in the image is that the edges are relatively obvious, this application embodiment can use an edge detection algorithm such as the Canny algorithm to extract the lane line edges in the image to obtain the first lane line edge detection result.

[0066] Step S130: Determine the first camera extrinsic parameter correction value of the autonomous vehicle based on the road perception information and the first lane line detection result.

[0067] Since the roadside equipment in this application embodiment is a roadside equipment that meets certain requirements from a preset set of roadside equipment, it can provide more accurate road perception information from the perspective of the roadside equipment. Therefore, based on the road perception information of the roadside equipment obtained by the aforementioned steps and the first lane detection result perceived by the autonomous vehicle, the camera extrinsic parameter correction value of the autonomous vehicle can be calculated in this application embodiment. The camera extrinsic parameter correction value may include, for example, heading angle correction value, pitch angle correction value and roll angle correction value.

[0068] Step S140: Determine the calibration result of the camera extrinsic parameters based on the first camera extrinsic parameter correction value.

[0069] To ensure the accuracy and reliability of the camera extrinsic parameter correction values, embodiments of this application may adopt certain verification strategies to verify the above-mentioned camera extrinsic parameter correction values, thereby determining the final calibration result of the camera extrinsic parameters based on the verification results.

[0070] The camera extrinsic parameter calibration method of this application corrects the camera extrinsic parameters of autonomous vehicles by using a pre-established and regularly maintained set of roadside equipment. It does not require the use of a fixed calibration workshop or external calibration site for calibration, thus improving the calibration efficiency and accuracy of camera extrinsic parameters.

[0071] In some embodiments of this application, the preset roadside equipment set is obtained by: acquiring road images captured by cameras of roadside equipment within a preset area, and performing lane line detection on the road images captured by the cameras of the roadside equipment to obtain a second lane line detection result, the second lane line detection result including lane line type and second lane line edge detection result; determining a second lane line corner detection result based on the second lane line edge detection result; and adding the roadside equipment to the preset roadside equipment set when the lane line type is a dashed lane line, the deviation of the second lane line edge detection result is less than a first preset deviation threshold, and the deviation of the second lane line corner detection result is less than a second preset deviation threshold.

[0072] In constructing a preset set of roadside devices, this application embodiment can analyze the road conditions of all roadside devices in the operating area of ​​autonomous vehicles and the camera imaging results on the roadside devices to find roadside devices that meet the calibration requirements.

[0073] Specifically, road images captured by the camera of the roadside device can be acquired and lane lines can be detected to obtain the lane line type and lane line edge detection results. If the lane line type is a dashed lane line, the lane line corner position can be further calculated based on the lane line edge detection results to obtain the lane line corner detection results. If the deviation of the lane line edge detection results and the deviation of the lane line corner detection results are both within the corresponding deviation threshold range, it indicates that the roadside device can provide an accurate basis for subsequent correction of camera extrinsic parameters. Its corresponding roadside device ID and absolute position information can be recorded in the preset roadside device set.

[0074] The above method can be used to find roadside equipment with flat surfaces and clear, parallel dashed lane lines, ensuring that the lane line edge detection results of the roadside equipment are correct. Then, the accurate lane line corner position can be calculated based on the lane line edge detection results, ensuring that the deviation of the lane line corner position is within a preset number of error pixels. The specific number of pixels can be flexibly set according to the image resolution, and is not specifically limited here.

[0075] In some embodiments of this application, the first camera extrinsic parameter correction value includes a heading angle correction value. Determining the first camera extrinsic parameter correction value of the autonomous vehicle based on the road perception information and the first lane line detection result includes: performing inverse perspective transformation processing on the first lane line edge detection result to obtain the second included angle information of the autonomous vehicle relative to the lane line; and determining the heading angle correction value based on the first included angle information of the autonomous vehicle relative to the lane line and the second included angle information of the autonomous vehicle relative to the lane line.

[0076] The camera extrinsic parameter correction values ​​in this application embodiment may include, for example, a heading angle correction value. The heading angle can be understood as the angle between the vehicle and the lane line. The roadside equipment captures road images containing only the stationary state of the autonomous vehicle, and projects the detected vehicle's bounding box and lane line recognition results onto the top view to calculate the first angle information of the vehicle body relative to the lane line. If the vehicle body is parallel to the lane line, the angle is 0°.

[0077] The first lane edge detection result perceived by the autonomous vehicle can be used to calculate the second angle information of the vehicle body relative to the lane line through inverse perspective mapping (IPM).

[0078] That is, the first included angle information is obtained based on the perception results of the roadside equipment, which can be considered to be sufficiently accurate angle information. The second included angle information is obtained based on the perception results of the autonomous vehicle. If there is no deviation in the camera extrinsic parameters of the autonomous vehicle, the first included angle information and the second included angle information should be basically consistent. Therefore, the embodiments of this application can perform iterative optimization calculation of the heading angle correction value based on the deviation between the first included angle information and the second included angle information, thereby obtaining the optimized heading angle correction value.

[0079] In some embodiments of this application, the first camera extrinsic parameter correction value includes a heading angle correction value and a pitch angle correction value. Determining the first camera extrinsic parameter correction value for the autonomous vehicle based on the road perception information and the first lane line detection result includes: performing inverse perspective transformation processing on the first lane line edge detection result based on the heading angle correction value; performing straight line fitting on the lane line edges based on the inverse perspective transformation processing result to obtain straight line fitting equations for multiple lane line edges; determining the straight line slope error between multiple lane line edges based on the straight line fitting equations for multiple lane line edges; and determining the pitch angle correction value based on the straight line slope error between multiple lane line edges.

[0080] The first camera extrinsic parameter correction value in this embodiment may further include a pitch angle correction value. Since the accuracy of the heading angle calibration affects the pitch angle calibration result, and the pitch angle calibration deviation mainly affects the parallel relationship between lane lines perceived by the autonomous vehicle, this embodiment can further calibrate the pitch angle based on the heading angle correction value calibrated in the aforementioned embodiment. That is, based on the heading angle correction value, IPM projection is performed on the first lane line edge detection result, and straight line fitting of the lane line edge is performed according to the IPM projection result to obtain multiple straight line fitting equations for lane line edges. Ideally, if the pitch angle extrinsic parameter of the autonomous vehicle is accurate, the slope of the straight line fitting equations of the multiple lane line edges obtained should be the same, that is, they should be parallel to each other. Therefore, the straight line slope error between multiple lane line edges can be determined based on the straight line slope of the straight line fitting equations of multiple lane line edges. By adjusting the pitch angle correction value, the straight line slope error between multiple lane line edges is minimized, thereby obtaining the optimal pitch angle correction value.

[0081] To further improve the accuracy and reliability of pitch angle correction values, IPM projection and straight line fitting can be performed on the first lane edge detection results perceived by the autonomous vehicle in multiple frames. The pitch angle correction value can then be calculated based on the straight line fitting equations of multiple lane edge detection results corresponding to the first lane edge detection results in multiple frames.

[0082] Furthermore, it should be noted that when constructing optimization constraints based on the slope error between multiple lane line edges, if the number of lane line edges perceived by the autonomous vehicle is small, the slope of the straight line obtained by fitting the lane line edges perceived by the roadside equipment can be used as a benchmark. The optimal pitch angle correction value is obtained by minimizing the error between the slope of the straight line corresponding to the lane line edge of the autonomous vehicle and the slope of the straight line edge corresponding to the roadside equipment.

[0083] In some embodiments of this application, the road perception information further includes the absolute position of the lane line corner points detected by the roadside equipment, and the first camera extrinsic parameter correction value includes a heading angle correction value, a pitch angle correction value, and a roll angle correction value. Determining the first camera extrinsic parameter correction value of the autonomous vehicle based on the road perception information and the first lane line detection result includes: determining the first lane line corner point detection result based on the first lane line edge detection result; determining the absolute position of the lane line corner points detected by the autonomous vehicle based on the first lane line corner point detection result and the camera parameters of the autonomous vehicle; and determining the roll angle correction value based on the heading angle correction value and the pitch angle correction value, and based on the absolute position of the lane line corner points detected by the autonomous vehicle and the absolute position of the lane line corner points detected by the roadside equipment.

[0084] The road perception information in this application embodiment also includes the absolute position of the lane line corner point detected by the roadside device. The absolute position of the lane line corner point detected by the roadside device can be obtained by assigning an absolute position value to the corner point of the dashed lane line identified by the roadside device based on the absolute position of the roadside device and the camera parameters of the roadside device.

[0085] The absolute position of the lane line corner point perceived by the autonomous vehicle can be obtained based on the lane line edge detection results and the camera parameters of the autonomous vehicle. Ideally, if the roll angle extrinsic parameter of the autonomous vehicle is accurate, then the absolute position of the lane line corner point perceived by the roadside equipment should be consistent with the absolute position of the lane line corner point perceived by the autonomous vehicle. Therefore, in this embodiment, the optimal roll angle correction value can be obtained by minimizing the deviation between the absolute position of the lane line corner point perceived by the roadside equipment and the absolute position of the lane line corner point perceived by the autonomous vehicle.

[0086] In some embodiments of this application, the first camera extrinsic parameter correction value includes a first camera extrinsic parameter correction value obtained based on multiple roadside devices, and the step of determining the calibration result of the camera extrinsic parameter based on the first camera extrinsic parameter correction value includes: fusing the first camera extrinsic parameter correction values ​​obtained based on multiple roadside devices to obtain a fused camera extrinsic parameter correction value; and determining the calibration result of the camera extrinsic parameter based on the fused camera extrinsic parameter correction value.

[0087] Considering that the camera extrinsic parameter correction values ​​obtained based on a single roadside device may not be accurate or reliable enough, this application embodiment can further obtain multiple first camera extrinsic parameter correction values ​​calculated by the autonomous vehicle within the perception range of multiple roadside devices through the aforementioned embodiment. Then, a certain fusion strategy is adopted to fuse the multiple first camera extrinsic parameter correction values. For example, the average value of the multiple first camera extrinsic parameter correction values ​​can be directly calculated to determine the fused camera extrinsic parameter correction value, or a weighted average method can be adopted to determine the fused camera extrinsic parameter correction value. The weight can be determined based on the reliability of the perception results of different roadside devices. The higher the reliability, the greater the corresponding weight. Finally, the calibration result of the camera extrinsic parameters is determined based on the fused camera extrinsic parameter correction value.

[0088] In some embodiments of this application, determining the calibration result of the camera extrinsic parameters based on the first camera extrinsic parameter correction value includes: obtaining a second camera extrinsic parameter correction value, the second camera extrinsic parameter correction value being obtained based on other roadside devices outside the preset roadside device set; determining the error between the first camera extrinsic parameter correction value and the second camera extrinsic parameter correction value; verifying the first camera extrinsic parameter correction value based on the error between the first camera extrinsic parameter correction value and the second camera extrinsic parameter correction value to obtain a first verification result; and determining the calibration result of the camera extrinsic parameters based on the first verification result.

[0089] To determine the accuracy of the camera extrinsic parameter correction values, embodiments of this application may also employ certain verification strategies to verify the camera extrinsic parameter correction values. For example, this can be implemented using roadside devices other than the preset set of roadside devices. The second camera extrinsic parameter correction value calculated based on the road perception results of other roadside devices is compared with the first camera extrinsic parameter correction value calculated based on the road perception results of roadside devices in the preset set of roadside devices. If the error between the two is less than a preset error threshold, it indicates that the first camera extrinsic parameter correction value is sufficiently reliable and accurate, and the calibration result of this camera extrinsic parameter is considered successful; otherwise, the calibration is considered unsuccessful.

[0090] In some embodiments of this application, determining the calibration result of the camera extrinsic parameters based on the first camera extrinsic parameter correction value includes: acquiring a second road image captured by the camera of the autonomous vehicle, and performing lane line detection on the second road image to obtain a third lane line detection result; acquiring corresponding high-precision map data based on the current position of the autonomous vehicle, and projecting the lane lines in the high-precision map data onto the second road image to obtain a lane line projection result; verifying the first camera extrinsic parameter correction value based on the third lane line detection result and the lane line projection result to obtain a second verification result; and determining the calibration result of the camera extrinsic parameters based on the second verification result.

[0091] In addition to the verification strategies described above, this application embodiment can also perform verification based on high-precision map data. First, a second road image captured by the autonomous vehicle's camera in a stationary state is acquired, and lane line edge detection is performed to obtain a third lane line detection result. Then, local high-precision map data corresponding to the current position of the autonomous vehicle is acquired. The current position can be obtained based on the autonomous vehicle's RTK positioning or fusion positioning module. The local high-precision map data contains lane lines and other data in the roads near the autonomous vehicle. Therefore, based on the calibrated camera extrinsic parameters, the lane lines in the local high-precision map data can be projected onto the second road image. Then, the lane line projection result is compared with the lane line edge detection result. If the lane line projection result falls within the middle area of ​​the left and right lane line edges corresponding to the same lane line, the camera extrinsic parameter calibration result is successful; otherwise, the calibration fails.

[0092] Of course, to further improve the accuracy of the verification results, a multi-frame data verification method can also be adopted. That is, if the lane lines in the local high-precision map data can be projected into the lane lines in the multi-frame road images when the vehicle is stationary, the calibration is considered successful. It should be noted that, due to the influence of perception capabilities, road factors, etc., the high-precision map data in this embodiment is only used as a verification strategy and is not used in the actual calibration process.

[0093] In summary, the camera extrinsic parameter calibration method of this application has achieved at least the following technical effects:

[0094] 1) By pre-establishing and regularly maintaining a set of roadside equipment, the impact of failing to correctly identify lane lines, road slopes, etc. is minimized, reducing the risk of calculation failure or large errors caused by the lack of effective specific markers during actual calculations;

[0095] 2) Using the perception results of roadside equipment as reference information, the three attitude extrinsic parameters of the autonomous vehicle are automatically corrected without human intervention;

[0096] 3) The scheme of multiple calibrations plus at least one verification ensures the correctness and reliability of the calibration results.

[0097] This application embodiment also provides a camera extrinsic parameter calibration device 200, such as... Figure 2 The diagram shows a schematic representation of a camera extrinsic parameter calibration device according to an embodiment of this application. The device 200 includes:

[0098] The acquisition unit 210 is used to acquire road perception information sent by roadside equipment, the road perception information including the first angle information of the autonomous vehicle relative to the lane line, and the roadside equipment is at least one of a preset set of roadside equipment.

[0099] The lane detection unit 220 is used to acquire a first road image captured by the camera of the autonomous vehicle, and to perform lane detection on the first road image to obtain a first lane detection result, wherein the first lane detection result includes a first lane edge detection result.

[0100] The first determining unit 230 is used to determine the first camera extrinsic parameter correction value of the autonomous vehicle based on the road perception information and the first lane line detection result.

[0101] The second determining unit 240 is used to determine the calibration result of the camera extrinsic parameters based on the first camera extrinsic parameter correction value.

[0102] In some embodiments of this application, the preset roadside equipment set is obtained by: acquiring road images captured by cameras of roadside equipment within a preset area, and performing lane line detection on the road images captured by the cameras of the roadside equipment to obtain a second lane line detection result, the second lane line detection result including lane line type and second lane line edge detection result; determining a second lane line corner detection result based on the second lane line edge detection result; and adding the roadside equipment to the preset roadside equipment set when the lane line type is a dashed lane line, the deviation of the second lane line edge detection result is less than a first preset deviation threshold, and the deviation of the second lane line corner detection result is less than a second preset deviation threshold.

[0103] In some embodiments of this application, the first camera extrinsic parameter correction value includes a heading angle correction value, and the first determining unit 230 is specifically used to: perform inverse perspective transformation processing on the first lane line edge detection result to obtain the second included angle information of the autonomous vehicle relative to the lane line; and determine the heading angle correction value based on the first included angle information of the autonomous vehicle relative to the lane line and the second included angle information of the autonomous vehicle relative to the lane line.

[0104] In some embodiments of this application, the first camera extrinsic parameter correction value includes a heading angle correction value and a pitch angle correction value. The first determining unit 230 is specifically used to: perform inverse perspective transformation processing on the first lane line edge detection result based on the heading angle correction value; perform straight line fitting of the lane line edge according to the inverse perspective transformation processing result to obtain a straight line fitting equation of multiple lane line edges; determine the straight line slope error between multiple lane line edges according to the straight line fitting equation of multiple lane line edges; and determine the pitch angle correction value according to the straight line slope error between multiple lane line edges.

[0105] In some embodiments of this application, the road perception information further includes the absolute position of the lane line corner point detected by the roadside equipment, and the first camera extrinsic parameter correction value includes a heading angle correction value, a pitch angle correction value, and a roll angle correction value. The first determining unit 230 is specifically used to: determine the first lane line corner point detection result based on the first lane line edge detection result; determine the absolute position of the lane line corner point detected by the autonomous vehicle based on the first lane line corner point detection result and the camera parameters of the autonomous vehicle; and determine the roll angle correction value based on the heading angle correction value and the pitch angle correction value, and based on the absolute position of the lane line corner point detected by the autonomous vehicle and the absolute position of the lane line corner point detected by the roadside equipment.

[0106] In some embodiments of this application, the first camera extrinsic parameter correction value includes a first camera extrinsic parameter correction value obtained based on multiple roadside devices. The second determining unit 240 is specifically used to: fuse the first camera extrinsic parameter correction values ​​obtained based on multiple roadside devices to obtain a fused camera extrinsic parameter correction value; and determine the calibration result of the camera extrinsic parameters based on the fused camera extrinsic parameter correction value.

[0107] In some embodiments of this application, the second determining unit 240 is specifically used to: obtain a second camera extrinsic parameter correction value, the second camera extrinsic parameter correction value being obtained based on other roadside devices outside the preset roadside device set; determine the error between the first camera extrinsic parameter correction value and the second camera extrinsic parameter correction value; verify the first camera extrinsic parameter correction value based on the error between the first camera extrinsic parameter correction value and the second camera extrinsic parameter correction value to obtain a first verification result; and determine the calibration result of the camera extrinsic parameter based on the first verification result.

[0108] In some embodiments of this application, the second determining unit 240 is specifically used for: acquiring a second road image captured by the camera of the autonomous vehicle, and performing lane line detection on the second road image to obtain a third lane line detection result; acquiring corresponding high-precision map data based on the current position of the autonomous vehicle, and projecting the lane lines in the high-precision map data onto the second road image to obtain a lane line projection result; verifying the first camera extrinsic parameter correction value based on the third lane line detection result and the lane line projection result to obtain a second verification result; and determining the calibration result of the camera extrinsic parameters based on the second verification result.

[0109] It is understood that the above-mentioned camera extrinsic parameter calibration device can implement each step of the camera extrinsic parameter calibration method provided in the foregoing embodiments. The relevant explanations of the camera extrinsic parameter calibration method are applicable to the camera extrinsic parameter calibration device, and will not be repeated here.

[0110] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 3 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0111] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 3 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0112] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0113] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a camera extrinsic parameter calibration device at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0114] Acquire road perception information sent by roadside equipment, the road perception information including the first angle information of the autonomous vehicle relative to the lane line, and the roadside equipment is at least one of a preset set of roadside equipment;

[0115] A first road image captured by the camera of an autonomous vehicle is acquired, and lane line detection is performed on the first road image to obtain a first lane line detection result, which includes a first lane line edge detection result.

[0116] The first camera extrinsic parameter correction value of the autonomous vehicle is determined based on the road perception information and the first lane line detection result.

[0117] The calibration result of the camera extrinsic parameters is determined based on the first camera extrinsic parameter correction value.

[0118] The above is as stated in this application. Figure 1The method executed by the camera extrinsic parameter calibration device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0119] The electronic device can also perform Figure 1 A method for calibrating camera extrinsic parameters, and implementation of the camera extrinsic parameter calibration device. Figure 1 The functions of the embodiments shown are not described in detail here.

[0120] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by an electronic device including multiple applications, enable the electronic device to perform... Figure 1 The method executed by the camera extrinsic parameter calibration device in the illustrated embodiment is specifically used to perform:

[0121] Acquire road perception information sent by roadside equipment, the road perception information including the first angle information of the autonomous vehicle relative to the lane line, and the roadside equipment is at least one of a preset set of roadside equipment;

[0122] A first road image captured by the camera of an autonomous vehicle is acquired, and lane line detection is performed on the first road image to obtain a first lane line detection result, which includes a first lane line edge detection result.

[0123] The first camera extrinsic parameter correction value of the autonomous vehicle is determined based on the road perception information and the first lane line detection result.

[0124] The calibration result of the camera extrinsic parameters is determined based on the first camera extrinsic parameter correction value.

[0125] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0127] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0128] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0129] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0130] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0131] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0132] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0133] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for calibrating camera extrinsic parameters, wherein, The method includes: Acquire road perception information sent by roadside equipment, the road perception information including the first angle information of the autonomous vehicle relative to the lane line, and the roadside equipment is at least one of a preset set of roadside equipment; A first road image captured by the camera of an autonomous vehicle is acquired, and lane line detection is performed on the first road image to obtain a first lane line detection result, which includes a first lane line edge detection result. The first camera extrinsic parameter correction value of the autonomous vehicle is determined based on the road perception information and the first lane line detection result. The calibration result of the camera extrinsic parameters is determined based on the first camera extrinsic parameter correction value; The first camera extrinsic parameter correction value includes a heading angle correction value and a pitch angle correction value. Determining the first camera extrinsic parameter correction value for the autonomous vehicle based on the road perception information and the first lane detection result includes: Based on the heading angle correction value, the first lane line edge detection result is subjected to inverse perspective transformation processing; Based on the results of the inverse perspective transformation, straight line fitting is performed on the lane line edges to obtain multiple straight line fitting equations for the lane line edges. Based on the straight line fitting equations of multiple lane line edges, determine the straight line slope error between multiple lane line edges; The pitch angle correction value is determined based on the straight slope error between the edges of multiple lane lines.

2. The method as described in claim 1, wherein, The pre-set set of roadside equipment is obtained in the following way: Acquire road images captured by cameras of roadside devices within a preset area, and perform lane line detection on the road images captured by the cameras of the roadside devices to obtain a second lane line detection result. The second lane line detection result includes lane line type and second lane line edge detection result. The corner detection results of the second lane line are determined based on the edge detection results of the second lane line. If the lane line type is a dashed lane line, the deviation of the second lane line edge detection result is less than a first preset deviation threshold, and the deviation of the second lane line corner detection result is less than a second preset deviation threshold, then the roadside device is added to the preset roadside device set.

3. The method as described in claim 1, wherein, The first camera extrinsic parameter correction value includes a heading angle correction value, and determining the first camera extrinsic parameter correction value of the autonomous vehicle based on the road perception information and the first lane detection result includes: The first lane line edge detection result is subjected to inverse perspective transformation to obtain the second included angle information of the autonomous vehicle relative to the lane line; The heading angle correction value is determined based on the first angle information of the autonomous vehicle relative to the lane line and the second angle information of the autonomous vehicle relative to the lane line.

4. The method as described in claim 1, wherein, The road perception information also includes the absolute position of the lane line corner points detected by the roadside equipment. The first camera extrinsic parameter correction values ​​include heading angle correction values, pitch angle correction values, and roll angle correction values. Determining the first camera extrinsic parameter correction values ​​for the autonomous vehicle based on the road perception information and the first lane line detection results includes: The detection results of the first lane line corner points are determined based on the first lane line edge detection results. The absolute position of the lane corner detected by the autonomous vehicle is determined based on the first lane corner detection result and the camera parameters of the autonomous vehicle. Based on the heading angle correction value and the pitch angle correction value, the roll angle correction value is determined according to the absolute position of the lane line corner point detected by the autonomous vehicle and the absolute position of the lane line corner point detected by the roadside equipment.

5. The method as described in claim 1, wherein, The first camera extrinsic parameter correction value includes first camera extrinsic parameter correction values ​​obtained based on multiple roadside devices, and the determination of the camera extrinsic parameter calibration result based on the first camera extrinsic parameter correction value includes: The first camera extrinsic parameter correction values ​​obtained from multiple roadside devices are fused to obtain the fused camera extrinsic parameter correction values. The calibration result of the camera extrinsic parameters is determined based on the fused camera extrinsic parameter correction values.

6. The method as described in any one of claims 1 to 5, wherein, The determination of the camera extrinsic calibration result based on the first camera extrinsic correction value includes: Obtain the second camera extrinsic parameter correction value, which is obtained based on other roadside devices outside the preset roadside device set; Determine the error between the first camera extrinsic parameter correction value and the second camera extrinsic parameter correction value; The first camera extrinsic parameter correction value is verified based on the error between the first camera extrinsic parameter correction value and the second camera extrinsic parameter correction value to obtain a first verification result; The calibration result of the camera extrinsic parameters is determined based on the first verification result.

7. The method as described in any one of claims 1 to 5, wherein, The determination of the camera extrinsic calibration result based on the first camera extrinsic correction value includes: Acquire a second road image captured by the camera of the autonomous vehicle, and perform lane line detection on the second road image to obtain a third lane line detection result; The corresponding high-precision map data is obtained based on the current location of the autonomous vehicle, and the lane lines in the high-precision map data are projected onto the second road image to obtain the lane line projection result. The correction value of the first camera extrinsic parameter is verified based on the third lane line detection result and the lane line projection result to obtain the second verification result. The calibration result of the camera extrinsic parameters is determined based on the second verification result.

8. A calibration device for camera extrinsic parameters, wherein, The device includes: An acquisition unit is used to acquire road perception information sent by roadside equipment, the road perception information including the first angle information of the autonomous vehicle relative to the lane line, and the roadside equipment is at least one of a preset set of roadside equipment. The lane detection unit is used to acquire a first road image captured by the camera of the autonomous vehicle, and to perform lane detection on the first road image to obtain a first lane detection result, which includes a first lane edge detection result. The first determining unit is used to determine the first camera extrinsic parameter correction value of the autonomous vehicle based on the road perception information and the first lane line detection result. The second determining unit is used to determine the calibration result of the camera extrinsic parameters based on the first camera extrinsic parameter correction value; The first camera extrinsic parameter correction values ​​include yaw angle correction values ​​and pitch angle correction values, and the first determining unit is specifically used for: Based on the heading angle correction value, the first lane line edge detection result is subjected to inverse perspective transformation processing; Based on the results of the inverse perspective transformation, straight line fitting is performed on the lane line edges to obtain multiple straight line fitting equations for the lane line edges. Based on the straight line fitting equations of multiple lane line edges, determine the straight line slope error between multiple lane line edges; The pitch angle correction value is determined based on the straight slope error between the edges of multiple lane lines.

9. An electronic device, comprising: processor; as well as A memory configured to store computer-executable instructions, which, when executed, cause the processor to perform the method of any one of claims 1 to 7.

Images