Camera calibration method and device

Abstract

The invention provides a camera calibration method and device. The method comprises the following steps: calculating internal and external parameters of an initial camera and a first vanishing point coordinate of a lane line in an image coordinate system according to an initial lane line image collected by a to-be-calibrated camera; obtaining a lane line point set under the image coordinate system according to a real-time lane line image collected by a to-be-calibrated camera, and if parallel target lane lines exist and vanishing points exist between the target lane lines, calculating second vanishing point coordinates of the target lane lines under the image coordinate system according to the lane line point set and internal and external parameters of the initial camera; and when the number of the second node coordinates is greater than a threshold value, calculating a slope pitch angle and a slope yaw angle according to the continuous frame vanishing point coordinate set and the first vanishing point coordinates, and updating initial camera external parameters in the initial camera internal and external parameters. According to the method and the device, the vehicle-mounted camera can be automatically calibrated and updated, so that the calibration precision and efficiency of the vehicle-mounted camera are guaranteed, and meanwhile, the flexibility and the universality of camera parameter calibration are improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a camera calibration method and device. Background Art

[0002] At present, with the rapid development of intelligent vehicle technology, on-board cameras, as key sensors for intelligent vehicles to achieve environmental perception, play a vital role in autonomous driving systems. Especially in the current L2 intelligent driving scenario, the calibration accuracy of on-board cameras directly affects the accuracy of the vehicle's perception of the surrounding environment. The traditional camera calibration method is to manually measure the physical coordinates of the calibration object, then take the image, and finally calculate the relationship between the image coordinates of the calibration object in the image and the actual physical coordinates to obtain the installation angle of the camera on the vehicle. In practice, it is found that during the driving process of the vehicle, due to factors such as uneven road surface, turning, acceleration, etc., the installation angle of the camera may change slightly, resulting in a large deviation between the image and the physical coordinates, thereby affecting the intelligent vehicle's accurate perception of the surrounding environment. On the other hand, the traditional calibration method cannot achieve automatic calibration and can only be manually operated in specific places, which limits the flexibility and universality of the calibration work, making it difficult for the traditional calibration method to meet the needs of intelligent vehicles in practical applications. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a camera calibration method and device that can automatically calibrate and update vehicle-mounted cameras without the need for manual calibration in specific locations, thereby improving the flexibility and universality of camera parameter calibration, improving the accuracy and efficiency of vehicle-mounted camera calibration, and ensuring the vehicle's accurate perception of the surrounding environment.

[0004] The first aspect of the present application provides a camera calibration method, comprising:

[0005] Collect the initial lane line image through the camera to be calibrated on the vehicle;

[0006] Calculate the initial camera internal and external parameters and the first vanishing point coordinates of the lane line in the image coordinate system according to the initial lane line image;

[0007] Acquire the real-time lane line image of the camera to be calibrated;

[0008] Acquire a lane line point set in an image coordinate system according to the real-time lane line image;

[0009] When a parallel target lane line is detected in the vehicle coordinate system according to the real-time lane line image, and a vanishing point exists in the target lane line in the image coordinate system, a second vanishing point coordinate of the target lane line in the image coordinate system is calculated according to the lane line point set and the initial camera internal and external parameters;

[0010] storing the second vanishing point coordinates into a continuous frame vanishing point coordinate set;

[0011] When the number of point coordinates in the continuous frame vanishing point coordinate set is greater than a preset number threshold, calculating a slope pitch angle and a slope yaw angle according to the continuous frame vanishing point coordinate set and the first vanishing point coordinates;

[0012] The initial camera extrinsic parameters in the initial camera extrinsic parameters are updated according to the slope pitch angle and the slope yaw angle.

[0013] In the above implementation process, the method can automatically calibrate and update the vehicle-mounted camera without the need for manual calibration in specific locations, thereby improving the flexibility and universality of camera parameter calibration, improving the accuracy and efficiency of vehicle-mounted camera calibration, and ensuring the vehicle's accurate perception of the surrounding environment.

[0014] Furthermore, the step of calculating initial camera internal and external parameters and first vanishing point coordinates of the lane line in the image coordinate system according to the initial lane line image includes:

[0015] Performing an initial calibration on the camera to be calibrated according to the initial lane line image to obtain initial camera internal and external parameters; wherein the initial camera internal and external parameters include camera internal parameters and initial camera external parameters;

[0016] The coordinates of the first vanishing point in the image coordinate system are calculated according to the initial camera internal and external parameters and the initial lane line image.

[0017] Furthermore, the calculating the second vanishing point coordinates of the target lane line in the image coordinate system according to the lane line point set and the initial camera internal and external parameters includes:

[0018] Performing coordinate transformation on the lane line point set according to the initial camera internal and external parameters to obtain a lane line 3D point set in a vehicle coordinate system;

[0019] Performing parallel lane line detection in a vehicle coordinate system according to the lane line 3D point set to obtain a detection result;

[0020] When it is determined that there are parallel lane lines according to the detection result, determining the parallel lane line equation according to the detection result and the lane line 3D point set; wherein the parallel lane line equation includes a straight line equation of the first lane line and a straight line equation of the second lane line;

[0021] When the first lane line and the second lane line have an intersection in the image coordinate system, calculating the coordinates of the intersection of the first lane line and the second lane line in the image coordinate system according to the parallel lane line equation;

[0022] The intersection coordinates are determined as second vanishing point coordinates of the first lane line and the second lane line in an image coordinate system.

[0023] Further, the calculating the slope pitch angle and the slope yaw angle according to the continuous frame vanishing point coordinate set and the first vanishing point coordinate includes:

[0024] Calculating an average vanishing point coordinate according to the continuous frame vanishing point coordinate set;

[0025] A slope pitch angle and a slope yaw angle are calculated according to the average vanishing point coordinates and the first vanishing point coordinates.

[0026] Furthermore, the method further comprises:

[0027] When it is determined according to the detection result that there are no parallel lane lines,

[0028] Alternatively, when it is determined according to the detection result that there are parallel lane lines, and the first lane line and the second lane line do not have an intersection in the image coordinate system, obtaining a currently stored set of vanishing point coordinates of continuous frames;

[0029] The coordinate data stored in the continuous frame vanishing point coordinate set is cleared.

[0030] A second aspect of the present application provides a camera calibration device, the camera calibration device comprising:

[0031] A collection unit, used for collecting an initial lane line image through a camera to be calibrated on the vehicle;

[0032] A first calculation unit, configured to calculate initial camera internal and external parameters and first vanishing point coordinates of the lane line in an image coordinate system according to the initial lane line image;

[0033] A first acquisition unit, used to acquire a real-time lane line image of the camera to be calibrated;

[0034] A second acquisition unit, configured to acquire a lane line point set in an image coordinate system according to the real-time lane line image;

[0035] A second calculation unit is used to calculate the second vanishing point coordinates of the target lane line in the image coordinate system according to the lane line point set and the initial camera internal and external parameters when a parallel target lane line is detected in the vehicle coordinate system according to the real-time lane line image and a vanishing point exists in the target lane line in the image coordinate system;

[0036] A storage unit, used for storing the second vanishing point coordinates into a continuous frame vanishing point coordinate set;

[0037] a third calculation unit, configured to calculate a slope pitch angle and a slope yaw angle according to the continuous frame vanishing point coordinate set and the first vanishing point coordinates when the number of point coordinates in the continuous frame vanishing point coordinate set is greater than a preset number threshold;

[0038] An updating unit is used to update the initial camera extrinsic parameters in the initial camera extrinsic parameters according to the slope pitch angle and the slope yaw angle.

[0039] Furthermore, the first computing unit includes:

[0040] A calibration subunit, configured to perform initial calibration on the camera to be calibrated according to the initial lane line image to obtain initial camera internal and external parameters; wherein the initial camera internal and external parameters include camera internal parameters and initial camera external parameters;

[0041] The first calculation subunit is used to calculate the first vanishing point coordinates in the image coordinate system according to the initial camera internal and external parameters and the initial lane line image.

[0042] Furthermore, the second computing unit includes:

[0043] A first conversion subunit is used for, when a parallel target lane line is detected in the vehicle coordinate system according to the real-time lane line image, and a vanishing point exists in the target lane line in the image coordinate system, to perform coordinate conversion on the lane line point set according to the initial camera internal and external parameters to obtain a lane line 3D point set in the vehicle coordinate system;

[0044] A detection subunit, configured to detect parallel lane lines in a vehicle coordinate system according to the lane line 3D point set, and obtain a detection result;

[0045] A determination subunit, configured to determine, when it is determined according to the detection result that there are parallel lane lines, a parallel lane line equation according to the detection result and the lane line 3D point set; wherein the parallel lane line equation includes a straight line equation of the first lane line and a straight line equation of the second lane line;

[0046] A second calculation subunit is used to calculate the coordinates of the intersection of the first lane line and the second lane line in the image coordinate system according to the parallel lane line equation when the first lane line and the second lane line have an intersection in the image coordinate system;

[0047] A determination subunit is used to determine the intersection coordinates as second vanishing point coordinates of the first lane line and the second lane line in an image coordinate system.

[0048] Further, the third calculation unit is specifically used to calculate the average vanishing point coordinates according to the continuous frame vanishing point coordinate set when the number of point coordinates in the continuous frame vanishing point coordinate set is greater than a preset number threshold; and calculate the slope pitch angle and the slope yaw angle according to the average vanishing point coordinates and the first vanishing point coordinates.

[0049] Furthermore, the second computing unit further includes:

[0050] an acquisition subunit, configured to acquire a currently stored set of vanishing point coordinates of continuous frames when it is determined according to the detection result that there are no parallel lane lines, or when it is determined according to the detection result that there are parallel lane lines and there is no intersection between the first lane line and the second lane line in the image coordinate system;

[0051] The clearing subunit is used to clear the coordinate data stored in the continuous frame vanishing point coordinate set.

[0052] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the camera calibration method described in any one of the first aspects of the present application.

[0053] A fourth aspect of the present application provides a computer-readable storage medium storing computer program instructions, wherein when the computer program instructions are read and executed by a processor, the camera calibration method described in any one of the first aspects of the present application is executed.

[0054] The beneficial effects of the present application are as follows: when a suitable road is found, the method can complete the camera calibration by running a certain distance on a parallel lane line, so that the method can be easily implemented in actual operations, thereby solving the difficulties of traditional low-level intelligent driving functions (the conversion between image space and real-world coordinates cannot achieve fast and simple camera calibration, and is relatively complicated in actual operations). BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 A schematic diagram of a camera calibration method provided in an embodiment of the present application;

[0057] Figure 2A schematic diagram of a flow chart of another camera calibration method provided in an embodiment of the present application;

[0058] Figure 3 A vanishing point calculation flow chart provided in an embodiment of the present application;

[0059] Figure 4 A schematic diagram of the principle of pinhole imaging provided in an embodiment of the present application;

[0060] Figure 5 An image formed by lane lines provided in an embodiment of the present application;

[0061] Figure 6 A schematic diagram of the calibration process of the pitch angle and the yawr angle provided in the embodiment of the present application;

[0062] Figure 7 A schematic diagram of the structure of a camera calibration device provided in an embodiment of the present application;

[0063] Figure 8 A schematic diagram of the structure of another camera calibration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0065] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0066] Example 1

[0067] Please see Figure 1 , Figure 1 A schematic diagram of a camera calibration method provided in this embodiment. The camera calibration method includes:

[0068] S101. Collect an initial lane line image using a camera to be calibrated on a vehicle.

[0069] S102, calculating initial camera internal and external parameters and the first vanishing point coordinates of the lane line in the image coordinate system according to the initial lane line image.

[0070] S103: Obtain a real-time lane line image of the camera to be calibrated.

[0071] S104. Acquire a lane line point set in an image coordinate system according to the real-time lane line image.

[0072] In this embodiment, the lane line point set includes at least one point coordinate set, and the point coordinates on the same line are in the same set.

[0073] S105. When a parallel target lane line is detected in the vehicle coordinate system according to the real-time lane line image, and a vanishing point exists in the target lane line in the image coordinate system, a second vanishing point coordinate of the target lane line in the image coordinate system is calculated according to the lane line point set and the initial camera internal and external parameters.

[0074] S106: Store the second vanishing point coordinates into a continuous frame vanishing point coordinate set.

[0075] S107: When the number of point coordinates in the continuous frame vanishing point coordinate set is greater than a preset number threshold, calculate the slope pitch angle and the slope yaw angle according to the continuous frame vanishing point coordinate set and the first vanishing point coordinate.

[0076] S108. Update the initial camera extrinsic parameters in the initial camera extrinsic parameters according to the slope pitch angle and the slope yaw angle.

[0077] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0078] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0079] It can be seen that the camera calibration method described in this embodiment can automatically calibrate and update the vehicle-mounted camera without the need for manual calibration in specific locations, thereby improving the flexibility and universality of camera parameter calibration, improving the accuracy and efficiency of vehicle-mounted camera calibration, and ensuring the vehicle's accurate perception of the surrounding environment.

[0080] Example 2

[0081] Please see Figure 2 , Figure 2 A schematic diagram of a camera calibration method provided in this embodiment. The camera calibration method includes:

[0082] S201. Collect an initial lane line image using a camera to be calibrated on a vehicle.

[0083] S202, performing initial calibration on the camera to be calibrated according to the initial lane line image to obtain initial camera internal and external parameters.

[0084] In this embodiment, the initial camera intrinsic and extrinsic parameters include camera intrinsic parameters and initial camera extrinsic parameters.

[0085] In this embodiment, in low-level LLD assisted driving, initial calibration can be performed using only one front-view camera (in this case, the vehicle is only equipped with one front-view camera) to obtain the initial camera internal and external parameters.

[0086] S203: Calculate the coordinates of the first vanishing point in the image coordinate system according to the initial camera internal and external parameters and the initial lane line image.

[0087] In this embodiment, the method can obtain the initial image vanishing point coordinates (int_x, int_y) through the initial camera internal and external parameters.

[0088] S204: Obtain a real-time lane line image of the camera to be calibrated.

[0089] S205. Acquire a lane line point set in an image coordinate system according to the real-time lane line image.

[0090] In this embodiment, the lane line point set includes at least one point coordinate set, and the point coordinates on the same line are in the same set.

[0091] In this embodiment, the method can obtain points in the lane line image through the perception detection model, and then use algorithm post-processing to put the points of the same line into the same point set to obtain the point set image_points of multiple lines (i.e., the lane line point set).

[0092] S206. When a parallel target lane line is detected in the vehicle coordinate system based on the real-time lane line image, and a vanishing point exists in the target lane line in the image coordinate system, the lane line point set is transformed according to the initial camera internal and external parameters to obtain a lane line 3D point set in the vehicle coordinate system.

[0093] In this embodiment, the method can use the initial camera internal and external point set image_points participating in the lane line, perform IPM projection on the image points and convert them into 3D points in the vehicle coordinate system to obtain the point set vehicle_points (i.e., the lane line 3D point set).

[0094] S207 , performing parallel lane line detection in the vehicle coordinate system according to the lane line 3D point set to obtain a detection result.

[0095] As an optional implementation, the method further includes:

[0096] When it is determined based on the detection results that there are no parallel lane lines,

[0097] Alternatively, when it is determined according to the detection result that there are parallel lane lines, and the first lane line and the second lane line do not have an intersection in the image coordinate system, the currently stored continuous frame vanishing point coordinate set is obtained;

[0098] Clear the coordinate data stored in the vanishing point coordinate set of consecutive frames.

[0099] In this embodiment, if no parallelism or no vanishing point that meets the conditions is detected in the image that has accumulated more than no_thred frames, the previously saved data is deleted and the calibration count is restarted.

[0100] S208. When it is determined based on the detection results that parallel lane lines exist, determine the parallel lane line equations based on the detection results and the lane line 3D point set; wherein the parallel lane line equations include the straight line equation of the first lane line and the straight line equation of the second lane line.

[0101] In this embodiment, the method can perform parallel detection and judgment on the points of vehicle_points. If it is confirmed that the two lane lines are parallel in the vehicle coordinate system, the image point set image_points of the corresponding two lines is taken out, and straight line fitting is performed to obtain two straight line equations.

[0102] S209: When the first lane line and the second lane line have an intersection in the image coordinate system, calculate the coordinates of the intersection between the first lane line and the second lane line in the image coordinate system according to the parallel lane line equation.

[0103] In this embodiment, the intersection point of the straight lines on the two images is found to obtain the intersection point coordinates (img_x, img_y).

[0104] S210: Determine the intersection coordinates as the second vanishing point coordinates of the first lane line and the second lane line in the image coordinate system.

[0105] In this embodiment, the method determines the intersection point as the vanishing point of the two lane lines on the image.

[0106] Please see Figure 3 , Figure 3 A vanishing point calculation flow chart is shown.

[0107] S211. Store the second vanishing point coordinates into a continuous frame vanishing point coordinate set.

[0108] In this embodiment, the method performs vanishing point detection and calculation on the lane lines detected in each frame of the image. If parallel lane lines are detected and vanishing points are detected for more than fit_thred consecutive frames, the future point data is saved.

[0109] S212: When the number of point coordinates in the continuous frame vanishing point coordinate set is greater than a preset number threshold, calculate an average vanishing point coordinate according to the continuous frame vanishing point coordinate set.

[0110] In this embodiment, when the method has obtained the vanishing point data of the fit_thred frame that meets the conditions, the average vanishing point coordinates (measure_x, measure_y) can be obtained by accumulating the vanishing points of multiple frames.

[0111] S213: Calculate the slope pitch angle and the slope yaw angle according to the average vanishing point coordinates and the first vanishing point coordinates.

[0112] In this embodiment, the method obtains the changed pitch angle (i.e., the slope pitch angle) and yaw angle (i.e., the slope yaw angle) by the following formula:

[0113]

[0114] In this embodiment, the slope calculation principle is as follows:

[0115] The conversion from the image coordinate system to the camera coordinate system is mainly based on the pinhole imaging principle, with the optical center o (pinhole) of the camera as the origin, the X-axis as the horizontal direction, the Y-axis as the vertical direction, and the Z-axis perpendicular to the image plane.

[0116] Please see Figure 4 , Figure 4 The principle of pinhole imaging is shown. However, the image displayed on the image plane is inverted. Therefore, as shown in the figure below, the image plane is moved to the other side to ensure that the image is positive when the two coordinate systems on the image plane are mapped into the camera coordinate system.

[0117] So the image formed by the lane line is Figure 5 As shown, the road ramp at this time is:

[0118]

[0119] That is, the width of the lane divided by the width of the road on the image is the slope pitch value at this point.

[0120] Based on this, if the pitch angle changes between two moments, the change values ​​of the two pitch angles can be calculated:

[0121]

[0122] The same applies to the yawr angle.

[0123] S214. Update the initial camera extrinsic parameters in the initial camera extrinsic parameters according to the slope pitch angle and the slope yaw angle.

[0124] In this embodiment, the method can calculate the camera installation change angle pitch and yaw angle through multiple frames, and then update the camera extrinsic parameters used later.

[0125] In this embodiment, Figure 6 A schematic diagram of the calibration process of the pitch angle and the yawr angle is shown.

[0126] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0127] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0128] It can be seen that the camera calibration method described in this embodiment can automatically calibrate and update the vehicle-mounted camera without the need for manual calibration in specific locations, thereby improving the flexibility and universality of camera parameter calibration, improving the accuracy and efficiency of vehicle-mounted camera calibration, and ensuring the vehicle's accurate perception of the surrounding environment.

[0129] Example 3

[0130] Please see Figure 7 , Figure 7 FIG. 1 is a schematic diagram of the structure of a camera calibration device provided in this embodiment. Figure 7 As shown, the camera calibration device includes:

[0131] The acquisition unit 310 is used to acquire an initial lane line image through a camera to be calibrated on the vehicle;

[0132] A first calculation unit 320, configured to calculate initial camera internal and external parameters and first vanishing point coordinates of the lane line in the image coordinate system according to the initial lane line image;

[0133] A first acquisition unit 330 is used to acquire a real-time lane line image of the camera to be calibrated;

[0134] A second acquisition unit 340 is used to acquire a lane line point set in an image coordinate system according to the real-time lane line image;

[0135] A second calculation unit 350 is used to calculate the coordinates of the second vanishing point of the target lane line in the image coordinate system according to the lane line point set and the initial camera internal and external parameters when a parallel target lane line is detected in the vehicle coordinate system according to the real-time lane line image and a vanishing point exists in the target lane line in the image coordinate system;

[0136] The storage unit 360 is used to store the second vanishing point coordinates into the continuous frame vanishing point coordinate set;

[0137] A third calculation unit 370, configured to calculate a slope pitch angle and a slope yaw angle according to the continuous frame vanishing point coordinate set and the first vanishing point coordinates when the number of point coordinates in the continuous frame vanishing point coordinate set is greater than a preset number threshold;

[0138] The updating unit 380 is used to update the initial camera extrinsic parameters in the initial camera extrinsic parameters according to the slope pitch angle and the slope yaw angle.

[0139] In this embodiment, the explanation of the camera calibration device can refer to the description in Embodiment 1 or Embodiment 2, and will not be further elaborated in this embodiment.

[0140] It can be seen that the camera calibration device described in this embodiment can automatically calibrate and update the vehicle-mounted camera without the need for manual calibration in specific locations, thereby improving the flexibility and universality of camera parameter calibration, improving the accuracy and efficiency of vehicle-mounted camera calibration, and ensuring the vehicle's accurate perception of the surrounding environment.

[0141] Example 4

[0142] Please see Figure 8 , Figure 8 FIG. 1 is a schematic diagram of the structure of a camera calibration device provided in this embodiment. Figure 8 As shown, the camera calibration device includes:

[0143] The acquisition unit 310 is used to acquire an initial lane line image through a camera to be calibrated on the vehicle;

[0144] A first calculation unit 320, configured to calculate initial camera internal and external parameters and first vanishing point coordinates of the lane line in the image coordinate system according to the initial lane line image;

[0145] A first acquisition unit 330 is used to acquire a real-time lane line image of the camera to be calibrated;

[0146] A second acquisition unit 340 is used to acquire a lane line point set in an image coordinate system according to the real-time lane line image;

[0147] A second calculation unit 350 is used to calculate the coordinates of the second vanishing point of the target lane line in the image coordinate system according to the lane line point set and the initial camera internal and external parameters when a parallel target lane line is detected in the vehicle coordinate system according to the real-time lane line image and a vanishing point exists in the target lane line in the image coordinate system;

[0148] The storage unit 360 is used to store the second vanishing point coordinates into the continuous frame vanishing point coordinate set;

[0149] A third calculation unit 370, configured to calculate a slope pitch angle and a slope yaw angle according to the continuous frame vanishing point coordinate set and the first vanishing point coordinates when the number of point coordinates in the continuous frame vanishing point coordinate set is greater than a preset number threshold;

[0150] The updating unit 380 is used to update the initial camera extrinsic parameters in the initial camera extrinsic parameters according to the slope pitch angle and the slope yaw angle.

[0151] As an optional implementation, the first calculation unit 320 includes:

[0152] The calibration subunit 321 is used to perform initial calibration on the camera to be calibrated according to the initial lane line image to obtain initial camera internal and external parameters; wherein the initial camera internal and external parameters include camera internal parameters and initial camera external parameters;

[0153] The first calculation subunit 322 is used to calculate the first vanishing point coordinates in the image coordinate system according to the initial camera internal and external parameters and the initial lane line image.

[0154] As an optional implementation, the second calculation unit 350 includes:

[0155] The first conversion subunit 351 is used to perform coordinate conversion on the lane line point set according to the initial camera internal and external parameters to obtain the lane line 3D point set in the vehicle coordinate system when a parallel target lane line is detected in the vehicle coordinate system according to the real-time lane line image and a vanishing point exists in the target lane line in the image coordinate system;

[0156] The detection subunit 352 is used to detect parallel lane lines in the vehicle coordinate system according to the lane line 3D point set to obtain a detection result;

[0157] The determination subunit 353 is used to determine the parallel lane line equation according to the detection result and the lane line 3D point set when it is determined that there are parallel lane lines according to the detection result; wherein the parallel lane line equation includes the straight line equation of the first lane line and the straight line equation of the second lane line;

[0158] A second calculation subunit 354 is used to calculate the coordinates of the intersection of the first lane line and the second lane line in the image coordinate system according to the parallel lane line equation when the first lane line and the second lane line have an intersection in the image coordinate system;

[0159] The determination subunit 355 is used to determine the intersection coordinates as the second vanishing point coordinates of the first lane line and the second lane line in the image coordinate system.

[0160] As an optional implementation, the third calculation unit 370 is specifically used to calculate the average vanishing point coordinates according to the continuous frame vanishing point coordinate set when the number of point coordinates in the continuous frame vanishing point coordinate set is greater than a preset number threshold; and calculate the slope pitch angle and the slope yaw angle according to the average vanishing point coordinates and the first vanishing point coordinates.

[0161] As an optional implementation manner, the second calculation unit 350 further includes:

[0162] The acquisition subunit 356 is used to acquire the currently stored continuous frame vanishing point coordinate set when it is determined according to the detection result that there are no parallel lane lines, or when it is determined according to the detection result that there are parallel lane lines and there is no intersection between the first lane line and the second lane line in the image coordinate system;

[0163] The clearing subunit 357 is used to clear the coordinate data stored in the continuous frame vanishing point coordinate set.

[0164] In this embodiment, the explanation of the camera calibration device can refer to the description in Embodiment 1 or Embodiment 2, and will not be further elaborated in this embodiment.

[0165] It can be seen that the camera calibration device described in this embodiment can automatically calibrate and update the vehicle-mounted camera without the need for manual calibration in specific locations, thereby improving the flexibility and universality of camera parameter calibration, improving the accuracy and efficiency of vehicle-mounted camera calibration, and ensuring the vehicle's accurate perception of the surrounding environment.

[0166] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the camera calibration method in Embodiment 1 or Embodiment 2 of the present application.

[0167] An embodiment of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the camera calibration method in Embodiment 1 or Embodiment 2 of the present application is executed.

[0168] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0169] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0170] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0171] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0172] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0173] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A camera calibration method, characterized in that: include: Collect the initial lane line image through the camera to be calibrated on the vehicle; Calculate the initial camera internal and external parameters and the first vanishing point coordinates of the lane line in the image coordinate system according to the initial lane line image; Acquire the real-time lane line image of the camera to be calibrated; Acquire a lane line point set in an image coordinate system according to the real-time lane line image; When a parallel target lane line is detected in the vehicle coordinate system according to the real-time lane line image, and a vanishing point exists in the target lane line in the image coordinate system, a second vanishing point coordinate of the target lane line in the image coordinate system is calculated according to the lane line point set and the initial camera internal and external parameters; storing the second vanishing point coordinates into a continuous frame vanishing point coordinate set; When the number of point coordinates in the continuous frame vanishing point coordinate set is greater than a preset number threshold, calculating a slope pitch angle and a slope yaw angle according to the continuous frame vanishing point coordinate set and the first vanishing point coordinates; The initial camera extrinsic parameters in the initial camera extrinsic parameters are updated according to the slope pitch angle and the slope yaw angle.

2. The camera calibration method according to claim 1, characterized in that: The calculating, according to the initial lane line image, initial camera internal and external parameters and the first vanishing point coordinates of the lane line in the image coordinate system includes: Performing an initial calibration on the camera to be calibrated according to the initial lane line image to obtain initial camera internal and external parameters; wherein the initial camera internal and external parameters include camera internal parameters and initial camera external parameters; The coordinates of the first vanishing point in the image coordinate system are calculated according to the initial camera internal and external parameters and the initial lane line image.

3. The camera calibration method according to claim 1, characterized in that: The calculating the second vanishing point coordinates of the target lane line in the image coordinate system according to the lane line point set and the initial camera internal and external parameters includes: Performing coordinate transformation on the lane line point set according to the initial camera internal and external parameters to obtain a lane line 3D point set in a vehicle coordinate system; Performing parallel lane line detection in a vehicle coordinate system according to the lane line 3D point set to obtain a detection result; When it is determined that there are parallel lane lines according to the detection result, determining the parallel lane line equation according to the detection result and the lane line 3D point set; wherein the parallel lane line equation includes a straight line equation of the first lane line and a straight line equation of the second lane line; When the first lane line and the second lane line have an intersection in the image coordinate system, calculating the coordinates of the intersection of the first lane line and the second lane line in the image coordinate system according to the parallel lane line equation; The intersection coordinates are determined as second vanishing point coordinates of the first lane line and the second lane line in an image coordinate system.

4. The camera calibration method according to claim 1, characterized in that: The calculating the slope pitch angle and the slope yaw angle according to the continuous frame vanishing point coordinate set and the first vanishing point coordinate comprises: Calculating an average vanishing point coordinate according to the continuous frame vanishing point coordinate set; A slope pitch angle and a slope yaw angle are calculated according to the average vanishing point coordinates and the first vanishing point coordinates.

5. The camera calibration method according to claim 3, characterized in that: The method further comprises: When it is determined according to the detection result that there are no parallel lane lines, Alternatively, when it is determined according to the detection result that there are parallel lane lines, and the first lane line and the second lane line do not have an intersection in the image coordinate system, obtaining a currently stored set of vanishing point coordinates of continuous frames; The coordinate data stored in the continuous frame vanishing point coordinate set is cleared.

6. A camera calibration device, characterized in that: The camera calibration device comprises: A collection unit, used for collecting an initial lane line image through a camera to be calibrated on the vehicle; A first calculation unit, configured to calculate initial camera internal and external parameters and first vanishing point coordinates of the lane line in an image coordinate system according to the initial lane line image; A first acquisition unit, used to acquire a real-time lane line image of the camera to be calibrated; A second acquisition unit, configured to acquire a lane line point set in an image coordinate system according to the real-time lane line image; A second calculation unit is used to calculate the second vanishing point coordinates of the target lane line in the image coordinate system according to the lane line point set and the initial camera internal and external parameters when a parallel target lane line is detected in the vehicle coordinate system according to the real-time lane line image and a vanishing point exists in the target lane line in the image coordinate system; A storage unit, used for storing the second vanishing point coordinates into a continuous frame vanishing point coordinate set; a third calculation unit, configured to calculate a slope pitch angle and a slope yaw angle according to the continuous frame vanishing point coordinate set and the first vanishing point coordinates when the number of point coordinates in the continuous frame vanishing point coordinate set is greater than a preset number threshold; An updating unit is used to update the initial camera extrinsic parameters in the initial camera extrinsic parameters according to the slope pitch angle and the slope yaw angle.

7. The camera calibration device according to claim 6, characterized in that: The first computing unit comprises: A calibration subunit, configured to perform initial calibration on the camera to be calibrated according to the initial lane line image to obtain initial camera internal and external parameters; wherein the initial camera internal and external parameters include camera internal parameters and initial camera external parameters; The first calculation subunit is used to calculate the first vanishing point coordinates in the image coordinate system according to the initial camera internal and external parameters and the initial lane line image.

8. The camera calibration device according to claim 6, characterized in that: The second computing unit comprises: A first conversion subunit is used for, when a parallel target lane line is detected in the vehicle coordinate system according to the real-time lane line image, and a vanishing point exists in the target lane line in the image coordinate system, to perform coordinate conversion on the lane line point set according to the initial camera internal and external parameters to obtain a lane line 3D point set in the vehicle coordinate system; A detection subunit, configured to detect parallel lane lines in a vehicle coordinate system according to the lane line 3D point set, and obtain a detection result; A determination subunit, configured to determine, when it is determined according to the detection result that there are parallel lane lines, a parallel lane line equation according to the detection result and the lane line 3D point set; wherein the parallel lane line equation includes a straight line equation of the first lane line and a straight line equation of the second lane line; A second calculation subunit is used to calculate the coordinates of the intersection of the first lane line and the second lane line in the image coordinate system according to the parallel lane line equation when the first lane line and the second lane line have an intersection in the image coordinate system; A determination subunit is used to determine the intersection coordinates as second vanishing point coordinates of the first lane line and the second lane line in an image coordinate system.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to perform the camera calibration method according to any one of claims 1 to 5.

10. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the camera calibration method according to any one of claims 1 to 5 is executed.

Images