LiDAR and camera extrinsic calibration method for autonomous driving system

Abstract

The present invention discloses a laser radar and camera extrinsic parameter calibration method for an autonomous driving system, wherein a vehicle equipped with a laser radar and a camera collects environmental data around the vehicle in real time during driving; extracts effective laser point clouds and effective pixels constituting the contour of an object respectively; obtains a set of laser point-pixel point coordinate pairs at the same position from the effective laser point cloud information and effective pixel information obtained; and finally obtains an extrinsic parameter matrix by combining a formula. The calibration method provided by the present invention does not require a specific calibration environment, has high versatility, and effectively reduces the R&D and production costs of the vehicle; at the same time, the method provided by the present invention has low computing power, high robustness, and also increases the accuracy of the calibration result; furthermore, the method provided by the present invention is more intelligent and can be adaptively calibrated according to the stored parameters.

Description

Technical Field

[0001] The present invention relates to the field of multi-sensor calibration, and in particular to a laser radar and camera extrinsic parameter calibration method for an autonomous driving system. Background Art

[0002] Multi-sensor calibration method is one of the research focuses in the field of autonomous driving. Cameras and laser radars are commonly used sensors in autonomous driving systems. The external parameter calibration of the two is the TF change of the camera coordinate system and the laser radar coordinate system. Therefore, the external parameter calibration is a four-dimensional rotation matrix. The four-dimensional rotation matrix is ​​a 4*4 rotation matrix composed of six variables: position coordinates (x, y, z), roll angle roll, pitch angle pitch, and heading angle yaw.

[0003] At present, the common joint calibration method of LiDAR and camera requires the use of a calibration plate or calibration in a calibration workshop. When using a calibration plate for calibration, due to the sparse nature of the LiDAR point cloud distribution density, it is impossible to extract valid contour points. When the calibration plate is far away from the vehicle, a larger calibration plate needs to be set. In addition, the calibration plate needs to be placed in different positions during the calibration process to optimize the calibration parameters of the camera and LiDAR. This calibration method is not only inconvenient and the calibration process is cumbersome, but also affects the calibration accuracy. If the calibration method of the calibration workshop is adopted, the cost is too high, which increases the calibration cost. Summary of the invention

[0004] Purpose of the invention: In view of the problems existing in the prior art, the present invention proposes a method for calibrating the external parameters of a lidar and a camera for an autonomous driving system which does not require a specific calibration environment and has high calibration accuracy.

[0005] Technical solution: To achieve the above-mentioned purpose, the present invention provides a method for calibrating the external parameters of a laser radar and a camera for an autonomous driving system, comprising the following steps:

[0006] Step 1: The vehicle equipped with both a laser radar and a camera collects environmental data around the vehicle in real time while driving;

[0007] Step 2: Extracting effective laser point clouds and effective pixels constituting the outline of the object from the environmental data around the vehicle collected in step 1, respectively; obtaining effective laser point cloud information and effective pixel information;

[0008] Step 3: Obtain a set of laser point-pixel coordinate pairs at the same position from the valid laser point cloud information and valid pixel information obtained in step 2;

[0009] Step 4: According to the formula:

[0010]

[0011] The external parameter T matrix is ​​obtained; wherein N represents the total number of laser point-pixel coordinate pairs at the same position in the set of laser point-pixel coordinate pairs at the same position; i represents the number of the laser point-pixel coordinate pair at the same position; xp i represents the x-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair, yp i represents the y-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair, zp i xc represents the z-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair; i represents the x-axis coordinate value of the pixel point in the i-th laser point-pixel coordinate pair, yc i represents the y-axis coordinate value of the pixel point in the i-th laser point-pixel coordinate pair; K represents the camera intrinsic parameter; Min is the minimum value to be solved.

[0012] Furthermore, the method for obtaining effective laser point cloud information in step 2 is:

[0013] Step 2-a1: construct a local 3D map based on the environmental data around the vehicle collected by the lidar in step 1;

[0014] Step 2-a2: Divide the obtained local 3D map into multiple 3D grids evenly;

[0015] Step 2-a3: Perform plane fitting on the point cloud in each 3D grid respectively; retain the 3D grids containing two or three planes;

[0016] Step 2-a4: Calculate the intersection lines between the planes in each retained 3D grid in turn; obtain the point cloud that constitutes each intersection line;

[0017] Step 2-a5: Calculate the eigenvector of the maximum eigenvalue of the point cloud constituting each intersection line in turn;

[0018] Step 2-a6: traverse the feature vectors between every two intersecting lines in adjacent 3D grids, determine the continuous line segments in space by the cosine similarity of the feature vectors of the two intersecting lines; and splice the continuous line segments;

[0019] Step 2-a7: Compare each spliced ​​line segment and unspliced ​​intersecting line obtained in step 2-a6 with the second threshold value respectively. If the length is less than the second threshold value, delete the corresponding line segment; if the length is not less than the second threshold value, retain it. This method can more accurately extract the contour information of the object.

[0020] Furthermore, the size of each 3D grid is 0.5*0.5*0.5 meters, which can effectively ensure the accuracy of extraction.

[0021] Furthermore, the method for obtaining the eigenvector of the maximum eigenvalue of the point cloud constituting each intersection line is as follows:

[0022] Step 2-a5-1: Calculate the coordinate mean of the point cloud constituting each intersection line;

[0023] Step 2-a5-2: Subtract the coordinates of each point in the intersection line from the corresponding coordinate mean to obtain the matrix X, and then multiply the matrix X and the transpose of the matrix X to obtain the covariance matrix of the current point; traverse all the point clouds that make up the intersection line to obtain the covariance matrix corresponding to each point in the intersection line;

[0024] Step 2-a5--3: Add all the covariance matrices obtained in step 2-a5-2 to obtain the covariance matrix of the corresponding intersection line;

[0025] Step 2-a5--4: The eigenvector at the maximum eigenvalue of the covariance matrix corresponding to the intersection line is the eigenvector of the maximum eigenvalue of the point cloud of each intersection line.

[0026] Further, the method for obtaining effective pixel information in step 2 is:

[0027] Step 2-b1: Extract the pixel points that constitute the outline of the object structure in the photo;

[0028] Step 2-b2: Perform line segment fitting on the pixel points obtained in step 2-b1 by a direct fitting method to obtain fitted line segments, and the pixel points constituting each fitted line segment are valid pixel information.

[0029] Furthermore, the method for obtaining the set of laser point-pixel point coordinate pairs at the same position in step 3 is:

[0030] Step 3-1: Project the effective laser point cloud information onto the plane of the image captured by the camera; obtain the effective laser point cloud information in the image plane;

[0031] Step 3-2: In the image plane, by calculating the distance between the effective laser point cloud and the effective pixel point, the effective pixel point closest to each effective laser point and the effective laser point closest to each effective pixel point are obtained respectively;

[0032] Step 3-3: Based on the results obtained in step 3-2, if the nearest valid pixel corresponding to a valid laser point is exactly the same as the nearest valid laser point corresponding to the valid pixel, then the valid laser point and the valid pixel are considered to be at the same position, and the coordinates of this set of valid laser points and valid pixel points are saved; all the results in step 3-2 are traversed to obtain the set of laser point-pixel coordinate pairs at the same position. In this way, all the pixels and laser points are traversed, considering the different distribution densities of pixel points and laser point clouds, and the results obtained are more accurate.

[0033] Furthermore, by collecting different types of objects around the vehicle, the corresponding external parameters are calibrated; the lane lines on the road are collected as environmental data to calibrate the heading angle in the external parameters; the signs or gantry on the road are collected as environmental data to calibrate the pitch angle and roll angle in the external parameters. This can effectively reduce the computing power and improve the robustness of the results.

[0034] Furthermore, the environmental data around the vehicle is the vehicle in front of the vehicle.

[0035] The present invention also provides a computer-readable medium storing software, wherein the software includes instructions that can be executed by one or more computers, and the instructions enable the one or more computers to perform operations through such execution, and the operations include the process of the laser radar and camera extrinsic parameter calibration method for an autonomous driving system as described above.

[0036] A computer system, comprising:

[0037] one or more processors;

[0038] A memory storing operable instructions, wherein when the instructions are executed by the one or more processors, the one or more processors perform operations, wherein the operations include the process of the above-mentioned laser radar and camera extrinsic parameter calibration method for an autonomous driving system.

[0039] Beneficial effects: Compared with the prior art, the calibration method provided by the present invention does not require a specific calibration environment, has high versatility, and effectively reduces the research and development and production costs of vehicles; at the same time, the method provided by the present invention has low computing power and high robustness, which also increases the accuracy of the calibration results; furthermore, the method provided by the present invention is more intelligent and can perform adaptive calibration based on the stored parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A flow chart of the method provided in Example 1;

[0041] Figure 2 A flow chart of the method provided in Example 2. DETAILED DESCRIPTION

[0042] The following will be combined with the accompanying drawings in the examples of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Embodiment 1:

[0044] like Figure 1 As shown, this embodiment provides a laser radar and camera extrinsic parameter calibration method for an autonomous driving system, which specifically includes the following steps:

[0045] Step 1: The main vehicle equipped with both a laser radar and a camera collects environmental data around the main vehicle in real time during driving; in this embodiment, the environmental data around the main vehicle includes vehicles in front of the main vehicle, buildings around the main vehicle, traffic equipment, etc.

[0046] Step 2: Extract the valid laser point cloud and valid pixels constituting the object contour from the environmental data around the vehicle collected in step 1, and obtain the valid laser point cloud information lidar_3d_points and the valid pixel information camera_2d_pixels;

[0047] Among them, the method to obtain effective laser point cloud information lidar_3d_points is:

[0048] Step 2-a1: construct a local 3D map based on the environmental data around the vehicle collected by the lidar in step 1;

[0049] Step 2-a2: Divide the obtained local 3D map into multiple 3D grids evenly; the size of each grid is: 0.5*0.5*0.5 meters.

[0050] Step 2-a3: Perform plane fitting on the point cloud in each 3D grid respectively; retain the 3D grids containing two or three planes;

[0051] Step 2-a4: Calculate the intersection lines between the planes in each retained 3D grid in turn; obtain the point cloud that constitutes each intersection line; and store the calculated point cloud at each intersection line in the corresponding intersection line list. Each intersection line has a corresponding intersection line list, which stores the coordinates of all points that constitute this intersection line.

[0052] Step 2-a5: sequentially calculate the eigenvector of the maximum eigenvalue of the point cloud constituting each intersection line; wherein the method for obtaining the eigenvector of the maximum eigenvalue of the point cloud constituting each intersection line is:

[0053] Step 2-a5-1: Calculate the coordinate mean of the point cloud that makes up each intersection line; that is, the coordinate mean of all points in the intersection line list of this intersection line.

[0054] Step 2-a5-2: Subtract the coordinates of each point in the intersection line from the mean of the coordinates obtained in step 2-a5-1 to obtain the matrix X, and then multiply the matrix X and the transpose of the matrix X to obtain the covariance matrix of the current point; traverse all the point clouds that make up the intersection line to obtain the covariance matrix corresponding to each point in the intersection line;

[0055] Step 2-a5--3: Add all the covariance matrices obtained in step 2-a5-2 to obtain the covariance matrix of the corresponding intersection line;

[0056] Step 2-a5--4: The eigenvector at the maximum eigenvalue of the covariance matrix corresponding to the intersection line is the eigenvector of the maximum eigenvalue of the point cloud of each intersection line.

[0057] Step 2-a6: Traverse the feature vectors between every two intersecting lines in adjacent 3D grids, determine the continuous line segments in space by the cosine similarity of the feature vectors of the two intersecting lines, and splice the continuous line segments. If the cosine similarity of the feature vectors of the two intersecting lines is less than the first threshold, the two intersecting lines are considered to be continuous line segments in space; the first threshold is 0.8 to 0.9, which can correct the deviation caused by the accuracy of the laser point cloud.

[0058] Step 2-a7: Compare each spliced ​​line segment and unspliced ​​intersection line obtained in step 2-a6 with the second threshold one by one. If its length is less than the second threshold, delete the corresponding line segment; if its length is not less than the second threshold, store all point clouds that constitute the corresponding intersection line in the data table; the point cloud information in the data table is the extracted valid laser point cloud information lidar_3d_points. Among them, the second threshold is 0.3 meters, which can prevent the detection of discontinuous corner edges. Ensure that the remaining is a valid line segment in space.

[0059] The method to get the valid pixel information camera_2d_pixels is:

[0060] Step 2-b1: Extract the pixel points that constitute the structural contour of the object in the photo; in this embodiment, the contours of the vehicle, the columns of the building, the corners of the wall, etc. in the image are mainly extracted by traditional methods such as GaussianBlur, canny, findContours, etc.

[0061] Step 2-b2: Perform line segment fitting on the pixel points obtained in step 2-b1 by a direct fitting method to obtain fitted line segments. The pixel points constituting each fitted line segment are valid pixel information camera_2d_pixels.

[0062] Step 3: Obtain a set of laser point-pixel coordinate pairs at the same position from the valid laser point cloud information and valid pixel information obtained in step 2;

[0063] The specific method for obtaining the set of laser point-pixel point coordinate pairs at the same position is:

[0064] Step 3-1: Project the effective laser point cloud information lidar_3d_points to the plane of the image captured by the camera; obtain the effective laser point cloud information lidar_2d_points in the image plane. In this embodiment, the camera plane projection is completed by initializing the external parameters and the camera intrinsic parameters. The initialization of external parameters can be completed by measuring after installing the laser radar and camera using tools such as rulers and protractors; after the vehicle has traveled for a period of time, a new external parameter matrix is ​​obtained by the method provided by the present invention, and the original external parameters are replaced as new initialization external parameters. The camera intrinsic parameters are given by the manufacturer, and their accuracy is high, and no calibration is required.

[0065] Step 3-2: On the image plane, calculate the distance length of all pixel points in the effective pixel information camera_2d_pixels and all points in the effective laser point cloud information idar_2d_points in the plane in turn, and select the points in lidar_2d_points closest to each point in camera_2d_pixels, and the distance between the nearest pixel point and the laser point is less than the first pixel threshold; then all pixel points in the effective pixel information camera_2d_pixels are searched to their corresponding nearest laser points; the first pixel threshold is 10 pixels, which can effectively remove interference points.

[0066] Calculate the distances between all points in the effective laser point cloud information lidar_2d_points in the image plane and all pixel points in the effective pixel information camera_2d_pixels in turn, and select the pixel point in the effective pixel information camera_2d_pixels closest to each point in the effective laser point cloud information lidar_2d_points in the distance plane; then all laser points in the effective laser point cloud information lidar_2d_points are searched to their corresponding nearest pixel points;

[0067] Step 3-3: Based on the result obtained in step 3-2, if the valid pixel point c_p searches for the nearest valid laser point l_p, and the nearest valid pixel point searched for by the valid laser point l_p is the valid pixel point c_p, then the valid pixel point c_p and the valid laser point l_p are considered to be at the same position in the 3D space, and the 3D point cloud coordinates of the valid laser point l_p and the pixel coordinates of the valid pixel point c_p are stored; traverse all the results in step 3-2 to get the set of laser point-pixel coordinate pairs at the same position.

[0068] Step 4: According to the formula:

[0069]

[0070] The external parameter T matrix is ​​obtained. In this embodiment, the environmental data around the main vehicle collected in step 1 is relatively diverse, so the obtained external parameter T matrix includes position coordinates (x, y, z), roll angle roll, pitch angle pitch and heading angle yaw; wherein N represents the total number of laser point-pixel coordinate pairs at the same position in the set of laser point-pixel coordinate pairs at the same position; i represents the number of the laser point-pixel coordinate pair at the same position; xp i represents the x-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair, yp i represents the y-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair, zp i xc represents the z-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair; i represents the x-axis coordinate value of the pixel point in the i-th laser point-pixel coordinate pair, yc i represents the y-axis coordinate value of the pixel point in the i-th laser point-pixel coordinate pair; K represents the camera intrinsic parameter; Min is the minimum value to be solved.

[0071] Embodiment 2:

[0072] This embodiment provides a laser radar and camera extrinsic parameter calibration method for an autonomous driving system, which mainly includes the following steps:

[0073] Step 1: The main vehicle equipped with both a laser radar and a camera collects environmental data around the main vehicle in real time during driving;

[0074] Step 2: Classify and save the environmental data around the main vehicle collected in step 1 according to different categories; repeat steps 3 to 5 according to different categories; the categories include lane lines on the road surface, road signs and gantries, surrounding buildings, etc.

[0075] Step 3: Extract the effective laser point cloud and effective pixels constituting the object contour from the data classified in step 2, and obtain the effective laser point cloud information and effective pixel information;

[0076] Among them, the lane lines on the road surface and the signboards on the road are extracted as two-dimensional plane lines, and the contour extraction method is used to extract the effective laser point cloud information and effective pixel information of the contours of the lane lines and the signboards on the road respectively.

[0077] The effective laser point cloud information and effective pixel information acquisition method disclosed in Example 1 is used to extract the effective laser point cloud information and effective pixel information of the gantry or surrounding buildings.

[0078] Step 4: Obtain a set of laser point-pixel coordinate pairs at the same position from the valid laser point cloud information and valid pixel information obtained in step 3;

[0079] The specific method for obtaining the set of laser point-pixel point coordinate pairs at the same position is:

[0080] Step 4-1: Project the effective laser point cloud information lidar_3d_points to the plane of the image captured by the camera; obtain the effective laser point cloud information lidar_2d_points in the image plane. In this embodiment, the camera plane projection is completed by initializing the external parameters and the camera intrinsic parameters. The initialization of external parameters can be completed by measuring after installing the laser radar and camera using tools such as rulers and protractors; after the vehicle has been driving for a period of time, a new external parameter matrix is ​​obtained by the method provided by the present invention, and the original external parameters are replaced as new initialization external parameters. The camera intrinsic parameters are given by the manufacturer, and their accuracy is high, and no calibration is required.

[0081] Step 4-2: On the image plane, calculate the distance length of all pixel points in the effective pixel information camera_2d_pixels and all points in the effective laser point cloud information idar_2d_points in the plane in turn, and select the points in lidar_2d_points closest to each point in camera_2d_pixels, and the distance between the nearest pixel point and the laser point is less than the first pixel threshold; then all pixel points in the effective pixel information camera_2d_pixels are searched to their corresponding nearest laser points; the first pixel threshold is 10 pixels, which can effectively remove interference points.

[0082] Calculate the distances between all points in the effective laser point cloud information lidar_2d_points in the image plane and all pixel points in the effective pixel information camera_2d_pixels in turn, and select the pixel point in the effective pixel information camera_2d_pixels closest to each point in the effective laser point cloud information lidar_2d_points in the distance plane; then all laser points in the effective laser point cloud information lidar_2d_points are searched to their corresponding nearest pixel points;

[0083] Step 4-3: Based on the result obtained in step 4-2, if the valid pixel point c_p searches for the nearest valid laser point l_p, and the nearest valid pixel point searched for by the valid laser point l_p is the valid pixel point c_p, then the valid pixel point c_p and the valid laser point l_p are considered to be at the same position in the 3D space, and the 3D point cloud coordinates of the valid laser point l_p and the pixel coordinates of the valid pixel point c_p are stored; traverse all the results in step 3-2 to get the set of laser point-pixel coordinate pairs at the same position.

[0084] Step 5: According to the formula:

[0085]

[0086] Get the external parameter T p , different external parameters T can be obtained according to different categories p and save; where N represents the total number of laser point-pixel coordinate pairs at the same position in the set of laser point-pixel coordinate pairs at the same position; i represents the number of the laser point-pixel coordinate pair at the same position; xp i represents the x-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair, yp i represents the y-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair, zp i xc represents the z-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair; i represents the x-axis coordinate value of the pixel point in the i-th laser point-pixel coordinate pair, yc i represents the y-axis coordinate value of the pixel point in the i-th laser point-pixel coordinate pair; K represents the camera intrinsic parameter; Min is the minimum value to be solved.

[0087] In this embodiment, the lane lines of the road are collected as environmental data to calibrate the heading angle in the external parameter matrix; that is, the set of laser point-pixel point coordinate pairs at the same position is obtained through the lane line data, and the external parameter T obtained in the above formula is pRepresents the heading angle in the external parameter matrix. The signboards or gantry on the road are collected as environmental data to calibrate the pitch angle and roll angle in the external parameters; that is, the set of laser point-pixel point coordinate pairs at the same position is obtained through the signboard or gantry data, then the external parameter T obtained in the above formula is p Represents the pitch angle and roll angle in the external parameter matrix. The surrounding buildings are collected as environmental data to calibrate the position information in the external parameter matrix. That is, the set of laser point-pixel point coordinate pairs at the same position is obtained through the surrounding building data. The external parameter T obtained in the above formula is p Represents the position information in the external parameter matrix.

[0088] Step 6: After all the data of all categories in step 2 are processed, the external parameters T obtained in step 5 are p The external parameter matrix of the lidar and camera is obtained by integration.

[0089] The present invention also provides a computer system, comprising: one or more processors; a memory, storing operable instructions, wherein the instructions, when executed by the one or more processors, enable the one or more processors to perform operations, wherein the operations include the process of the aforementioned laser radar and camera extrinsic parameter calibration method for an autonomous driving system.

[0090] It should be understood that the examples of the laser radar and camera extrinsic calibration methods for the automatic driving system of the present invention can be in any computer system with data storage and data processing, and the aforementioned computer system can be at least one electronic processing system or electronic device including a processor and a memory, such as a PC, whether it is a personal PC, a commercial PC, or a PC for graphics processing, or a server-level PC. These PCs realize wired and / or wireless data transmission, especially image data, by having a data interface and / or a network interface.

[0091] In other embodiments, the computer system may also be a server, especially a cloud server, having data storage, processing and network communication functions.

[0092] An exemplary computer system generally includes at least one processor, a memory, and a network interface connected by a system bus. The network interface is used to communicate with other devices / systems.

[0093] The processor is used to provide system calculations and control.

[0094] The memory includes non-volatile memory and cache.

[0095] Non-volatile memory generally has a massive storage capacity and can store operating systems and computer programs. These computer programs may include operable instructions that, when executed by one or more processors, enable the one or more processors to execute the process of the laser radar and camera extrinsic parameter calibration method for an autonomous driving system of the aforementioned embodiment of the present invention.

[0096] In necessary or reasonable implementations, the aforementioned computer system, whether a PC device or a server, may also include more or fewer components than those shown in the diagram, or a combination, or use different hardware, software or other components or different deployment methods.

Claims

1. A laser radar and camera extrinsic calibration method for an autonomous driving system, characterized in that: The following steps are involved: Step 1: The vehicle equipped with both a laser radar and a camera collects environmental data around the vehicle in real time while driving; Step 2: Extracting effective laser point clouds and effective pixels constituting the outline of the object from the environmental data around the vehicle collected in step 1, respectively; obtaining effective laser point cloud information and effective pixel information; Step 3: Obtain a set of laser point-pixel coordinate pairs at the same position from the valid laser point cloud information and valid pixel information obtained in step 2; Step 4: According to the formula ； The external parameter T matrix is ​​obtained; wherein N represents the total number of laser point-pixel coordinate pairs at the same position in the set of laser point-pixel coordinate pairs at the same position; i represents the number of the laser point-pixel coordinate pair at the same position; xp i represents the x-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair, yp i represents the y-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair, zp i xc represents the z-axis coordinate value of the laser point in the i-th laser point-pixel coordinate pair; i represents the x-axis coordinate value of the pixel point in the i-th laser point-pixel coordinate pair, yc i represents the y-axis coordinate value of the pixel point in the i-th laser point-pixel coordinate pair; K represents the camera internal parameter; Min is the minimum value to be solved; The method for obtaining effective laser point cloud information in step 2 is: Step 2-a1: construct a local 3D map based on the environmental data around the vehicle collected by the lidar in step 1; Step 2-a2: Divide the obtained local 3D map into multiple 3D grids evenly; Step 2-a3: Perform plane fitting on the point cloud in each 3D grid respectively; retain the 3D grids containing two or three planes; Step 2-a4: Calculate the intersection lines between the planes in each retained 3D grid in turn; obtain the point cloud that constitutes each intersection line; Step 2-a5: Calculate the eigenvector of the maximum eigenvalue of the point cloud constituting each intersection line in turn; Step 2-a6: traverse the feature vectors between every two intersecting lines in adjacent 3D grids, determine the continuous line segments in space by the cosine similarity of the feature vectors of the two intersecting lines; and splice the continuous line segments; Step 2-a7: Compare each spliced ​​line segment and unspliced ​​intersecting line obtained in step 2-a6 with the second threshold value respectively. If the length of each line segment is less than the second threshold value, delete the corresponding line segment; if the length of each line segment is not less than the second threshold value, retain the corresponding line segment. The method for obtaining effective pixel information in step 2 is: Step 2-b1: Extract the pixel points that constitute the outline of the object structure in the photo; Step 2-b2: Perform line segment fitting on the pixel points obtained in step 2-b1 by a direct fitting method to obtain fitted line segments, and the pixel points constituting each fitted line segment are valid pixel information; The method for obtaining the set of laser point-pixel point coordinate pairs at the same position in step 3 is: Step 3-1: Project the effective laser point cloud information onto the plane of the image captured by the camera; obtain the effective laser point cloud information in the image plane; Step 3-2: In the image plane, by calculating the distance between the effective laser point cloud and the effective pixel point, the effective pixel point closest to each effective laser point and the effective laser point closest to each effective pixel point are obtained respectively; Step 3-3: Based on the results obtained in step 3-2, if the nearest valid pixel point corresponding to a valid laser point is exactly the same as the nearest valid laser point of the corresponding valid pixel point, then the valid laser point and the valid pixel point are considered to be at the same position, and the coordinates of this set of valid laser points and valid pixel points are saved; traverse all the results in step 3-2 to get the set of laser point-pixel coordinate pairs at the same position.

2. The laser radar and camera extrinsic parameter calibration method for an autonomous driving system according to claim 1, characterized in that: The size of each 3D grid is: 0.5*0.5*0.5 meters.

3. The laser radar and camera extrinsic parameter calibration method for an autonomous driving system according to claim 1, characterized in that: The method for obtaining the eigenvector of the maximum eigenvalue of the point cloud that constitutes each intersection line is: Step 2-a5-1: Calculate the coordinate mean of the point cloud constituting each intersection line; Step 2-a5-2: Subtract the coordinates of each point in the intersection line from the corresponding coordinate mean to obtain the matrix X, and then multiply the matrix X and the transpose of the matrix X to obtain the covariance matrix of the current point; traverse all the point clouds that make up the intersection line to obtain the covariance matrix corresponding to each point in the intersection line; Step 2-a5--3: Add all the covariance matrices obtained in step 2-a5-2 to obtain the covariance matrix of the corresponding intersection line; Step 2-a5--4: The eigenvector at the maximum eigenvalue of the covariance matrix corresponding to the intersection line is the eigenvector of the maximum eigenvalue of the point cloud of each intersection line.

4. The laser radar and camera extrinsic parameter calibration method for an autonomous driving system according to claim 1, characterized in that: The corresponding external parameters are calibrated by collecting different objects around the vehicle; the lane lines on the road are collected as environmental data to calibrate the heading angle in the external parameters; the signboards or gantries on the road are collected as environmental data to calibrate the pitch angle and roll angle in the external parameters.

5. The laser radar and camera extrinsic calibration method for an autonomous driving system according to claim 1, characterized in that: The environmental data around the vehicle is the vehicle in front of the vehicle.

6. A computer-readable medium storing software, characterized in that: The software includes instructions that can be executed by one or more computers, and the instructions enable the one or more computers to perform operations through such execution, and the operations include the process of the laser radar and camera extrinsic parameter calibration method for an autonomous driving system as described in any one of claims 1-5.

7. A computer system, characterized in that: include: one or more processors; A memory storing operable instructions, wherein the instructions, when executed by the one or more processors, cause the one or more processors to perform operations, wherein the operations include the process of the laser radar and camera extrinsic parameter calibration method for an autonomous driving system as described in any one of claims 1 to 5.

Images