Laser radar external parameter calibration method and device, computer equipment and storage medium

Abstract

The application relates to the technical field of laser radars, and discloses a laser radar external parameter calibration method and device, computer equipment and a storage medium, which comprise the following steps: acquiring vehicle information and point cloud data in a preset area; performing point cloud segmentation processing on the point cloud data to obtain ground point cloud data; determining a first rotation matrix based on the ground point cloud data and a target normal vector; determining calibrated ground point cloud data and calibrated point cloud data based on the first rotation matrix, the ground point cloud data and the point cloud data; performing decomposition processing on the calibrated point cloud data to obtain at least two calibration rod point cloud data; determining a rotation angle based on the calibration rod point cloud data; determining a second rotation matrix based on the rotation angle and the calibration rod point cloud data; determining a target rotation matrix based on the first rotation matrix and the second rotation matrix; and calibrating the laser radar external parameter based on the target rotation matrix, the rotation angle, the calibrated ground point cloud, the calibration rod point cloud and the vehicle information.

Description

Technical Field

[0001] This invention relates to the field of lidar technology, specifically to a method, apparatus, computer equipment, and storage medium for calibrating external parameters of lidar. Background Technology

[0002] With the rapid development of autonomous driving technology, LiDAR has been widely adopted as an important perception device. Current technological development generally believes that LiDAR is essential for L3 or higher level autonomous vehicles, as it can accurately acquire three-dimensional point cloud data of the surrounding environment.

[0003] Currently, lidar extrinsic parameter calibration methods often employ a plane fitting method using a calibration board. Specifically, this involves: acquiring an image of the calibration board (ensuring the board is placed on a horizontal plane and maintaining a constant distance between the board and the lidar); extracting features from the calibration board (using image processing techniques to extract feature points on the board, such as corner or edge points); and using a plane fitting algorithm to fit the calibration board to a plane based on the extracted feature points. Based on the plane fitting results, the lidar's intrinsic and extrinsic parameters are then calculated.

[0004] However, using a calibration plate for planar fitting often requires the use of a special calibration plate or structure, and the calibration environment must have specific lighting and geometric conditions, which makes the calibration process for lidar complex and costly. Summary of the Invention

[0005] In view of this, the present invention provides a calibration method, apparatus, computer equipment and storage medium for lidar extrinsic parameters, to solve the problem that the use of calibration boards for plane fitting methods often requires special calibration boards or structures and specific lighting and geometric conditions in the calibration environment, resulting in the complexity and high cost of lidar calibration.

[0006] In a first aspect, the present invention provides a method for calibrating the extrinsic parameters of a lidar, the method comprising: acquiring vehicle information and point cloud data within a preset area; performing point cloud segmentation processing on the point cloud data to obtain ground point cloud data; determining a first rotation matrix based on the ground point cloud data and a target normal vector; determining calibrated ground point cloud data and calibrated point cloud data based on the first rotation matrix, the ground point cloud data, and the point cloud data; decomposing the calibrated point cloud data to obtain at least two calibration rod point cloud data; determining a rotation angle based on the calibration rod point cloud data; determining a second rotation matrix based on the rotation angle and the calibration rod point cloud data; determining a target rotation matrix based on the first rotation matrix and the second rotation matrix; and calibrating the extrinsic parameters of the lidar based on the target rotation matrix, the rotation angle, the calibrated ground point cloud, the calibration rod point cloud, and the vehicle information.

[0007] In an optional implementation, point cloud data is segmented to obtain ground point cloud data, including: determining at least three point cloud data to be processed from the point cloud data; performing plane fitting processing on the coordinate points of the point cloud data to be processed to determine the processing coefficients corresponding to the point cloud data to be processed; determining a homography matrix based on the processing coefficients; determining the distance of each point cloud data from the plane formed by the point cloud data to be processed based on the homography matrix; using point cloud data with a distance less than a preset distance as ground point cloud data; detecting whether the number of ground point cloud data is less than a preset number; if the number is less than the preset number, using point cloud data with a distance less than the preset distance as ground point cloud data; repeating the step of determining at least three point cloud data to be processed from the point cloud data until the number is less than the preset number and the step of using point cloud data with a distance less than the preset distance as ground point cloud data, until the loop condition is met, and finally using the final point cloud data as ground point cloud data.

[0008] In one optional implementation, determining a first rotation matrix based on ground point cloud data and a target normal vector includes: determining the normal vector of the ground point cloud data based on the ground point cloud data; and determining the first rotation matrix based on the normal vector of the ground point cloud data and the target normal vector.

[0009] In one optional implementation, determining the normal vector of the ground point cloud data based on the ground point cloud data includes: determining the ground point cloud covariance matrix based on the ground point cloud data; performing singular value decomposition on the ground point cloud covariance matrix to obtain an eigenvector matrix; comparing the eigenvalues ​​of each eigenvector matrix to determine the smallest eigenvalue; and determining the normal vector of the ground point cloud data based on the eigenvalue.

[0010] In one optional implementation, determining the first rotation matrix based on the normal vectors of the ground point cloud data and the target normal vector includes: determining the vector product and the included angle based on the normal vectors of the ground point cloud data and the target normal vector; determining the unit vectors in the same direction as the vector product; and determining the first rotation matrix based on the included angle and the unit vectors.

[0011] In one optional implementation, determining the rotation angle based on the calibration rod point cloud data includes: determining the average abscissa and average ordinate of each calibration rod based on the calibration rod point cloud data; and determining the rotation angle based on the average abscissa and average ordinate.

[0012] In one optional implementation, the vehicle information includes rear axle information and tire radius information; wherein, the calibration of the lidar extrinsic parameters based on the target rotation matrix, rotation angle, calibrated ground point cloud, calibration rod point cloud, and vehicle information includes: determining the number of calibrated ground point clouds based on the calibrated ground point cloud data; determining the average value of the calibrated ground point cloud data based on the coordinates of each calibrated ground point cloud data and the number of calibrated ground point cloud data; determining the coordinates of the calibration rod point cloud data near the vehicle based on the coordinates of the calibration rod point cloud data; and determining the coordinates of the calibration rod point cloud data near the vehicle based on the coordinates of the calibration rod point cloud data. The coordinates of the vehicle's calibration pole point cloud are rotated around the Z-axis by a rotation angle to obtain the coordinates of the target calibration pole point cloud data. The first calibration parameter is determined based on the distance from the x-coordinate of the target calibration pole point cloud data to the rear axle information. The second calibration parameter is determined based on the y-coordinate of the target calibration pole point cloud data. The third calibration parameter is determined based on the tire radius information and the average value of the calibrated ground point cloud data. The target rotation matrix is ​​then decomposed using Eulerian decomposition to determine the fourth calibration parameter. Based on the first, second, third, and fourth calibration parameters, the extrinsic parameters of the lidar are calibrated.

[0013] Secondly, the present invention provides a calibration device for lidar extrinsic parameters. The device includes: an acquisition module for acquiring vehicle information and point cloud data within a preset area; a segmentation processing module for segmenting the point cloud data to obtain ground point cloud data; a first determination module for determining a first rotation matrix based on the ground point cloud data and a target normal vector; a second determination module for determining calibrated ground point cloud data and calibrated point cloud data based on the first rotation matrix, the ground point cloud data, and the point cloud data; a decomposition processing module for decomposing the calibrated point cloud data to obtain at least two calibration rod point cloud data sets; a third determination module for determining a rotation angle based on the calibration rod point cloud data; a fourth determination module for determining a second rotation matrix based on the rotation angle and the calibration rod point cloud data; a fifth determination module for determining a target rotation matrix based on the first and second rotation matrices; and a calibration module for calibrating lidar extrinsic parameters based on the target rotation matrix, rotation angle, calibrated ground point cloud data, calibration rod point cloud data, and vehicle information.

[0014] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the calibration method for lidar extrinsic parameters described in the first aspect or any corresponding embodiment thereof.

[0015] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the calibration method for lidar extrinsic parameters according to the first aspect or any corresponding embodiment thereof.

[0016] This invention provides a method, apparatus, computer equipment, and storage medium for calibrating the extrinsic parameters of a lidar. It determines a first rotation matrix using ground point cloud data and a target normal vector. Then, it determines calibrated ground point cloud data and calibrated point cloud data using the first rotation matrix, ground point cloud data, and point cloud data. Next, it determines the calibration rod point cloud data. Finally, it determines a second rotation matrix using the calibration rod point cloud data and rotation angle. The target rotation matrix is ​​then determined using the first and second rotation matrices. Finally, the lidar extrinsic parameters are calibrated using the target rotation matrix, rotation angle, calibrated ground point cloud, calibration rod point cloud, and vehicle information. Since this embodiment only requires a calibration rod, vehicle, and lidar to calibrate the lidar extrinsic parameters, it eliminates the need for a specific calibration environment and special calibration boards or structures, thereby reducing costs and the complexity of lidar calibration. Attached Figure Description

[0017] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a method for calibrating the extrinsic parameters of a lidar according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of a radar calibration scenario for a laser radar extrinsic parameter calibration method according to an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of another radar calibration scenario based on the calibration method for lidar extrinsic parameters according to an embodiment of the present invention;

[0021] Figure 4 This is a flowchart illustrating another method for calibrating the extrinsic parameters of a lidar according to an embodiment of the present invention;

[0022] Figure 5 This is a flowchart illustrating another method for calibrating the extrinsic parameters of a lidar according to an embodiment of the present invention;

[0023] Figure 6This is a flowchart illustrating another method for calibrating the extrinsic parameters of a lidar according to an embodiment of the present invention;

[0024] Figure 7 This is a structural block diagram of a laser radar extrinsic parameter calibration device according to an embodiment of the present invention;

[0025] Figure 8 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Based on relevant technologies, it is known that the external parameter calibration of LiDAR is one of the key steps to ensure the acquisition of accurate point cloud data. Only by determining the positional relationship between the LiDAR sensor device and the vehicle itself can the captured point cloud information be more accurately provided for subsequent tasks such as positioning, target detection, and path planning.

[0028] Currently, lidar extrinsic parameter calibration methods often employ a plane fitting method using a calibration board. Specifically, this involves: acquiring an image of the calibration board (ensuring the board is placed on a horizontal plane and maintaining a constant distance between the board and the lidar); extracting features from the calibration board (using image processing techniques to extract feature points on the board, such as corner or edge points); and using a plane fitting algorithm to fit the calibration board to a plane based on the extracted feature points. Based on the plane fitting results, the lidar's intrinsic and extrinsic parameters are then calculated.

[0029] However, using a calibration plate for planar fitting often requires the use of a special calibration plate or structure, and the calibration environment must have specific lighting and geometric conditions, which makes the calibration process for lidar complex and costly.

[0030] Based on this, this embodiment provides a calibration method for lidar extrinsic parameters. A first rotation matrix is ​​determined using ground point cloud data and the target normal vector. Then, calibrated ground point cloud data and calibrated point cloud data are determined using the first rotation matrix, ground point cloud data, and point cloud data. Next, calibration rod point cloud data is determined. A second rotation matrix is ​​determined using the calibration rod point cloud data and rotation angle. Finally, the target rotation matrix is ​​determined using the first and second rotation matrices. Finally, the lidar extrinsic parameters are calibrated using the target rotation matrix, rotation angle, calibrated ground point cloud, calibration rod point cloud, and vehicle information. Since this embodiment only requires a calibration rod, vehicle, and lidar to calibrate the lidar extrinsic parameters, it eliminates the need for a specific calibration environment and special calibration boards or structures, thereby reducing costs and the complexity of lidar calibration.

[0031] According to an embodiment of the present invention, a calibration method for the extrinsic parameters of a lidar is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] This embodiment provides a method for calibrating the extrinsic parameters of a lidar system, which can be used in computer equipment such as computers and servers. Figure 1 This is a flowchart illustrating a method for calibrating the extrinsic parameters of a lidar according to an embodiment of the present invention, as shown below. Figure 1 As shown, the process includes the following steps:

[0033] Step S101: Obtain vehicle information and point cloud data within a preset area.

[0034] Figure 2 A schematic diagram of a radar calibration scenario is shown, illustrating a method for calibrating the extrinsic parameters of a lidar system. First, the radar calibration scenario can be pre-arranged. This scenario may include: a laser pointer, a plumb line, lidar host computer software or open-source robot vision software (Robot Visualization, RViz), two calibration rods, a ruler, etc. Specifically, combined with... Figure 2 As shown, the centerline of the vehicle body is found by using a plumb line and a laser pointer. Two calibration rods are placed on the centerline in front of the vehicle (these calibration rods must be completely within the field of view of the lidar). At the same time, the horizontal distance between the two calibration rods is more than 2m. The lidar extrinsic parameters are calibrated through the lidar calibration scenario.

[0035] Figure 3 This diagram illustrates another radar calibration scenario for a method of calibrating lidar extrinsic parameters. A preset area can be used to characterize the lidar scanning area. Specifically, it combines... Figure 3 As shown, the preset area can include the ground between the calibration pole and the vehicle, and the area of ​​the calibration pole.

[0036] Vehicle information can be used to represent tire information and rear axle information. Tire information may include tire radius information, and rear axle information may include rear axle center point information, etc., without specific limitations. Point cloud data can be used to represent point cloud data corresponding to the ground and point cloud data corresponding to calibration poles. Specifically, point cloud data can be point cloud data obtained after scanning by a LiDAR, and then computer equipment can acquire the point cloud data obtained after the LiDAR scan.

[0037] Step S102: Perform point cloud segmentation on the point cloud data to obtain ground point cloud data.

[0038] Ground point cloud data can be used to represent the set of points belonging to the ground in point cloud data acquired from LiDAR. The purpose of ground point cloud extraction is to reduce interference from non-ground points in subsequent tasks and improve the efficiency and accuracy of point cloud data processing. Specifically, point cloud segmentation can be performed using methods such as surface-based segmentation, attribute-based segmentation, and deep learning methods, etc., without specific limitations, and can be implemented by those skilled in the art.

[0039] Step S103: Determine the first rotation matrix based on the ground point cloud data and the target normal vector.

[0040] The first rotation matrix can be used to characterize the rotation matrix used for the initial calibration of point cloud data and ground point cloud data. The ground normal vector corresponding to the ground point cloud data can be obtained from the ground point cloud data, and the target normal vector is (0, 0, 1). The included angle can be determined using the ground normal vector and the target normal vector, and based on the included angle, the target normal vector, and the ground normal vector, the first rotation matrix can be determined.

[0041] Step S104: Based on the first rotation matrix, ground point cloud data, and point cloud data, determine the calibrated ground point cloud data and the calibrated point cloud data.

[0042] The first rotation matrix can change the direction of a vector after multiplying it, without changing its magnitude. Therefore, by multiplying ground point cloud data and point cloud data by the aforementioned first rotation matrix, respectively, calibrated ground point cloud data and calibrated point cloud data can be obtained.

[0043] Step S105: Decompose the calibrated point cloud data to obtain at least two calibration pole point cloud data.

[0044] Since the calibrated point cloud data includes calibration rod point cloud data, it needs to be decomposed to obtain at least two calibration rod point cloud data sets. One method for decomposing the calibrated point cloud data is to determine the positions of the two calibration rods in the point cloud. Specifically, this can be achieved by searching for specific feature points or shapes in the point cloud. Based on the positions of the calibration rods, the target point cloud data is divided into two parts, each corresponding to one of the two calibration rods. This can be achieved by setting an appropriate threshold or boundary; for example, points within a certain range from the calibration rods are considered as calibration rod point cloud data. Further processing is then performed on the two segmented point cloud data sets to extract the calibration rod point cloud data. This can be achieved by extracting specific feature points or shapes, such as by fitting straight lines or circles to extract the edge points of the calibration rods, thus obtaining at least two calibration rod point cloud data sets.

[0045] Step S106: Determine the rotation angle based on the calibration rod point cloud data.

[0046] The rotation angle can be the same as the rotation angle used during the second calibration. The rotation angle can be 30°, 40°, etc., and is not specifically limited here. The rotation angle can be determined using the x-coordinate and y-coordinate of the calibration rod point cloud data. The specific determination method is described in detail below.

[0047] Step S107: Determine the second rotation matrix based on the rotation angle and the calibration rod point cloud data.

[0048] The second rotation matrix can be used to characterize the rotation matrix used during the second calibration. This second rotation matrix can be determined using the rotation angle and the calibration rod point cloud data. The specific determination method is described in detail below.

[0049] Step S108: Determine the target rotation matrix based on the first rotation matrix and the second rotation matrix.

[0050] The target rotation matrix can be used to characterize the rotation matrix used to determine the extrinsic parameters of the lidar. Specifically, the target rotation matrix can be determined by multiplying the first rotation matrix and the second rotation matrix.

[0051] Step S109: Calibrate the extrinsic parameters of the lidar based on the target rotation matrix, rotation angle, calibrated ground point cloud, calibration pole point cloud, and vehicle information.

[0052] The target rotation matrix allows us to determine the roll, pitch, and yaw parameters of the LiDAR extrinsic parameters. Roll describes the LiDAR's lateral scan, pitch describes its longitudinal scan, and yaw describes its left-right scan. The position parameters of the LiDAR extrinsic parameters can be determined using the rotation angle, the calibrated ground point cloud, the calibration pole point cloud, and vehicle information.

[0053] This embodiment provides a method for calibrating the extrinsic parameters of a lidar. A first rotation matrix is ​​determined using ground point cloud data and the target normal vector. Then, calibrated ground point cloud data and calibrated point cloud data are determined using the first rotation matrix, ground point cloud data, and point cloud data. Next, calibration rod point cloud data is determined. A second rotation matrix is ​​determined using the calibration rod point cloud data and the rotation angle. Finally, the target rotation matrix is ​​determined using the first and second rotation matrices. The lidar extrinsic parameters are calibrated using the target rotation matrix, rotation angle, calibrated ground point cloud, calibration rod point cloud, and vehicle information. Since this embodiment only requires a calibration rod, vehicle, and lidar to calibrate the lidar extrinsic parameters, it eliminates the need for a specific calibration environment and special calibration boards or structures, thereby reducing costs and the complexity of lidar calibration.

[0054] like Figure 4 As shown, in an optional implementation, step S102 includes:

[0055] Step S1021: Identify at least three point cloud data points to be processed from the point cloud data.

[0056] The point cloud data to be processed can be point cloud data used to construct a plane. Specifically, any three point cloud data points can be extracted from the point cloud data to be processed.

[0057] Step S1022: Perform plane fitting processing on the coordinate points of the point cloud data to be processed to determine the processing coefficients corresponding to the point cloud data to be processed.

[0058] Plane fitting processing can be described as the process of fitting discrete point cloud data to obtain an approximate planar model. Specifically, based on the extracted point cloud data to be processed, a linear equation can be determined, such as ax + by + cz = 1; where a, b, and c are processing coefficients. Then, the above processing system can be determined using the coordinates corresponding to the point cloud data to be processed. For example, if ax + by + cz = 1, and the coordinates corresponding to the point cloud data to be processed are A(x1, y1, z1), B(x2, y2, z2), and C(x3, y3, z3), then the system can be determined using the formula: Find a, b, and c.

[0059] Step S1023: Determine the homography matrix based on the processing coefficients.

[0060] The homography matrix is ​​a projection matrix from one plane to another. Specifically, it can be expressed by the formula: Determine the homography matrix. Here, a, b, and c can be obtained through step S1022 above, and are not specifically limited here.

[0061] Step S1024: Based on the homography matrix, determine the distance of each point cloud data from the plane formed by the point cloud data to be processed.

[0062] Specifically, the coordinates of the point cloud data in the original coordinate system are transformed using a homography matrix to obtain their coordinates in the new coordinate system. Based on the coordinates of each point cloud data point in the new coordinate system, the distances between them and the point cloud data to be processed in the new coordinate system are calculated. Since the homography matrix is ​​a combination of translation, rotation, and scaling transformations, it is necessary to convert the distances between each point cloud data point and the point cloud data to be processed in the new coordinate system back to the original coordinate system. Therefore, it is necessary to convert the distances between each point cloud data point and the point cloud data to be processed in the new coordinate system back to the original coordinate system, so as to obtain the distances between each point cloud data point and the plane formed by the point cloud data to be processed.

[0063] Step S1025: Point cloud data with a distance less than a preset distance are used as ground point cloud data.

[0064] The preset distance can be used to characterize the distance that satisfies the ground point cloud data. Specifically, after obtaining the above distances, the computer device can compare the various distances and use the point cloud data with distances less than the preset distance as ground point cloud data. For example, if the distances include 5m, 6m, and 7m, and the preset distance is 6.5m, then the point cloud data with distances of 5m and 6m are ground point cloud data.

[0065] Step S1026: Detect whether the number of ground point cloud data is less than the preset number.

[0066] After confirming that the point cloud data is ground point cloud data, it can be stored in a set, and the number of ground point cloud data in the set can be checked to see if it is less than a preset number. The preset number can be M or N, without specific limitation. If the number of ground point cloud data is greater than the preset number, the number of ground point cloud data and the plane formed by the point cloud data to be processed will be updated.

[0067] Step S1027: If the number is less than the preset number, the point cloud data with a distance less than the preset distance shall be used as ground point cloud data.

[0068] If the number is less than the preset number, then the point cloud data with a distance less than the preset distance can be directly used as the ground point cloud data.

[0069] Step S1028: Repeat the step of determining at least three point cloud data to be processed from the point cloud data until the number is less than a preset number, and taking the point cloud data with a distance less than a preset distance as ground point cloud data, until the loop condition is met, and then take the final point cloud data as ground point cloud data.

[0070] The loop condition can include two conditions. The loop condition is satisfied when either condition is met. Condition 1: The number of ground point cloud data points is not updated for n consecutive loops (e.g., Q times, Q1 times, etc., which can be set by the user and are not specifically limited here). Condition 2: The total number of loops reaches the set upper limit m times (e.g., W times, W1 times, etc., which can be set by the user and are not specifically limited here). Specifically, the computer device repeatedly executes steps S1021 to S1027 until either condition 1 or condition 2 is met, and the final number of point cloud data points is taken as the ground point cloud data.

[0071] The lidar extrinsic parameter calibration method provided in this embodiment can ensure that the amount of ground point cloud data is at an optimal level by repeatedly updating the amount of ground point cloud data, so that the lidar extrinsic parameters can be determined more accurately when calibrating them in the future.

[0072] In an optional implementation, step S103 includes:

[0073] Step S1031: Determine the normal vector of the ground point cloud data based on the ground point cloud data.

[0074] A normal vector is a concept in spatial analytic geometry; it is perpendicular to a plane and represents the direction of that plane. Specifically, the normal vector of a ground point cloud can be determined using its x-coordinate, y-coordinate, and the number of data points. The specific determination method is described in detail below.

[0075] Specifically, step S1031 includes:

[0076] Step a1: Determine the ground point cloud covariance matrix based on the ground point cloud data.

[0077]

[0078] Where C is the ground point cloud covariance matrix, x is the x-coordinate, y is the y-coordinate, z is the z-coordinate value, and n is the x-coordinate. g This refers to the amount of ground point cloud data.

[0079] Step a2: Perform singular value decomposition on the ground point cloud covariance matrix to obtain the eigenvector matrix.

[0080] M=U∑V T Where U is an m×m real symmetric orthogonal matrix, ∑ is a positive semi-definite m×n matrix, V is an n×n real symmetric orthogonal matrix, and M is the eigenvector matrix. Here, m is the number of rows in the aforementioned ground point cloud covariance matrix, and n is the number of columns in the aforementioned ground point cloud covariance matrix.

[0081] Step a3: Compare the eigenvalues ​​of each eigenvector matrix to determine the smallest eigenvalue.

[0082] Step a4: Determine the normal vector of the ground point cloud data based on the feature values.

[0083] After determining the eigenvalues ​​corresponding to each of the aforementioned ground point cloud data sets, the computer equipment can compare the eigenvalues ​​of each ground point cloud data set to determine the smallest eigenvalue. For example, ground point cloud data sets K, K1, and K2 have eigenvalues ​​L1, L2, and L3 respectively. Among these, eigenvalue L1 is greater than eigenvalue L2, which is greater than eigenvalue L3. Therefore, eigenvalue L3 corresponding to ground point cloud data set K2 is the smallest eigenvalue, and the normal vector corresponding to ground point cloud data set K2 is determined using this smallest eigenvalue.

[0084] Preferably, a certain number of the above point cloud data are randomly selected to determine the ground normal vector. Is the direction upwards from the ground? Specifically, a certain number of point cloud sets O(p1, p2, ..., p) are randomly selected from the point cloud data. n And calculate the optimal planar point cloud center point. pass Determine the ground normal vector Is the direction upwards from the ground?

[0085] in, p is the ground normal vector. i For point cloud data, This is used to represent the positive or negative value of the calculation result. Specifically, if the positive direction is less than zero, the ground normal vector is in the opposite direction; if... If the value is greater than zero, the ground normal vector takes the positive direction.

[0086] Step S1032: Determine the first rotation matrix based on the normal vector of the ground point cloud data and the target normal vector.

[0087] After determining the normal vector and target normal vector of the ground point cloud data, the computer device can specifically, as described in step S1032 above:

[0088] Step b1: Determine the vector product and the included angle based on the normal vector of the ground point cloud data and the target normal vector.

[0089] Let the normal vector of the ground point cloud data be (a i ,b i ,c i The target normal vector is (0, 0, 1), obtained using the formula: Determine the vector product; where, It is a vector product.

[0090] Through the formula: Determine the included angle; where θ is the included angle. The normal vector of the ground point cloud data. The target normal vector.

[0091] Step b2: Determine the unit vectors in the same direction as the vector product.

[0092] Through the above vectors Able to determine the above vectors Unit vectors in the same direction

[0093] Step b3: Determine the first rotation matrix based on the included angle and the unit vector.

[0094]

[0095] Where R1 is the first rotation matrix and θ is the included angle.

[0096] In an optional implementation, step S106 includes:

[0097] Step S1061: Based on the calibration pole point cloud data, determine the average abscissa and average ordinate of each calibration pole.

[0098] Since the calibration pole point cloud data is in a coordinate system, the point cloud data of each calibration pole can be represented using a coordinate system. The computer equipment can average the point cloud data of all calibration poles for each calibration pole to obtain the corresponding average x-coordinate and average y-coordinate. For example, if the calibration pole includes calibration pole point cloud data J1, J2, and J3, the average x-coordinate and average y-coordinate of calibration pole point cloud data J1, J2, and J3 can be averaged to obtain the average x-coordinate and average y-coordinate of that calibration pole.

[0099] Step S1062: Determine the rotation angle based on the average horizontal coordinate and the average vertical coordinate.

[0100] Where yaw1 is the rotation angle. as well as The vertical and horizontal coordinates of the calibration rod closest to the vehicle. as well as The vertical and horizontal coordinates are for the calibration rods that are far from the vehicle.

[0101] The above step S107 includes: Where R2 is the second rotation matrix.

[0102] The lidar extrinsic parameter calibration method provided in this embodiment can accurately determine the rotation angle by determining the average abscissa and average ordinate corresponding to each calibration rod, thereby enabling more accurate determination of the lidar extrinsic parameters during subsequent calibration.

[0103] like Figure 5 As shown, in an optional implementation, the vehicle information includes rear axle information and tire radius information; wherein, step S109 includes:

[0104] Step S1091: Determine the number of calibrated ground point clouds based on the calibrated ground point cloud data.

[0105] The number of calibrated ground point cloud data can be determined by the ground point cloud data itself. If there are 5 ground point cloud data points, then the corresponding number is 5; if there are 15 ground point cloud data points, then the corresponding number is 15. No specific limit is imposed here.

[0106] Step S1092: Determine the average value of the calibrated ground point cloud data based on the coordinates of each calibrated ground point cloud data and the number of calibrated ground point cloud data.

[0107] in, Let z be the average value, n be the number of ground point clouds after calibration, and z be the average value. i Let z be the z-coordinate of the i-th point.

[0108] Step S1093: Based on the coordinates of the calibration pole point cloud data, determine the coordinates of the calibration pole point cloud data closest to the vehicle.

[0109] The calibration pole point cloud data includes calibration pole point cloud data that is close to the vehicle and calibration pole point cloud data that is far from the vehicle. Therefore, the coordinates of the calibration pole point cloud data that is close to the vehicle can be determined by using the coordinates of the calibration pole point cloud data.

[0110] Step S1094: Based on the coordinates of the calibration pole point cloud near the vehicle, rotate it around the Z-axis according to the rotation angle to obtain the coordinates of the target calibration pole point cloud data.

[0111] The Z-axis can be used to characterize the vehicle's height. Specifically, the coordinates of the calibration pole point cloud data closest to the vehicle can be rotated along the vehicle's height direction to obtain the coordinates of the target calibration pole point cloud data. For example: After rotation, it can become

[0112] Step S1095: Determine the first calibration parameter based on the distance from the abscissa of the target calibration pole point cloud data to the rear axis information.

[0113] Where X is the first calibration parameter, x pn The distance from the nearest calibration rod to the center of the rear axle. The x-coordinate of the target calibration pole point cloud data.

[0114] Step S1096: Determine the second calibration parameter based on the ordinate of the target calibration pole point cloud data.

[0115] Where y is the second calibration parameter. The vertical coordinate of the target calibration pole point cloud data.

[0116] Step S1097: Determine the third calibration parameter based on the tire radius information and the average value of the calibrated ground point cloud data.

[0117] Where z is the third calibration parameter, The value of the calibrated ground point cloud data is denoted as r, where r represents the tire radius. The tire radius can be 0.5m, 0.7m, etc., and is not specifically limited here.

[0118] Step S1098: Perform Euler decomposition on the target rotation matrix to determine the fourth calibration parameter.

[0119] Euler decomposition is a method that decomposes a rotation matrix into three rotation axes and corresponding angles, which can be used to obtain the values ​​of roll, pitch, and yaw. The fourth calibration parameter includes the values ​​of roll, pitch, and yaw.

[0120] Step S1099: Based on the first calibration parameter, the second calibration parameter, the third calibration parameter, and the fourth calibration parameter, calibrate the external parameters of the lidar.

[0121] After determining the above calibration parameters, the external parameters of the lidar can be calibrated using these parameters.

[0122] The lidar extrinsic parameter calibration method provided in this embodiment determines the aforementioned calibration parameters by determining the tire height and the distance from the calibration rod to the rear axle center. Compared to other current single lidar calibration methods that can only calibrate three parameters (roll, pitch, and yaw), this method can reduce costs while improving the accuracy of lidar calibration.

[0123] Combination Figure 6 As shown, in one optional embodiment, a method for calibrating the extrinsic parameters of a lidar is provided, the method comprising:

[0124] First, set up the scene, as described above. Figure 2 as well as Figure 3 The content described herein will not be elaborated upon here. The original point cloud is scanned using a lidar scanner. The original point cloud includes a ground point cloud and a calibration pole point cloud. Ground segmentation is performed on the original point cloud to obtain a ground point cloud. Through steps S1031 to b3, a rotation matrix R1 (i.e., the first rotation matrix) is obtained from the ground point cloud. Rotation matrix R1 is used to rotate the ground point cloud and the original point cloud to obtain the target point cloud after the first calibration (the ground point cloud after the first calibration and the original point cloud after the first calibration). The original point cloud after the first calibration is segmented to obtain the calibration pole point cloud. Then, through steps S1061, S1062, and S17, a rotation matrix R2 (i.e., the second rotation matrix) is obtained from the calibration pole point cloud. Multiplying rotation matrices R1 and R2 yields the total rotation matrix R (i.e., the target rotation matrix). Euler decomposition of the total rotation matrix R yields Roll, Pitch, and Yaw. Using the tire radius, the distance between the calibration rod and the rear axle center, and the target point cloud after the second calibration, x, y, and z can be obtained through steps S1091 to S1099.

[0125] The calibration method for lidar extrinsic parameters provided in this embodiment is simpler to set up and more feasible than other calibration methods (such as using a wall to correct the yaw angle).

[0126] Furthermore, compared to other single-laser calibration methods that can only calibrate three calibration parameters—roll, pitch, and yaw—this invention can automatically calculate the above six calibration parameters by simply measuring the tire height and the distance from the calibration rod to the center of the rear axle.

[0127] This embodiment also provides a calibration device for lidar extrinsic parameters, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0128] This embodiment provides a calibration device for the extrinsic parameters of a lidar system, such as... Figure 7As shown, the system includes: an acquisition module 701 for acquiring vehicle information and point cloud data within a preset area; a segmentation processing module 702 for segmenting the point cloud data to obtain ground point cloud data; a first determination module 703 for determining a first rotation matrix based on the ground point cloud data and the target normal vector; a second determination module 704 for determining calibrated ground point cloud data and calibrated point cloud data based on the first rotation matrix, the ground point cloud data, and the point cloud data; a decomposition processing module 705 for decomposing the calibrated point cloud data to obtain at least two calibration rod point cloud data; a third determination module 706 for determining a rotation angle based on the calibration rod point cloud data; a fourth determination module 707 for determining a second rotation matrix based on the rotation angle and the calibration rod point cloud data; a fifth determination module 708 for determining a target rotation matrix based on the first and second rotation matrices; and a calibration module 709 for calibrating the extrinsic parameters of the lidar based on the target rotation matrix, rotation angle, calibrated ground point cloud, calibration rod point cloud, and vehicle information.

[0129] In some optional implementations, the segmentation processing module 702 includes: a first determining unit, configured to determine at least three point cloud data to be processed from the point cloud data; a second determining unit, configured to perform plane fitting processing on the coordinate points of the point cloud data to be processed to determine the processing coefficients corresponding to the point cloud data to be processed; a third determining unit, configured to determine the homography matrix based on the processing coefficients; a fourth determining unit, configured to determine the distance of each point cloud data from the plane formed by the point cloud data to be processed based on the homography matrix; a fifth determining unit, configured to use point cloud data with a distance less than a preset distance as ground point cloud data; a detection unit, configured to detect whether the number of ground point cloud data is less than a preset number; a sixth determining unit, configured to use point cloud data with a distance less than a preset distance as ground point cloud data if the number is less than the preset number; and a repetition execution unit, configured to repeatedly execute the step of determining at least three point cloud data to be processed from the point cloud data to the step of using point cloud data with a distance less than a preset distance as ground point cloud data if the number is less than the preset number, until the loop condition is met, and then use the final point cloud data as ground point cloud data.

[0130] In some optional implementations, the first determining module 703 includes: a seventh determining unit, used to determine the normal vector of the ground point cloud data based on the ground point cloud data; and an eighth determining unit, used to determine the first rotation matrix based on the normal vector of the ground point cloud data and the target normal vector.

[0131] In some optional implementations, the seventh determining unit includes: a first determining subunit, used to determine the ground point cloud covariance matrix based on the ground point cloud data; a singular value decomposition subunit, used to perform singular value decomposition on the ground point cloud covariance matrix to obtain an eigenvector matrix; a comparison subunit, used to compare the eigenvalues ​​of each eigenvector matrix to determine the smallest eigenvalue; and a second determining subunit, used to determine the normal vector of the ground point cloud data based on the eigenvalue.

[0132] In some optional implementations, the eighth determining unit includes: a third determining subunit for determining a vector product and an included angle based on the normal vector of the ground point cloud data and the target normal vector; a fourth determining subunit for determining a unit vector in the same direction as the vector product; and a fifth determining subunit for determining a first rotation matrix based on the included angle and the unit vector.

[0133] In some optional implementations, the third determining module 706 includes: a ninth determining unit, used to determine the average abscissa and average ordinate of each calibration rod based on the calibration rod point cloud data; and a tenth determining unit, used to determine the rotation angle based on the average abscissa and average ordinate.

[0134] In some optional implementations, the vehicle information includes rear axle information and tire radius information; wherein, the calibration module 709 includes: an eleventh determining unit, used to determine the number of calibrated ground point cloud data based on the calibrated ground point cloud data; a twelfth determining unit, used to determine the average value of the calibrated ground point cloud data based on the coordinates of each calibrated ground point cloud data and the number of calibrated ground point cloud data; a thirteenth determining unit, used to determine the coordinates of the calibration rod point cloud data near the vehicle based on the coordinates of the calibration rod point cloud data; and a rotation unit, used to rotate the calibration rod point cloud near the vehicle about the Z-axis by a rotation angle. The system obtains the coordinates of the target calibration pole point cloud data; the fourteenth determination unit determines the first calibration parameter based on the distance from the horizontal coordinate of the target calibration pole point cloud data to the rear axis information; the fifteenth determination unit determines the second calibration parameter based on the vertical coordinate of the target calibration pole point cloud data; the sixteenth determination unit determines the third calibration parameter based on the tire radius information and the average value of the calibrated ground point cloud data; the seventeenth determination unit performs Eulerian decomposition on the target rotation matrix to determine the fourth calibration parameter; and the calibration unit calibrates the lidar extrinsic parameters based on the first, second, third, and fourth calibration parameters.

[0135] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0136] In this embodiment, the calibration device for the external parameters of the lidar is presented in the form of a functional unit. Here, a functional unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0137] This invention also provides a computer device having the above-described features. Figure 7 The calibration device for the external parameters of the lidar shown is shown.

[0138] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 8 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 8 Take a processor 10 as an example.

[0139] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0140] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0141] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0142] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0143] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0144] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0145] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for calibrating the extrinsic parameters of a lidar system, characterized in that, include: Acquire vehicle information and point cloud data within a preset area; The point cloud data is segmented to obtain ground point cloud data; Based on the ground point cloud data and the target normal vector, determine the first rotation matrix; Based on the first rotation matrix, the ground point cloud data, and the point cloud data, the calibrated ground point cloud data and the calibrated point cloud data are determined; The calibrated point cloud data is decomposed to obtain at least two calibration pole point cloud data sets. Based on the calibration pole point cloud data, determine the rotation angle; Based on the rotation angle and the calibration rod point cloud data, determine the second rotation matrix; Based on the first rotation matrix and the second rotation matrix, determine the target rotation matrix; The extrinsic parameters of the lidar are calibrated based on the target rotation matrix, the rotation angle, the calibrated ground point cloud, the calibration rod point cloud, and the vehicle information. The vehicle information includes rear axle information and tire radius information; wherein, the calibration of the lidar extrinsic parameters based on the target rotation matrix, the rotation angle, the calibrated ground point cloud, the calibration rod point cloud, and the vehicle information includes: determining the number of calibrated ground point clouds based on the calibrated ground point cloud data; determining the average value of the calibrated ground point cloud data based on the coordinates of each calibrated ground point cloud data and the number of calibrated ground point cloud data; determining the coordinates of the calibration rod point cloud data near the vehicle based on the coordinates of the calibration rod point cloud data; and determining the coordinates of the calibration rod point cloud data near the vehicle based on the coordinates of the calibration rod point cloud data. The coordinates of the point cloud are rotated around the Z-axis according to the rotation angle to obtain the coordinates of the target calibration pole point cloud data; a first calibration parameter is determined based on the distance from the x-coordinate of the target calibration pole point cloud data to the rear axis information; a second calibration parameter is determined based on the y-coordinate of the target calibration pole point cloud data; a third calibration parameter is determined based on the tire radius information and the average value of the calibrated ground point cloud data; an Eulerian decomposition is performed on the target rotation matrix to determine a fourth calibration parameter; and the extrinsic parameters of the lidar are calibrated based on the first, second, third, and fourth calibration parameters.

2. The method for calibrating the extrinsic parameters of a lidar according to claim 1, characterized in that, The step of performing point cloud segmentation on the point cloud data to obtain ground point cloud data includes: Identify at least three point cloud data sets to be processed from the point cloud data; The coordinate points of the point cloud data to be processed are subjected to plane fitting processing to determine the processing coefficients corresponding to the point cloud data to be processed. Based on the processing coefficients, determine the homography matrix; Based on the homography matrix, the distance of each point cloud data from the plane formed by the point cloud data to be processed is determined; Point cloud data whose distance is less than a preset distance are used as ground point cloud data; Detect whether the number of ground point cloud data is less than a preset number; If the quantity is less than the preset quantity, the point cloud data with a distance less than the preset distance shall be used as the ground point cloud data; The step of repeatedly executing the step of determining at least three point cloud data to be processed from the point cloud data until the number is less than the preset number, and taking the point cloud data with a distance less than the preset distance as the ground point cloud data, is repeated until the loop condition is met, and the final point cloud data is taken as the ground point cloud data.

3. The method for calibrating the extrinsic parameters of a lidar according to claim 1, characterized in that, Based on the ground point cloud data and the target normal vector, the first rotation matrix is ​​determined, including: Based on the ground point cloud data, determine the normal vector of the ground point cloud data; The first rotation matrix is ​​determined based on the normal vector of the ground point cloud data and the target normal vector.

4. The method for calibrating the extrinsic parameters of a lidar according to claim 3, characterized in that, Determining the normal vector of the ground point cloud data based on the ground point cloud data includes: Based on the aforementioned ground point cloud data, determine the ground point cloud covariance matrix; Singular value decomposition is performed on the ground point cloud covariance matrix to obtain the eigenvector matrix; By comparing the eigenvalues ​​of each of the eigenvector matrices, the smallest eigenvalue is determined; Based on the feature values, the normal vector of the ground point cloud data is determined.

5. The method for calibrating the extrinsic parameters of a lidar according to claim 3, characterized in that, Based on the normal vectors of the ground point cloud data and the target normal vector, the first rotation matrix is ​​determined, including: Based on the normal vector and target normal vector of the ground point cloud data, determine the vector product and the included angle; Determine the unit vectors in the same direction as the product of the vectors; The first rotation matrix is ​​determined based on the included angle and the unit vector.

6. The method for calibrating the extrinsic parameters of a lidar according to claim 1, characterized in that, The step of determining the rotation angle based on the calibration pole point cloud data includes: Based on the point cloud data of the calibration poles, the average abscissa and average ordinate of each calibration pole are determined. The rotation angle is determined based on the average horizontal coordinate and the average vertical coordinate.

7. A calibration device for the extrinsic parameters of a lidar system, characterized in that, The device includes: The acquisition module is used to acquire vehicle information and point cloud data within a preset area; The segmentation processing module is used to perform point cloud segmentation processing on the point cloud data to obtain ground point cloud data; The first determining module is used to determine the first rotation matrix based on the ground point cloud data and the target normal vector; The second determining module is used to determine the calibrated ground point cloud data and the calibrated point cloud data based on the first rotation matrix, the ground point cloud data and the point cloud data. The decomposition processing module is used to decompose the calibrated point cloud data to obtain at least two calibration rod point cloud data. The third determining module is used to determine the rotation angle based on the calibration rod point cloud data; The fourth determining module is used to determine the second rotation matrix based on the rotation angle and the calibration rod point cloud data; The fifth determining module is used to determine the target rotation matrix based on the first rotation matrix and the second rotation matrix; The calibration module is used to calibrate the extrinsic parameters of the lidar based on the target rotation matrix, the rotation angle, the calibrated ground point cloud, the calibration rod point cloud, and the vehicle information. The calibration module includes: an eleventh determining unit, used to determine the number of calibrated ground point clouds based on the calibrated ground point cloud data; a twelfth determining unit, used to determine the average value of the calibrated ground point cloud data based on the coordinates of each calibrated ground point cloud data and the number of calibrated ground point cloud data; a thirteenth determining unit, used to determine the coordinates of the calibration pole point cloud data near the vehicle based on the coordinates of the calibration pole point cloud data; and a rotation unit, used to rotate the calibration pole point cloud data near the vehicle around the Z-axis by a rotation angle to obtain the coordinates of the target calibration pole point cloud data; the tenth... The four determination units are used to determine the first calibration parameter based on the distance from the abscissa of the target calibration pole point cloud data to the rear axis information; the fifteenth determination unit is used to determine the second calibration parameter based on the ordinate of the target calibration pole point cloud data; the sixteenth determination unit is used to determine the third calibration parameter based on the tire radius information and the average value of the calibrated ground point cloud data; the seventeenth determination unit is used to perform Eulerian decomposition on the target rotation matrix to determine the fourth calibration parameter; and the calibration unit is used to calibrate the lidar extrinsic parameters based on the first, second, third, and fourth calibration parameters.

8. A computer device, characterized in that, include: A memory and a processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the calibration method for the external parameters of the lidar as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the calibration method for the external parameters of the lidar as described in any one of claims 1 to 6.

Images