External parameter calibration method and device, computer device and storage medium

Abstract

The application relates to an external parameter calibration method and device, computer equipment and a storage medium. The method comprises the following steps: determining a driving track point of a laser radar according to collected point cloud data, determining a position measurement point of a global positioning system (GPS) according to collected longitude and latitude data, constructing a position constraint according to the driving track point of the laser radar and the position measurement point of the GPS, and solving a target of the position constraint to obtain an external parameter matrix of the laser radar to the GPS. The method provides an external parameter calibration method of the GPS and the laser radar, and the external parameter matrix of the GPS is solved according to the position constraint constructed between the driving track point of the laser radar and the position measurement point of the GPS, so that the calibration accuracy of the external parameter calibration can be ensured.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to an external parameter calibration method, apparatus, computer equipment, and storage medium. Background Technology

[0002] Simultaneous Localization and Mapping (SLAM) algorithms for high-precision map building and real-time vehicle localization is a mature and widely recognized reliable approach to autonomous driving. Fusion sensing and localization using multiple sensors is an effective method to improve the reliability of this approach. LiDAR and GPS are common sensors used in autonomous vehicles, and their joint calibration is a fundamental prerequisite for high-precision environmental map building and accurate real-time vehicle localization.

[0003] In both SLAM and sensor extrinsic parameter calibration, calculating radar pose changes through point cloud matching is the most fundamental method. The Normal Distributions Transform (NDT) algorithm is one of the commonly used classic matching algorithms. However, its current drawback is that it uses a fixed-size grid during mesh generation. If the grid is too large, it can lead to the loss of some local features; if it is too small, it will increase the computational load significantly and lose statistical universality. Some data can also have an excessive impact on the results. In addition, finding the grid corresponding to each point quickly is crucial for search speed. On the other hand, in GPS and lidar extrinsic parameter calibration, a common calibration method is to integrate the motion changes of the inertial measurement unit (IMU) or global navigation satellite system (GNSS). Combining this pose change with the lidar pose estimate, the current radar scan frame is transformed. The Iterative Closest Point (ICP) algorithm is used to register the point clouds of the two frames before and after the transformation to obtain the transformation matrix estimate, thereby calculating the extrinsic parameter transformation matrix of the lidar and IMU or GNSS. The methods described above are mostly used for extrinsic parameter calibration of IMUs and LiDARs on small mobile platforms such as drones, where attitude changes are rapid. These methods require highly non-planar motion and constraints on three-axis attitude. However, they are unsuitable for autonomous vehicles, which have a large range of motion, minimal changes in motion perpendicular to the ground, and are far from surrounding buildings with weaker constraints. This results in these methods failing to provide accurate estimates of the extrinsic parameter matrices for the onboard sensors.

[0004] Therefore, existing GPS and lidar extrinsic parameter calibration methods suffer from inaccurate calibration. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, computer equipment, and computer-readable storage medium for extrinsic parameter calibration of GPS and lidar, which can improve calibration accuracy and address the above-mentioned technical problems.

[0006] Firstly, this application provides a method for calibrating external parameters. The method includes:

[0007] The driving trajectory points of the lidar are determined based on the collected point cloud data;

[0008] The location measurement points of the Global Positioning System (GPS) are determined based on the collected latitude and longitude data;

[0009] Position constraints are constructed based on the driving trajectory points of the lidar and the position measurement points of the GPS, and the external parameter matrix of the lidar to the GPS is obtained by solving the position constraints as the objective.

[0010] In one embodiment, determining the driving trajectory points of the lidar based on the collected point cloud data includes:

[0011] The first frame of point cloud data in the collected point cloud data is determined as the reference point cloud data, and the next frame of point cloud data in the reference point cloud data is determined as the point cloud data to be registered.

[0012] The reference point cloud data and the point cloud data to be registered are registered to obtain the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

[0013] The current point cloud data to be registered is redefined as the new reference point cloud data, and the next frame of point cloud data of the new reference point cloud data is defined as the new point cloud data to be registered. Then, the step of registering the reference point cloud data and the point cloud data to be registered is returned to be executed until the point cloud data of all frames in the acquired point cloud data is registered, and multiple sets of LiDAR pose transformation matrices are obtained.

[0014] The driving trajectory points of the lidar are determined based on the pose transformation matrices of multiple lidar units.

[0015] In one embodiment, the registration of the reference point cloud data and the point cloud data to be registered to obtain the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered includes:

[0016] The Normal Distribution Transform (NDT) algorithm is used to register the reference point cloud data and the point cloud data to be registered, thereby obtaining the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

[0017] In one embodiment, the registration of the reference point cloud data and the point cloud data to be registered to obtain the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered includes:

[0018] The reference point cloud data is divided into spherical coordinate grids, and an octree is constructed using the center point of each spherical grid.

[0019] Perform spherical coordinate point cloud transformation on the point cloud data to be registered to obtain the transformed point cloud points;

[0020] Search the octree for grid points that match each of the point cloud points;

[0021] The pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered is determined based on the searched grid points and the matched point cloud points.

[0022] In one embodiment, the step of performing spherical coordinate point cloud transformation on the point cloud data to be registered to obtain the transformed point cloud points includes:

[0023] The coordinates of the point cloud data to be registered are transformed into segments, sectors, and rings to obtain the transformed point cloud points.

[0024] In one embodiment, determining the driving trajectory points of the lidar based on the searched grid points and matching point cloud points includes:

[0025] The likelihood function is maximized for the searched grid points and matched point cloud points to determine the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

[0026] In one embodiment, determining the driving trajectory points of the lidar based on the searched grid points and matching point cloud points includes:

[0027] The Normal Distribution Transform (NDT) algorithm is used to register the searched grid points with the matched point cloud points to determine the driving trajectory points of the lidar.

[0028] In one embodiment, determining the location measurement point of the Global Positioning System (GPS) based on the collected latitude and longitude data includes:

[0029] The collected latitude and longitude data are converted into distance values ​​in the Mercator plane projection coordinate system to obtain the location measurement points of the GPS.

[0030] In one embodiment, the step of constructing position constraints based on the driving trajectory points of the lidar and the position measurement points of the GPS includes:

[0031] A KD tree is constructed using the GPS location measurement points, and a K-nearest neighbor search is performed on the driving trajectory point of each lidar to determine a preset number of GPS location measurement points that are closest to the driving trajectory point of each lidar.

[0032] Calculate the center point of the preset number of GPS location measurement points;

[0033] The position constraint is constructed using the center point and the corresponding driving trajectory point of the lidar.

[0034] In one embodiment, the step of solving for the extrinsic parameter matrix of the laser radar to the GPS with the position constraint as the objective includes:

[0035] Using the positional constraint as the objective, the Ceres algorithm is used to minimize the distance residual between the center point and the corresponding driving trajectory point of the lidar, and the corresponding extrinsic parameter matrix is ​​output.

[0036] Secondly, this application also provides an external parameter calibration device. The device includes:

[0037] The first determining module is used to determine the driving trajectory points of the lidar based on the collected point cloud data;

[0038] The second determining module is used to determine the location measurement points of the Global Positioning System (GPS) based on the collected latitude and longitude data;

[0039] The solution module is used to construct position constraints based on the driving trajectory points of the lidar and the position measurement points of the GPS, and to solve for the position constraints to obtain the extrinsic parameter matrix of the lidar to the GPS.

[0040] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described in the first aspect.

[0041] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the method described in the first aspect.

[0042] The aforementioned extrinsic parameter calibration method, apparatus, computer equipment, and storage medium determine the driving trajectory points of the lidar based on collected point cloud data and the position measurement points of the Global Positioning System (GPS) based on collected latitude and longitude data. Then, position constraints are constructed based on the lidar's driving trajectory points and the GPS position measurement points, and the extrinsic parameter matrix from the lidar to GPS is obtained by solving for these position constraints. This method provides an extrinsic parameter calibration method for GPS and lidar, and by solving for the GPS extrinsic parameter matrix based on the position constraints constructed between the lidar's driving trajectory points and the GPS position measurement points, the accuracy of the extrinsic parameter calibration can be guaranteed. Attached Figure Description

[0043] Figure 1 This is a diagram illustrating the application environment of an extrinsic parameter calibration method in one embodiment.

[0044] Figure 2 This is a flowchart illustrating an external parameter calibration method in one embodiment;

[0045] Figure 3 for Figure 2 A flowchart illustrating one implementation of S101 in the embodiment;

[0046] Figure 4 for Figure 3 A flowchart illustrating one implementation of S202 in the embodiment;

[0047] Figure 5 for Figure 1 A flowchart illustrating one implementation of S103 in the embodiment;

[0048] Figure 6 This is a flowchart illustrating an external parameter calibration method in one embodiment;

[0049] Figure 7 for Figure 6 A flowchart illustrating one implementation of S5031 in the embodiment;

[0050] Figure 8 This is a schematic diagram of the external parameter calibration system in one embodiment;

[0051] Figure 9 This is a structural block diagram of an external parameter calibration device in one embodiment;

[0052] Figure 10 This is a structural block diagram of an external parameter calibration device in one embodiment;

[0053] Figure 11 This is a structural block diagram of an external parameter calibration device in one embodiment;

[0054] Figure 12This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0056] The external parameter calibration method provided in this application embodiment can be applied to, for example... Figure 1 The application environment is shown. LiDAR 110 and GPS 120 are installed on the vehicle, and data processing platform 130 communicates with LiDAR 110 and GPS 120 via a network. LiDAR 110 is used to collect point cloud data of the area surrounding the vehicle; GPS 120 is used to collect the vehicle's location information; data processing platform 130 is used to analyze and process the point cloud data and location information to calibrate the extrinsic parameters of LiDAR 110 and GPS 120, and to construct a map of the area surrounding the vehicle based on the extrinsic parameter calibration. The vehicle can be a conventional vehicle or an autonomous vehicle. Data processing platform 130 can be a terminal, which can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. Data processing platform 130 can also be a server, which can be a standalone server or a server cluster composed of multiple servers. LiDAR 110 can be a vehicle-mounted LiDAR.

[0057] In one embodiment, such as Figure 2 As shown, an extrinsic parameter calibration method is provided, which is applied to the extrinsic parameter calibration of lidar and GPS. This method is applied to... Figure 1 Taking the data processing platform in China as an example, the following steps are included:

[0058] S101 determines the driving trajectory points of the lidar based on the collected point cloud data.

[0059] Point cloud data is obtained by synchronizing the LiDAR with the vehicle's GPS, using relevant data acquisition software to capture the three-dimensional coordinate measurements of the vehicle's surrounding environment during driving, and then saving them synchronously with the GPS data.

[0060] In this embodiment, the lidar can be installed on the vehicle and collect point cloud data of the area surrounding the vehicle while it is in motion. The collected point cloud data is then sent to a data processing platform. Upon receiving the point cloud data, the platform analyzes it to determine the lidar's pose during vehicle movement, i.e., its trajectory points. It should be noted that the vehicle can collect point cloud data within a closed, annular area to improve the accuracy of subsequent calculations of the extrinsic parameter matrix based on this data.

[0061] S102, determine the location measurement point of the Global Positioning System (GPS) based on the collected latitude and longitude data.

[0062] Among them, the latitude and longitude data are obtained by synchronizing the GPS with the LiDAR on the vehicle, and then using relevant data acquisition software to obtain the latitude and longitude measurement values ​​of the on-board GPS during the driving process, which are then saved synchronously with the LiDAR point cloud data.

[0063] In this embodiment, a GPS device can be installed on the vehicle and collect latitude and longitude data of the area surrounding the vehicle while it is in motion. This collected data is then sent to a data processing platform. Upon receiving the data, the platform analyzes and processes it to determine the GPS location measurement points during the vehicle's movement. Specifically, after obtaining the GPS location measurement points, the platform saves them in PointXYZ data format within a Point Cloud Library (PCL) for later processing. It should be noted that the vehicle can collect GPS latitude and longitude data within a closed, circular area to improve the accuracy of subsequent calculations of the extrinsic parameter matrix based on this data.

[0064] S103: Based on the driving trajectory points of the lidar and the position measurement points of the GPS, position constraints are constructed, and the solution is performed with the position constraints as the objective to obtain the extrinsic parameter matrix of lidar to GPS.

[0065] When the data processing platform acquires the driving trajectory points of the LiDAR and the position measurement points of the GPS, it can first determine the GPS position measurement points that match the LiDAR driving trajectory points and calculate the distances between all LiDAR driving trajectory points and the matched GPS position measurement points. Then, based on the distances between all LiDAR driving trajectory points and the matched GPS position measurement points, iteratively calculate the extrinsic parameter matrix between the LiDAR and GPS. Specifically, in determining the extrinsic parameter matrix between the LiDAR and GPS, the extrinsic parameter matrix can be calculated once for each frame of LiDAR driving trajectory points and each frame of GPS position measurement points. Then, the extrinsic parameter matrix obtained in the previous frame is used as the initial value for calculating the extrinsic parameter matrix in the next frame for iterative calculation until the final extrinsic parameter matrix is ​​calculated based on the LiDAR driving trajectory points and GPS position measurement points of all frames.

[0066] The extrinsic parameter calibration method provided in the above embodiments determines the driving trajectory points of the lidar based on the collected point cloud data, and determines the position measurement points of the Global Positioning System (GPS) based on the collected latitude and longitude data. Then, position constraints are constructed based on the lidar's driving trajectory points and the GPS position measurement points, and the extrinsic parameter matrix from the lidar to the GPS is obtained by solving for these position constraints. This method provides an extrinsic parameter calibration method for GPS and lidar, and by solving for the GPS extrinsic parameter matrix based on the position constraints constructed between the lidar's driving trajectory points and the GPS position measurement points, the accuracy of the extrinsic parameter calibration can be guaranteed.

[0067] In one embodiment, a method for accurately calculating the driving trajectory points of the lidar is also provided. Therefore, this is one implementation of the above-mentioned S101 "determining the driving trajectory points of the lidar based on the collected point cloud data," such as... Figure 3 As shown, this implementation method includes:

[0068] S201, the first frame of point cloud data in the collected point cloud data is determined as the reference point cloud data, and the next frame of point cloud data in the reference point cloud data is determined as the point cloud data to be registered.

[0069] The data processing platform can process point cloud data frame by frame. In determining the driving trajectory points of the LiDAR based on the collected point cloud data, it is necessary to process the point cloud data frame by frame. Furthermore, when processing each frame of point cloud data, it is necessary to register it with reference point cloud data to determine the LiDAR's driving trajectory points. Therefore, in this embodiment, the first frame of point cloud data is extracted from the collected point cloud data as the reference point cloud data, and the next frame of point cloud data is extracted as the point cloud data to be registered.

[0070] S202, register the reference point cloud data and the point cloud data to be registered to obtain the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

[0071] The pose transformation matrix of the lidar can be the pose transformation matrix between a frame of reference point cloud data and a frame of point cloud data to be registered.

[0072] In this embodiment, when the data processing platform determines the reference point cloud data and the point cloud data to be registered, it can use the corresponding point cloud registration algorithm to register the reference point cloud data and the point cloud data to be registered, thereby obtaining the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered. For example, the Normal Distribution Transform (NDT) algorithm can be used to register the reference point cloud data and the point cloud data to be registered, thereby obtaining the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

[0073] S203, re-determine the current point cloud to be registered as the new reference point cloud, and determine the next frame of point cloud data of the new reference point cloud as the new point cloud to be registered, and return to execute the step of registering the reference point cloud and the point cloud to be registered, until the point cloud data of all frames in the acquired point cloud data are registered, and obtain multiple sets of LiDAR pose transformation matrices.

[0074] This embodiment uses a cyclical process. Specifically, when the data processing platform obtains the pose transformation matrix of the lidar based on the aforementioned steps S201-S202, that is, after obtaining the pose transformation matrix of the lidar between the reference point cloud data of the first frame and the point cloud data to be registered in the next frame, it can continue to calculate the pose transformation matrix of the lidar between the reference point cloud data of the second frame and the point cloud data to be registered in the next frame based on the aforementioned steps S201-S202. Then, it continues to calculate the pose transformation matrix of the lidar between the reference point cloud data of the third frame and the point cloud data to be registered in the next frame based on the aforementioned steps S201-S202. This process is repeated until the pose transformation matrices of the lidar between all frames in the point cloud data are calculated, resulting in multiple sets of lidar pose transformation matrices. Each set of lidar pose transformation matrices corresponds to the pose transformation matrix between two frames of point cloud data.

[0075] S204, determine the driving trajectory points of the lidar based on the pose transformation matrix of multiple lidars.

[0076] Once the data processing platform acquires the pose transformation matrices of multiple LiDAR sensors, it can analyze these matrices—that is, analyze the pose transformation of the LiDAR sensors across different frames of point cloud data—to determine the driving trajectory points of the LiDAR sensors during vehicle operation. This embodiment directly constructs position constraints from the GPS position measurement points and the LiDAR driving trajectory points, performing a nonlinear iterative solution to improve the efficiency and accuracy of calculating the extrinsic parameters between the two.

[0077] Furthermore, one implementation of the above S202 is provided, such as... Figure 4 As shown, this implementation method includes:

[0078] S301 divides the baseline point cloud data into spherical coordinate grids and constructs an octree using the center point of each spherical grid.

[0079] When the data processing platform acquires the reference point cloud data, it can convert the three-dimensional spatial coordinates of the reference point cloud data into spherical coordinates, and then divide the reference point cloud data into spherical coordinate grids. Then, an octree is constructed with the center point of each spherical grid. Since the parameters of the spherical coordinate grid division are preset, the number of grids is a fixed value. That is, the minimum granularity of the octree constructed based on the grid is fixed. This can greatly improve the efficiency of tree construction and search, and facilitate the rapid completion of the corresponding spherical coordinate grid lookup and pose matching calculation of the point cloud data to be registered.

[0080] S302, perform spherical coordinate point cloud transformation on the point cloud data to be registered to obtain the transformed point cloud points.

[0081] In this embodiment, when the data processing platform acquires the point cloud data to be registered, it can perform coordinate transformations on the point cloud data to be registered into segments, sectors, and rings to obtain the transformed point cloud points. Specifically, the point cloud data to be registered, which is in a three-dimensional Cartesian coordinate system, is transformed into a spherical coordinate system, dividing it from an unordered point cloud into an ordered point cloud. The division criteria are as follows: segment, sector, and ring. Among them, the segment division is based on the vertical field of view (FOV) of the lidar. v Combining the pitch angle parameter δθ, the number of segments is M = FOV. v / δθ, sector division is based on the horizontal field of view (FOV) of the lidar. h Combined with azimuth parameters The number of sectors is Rings can be defined by manually limiting the maximum distance *r* in the point cloud and combining this with the distance parameter *δr*. The number of rings is then *R* = *r* / *δr*. Based on this principle, for example, the *i*th point *p* in the point cloud data to be registered... i(x i ,y i ,z i The division or transformation is carried out using the following relationships (1)-(3):

[0082]

[0083]

[0084]

[0085] In the above formula, δθ represents the pitch angle parameter; δr represents the azimuth parameter; δr represents the distance parameter. Represents point cloud point p i The segment coordinates after spherical coordinate point cloud transformation; Represents point cloud point p i Sector coordinates after spherical coordinate point cloud transformation; ring represents point cloud point p. i The coordinates of the ring area after spherical coordinate point cloud transformation. In this embodiment, spherical coordinate grid division is used, which is proposed to address the characteristic of dense point clouds at near distances and sparse point clouds at distant distances in mechanical lidar. The purpose is to ensure that the difference in the number of point clouds in the divided voxels is not large, thus avoiding the problem of large differences in point cloud feature distribution caused by traditional cubic pixel division with fixed resolution. This can improve the registration accuracy of subsequent registration based on the transformed point cloud points.

[0086] S303 searches the octree for grid points that match the points in the point cloud.

[0087] After constructing an octree based on the center point of each spherical grid, we can further search on the octree using each point cloud point in the converted point cloud data to be registered as the search target, and search for the grid point that matches each point cloud point, which is the center point of the aforementioned spherical grid.

[0088] S304, determine the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered based on the searched grid points and the matched point cloud points.

[0089] This embodiment relates to a specific method for determining the pose transformation matrix of a lidar, providing two methods. The first method involves maximizing the likelihood function of the searched grid points and the matched point cloud points to determine the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered. The maximum likelihood function can be determined by the following relationship (4):

[0090]

[0091] Where T(t,s) represents the transformation function, t represents the searched grid point, and s represents the point cloud point that matches the searched grid point. p(T(t,s)) represents the probability density function.

[0092] The second method is to use the Normal Distribution Transform (NDT) algorithm to register the searched grid points and the matched point cloud points, and determine the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

[0093] In the above embodiments, spherical coordinate grid partitioning is used to adapt to the "dense near, sparse far" distribution of point cloud data acquired by LiDAR, thereby improving the inaccurate local feature description caused by the fixed size of cubic grid partitioning in traditional NDT matching algorithms. Therefore, the above embodiments belong to an improved NDT matching algorithm. In addition, by preset parameters, the number of spherical grid partitions can be ensured to be a constant value. The minimum granularity can be determined by constructing an octree with the grid center point, thereby stabilizing the tree construction cost and improving the efficiency of finding the corresponding grid for the current scan point.

[0094] Optionally, one implementation of the above S102 is provided, namely, the above S102 "determines the location measurement point of the Global Positioning System (GPS) based on the collected latitude and longitude data" specifically includes: converting the collected latitude and longitude data into distance values ​​in the Mercator plane projection coordinate system to obtain the GPS location measurement point.

[0095] Once the data processing platform obtains GPS latitude and longitude data, it can obtain the GPS planar distance value in the Mercator (Universal Transverse Mercator, UTM) coordinate system, i.e., the GPS location measurement point, according to the corresponding latitude and longitude projection method.

[0096] In one embodiment, a method for constructing position constraints based on the driving trajectory points of the LiDAR and the position measurement points of the GPS is provided, namely, "constructing position constraints based on the driving trajectory points of the LiDAR and the position measurement points of the GPS" in S103 above. Figure 5 As shown, it includes:

[0097] S401, construct a KD tree using GPS position measurement points, and perform a K-nearest neighbor search at each lidar driving trajectory point to determine a preset number of GPS position measurement points closest to each lidar driving trajectory point.

[0098] The preset number can be 3, 4, 5, etc., with 5 being the optimal preset number. In this embodiment, when the data processing platform obtains the GPS location measurement points and the LiDAR driving trajectory points, it can construct a KD tree using the GPS location measurement points and iterate through the driving trajectory points of each LiDAR. Then, it uses the K-nearest neighbor search method to find the preset number of GPS location measurement points closest to each LiDAR driving trajectory point. For example, it can find the 5 GPS location measurement points closest to the current LiDAR driving trajectory point.

[0099] S402, Calculate the center point of a preset number of GPS location measurement points.

[0100] Once a predetermined number of GPS position measurement points are found that are closest to the driving trajectory points of each LiDAR, the center points of these predetermined number of GPS position measurement points can be further determined. Specifically, this can be done by constructing a geometric figure of the predetermined number of GPS position measurement points and then determining the center point of this geometric figure as the center point of the predetermined number of GPS position measurement points. Alternatively, existing center point calculation methods can be used to calculate the center points of the predetermined number of GPS position measurement points.

[0101] S403 constructs position constraints using the center point and the corresponding LiDAR driving trajectory points.

[0102] When the data processing platform obtains the driving trajectory point of each LiDAR and the center point of the GPS position measurement point corresponding to the driving trajectory point of each LiDAR, it can further establish positional constraints between the driving trajectory point of each LiDAR and the center point of the GPS position measurement point corresponding to the driving trajectory point of each LiDAR. That is, the distance between the driving trajectory point of each LiDAR and the center point of the GPS position measurement point corresponding to the driving trajectory point of each LiDAR is calculated as a constraint condition, so that the solution can be performed based on the constraint condition later.

[0103] Furthermore, when the data processing platform specifically executes the steps of S403 above, it can specifically perform the following: with position constraints as the objective, use the Ceres algorithm to minimize the distance residual between the center point and the corresponding LiDAR driving trajectory point, and output the corresponding extrinsic parameter matrix.

[0104] The Ceres algorithm is a nonlinear iterative solution method that solves the problem by minimizing the distance residuals in the x, y, and z directions. Following the steps of the previous embodiment, after determining the positional constraints, these constraints can be used as the target. The Ceres algorithm is then used to minimize the distance residuals between the center point and the corresponding LiDAR trajectory points, calculating the corresponding extrinsic parameter matrix, which is the extrinsic parameter matrix from LiDAR to GPS.

[0105] Based on all the above embodiments, an external parameter calibration method for lidar and GPS calibration is provided, such as... Figure 6 As shown, the method includes:

[0106] S501 acquires the collected point cloud data and the collected latitude and longitude data.

[0107] S502, the first frame of point cloud data in the collected point cloud data is determined as the reference point cloud data, and the next frame of point cloud data in the reference point cloud data is determined as the point cloud data to be registered.

[0108] S503, registers the reference point cloud data and the point cloud data to be registered, and obtains the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

[0109] S504, re-determine the current point cloud to be registered as the new reference point cloud, and determine the next frame of point cloud data of the new reference point cloud as the new point cloud to be registered, and return to execute step S503 until the point cloud data of all frames in the acquired point cloud data are registered, and obtain multiple sets of LiDAR pose transformation matrices.

[0110] S505 determines the driving trajectory points of the lidar based on the pose transformation matrix of multiple lidars.

[0111] S506 converts the collected latitude and longitude data into distance values ​​in the Mercator plane projection coordinate system to obtain the GPS location measurement points.

[0112] S507 constructs a KD tree using GPS location measurement points and performs a K-nearest neighbor search at each lidar's driving trajectory point to determine a preset number of GPS location measurement points closest to each lidar's driving trajectory point.

[0113] S508, calculate the center point of a preset number of GPS position measurement points, and construct position constraints using the center point and the corresponding LiDAR driving trajectory points.

[0114] S509, with position constraints as the objective, uses the Ceres algorithm to minimize the distance residual between the center point and the corresponding LiDAR driving trajectory point, and outputs the corresponding extrinsic parameter matrix.

[0115] Among them, such as Figure 7 As shown, the above S503 includes the following steps:

[0116] S5031 divides the reference point cloud data into spherical coordinate grids and constructs an octree using the center point of each spherical grid.

[0117] S5032 performs spherical coordinate point cloud transformation on the point cloud data to be registered, and obtains the transformed point cloud points.

[0118] S5033 searches for grid points that match the points in the point cloud on an octree.

[0119] S5034, the likelihood function is maximized for the searched grid points and matched point cloud points to determine the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

[0120] Each of the above steps has been described in the foregoing embodiments. For details, please refer to the foregoing content. They will not be repeated here.

[0121] Based on the methods described in any of the above embodiments, this application also provides an external parameter calibration system, such as... Figure 8 As shown, the extrinsic parameter calibration system includes a laser SLAM module, a GPS data processing module, and a trajectory processing module. The laser SLAM module determines the driving trajectory points of the lidar based on the collected point cloud data. The GPS data processing module determines the GPS position measurement points based on the collected latitude and longitude data. The trajectory processing module constructs position constraints based on the lidar's driving trajectory points and the GPS position measurement points, and solves for these constraints to obtain the extrinsic parameter matrix from the lidar to the GPS. This extrinsic parameter calibration system can solve the problem of rapid extrinsic parameter calibration of lidar for autonomous vehicles in general scenarios without specific landmarks, using GPS with only position measurement values.

[0122] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0123] Based on the same inventive concept, this application also provides an external parameter calibration device for implementing the external parameter calibration method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more external parameter calibration device embodiments provided below can be found in the limitations of the external parameter calibration method described above, and will not be repeated here.

[0124] In one embodiment, such as Figure 9 As shown, an external parameter calibration device is provided, comprising:

[0125] The first determining module 11 is used to determine the driving trajectory points of the lidar based on the collected point cloud data.

[0126] The second determining module 12 is used to determine the location measurement point of the Global Positioning System (GPS) based on the collected latitude and longitude data.

[0127] The solution module 13 is used to construct position constraints based on the driving trajectory points of the lidar and the position measurement points of the GPS, and to solve the position constraints as the target to obtain the extrinsic parameter matrix of the lidar to the GPS.

[0128] In one embodiment, the first determining module 11 described above, such as Figure 10 As shown, it includes:

[0129] The first determining unit 111 is used to determine the first frame of point cloud data in the collected point cloud data as the reference point cloud data, and to determine the next frame of point cloud data of the reference point cloud data as the point cloud data to be registered.

[0130] The first registration unit 112 is used to register the reference point cloud data and the point cloud data to be registered to obtain the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

[0131] The second registration unit 113 is used to re-determine the current point cloud data to be registered as the new reference point cloud data, and to determine the next frame of point cloud data of the new reference point cloud data as the new point cloud data to be registered, and return to execute the step of registering the reference point cloud data and the point cloud data to be registered, until the point cloud data of all frames in the acquired point cloud data are registered, and obtain multiple sets of lidar pose transformation matrices;

[0132] The second determining unit 114 is used to determine the driving trajectory points of the lidar based on the pose transformation matrices of the multiple lidars.

[0133] In one embodiment, the first registration unit 112 is specifically used to register the reference point cloud data and the point cloud data to be registered using the Normal Distribution Transform (NDT) algorithm, so as to obtain the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

[0134] In one embodiment, the first registration unit 112 is specifically used to divide the reference point cloud data into spherical coordinate grids and construct an octree with the center point of each spherical grid; perform spherical coordinate point cloud transformation on the point cloud data to be registered to obtain the transformed point cloud points; search for grid points that match each point cloud point on the octree; and determine the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered based on the searched grid points and the matched point cloud points.

[0135] In one embodiment, the second determining module 12 is specifically used to convert the collected latitude and longitude data into distance values ​​in the Mercator plane projection coordinate system to obtain the location measurement point of the GPS.

[0136] In one embodiment, the solving module 13 described above, such as Figure 11 As shown, it includes:

[0137] The third determining unit 131 is used to construct a KD tree with the location measurement points of the GPS, and perform a K-nearest neighbor search at each driving trajectory point of the lidar to determine a preset number of GPS location measurement points that are closest to each driving trajectory point of the lidar.

[0138] Calculation unit 132 is used to calculate the center point of the preset number of GPS location measurement points;

[0139] The construction unit 133 is used to construct the position constraint with the center point and the corresponding driving trajectory point of the lidar.

[0140] In one embodiment, the solution module 13 is specifically used to minimize the distance residual between the center point and the corresponding driving trajectory point of the lidar using the Ceres algorithm with the position constraint as the objective, and output the corresponding extrinsic parameter matrix.

[0141] Each module in the aforementioned external parameter calibration device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0142] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 12As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an external parameter calibration method. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0143] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0144] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0145] The driving trajectory points of the lidar are determined based on the collected point cloud data;

[0146] The location measurement points of the Global Positioning System (GPS) are determined based on the collected latitude and longitude data;

[0147] Position constraints are constructed based on the driving trajectory points of the lidar and the position measurement points of the GPS, and the external parameter matrix of the lidar to the GPS is obtained by solving the position constraints as the objective.

[0148] The computer device provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0149] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0150] The driving trajectory points of the lidar are determined based on the collected point cloud data;

[0151] The location measurement points of the Global Positioning System (GPS) are determined based on the collected latitude and longitude data;

[0152] Position constraints are constructed based on the driving trajectory points of the lidar and the position measurement points of the GPS, and the external parameter matrix of the lidar to the GPS is obtained by solving the position constraints as the objective.

[0153] The computer-readable storage medium provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0154] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0155] The driving trajectory points of the lidar are determined based on the collected point cloud data;

[0156] The location measurement points of the Global Positioning System (GPS) are determined based on the collected latitude and longitude data;

[0157] Position constraints are constructed based on the driving trajectory points of the lidar and the position measurement points of the GPS, and the external parameter matrix of the lidar to the GPS is obtained by solving the position constraints as the objective.

[0158] The computer program product provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0159] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0160] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0161] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for calibrating external parameters, characterized in that, The method includes: The driving trajectory points of the lidar are determined based on the collected point cloud data; The collected latitude and longitude data are projected onto a preset plane coordinate system to obtain the position measurement points of the Global Positioning System (GPS) during the vehicle's movement. Position constraints are constructed based on the driving trajectory points of the lidar and the position measurement points of the GPS, and the external parameter matrix of the lidar to the GPS is obtained by solving the position constraints as the objective. The process of determining the driving trajectory points of the lidar based on the collected point cloud data includes: The first frame of point cloud data in the collected point cloud data is determined as the reference point cloud data, and the next frame of point cloud data in the reference point cloud data is determined as the point cloud data to be registered. The process of registering the reference point cloud data and the point cloud data to be registered to obtain the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered includes: dividing the reference point cloud data into spherical coordinate grids and constructing an octree with the center point of each spherical grid; performing coordinate transformation on the point cloud data to be registered by segmenting, sectoring, and ringing to divide the disordered point cloud into an ordered point cloud, obtaining the transformed point cloud points; searching for grid points that match each point cloud point on the octree; and determining the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered based on the searched grid points and the matched point cloud points; wherein, the segmenting includes division based on the vertical field of view and elevation angle parameters of the lidar, the sectoring includes division based on the horizontal field of view and azimuth angle parameters of the lidar, and the ringing includes division based on the maximum range and distance parameters of the point cloud. The current point cloud data to be registered is redefined as the new reference point cloud data, and the next frame of point cloud data of the new reference point cloud data is defined as the new point cloud data to be registered. Then, the step of registering the reference point cloud data and the point cloud data to be registered is returned to be executed until the point cloud data of all frames in the acquired point cloud data is registered, and multiple sets of LiDAR pose transformation matrices are obtained. The driving trajectory points of the lidar are determined based on the pose transformation matrices of multiple lidar units.

2. The method according to claim 1, characterized in that, The registration of the reference point cloud data and the point cloud data to be registered to obtain the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered includes: The Normal Distribution Transform (NDT) algorithm is used to register the reference point cloud data and the point cloud data to be registered, thereby obtaining the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

3. The method according to claim 1, characterized in that, The step of determining the driving trajectory points of the lidar based on the searched grid points and the matched point cloud points includes: The likelihood function is maximized for the searched grid points and matched point cloud points to determine the pose transformation matrix of the lidar between the reference point cloud data and the point cloud data to be registered.

4. The method according to claim 1, characterized in that, The step of determining the driving trajectory points of the lidar based on the searched grid points and the matched point cloud points includes: The Normal Distribution Transform (NDT) algorithm is used to register the searched grid points with the matched point cloud points to determine the driving trajectory points of the lidar.

5. The method according to any one of claims 1-4, characterized in that, The process of determining the location measurement points of the Global Positioning System (GPS) based on the collected latitude and longitude data includes: The collected latitude and longitude data are converted into distance values ​​in the Mercator plane projection coordinate system to obtain the location measurement points of the GPS.

6. The method according to claim 1, characterized in that, The step of constructing position constraints based on the driving trajectory points of the lidar and the position measurement points of the GPS includes: A KD tree is constructed using the GPS location measurement points, and a K-nearest neighbor search is performed on the driving trajectory point of each lidar to determine a preset number of GPS location measurement points that are closest to the driving trajectory point of each lidar. Calculate the center point of the preset number of GPS location measurement points; The position constraint is constructed using the center point and the corresponding driving trajectory point of the lidar.

7. The method according to claim 6, characterized in that, The process of solving for the extrinsic parameters of the laser radar to the GPS, with the position constraint as the objective, includes: Using the positional constraint as the objective, the Ceres algorithm is used to minimize the distance residual between the center point and the corresponding driving trajectory point of the lidar, and the corresponding extrinsic parameter matrix is ​​output.

8. An external parameter calibration device for implementing the external parameter calibration method as described in claim 1, characterized in that, The device includes: The first determining module is used to determine the driving trajectory points of the lidar based on the collected point cloud data; The second determining module is used to determine the location measurement points of the Global Positioning System (GPS) based on the collected latitude and longitude data; The solution module is used to construct position constraints based on the driving trajectory points of the lidar and the position measurement points of the GPS, and to solve for the position constraints to obtain the extrinsic parameter matrix of the lidar to the GPS.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images