External parameter calibration method and device for vehicle-mounted multi-sensor fusion

Abstract

The present invention provides a method and device for calibrating external parameters of vehicle-mounted multi-sensor fusion, which collects raw data obtained by multiple sensors; constructs an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, and a lever arm priori residual model based on the raw data, solves the model through a graph optimization method, and obtains an initial calibration result of the external parameters; uses data in a sliding window to construct and solve the residual model to obtain a coarse optimization calibration result; constructs and solves the model based on the raw data and the coarse optimization calibration result to obtain a final calibration result, adopts a coarse-to-fine iterative calibration method, and completes the calibration without the need for convergence of the combined navigation system. In addition, multiple residual models are simultaneously optimized in the same optimization problem, and multiple sensor external parameters are calibrated at one time, which can ensure the global consistency of the multi-sensor external parameters and meet the requirements of high-precision positioning and attitude determination of the multi-source fusion navigation system.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a method and device for calibrating external parameters of vehicle-mounted multi-sensor fusion. Background Art

[0002] LiDAR (Light Detection and Ranging), Inertial Measurement Unit (IMU), and Global Navigation Satellite System (GNSS) are widely used in multi-source fusion navigation due to their powerful complementary characteristics. High-precision extrinsic parameters are crucial for the positioning and attitude accuracy of multi-source fusion navigation. Multi-sensor fusion methods using LiDAR, IMU, and GNSS can accurately locate vehicles in complex scenarios and perform high-precision environmental modeling. However, this requires that the extrinsic parameters between the sensors are known. The extrinsic parameters of LiDAR and IMU are the rotational transformation matrix and translational transformation matrix between the LiDAR and IMU, while the extrinsic parameters of GNSS are the antenna arm value. The antenna arm value of GNSS is usually obtained through manual measurement or digital modeling, but human measurement errors and installation errors can lead to inaccurate antenna arm values. The transformation matrix between the LiDAR and IMU cannot generally be measured directly, especially the rotational component. Even with a total station, it is difficult to select a reference point representing the sensor coordinate system. Therefore, calibration methods such as track alignment are often used. However, the restricted motion of ground-based vehicles leads to insufficient excitation of the IMU, posing an even greater challenge for accurate parameter estimation. Furthermore, these calibration methods require the convergence of the integrated navigation system to obtain calibration results. Furthermore, methods that separately calibrate the external parameters of the LiDAR, IMU, and GNSS systems can produce inconsistent calibration results, making them unable to meet the high-precision positioning and attitude determination requirements of multi-source fusion navigation systems. Summary of the Invention

[0003] The present invention provides a method and device for calibrating the external parameters of a vehicle-mounted multi-sensor fusion system, which is used to solve the defects of the traditional multi-source fusion sensor external parameter calibration method, such as inaccurate parameters estimated, the need for the combined navigation system to converge to obtain calibration results, and the inconsistency of estimates caused by separate calibration, which cannot meet the high-precision positioning and attitude determination requirements of the multi-source fusion navigation system.

[0004] The present invention provides a method for calibrating external parameters of vehicle-mounted multi-sensor fusion, comprising:

[0005] Collecting raw data acquired by a plurality of sensors, the plurality of sensors including a lidar, an inertial measurement unit, and a global navigation satellite system;

[0006] constructing an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, and a lever arm prior residual model based on the raw data, solving the inertial measurement unit measurement data residual model and the global navigation satellite system measurement data residual model by a graph optimization method, and obtaining initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system;

[0007] Setting a sliding window, using the raw data within the sliding window and the initial calibration results of the extrinsic parameters of the inertial measurement unit and the global navigation satellite system to construct and solve an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, a lever arm priori residual model, a lidar and inertial measurement unit extrinsic parameter priori residual model, and a lidar inter-frame point cloud alignment residual model, to obtain a coarse optimization calibration result of the extrinsic parameters of the lidar, inertial measurement unit, and global navigation satellite system;

[0008] Based on the raw data acquired by the multiple sensors and the rough optimization calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system, the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm priori residual model, the lidar and inertial measurement unit external parameter priori residual model and the lidar global feature alignment residual model are constructed and solved to obtain the final calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system. The lidar global feature alignment residual model and the lidar inter-frame point cloud alignment residual model are constructed based on data from different ranges.

[0009] According to a method for calibrating external parameters of vehicle-mounted multi-sensor fusion provided by the present invention, the method constructs an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, and a lever arm priori residual model based on the raw data, including:

[0010] Calculating motion information of the inertial measurement unit at each sampling moment based on raw data acquired by the inertial measurement unit and the global navigation satellite system;

[0011] interpolating the motion information of each sampling moment of the inertial measurement unit to obtain key frame motion information;

[0012] A pre-integration residual function, a carrier differential positioning residual function and an antenna arm priori residual function are constructed using key frame motion information and the original data obtained by the inertial measurement unit and the global navigation satellite system, wherein the pre-integration residual function is a motion information deviation residual function between two adjacent key frames; the carrier differential positioning residual function is a residual function between the antenna position and velocity of the global navigation satellite system predicted by the inertial navigation system based on the inertial measurement unit and the antenna position and velocity solved by the carrier differential positioning of the global navigation satellite system; and the antenna arm priori residual function is a distance constraint function from the antenna to the inertial measurement unit.

[0013] According to a method for calibrating external parameters of vehicle-mounted multi-sensor fusion provided by the present invention, the raw data acquired by the inertial measurement unit and the global navigation satellite system include:

[0014] The target angular velocity and acceleration obtained by the inertial measurement unit and the global navigation satellite system carrier differential positioning speed measurement data.

[0015] According to a method for calibrating external parameters of vehicle-mounted multi-sensor fusion provided by the present invention, the motion information of the inertial measurement unit at each sampling moment includes:

[0016] At least one of target position, velocity, attitude, acceleration bias, and gyroscope constant drift information.

[0017] According to a method for calibrating extrinsic parameters of vehicle-mounted multi-sensor fusion provided by the present invention, the method solves the residual model of the inertial measurement unit measurement data and the residual model of the global navigation satellite system measurement data by a graph optimization method to obtain initial calibration results of the extrinsic parameters of the inertial measurement unit and the global navigation satellite system, including:

[0018] The key frame motion information corresponding to the minimum output residuals of the pre-integration residual function, the carrier differential positioning residual function and the antenna arm priori residual function and the antenna arm value of the global navigation satellite system are used as the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system.

[0019] According to a method for calibrating extrinsic parameters of vehicle-mounted multi-sensor fusion provided by the present invention, a sliding window is set, and the raw data within the sliding window and the initial calibration results of the extrinsic parameters of the inertial measurement unit and the global navigation satellite system are used to construct and solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm priori residual model, the lidar and inertial measurement unit extrinsic parameter priori residual model, and the lidar inter-frame point cloud alignment residual model, to obtain the coarse optimization calibration results of the extrinsic parameters of the lidar, the inertial measurement unit, and the global navigation satellite system, including:

[0020] Select the starting keyframe and ending keyframe of the sliding window, and use the raw data in the sliding window and the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system to build the residual model of the inertial measurement unit measurement data, the residual model of the global navigation satellite system measurement data, the prior residual model of the arm, the prior residual model of the external parameters of the lidar and the inertial measurement unit, and the residual model of the lidar inter-frame point cloud alignment;

[0021] Using the LM algorithm to solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the lever arm prior residual model, the lidar and inertial measurement unit extrinsic parameter prior residual model, and the lidar inter-frame point cloud alignment residual model, and record the solution results;

[0022] Repeat the above steps until the standard deviation of the conversion matrix between the lidar and the inertial measurement unit output for a preset number of consecutive times is less than a set threshold, obtaining the first optimization calibration result of the external parameters of the lidar, inertial measurement unit and global navigation satellite system;

[0023] The first optimization calibration result of the external parameters is optimized twice to obtain a rough optimization calibration result of the external parameters.

[0024] According to a method for calibrating external parameters of vehicle-mounted multi-sensor fusion provided by the present invention, the method for constructing a residual model for point cloud alignment between laser radar frames includes:

[0025] Project the point cloud data between adjacent key frames to the radar coordinate system corresponding to the previous key frame;

[0026] Project all point clouds in the radar coordinate system to the world coordinate system based on the key frame motion information, and obtain matching point pairs between any two frames of point clouds through nearest neighbor search;

[0027] The distance residual summation function of all matching point pairs is used as the residual model of point cloud alignment between lidar frames.

[0028] According to the present invention, a method for calibrating external parameters of vehicle-mounted multi-sensor fusion is provided, which further includes:

[0029] Reselect the starting keyframe and ending keyframe of the sliding window, and use the data corresponding to the first optimized calibration result in the window to construct the inertial measurement unit measurement data residual model, the lidar global feature alignment residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit extrinsic prior residual model, and the lidar global feature alignment residual model;

[0030] Use the LM algorithm to solve the inertial measurement unit measurement data residual model, the lidar global feature alignment residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit extrinsic prior residual model, and the lidar global feature alignment residual model, and record the solution results;

[0031] Repeat the above steps until the standard deviation of the conversion matrix between the lidar and the inertial measurement unit output for a consecutive preset number of times is less than the set threshold, and obtain the second optimization calibration result of the external parameters of the lidar, inertial measurement unit and global navigation satellite system. The second optimization calibration result is used as the coarse optimization calibration result of the external parameters of each sensor.

[0032] According to a method for calibrating external parameters of vehicle-mounted multi-sensor fusion provided by the present invention, the method for constructing a lidar global feature alignment residual model includes:

[0033] Project all points into the world coordinate system based on the key frame motion information and construct an octree;

[0034] Each node of the octree is marked as line or surface features through principal component analysis;

[0035] Use the point clouds within all nodes marked as line or surface features to build a matching error model and obtain the lidar global feature alignment residual model;

[0036] Among them, for a node marked as a line feature, the point cloud matching error is the second largest eigenvalue of the covariance matrix constructed using all points in the node; for a node marked as a surface feature, the point cloud matching error is the minimum eigenvalue of the covariance matrix constructed using all points in the node, and the sum of the errors of all marked nodes is the point cloud matching error.

[0037] According to a vehicle-mounted multi-sensor fusion extrinsic parameter calibration method provided by the present invention, based on the raw data obtained by the multiple sensors and the rough optimization calibration results of the extrinsic parameters of the lidar, inertial measurement unit and global navigation satellite system, an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, a lever arm priori residual model, a lidar and inertial measurement unit extrinsic parameter priori residual model and a lidar global feature alignment residual model are constructed and solved to obtain the final calibration results of the extrinsic parameters of the lidar, inertial measurement unit and global navigation satellite system, including:

[0038] Based on all the point cloud data acquired by the lidar, the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit extrinsic parameter prior residual model and the lidar global feature alignment residual model are constructed to perform secondary optimization on the rough optimization calibration results of the external parameters of each sensor to obtain the final calibration results of the external parameters of each sensor.

[0039] The present invention also provides an external parameter calibration device for vehicle-mounted multi-sensor fusion, comprising:

[0040] a collection module, configured to collect raw data acquired by a plurality of sensors, the plurality of sensors including a lidar, an inertial measurement unit, and a global navigation satellite system;

[0041] an initial calibration module, configured to construct an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, and a lever arm prior residual model based on the raw data, and solve the inertial measurement unit measurement data residual model and the global navigation satellite system measurement data residual model using a graph optimization method to obtain initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system;

[0042] a coarse optimization calibration module, configured to set a sliding window, use the raw data within the sliding window and the initial calibration results of the extrinsic parameters of the inertial measurement unit and the global navigation satellite system to construct and solve an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, a lever arm priori residual model, a lidar and inertial measurement unit extrinsic parameter priori residual model, and a lidar inter-frame point cloud alignment residual model, and obtain a coarse optimization calibration result of the extrinsic parameters of the lidar, inertial measurement unit, and global navigation satellite system;

[0043] A global optimization calibration module is used to construct and solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm priori residual model, the lidar and inertial measurement unit external parameter priori residual model and the lidar global feature alignment residual model based on the raw data obtained by the multiple sensors and the rough optimization calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system, so as to obtain the final calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system. The lidar global feature alignment residual model and the lidar inter-frame point cloud alignment residual model are constructed based on data from different ranges.

[0044] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the external parameter calibration method for vehicle-mounted multi-sensor fusion as described in any one of the above is implemented.

[0045] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the external parameter calibration method for vehicle-mounted multi-sensor fusion as described in any one of the above is implemented.

[0046] The present invention provides a method and device for calibrating external parameters of vehicle-mounted multi-sensor fusion, which collects raw data obtained by multiple sensors, including a lidar, an inertial measurement unit, and a global navigation satellite system; constructs an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, and a lever arm priori residual model based on the raw data, solves the inertial measurement unit measurement data residual model and the global navigation satellite system measurement data residual model through a graph optimization method, and obtains initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system; sets a sliding window, uses the raw data within the sliding window and the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system, and uses the raw data within the sliding window and the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system to construct and solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the lever arm priori residual model, the lidar and ... The coarse optimization calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system are obtained by the prior residual model of the external parameters of the lidar, inertial measurement unit and the residual model of the lidar inter-frame point cloud alignment; based on the original data obtained by multiple sensors and the coarse optimization calibration results of the external parameters of the lidar, inertial measurement unit and the global navigation satellite system, the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit extrinsic parameter prior residual model and the lidar global feature alignment residual model are constructed and solved to obtain the final calibration results of the external parameters of the lidar, inertial measurement unit and the global navigation satellite system. The coarse-to-fine iterative calibration method is adopted, and the calibration can be completed without the convergence of the combined navigation system. Moreover, the external parameters of multiple sensors are calibrated simultaneously through local and global optimization, which can ensure the consistency of global features, solve the problem of inconsistent estimation during separate calibration, and meet the needs of high-precision positioning and attitude determination of the multi-source fusion navigation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1This is one of the flow charts of the external parameter calibration method for vehicle-mounted multi-sensor fusion provided by the present invention;

[0049] Figure 2 This is the second flow chart of the external parameter calibration method for vehicle-mounted multi-sensor fusion provided by the present invention;

[0050] Figure 3 This is the third flow chart of the external parameter calibration method for vehicle-mounted multi-sensor fusion provided by the present invention;

[0051] Figure 4 It is a structural diagram of the external parameter calibration device for vehicle-mounted multi-sensor fusion provided by the present invention;

[0052] Figure 5 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0053] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0054] Figure 1 The flowchart of the external parameter calibration method of vehicle-mounted multi-sensor fusion provided by the embodiment of the present invention is as follows Figure 1 As shown, the external parameter calibration method for vehicle-mounted multi-sensor fusion provided by the embodiment of the present invention includes:

[0055] Step 101: collecting raw data acquired by multiple sensors, including a lidar, an inertial measurement unit, and a global navigation satellite system;

[0056] Step 102: constructing an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, and a lever arm prior residual model based on the original data, solving the inertial measurement unit measurement data residual model and the global navigation satellite system measurement data residual model using a graph optimization method, and obtaining initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system;

[0057] In an embodiment of the present invention, a combined navigation solution is performed based on the original data, and then an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model and a lever arm priori residual model are constructed.

[0058] Step 103: Set a sliding window, and use the raw data within the sliding window and the initial calibration results of the extrinsic parameters of the inertial measurement unit and the global navigation satellite system to construct and solve the residual model of the inertial measurement unit measurement data, the residual model of the global navigation satellite system measurement data, the prior residual model of the lever arm, the prior residual model of the extrinsic parameters of the lidar and inertial measurement unit, and the residual model of the lidar inter-frame point cloud alignment, to obtain the coarse optimization calibration results of the extrinsic parameters of the lidar, inertial measurement unit, and global navigation satellite system;

[0059] Step 104: Based on the raw data obtained by multiple sensors and the rough optimization calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system, a residual model of the inertial measurement unit measurement data, a residual model of the global navigation satellite system measurement data, a priori residual model of the arm, a priori residual model of the external parameters of the lidar and inertial measurement unit, and a residual model of the lidar global feature alignment are constructed and solved to obtain the final calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system.

[0060] The lidar global feature alignment residual model and the lidar inter-frame point cloud alignment residual model are constructed based on different range data and different principles.

[0061] The residual model of lidar inter-frame point cloud alignment is the distance between the matching points of the two frame point clouds. The residual model of lidar global feature alignment is the thickness of the surface and the thickness of the line composed of points belonging to the same plane or the same line feature in all frame point clouds.

[0062] Since the lidar global feature alignment residual model is built based on global data, and the lidar inter-frame point cloud alignment residual model is built based on local data, it can achieve iterative calibration from coarse to fine. Moreover, in the optimization problem, multiple residuals are optimized simultaneously, and the calibration of multi-sensor extrinsic parameters is completed at one time, which can ensure the global consistency of the extrinsic parameters.

[0063] In an embodiment of the present invention, the final calibration results of the external parameters of the laser radar, the inertial measurement unit and the global navigation satellite system include the conversion matrix between the laser radar and the inertial measurement unit, and the antenna arm value of the global navigation satellite system.

[0064] In traditional methods for calibrating the extrinsic parameters of lidar, inertial measurement units, and global navigation satellite systems (GNSS), antenna arm values for GNSS systems are typically obtained through manual measurement or digital modeling. However, human measurement errors and installation errors can lead to inaccurate antenna arm values. The transformation matrix between the lidar and inertial measurement unit (IMU) is generally impossible to measure directly, especially the rotational component. Even with a total station, it is difficult to select a reference point representing the sensor coordinate system. Therefore, calibration is often performed using methods such as track alignment. However, the restricted motion of ground-based vehicles results in insufficient excitation of the IMU, posing a greater challenge to accurately estimating parameters. Furthermore, these calibration methods require convergence of the integrated navigation system to obtain calibration results. Furthermore, methods that separately calibrate the extrinsic parameters of the lidar, IMU, and GNSS systems can produce inconsistent estimates, failing to meet the requirements of high-precision positioning and attitude determination for multi-source fusion navigation systems.

[0065] The external parameter calibration method for vehicle-mounted multi-sensor fusion provided by the embodiment of the present invention collects raw data obtained by multiple sensors, and the multiple sensors include lidar, inertial measurement unit and global navigation satellite system; constructs the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model and the lever arm priori residual model based on the raw data, and solves the inertial measurement unit measurement data residual model and the global navigation satellite system measurement data residual model through a graph optimization method to obtain the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system; sets a sliding window, uses the raw data in the sliding window and the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system to construct and solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the lever arm priori residual model, the lidar and inertial measurement unit external parameter priori residual model and the lidar inter-frame point cloud alignment residual model to obtain the lidar, inertial measurement unit external parameter priori residual model and the lidar inter-frame point cloud alignment residual model The coarse optimization calibration results of the external parameters of the measurement unit and the global navigation satellite system are obtained; based on the original data obtained by the multiple sensors and the coarse optimization calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system, the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm priori residual model, the lidar and inertial measurement unit external parameter priori residual model and the lidar global feature alignment residual model are constructed and solved to obtain the final calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system, firstly through the local optimization of the lidar inter-frame point cloud alignment residual model and then through the global optimization of the lidar global feature alignment residual model, to achieve iterative calibration from coarse to fine, and calibration can be performed without the convergence of the combined navigation system, and the consistency of the global features can be guaranteed by local and global optimization, so as to solve the problem of inconsistent estimation during separate calibration and meet the requirements of high-precision positioning and attitude determination of the multi-source fusion navigation system.

[0066] Based on any of the above embodiments, Figure 2 As shown in FIG, based on the original data, an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model and a lever arm prior residual model are constructed, including:

[0067] Step 201: Calculate the motion information of the inertial measurement unit at each sampling moment based on the raw data obtained by the inertial measurement unit and the global navigation satellite system;

[0068] In the embodiment of the present invention, the raw data acquired by the inertial measurement unit and the global navigation satellite system include but are not limited to: target angular velocity and acceleration acquired by the inertial measurement unit and global navigation satellite system carrier differential positioning velocity measurement data.

[0069] The global navigation satellite system carrier differential positioning speed measurement data includes speed information in the latitude, longitude, altitude and northeast celestial coordinate systems.

[0070] Step 202: interpolate the motion information of each sampling moment of the inertial measurement unit to obtain key frame motion information;

[0071] In an embodiment of the present invention, a carrier differential positioning / inertial navigation system based on an inertial measurement unit is used to calculate the motion information of the inertial measurement unit at each sampling moment. The motion information of the inertial measurement unit at each sampling moment includes but is not limited to: target position, speed, attitude, acceleration bias and gyroscope constant drift information.

[0072] Step 203: Use the key frame motion information and the original data obtained by the inertial measurement unit and the global navigation satellite system to construct a pre-integration residual function, a carrier differential positioning residual function and an antenna arm priori residual function, wherein the pre-integration residual function is a motion information deviation residual function between two adjacent key frames; the carrier differential positioning residual function is a residual function between the antenna position and velocity of the global navigation satellite system predicted by the inertial navigation system based on the inertial measurement unit and the antenna position and velocity solved by the carrier differential positioning of the global navigation satellite system; and the antenna arm priori residual function is a distance constraint function from the antenna to the inertial measurement unit.

[0073] For example, the motion information of the inertial measurement unit at the whole second is obtained as the optimization quantity. The raw data obtained by the inertial measurement unit and the global navigation satellite system are used to construct the pre-integration residual and carrier differential positioning residual functions respectively. Based on graph optimization, the posture at the whole second is further optimized by minimizing the inertial measurement unit pre-integration and carrier differential positioning residuals. Specifically, the following are included:

[0074] The motion information of the GNSS carrier differential positioning sampling moment is defined as a keyframe, with one keyframe per second. The initial keyframe value is obtained by interpolating the motion information estimated by the integrated navigation. A graph optimization problem is constructed using the keyframe information and the original data. By solving the graph optimization problem, more accurate keyframe motion information and the GNSS antenna arm are obtained. The graph optimization problem includes the pre-integrated residual function, the carrier differential positioning residual, and the antenna arm prior residual.

[0075] Based on any of the above embodiments, solving the inertial measurement unit measurement data residual model and the global navigation satellite system measurement data residual model by a graph optimization method to obtain initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system, including:

[0076] By minimizing the output residuals of the pre-integration residual function, the carrier differential positioning residual function and the antenna arm prior residual function, the key frame motion information corresponding to the inertial measurement unit and the initial calibration data of the antenna arm of the global navigation satellite system are obtained.

[0077] Based on any of the above embodiments, Figure 3 As shown, step 103 specifically includes:

[0078] In the embodiment of the present invention, the data collected by the laser within a time period are organized in a unified timestamp coordinate system, so that motion distortion can be removed.

[0079] Step 301: Select the starting keyframe and the ending keyframe of the sliding window, and use the raw data in the sliding window and the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system to construct the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit external parameter prior residual model, and the lidar inter-frame point cloud alignment residual model;

[0080] Step 302: Use the LM algorithm to solve the residual model of the inertial measurement unit measurement data, the residual model of the global navigation satellite system measurement data, the prior residual model of the lever arm, the prior residual model of the external parameters of the lidar and the inertial measurement unit, and the residual model of the lidar inter-frame point cloud alignment, and record the solution results;

[0081] Step 303: Repeat steps 301 and 302 until the standard deviation of the conversion matrix between the lidar and the inertial measurement unit output for a preset number of consecutive times is less than a set threshold, thereby obtaining the first optimization calibration result of the external parameters of the lidar, inertial measurement unit and global navigation satellite system.

[0082] In an embodiment of the present invention, a method for constructing a residual model for point cloud alignment between laser radar frames includes:

[0083] Project the point cloud data between adjacent key frames to the radar coordinate system corresponding to the previous key frame;

[0084] Project all point clouds in the radar coordinate system to the world coordinate system based on the key frame motion information, and obtain matching point pairs between any two frames of point clouds through nearest neighbor search;

[0085] The distance residual summation function of all matching point pairs is used as the residual model of point cloud alignment between lidar frames.

[0086] The accuracy of the calibration results can be improved by performing a first-level optimization on the locally optimized lidar inter-frame point cloud alignment residual model.

[0087] Step 304: reselect the starting keyframe and the ending keyframe of the sliding window, and use the corresponding data of all the first optimization calibration results in the window to construct the inertial measurement unit measurement data residual model, the lidar global feature alignment residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit extrinsic prior residual model, and the lidar global feature alignment residual model;

[0088] Step 305: Use the LM (Levenberg-Marquardt) algorithm to solve the inertial measurement unit measurement data residual model, the lidar global feature alignment residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit extrinsic prior residual model, and the lidar global feature alignment residual model, and record the solution results;

[0089] Step 306: Repeat steps 304 and 305 until the standard deviation of the conversion matrix between the lidar and the inertial measurement unit output for a preset number of consecutive times is less than the set threshold, and obtain the second optimization calibration result of the external parameters of the lidar, inertial measurement unit and global navigation satellite system, and use the second optimization calibration result as the coarse optimization calibration result of the external parameters of each sensor.

[0090] In an embodiment of the present invention, a method for constructing a lidar global feature alignment residual model includes:

[0091] Project all points into the world coordinate system based on the key frame motion information and construct an octree;

[0092] Each node of the octree is marked as line or surface features through principal component analysis;

[0093] Use the point clouds within all nodes marked as line or surface features to build a matching error model and obtain the lidar global feature alignment residual model;

[0094] Among them, for a node marked as a line feature, the point cloud matching error is the second largest eigenvalue of the covariance matrix constructed using all points in the node; for a node marked as a surface feature, the point cloud matching error is the minimum eigenvalue of the covariance matrix constructed using all points in the node, and the sum of the errors of all marked nodes is the point cloud matching error.

[0095] In this embodiment of the present invention, a sliding window coarse optimization is used, where a sliding window is defined as the time interval from the start keyframe to the end keyframe. Using all the information within the sliding window, the GNSS antenna arm and keyframe motion information, as well as the point cloud data collected by the lidar, the IMU measurement results, and the GNSS-RTK positioning and velocity measurement results, a graph optimization problem is constructed and solved. The sliding window calculation is performed until the set termination condition is met, specifically including:

[0096] Based on the keyframe motion information and LiDAR-IMU extrinsic parameters, the starting and ending keyframes of the optimization sliding window are selected. A graph optimization problem is constructed using the raw data within the sliding window and the initial calibration results of the extrinsic parameters of the inertial measurement unit and global navigation satellite system. The graph optimization problem includes the residual model described in the previous steps and also includes a residual model for the extrinsic parameters between the LiDAR and IMU measurements. The prior residuals of the extrinsic parameters between the LiDAR and IMU measurements are the prior residuals of the relative angle and relative distance between the two sensors obtained from the digital model.

[0097] During the second local optimization, only the construction method of the point cloud matching error is changed.

[0098] Based on any of the above embodiments, based on the raw data obtained by multiple sensors and the rough optimization calibration results of the external parameters of the lidar, inertial measurement unit, and global navigation satellite system, a residual model of the inertial measurement unit measurement data, a residual model of the global navigation satellite system measurement data, a priori residual model of the lever arm, a priori residual model of the external parameters of the lidar and inertial measurement unit, and a residual model of the lidar global feature alignment are constructed and solved to obtain the final calibration results of the external parameters of the lidar, inertial measurement unit, and global navigation satellite system, including:

[0099] Based on all the point cloud data acquired by the lidar, the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit extrinsic parameter prior residual model and the lidar global feature alignment residual model are constructed to perform secondary optimization on the rough optimization calibration results of the external parameters of each sensor to obtain the final calibration results of the external parameters of each sensor.

[0100] In the embodiment of the present invention, the secondary optimization is to use the data of all key frames to construct a residual model in the same manner as step 304 in the above embodiment and use the LM method to solve the model until convergence, which will not be repeated here.

[0101] The external parameter calibration method for vehicle-mounted multi-sensor fusion provided by an embodiment of the present invention adopts an iterative calibration method from coarse to fine, and calibration can be performed without the convergence of the combined navigation system; and a two-stage joint optimization calibration method is adopted, in which different point cloud matching residuals are used in the two-stage optimization: the first stage uses the point-to-point matching residuals across multiple frames of point cloud data to calculate the coarse optimization value of the external parameters of each sensor; the second stage uses the point-to-globally consistent line and surface feature matching residuals across multiple frames of point cloud data to ensure the consistency of global features, thereby further improving the accuracy of the calibration results.

[0102] Because the Global Navigation Satellite System (GNSS) is an active positioning system, carrier differential positioning may exhibit gross errors when satellite signals are interfered with or obscured by the environment. Furthermore, the motion of ground vehicles provides limited excitation to the inertial measurement unit (IMU). This embodiment of the present invention, through the tight coupling of multiple sensors, simultaneously eliminates gross errors in carrier differential positioning, overcomes issues such as insufficient IMU motion excitation and point cloud data distortion, and ultimately achieves more consistent multi-sensor external parameters.

[0103] The following describes the external parameter calibration device for vehicle-mounted multi-sensor fusion provided by the present invention. The external parameter calibration device for vehicle-mounted multi-sensor fusion described below and the external parameter calibration method for vehicle-mounted multi-sensor fusion described above can correspond to each other.

[0104] Figure 4 A schematic diagram of an external parameter calibration device for vehicle-mounted multi-sensor fusion according to an embodiment of the present invention is shown in FIG. Figure 4 As shown, the external parameter calibration device for vehicle-mounted multi-sensor fusion provided by an embodiment of the present invention includes:

[0105] A collection module 401 is configured to collect raw data acquired by a plurality of sensors, including a lidar, an inertial measurement unit, and a global navigation satellite system;

[0106] An initial calibration module 402 is configured to construct an IMU measurement data residual model, a GNSS measurement data residual model, and a lever arm prior residual model based on the raw data, solve the IMU measurement data residual model and the GNSS measurement data residual model using a graph optimization method, and obtain initial calibration results of the extrinsic parameters of the IMU and GNSS;

[0107] A coarse optimization calibration module 403 is configured to set a sliding window, use the raw data within the sliding window and the initial calibration results of the extrinsic parameters of the inertial measurement unit and the global navigation satellite system to construct and solve the residual model of the inertial measurement unit measurement data, the residual model of the global navigation satellite system measurement data, the prior residual model of the lever arm, the prior residual model of the extrinsic parameters of the lidar and inertial measurement unit, and the residual model of the lidar inter-frame point cloud alignment, and obtain the coarse optimization calibration results of the extrinsic parameters of the lidar, inertial measurement unit, and global navigation satellite system;

[0108] The global optimization calibration module 404 is used to construct and solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm priori residual model, the lidar and inertial measurement unit external parameter priori residual model and the lidar global feature alignment residual model based on the raw data obtained by the multiple sensors and the rough optimization calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system, so as to obtain the final calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system.

[0109] The external parameter calibration device for vehicle-mounted multi-sensor fusion provided by an embodiment of the present invention collects raw data obtained by multiple sensors, and the multiple sensors include lidar, inertial measurement unit and global navigation satellite system; constructs an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model and a lever arm priori residual model based on the raw data, and solves the inertial measurement unit measurement data residual model and the global navigation satellite system measurement data residual model through a graph optimization method to obtain the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system; sets a sliding window, uses the raw data in the sliding window and the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system to construct and solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the lever arm priori residual model, the lidar and inertial measurement unit external parameter priori residual model and the lidar inter-frame point cloud alignment residual model to obtain the lidar , coarse optimization calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system; based on the original data obtained by the multiple sensors and the coarse optimization calibration results of the external parameters of the lidar, inertial measurement unit and the global navigation satellite system, the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm priori residual model, the lidar and inertial measurement unit external parameter priori residual model and the lidar global feature alignment residual model are constructed and solved to obtain the final calibration results of the external parameters of the lidar, inertial measurement unit and the global navigation satellite system, and an iterative calibration method from coarse to fine is adopted, and calibration can be performed without the convergence of the combined navigation system. Moreover, by simultaneously calibrating the global navigation satellite system antenna arm and the external parameters between the lidar and the inertial measurement unit through local and global optimization, the consistency of global features can be ensured, the problem of inconsistent estimation during separate calibration is solved, and the requirements of high-precision positioning and attitude determination of the multi-source fusion navigation system are met.

[0110] Figure 5 An example of a physical structure diagram of an electronic device is shown below. Figure 5As shown, the electronic device may include: a processor (processor) 510, a communication interface (Communications Interface) 520, a memory (memory) 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logic instructions in the memory 530 to execute the external parameter calibration method of the vehicle-mounted multi-sensor fusion, the method comprising: collecting raw data obtained by multiple sensors, the multiple sensors including lidar, inertial measurement unit and global navigation satellite system; constructing an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model and a lever arm prior residual model based on the raw data, solving the inertial measurement unit measurement data residual model and the global navigation satellite system measurement data residual model by a graph optimization method, and obtaining the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system; setting a sliding window, using the raw data in the sliding window and the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system to construct and solve the inertial measurement unit measurement data residual model. The method comprises the following steps: constructing a measurement data residual model, a global navigation satellite system measurement data residual model, a lever arm prior residual model, a lidar and inertial measurement unit extrinsic parameter prior residual model and a lidar inter-frame point cloud alignment residual model to obtain a coarse optimization calibration result of the external parameters of the lidar, inertial measurement unit and global navigation satellite system; based on the original data acquired by the multiple sensors and the coarse optimization calibration result of the external parameters of the lidar, inertial measurement unit and global navigation satellite system, constructing and solving the inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, a lever arm prior residual model, a lidar and inertial measurement unit extrinsic parameter prior residual model and a lidar global feature alignment residual model to obtain a final calibration result of the external parameters of the lidar, inertial measurement unit and global navigation satellite system.

[0111] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0112] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the external parameter calibration method for vehicle-mounted multi-sensor fusion provided by the above-mentioned methods, the method comprising: collecting raw data acquired by multiple sensors, the multiple sensors including lidar, inertial measurement unit and global navigation satellite system; constructing an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model and a lever arm priori residual model based on the raw data, solving the inertial measurement unit measurement data residual model and the global navigation satellite system measurement data residual model by a graph optimization method, and obtaining the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system; setting a sliding window, using the raw data in the sliding window and the external parameters of the inertial measurement unit and the global navigation satellite system The initial calibration results are used to construct and solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit external parameter prior residual model and the lidar inter-frame point cloud alignment residual model to obtain the rough optimization calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system; based on the original data obtained by the multiple sensors and the rough optimization calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system, the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit external parameter prior residual model and the lidar global feature alignment residual model are constructed and solved to obtain the final calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system.

[0113] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0114] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for calibrating external parameters of vehicle-mounted multi-sensor fusion, characterized in that: include: Collecting raw data acquired by a plurality of sensors, the plurality of sensors including a lidar, an inertial measurement unit, and a global navigation satellite system; constructing an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, and a lever arm priori residual model based on the raw data, solving the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, and the lever arm priori residual model using a graph optimization method to obtain initial calibration results of the extrinsic parameters of the inertial measurement unit and the global navigation satellite system; Setting a sliding window, using the raw data within the sliding window and the initial calibration results of the extrinsic parameters of the inertial measurement unit and the global navigation satellite system to construct and solve an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, a lever arm priori residual model, a lidar and inertial measurement unit extrinsic parameter priori residual model, and a lidar inter-frame point cloud alignment residual model, to obtain a coarse optimization calibration result of the extrinsic parameters of the lidar, inertial measurement unit, and global navigation satellite system; Based on the raw data obtained by the multiple sensors and the rough optimization calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system, the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm priori residual model, the lidar and inertial measurement unit external parameter priori residual model and the lidar global feature alignment residual model are constructed and solved to obtain the final calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system.

2. The external parameter calibration method for vehicle-mounted multi-sensor fusion according to claim 1 is characterized in that: The constructing of an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, and a lever arm priori residual model based on the original data includes: Calculating motion information of the inertial measurement unit at each sampling moment based on raw data acquired by the inertial measurement unit and the global navigation satellite system; interpolating the motion information of each sampling moment of the inertial measurement unit to obtain key frame motion information; The key frame motion information and the original data obtained based on the inertial measurement unit and the global navigation satellite system are used to construct a pre-integration residual function, a carrier differential positioning residual function and an antenna arm priori residual function, wherein the pre-integration residual function is a motion information deviation residual function between two adjacent key frames; the carrier differential positioning residual function is a residual function between the antenna position and velocity of the global navigation satellite system predicted by the inertial navigation system based on the inertial measurement unit and the antenna position and velocity solved by the carrier differential positioning of the global navigation satellite system; the antenna arm priori residual function is a distance constraint function from the antenna to the inertial measurement unit.

3. The external parameter calibration method for vehicle-mounted multi-sensor fusion according to claim 2 is characterized in that: The raw data acquired by the inertial measurement unit and the global navigation satellite system include: The inertial measurement unit obtains the target angular velocity and acceleration, as well as the global navigation satellite system carrier differential positioning speed data.

4. The external parameter calibration method for vehicle-mounted multi-sensor fusion according to claim 2, characterized in that: The motion information of the inertial measurement unit at each sampling moment includes: At least one of target position, velocity, attitude, acceleration bias, and gyroscope constant drift information.

5. The external parameter calibration method for vehicle-mounted multi-sensor fusion according to claim 2 is characterized in that: The graph optimization method is used to solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model and the lever arm prior residual model to obtain the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system, including: The key frame motion information corresponding to the minimum output residuals of the pre-integration residual function, the carrier differential positioning residual function and the antenna arm priori residual function and the antenna arm value of the global navigation satellite system are used as the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system.

6. The external parameter calibration method for vehicle-mounted multi-sensor fusion according to claim 5, characterized in that: The method uses the raw data in the sliding window and the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system to construct and solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm priori residual model, the lidar and inertial measurement unit external parameter priori residual model, and the lidar inter-frame point cloud alignment residual model to obtain the coarse optimization calibration results of the external parameters of the lidar, inertial measurement unit, and global navigation satellite system, including: Select the starting keyframe and ending keyframe of the sliding window, and use the raw data in the sliding window and the initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system to build the residual model of the inertial measurement unit measurement data, the residual model of the global navigation satellite system measurement data, the prior residual model of the arm, the prior residual model of the external parameters of the lidar and the inertial measurement unit, and the residual model of the lidar inter-frame point cloud alignment; Using the LM algorithm to solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the lever arm prior residual model, the lidar and inertial measurement unit extrinsic parameter prior residual model, and the lidar inter-frame point cloud alignment residual model, and record the solution results; Repeat the above steps until the standard deviation of the conversion matrix between the lidar and the inertial measurement unit output for a preset number of consecutive times is less than a set threshold, obtaining the first optimization calibration result of the external parameters of the lidar, inertial measurement unit and global navigation satellite system; The first optimization calibration result of the external parameters is optimized twice to obtain a rough optimization calibration result of the external parameters.

7. The external parameter calibration method for vehicle-mounted multi-sensor fusion according to claim 6, characterized in that: The method for constructing a residual model of point cloud alignment between laser radar frames includes: Project the point cloud data between adjacent key frames to the radar coordinate system corresponding to the previous key frame; Project all point clouds in the radar coordinate system to the world coordinate system based on the key frame motion information, and obtain matching point pairs between any two frames of point clouds through nearest neighbor search; The distance residual summation function of all matching point pairs is used as the residual model of point cloud alignment between lidar frames.

8. The external parameter calibration method for vehicle-mounted multi-sensor fusion according to claim 6, characterized in that: The second optimization of the first optimization calibration result of the external parameter to obtain a rough optimization calibration result of the external parameter includes: Reselect the starting keyframe and ending keyframe of the sliding window, and use the data corresponding to the first optimized calibration result in the window to construct the inertial measurement unit measurement data residual model, the lidar global feature alignment residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit extrinsic prior residual model, and the lidar global feature alignment residual model; Use the LM algorithm to solve the inertial measurement unit measurement data residual model, the lidar global feature alignment residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit extrinsic prior residual model, and the lidar global feature alignment residual model, and record the solution results; Repeat the above steps until the standard deviation of the conversion matrix between the lidar and the inertial measurement unit output for a consecutive preset number of times is less than the set threshold, and obtain the second optimization calibration result of the external parameters of the lidar, inertial measurement unit and global navigation satellite system. The second optimization calibration result is used as the coarse optimization calibration result of the external parameters of each sensor.

9. The external parameter calibration method for vehicle-mounted multi-sensor fusion according to claim 8, characterized in that: The method for constructing a lidar global feature alignment residual model includes: Projecting all points into a world coordinate system according to key frame motion information, and constructing an octree based on the data in the world coordinate system; Marking each node of the octree as a line feature or a surface feature through principal component analysis; Use the point clouds within all nodes marked as line or surface features to build a matching error model and obtain the lidar global feature alignment residual model; Among them, for a node marked as a line feature, the point cloud matching error is the second largest eigenvalue of the covariance matrix constructed using all points in the node; for a node marked as a surface feature, the point cloud matching error is the minimum eigenvalue of the covariance matrix constructed using all points in the node, and the sum of the errors of all marked nodes is the point cloud matching error.

10. The external parameter calibration method for vehicle-mounted multi-sensor fusion according to claim 7, characterized in that: The method comprises constructing and solving an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, a lever arm priori residual model, a lidar and inertial measurement unit extrinsic parameter priori residual model, and a lidar global feature alignment residual model based on the raw data acquired by the multiple sensors and the coarse optimization calibration results of the external parameters of the lidar, inertial measurement unit, and global navigation satellite system, to obtain a final calibration result of the external parameters of the lidar, inertial measurement unit, and global navigation satellite system, including: Based on all the point cloud data acquired by the lidar, the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm prior residual model, the lidar and inertial measurement unit extrinsic parameter prior residual model and the lidar global feature alignment residual model are constructed to perform secondary optimization on the rough optimization calibration results of the external parameters of each sensor to obtain the final calibration results of the external parameters of each sensor.

11. An external parameter calibration device for vehicle-mounted multi-sensor fusion, characterized in that: include: a collection module, configured to collect raw data acquired by a plurality of sensors, the plurality of sensors including a lidar, an inertial measurement unit, and a global navigation satellite system; an initial calibration module, configured to construct an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, and a lever arm prior residual model based on the raw data, and solve the inertial measurement unit measurement data residual model and the global navigation satellite system measurement data residual model using a graph optimization method to obtain initial calibration results of the external parameters of the inertial measurement unit and the global navigation satellite system; a coarse optimization calibration module, configured to set a sliding window, and use the raw data within the sliding window and the initial calibration results of the extrinsic parameters of the inertial measurement unit and the global navigation satellite system to construct and solve an inertial measurement unit measurement data residual model, a global navigation satellite system measurement data residual model, a lever arm priori residual model, a lidar and inertial measurement unit extrinsic parameter priori residual model, and a lidar inter-frame point cloud alignment residual model, to obtain a coarse optimization calibration result of the extrinsic parameters of the lidar, inertial measurement unit, and global navigation satellite system; A global optimization calibration module is used to construct and solve the inertial measurement unit measurement data residual model, the global navigation satellite system measurement data residual model, the arm priori residual model, the lidar and inertial measurement unit extrinsic parameter priori residual model and the lidar global feature alignment residual model based on the raw data obtained by the multiple sensors and the rough optimization calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system, so as to obtain the final calibration results of the external parameters of the lidar, inertial measurement unit and global navigation satellite system.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the external parameter calibration method for vehicle-mounted multi-sensor fusion is implemented as described in any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the external parameter calibration method for vehicle-mounted multi-sensor fusion is implemented as described in any one of claims 1 to 10.

Images