Rapid joint calibration method for laser radar and GPS-RTK

Abstract

The invention relates to the technical field of multi-sensor joint calibration, in particular to a laser radar and GPS-RTK (global positioning system-real time kinematic) rapid joint calibration method, which comprises the following steps of: automatically acquiring an initialized structure external parameter value as a calibration initial value by using a structure external parameter self-checking device; then, geometric features in the point cloud are extracted through a depth feature extraction network, front and back frame matching is carried out, and the mileage is calculated; dynamically adjusting data selection logic according to the conditions of the GPS-RTK and the mileage data, and distributing different weights for different points; and finally, accurately solving the external parameters of the structure and the external parameters of the system by constructing a minimum error optimization problem with a weight. According to the method, the dependence on manual calibration points is reduced, the complexity is reduced, the calibration efficiency and precision are remarkably improved, and the method is suitable for scenes with high-precision positioning requirements such as automatic driving and unmanned aerial vehicle navigation.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-sensor joint calibration, and in particular to a rapid joint calibration method of a laser radar and a GPS-RTK. Background Art

[0002] With the development of autonomous driving technology, accurate vehicle positioning and environmental perception have become crucial. Laser radar (LiDAR) and Global Positioning System Real-Time Kinematic Measurement Technology (GPS-RTK) are two sensor technologies widely used in vehicle positioning and environmental modeling. LiDAR can provide high-precision three-dimensional environmental data, while GPS-RTK can provide centimeter-level positioning accuracy. However, the existing LiDAR and GPS-RTK calibration methods have some limitations. First, traditional calibration methods are time-consuming and usually require specific calibration sites and markers, which not only increases the calibration cost but also limits the flexibility of the calibration process. Secondly, existing calibration methods often require complex mathematical models and a lot of manual adjustments, which not only increases the technical threshold but also affects the degree of automation of the calibration process.

[0003] In the existing disclosed invention technology, for example, a patent with publication number CN109597054A discloses a laser radar calibration method, which includes: using the laser radar to obtain distance measurement values ​​and corresponding reflectivity measurement values ​​of a calibration object under multiple calibration distance values; splitting the distance measurement values ​​and the corresponding reflectivity measurement values ​​into at least two groups according to the size of the distance measurement values; further splitting the distance measurement values ​​and the corresponding reflectivity measurement values ​​in each group into at least two sub-groups according to the size of the reflectivity measurement values ​​in each group; and fitting an error correction function with the reflectivity measurement value as an input variable according to the error value between the distance measurement value and the corresponding calibration distance value and the corresponding reflectivity measurement value in each sub-group.

[0004] The above patent processing flow is relatively complicated in data calculation, especially in large-scale data processing. When multiple splitting and fitting are performed, it will lead to high computing overhead and the problems described in the background technology exist. Summary of the invention

[0005] The purpose of the embodiment of the present application is to provide a rapid joint calibration method of a laser radar and a GPS-RTK, which is used to solve the above technical problems. The technical solution of the rapid joint calibration method of a laser radar and a GPS-RTK includes the following steps:

[0006] S1: Automatically identify the positions of the LiDAR sensor and GPS-RTK antenna through the structure external parameter self-checking device, and perform initialization calculation of the structure external parameters;

[0007] S2: Obtain information about fixed features in the operating environment through LiDAR, and record and analyze multimodal data during the motion platform's travel;

[0008] S3: extract the real-time driving environment conditions of the motion platform, and dynamically adjust the processing window and threshold value of the lidar data and GPS-RTK data according to the real-time driving environment conditions to obtain valid data in the real-time driving environment conditions;

[0009] S4: Select the LiDAR mileage point-northeast celestial coordinate point pair in the valid data according to the distance selection algorithm, calculate the preliminary system extrinsic parameters through the plane calibration algorithm, construct the optimization problem of minimizing the error, perform nonlinear optimization, and iteratively solve the system extrinsic parameters and structural extrinsic parameters;

[0010] S5: According to the system external parameters and structural external parameters, complete the joint calibration of the laser radar and GPS-RTK, obtain the structural external parameters of the laser radar to the GPS-RTK installation position and the system external parameters of the northeast celestial coordinate system and the geocentric earth-fixed coordinate system initialized by GPS-RTK, and complete the conversion between absolute geographic coordinates and laser mileage.

[0011] Specifically, S1 includes the following specific steps:

[0012] S11: Install the laser radar and GPS-RTK on the motion platform, and operate the motion platform to the bottom of the structure external parameter self-checking device, wherein the structure external parameter self-checking device is composed of a depth camera suspended above the motion platform and an AI computing unit connected thereto;

[0013] Specifically, the suspension height of the depth camera is determined according to the height of the motion platform. Exemplarily, in this embodiment, when the position image taken by the depth camera is covered by both the laser radar and the GPS-RTK antenna, it is the minimum suspension height of the depth camera.

[0014] S12: Acquire a position image of a laser radar sensor on the motion platform through the depth camera;

[0015] S13: Automatically extract the position image of the lidar sensor through the AI ​​algorithm and perform initialization calculation of the structural external parameters.

[0016] Specifically, in S11, the AI ​​computing unit includes: the convolutional neural network used is a trained target detection network or a semantic segmentation network, which is used to detect or segment two types of targets: lidar and GPS-RTK antenna.

[0017] Specifically, S13 also includes the following specific steps:

[0018] S131: Identify the laser radar and GPS-RTK antennas in the position image of the laser radar sensor through a convolutional neural network, and calculate the three-dimensional coordinates of the laser radar and GPS-RTK through pixel coordinates, actual depth of pixels and camera internal parameters;

[0019] S132: importing the three-dimensional coordinates of GPS-RTK into the relative posture calculation strategy, and calculating and obtaining the relative posture between the laser radar and the GPS-RTK antenna;

[0020] S133: extracting the relative position and posture between the laser radar and the GPS-RTK antenna to form the initial value of the structural external parameter.

[0021] Furthermore, in S131, the calculation strategy of the three-dimensional coordinates is specifically:

[0022]

[0023] Where z is the converted metric depth, obtained from the depth map of the depth camera;

[0024] (q, p) are the pixel coordinates of LiDAR and GPS-RTK;

[0025] (x, y, z) is the physical coordinate in the depth camera coordinate system, that is, the physical coordinate of the laser radar and GPS-RTK;

[0026] f x ,f y are the focal lengths of the depth camera in the horizontal and vertical directions respectively;

[0027] C x ,C y The main point of the camera.

[0028] Specifically, S2 includes the following steps:

[0029] S21: extracting the real-time driving environment conditions of the motion platform. When the motion platform is close to the feature object, if the GPS-RTK signal is interfered, then execute step S3, otherwise execute step S22;

[0030] S22: The mobile motion platform obtains fixed feature object information in the operating environment through a laser radar, wherein the fixed feature object information includes: edge, corner and surface data of the feature object;

[0031] Furthermore, the fixed feature information also includes: the facade or outer corner of a building, a publicity sign, a telephone pole, a tree, and a stationary vehicle;

[0032] S23: simultaneously recording multimodal data of the motion platform during its travel, wherein the multimodal data includes: laser radar point cloud, GPS-RTK latitude and longitude information, and motion trajectory data of the IMU;

[0033] S24: The geometric features of the LiDAR point cloud are extracted in real time through a deep feature extraction network, and the motion trajectory data of the IMU is combined to assist in calculating the inter-frame LiDAR mileage data.

[0034] Furthermore, S24 includes the following specific steps:

[0035] S241: Introduce a feature extraction network to automatically extract local and global geometric features of the point cloud from the geometric features of the LiDAR point cloud;

[0036] Wherein, the feature extraction network is one of a deep convolutional neural network and a graph neural network;

[0037] S242: Global feature matching of point clouds between frames using Transformer network;

[0038] S243: Use multi-scale feature fusion mechanism to extract point cloud features from different scales;

[0039] S244: Use GPU to accelerate the inference process of deep learning network and complete real-time matching of point clouds between frames;

[0040] S245: After completing the inter-frame matching, extract the six degrees of freedom transformation between the two frames of point clouds as the mileage data of the laser radar for subsequent external parameter calculation and optimization, wherein the six degrees of freedom include: three translation quantities and three rotation quantities.

[0041] Specifically, S3 includes the following specific steps:

[0042] S31: extracting real-time driving environment conditions of the motion platform, wherein the real-time driving environment conditions include noise characteristics, laser characteristics and GPS signal strength characteristics;

[0043] The noise feature is the density of the laser radar point cloud;

[0044] The laser feature is the average distance between points in the feature area;

[0045] The GPS signal strength characteristics are GPS status bit and number of satellites;

[0046] S32: Preset the threshold range of the laser radar point cloud density, the threshold range of the average distance between points in the feature area, the threshold of the GPS status bit and the threshold of the number of satellites, and simultaneously preset the initial threshold range, and screen the real-time driving environment conditions according to the above threshold ranges, wherein the initial threshold range is 25-30;

[0047] S33: when the noise feature exceeds the threshold range of the lidar point cloud density, increase the time window of the lidar data;

[0048] When the laser feature exceeds the threshold range of the average distance between points in the feature area, the data is judged to be unusable and is removed;

[0049] When the GPS signal strength characteristic is less than the threshold of the GPS status bit and the threshold of the number of satellites at the same time, the time window of the GPS data is increased and the threshold value when the laser radar matches the GPS-RTK data is increased; the threshold value when the laser radar matches the GPS-RTK data includes: the distance threshold and the error threshold in the ICP algorithm;

[0050] S34: According to S32-S33, the real-time driving environment conditions after the screening process are marked as valid data, and the time window size is adaptively controlled for the process of obtaining the valid data.

[0051] Furthermore, S3 also includes the following specific steps:

[0052] S35: When the GPS signal strength feature is less than the threshold of the GPS status bit and the threshold of the number of satellites at the same time or the noise feature exceeds the threshold range of the laser radar point cloud density, the weight distribution of the sensor data is adjusted and recorded in real time, and weight constraints are provided for subsequent optimization calculations. The weight distribution strategy specifically includes:

[0053] w=w gps × lidar-noise × feature ;

[0054] Among them, w is the total weight value; w gps is the GPS-RTK weight factor;

[0055] num star Real-time GPS-RTK satellite search number;

[0056] w lidar-noise is the laser radar noise weight factor; w feature is the LiDAR feature significance weight factor.

[0057] Specifically, S4 includes the following specific steps:

[0058] S41: completing the conversion of the longitude and latitude data and the northeast celestial coordinate system through a coordinate conversion algorithm, obtaining a pair of laser radar mileage points and northeast celestial coordinate points, and selecting a pair of laser radar mileage points and northeast celestial coordinate points in the valid data according to a distance selection algorithm; wherein the distance selection algorithm specifically selects points by the distance between the point pairs and the constraint condition that the three points are not collinear with each other; the number of the laser radar mileage points and northeast celestial coordinate points in the valid data is greater than or equal to 4;

[0059] S42: Calculate preliminary system extrinsic parameters through a plane calibration algorithm.

[0060] Furthermore, in S41, the coordinate conversion algorithm specifically includes:

[0061] S411: Convert the longitude and latitude data to the Earth-centered Earth-fixed rectangular coordinate system, as follows:

[0062]

[0063] Among them, (X, Y, Z) are the coordinates converted to the Earth-centered Earth-fixed rectangular coordinate system;

[0064] (lon, lat, alt) are the longitude and latitude coordinates in the geodetic coordinate system; e is the eccentricity of the earth ellipsoid; N is the radius of curvature of the reference ellipsoid;

[0065] S412: extracting the geocentric earth-fixed rectangular coordinate system and converting the geocentric earth-fixed rectangular coordinate system to the northeast celestial coordinate system, specifically including: calculating the northeast celestial coordinates (e, n, u) of (X, Y, Z) at the coordinate origin with coordinates (x0, y0, z0);

[0066]

[0067]

[0068] Among them, the latitude and longitude of the coordinate origin with coordinates (x0, y0, z0) is (lon0, lat0, alt0);

[0069] (Δx, Δy, Δz) is the coordinate transformation difference matrix.

[0070] Specifically, in S42, the plane calibration algorithm includes:

[0071] S421: making assumptions before solving, wherein the assumptions before solving include: considering only heading information, assuming that the origins of the translation amounts coincide, and making a plane assumption for the rotation amounts;

[0072] S422: translating the mileage information and the northeast celestial coordinates respectively by removing the centroid coordinates;

[0073] S423: Using two segments of trajectory points, construct an iterative closest point problem, convert the optimization problem of minimizing the error to be solved into an optimization problem of point pair fitting, solve the rotation matrix R through SVD decomposition, and extract the heading angle from R.

[0074] Specifically, in S423, the prerequisite data for constructing the optimization problem of minimizing the error to be solved includes: based on the lidar trajectory and the GPS-RTK trajectory, combined with the point weights determined in S3; the optimization algorithms used in the optimization problem include: least squares method and Ceres optimization library.

[0075] A computer-readable storage medium stores a computer program, which, when executed by a processor, enables the computer to execute the method for rapid joint calibration of a laser radar and GPS-RTK.

[0076] An electronic device includes a memory, a processor, and instructions stored in the memory and executable on the processor. When the processor executes the instructions, the above-mentioned rapid joint calibration method of a laser radar and GPS-RTK is implemented.

[0077] Compared with the prior art, the present invention according to the above scheme has the following beneficial effects: a fast joint calibration method of laser radar and GPS-RTK, which uses laser radar data and Fast-LIO algorithm to perform mileage calculation, generates continuous relative displacement information, converts the laser radar mileage information into GPS-RTK mileage data through rough structural extrinsics, and performs conditional plane calibration with the northeast sky data converted by GPS-RTK, so as to obtain the initial values ​​of the system extrinsics. Then, the laser mileage data is matched with the northeast sky coordinates, and an optimization problem is constructed. The precise system extrinsics and structural extrinsics are iteratively approximated from the initial values ​​of the system and structural extrinsics, so as to achieve high-precision joint calibration of the two. Compared with traditional methods, the present invention reduces the dependence on manual calibration points, reduces complexity, and significantly improves the efficiency and accuracy of calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0079] in:

[0080] Figure 1 A schematic diagram of a rapid joint calibration method of a laser radar and GPS-RTK according to the present invention;

[0081] Figure 2 It is a right view of the laser radar installation in a rapid joint calibration method of a laser radar and GPS-RTK of the present invention;

[0082] Figure 3 It is a top view of the laser radar installation and a schematic diagram of the motion trajectory of the motion platform in a rapid joint calibration method of a laser radar and GPS-RTK of the present invention.

[0083] Reference numerals:

[0084] 1. Fixed top surface; 2. Depth camera; 3. GPS antenna; 4. LiDAR; 5. Motion platform; 6. Motion trajectory. DETAILED DESCRIPTION

[0085] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0086] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0087] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0088] Example 1

[0089] like Figure 1 As shown, a rapid joint calibration method of a laser radar and GPS-RTK according to an embodiment of the present invention is as follows: Figure 1 As shown, the specific steps are as follows:

[0090] S1: Automatically identify the positions of the LiDAR sensor and GPS-RTK antenna through the structure external parameter self-checking device, and perform initialization calculation of the structure external parameters;

[0091] S1 includes the following specific steps:

[0092] S11: Install the laser radar and GPS-RTK on the motion platform, and operate the motion platform to the bottom of the structure external parameter self-checking device, wherein the structure external parameter self-checking device is composed of a depth camera suspended above the motion platform and an AI computing unit connected thereto;

[0093] For example, in this embodiment, the selected laser radar model is MID360, which can perform 360° non-repetitive scanning and has an internal integrated IMU. It is installed on a passenger car, and the passenger car is placed in a scene with obvious fixed features. In this embodiment, the scene with obvious fixed features is a scene with buildings, trees, and stationary vehicles.

[0094] Specifically, the hanging height of the depth camera is determined according to the height of the motion platform. For example, in this embodiment, when the position image captured by the depth camera is covered by both the laser radar and the GPS-RTK antenna, it is the minimum hanging height of the depth camera. In this embodiment, it should be noted that the motion platform needs to be a certain distance away from buildings or trees that can obviously block the GPS signal to avoid interference with the GPS-RTK signal during the calibration process. In this embodiment, the distance is selected to be 5m.

[0095] In addition, in this embodiment, during the installation of the laser radar and the RTK antenna, it is necessary to ensure that the centers of the laser radar and the RTK antenna are on the central axis of the vehicle and that their heights are in the same plane;

[0096] S12: Acquire a position image of a laser radar sensor on the motion platform through the depth camera;

[0097] S13: Automatically extract the position image of the lidar sensor through the AI ​​algorithm and perform initialization calculation of the structural external parameters.

[0098] In S11, the AI ​​computing unit includes: the convolutional neural network used is a trained target detection network or a semantic segmentation network, which is used to detect or segment two types of targets: lidar and GPS-RTK antenna.

[0099] S13 also includes the following specific steps:

[0100] S131: Identify the laser radar and GPS-RTK antennas in the position image of the laser radar sensor through a convolutional neural network, and calculate the three-dimensional coordinates of the laser radar and GPS-RTK through pixel coordinates, actual depth of pixels and camera internal parameters;

[0101] S132: importing the three-dimensional coordinates of GPS-RTK into the relative posture calculation strategy, and calculating and obtaining the relative posture between the laser radar and the GPS-RTK antenna;

[0102] S133: extracting the relative position and posture between the laser radar and the GPS-RTK antenna to form the initial value of the structural external parameter.

[0103] In S131, the calculation strategy of the three-dimensional coordinates is specifically:

[0104]

[0105] Where z is the converted metric depth, obtained from the depth map of the depth camera;

[0106] (q, p) are the pixel coordinates of LiDAR and GPS-RTK;

[0107] (x, y, z) is the physical coordinate in the depth camera coordinate system, that is, the physical coordinate of the laser radar and GPS-RTK;

[0108] f x ,f y are the focal lengths of the depth camera in the horizontal and vertical directions respectively;

[0109] C x ,C y The main point of the camera.

[0110] S2: Obtain information about fixed features in the operating environment through LiDAR, and record and analyze multimodal data during the motion platform's travel;

[0111] S2 includes the following specific steps:

[0112] S21: extracting the real-time driving environment conditions of the motion platform. When the motion platform is close to the feature object, if the GPS-RTK signal is interfered, then execute step S3, otherwise execute step S22;

[0113] S22: The mobile motion platform obtains fixed feature object information in the operating environment through a laser radar, wherein the fixed feature object information includes: edge, corner and surface data of the feature object;

[0114] Furthermore, the fixed feature information also includes: the facade or outer corner of a building, a publicity sign, a telephone pole, a tree, and a stationary vehicle;

[0115] S23: simultaneously recording multimodal data of the motion platform during its travel, wherein the multimodal data includes: laser radar point cloud, GPS-RTK latitude and longitude information, and motion trajectory data of the IMU;

[0116] S24: The geometric features of the LiDAR point cloud are extracted in real time through a deep feature extraction network, and the motion trajectory data of the IMU is combined to assist in calculating the inter-frame LiDAR mileage data.

[0117] S24 includes the following specific steps:

[0118] S241: Introduce a feature extraction network to automatically extract local and global geometric features of the point cloud from the geometric features of the LiDAR point cloud;

[0119] Wherein, the feature extraction network is one of a deep convolutional neural network and a graph neural network;

[0120] Exemplarily, in this embodiment, the feature extraction network is FCGF or SuperPoint;

[0121] S242: Use the Transformer network to perform global feature matching of point clouds between frames. The Transformer network is a deep learning architecture widely used in the field of natural language processing. The Transformer architecture captures the long-distance dependencies between point cloud frames to ensure high matching accuracy in scenes with sparse or repeated textures or uneven point cloud density.

[0122] S243: Use multi-scale feature fusion mechanism to extract point cloud features from different scales;

[0123] In this embodiment, a multi-scale feature fusion mechanism is used to extract point cloud features from different scales, ensuring that both low-resolution features at long distances and high-resolution features at close ranges can be processed, thereby optimizing detail matching between frames. The accuracy and consistency of feature extraction are maintained in a wide range of scenes;

[0124] S244: Use GPU to accelerate the inference process of deep learning network and complete real-time matching of point clouds between frames;

[0125] In this embodiment, efficient parallel computing is used to accelerate the extraction and matching process of point cloud features, and the computational efficiency of the overall calibration can be significantly improved, especially in large-scale point cloud data.

[0126] S245: After completing the inter-frame matching, extract the six degrees of freedom transformation between the two frames of point clouds as the mileage data of the laser radar for subsequent external parameter calculation and optimization, wherein the six degrees of freedom include: three translation amounts and three rotation amounts;

[0127] Exemplarily, in this embodiment, the six degrees of freedom transformation are obtained by: measuring the distance d between the laser radar and the center of the RTK antenna by a tape measure, and then determining the translation T = [d, 0, 0] and the rotational freedom R = [0, 0, 0] of the two;

[0128] S3: extract the real-time driving environment conditions of the motion platform, and dynamically adjust the processing window and threshold value of the lidar data and GPS-RTK data according to the real-time driving environment conditions to obtain valid data in the real-time driving environment conditions;

[0129] S3 includes the following specific steps:

[0130] S31: extracting real-time driving environment conditions of the motion platform, wherein the real-time driving environment conditions include noise characteristics, laser characteristics and GPS signal strength characteristics;

[0131] The noise feature is the density of the laser radar point cloud;

[0132] The laser feature is the average distance between points in the feature area;

[0133] The GPS signal strength characteristics are GPS status bit and number of satellites;

[0134] S32: Preset the threshold range of the laser radar point cloud density, the threshold range of the average distance between points in the feature area, the threshold of the GPS status bit and the threshold of the number of satellites, and simultaneously preset the initial threshold range, and screen the real-time driving environment conditions according to the above threshold ranges, wherein the initial threshold range is 25-30;

[0135] S33: when the noise feature exceeds the threshold range of the laser radar point cloud density, increasing the time window of the laser radar data; in this embodiment, increasing the time window of the laser radar data can increase the amount of accumulated data to smooth the noise interference and ensure that the extracted features are more stable;

[0136] When the laser feature exceeds the threshold range of the average distance between points in the feature area, the data is judged to be unusable and is removed;

[0137] When the GPS signal strength characteristic is less than the threshold of the GPS status bit and the threshold of the number of satellites at the same time, the time window of the GPS data is increased and the threshold value when the laser radar matches the GPS-RTK data is increased; the threshold value when the laser radar matches the GPS-RTK data includes: the distance threshold and the error threshold in the ICP algorithm. In this embodiment, the threshold value when the laser radar matches the GPS-RTK data is increased to tolerate large errors between the data and ensure that limited feature point pairs can still be matched in harsh environments. When the environmental conditions improve, the system will automatically reduce the threshold value to improve the matching accuracy and optimize the external parameter calculation results.

[0138] S34: According to S32-S33, the real-time driving environment conditions after the screening process are marked as valid data, and the time window size is adaptively controlled for the process of obtaining the valid data.

[0139] S3 also includes the following specific steps:

[0140] S35: When the GPS signal strength feature is less than the threshold of the GPS status bit and the threshold of the number of satellites at the same time or the noise feature exceeds the threshold range of the laser radar point cloud density, the weight distribution of the sensor data is adjusted and recorded in real time, and weight constraints are provided for subsequent optimization calculations. The weight distribution strategy specifically includes:

[0141] w=w gps × lidar-noise × feature ;

[0142] Among them, w is the total weight value; w gps is the GPS-RTK weight factor;

[0143] num star is the number of real-time GPS-RTK satellite searches. In this embodiment, the more real-time GPS-RTK satellite searches are performed, the more stable the GPS signal is.

[0144] w lidar-noise is the laser radar noise weight factor; w feature is the LiDAR feature significance weight factor.

[0145] In this embodiment, a method for acquiring laser radar data and GPS-RTK data is provided, including: using the Fast-LIO algorithm to perform feature matching on the laser radar point clouds of the previous and next two frames, and calculating the mileage data of the laser radar, if the time interval between the current laser radar mileage and the GPS-RTK data acquisition is less than a threshold value of 0.1s, the two data are combined into a data pair for processing;

[0146] In this embodiment, S3 also includes: judging whether the GPS-RTK data in the data pair is valid; if valid, continue; otherwise, it is necessary to continue to loop to obtain data and perform the processing of step S4; the GPS-RTK data validity judgment conditions include: 1) GPS-RTK initialization has been completed, that is, the longitude and latitude data is greater than 0; 2) the status bit in the GPS-RTK data is 4; 3) the number of satellites in the GPS-RTK data exceeds the set threshold value, and the threshold value is set to 25 to 30;

[0147] S4: Select the LiDAR mileage point-northeast celestial coordinate point pair in the valid data according to the distance selection algorithm, calculate the preliminary system extrinsic parameters through the plane calibration algorithm, construct the optimization problem of minimizing the error, perform nonlinear optimization, and iteratively solve the accurate system extrinsic parameters and structural extrinsic parameters;

[0148] S4 includes the following specific steps:

[0149] S41: completing the conversion of the longitude and latitude data and the northeast celestial coordinate system through a coordinate conversion algorithm, obtaining a pair of laser radar mileage points and northeast celestial coordinate points, and selecting a pair of laser radar mileage points and northeast celestial coordinate points in the valid data according to a distance selection algorithm; wherein the distance selection algorithm specifically selects points by the distance between the point pairs and the constraint condition that the three points are not collinear with each other; the number of the laser radar mileage points and northeast celestial coordinate points in the valid data is greater than or equal to 4;

[0150] S42: Calculate preliminary system extrinsic parameters through a plane calibration algorithm.

[0151] In S41, the coordinate conversion algorithm specifically includes:

[0152] S411: Convert the longitude and latitude data to the Earth-centered Earth-fixed rectangular coordinate system, as follows:

[0153]

[0154] Among them, (X, Y, Z) are the coordinates converted to the Earth-centered Earth-fixed rectangular coordinate system;

[0155] (lon, lat, alt) are the longitude and latitude coordinates in the geodetic coordinate system; e is the eccentricity of the earth ellipsoid; N is the radius of curvature of the reference ellipsoid;

[0156] S412: extracting the geocentric earth-fixed rectangular coordinate system and converting the geocentric earth-fixed rectangular coordinate system to the northeast celestial coordinate system, specifically including: calculating the northeast celestial coordinates (e, n, u) of (X, Y, Z) at the coordinate origin with coordinates (x0, y0, z0);

[0157]

[0158]

[0159] Among them, the latitude and longitude of the coordinate origin with coordinates (x0, y0, z0) is (lon0, lat0, alt0);

[0160] (Δx, Δy, Δz) is the coordinate transformation difference matrix.

[0161] In S42, the plane calibration algorithm includes:

[0162] S421: making assumptions before solving, wherein the assumptions before solving include: considering only heading information, assuming that the origins of the translation amounts coincide, and making a plane assumption for the rotation amounts;

[0163] S422: translating the mileage information and the northeast celestial coordinates respectively by removing the centroid coordinates; illustratively, in this embodiment, the mileage information and the northeast celestial coordinates are translated respectively, and since the data are all in metric system and no scaling is involved, only the rotation amount is left;

[0164] S423: Using two segments of trajectory points, construct an iterative closest point problem, convert the optimization problem of minimizing the error to be solved into an optimization problem of point pair fitting, solve the rotation matrix R through SVD decomposition, and extract the heading angle from R.

[0165] In S423, the construction pre-data of the optimization problem of minimizing the error to be solved includes: based on the laser radar trajectory and the GPS-RTK trajectory, combined with the point weight determined in S3; in this example, the point weight represents the quality of the collected data; the optimization algorithm used in the optimization problem includes: least squares method and Ceres optimization library;

[0166] The optimization algorithm in the Ceres optimization library is used to iteratively solve the precise system extrinsics and structural extrinsics. In this example, the optimization algorithm is used to minimize the error between the LiDAR trajectory and the GPS-RTK trajectory, and calculate the best rotation matrix and translation vector.

[0167] S5: Based on the precise system extrinsics and structural extrinsics, complete the joint calibration of the laser radar and GPS-RTK, obtain the structural extrinsics of the laser radar to the GPS-RTK installation position and the system extrinsics of the northeast celestial coordinate system and the geocentric earth-fixed coordinate system initialized by GPS-RTK, and complete the conversion between absolute geographic coordinates and laser mileage.

[0168] Example 2

[0169] like Figure 2 , Figure 3 As shown, in this embodiment, the depth camera 2 is installed on the fixed top surface 1 for acquiring image data, the laser radar 4 is installed in the middle of the front end of the motion platform 5, and the two GPS antennas 3 are symmetrically arranged on both sides of the rear of the laser radar 4 and are located on the same plane of the motion platform 5. During the installation process, it is necessary to ensure that the center of the two GPS antennas 3 and the laser radar 4 are connected to the center axis of the motion platform 5. During the calibration process, the motion platform 5 is controlled to move around the figure 8 to obtain the motion trajectory 6, and the selected scene needs to meet the requirements of having obvious fixed features and a certain distance from the obstruction to avoid signal interference affecting the iteration time and data validity during calibration.

[0170] Example 3

[0171] This embodiment provides an electronic device, including: a processor and a memory, wherein the memory stores instructions that can be called by the processor;

[0172] The processor executes the above-mentioned rapid joint calibration method of laser radar and GPS-RTK by calling the instructions stored in the memory.

[0173] The electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) and one or more memories, wherein the memory stores at least one computer program, which is loaded and executed by the processor to implement a rapid joint calibration method of a laser radar and GPS-RTK provided in the above method embodiment. The electronic device may also include other components for implementing the functions of the device, for example, the electronic device may also have components such as a wired or wireless network interface and an input and output interface to input and output data. This embodiment will not be described in detail here.

[0174] Example 4

[0175] This embodiment provides a computer-readable storage medium having a rewritable computer program stored thereon;

[0176] When the computer program runs on a computer device, the computer device executes the above-mentioned rapid joint calibration method of laser radar and GPS-RTK.

[0177] For example, the computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0178] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0179] It should be understood that determining B based on A does not mean determining B only based on A. B can also be determined based on A and / or other information.

[0180] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center through a wired network or / and a wireless network. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0181] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the present invention can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0182] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0183] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only one, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0184] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0185] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0186] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0187] In summary, the present invention is for Figure 2 , Figure 3 The calibration of the laser radar 4 and the GPS antenna 3 shown in the figure requires fewer constraints and assumptions. By using existing buildings, signs, stationary vehicles, etc. as reference objects, the hard constraints brought by fixed calibration objects can be avoided. Moreover, it can be performed online, and the solution can be continuously iterated through the optimization algorithm, thereby achieving fast and high-precision sensor joint calibration.

[0188] Compared with the prior art, the present invention according to the above scheme has the following beneficial effects: a fast joint calibration method of laser radar and GPS-RTK, which uses laser radar data and Fast-LIO algorithm to perform mileage calculation, generates continuous relative displacement information, converts the laser radar mileage information into GPS-RTK mileage data through rough structural extrinsics, and performs conditional plane calibration with the northeast sky data converted by GPS-RTK, so as to obtain the initial values ​​of the system extrinsics. Then, the laser mileage data is matched with the northeast sky coordinates, and an optimization problem is constructed. The precise system extrinsics and structural extrinsics are iteratively approximated from the initial values ​​of the system and structural extrinsics, so as to achieve high-precision joint calibration of the two. Compared with traditional methods, the present invention reduces the dependence on manual calibration points, reduces complexity, and significantly improves the efficiency and accuracy of calibration.

[0189] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention to be protected. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A rapid joint calibration method for laser radar and GPS-RTK, characterized in that: The method comprises the following specific steps: S1: Automatically identify the positions of the LiDAR sensor and GPS-RTK antenna through the structure external parameter self-checking device, and perform initialization calculation of the structure external parameters; S2: Obtain information about fixed features in the operating environment through LiDAR, and record and analyze multimodal data during the motion platform's travel; S3: extract the real-time driving environment conditions of the motion platform, and dynamically adjust the processing window and threshold value of the lidar data and GPS-RTK data according to the real-time driving environment conditions to obtain valid data in the real-time driving environment conditions; S4: Select the LiDAR mileage point-northeast celestial coordinate point pair in the valid data according to the distance selection algorithm, calculate the preliminary system extrinsic parameters through the plane calibration algorithm, construct the optimization problem of minimizing the error, perform nonlinear optimization, and iteratively solve the system extrinsic parameters and structural extrinsic parameters; S5: According to the system external parameters and structural external parameters, complete the joint calibration of the laser radar and GPS-RTK, obtain the structural external parameters of the laser radar to the GPS-RTK installation position and the system external parameters of the northeast celestial coordinate system and the geocentric earth-fixed coordinate system initialized by GPS-RTK, and complete the conversion between absolute geographic coordinates and laser mileage.

2. The rapid joint calibration method of laser radar and GPS-RTK according to claim 1 is characterized in that: S1 includes the following specific steps: S11: Install the laser radar and GPS-RTK on the motion platform, and operate the motion platform to the bottom of the structure external parameter self-checking device, wherein the structure external parameter self-checking device is composed of a depth camera suspended above the motion platform and an AI computing unit connected thereto; S12: Acquire a position image of a laser radar sensor on the motion platform through the depth camera; S13: Automatically extract the position image of the lidar sensor through the AI ​​algorithm and perform initialization calculation of the structural external parameters.

3. The rapid joint calibration method of laser radar and GPS-RTK according to claim 2 is characterized in that: S13 also includes the following specific steps: S131: Identify the laser radar and GPS-RTK antennas in the position image of the laser radar sensor through a convolutional neural network, and calculate the three-dimensional coordinates of the laser radar and GPS-RTK through pixel coordinates, actual depth of pixels and camera internal parameters; S132: importing the three-dimensional coordinates of GPS-RTK into the relative posture calculation strategy, and calculating and obtaining the relative posture between the laser radar and the GPS-RTK antenna; S133: extracting the relative position and posture between the laser radar and the GPS-RTK antenna to form the initial value of the structural external parameter.

4. The rapid joint calibration method of laser radar and GPS-RTK according to claim 3 is characterized in that: S2 includes the following specific steps: S21: extracting the real-time driving environment conditions of the motion platform. When the motion platform is close to the feature object, if the GPS-RTK signal is interfered, then execute step S3, otherwise execute step S22; S22: The mobile motion platform obtains fixed feature object information in the operating environment through a laser radar, wherein the fixed feature object information includes: edge, corner and surface data of the feature object; S23: simultaneously recording multimodal data of the motion platform during its travel, wherein the multimodal data includes: laser radar point cloud, GPS-RTK latitude and longitude information, and motion trajectory data of the IMU; S24: The geometric features of the LiDAR point cloud are extracted in real time through a deep feature extraction network, and the motion trajectory data of the IMU is combined to assist in calculating the inter-frame LiDAR mileage data.

5. The rapid joint calibration method of laser radar and GPS-RTK according to claim 4 is characterized in that: S24 includes the following specific steps: S241: Introduce a feature extraction network to automatically extract local and global geometric features of the point cloud from the geometric features of the LiDAR point cloud; S242: Global feature matching of point clouds between frames using Transformer network; S243: Use multi-scale feature fusion mechanism to extract point cloud features from different scales; S244: Use GPU to accelerate the inference process of deep learning network and complete real-time matching of point clouds between frames; S245: After completing the inter-frame matching, extract the six degrees of freedom transformation between the two frames of point clouds as the mileage data of the laser radar for subsequent external parameter calculation and optimization, wherein the six degrees of freedom include: three translation quantities and three rotation quantities.

6. The rapid joint calibration method of laser radar and GPS-RTK according to claim 5, characterized in that: S3 includes the following specific steps: S31: extracting real-time driving environment conditions of the motion platform, wherein the real-time driving environment conditions include noise characteristics, laser characteristics and GPS signal strength characteristics; The noise feature is the density of the laser radar point cloud; The laser feature is the average distance between points in the feature area; The GPS signal strength characteristics are GPS status bit and number of satellites; S32: Preset the threshold range of the laser radar point cloud density, the threshold range of the average distance between points in the feature area, the threshold of the GPS status bit and the threshold of the number of satellites, simultaneously preset the initial range of the threshold, and screen the real-time driving environment conditions according to the above threshold ranges; S33: when the noise feature exceeds the threshold range of the lidar point cloud density, increase the time window of the lidar data; When the laser feature exceeds the threshold range of the average distance between points in the feature area, the data is judged to be unusable and is removed; When the GPS signal strength characteristic is less than the threshold of the GPS status bit and the threshold of the number of satellites at the same time, the time window of the GPS data is increased and the threshold value when the laser radar matches the GPS-RTK data is increased; the threshold value when the laser radar matches the GPS-RTK data includes: the distance threshold and the error threshold in the ICP algorithm; S34: According to S32-S33, the real-time driving environment conditions after the screening process are marked as valid data, and the time window size is adaptively controlled for the process of obtaining the valid data.

7. The rapid joint calibration method of laser radar and GPS-RTK according to claim 6 is characterized in that: S3 also includes the following specific steps: S35: When the GPS signal strength feature is less than the threshold of the GPS status bit and the threshold of the number of satellites at the same time or the noise feature exceeds the threshold range of the laser radar point cloud density, the weight distribution of the sensor data is adjusted and recorded in real time, and weight constraints are provided for subsequent optimization calculations. The weight distribution strategy specifically includes: in=in gps ×in lidar-noise ×in feature ; Among them, w is the total weight value; w gps is the GPS-RTK weight factor; num star Search satellite numbers for real-time GPS-RTK; w lidar-noise is the laser radar noise weight factor; w feature is the LiDAR feature significance weight factor.

8. The rapid joint calibration method of laser radar and GPS-RTK according to claim 7, characterized in that: S4 includes the following specific steps: S41: completing the conversion of the longitude and latitude data and the northeast celestial coordinate system through a coordinate conversion algorithm, obtaining a pair of laser radar mileage points and northeast celestial coordinate points, and selecting a pair of laser radar mileage points and northeast celestial coordinate points in the valid data according to a distance selection algorithm; wherein the distance selection algorithm specifically selects points by the distance between the point pairs and the constraint condition that the three points are not collinear with each other; the number of the laser radar mileage points and northeast celestial coordinate points in the valid data is greater than or equal to 4; S42: Calculate preliminary system extrinsic parameters through a plane calibration algorithm.

9. The rapid joint calibration method of laser radar and GPS-RTK according to claim 8, characterized in that: In S41, the coordinate conversion algorithm specifically includes: S411: Convert the longitude and latitude data to the Earth-centered Earth-fixed rectangular coordinate system, as follows: Among them, (X, Y, Z) are the coordinates converted to the Earth-centered Earth-fixed rectangular coordinate system; (lon, lat, alt) are the longitude and latitude coordinates in the geodetic coordinate system; e is the eccentricity of the earth ellipsoid; N is the radius of curvature of the reference ellipsoid; S412: extracting the geocentric earth-fixed rectangular coordinate system and converting the geocentric earth-fixed rectangular coordinate system to the northeast celestial coordinate system, specifically including: calculating the northeast celestial coordinates (e, n, u) of (X, Y, Z) at the coordinate origin with coordinates (x0, y0, z0); Among them, the latitude and longitude of the coordinate origin with coordinates (x0, y0, z0) is (lon0, lat0, alt0); (Δx, Δy, Δz) is the coordinate transformation difference matrix.

10. The rapid joint calibration method of laser radar and GPS-RTK according to claim 9, characterized in that: In S42, the plane calibration algorithm includes: S421: making assumptions before solving, wherein the assumptions before solving include: considering only heading information, assuming that the origins of the translation amounts coincide, and making a plane assumption for the rotation amounts; S422: translating the mileage information and the northeast celestial coordinates respectively by removing the centroid coordinates; S423: Using two segments of trajectory points, construct an iterative closest point problem, convert the optimization problem of minimizing the error to be solved into an optimization problem of point pair fitting, solve the rotation matrix R through SVD decomposition, and extract the heading angle from R.

11. The rapid joint calibration method of laser radar and GPS-RTK according to claim 10, characterized in that: In S423, the construction of the pre-data of the optimization problem of minimizing the error to be solved includes: based on the laser radar trajectory and the GPS-RTK trajectory, combined with the point weights determined in S3; The optimization algorithms used in the optimization problem include: least square method and Ceres optimization library.

12. The rapid joint calibration method of laser radar and GPS-RTK according to claim 11, characterized in that: In S11, the AI ​​computing unit includes: the convolutional neural network used is a trained target detection network or a semantic segmentation network, which is used to detect or segment two types of targets: lidar and GPS-RTK antenna.

13. The rapid joint calibration method of laser radar and GPS-RTK according to claim 12, characterized in that: In S131, the calculation strategy of the three-dimensional coordinates is specifically: Where z is the converted metric depth, obtained from the depth map of the depth camera; (q, p) are the pixel coordinates of LiDAR and GPS-RTK; (x, y, z) is the physical coordinate in the depth camera coordinate system, that is, the physical coordinate of the laser radar and GPS-RTK; f x ,f y are the focal lengths of the depth camera in the horizontal and vertical directions respectively; C x ,C y The main point of the camera.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, a rapid joint calibration method of a laser radar and a GPS-RTK is implemented as described in any one of claims 1 to 13.

15. An electronic device, characterized in that: include: A memory for storing instructions; A processor is used to execute the instruction so that the device performs the operation of implementing a rapid joint calibration method of a laser radar and GPS-RTK as described in any one of claims 1 to 13.

Images