An external parameter calibration method, device, apparatus, server and vehicle-mounted computing device

Abstract

The application discloses an external parameter calibration method, device, equipment, server and vehicle-mounted computing equipment. The method comprises the following steps: acquiring point cloud data and pose data; constructing a target function according to the point cloud data, the pose data, first feature information and a current first conversion matrix, wherein the first feature information is obtained by performing feature extraction on the point cloud data based on a first neural network model, and the first conversion matrix is a conversion matrix from a first coordinate corresponding to the point cloud data to a second coordinate corresponding to the pose data; and updating the first conversion matrix according to the target function. In the application, the feature information extracted by the deep neural network is used to construct the target function for updating the first conversion matrix, which helps to improve the accuracy of external parameter calibration and further improve the accuracy of system mapping, positioning, navigation and other functions.

Description

Technical Field

[0001] This application relates to the field of calibration technology, and in particular to an external parameter calibration method, apparatus, equipment, server, and vehicle-mounted computing device. Background Technology

[0002] Currently, radar (hereinafter referred to as LiDAR), global navigation satellite system (GNSS), and inertial measurement unit (IMU) are widely used in autonomous driving technology. High-precision mapping, localization, perception, and navigation of autonomous vehicles all rely on the integrated use of these three systems. This integrated use can significantly reduce downstream workload and improve the accuracy of the final results.

[0003] Currently, high-precision navigation inertial measurement unit (GNSS-IMU) systems have effectively fused GNSS and IMU information for output. However, the fusion of LiDAR and GNSS-IMU systems still heavily relies on the extrinsic parameter calibration of both systems. This calibration involves determining the precise three-dimensional transformation relationship between the LiDAR and GNSS-IMU systems to convert data from the LiDAR system to the GNSS-IMU coordinate system (or vice versa), ultimately enabling the fusion of the two datasets and providing more accurate fused data for subsequent work.

[0004] Therefore, accurate external parameter calibration of Lidar and GNSS-IMU systems can effectively reduce subsequent fusion optimization work and improve the accuracy and speed of various tasks. Conversely, if the external parameter calibration is too coarse, further data optimization will be required in subsequent work, increasing the workload. Summary of the Invention

[0005] This application provides an external parameter calibration method, apparatus, and device to achieve more accurate external parameter calibration, thereby improving system accuracy.

[0006] In a first aspect, embodiments of this application provide an extrinsic parameter calibration method, comprising: acquiring first data, the first data including point cloud data and pose data; constructing an objective function based on the first data, first feature information and a current first transformation matrix, wherein the first feature information is obtained by feature extraction of the point cloud data based on a first neural network model, and the first transformation matrix is ​​a transformation matrix from a first coordinate corresponding to the point cloud data to a second coordinate corresponding to the pose data; and updating the first transformation matrix according to the objective function.

[0007] In traditional extrinsic parameter calibration, the objective function is constructed using only simple point-to-point Euclidean distance. However, point cloud data is characterized by uneven distribution and sparseness, making Euclidean distance unsuitable for reflecting the differences between two point clouds, resulting in low calibration accuracy. In this embodiment, however, the objective function is constructed using feature information extracted from a deep neural network, which helps improve the accuracy of extrinsic parameter calibration, thereby enhancing the accuracy of the system's mapping, localization, and navigation functions. Furthermore, in traditional extrinsic parameter calibration, even if feature points are extracted, only key points are randomly downsampled, without utilizing their feature information, such as semantic information. In contrast, this embodiment comprehensively considers the extracted feature information, further improving the accuracy of extrinsic parameter calibration.

[0008] In one possible implementation, the step of constructing a target function based on the first data, the first feature information, and the current first transformation matrix, and updating the first transformation matrix based on the target function, includes: constructing a target function based on the first data, the first feature information, and the current first transformation matrix; updating the current first transformation matrix based on the target function; detecting whether the accuracy of the updated first transformation matrix and / or the number of times the current first transformation matrix has been updated satisfy the preset condition; if satisfied, using the updated first transformation matrix as the target transformation matrix; if not satisfied, replacing the current first transformation matrix with the updated first transformation matrix, and returning to continue executing the process of constructing the target function based on the first data, the first feature information, and the current first transformation matrix. Repeatedly executing the above operation of updating the first transformation matrix helps to improve the accuracy of the first transformation matrix.

[0009] In one possible implementation, before constructing the objective function based on the first data, the first feature information, and the current first transformation matrix, the method further includes: performing motion compensation on the point cloud data based on the pose data and the current first transformation matrix, correcting the coordinates of points scanned in each frame to their coordinates at the same time. Motion compensation helps solve the distortion problem of the point cloud data and further improves calibration accuracy.

[0010] In one possible implementation, constructing the objective function based on the first data, the first feature information, and the current first transformation matrix includes: extracting features from the motion-compensated point cloud data based on the first neural network model to obtain second feature information; and constructing the objective function based on the first data, the second feature information, and the current first transformation matrix. Performing feature extraction on the motion-compensated point cloud data helps improve the accuracy of the extracted feature information, thereby improving the calibration accuracy.

[0011] In one possible implementation, the motion compensation of the point cloud data based on the pose data and the current first transformation matrix includes: determining a target time for scanning a target frame, where the target frame is any frame in the acquired point cloud data; determining a second transformation matrix corresponding to the target time based on the pose data at the target time and the current first transformation matrix, where the second transformation matrix is ​​a transformation matrix from the first coordinate to the third coordinate; determining a second transformation matrix corresponding to the first time for a point scanned at the first time in the target frame based on the pose data at the first time and the current first transformation matrix; and correcting the coordinates of the target points scanned at the first time to the coordinates of the target points corresponding to the target time based on the second transformation matrix corresponding to the target time and the second transformation matrix corresponding to the first time.

[0012] In one possible implementation, determining the second transformation matrix corresponding to the target time based on the pose data at the target time and the current first transformation matrix includes: determining a third transformation matrix corresponding to the target time based on the pose data at the target time, wherein the third transformation matrix is ​​a transformation matrix from the second coordinate to the third coordinate; determining the second transformation matrix corresponding to the target time based on the current first transformation matrix and the third transformation matrix corresponding to the target time, wherein the second transformation matrix is ​​a transformation matrix from the first coordinate to the third coordinate; determining the second transformation matrix corresponding to the first time based on the pose data at the first time and the current first transformation matrix includes: determining a third transformation matrix corresponding to the first time based on the pose data at the first time; determining the second transformation matrix corresponding to the first time based on the current first transformation matrix and the third transformation matrix corresponding to the first time.

[0013] In one possible implementation, constructing the objective function based on the first data, the first feature information, and the current first transformation matrix includes: performing motion compensation on the point cloud data; extracting second feature information from the motion-compensated point cloud data based on the first neural network; transforming the motion-compensated point cloud data to a third coordinate system based on the pose data and the current first transformation matrix; and constructing the objective function based on the point cloud data transformed to the third coordinate system and the second feature information. Constructing the objective function based on the motion-compensated point cloud data and feature information is more conducive to improving the accuracy of extrinsic parameter calibration.

[0014] In one possible implementation, constructing the objective function based on the point cloud data transformed to the third coordinates and the second feature information includes: constructing the objective function according to the following formula:

[0015]

[0016] in, Let N represent the objective function, and let N represent the number of points in the point cloud data or the number of feature points extracted based on the first neural network model. This represents the i-th point. Let k be the k nearest neighbors of the i-th point. The distance function represents the distance function, which includes a point-to-point distance function, a point-to-line distance function, or a point-to-surface distance function.

[0017] In one possible implementation, the point-to-point distance function is as follows:

[0018]

[0019]

[0020] in, The function represents the distance between points. These represent two distinct points, and e represents a natural number. They represent Features extracted by a deep neural network, wherein the features include K dimensions, and , Representation of features The distance between them express The traditional point-to-point distance function typically uses Euclidean distance, but this application proposes a new point-to-point distance function, which helps to further improve calibration accuracy.

[0021] In one possible implementation, the step of converting the motion-compensated point cloud data to a third coordinate based on the pose data and the current first transformation matrix includes: for the point cloud data of the target frame corrected to the target time after motion compensation, determining a third transformation matrix corresponding to the target time based on the pose data at the target time, wherein the third transformation matrix is ​​a transformation matrix from the second coordinate to the third coordinate, and the target frame is any frame in the acquired point cloud data; determining a second transformation matrix corresponding to the target time based on the third transformation matrix corresponding to the target time and the current first transformation matrix, wherein the second transformation matrix is ​​a transformation matrix from the first coordinate to the third coordinate; and converting the coordinates of each point in the target frame to the third coordinate based on the third transformation matrix.

[0022] In one possible implementation, the method further includes optimizing the objective function using a nonlinear optimization solver. Optimizing the objective function also helps improve calibration accuracy.

[0023] In one possible implementation, the nonlinear optimization solver includes the Levenberg-Marquardt algorithm.

[0024] In one possible implementation, after acquiring the first data, the method further includes: determining the currently existing first transformation matrix based on the point cloud data and the pose data, using a mapping algorithm and a hand-eye calibration algorithm. This application also provides a coarse calibration method, which can further improve the calibration accuracy before employing the aforementioned higher-precision calibration method.

[0025] In one possible implementation, determining the currently existing first transformation matrix based on the point cloud data and the pose data using a mapping algorithm and a hand-eye calibration algorithm includes: performing motion compensation on the point cloud data based on the pose data and the initial transformation matrix, correcting the coordinates of points scanned in each frame to coordinates at the same time, wherein the initial transformation matrix is ​​a preset initial first transformation matrix; extracting feature information from the motion-compensated point cloud data based on a deep neural network; determining the relative pose of the center point of each frame of the point cloud using a mapping algorithm based on the motion-compensated point cloud data and the feature information; and determining the currently existing first transformation matrix using a hand-eye calibration algorithm based on the pose data and the relative pose.

[0026] In one possible implementation, the mapping algorithm is the SC-LeGO-LOAM mapping algorithm.

[0027] In one possible implementation, the pose data includes precision, latitude, and attitude angle.

[0028] Secondly, embodiments of this application provide an external parameter calibration device, the device including modules / units for performing the first aspect and any possible design method of the first aspect; these modules / units can be implemented in hardware or by hardware executing corresponding software.

[0029] For example, it may include an acquisition module for acquiring first data, the first data including point cloud data and pose data; a processing module for constructing an objective function based on the first data, first feature information and a current first transformation matrix, the first feature information being obtained by feature extraction of the point cloud data based on a first neural network model, and the first transformation matrix being a transformation matrix from a first coordinate corresponding to the point cloud data to a second coordinate corresponding to the pose data; and updating the first transformation matrix according to the objective function.

[0030] Thirdly, embodiments of this application provide a server, the server including a processor, the processor being configured to perform the extrinsic parameter calibration method as described in the first aspect and any possible implementation thereof.

[0031] Fourthly, embodiments of this application provide an in-vehicle computing device, the in-vehicle computing device including a processor, the processor being configured to execute the external parameter calibration method as described in the first aspect and any possible implementation thereof.

[0032] Fifthly, embodiments of this application provide an external parameter calibration system, including a server as described in the fourth aspect and an in-vehicle computing device as described in the fifth aspect.

[0033] In one possible implementation, the server performs the extrinsic calibration method as described in the first aspect and any possible implementation above.

[0034] In another possible implementation, the on-board computing device performs the extrinsic calibration method as described in the first aspect and any possible implementation above.

[0035] In another possible implementation, the server and the in-vehicle computing device jointly perform the extrinsic parameter calibration method as described in the first aspect and any possible implementation above.

[0036] Sixthly, embodiments of this application provide a chip system including a processor and potentially a memory, for implementing the external parameter calibration method described in the first aspect and any possible implementation thereof. The chip system may be composed of chips or may include chips and other discrete devices.

[0037] In a seventh aspect, embodiments of this application provide a computer-readable storage medium storing computer-readable instructions that, when executed on a computer, cause the method described in any possible implementation of the first aspect to be performed.

[0038] Eighthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the extrinsic parameter calibration method as described in the first aspect and any possible implementation to be executed. Attached Figure Description

[0039] Figure 1 This is a schematic diagram illustrating an application scenario provided in the embodiments of this application;

[0040] Figure 2 This is one of the schematic diagrams of a traditional external parameter calibration process provided in the embodiments of this application;

[0041] Figure 3 This is the second schematic diagram of a traditional external parameter calibration process provided for embodiments of this application;

[0042] Figure 4 One of the flowcharts for the external parameter calibration method provided in the embodiments of this application;

[0043] Figure 5 A schematic diagram illustrating data acquisition provided in an embodiment of this application;

[0044] Figure 6 A flowchart illustrating the construction of the objective function provided in an embodiment of this application;

[0045] Figure 7 A schematic diagram of the coarse calibration process provided for embodiments of this application;

[0046] Figure 8 A second schematic flowchart illustrating the external parameter calibration method provided in this application embodiment;

[0047] Figure 9 This is a schematic diagram of the external parameter calibration device provided in the embodiments of this application;

[0048] Figure 10 This is a schematic diagram of the external parameter calibration device provided in the embodiments of this application. Detailed Implementation

[0049] The external parameter calibration scheme proposed in this application can be applied to a variety of fields. For example, it can be applied to, but is not limited to, vehicle radar, UAV radar, and other fields.

[0050] Figure 1An exemplary application scenario diagram of an embodiment of this application is provided. In this scenario, radar sensors and GNSS-IMU sensors can be installed on vehicles, and the sensors can be applied to advanced driving assistance systems (ADAS) (such as autonomous driving), robots, drones, connected vehicles, security monitoring, and other fields. Figure 1 Taking the deployment of radar sensors at the front of a vehicle as an example, the radar sensor deployed at the front of the vehicle can perceive a fan-shaped area as shown in the solid-line box. This fan-shaped area can be considered the radar perception area. When the radar sensor detects a target object in the radar perception area, it transmits the radar signal information to the processing module for further processing. The GNSS-IMU sensor can also transmit the collected data to the processing module. After receiving the information from the radar sensor and GNSS-IMU sensor, the processing module outputs the measurement information of the target object (e.g., the relative distance, angle, relative speed, and pose of the target object and the vehicle). It should be noted that the processing module here can be a computer or a software module within a computer independent of the radar sensor, such as the processing module in an onboard computer system, or it can be a computer or a software module within a computer deployed in the radar sensor or GNSS-IMU sensor; there is no limitation here.

[0051] Radar sensors can be classified into long-range radar (LRR), medium-range radar (MRR), and short-range radar (SRR) based on their different measurement ranges.

[0052] The LRR features distance measurement and collision avoidance capabilities, and is widely used in adaptive cruise control (ACC), forward collision warning (FCW), and automatic emergency braking (AEB). The LRR's ACC, FCW, and AEB functions provide significant safety alerts when drivers are distracted, fatigued, or using mobile phones and fail to notice the road ahead.

[0053] MRR and SRR feature blind spot detection (BSD), lane change assistance (LCA), rear cross traffic alert (RCTA), exit assistant function (EAF), and forward cross traffic alert (FCTA), enabling them to accurately detect targets within a certain range in front of, behind, to the sides of the vehicle. As a typical application in ADAS systems, SRR, in areas such as BSD and LCA, can effectively reduce the risk of accidents caused by poor driver visibility in adverse weather conditions such as nighttime, fog, and heavy rain, and prevent potential collisions with adjacent lanes and blind spots during lane changes.

[0054] Different application scenarios have different requirements for radar detection range, and LRR, MRR and SRR all play important roles in ADAS.

[0055] As can be seen, mounting the aforementioned radar sensors and GNSS-IMU sensors on the vehicle body allows for real-time or periodic acquisition of measurement information such as the vehicle's latitude and longitude, speed, orientation, and distances to surrounding objects. This measurement information can then be used to achieve assisted driving or autonomous driving. For example, latitude and longitude can be used to determine the vehicle's position, speed and orientation can be used to determine the vehicle's direction and destination over a future period, or the distances to surrounding objects can be used to determine the number and density of obstacles around the vehicle. Achieving high-precision positioning, obstacle recognition, and navigation relies on the high-precision external parameter calibration results of the radar system.

[0056] Extrinsic parameter calibration in radar systems involves determining the three-dimensional coordinate transformation relationship between the radar system and the GNSS-IMU system. Accurate calibration is fundamental to multi-sensor fusion; obtaining precise calibration results is essential for high-precision map creation, vehicle localization, and navigation.

[0057] The calibration procedures for commonly used radar systems and GNSS-IMU systems can be as follows: Figure 2The process includes the following steps: ① Set an initial transformation matrix from radar system coordinates to GNSS-IMU system coordinates (hereinafter referred to as the IMU-Lidar transformation matrix), for example, set it to an identity matrix. ② Obtain pose data from the GNSS-IMU system and point cloud data from the radar system. ③ Combine the current IMU-Lidar transformation matrix to transform the point cloud data in each frame to the world coordinate system. ④ Optimize the IMU-Lidar transformation matrix using an objective function (e.g., k-nn distance sum). ⑤ Determine whether the current optimization meets the stopping condition. If not, repeat steps ②-⑤; if it does, output the current IMU-Lidar transformation matrix. Here, world coordinates refer to the absolute coordinates of the system. Since the radar (or GNSS-IMU) sensor can be placed anywhere in the environment, a reference coordinate system is selected in the environment to describe the position of the radar (or GNSS-IMU) sensor and to describe the position of any object in the environment. This coordinate system is called the world coordinate system.

[0058] Another common calibration procedure for radar systems and GNSS-IMU systems can be as follows: Figure 3 As shown, the process includes the following steps: ① Obtain a coarse-precision IMU-Lidar transformation matrix through manual measurement. ② Acquire pose data from a multi-frame GNSS-IMU system and point cloud data from a radar system. ③ Combine the current IMU-Lidar transformation relationship to remove distortion from the point cloud data in each frame, and then transform it to the world coordinate system. ④ Optimize the IMU-Lidar transformation matrix using an objective function. ⑤ Determine whether the current optimization meets the stopping condition. If not, repeat steps ②-⑤; if it does, output the current IMU-Lidar transformation matrix.

[0059] However, the aforementioned extrinsic parameter calibration processes all suffer from insufficient accuracy. To further improve the accuracy of extrinsic parameter calibration, this application provides an extrinsic parameter calibration method and apparatus to enhance the accuracy of extrinsic parameter calibration, thereby enabling high-precision map creation, precise vehicle positioning, and navigation.

[0060] See Figure 4 The figure shows a flowchart illustrating the external parameter calibration method improved in this application embodiment. The method may include the following steps:

[0061] Step 401: Obtain the first data, which includes point cloud data and pose data.

[0062] Point cloud data can be point cloud data obtained from radar system scans; for example, a single radar scan can yield one frame of point cloud data. Alternatively, point cloud data can also be images containing point cloud information.

[0063] Pose data can be pose data detected by a GNSS-IMU system. Pose represents position and attitude, and pose data detected by a GNSS-IMU system can include longitude, latitude, and attitude angles (such as pitch angle, roll angle, and heading angle, or one or more of these). In some embodiments, pose data may also include altitude.

[0064] In the above steps, multiple frames of point cloud data and pose data detected by the GNSS-IMU system at multiple times can be acquired to complete the extrinsic parameter calibration.

[0065] If the above-mentioned extrinsic parameter calibration method is applied to systems for mapping and positioning in planar motion, such as systems for precise vehicle positioning, then when acquiring point cloud data and pose data, the vehicle can be positioned in a wide area at a speed of 20-40 km / h. Figure 5 The figure-eight motion shown can be illustrated by having the vehicle move in one or two figure-eight patterns. During this movement, point cloud data and pose data are collected by the radar system and GNSS-IMU system installed on the vehicle. To obtain more accurate data, a GNSS-IMU system with a pose deviation of less than 0.02 can be used for data acquisition.

[0066] Step 402: Construct an objective function based on the point cloud data, pose data, first feature information, and the current first transformation matrix. The first feature information is obtained by feature extraction from the point cloud data using the first neural network model.

[0067] The first neural network model can be a deep neural network model. Deep neural networks are an important technology in the field of machine learning. Based on deep neural networks, feature extraction can be performed on point cloud data to obtain feature point clouds and corresponding semantic information. For example, the boundary points of a target object can be extracted, and the corresponding semantic information is used to indicate that the point is a boundary point. Similarly, the center point of the target object can be extracted, and the corresponding semantic information is used to indicate that the point is a center point, and so on. Using efficient deep neural networks, feature extraction can be performed relatively efficiently and accurately.

[0068] The first transformation matrix is ​​the transformation matrix from the first coordinate corresponding to the point cloud data to the second coordinate corresponding to the pose data. It can also be understood as the transformation matrix from radar coordinates to GNSS-IMU system coordinates. The current first transformation matrix can be a preset initial transformation matrix or a coarse-precision first transformation matrix obtained through a certain method.

[0069] Step 403: Update the current first transformation matrix according to the objective function.

[0070] In the above embodiments, constructing an objective function based on the feature information extracted by the neural network helps to optimize the objective function, thereby helping to improve the accuracy of extrinsic parameter calibration.

[0071] To further improve the accuracy of the extrinsic parameter calibration, the accuracy of the first transformation matrix can be improved through multiple iterations. Specifically, steps 402 and 403 can be repeatedly executed. After each execution of steps 402 and 403, it is checked whether the preset stopping condition has been met. If it has been met, the updated first transformation matrix is ​​used as the target transformation matrix; otherwise, steps 402 and 403 are executed again. The target transformation matrix is ​​the final transformation matrix output by the above extrinsic parameter calibration process.

[0072] Optionally, the preset stopping condition may include the accuracy of the obtained first transformation matrix reaching a preset accuracy, such as the preset parameters in the first transformation matrix reaching a preset standard. Alternatively, the preset stopping condition may also include repeating steps 402 and 403 a preset maximum number of times; or, the preset stopping condition may include both the accuracy reaching the preset accuracy and the number of iterations reaching the preset maximum number of iterations. That is, if the current accuracy of the first transformation matrix does not reach the preset accuracy, and the current number of iterations does not reach the preset maximum number of iterations, then steps 402 and 403 are repeated; otherwise, the iteration ends, and the updated first transformation matrix is ​​output. Setting a stopping condition helps to reduce unnecessary calculations while meeting the accuracy requirements of the target transformation matrix.

[0073] When steps 402 and 403 are executed for the first time, the existing first transformation matrix can be a coarse-precision first transformation matrix obtained in a certain way, or it can be an initial first transformation matrix obtained by assigning initial values.

[0074] When steps 402 and 403 are not executed for the first time, that is, when steps 402 and 403 are executed for the Nth time (N is an integer greater than or equal to 2) in the case of multiple iterations, the current first transformation matrix is ​​the updated first transformation matrix obtained in the previous iteration process (i.e., steps 402 and 403).

[0075] In each iteration, feature information is extracted based on the neural network, and an objective function is constructed based on the extracted feature information. Then, the first transformation matrix is ​​updated and optimized according to the newly constructed objective function. The first transformation matrix obtained by using the extrinsic parameter calibration method provided in this application is more helpful in improving the accuracy of extrinsic parameter calibration compared with the traditional transformation matrix obtained by constructing an objective function based solely on Euclidean distance. This is more conducive to improving the accuracy of the system in functions such as mapping, positioning, and navigation.

[0076] To further improve the accuracy of extrinsic parameter calibration, motion compensation can be performed on the acquired point cloud data to correct the coordinates of the points scanned in each frame to the coordinates at the same time. Correspondingly, when extracting features from the point cloud data in the subsequent process, the feature extraction is performed on the motion-compensated point cloud data based on the first neural network model.

[0077] It should be understood that a radar scan occurs within a time period. For example, when the radar scans from left to right, it scans the left side of the target object first, then the right side. Therefore, in a single frame of point cloud data, the left-side point cloud data is acquired before the right-side data. If the target object is in motion, the resulting frame of point cloud data may contain distortion. Therefore, motion compensation of the point cloud data can resolve this distortion problem and improve calibration accuracy.

[0078] To facilitate processing, a target time can be configured for the target frame to be motion compensated. This could be the initial time of the scan, the final time of the scan, or another time during the scan process. Then, the coordinates of points in each frame of point cloud data are corrected to their corresponding target time. Furthermore, the same target time selection method can be used for each frame of point cloud data; for example, the initial time of each scan can be used as the target time for each frame.

[0079] Then, based on the pose data of the target time and the current first transformation matrix, the second transformation matrix corresponding to the target time can be determined. The second transformation matrix is ​​the transformation matrix from the first coordinate to the third coordinate, which can also be understood as the transformation matrix from radar coordinates to the third coordinate. Next, for the points scanned at the first time in the target frame (any time other than the target time within the target frame scanning period), the second transformation matrix corresponding to that first time is determined based on the pose data of that first time and the current first transformation matrix. Thus, the coordinates of the target points scanned at the first time can be corrected to the coordinates of the target points corresponding to the target time based on the second transformation matrix corresponding to the target time and the second transformation matrix corresponding to the first time. The third coordinate system can be the world coordinate system; the following example uses the world coordinate system.

[0080] In one specific embodiment, when performing motion compensation on the target frame point cloud data, the transformation matrix from the second coordinate to the world coordinate corresponding to the target time (t0) can be obtained based on the pose data of the target time in the point cloud data of that frame. This can also be understood as the transformation matrix from GNSS-IMU system coordinates to world coordinates (hereinafter referred to as the "third transformation matrix"); then, based on the current first transformation matrix... The third transformation matrix corresponding to the target time Determine the second transformation matrix corresponding to the target time. Specifically, it can be shown in formula (1):

[0081] (1)

[0082] Similarly, for the points scanned at the first (ti) time in this frame, the third transformation matrix corresponding to time ti is determined based on the pose data at the first time. According to the current first transformation matrix The third transformation matrix corresponding to the first time step Determine the second transformation matrix corresponding to the first time step. Specifically, it can be shown in formula (2):

[0083] (2)

[0084] Then, based on the second transformation matrix corresponding to the target time... The second transformation matrix corresponding to the first time step Determine the transformation matrix of the first coordinate corresponding to the point cloud data from the first time step to the target time step. Specifically, it can be shown in formula (3):

[0085] (3)

[0086] Based on the obtained transformation matrix from time ti to time t0 This allows the coordinates of the point scanned at time ti to be corrected to the coordinates of that point at the target time.

[0087] Accordingly, if motion compensation has been performed on the point cloud data, then during feature extraction, features are extracted from the motion-compensated point cloud data to obtain the second feature information. In step 402 above, the objective function is constructed based on the first data, the second feature information, and the current first transformation matrix.

[0088] It should be understood that the difference between the second feature information and the first feature information lies in whether the point cloud data input to the first neural network model has undergone motion compensation. Essentially, both are feature information obtained by feature extraction from the point cloud data. In cases where the motion compensation effect differs, the obtained first and second feature information may be the same or different. For example, the effect of motion compensation may differ between the initial motion compensation and the subsequent motion compensation after multiple iterations.

[0089] In one possible implementation, when constructing the objective function according to step 402 above, the following approach can be used: Figure 6 The objective function construction process shown is as follows:

[0090] Step b1: Based on the pose data and the current first transformation matrix, transform the point cloud data to the third coordinate system.

[0091] For example, the third coordinate system is the world coordinate system; for each frame of point cloud data, the third transformation matrix corresponding to the target time can be determined according to the pose data, and the second transformation matrix corresponding to the target time can be determined according to the third transformation matrix corresponding to the target time and the current first transformation matrix (as shown in the aforementioned formula (1)). The first coordinate corresponding to the point cloud data can be converted to the world coordinate according to the second transformation matrix.

[0092] If motion compensation has been performed on the point cloud data, and the aforementioned embodiments were used during motion compensation (i.e., motion compensation was performed according to formulas (1) to (3)), then when converting the point cloud data to world coordinates, the second transformation matrix obtained during motion compensation can be used directly without repeated calculation.

[0093] Step b2: Construct an objective function based on the point cloud data after transformation to the third coordinate system and the extracted feature information.

[0094] As mentioned above, if motion compensation is performed on the point cloud data, the feature information in step b2 above is the first feature information; if motion compensation is performed on the point cloud data, the feature information in step b2 above is the second feature information.

[0095] In a specific embodiment, the objective function can be constructed according to formula (4):

[0096]

[0097] in, Let N represent the objective function; N represents the number of points in the point cloud data or the number of feature points extracted based on the first neural network model, i.e., it can be... This represents the i-th point. Let k represent the k nearest neighbors of the i-th point. This represents a distance function, which can be a point-to-point distance function, a point-to-line distance function, or a point-to-surface distance function, etc.

[0098] Traditional point-to-point distance functions typically use Euclidean distance. However, to further improve calibration accuracy, this application proposes a new point-to-point distance function, as shown in formulas (5) and (6):

[0099] (5)

[0100] (6)

[0101] in, The function represents the distance between points. These represent two distinct points, and e represents a natural number. They represent Features extracted by a deep neural network, wherein the features include K dimensions, and , Representation of features The distance between them (e.g., KL divergence). express The Euclidean distance between them.

[0102] Furthermore, after step b2, the objective function can be optimized using a nonlinear optimization solver, which helps to further improve the calibration accuracy. For example, the Levenberg-Marquardt algorithm (LM algorithm for short) can be used to optimize the objective function.

[0103] If this extrinsic parameter calibration method is applied to planar motion, the constraints can be optimized when constructing the objective function so that the first transformation matrix obtained in this embodiment can be applied to 3-DOF data (X, Y coordinates and yaw angle) of planar motion, rather than only to 6-DOF data in space. For example, when processing extrinsic parameter calibration of vehicle-mounted lidar and GNSS-IMU systems, because the car is moving on a plane, there is rich variation information in only 3 degrees of freedom. The calibration results have large errors in the other 3 degrees of freedom, which may lead to blurry mapping. Being able to apply to 3-DOF data helps to improve mapping accuracy.

[0104] In addition, to further improve the calibration accuracy of the external parameters, a coarse calibration of the first transformation matrix can be performed before executing steps 402 and 403. That is, the coarse calibration of the first transformation matrix is ​​performed first, and then the fine calibration of the first transformation matrix is ​​performed according to steps 402 and 403.

[0105] When performing coarse calibration of the first transformation matrix, the first transformation matrix can be determined based on point cloud data and pose data using mapping algorithms and hand-eye calibration algorithms. This determined first transformation matrix serves as the "current first transformation matrix" in steps 402 and 403 above. Alternatively, when steps 402 and 403 need to be repeated, the coarsely calibrated first transformation matrix can be used as the current first transformation matrix when steps 402 and 403 are executed for the first time. Specifically, the following methods can be used: Figure 7 The coarse calibration procedure is shown below:

[0106] Step 701: Perform motion compensation on the point cloud data based on the pose data and the initial transformation matrix, and correct the coordinates of the points in each frame of point cloud data to the coordinates at the target time of that frame.

[0107] As mentioned earlier, motion compensation helps solve the problem of point cloud distortion. The initial transformation matrix is ​​a preset first transformation matrix, which can be an identity matrix, an initial matrix set based on empirical values, or a coarse-precision first transformation matrix obtained through other methods.

[0108] Step 702: Based on the first neural network model, feature information is obtained by extracting features from the motion-compensated point cloud data.

[0109] As mentioned earlier, deep neural networks can be used to extract features efficiently and accurately, yielding feature points and semantic information.

[0110] Step 703: Based on the motion-compensated point cloud data and the extracted feature information, use a mapping algorithm to determine the relative pose of the center point of each frame of the point cloud.

[0111] For example, motion-compensated point cloud data, feature points extracted by a deep network, and semantic information can be used as inputs to reconstruct a point cloud map using the SC-LeGO-LOAM mapping algorithm, and finally output the relative pose of the center points of each frame of the point cloud.

[0112] Step 704: Based on the relative pose of the center point of each frame of point cloud data and the acquired pose data, determine the first transformation matrix using the hand-eye calibration algorithm, which is the current first transformation matrix when steps 402 and 403 are executed for the first time.

[0113] To establish a relationship between the coordinate systems of the camera (i.e., the robot's eye) and other sensors (i.e., the robot's hand), it is necessary to calibrate the coordinate systems of the camera and other sensors. This calibration process is called hand-eye calibration. Using the principle of the hand-eye calibration algorithm, the first transformation matrix is ​​obtained, thus completing the coarse calibration.

[0114] To better understand the above embodiments of this application, the following is in conjunction with... Figure 8 Let's illustrate with examples. Figure 8 An extrinsic parameter calibration process is provided, which specifically includes:

[0115] Step 801: Acquire point cloud data scanned by radar and pose data detected by GNSS-IMU.

[0116] Step 802: Perform motion compensation on each frame of point cloud data based on the initial first transformation matrix (such as the identity matrix).

[0117] Step 803: Extract features from the motion-compensated point cloud data based on a deep neural network.

[0118] Step 804: Based on the motion-compensated point cloud data and the extracted feature information, reconstruct the point cloud map using a mapping algorithm and obtain the relative pose of the center point of each frame of the point cloud.

[0119] For example, the mapping algorithm is SC-LeGO-LOAM.

[0120] Step 805: Update the first transformation matrix using the hand-eye calibration algorithm based on the relative pose of the center point of each frame of point cloud data and the pose data obtained by the GNSS-IMU system.

[0121] Step 806: Perform motion compensation on each frame of point cloud data based on the updated first transformation matrix.

[0122] Step 807: Extract features from the motion-compensated point cloud data based on a deep neural network.

[0123] Step 808: Construct an objective function based on the extracted feature information, point cloud data, pose data, and the current first transformation matrix.

[0124] Step 809: Update the current first transformation matrix according to the objective function.

[0125] Step 810: Determine whether the precision of the first transformation matrix has reached the preset threshold or whether the current iteration number has reached the preset maximum iteration number; if both are not true, repeat steps 806 to 809 based on the updated first transformation matrix; otherwise, end the process and output the current first transformation matrix.

[0126] Steps 802 to 805 are coarse calibration processes, and steps 806 to 809 are fine calibration processes.

[0127] In addition, during the fine calibration process in steps 806 to 809, new point cloud data and pose data can be acquired during each iteration or during an iteration over a period of time for fine calibration.

[0128] In traditional extrinsic parameter calibration, the objective function is constructed using only simple point-to-point Euclidean distance. However, point cloud data is characterized by uneven distribution and sparseness, making Euclidean distance unsuitable for reflecting the differences between two point clouds, resulting in low calibration accuracy. In this embodiment, however, the objective function is constructed using feature information extracted from a deep neural network, which helps improve the accuracy of extrinsic parameter calibration and thus enhances the accuracy of the system's mapping, localization, and navigation functions. Furthermore, in traditional extrinsic parameter calibration, even if feature points are extracted, only key points are randomly downsampled, without utilizing their feature information, such as semantic information. This embodiment, however, comprehensively considers the extracted feature information, further improving the accuracy of extrinsic parameter calibration. Additionally, by optimizing the constraints when constructing the objective function, it can be made suitable for 3-DOF data, further improving the mapping and localization of planar moving objects.

[0129] Based on the same technical concept, this application also provides a parameter calibration device for implementing the above method embodiments. For example... Figure 9 As shown, the external parameter calibration device may include an acquisition module 901 and a processing module 902.

[0130] Specifically, the acquisition module 901 is used to acquire point cloud data from radar scanning and pose data detected by the navigation inertial measurement unit system;

[0131] The processing module 902 is used to construct an objective function based on the first data, the first feature information and the current first transformation matrix. The first feature information is obtained by extracting features from the point cloud data based on the first neural network model. The first transformation matrix is ​​the transformation matrix from the first coordinate corresponding to the point cloud data to the second coordinate corresponding to the pose data. The first transformation matrix is ​​updated according to the objective function.

[0132] In one possible implementation, the processing module 902, when constructing a target function based on the first data, the first feature information, and the current first transformation matrix, and updating the first transformation matrix based on the target function, specifically performs the following process repeatedly until a preset condition is met: constructing a target function based on the first data, the first feature information, and the current first transformation matrix; updating the current first transformation matrix based on the target function; detecting whether the preset condition is met; if it is met, using the updated first transformation matrix as the target transformation matrix; if it is not met, replacing the current first transformation matrix with the updated first transformation matrix, and returning to continue executing the process of constructing the target function based on the first data, the first feature information, and the current first transformation matrix.

[0133] In one possible implementation, the preset conditions may include, but are not limited to, the following: the precision of the first transformation matrix obtained by updating reaches a preset precision, and / or, the number of times the processing is repeated reaches a preset maximum number of times.

[0134] In one possible implementation, before constructing the objective function based on the first data, the first feature information, and the current first transformation matrix, the processing module 902 is further configured to: perform motion compensation on the point cloud data based on the pose data and the current first transformation matrix, and correct the coordinates of the points scanned in each frame to the coordinates at the same time.

[0135] In one possible implementation, the processing module 902, when constructing the objective function based on the first data, the first feature information, and the current first transformation matrix, specifically performs the following: extracts features from the motion-compensated point cloud data based on the first neural network model to obtain the second feature information; and constructs the objective function based on the first data, the second feature information, and the current first transformation matrix.

[0136] In one possible implementation, the processing module 902, when performing motion compensation on the point cloud data based on the pose data and the current first transformation matrix, specifically performs the following: determining the target time for scanning a target frame, where the target frame is any frame in the acquired point cloud data; determining the second transformation matrix corresponding to the target time based on the pose data at the target time and the current first transformation matrix, where the second transformation matrix is ​​a transformation matrix from the first coordinate to the third coordinate; determining the second transformation matrix corresponding to the first time for the points scanned at the first time in the target frame based on the pose data at the first time and the current first transformation matrix; and correcting the coordinates of the target points scanned at the first time to the coordinates of the target points corresponding to the target time based on the second transformation matrix corresponding to the target time and the second transformation matrix corresponding to the first time.

[0137] In one possible implementation, when processing module 902 determines the second transformation matrix corresponding to the target time based on the pose data at the target time and the current first transformation matrix, it is specifically used to: determine the third transformation matrix corresponding to the target time based on the pose data at the target time, wherein the third transformation matrix is ​​a transformation matrix from the second coordinate to the third coordinate; determine the second transformation matrix corresponding to the target time based on the current first transformation matrix and the third transformation matrix corresponding to the target time, wherein the second matrix is ​​a transformation matrix from the first coordinate to the third coordinate; when processing module 902 determines the second transformation matrix corresponding to the first time based on the pose data at the first time and the current first transformation matrix, it is specifically used to: determine the third transformation matrix corresponding to the first time based on the pose data at the first time; determine the second transformation matrix corresponding to the first time based on the current first transformation matrix and the third transformation matrix corresponding to the first time.

[0138] In one possible implementation, the processing module 902, when constructing the objective function based on the first data, the first feature information, and the current first transformation matrix, is specifically used for: performing motion compensation on the point cloud data; extracting second feature information from the motion-compensated point cloud data based on the first neural network; transforming the motion-compensated point cloud data to a third coordinate based on the pose data and the current first transformation matrix; and constructing the objective function based on the point cloud data transformed to the third coordinate and the second feature information.

[0139] In one possible implementation, the processing module, when constructing the objective function based on the point cloud data after transformation to the third coordinate and the second feature information, is specifically used to: construct the objective function according to the above formula (4).

[0140] In one possible implementation, the point-to-point distance function is as shown in Equations (5) and (6) above.

[0141] In one possible implementation, the processing module, when transforming the motion-compensated point cloud data to the third coordinate based on the pose data and the current first transformation matrix, specifically performs the following: for the point cloud data of the target frame corrected to the target time after motion compensation, it determines the third transformation matrix corresponding to the target time based on the pose data at the target time. The third transformation matrix is ​​a transformation matrix from the second coordinate to the third coordinate, and the target frame is any frame in the acquired point cloud data; it determines the second transformation matrix corresponding to the target time based on the third transformation matrix corresponding to the target time and the current first transformation matrix. The second transformation matrix is ​​a transformation matrix from the first coordinate to the third coordinate; and it transforms the coordinates of each point in the target frame to the third coordinate based on the third transformation matrix.

[0142] In one possible implementation, the processing module 902 is further configured to: optimize the objective function using a nonlinear optimization solver.

[0143] In one possible implementation, the nonlinear optimization solver includes the Levenberg-Marquardt algorithm.

[0144] In one possible implementation, after the acquisition module 901 acquires the first data, the processing module 902 is further configured to: determine the first transformation matrix based on the point cloud data and pose data, using a mapping algorithm and a hand-eye calibration algorithm.

[0145] In one possible implementation, the processing module 902, when determining the first transformation matrix based on point cloud data and pose data using a mapping algorithm and a hand-eye calibration algorithm, specifically performs the following: motion compensation on the point cloud data based on the pose data and the initial transformation matrix, correcting the coordinates of the points scanned in each frame to the coordinates at the same time, wherein the initial transformation matrix is ​​a preset initial first transformation matrix; extracting feature information from the motion-compensated point cloud data based on a deep neural network; determining the relative pose of the center point of each frame of the point cloud using a mapping algorithm based on the motion-compensated point cloud data and feature information; and determining the current first transformation matrix using a hand-eye calibration algorithm based on the pose data and the relative pose.

[0146] In one possible implementation, the graph construction algorithm is the SC-LeGO-LOAM graph construction algorithm.

[0147] In one possible implementation, the pose data includes longitude, latitude, and attitude angle.

[0148] Based on the same technical concept, this application also provides an external parameter calibration device for implementing the above method embodiments. For example... Figure 10 As shown, the extrinsic calibration device may include: a processor 1001 and a memory 1002 connected to the processor 1001; the processor 1001 is used to run instructions or programs in the memory 1002 to cause the extrinsic calibration device to perform the extrinsic calibration method as described in any of the preceding possible implementations.

[0149] Based on the same technical concept, embodiments of this application also provide a server, which includes a processor for executing the extrinsic parameter calibration method as described in any of the above implementations.

[0150] Based on the same technical concept, embodiments of this application also provide an in-vehicle computing device, which includes a processor for executing the external parameter calibration method as described in any of the above implementations.

[0151] This application also provides a chip system, which includes a processor and may further include a memory, for implementing the external parameter calibration method described in any of the above implementations. The chip system may be composed of chips or may include chips and other discrete devices.

[0152] Furthermore, the chip system may also include a communication interface for communicating with other devices. For example, this communication interface can connect to radar systems or GNSS-IMU systems to acquire point cloud data and pose data.

[0153] Based on the same technical concept, embodiments of this application also provide a computer-readable storage medium storing computer-readable instructions that, when executed on a computer, cause the external parameter calibration method described in any of the aforementioned possible implementations to be executed.

[0154] This application provides a computer program product containing instructions that, when run on a computer, causes the computer to execute the above-described external parameter calibration method.

[0155] It should be understood that in the description of this application, terms such as "first," "second," and "third" are used only for distinguishing purposes and should not be construed as indicating or implying relative importance or order. References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in still other embodiments" appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0156] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0158] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0159] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0160] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0161] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of the embodiments of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

Claims

1. A method for calibrating an extrinsic parameter, characterized in that, The method comprises: acquiring first data, the first data comprising point cloud data and pose data; constructing a target function according to the first data, first feature information and a current first conversion matrix, the first feature information being obtained by performing feature extraction on the point cloud data based on a first neural network model, and the first conversion matrix being a conversion matrix from a first coordinate corresponding to the point cloud data to a second coordinate corresponding to the pose data; updating the first conversion matrix according to the target function; the step of constructing the target function according to the first data, first feature information and the current first conversion matrix comprises: converting the point cloud data to a third coordinate system according to the pose data and the current first conversion matrix; and constructing the target function based on the point cloud data converted to the third coordinate system and the extracted feature information; wherein the target function is determined according to a distance function of each point and k nearest neighbor points; the each point is a point in the cloud data, or is a feature point extracted based on the first neural network model; the distance function is determined according to a feature distance of features extracted by a deep neural network and an Euclidean distance of the two points.

2. The method of claim 1, wherein, the step of constructing the target function according to the first data, first feature information and the current first conversion matrix, and updating the first conversion matrix according to the target function comprises: constructing the target function according to the first data, first feature information and the current first conversion matrix; updating the current first conversion matrix according to the target function; detecting whether the precision of the updated first conversion matrix and / or the number of times of performing the updating of the current first conversion matrix satisfies a preset condition, if yes, taking the updated first conversion matrix as a target conversion matrix, and if no, replacing the current first conversion matrix with the updated first conversion matrix and returning to continue the processing of constructing the target function according to the first data, first feature information and the current first conversion matrix.

3. The method of claim 1 or 2, wherein, before constructing the target function according to the first data, first feature information and the current first conversion matrix, the method further comprises: performing motion compensation on the point cloud data according to the pose data and the current first conversion matrix, and correcting the coordinates of points scanned in each frame to coordinates at the same time.

4. The method of claim 3, wherein, the step of constructing the target function according to the first data, first feature information and the current first conversion matrix comprises: performing feature extraction on the motion-compensated point cloud data based on the first neural network model to obtain second feature information; constructing the target function according to the first data, the second feature information and the current first conversion matrix.

5. The method of claim 3, wherein, the step of performing motion compensation on the point cloud data according to the pose data and the current first conversion matrix comprises: determining a target time for a target frame, the target frame being any one of the acquired point cloud data; determining a second conversion matrix corresponding to the target time according to the pose data of the target time and the current first conversion matrix, the second conversion matrix being a conversion matrix from the first coordinate to a third coordinate; and For the point scanned at the first time in the target frame, a second conversion matrix corresponding to the first time is determined according to the pose data of the first time and the current first conversion matrix; According to the second conversion matrix corresponding to the target time and the second conversion matrix corresponding to the first time, the coordinates of the target point scanned at the first time are corrected to the coordinates of the target point corresponding to the target time.

6. The method of claim 5, wherein, The second conversion matrix corresponding to the target time is determined according to the pose data of the target time and the current first conversion matrix, and the second conversion matrix corresponding to the first time is determined according to the pose data of the first time and the current first conversion matrix. The second conversion matrix corresponding to the target time is determined according to the pose data of the target time and the current first conversion matrix, and the second conversion matrix corresponding to the first time is determined according to the pose data of the first time and the current first conversion matrix. The second conversion matrix corresponding to the target time is determined according to the pose data of the target time and the current first conversion matrix, and the second conversion matrix corresponding to the first time is determined according to the pose data of the first time and the current first conversion matrix. The second conversion matrix corresponding to the target time is determined according to the pose data of the target time and the current first conversion matrix, and the second conversion matrix corresponding to the first time is determined according to the pose data of the first time and the current first conversion matrix.

7. The method of claim 1 or 2, wherein, The target function is constructed according to the first data, the first feature information and the current first conversion matrix, and the target function is constructed according to the first data, the first feature information and the current first conversion matrix. Motion compensation is performed on the point cloud data; Second feature information is extracted from the motion-compensated point cloud data based on the first neural network; The motion-compensated point cloud data is converted to a third coordinate according to the pose data and the current first conversion matrix; The target function is constructed according to the converted point cloud data to the third coordinate and the second feature information.

8. The method of claim 7, wherein, The target function is constructed according to the converted point cloud data to the third coordinate and the second feature information. The target function is constructed according to the following formula: wherein, denotes an objective function, N denotes a number of all points in the point cloud data or a number of feature points extracted based on the first neural network model, denotes the i-th point, denotes k nearest neighbors of the i-th point, denotes a distance function, the distance function comprising a point-to-point distance function, a point-to-line distance function, or a point-to-plane distance function.

9. The method of claim 8, wherein, The point-to-point distance function is as follows: wherein, represents a point-to-point distance function, respectively represent two different points, e represents a natural number, respectively represent features extracted by a deep neural network, the features include K dimensions, and , represents a distance between features , represents a Euclidean distance between .

10. The method of claim 7, wherein, The motion-compensated point cloud data is converted to a third coordinate according to the pose data and the current first conversion matrix. For the point cloud data of the target frame after motion compensation and correction to the target time, a third conversion matrix corresponding to the target time is determined according to the pose data of the target time, and the third conversion matrix is a conversion matrix from the second coordinate to the third coordinate. A second conversion matrix corresponding to the target time is determined according to the third conversion matrix corresponding to the target time and the current first conversion matrix, and the second conversion matrix is a conversion matrix from the first coordinate to the third coordinate. The coordinates of each point in the target frame are converted to the third coordinate according to the third conversion matrix.

11. The method of claim 1 or 2, wherein, After obtaining the first data, the method further comprises: A first conversion matrix is determined by using a mapping algorithm and a hand-eye calibration algorithm according to the point cloud data and the pose data.

12. The method of claim 11, wherein, A first conversion matrix is determined by using a mapping algorithm and a hand-eye calibration algorithm according to the point cloud data and the pose data. Motion compensation is performed on the point cloud data according to the pose data and an initial conversion matrix, the initial conversion matrix being a preset initial first conversion matrix, so as to correct the coordinates of points scanned in each frame to coordinates at the same time instant. Feature information is extracted from the motion-compensated point cloud data based on the first neural network model. A relative pose of a center point of each frame of point cloud is determined by using a mapping algorithm according to the motion-compensated point cloud data and the feature information. A current first conversion matrix is determined by using a hand-eye calibration algorithm according to the pose data and the relative pose.

13. The method of claim 1 or 2, wherein, The pose data includes longitude, latitude and attitude angle.

14. An external parameter calibration device, characterized in that The method comprises: An acquisition module is configured to acquire first data, the first data including point cloud data and pose data; A processing module is configured to construct a target function according to the first data, first feature information and a current first conversion matrix, the first feature information being obtained by performing feature extraction on the point cloud data based on a first neural network model, and the first conversion matrix being a conversion matrix from first coordinates corresponding to the point cloud data to second coordinates corresponding to the pose data; and update the first conversion matrix according to the target function. When the processing module constructs the target function according to the first data, first feature information and the current first conversion matrix, the processing module is specifically configured to: convert the point cloud data to a third coordinate system according to the pose data and the current first conversion matrix; and construct the target function based on the point cloud data converted to the third coordinate system and the extracted feature information. The target function is determined according to a distance function of each point and k nearest neighbor points. The distance function is determined according to a feature distance of features extracted by a deep neural network and an Euclidean distance of the two points. When the processing module constructs the target function according to the first data, first feature information and the current first conversion matrix and updates the first conversion matrix according to the target function, the processing module is specifically configured to:

15. The apparatus of claim 14, wherein, construct the target function according to the first data, first feature information and the current first conversion matrix; update the current first conversion matrix according to the target function; detect whether the accuracy of the updated first conversion matrix and / or the number of times of updating the current first conversion matrix satisfies a preset condition, if yes, take the updated first conversion matrix as a target conversion matrix, and if no, replace the current first conversion matrix with the updated first conversion matrix and return to continue to construct the target function according to the first data, first feature information and the current first conversion matrix. Before the processing module constructs the target function according to the first data, first feature information and the current first conversion matrix, the processing module is further configured to:

16. The apparatus of claim 14 or 15, wherein, motion compensate the point cloud data according to the pose data and the current first conversion matrix, and correct the coordinates of points scanned in each frame to coordinates at the same time instant. ​ 17. The apparatus of claim 16, wherein, The processing module is specifically configured to: extract the second feature information based on the first neural network model and the point cloud data after motion compensation. The processing module is specifically configured to:

18. The apparatus of claim 16, wherein, determine a target time according to a target frame scan, the target frame being any one of the acquired point cloud data frames; determine a second conversion matrix corresponding to the target time according to the pose data of the target time and the current first conversion matrix, the second conversion matrix being a conversion matrix from the first coordinate to the third coordinate; determine a second conversion matrix corresponding to the first time according to the pose data of the first time and the current first conversion matrix for the point scanned at the first time in the target frame; correct the coordinates of the target point scanned at the first time to the coordinates of the target point corresponding to the target time according to the second conversion matrix corresponding to the target time and the second conversion matrix corresponding to the first time. The processing module is specifically configured to:

19. The apparatus of claim 18, wherein, determine a third conversion matrix corresponding to the target time according to the pose data of the target time, the third conversion matrix being a conversion matrix from the second coordinate to the third coordinate; and determine the second conversion matrix corresponding to the target time according to the current first conversion matrix and the third conversion matrix corresponding to the target time, the second conversion matrix being a conversion matrix from the first coordinate to the third coordinate. The processing module is specifically configured to: determine a third conversion matrix corresponding to the first time according to the pose data of the first time; and determine the second conversion matrix corresponding to the first time according to the current first conversion matrix and the third conversion matrix corresponding to the first time. The processing module is specifically configured to:

20. The apparatus of claim 14 or 15, wherein, perform motion compensation on the point cloud data; extract the second feature information based on the first neural network and the point cloud data after motion compensation; convert the point cloud data after motion compensation to the third coordinate according to the pose data and the current first conversion matrix; construct the target function according to the point cloud data converted to the third coordinate and the second feature information. The processing module is specifically configured to:

21. The apparatus of claim 20, wherein, construct the target function according to the following formula: The point-to-point distance function is as follows: wherein, represents a target function, N represents a number of all points in the point cloud data or a number of feature points extracted based on the first neural network model, represents the i-th point, represents k nearest neighbor points of the i-th point, represents a distance function, the distance function including a point-to-point distance function, a point-to-line distance function, or a point-to-plane distance function.

22. The apparatus of claim 21, wherein, The processing module is specifically configured to: wherein, represents a point-to-point distance function, respectively represent two different points, e represents a natural number, respectively represent features extracted by a deep neural network, the features include K dimensions, and , represents a distance between features , represents a Euclidean distance between .

23. The apparatus of claim 20, wherein, convert the point cloud data after motion compensation to the third coordinate according to the pose data and the current first conversion matrix. The point cloud data of the target frame is corrected to the point cloud data of the target time after motion compensation, a third conversion matrix corresponding to the target time is determined according to the pose data of the target time, the third conversion matrix is a conversion matrix from the second coordinate to a third coordinate, and the target frame is any frame of the obtained point cloud data; A second conversion matrix corresponding to the target time is determined according to the third conversion matrix corresponding to the target time and the current first conversion matrix, and the second conversion matrix is a conversion matrix from the first coordinate to the third coordinate; The coordinates of each point in the target frame are converted to the third coordinate according to the third conversion matrix.

24. The apparatus of claim 14 or 15, wherein, After the acquisition module acquires the first data, the processing module is further configured to: A first conversion matrix is determined by using a mapping algorithm and a hand-eye calibration algorithm according to the point cloud data and the pose data.

25. The apparatus of claim 24, wherein, When the processing module determines the first conversion matrix by using the mapping algorithm and the hand-eye calibration algorithm according to the point cloud data and the pose data, the processing module is specifically configured to: The point cloud data is motion compensated according to the pose data and an initial conversion matrix, and the coordinates of points scanned in each frame are corrected to the coordinates at the same time, and the initial conversion matrix is a preset initial first conversion matrix; Feature information is extracted from the motion-compensated point cloud data based on a deep neural network; A relative pose of a center point of each frame of point cloud is determined by using a mapping algorithm according to the motion-compensated point cloud data and the feature information; A current first conversion matrix is determined by using a hand-eye calibration algorithm according to the pose data and the relative pose.

26. The apparatus of claim 14 or 15, wherein, The pose data includes longitude, latitude and attitude angle.

27. A server, comprising: The server comprises a processor configured to perform the external parameter calibration method according to any one of claims 1-13.

28. An in-vehicle computing device, comprising: The vehicle-mounted computing device comprises a processor configured to perform the external parameter calibration method according to any one of claims 1-13.

29. A computer-readable storage medium, characterized in that, A computer readable instruction is stored, and when the computer readable instruction runs on a computer, the method according to any one of claims 1-13 is executed.

Images