Laser radar external parameter calibration method and device, readable medium and program product

Abstract

The invention relates to the technical field of intelligent auxiliary driving, in particular to a laser radar external parameter calibration method and device, a readable medium and a program product. In the vehicle driving process, a first point cloud collected by a laser radar of the vehicle at a first moment and a second point cloud collected by the laser radar at a second moment are obtained, and the second moment is after the first moment; determining a plurality of first feature points in the first point cloud; based on the positions of the first feature points, the driving data of the vehicle from the first moment to the second moment, and the original external parameters of the laser radar, determining the predicted position of each first feature point at the second moment; determining second feature points corresponding to the first feature points in the second point cloud; and according to the deviation between the predicted position of the first feature point and the position of the second feature point, optimizing the original external parameter to obtain a calibration result of the external parameter. In this way, calibration of the external parameters of the laser radar can be completed without depending on a target, and the accuracy of vehicle positioning and environment sensing is improved.

Description

[0001] This application is a divisional application. The application number of the parent application is 202511821379.7, and the application date of the parent application is December 4, 2025. The entire contents of the parent application are incorporated herein by reference. Technical Field

[0002] This application relates to the field of intelligent assisted driving technology, specifically to lidar extrinsic parameter calibration methods, equipment, readable media, and program products. Background Technology

[0003] In driver assistance systems, LiDAR, as one of the core sensors, can perceive environmental elements around the vehicle in real time using point cloud recognition technology. To achieve accurate localization of these environmental elements, the extrinsic parameters of the LiDAR (i.e., the external parameters of the LiDAR coordinate system relative to the vehicle coordinate system, VCS) play a crucial role. Through these extrinsic parameters, driver assistance and autonomous driving systems can precisely map the environmental elements captured by the LiDAR onto the vehicle coordinate system, thereby providing reliable spatial reference information for path planning, obstacle avoidance, and decision-making.

[0004] However, vehicles equipped with lidar need to undergo extrinsic parameter calibration in scenarios lacking targets after leaving the factory. Existing targetless lidar extrinsic parameter calibration methods suffer from technical challenges such as difficulty in feature extraction, high requirements for the scene, and low automation, resulting in low calibration accuracy, difficulty in convergence, and low efficiency. Summary of the Invention

[0005] To address the issues of low accuracy, convergence difficulties, and low efficiency in calibrating the extrinsic parameters of lidar in targetless environments due to challenges in feature extraction, high scene requirements, and low automation, embodiments of this application provide a method, apparatus, readable medium, and program product for calibrating lidar extrinsic parameters.

[0006] In a first aspect, embodiments of this application provide a calibration method for the extrinsic parameters of a lidar, applied to an in-vehicle electronic device. The method includes: during vehicle operation, acquiring a first point cloud collected by the vehicle's lidar at a first moment and a second point cloud collected at a second moment, wherein the second moment is after the first moment; determining multiple first feature points in the first point cloud; determining the predicted position of each first feature point at the second moment based on the position of the first feature points, the vehicle's driving data from the first moment to the second moment, and the original extrinsic parameters of the lidar; determining second feature points in the second point cloud corresponding to the predicted positions of each first feature point; and adjusting the yaw angle in the original extrinsic parameters according to the deviation between the predicted positions of the first feature points and the corresponding positions of the second feature points to obtain a calibration result for the yaw angle.

[0007] The method provided in this application can optimize the yaw angle in the original extrinsic parameters of the lidar by using the point cloud collected by the lidar at two time points and the driving data of the vehicle during the time interval between the two time points to obtain the calibration result of the lidar yaw angle. It does not rely on a fixed target in the vehicle driving environment, which reduces the environmental requirements for lidar extrinsic parameter calibration and improves the calibration accuracy.

[0008] In one possible implementation of the first aspect described above, determining the second feature points corresponding to the predicted positions of each first feature point in the second point cloud includes: determining the three points in the second point cloud closest to the predicted positions of each first feature point, and using these three points as the three second feature points corresponding to the first feature points; adjusting the yaw angle in the original extrinsic parameters based on the deviation between the predicted positions of the first feature points and the corresponding positions of the second feature points to obtain a calibration result for the yaw angle, including: calculating the first distance between the predicted positions of each first feature point and the first plane corresponding to the first feature point, wherein the first plane corresponding to the first feature point is the plane determined by the three second feature points corresponding to the first feature point; and adjusting the yaw angle based on the first distance corresponding to each first feature point to obtain a calibration result for the yaw angle.

[0009] The method provided in this application matches the predicted position of the first feature point at the first time moment with the actual point cloud at the second time moment, and optimizes the deviation between the predicted position and the actual point cloud by using the distance from the predicted position to the plane determined by the actual point cloud as a constraint, thereby optimizing the yaw angle in the original extrinsic parameters and ensuring the accuracy of the calibration.

[0010] In one possible implementation of the first aspect described above, the lidar includes multiple scanning beams; determining multiple first feature points in a first point cloud includes: grouping the points in the first point cloud according to the scanning beams they belong to, to obtain multiple groups of points; determining candidate points in each group of points, and determining at least some of the candidate points as first feature points, wherein the candidate points are points in each group whose curvature value exceeds a first threshold.

[0011] The method provided in this application involves grouping the point cloud acquired by the lidar according to the scanning beam and selecting points with curvature values ​​greater than a first threshold as first feature points. These points have significant geometric features, relatively stable positions, and high recognizability.

[0012] Using these points for lidar extrinsic parameter calibration can reduce calibration errors and improve calibration accuracy and robustness.

[0013] In one possible implementation of the first aspect above, the curvature of a point is the average of the depth differences between the point and its N neighboring points, where N is a positive integer greater than or equal to 5 and less than or equal to 15.

[0014] In one possible implementation of the first aspect above, the scanning beam includes M segments connected in sequence, where M is a positive integer greater than or equal to 2; determining candidate points in each group of points and determining at least some of the candidate points as first feature points includes: dividing each group of points into M point sets, where the M point sets are respectively located in the M segments of the scanning beam; determining candidate points in each point set; and selecting the top K candidate points with the largest curvature values ​​in each point set as first feature points, where K is a positive integer greater than or equal to 15 and less than or equal to 20.

[0015] In one possible implementation of the first aspect above, before determining candidate points in each of the multiple sets of points, the method further includes: deleting points in each set of points whose curvature values ​​exceed a second threshold, the second threshold being greater than a first threshold.

[0016] In one possible implementation of the first aspect above, the first threshold is 0.1, and / or the second threshold is 0.4.

[0017] Secondly, embodiments of this application provide a calibration method for the extrinsic parameters of a lidar, applied to an in-vehicle electronic device. The method includes: acquiring a third point cloud collected by the vehicle's lidar at a third moment during vehicle operation; selecting points with heights less than a third threshold from the third point cloud to construct a fourth point cloud; acquiring the vehicle's attitude information at the third moment; correcting the initial position of the fourth point cloud based on the attitude information to obtain the corrected position of the fourth point cloud; constructing a second plane based on the corrected position of the fourth point cloud; and obtaining the calibration results of the lidar's pitch angle, roll angle, and height based on the plane parameters of the second plane.

[0018] The method provided in this application corrects the point cloud data acquired by LiDAR using vehicle attitude information, fits a plane equation based on the corrected point cloud data, and obtains the calibration results of LiDAR pitch angle, roll angle, and altitude based on the parameters of the plane equation. The introduction of vehicle attitude information eliminates the impact of vehicle vibrations caused by uneven ground or aggressive driving, improving the accuracy of ground modeling and thus enhancing the accuracy and robustness of LiDAR extrinsic parameter calibration.

[0019] In one possible implementation of the second aspect above, before correcting the initial position of the fourth point cloud based on the attitude information, the method further includes: removing invalid points in the fourth point cloud; wherein, the method for determining invalid points in the fourth point cloud includes: constructing a third plane based on the fourth point cloud; calculating a second distance from each point in the fourth point cloud to the third plane; and determining points whose second distance is greater than a fourth threshold as invalid points in the fourth point cloud.

[0020] The method provided in this application removes points that are far from the third plane. These points have significant differences compared to other points in the fourth point cloud and are not ground points, so they are invalid points. By removing these invalid points, the accuracy of the external parameter calibration of the lidar in this application can be guaranteed.

[0021] In one possible implementation of the second aspect above, the attitude information includes the vehicle's pitch angle and roll angle; the initial position of the fourth point cloud is corrected based on the attitude information to obtain the corrected position of the fourth point cloud, including: determining a rotation matrix based on the vehicle's pitch angle and roll angle; and correcting the initial position of the fourth point cloud based on the rotation matrix to obtain the corrected position of the fourth point cloud.

[0022] In one possible implementation of the second aspect above, constructing a second plane based on the corrected position of the fourth point cloud includes: constructing the second plane based on a random sampling consensus algorithm according to the corrected position of the fourth point cloud.

[0023] Thirdly, embodiments of this application provide an in-vehicle electronic device, including one or more processors; one or more memories; the one or more memories storing one or more programs, which, when executed by one or more processors, cause the in-vehicle device to perform the calibration method for lidar extrinsic parameters of the first aspect and any possible implementation of the first aspect, and to perform the calibration method for lidar extrinsic parameters of the second aspect and any possible implementation of the second aspect.

[0024] Fourthly, embodiments of this application provide a computer-readable medium storing instructions that, when executed on an electronic device, cause the electronic device to perform a calibration method for lidar extrinsic parameters according to the first aspect and any possible implementation thereof, as well as a calibration method for lidar extrinsic parameters according to the second aspect and any possible implementation thereof.

[0025] Fifthly, embodiments of this application provide a computer program product comprising computer instructions that, when executed by an electronic device, cause the electronic device to perform a calibration method for lidar extrinsic parameters of the first aspect and any possible implementation thereof, and to perform a calibration method for lidar extrinsic parameters of the second aspect and any possible implementation thereof.

[0026] The beneficial effects of the third to fifth aspects mentioned above can be found in the first and second aspects, as well as the beneficial effects of any possible implementation of the first aspect and any possible implementation of the second aspect, and will not be repeated here. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings are described below.

[0028] Figure 1A According to some embodiments of this application, a schematic diagram of a scenario for lidar extrinsic parameter calibration is shown;

[0029] Figure 1B According to some embodiments of this application, a scenario diagram for lidar extrinsic parameter calibration is shown. Figure 2 ;

[0030] Figure 2 According to some embodiments of this application, a scenario diagram for lidar extrinsic parameter calibration is shown. Figure 3 ;

[0031] Figure 3 According to some embodiments of this application, a flowchart of a method for calibrating the extrinsic parameters of a lidar is shown;

[0032] Figure 4 According to some embodiments of this application, a flowchart of a method for determining a first feature point is shown;

[0033] Figure 5 According to some embodiments of this application, a schematic diagram of a yaw angle calibration method is shown;

[0034] Figure 6 According to some embodiments of this application, a flowchart of another method for calibrating the extrinsic parameters of a lidar is shown;

[0035] Figure 7 According to some embodiments of this application, a flowchart of a preprocessing method for a fourth point cloud is shown;

[0036] Figure 8 According to some embodiments of this application, a schematic diagram of the structure of an electronic device is shown;

[0037] Figure 9 According to some embodiments of this application, a block diagram of a system-on-a-chip is shown. Detailed Implementation

[0038] To facilitate understanding of the solutions in the embodiments of this application by those skilled in the art, the terms involved in the embodiments of this application will be explained below.

[0039] (1) External parameters of lidar.

[0040] The external parameters of vehicle-mounted LiDAR mainly include the rotation parameters (pitch, roll, yaw) and displacement parameters (radar altitude h) of the LiDAR relative to the vehicle coordinate system (VCS). The calibration of these parameters is a key step in achieving the accuracy of vehicle environmental perception.

[0041] The following combination Figures 1A to 2 The extrinsic parameters of lidar are described in detail. Figure 1A and Figure 1B This is a schematic diagram of the vehicle body coordinate system (VCS). Figure 2 It is a schematic diagram of pitch, roll, yaw, and radar altitude h in the vehicle coordinate system.

[0042] like Figure 1A and Figure 1B As shown, a VCS coordinate system is established with the projection point of the vehicle's rear axle center onto the ground as the origin. The X, Y, and Z directions represent the vehicle's front, left, and up directions, respectively. For example, when the user is sitting in the driver's seat facing the steering wheel, the X direction represents the user's front-to-back direction, the Y direction represents the user's left-to-right direction, and the Z direction represents the user's up-to-down direction.

[0043] like Figure 2 As shown, the roll, pitch, and yaw angles of the lidar are rotation angles in the VCS coordinate system with the X, Y, and Z axes as rotation axes and the right-hand screw rule as the rotation direction. Facing the rotation axis, the clockwise angle is negative, and the counterclockwise angle is positive. The lidar height h is the height at which the lidar reaches the ground (XY plane).

[0044] In some scenarios, when a vehicle with assisted driving is driving in real time, the onboard LiDAR will collect environmental elements around the vehicle and map them onto the vehicle coordinate system through the LiDAR extrinsic parameters, providing decision-making information such as path planning and obstacle avoidance for autonomous or assisted driving.

[0045] Existing methods for calibrating the extrinsic parameters of lidar typically involve calibrating the vehicle before it leaves the factory in a workshop with fixed targets as environmental references. However, after the lidar is put into use, its extrinsic parameters dynamically change due to factors such as vehicle vibration, prolonged use, and mechanical wear during actual vehicle operation. This causes the original calibration results to gradually deviate from the true values, resulting in inaccurate vehicle positioning and environmental perception.

[0046] Therefore, after vehicles equipped with lidar leave the factory, their extrinsic parameters still need to be calibrated in scenarios lacking targets. Existing targetless lidar extrinsic parameter calibration methods suffer from technical challenges such as difficulty in feature extraction, high requirements for the scene, and low automation, resulting in low calibration accuracy, difficulty in convergence, and low efficiency.

[0047] To address the aforementioned issues, this application provides a method for calibrating the extrinsic parameters of a lidar. During vehicle operation, point cloud data collected by the lidar at a first and second time point are acquired. Multiple first feature points are identified within the first point cloud. Based on the positions of the first feature points, vehicle driving data, and the original extrinsic parameters, the position of each first feature point at the second time point is predicted. Second feature points corresponding to the predicted positions of each first feature point are found in the second point cloud. By comparing the deviation between the predicted position and the actual position (second feature point) at the second time point, the yaw angle in the original extrinsic parameters is optimized to obtain the calibration result of the lidar's yaw angle.

[0048] Thus, only the point cloud data of the lidar at two time points and the vehicle's driving data are needed to calibrate the lidar's yaw angle without a fixed target, reducing the environmental requirements for calibration and improving calibration accuracy.

[0049] It is understood that this application does not limit the type or brand of lidar. As long as it can emit a laser beam and receive reflected signals to obtain relevant information about the target, such as the target distance, azimuth, and height, it can determine parameters such as roll angle, pitch angle, yaw angle, and radar height in the vehicle coordinate system (VCS) constructed in this application, so as to realize the perception and monitoring of the vehicle's surrounding environment.

[0050] It is understood that the execution subject of each step of the lidar extrinsic parameter calibration method shown in the embodiments of this application can be an in-vehicle electronic device. For ease of description, the execution subject of each step will not be described again in the following description of each step.

[0051] The calibration of the external parameters of the lidar provided in this application is described below with reference to the accompanying drawings. Figure 3 A flowchart illustrating a method for calibrating the extrinsic parameters of a lidar system is shown. Figure 3 As shown, the method may include the following S301-S305.

[0052] S301: During vehicle operation, acquire the first point cloud collected by the vehicle's lidar at the first moment and the second point cloud collected at the second moment, which is after the first moment.

[0053] It is understandable that, during vehicle operation, LiDAR, as one of the core sensors, needs to be able to perceive the environmental elements around the vehicle in real time. LiDAR generates a set of three-dimensional spatial points, i.e., a point cloud, by emitting laser beams and receiving the reflected signals, combined with spatial positioning technology.

[0054] For example, the time difference between the second moment and the first moment can be 0.01s to 1s, such as 0.02s, 0.1s, etc.

[0055] S302: Determine multiple first feature points in the first point cloud.

[0056] It is understandable that the first point cloud is the raw data collected by the lidar. The raw data is large in volume and contains flaws. If the raw first point cloud is not preprocessed, it will affect the calibration accuracy of the lidar extrinsic parameters. In addition, the large amount of data will increase the calibration difficulty.

[0057] Therefore, in order to ensure the accuracy and efficiency of subsequent calibration of lidar extrinsic parameters, this application extracts the first feature point from the first point cloud.

[0058] In some embodiments of this application, the first feature point can be a corner point that differs significantly from the surrounding point cloud (such as a point located at the edge of an obstacle or at a corner in the road where the vehicle is traveling). Compared with the surrounding point cloud, these corner points have distinct geometric features and are easy to identify and distinguish. Therefore, the external parameter calibration of the lidar based on the corner points can reduce errors and improve calibration accuracy.

[0059] For the sake of narrative coherence, please refer to the following text for the method of determining the first feature point. Figure 4 Some details will not be repeated here.

[0060] S303: Based on the position of the first feature point, the vehicle's driving data from the first time to the second time, and the raw extrinsic parameters of the lidar, determine the predicted position of each first feature point at the second time.

[0061] As can be seen from the above, the point cloud collected by the lidar is a set of three-dimensional points. Therefore, the position of the first feature point determined based on the first point cloud can be represented by its corresponding three-dimensional coordinates in the VCS coordinate system.

[0062] In some embodiments of this application, the vehicle's driving data from the first moment to the second moment can be provided by the vehicle's three-dimensional odometer.

[0063] It is understood that the original extrinsic parameters of the LiDAR are the original calibration parameters when the vehicle leaves the factory. This application obtains the true extrinsic parameters of the LiDAR during vehicle operation by optimizing the original extrinsic parameters. In some embodiments of this application, the predicted position of each first feature point at the second time moment can be obtained based on the following calculation method:

[0064]

[0065] in, The point cloud at time t-1 (the first time point) is the first point cloud. Let i be the i-th point in the point cloud (first point cloud) at time t-1; Let i be the predicted position of the i-th point in the point cloud (first point cloud) at time t (second time). It is the set of the predicted locations mentioned above; It is the homogeneous transformation matrix of the vehicle from time t-1 to time t (from the first time to the second time), which describes the vehicle's driving data (such as the vehicle's pose change) from time t-1 to time t (from the first time to the second time). This is a matrix that characterizes the original extrinsic parameters of the lidar sensor in the VCS coordinate system.

[0066] In the above formula, the predicted position of the lidar point cloud at the first moment is calculated at the second moment based on the original extrinsic parameters of the lidar and the pose transformation matrix of the vehicle from the first moment to the second moment.

[0067] In this embodiment, the calculation of the predicted position references the original extrinsic parameters of the LiDAR and vehicle motion information (pose transformation matrix). The predicted position determined in this application represents the position where each feature point in the point cloud at the first moment "should" appear at the second moment, assuming the environment is static and the vehicle moves precisely according to the given pose transformation. By comparing and analyzing these predicted positions with the LiDAR point cloud actually acquired at the second moment, the original extrinsic parameters of the LiDAR can be optimized.

[0068] S304: Determine the second feature point in the second point cloud that corresponds to the predicted position of each first feature point.

[0069] In some embodiments of this application, the three points closest to the predicted positions of each first feature point are determined in the second point cloud, and these three points are used as the three second feature points corresponding to the first feature points.

[0070] In some embodiments of this application, for example, the Iterative ClosestPoint (ICP) algorithm can be used to determine the predicted position of each first feature point at each second time step. By performing feature matching with points in the actual second point cloud in the VCS coordinate system acquired by the lidar at the second moment, we can find points in the second point cloud that correspond to each predicted position. The three most recent points, that is, the three second feature points , , .

[0071] In some embodiments of this application, when using the Iterative Nearest Point (ICP) algorithm to find the nearest neighbor, three nearest neighbors are selected for each predicted location. Compared to selecting one nearest neighbor, the approach of this application helps to improve the robustness of subsequent lidar extrinsic parameter calibration.

[0072] S305: Based on the deviation between the predicted position of the first feature point and the position of the corresponding second feature point, adjust the yaw angle in the original extrinsic parameters to obtain the calibration result of the yaw angle.

[0073] As shown in S304, the predicted position of each first feature point... The three corresponding second feature points are , , The original extrinsic parameter matrix of the lidar is: .

[0074] In this embodiment of the application, the original extrinsic parameter matrix of the lidar This can include the original yaw angle, original pitch angle, original roll angle, and original altitude. For the original pitch angle, original roll angle, and original altitude, it is assumed that their original values ​​are already accurately calibrated. Only the yaw angle among the original extrinsic parameters is optimized to obtain the calibrated yaw angle result. For the specific process of yaw angle optimization, please refer to [link to relevant documentation]. Figure 5 Partial description.

[0075] As described above in S301 to S305, the calibration method provided in this application embodiment can perform high-precision online calibration of the yaw angle in the extrinsic parameters of a lidar system during dynamic vehicle operation. It eliminates the need for a static calibration environment or high-precision manual targets, overcoming the dependence of traditional calibration methods on specific scenarios and external equipment, and significantly improving the robustness and practicality of the lidar system in real, complex traffic environments.

[0076] The following is about Figure 3 This application provides a detailed explanation of each step in the calibration method for the external parameters of a lidar.

[0077] First, the method for determining the first feature point involved in S302 above will be described. Figure 4 A flowchart illustrating a method for determining the first feature point is shown. Figure 4As shown, the method may include the following S401-S402.

[0078] S401: Group the points in the first point cloud according to the scan beam they belong to, to obtain multiple groups of points.

[0079] It is understandable that a LiDAR system comprises multiple scanning beams, each beam corresponding to beam information. The point cloud is grouped based on the beam information, with points on the same beam forming a group. Points on the same scanning beam are arranged in the scanning order. The points in the first point cloud are grouped according to the beam they belong to, following the acquisition rules of LiDAR data. This also facilitates the identification of each point's neighboring points and the geometric changes that the point undergoes compared to its neighbors.

[0080] S402: Determine candidate points in each group of points from multiple groups of points, and determine at least some of the candidate points as first feature points, wherein the candidate points are the points in each group of points whose curvature values ​​exceed a first threshold.

[0081] In some embodiments of this application, the curvature value of the aforementioned point is the average of the depth differences between the point and its N adjacent points, where N is a positive integer greater than or equal to 5 and less than or equal to 15. For example, N can be 10, in which case the curvature value of each point in the first point cloud is the average of the depths of that point and its five adjacent points before and after it. It is understood that in other examples, N can also be 8, 12, or other values.

[0082] For example, this application provides a method for calculating any point in a first point cloud. curvature value The method is as follows:

[0083]

[0084] in, Let i be the i-th point in the first point cloud. The modulus at that point; For point The curvature value; for The set of neighboring points, Let N be the size of the set; for example, when N is 10. It contains points Five neighboring points before and after, =10; for The point in, that is The nearest point; For point With sets All neighboring points The magnitude of the sum of vector differences.

[0085] In some embodiments of this application, after calculating the curvature value of each point using the above formula, candidate points can be selected based on the magnitude of the curvature value. The curvature value of a candidate point is greater than a first threshold. .

[0086] In some embodiments of this application, the scanning beam of the lidar includes M segments connected sequentially, where M is a positive integer greater than or equal to 2. For example, M can be 6, meaning each beam of the lidar can be divided into 6 segments. Correspondingly, the determination of the first feature point can also be based on the segmentation of the lidar scanning beam. Points on each beam segment can be treated as a point set, and the point cloud corresponding to that beam can be further divided into M point sets. Candidate points and the first feature point are then determined in each point set. It is understood that in other examples, M can also be 4, 10, or other values. In some embodiments, M can be less than 20.

[0087] In some embodiments of this application, after candidate points are selected, in each point set, each point in the first point cloud is sorted according to its corresponding curvature value. The points are sorted in descending order of their curvature values, and the top K candidate points with the largest curvature values ​​are selected as the first feature points. K can be a positive integer greater than or equal to 15 and less than or equal to 20. For example, K can be 15, 18, 20, etc.

[0088] In some embodiments of this application, before determining candidate points in each group of points from multiple groups of points, the processing of the first point cloud further includes: deleting points in each group whose curvature values ​​exceed a second threshold. The point, the second threshold Greater than the first threshold .

[0089] It is understandable that points with excessively large curvature values ​​indicate that their depth difference compared to neighboring points is too large. These points may be obstructed points or points on the parallel beam generated by lidar scanning. Therefore, these defective points should be removed to reduce calibration errors.

[0090] In some embodiments of this application, after determining the first feature point, a feature homogenization operation is required to prevent the sampling of the first feature point from being overly concentrated in the same area. When a candidate point is selected as the first feature point, its surrounding L neighboring points are marked (L is a positive integer; for example, when L is 10, five points before and after it are marked). The marked neighboring points are not supported to be determined as the first feature point, so as to ensure that the distribution of the first feature point is relatively uniform.

[0091] Exemplary, in some embodiments of this application, the first threshold The value can be 0.1, and / or, the second threshold. The value can be 0.4.

[0092] It is understood that determining the first feature point in the first point cloud is a preprocessing method for point cloud data. In some embodiments of this application, the preprocessing of the first point cloud may also include removing point cloud distortion caused by vehicle motion based on vehicle speed information and the timestamp of each point in the first point cloud, ensuring the authenticity of the geometric position of the first feature point, improving calibration accuracy, reducing noise interference in the original yaw angle optimization process, and avoiding the yaw angle calibration result from getting trapped in a local optimum.

[0093] The method for calibrating the yaw angle mentioned in S305 above is described below. Figure 5 A flowchart illustrating a method for calibrating the yaw angle is shown. Figure 5 As shown, the method may include the following S501-S502.

[0094] S501: Calculate the first distance between the predicted position of each first feature point and the first plane corresponding to the first feature point, wherein the first plane corresponding to the first feature point is the plane determined by the three second feature points corresponding to the predicted position of the first feature point.

[0095] As can be seen from the above, each predicted position The three corresponding second feature points are respectively , , It's understandable that three points can determine a unique plane. (This refers to the first plane, also known as the first plane.) The normal vector corresponding to the first plane is... Then the distance from each predicted position to the first plane can be calculated. = .

[0096] S502: Based on the first distance corresponding to each first feature point, adjust the yaw angle to obtain the calibration result of the yaw angle.

[0097] As can be seen from the above, the distance from each predicted position to the first plane is... = Based on this, residuals are constructed. Using this residual as a constraint, optimize the original yaw angle in the original extrinsic parameters to minimize this residual. The value of the yaw angle is the optimal yaw angle, which is also the calibration result. This calibration result minimizes the sum of the squares of the distances from each predicted position to its corresponding first plane, thereby correcting the yaw angle error to the maximum extent and improving the accuracy of the calibration.

[0098] This application is not limited to the specific algorithm for optimizing the yaw angle in the process of reducing the above residual. For example, gradient-based optimization methods, gradient-free optimization methods, linear least squares simplification methods, etc. can be used, as long as the yaw angle that minimizes the above residual can be obtained in the iterative calculation process.

[0099] It should be noted that the above embodiments of this application calibrate the external parameters of the lidar by fixing the original pitch angle, original roll angle and original altitude in the original external parameters, and optimizing the original yaw angle to obtain the calibration result of the yaw angle. Similarly, any three original parameters can be fixed to optimize the remaining original parameter to obtain the calibration result of that original parameter. This application does not limit this.

[0100] As explained above regarding the extrinsic parameters of vehicle-mounted LiDAR, in addition to yaw angle, the extrinsic parameters of LiDAR can also include pitch angle, roll angle, and radar altitude. This application provides an alternative calibration method for these three parameters: pitch angle, roll angle, and altitude.

[0101] This method can be based on the following preconditions: that the road on which the vehicle travels is a smooth plane, that the road surface is perpendicular to the direction of gravity, and that the vehicle's inertial measurement unit (IMU) sensors have been accurately calibrated in advance. These preconditions will not be elaborated upon further in the following description.

[0102] Figure 6 A flowchart illustrating another method for calibrating the extrinsic parameters of a lidar system provided in this application is shown below. Figure 6 The method may include S601-S606:

[0103] S601: During vehicle operation, acquire the third point cloud collected by the vehicle's lidar at the third moment.

[0104] It is understandable that the third point cloud is also a collection of three-dimensional spatial points that can characterize the environment around the vehicle, collected by the vehicle-mounted lidar, and will not be elaborated on here.

[0105] S602: Select points from the third point cloud whose height is less than the third threshold to construct the fourth point cloud.

[0106] In some embodiments of this application, for the third point cloud Through the third threshold Filter the data to select those with a height less than the third threshold. The points are used to construct the fourth point cloud. :

[0107] ,in It is the third point cloud. The point in the middle.

[0108] For example, the third threshold can be 0.2m to 2m, such as 1m, 0.8m, 0.5m, 0.2m, etc.

[0109] S603: Obtain the vehicle's attitude information at the third moment.

[0110] In some embodiments of this application, the vehicle's attitude information at a third moment is obtained through the vehicle's three-dimensional odometer. The vehicle's attitude information may include the vehicle's pitch angle in the world coordinate system. and roll angle .

[0111] S604: Correct the initial position of the fourth point cloud based on the attitude information to obtain the corrected position of the fourth point cloud.

[0112] In some embodiments of this application, the vehicle's attitude information includes the vehicle's pitch angle. and roll angle Then, the initial position correction of the fourth point cloud based on the vehicle's attitude information can be determined according to the vehicle's pitch angle. and vehicle roll angle Determine the rotation matrix :

[0113] ,in,

[0114] , .

[0115] Based on rotation matrix Regarding the fourth point cloud The process of correcting the initial position to obtain the corrected position of the fourth point cloud is as follows:

[0116] For any point in the fourth point cloud The fourth cloud after correction .

[0117] It is understandable that when using vehicle-mounted LiDAR to collect point clouds, changes in vehicle attitude can cause systematic deviations in the initial position of the point cloud in space, especially when driving on complex terrain (such as slopes and curves). Therefore, this application's dynamic correction of point clouds based on vehicle attitude can improve calibration accuracy.

[0118] S605: Construct the second plane based on the corrected position of the fourth point cloud.

[0119] The fourth point cloud after correction For example, in some embodiments of this application, a second plane can be constructed based on a random sampling consensus algorithm to perform plane fitting.

[0120] The equation of the second plane is: .

[0121] S606: Based on the planar parameters of the second plane, the calibration results of the lidar's pitch angle, roll angle, and altitude are obtained.

[0122] From S605, we know that the plane equation of the second plane is: The unit normal vector of the second plane. .

[0123] The parameters (unit normal vector) contained in the plane equations based on these second planes ,intercept The height of the lidar can be calculated. The roll angle of the lidar The elevation angle of the lidar ,in .

[0124] It is understood that the calibration method provided in this application selects low-altitude point clouds as the basis for fitting the plane and combines the attitude information provided by the three-dimensional odometry for spatial correction, so that the calibration process can be embedded into the daily operation of the vehicle. It does not need to rely on a specific calibration field or artificially arranged target facilities. The external parameter calibration can be completed simply by the vehicle driving normally on a regular road. It supports continuous monitoring and dynamic updating of external parameter parameters, thereby improving the autonomy and robustness of the system.

[0125] Furthermore, it is understood that the embodiments of this application can calibrate radar extrinsic parameters based on the difference between the plane fitted by the low-altitude point cloud and the plane where the current road surface is located. In some embodiments, after performing coordinate transformation on the radar point cloud data based on the calibrated radar extrinsic parameters, the plane fitted based on the transformed point cloud data can be the plane where the current road surface is located. For example, when the current road surface is perpendicular to the direction of gravity, the plane where the current road surface is located is an ideal horizontal plane.

[0126] In some embodiments of this application, before correcting the initial position of the fourth point cloud based on attitude information in S604, the above-mentioned lidar extrinsic parameter calibration method further includes preprocessing of the fourth point cloud. Figure 7 A flowchart illustrating a preprocessing method for fourth point clouds provided in this application is shown below. Figure 7 The method may include S701-S704:

[0127] S701: Construct the third plane based on the fourth point cloud.

[0128] In some embodiments of this application, it is assumed that the ground is a planar model, and its general equation is: ,in It is the normal vector (unit vector) of the plane. The intercept is calculated using a random sampling consensus algorithm from the fourth point cloud. Estimate the plane parameters and unit normal vector of the third plane. and intercept This leads to the plane equation of the third plane: .

[0129] S702: Calculate the second distance from each point in the fourth point cloud to the third plane.

[0130] Regarding the fourth point cloud Every point in Calculate its distance to the third plane The second distance The calculation method for point-to-surface is as described in section S501 above, and will not be repeated here.

[0131] S703: Points whose second distance is greater than the fourth threshold are identified as invalid points in the fourth point cloud.

[0132] S704: Remove invalid points from the fourth point cloud.

[0133] In some embodiments of this application, for the fourth point cloud Through the fourth threshold After filtering, the updated fourth point cloud is obtained. : .

[0134] It's understandable that after removing invalid points, the subsequent processing of the fourth point cloud... Correction can be performed based on the updated fourth point cloud, which helps improve correction efficiency and the accuracy of subsequent calibration.

[0135] For example, the fourth threshold can be 0.005m to 0.05m, such as 0.01m, 0.02m, etc.

[0136] In summary, the targetless high-precision lidar extrinsic parameter calibration method provided in this application uses corner features for point cloud prediction and matching, and employs the distance from a point to a surface as a constraint. This improves matching accuracy while saving computational power and enhancing the robustness of the calibration method, which is beneficial for deployment in intelligent driving chips. This application also incorporates vehicle pose change information, eliminating bumps caused by uneven ground or aggressive driving. This reduces the operational requirements for calibration and the demands on the driver; it also improves the accuracy of ground modeling, thereby enhancing the accuracy of extrinsic parameter calibration and the robustness of the calibration process. Furthermore, this application separates the calibration of lidar extrinsic parameters, employing different calibration methods for different extrinsic parameters, which improves calibration accuracy.

[0137] In some embodiments, this application also provides a readable storage medium storing a program or instructions that, when executed on an electronic device, cause the electronic device to perform the calibration method for lidar extrinsic parameters described in the above embodiments.

[0138] In some embodiments, this application also provides a program product, including: a program or instructions, which, when run on an electronic device, cause the electronic device to perform the calibration method for lidar extrinsic parameters described in the above embodiments.

[0139] In some embodiments, this application also provides an in-vehicle electronic device, which includes: one or more processors; one or more memories; the one or more memories storing one or more programs, which, when executed by one or more processors, cause the in-vehicle electronic device to perform the calibration method for lidar extrinsic parameters described in the above embodiments.

[0140] Now for reference Figure 8The diagram shows a block diagram of an electronic device 800 according to some embodiments of this application. The electronic device 800 may include one or more processors 801 coupled to a controller hub 803. In at least one embodiment, the controller hub 803 communicates with the processor 801 via a multi-branch bus such as a Front Side Bus (FSB) or a point-to-point interface such as a QuickPath Interconnect (QPI). The processor 801 executes instructions controlling general types of data processing operations. In one embodiment, the controller hub 803 includes, but is not limited to, a Graphics & Memory Controller Hub (GMCH) (not shown) and an Input / Output Hub (IOH) (which may be on a separate chip) (not shown), wherein the GMCH includes memory and a graphics controller and is coupled to the IOH.

[0141] Electronic device 800 may also include a coprocessor 802 and a memory 804 coupled to a controller hub 803. Alternatively, one or both of the memory and GMCH may be integrated within the processor (as described in this application), with memory 804 and coprocessor 802 directly coupled to processor 801 and controller hub 803, which is on a single chip with IOH.

[0142] Memory 804 may be, for example, Dynamic Random Access Memory (DRAM), Phase Change Memory (PCM), or a combination of both. Memory 804 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. The computer-readable storage medium stores instructions, specifically, temporary and permanent copies of those instructions. These instructions may include: instructions that, when executed by at least one processor, cause electronic device 800 to implement the calibration method for lidar extrinsic parameters provided in the embodiments of this application. When the instructions are executed on a computer, they cause the computer to perform the calibration method for lidar extrinsic parameters provided in the embodiments of this application.

[0143] In one embodiment, the coprocessor 802 is a dedicated processor, such as, for example, a high-throughput many-integrated core (MIC) processor, a network or communication processor, a compression engine, a graphics processor, general-purpose computing on graphics processing units (GPGPU), or an embedded processor, etc. Optional properties of the coprocessor 802 are indicated by dashed lines. Figure 8 middle.

[0144] In one embodiment, electronic device 800 may further include a Network Interface Controller (NIC) 806. The network interface 806 may include a transceiver for providing a radio interface for electronic device 800 to communicate with any other suitable device, such as a front-end module, antenna, etc. In various embodiments, the network interface 806 may be integrated with other components of electronic device 800. The network interface 806 can implement the functions of the communication unit in the above embodiments.

[0145] Electronic device 800 may further include input / output (I / O) device 805. Input / output device 805 may include: a user interface designed to enable a user to interact with electronic device 800; a peripheral component interface designed to enable peripheral components to interact with electronic device 800; and / or sensors designed to determine environmental conditions and / or location information related to electronic device 800.

[0146] It is worth noting that, Figure 8 This is merely an example. That is, although... Figure 8 The electronic device 800 shown includes multiple devices such as a processor 801, a controller hub 803, and a memory 804. However, in practical applications, devices using the methods of this application may include only a portion of the devices in the electronic device 800. For example, it may include only the processor 801 and the network interface 806. Figure 8 The properties of the optional devices are shown by dashed lines.

[0147] Now for reference Figure 9 The diagram shown is a block diagram of a System-on-Chip (SoC) 900 according to some embodiments of this application. Figure 9 In the diagram, similar components share the same reference numerals. Additionally, dashed boxes are an optional feature for more advanced SoCs. Figure 9In this embodiment, SoC 900 includes: an interconnect unit 950 coupled to processor 910; a system proxy unit 980; a bus controller unit 990; an integrated memory controller unit 940; a group or one or more coprocessors 920, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random access memory (SRAM) unit 930; and a direct memory access (DMA) unit 960. In one embodiment, coprocessor 1920 includes a dedicated processor, such as, for example, a network or communication processor, a compression engine, general-purpose computing on graphics processing units (GPGPU), a high-throughput MIC processor, or an embedded processor.

[0148] Static Random Access Memory (SRAM) cell 930 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. The computer-readable storage medium stores instructions, specifically, temporary and permanent copies of the instructions. The instructions may include: instructions that, when executed by at least one processor, cause the SoC to implement the lidar extrinsic parameter calibration method disclosed in embodiments of this application. When the instructions are executed on a computer, they cause the computer to perform the lidar extrinsic parameter calibration method disclosed in embodiments of this application.

[0149] It is understood that, as used herein, the term “module” may refer to or include, or be part of, an application-specific integrated circuit (ASIC), electronic circuitry, a processor (shared, dedicated, or grouped) and / or memory that executes one or more software or firmware programs, combinational logic circuitry, and / or other suitable hardware components that provide the described functionality.

[0150] It is understood that in the various embodiments of this application, the processor may be a microprocessor, a digital signal processor, a microcontroller, etc., and / or any combination thereof. According to another aspect, the processor may be a single-core processor, a multi-core processor, etc., and / or any combination thereof.

[0151] The embodiments disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0152] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0153] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0154] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or via other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagated signals. Therefore, machine-readable media includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0155] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0156] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0157] It should be noted that in the examples and description of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0158] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made thereto without departing from the scope of this application.

Claims

1. A method for calibrating the extrinsic parameters of a lidar system, applied to vehicle-mounted electronic equipment, characterized in that, The method includes: During the vehicle's operation, the third point cloud collected by the vehicle's lidar at the third moment is acquired; From the third point cloud, select points with heights less than the third threshold to construct the fourth point cloud; Obtain the attitude information of the vehicle at the third moment; The initial position of the fourth point cloud is corrected based on the attitude information to obtain the corrected position of the fourth point cloud; Based on the corrected position of the fourth point cloud, a second plane is constructed; Based on the planar parameters of the second plane, the calibration results of the pitch angle, roll angle, and altitude of the lidar are obtained.

2. The method according to claim 1, characterized in that, Before correcting the initial position of the fourth point cloud based on the attitude information, the method further includes: Remove invalid points from the fourth point cloud; The method for determining invalid points in the fourth point cloud includes: Based on the fourth point cloud, construct the third plane; Calculate the second distance from each point in the fourth point cloud to the third plane; Points whose second distance is greater than the fourth threshold are identified as invalid points in the fourth point cloud.

3. The method according to claim 1, characterized in that, The attitude information includes the vehicle's pitch angle and roll angle; The step of correcting the initial position of the fourth point cloud based on the attitude information to obtain the corrected position of the fourth point cloud includes: The rotation matrix is ​​determined based on the vehicle's pitch angle and roll angle. Based on the rotation matrix, the initial position of the fourth point cloud is corrected to obtain the corrected position of the fourth point cloud.

4. The method according to claim 1, characterized in that, The construction of the second plane based on the corrected position of the fourth point cloud includes: Based on the corrected position of the fourth point cloud, the second plane is constructed using a random sampling consensus algorithm.

5. The method according to claim 1, characterized in that, The second plane is the plane where the vehicle is currently located on the road surface.

6. The method according to claim 1, characterized in that, The attitude information is obtained through the vehicle's three-dimensional odometer.

7. The method according to claim 2, characterized in that, The third threshold is 0.2m to 2m; and / or, the fourth threshold is 0.005m to 0.05m.

8. A vehicle-mounted electronic device, characterized in that, It includes one or more processors; one or more memories; the one or more memories storing one or more programs, which, when executed by the one or more processors, cause the vehicle-mounted electronic device to perform the calibration method for the extrinsic parameters of the lidar as described in any one of claims 1 to 7.

9. A computer-readable medium, characterized in that, The readable medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the calibration method for the extrinsic parameters of the lidar as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The program product includes computer instructions, which, when executed by an electronic device, enable the electronic device to perform the calibration method for the external parameters of the lidar as described in any one of claims 1 to 7.

Images