Parameter calibration method and device of laser radar, vehicle and storage medium

Abstract

The invention provides a parameter calibration method and device for a laser radar, a vehicle and a storage medium, and the method comprises the steps: obtaining first point cloud data collected at a previous moment and second point cloud data collected at a current moment in a driving environment of the vehicle based on the current parameter of the laser radar; based on a matching feature between the first point cloud data and the second point cloud data, determining an actual observation value of the matching feature in the second point cloud data; predicting a matching feature in the first point cloud data based on the motion information of the vehicle, and determining a predicted observation value of the matching feature at the current moment; under the condition that the difference value between the predicted observation value and the actual observation value exceeds a preset deviation range, performing minimum error solution on the actual observation value and the predicted observation value to obtain a target parameter estimation value of the laser radar; and calibrating the current parameter of the laser radar by using the target parameter estimation value. According to the invention, online calibration of the parameters of the laser radar can be realized.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a method, apparatus, vehicle, and computer-readable storage medium for calibrating parameters of a lidar. Background Technology

[0002] As a key sensor in autonomous driving systems, LiDAR (Light Detection and Ranging) constructs three-dimensional point cloud data of the environment by emitting laser beams and receiving reflected signals. Its calibration quality directly affects the accuracy and reliability of the perception system; therefore, maintaining the accuracy and stability of LiDAR parameters during vehicle operation is crucial.

[0003] In related technologies, calibration is typically performed before shipment or during after-sales maintenance, relying on manual operation and calibration equipment specific to certain scenarios. This type of method calculates and adjusts the parameters of the lidar by comparing the geometric relationship between the acquired point cloud and a known calibration board to achieve calibration.

[0004] However, these calibration methods are all offline and non-real-time. Due to factors such as bumps, collisions, and thermal expansion and contraction during long-term vehicle use, the parameters of the initial LiDAR calibration will change slowly, such as temperature drift or mechanical drift. The calibration methods in these technologies cannot detect and correct these changes in time, which may seriously affect the safety of autonomous driving. Summary of the Invention

[0005] To address the aforementioned issues, this application provides a method, apparatus, vehicle, and computer-readable storage medium for calibrating the parameters of a lidar, enabling online calibration of lidar parameters and timely correction of lidar parameters, thereby improving the safety of autonomous driving.

[0006] The technical solution of this application is implemented as follows: Firstly, this application provides a parameter calibration method for a lidar, the method comprising: Based on the current parameters of the lidar, acquire the first point cloud data collected by the vehicle in the driving environment at the previous moment and the second point cloud data collected at the current moment. Based on the matching features between the first point cloud data and the second point cloud data, determine the actual observed value of the matching features in the second point cloud data; Based on the vehicle's motion information, the matching features in the first point cloud data are predicted to determine the predicted observation value of the matching features at the current moment. If the difference between the predicted and actual observations exceeds the preset deviation range, the minimum error solution is performed on the predicted and actual observations to obtain the estimated target parameters of the lidar. The current parameters of the lidar are calibrated using the estimated target parameters.

[0007] Secondly, this application provides a parameter calibration device for a lidar, the parameter calibration device comprising an acquisition module, a determination module, a calculation module, and a calibration module, wherein: The acquisition module is configured to acquire the first point cloud data collected by the vehicle in the driving environment at the previous moment and the second point cloud data collected at the current moment, based on the current parameters of the LiDAR. The determination module is configured to determine the actual observed value of the matching feature in the second point cloud data based on the matching feature between the first point cloud data and the second point cloud data; and to predict the matching feature in the first point cloud data based on the vehicle's motion information to determine the predicted observed value of the matching feature at the current moment. The calculation module is configured to perform minimum error calculation on the predicted and actual observations when the difference between the predicted and actual observations exceeds a preset deviation range, thereby obtaining the estimated target parameters of the lidar. The calibration module is configured to calibrate the current parameters of the lidar using the target parameter estimates.

[0008] Thirdly, this application provides a vehicle including a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the program to implement the method steps provided in the first aspect above.

[0009] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method steps provided in the first aspect above.

[0010] This application provides a parameter calibration method, apparatus, vehicle, and computer storage medium for a lidar. The lidar parameter calibration method includes at least the following steps: acquiring first point cloud data collected by the vehicle in the driving environment at the previous moment and second point cloud data collected at the current moment, based on the current parameters of the lidar; determining the actual observed value of the matching feature in the second point cloud data based on the matching feature between the first and second point cloud data; predicting the matching feature in the first point cloud data based on the vehicle's motion information to determine the predicted observed value of the matching feature at the current moment; if the difference between the predicted and actual observed values ​​exceeds a preset deviation range, performing a minimum error calculation on the predicted and actual observed values ​​to obtain an estimated value of the lidar's target parameters; and calibrating the current parameters of the lidar using the estimated value of the target parameters. This application enables online calibration of lidar parameters.

[0011] In the solution provided in this application, point cloud data of the lidar at adjacent time points under the current parameters are acquired and feature matching is performed to obtain the actual observed value of the matching feature in the point cloud data at the current time. That is, this actual observed value is an actual value measured by the lidar under the current parameters. Then, based on the vehicle's motion information, the position of the matching feature in the first point cloud data at the current time is predicted under the condition that the lidar's current parameters are completely accurate, thus obtaining the predicted observed value. For the same matching feature, it can be understood that if the difference between the predicted observed value and the actual observed value of the matching feature is within a preset deviation range, it indicates that the lidar's current parameters are correct. If the difference between the predicted observed value and the actual observed value of the matching feature exceeds the preset deviation range, it indicates that the lidar's current parameters have changed and need to be calibrated. The target parameter estimate of the lidar can be obtained by solving for the minimum error between the predicted and actual observed values. The current parameters of the lidar are then calibrated using the target parameter estimate. The solution provided in this application, on the one hand, transforms the calibration method of LiDAR parameters from offline and periodic operations to an online and real-time process, enabling timely and automatic compensation for parameter drift, avoiding silent degradation of the perception system, and improving the safety and reliability of autonomous driving; on the other hand, it eliminates the need for manual labor or specific scenarios, enabling real-time calibration during normal vehicle operation, thereby effectively compensating for parameter drift caused by bumps, thermal expansion and contraction, and improving the accuracy and timeliness of LiDAR calibration; simultaneously, the matching features include natural features such as ground markings (e.g., arrows and zebra crossings), lamppost corners, building wall edges, and traffic sign edges. By utilizing rich natural features in the calibration process, the high convenience and automation of calibration can be demonstrated, significantly reducing the manpower, time, and site costs required by traditional calibration methods, thus removing obstacles to the large-scale deployment of autonomous driving. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating a parameter calibration method for a lidar provided in an embodiment of this application. Figure 1 ; Figure 2 This is a flowchart illustrating a parameter calibration method for a lidar provided in an embodiment of this application. Figure 2 ; Figure 3 This is a schematic diagram of the structure of a parameter calibration system for a lidar provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a parameter calibration device for a lidar provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a vehicle provided in an embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0015] In the following description, references to "some embodiments," "this embodiment," "this application embodiment," and examples, etc., describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments and may be combined with each other without conflict.

[0016] The descriptions such as "first," "second," and "third" appearing in the embodiments of this application do not have a specific meaning (such as no order, nor do they indicate a special limitation on the number of devices in the embodiments of this application), but are merely for the purpose of clearly describing the embodiments of this application and do not constitute any limitation on the embodiments of this application.

[0017] Before providing a more detailed description of the embodiments of this application, the nouns and terms that may be involved in the embodiments of this application will be explained. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0018] Inertial Measurement Unit (IMU): Used to measure angular velocity and acceleration along three axes (X-axis, Y-axis, and Z-axis).

[0019] Global Navigation Satellite System (GNSS): Used to receive absolute position information from satellite signals.

[0020] Telematics Box (TBOX): Also known as a telematics control unit, it is mainly responsible for communication between the vehicle and the cloud, mobile phones, etc., to realize functions such as remote control and safety monitoring.

[0021] Human Machine Interface (HMI): It is the core medium for drivers to interact with vehicle systems. It transmits vehicle information and receives user commands through screens, voice, touch, and other means, thereby improving driving safety and user experience.

[0022] Vehicle Control Unit (VCU): Also known as the vehicle control unit, it is the core control module in a car, equivalent to the "brain" of the car, responsible for key functions such as power coordination, energy management, and driving mode switching.

[0023] Telematics Service Provider (TSP): Responsible for integrating resources from automakers, equipment suppliers, and network operators to provide car owners with comprehensive solutions such as navigation, remote control, and safety services.

[0024] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies or terms of the embodiments of this application are described below. The following relevant technologies or terms are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0025] Based on this, in order to solve the problem that the parameter calibration methods of lidar in the related technologies proposed in the background art cannot achieve online calibration and face the problem of not being able to correct the lidar parameters in a timely manner, the embodiments of this application provide the following lidar parameter calibration methods, devices, vehicles and computer-readable storage media, etc.

[0026] Figure 1 This is a flowchart illustrating a parameter calibration method for a lidar provided in an embodiment of this application. Figure 1 .like Figure 1 As shown, the parameter calibration method for lidar provided in this application includes the following steps: S101. Based on the current parameters of the lidar, acquire the first point cloud data collected by the vehicle in the driving environment at the previous moment and the second point cloud data collected at the current moment.

[0027] In one example, the current parameters of the LiDAR can be its external parameters, also known simply as "external parameters". The external parameters of the LiDAR reflect the transformation relationship between the "radar coordinate system" and the "vehicle coordinate system", precisely defining the position and angle of the LiDAR relative to a fixed coordinate system of the vehicle (usually with the center of the rear axle of the vehicle as the origin).

[0028] The external parameters of a lidar include two parts: position (Translation Vector) and angle (Rotation Matrix R). The position includes the x-axis, y-axis, and z-axis, which can be represented by the coordinate system [x, y, z]. The angle includes pitch, roll, and yaw.

[0029] Understandably, based on the current parameters of the LiDAR, point cloud data streams of the vehicle in the driving environment are continuously collected, forming a temporal frame queue. In this embodiment, the temporal frame queue can be maintained in real time, and the point cloud data of the previous frame and the point cloud data of the current frame can be extracted and correlated.

[0030] The previous frame refers to the previous moment, and the point cloud data of the previous frame is the first point cloud data; the current frame refers to the current moment, and the point cloud data of the current frame is the second point cloud data.

[0031] S102. Based on the matching features between the first point cloud data and the second point cloud data, determine the actual observed value of the matching features in the second point cloud data.

[0032] In one embodiment, the acquired first and second point cloud data can be applied directly without processing; in another embodiment, the acquired first and second point cloud data can be preprocessed before application. This application does not impose any particular limitation on this approach. The following explanation will use the example of preprocessing the acquired first and second point cloud data before application to illustrate how to improve the quality of point cloud data and reduce errors.

[0033] For example, the preprocessing of the acquired first point cloud data and second point cloud data may include operations such as filtering, denoising, and motion distortion removal.

[0034] In one embodiment, preset features can be extracted from the first point cloud data and the second point cloud data. The preset features are pre-defined features, which are matching features in the two point cloud data. By performing association matching on the preset features extracted from the two point cloud data, cross-frame tracking of the same feature can be achieved.

[0035] It is understood that there may be one or more preset features, and this application does not make any special restrictions on this.

[0036] If there is only one preset feature, during extraction, if one frame of two point cloud data has the preset feature but the other frame does not, the point cloud data can be re-acquired.

[0037] When there are multiple preset features, during extraction, one preset feature can be selected as the target feature from the multiple preset features. Check whether the target feature is present in both frames of point cloud data. If it is present in both frames, then the target feature is the matching feature in the two frames of point cloud data. If one frame of two frames of point cloud data has the target feature but the other frame does not, a new preset feature can be selected as the new target feature from the multiple preset features. If all preset features have been exhausted and the situation still exists where one frame of two frames of point cloud data has the target feature but the other frame does not, then the point cloud data can be acquired again.

[0038] In another embodiment, stable and unique natural features can be extracted from the first point cloud data and the second point cloud data. These stable and unique natural features are the matching features in the two point cloud data. By performing correlation matching on the stable and unique natural features extracted from the two point cloud data, cross-frame tracking of the same natural feature can be achieved.

[0039] It is understandable that there may be one or more stable and unique natural features, and this application does not make any particular limitation on this.

[0040] If there is only one stable and unique natural feature, during extraction, if one frame of two point cloud data has a stable and unique natural feature while the other frame does not, the point cloud data can be re-acquired.

[0041] When there are multiple stable and unique natural features, during extraction, one stable and unique natural feature can be selected as the specific feature from the multiple stable and unique natural features. Check whether the specific feature is present in both frames of point cloud data. If it is present in both frames, then the specific feature is the matching feature in the two frames of point cloud data. If one frame of two frame of point cloud data has the specific feature but the other frame does not, a new stable and unique natural feature can be selected as the new specific feature from the multiple stable and unique natural features. If all stable and unique natural features have been exhausted and the situation still exists where one frame of two frame of point cloud data has the specific feature but the other frame does not, then the point cloud data can be acquired again.

[0042] It is understood that, in another embodiment, the preset feature may be a stable and unique natural feature.

[0043] For example, stable and unique natural features may include, but are not limited to: ground markings (such as arrows and zebra crossings), lamppost corners, building wall edges, and traffic sign edges. That is, matching features can be natural features such as ground markings (such as arrows and zebra crossings), lamppost corners, building wall edges, and traffic sign edges.

[0044] Understandably, utilizing rich natural features in the calibration process can demonstrate the high convenience and automation of calibration, significantly reducing the manpower, time, and site costs required by traditional calibration methods, thus removing obstacles to the large-scale deployment of autonomous driving.

[0045] This application does not impose any particular limitation on whether the preset feature is a stable and unique natural feature. The following explanation will take the example of a preset feature being a stable and unique natural feature.

[0046] S103. Based on the vehicle's motion information, predict the matching features in the first point cloud data and determine the predicted observation value of the matching features at the current moment.

[0047] In one embodiment, the vehicle's motion information can be collected by a high-precision integrated navigation system to obtain the vehicle's motion information provided by the high-precision integrated navigation system.

[0048] For example, the vehicle's motion information may include the vehicle's rotation matrix (R_vehicle) and translation vector (t_vehicle).

[0049] Understandably, it is necessary to obtain vehicle motion information multiple times consecutively between the time when the first point cloud data is obtained and the time when the second point cloud data is obtained. Only in this way can the vehicle's trajectory during this period be obtained, and the predicted observation value of the matching feature at the current time be predicted, thereby ensuring the real-time performance and accuracy of the prediction.

[0050] In one embodiment, matching features in the first point cloud data are predicted based on the vehicle's motion information. The prediction determines the position of the matching feature observed in the previous moment, given that the current parameters of the LiDAR are the original parameters. The original parameters are the parameters of the LiDAR that were calibrated before leaving the factory and are completely accurate (i.e., no drift has occurred).

[0051] In another embodiment, the matching features in the first point cloud data are predicted based on the vehicle's motion information. The prediction determines the position of the matching features observed in the previous moment at the current moment, assuming that the current parameters of the lidar are the parameters after the last online calibration.

[0052] This application does not specifically limit whether the determined matching feature is based on the current parameters of the LiDAR as the original parameters or the parameters after the previous online calibration. Instead, it exemplifies how the determined matching feature should appear at the current moment if the current parameters of the LiDAR are the original parameters. This is to ensure that when the current parameters of the LiDAR need calibration, the calibrated LiDAR parameters are identical to the fully accurate parameters calibrated before leaving the factory, thereby reducing unexpected downtime and vehicle return-to-factory repairs due to sensor malfunctions and lowering the maintenance costs of autonomous vehicles.

[0053] S104. When the difference between the predicted and actual observations exceeds the preset deviation range, the minimum error is calculated between the predicted and actual observations to obtain the estimated target parameters of the lidar.

[0054] In one example, for the same matching feature, if the predicted and actual observed values ​​of the matching feature are the same, it means that the current parameters of the lidar are correct; if the predicted and actual observed values ​​of the matching feature are different, it means that the current parameters of the lidar have changed and need to be calibrated.

[0055] Understandably, in practical applications, the question of whether the predicted and actual observations are the same is usually measured by whether the difference between the predicted and actual observations is within the preset deviation range. Therefore, the preset deviation range can be increased.

[0056] In another example, for the same matching feature, if the difference between the predicted and actual observed values ​​of the matching feature is within a preset deviation range, it indicates that the current parameters of the lidar are correct; if the difference between the predicted and actual observed values ​​of the matching feature exceeds the preset deviation range, it indicates that the current parameters of the lidar have changed and need to be calibrated.

[0057] This application does not specifically limit the criteria for determining whether calibration is required. The following example uses the addition of a preset deviation range as a criterion for illustrative purposes.

[0058] It is understandable that the difference between the predicted and actual observations is the reprojection error. In one embodiment, a prediction algorithm can be used to solve for the minimum error between the predicted and actual observations to obtain the target parameter estimate of the lidar, which can also be called the "optimal extrinsic parameter estimate of the lidar".

[0059] The prediction algorithm can be the least squares method. Furthermore, the least squares method can be a nonlinear least squares method.

[0060] For example, specifically, a nonlinear least squares optimization problem is constructed with the goal of finding an optimal set of LiDAR parameters to minimize the total error between the predicted and actual values ​​of all feature point pairs matching the features. This problem is solved through an iterative optimization algorithm, and the final output is the target parameter estimate (i.e., the optimal extrinsic parameter estimate) at the current time.

[0061] S105. Use the target parameter estimate to calibrate the current parameters of the lidar.

[0062] In one embodiment, after obtaining the target parameter estimate, it can be directly used to calibrate the current parameters of the LiDAR. In another embodiment, after obtaining the target parameter estimate, its health status can be assessed first. If the health status assessment is passed, the target parameter estimate is then used to calibrate the current parameters of the LiDAR. If the health status assessment is not passed, the current parameters of the LiDAR are not calibrated temporarily; instead, fault prediction is performed on the LiDAR. This application does not impose any particular limitation on this approach. The following example of assessing the health status after obtaining the target parameter estimate is provided to illustrate this point and reduce interference caused by low-quality or abnormal estimates. (Refer to...) Figure 2 .

[0063] Figure 2 This is a flowchart illustrating a parameter calibration method for a lidar provided in an embodiment of this application. Figure 2 ,like Figure 2 As shown, calibrating the current parameters of the lidar using the estimated target parameters includes the following steps: S105A. Assess the health of the target parameter estimates and determine the health assessment results of the target parameter estimates.

[0064] In one embodiment, confidence level can be used to assess the health of the target parameter estimate; in another embodiment, stability can be used to assess the health of the target parameter estimate; in yet another embodiment, confidence level and stability can be combined to assess the health of the target parameter estimate. This application does not impose any particular limitation on these methods. The three implementation methods are described in detail below, with the following example illustrating the assessment of the health of the target parameter estimate by combining confidence level and stability.

[0065] When using confidence levels to assess the health of estimated target parameters: For example, the confidence level of the target parameter estimate can be determined first; then, if the confidence level of the target parameter estimate is greater than or equal to the preset confidence level, the health assessment result can be determined to indicate that the target parameter estimate passes the health assessment; if the confidence level of the target parameter estimate is less than the preset confidence level, the health assessment result can be determined to indicate that the target parameter estimate fails the health assessment.

[0066] The confidence level can be the matching error, the number of feature points involved in the matching, etc., and this application does not impose any special restrictions on it.

[0067] It is understandable that when the confidence level is the matching error, the preset confidence level is the preset matching error; when the confidence level is the number of feature points participating in the matching, the preset confidence level is the preset number of feature points participating in the matching. In other words, the type of preset confidence level needs to be consistent with the type of the calculated confidence level.

[0068] Understandably, using a pre-set confidence level as a health assessment standard allows for a quantitative assessment of the reliability of the target parameter estimates. When the confidence level of the target parameter estimate is lower than the pre-set confidence level, it indicates that the obtained target parameter estimate may have a large error or anomaly. In this case, the obtained target parameter estimate will not be used to calibrate the current parameters of the lidar, thereby preventing safety hazards caused by the introduction of erroneous parameters.

[0069] It should be noted that if the estimated target parameters obtained in this study have large errors or anomalies, it may mean that the LiDAR has malfunctioned. Fault type can be predicted to provide early warning before the LiDAR performance degrades significantly or fails completely, enabling predictive maintenance and further improving the safety of the autonomous driving system.

[0070] When using stability to assess the health of the estimated target parameters: For example, the difference between the target parameter estimate and the historical target parameter estimate can be determined first; if the difference is within a preset range, the health assessment result indicates that the target parameter estimate has passed the health assessment; if the difference is outside the set range, the health assessment result indicates that the target parameter estimate has failed the health assessment.

[0071] Understandably, if the difference between the estimated target parameter and the historical target parameter is within a preset range, it indicates that the estimated target parameter obtained this time is relatively stable (or reliable); if the difference between the estimated target parameter and the historical target parameter is outside the preset range, it indicates that the estimated target parameter obtained this time is unstable (or unreliable).

[0072] The historical target parameter estimates can be the target parameter estimates obtained from the previous health assessment, or they can be the fully accurate parameters calibrated by the lidar before leaving the factory, etc. This application does not make any special restrictions on them.

[0073] In one embodiment, the difference between the target parameter estimate and the historical target parameter estimate can be obtained by directly subtracting the historical target parameter estimate from the target parameter estimate, or by subtracting the target parameter estimate from the historical target parameter estimate.

[0074] In another embodiment, the difference between the target parameter estimate and the historical target parameter estimate can be the variance of the fluctuation between the target parameter estimate and the historical target parameter estimate.

[0075] This application does not specifically limit the type of the difference between the estimated target parameter and the historical estimated target parameter. The following example illustrates this by taking the difference between the estimated target parameter and the historical estimated target parameter as the variance of fluctuation.

[0076] Understandably, by comparing the difference between the current estimate and historical estimates with a preset range, the reliability of the obtained target parameter estimates can be determined. If the difference is within the preset range, the obtained target parameter estimate is relatively stable / reliable, and can be used to update the current parameters of the LiDAR. If the difference is outside the preset range, the obtained target parameter estimate is unstable / unreliable, and will not be used to update the current parameters of the LiDAR, thus preventing safety hazards caused by erroneous parameters.

[0077] It should be noted that if the estimated target parameters obtained in this case are unstable or unreliable, it may mean that the LiDAR has malfunctioned. Fault type can be predicted to provide early warning before the LiDAR performance significantly degrades or completely fails, enabling predictive maintenance and further improving the safety of the autonomous driving system.

[0078] When combining confidence and stability to assess the health of the target parameter estimates: For example, if the confidence level of the target parameter estimate is greater than or equal to a preset confidence level and the difference between the target parameter estimate and the historical target parameter estimate is within a preset range, the health assessment result indicates that the target parameter estimate has passed the health assessment; if either of these conditions is met, the health assessment result indicates that the target parameter estimate has not passed the health assessment.

[0079] Understandably, adopting a scheme that combines confidence and stability to assess the health of the target parameter estimates can serve as a double verification, preventing misjudgments that occur when only one verification method is used.

[0080] S105B: If the health assessment results indicate that the target parameter estimate passes the health assessment, the current parameters of the lidar are calibrated using the target parameter estimate.

[0081] Understandably, adding a health assessment step before calibration ensures that only target parameter estimates that have undergone health assessment are used to update the current parameters of the LiDAR. This mechanism avoids interference caused by low-quality or outlier estimates, improves the robustness of the calibration process, and further guarantees the stability of the autonomous driving system.

[0082] S105C: When the estimated value of the target parameter indicated by the health assessment result fails the health assessment, a fault prediction is made for the lidar.

[0083] In one embodiment, fault prediction for the lidar can be performed based on at least one of the following judgment strategies.

[0084] The judgment strategy may include: Strategy 1: Based on whether multiple current parameters obtained within a first preset time period have undergone linear drift.

[0085] That is, if multiple current parameters obtained within a first preset time period exhibit linear drift, a level one fault is determined to have occurred in the lidar.

[0086] Specifically, within a first preset time period, the current parameters of the lidar are obtained multiple times through the sensor. The current parameters (i.e., current external parameters) of multiple lidars are sorted according to the order of acquisition time. The current parameters of the lidar at the next moment are subtracted from the current parameters of the lidar at the previous moment, and so on, traversing all the current parameters of the lidars. It is observed whether the current parameters of the lidars undergo a slow and continuous linear drift. If this occurs, it is determined that the lidar has experienced a first-level fault.

[0087] It's important to note that linear drift in the external parameters of a LiDAR system refers to a slow but continuous relative displacement or rotational deviation between the LiDAR's coordinate system and the vehicle's coordinate system. This drift can lead to spatial positional discrepancies during multi-sensor data fusion, affecting positioning accuracy and obstacle detection precision. Linear drift primarily manifests as a continuous relative offset between the two coordinate systems along the X / Y / Z axes, or a slow rotational deviation around the vertical or horizontal axes. For example, if the LiDAR's installation location is tilted or vibrates, its coordinate system may gradually deviate from its initial alignment with the vehicle's coordinate system during driving.

[0088] For example, a Level 1 failure of a lidar could indicate that the lidar's support is slowly deforming due to material fatigue.

[0089] Strategy 2: Based on whether the confidence level of the multiple target parameter estimates obtained within the second preset time period is less than the preset confidence level.

[0090] That is, if the confidence level of multiple target parameter estimates obtained within the second preset time period is less than the preset confidence level, it is determined that the lidar has experienced a level two fault.

[0091] Specifically, within the second preset time period, target parameter estimates are obtained multiple times, and the confidence level of each target parameter estimate is calculated; if the confidence level of all target parameter estimates is less than the preset confidence level within the second preset time period, it is determined that the lidar has experienced a level two fault.

[0092] For example, a level 2 failure in a lidar could indicate that the internal mirrors of the lidar are dirty, icy, or that the scanning mechanism is degraded.

[0093] Strategy 3: Whether at least one axial parameter in the target parameter estimate undergoes a step change.

[0094] That is, if at least one axial parameter in the target parameter estimate undergoes a step change, the lidar is determined to have a level three fault.

[0095] Specifically, at least one axial parameter of the lidar can be acquired by a sensor. The same axial parameter in the target parameter estimate that is the same as that acquired by the lidar by the sensor is compared. If there is a step change in the same axial parameter in the target parameter estimate that is the same as that acquired by the lidar by the sensor, it is determined that the lidar has a level three fault.

[0096] For example, the current Z-axis pitch angle of the lidar is collected by the sensor and compared with the Z-axis pitch angle in the target parameter estimate. If the Z-axis pitch angle in the target parameter estimate changes abruptly, it is determined that the lidar has a level three fault.

[0097] For example, a level 3 fault in a lidar could indicate that the lidar's mounting bracket has been impacted or loosened.

[0098] Among them, the danger level of a Level 1 fault is lower than that of a Level 2 fault, and the danger level of a Level 2 fault is lower than that of a Level 3 fault.

[0099] It should be noted that the above three strategies can be implemented simultaneously, or strategy one can be executed first. After executing strategy one and confirming that the LiDAR has not experienced a level-one fault, strategy two can be executed. After executing strategy two and confirming that the LiDAR has not experienced a level-two fault, strategy three can be executed to determine whether the LiDAR has experienced a level-three fault. This application does not specifically limit the execution method of the LiDAR fault prediction strategy. The following explanation uses the simultaneous execution of the above three strategies as an example to improve the diagnostic speed and achieve "early detection, early diagnosis, and early treatment".

[0100] Understandably, by analyzing parameter change patterns, potential fault signs can be detected in advance before the performance of lidar deteriorates significantly or fails, providing a basis for subsequent early warning and maintenance decisions, helping to achieve predictive maintenance and reduce the risks caused by sudden failures.

[0101] Understandably, by defining different levels of fault judgment criteria and corresponding to different levels of danger, the health status of lidar can be assessed more accurately, enhancing intelligent diagnostic capabilities.

[0102] To improve the safety of autonomous driving systems, the parameter calibration method for lidar provided in this application embodiment can also implement corresponding early warning measures after predicting the lidar.

[0103] As an optional embodiment, the parameter calibration method for lidar provided in this application embodiment further includes: issuing an early warning based on the fault level of the lidar when it is determined that the lidar has malfunctioned.

[0104] In some embodiments, when the fault level is a Level 1 fault, the warning includes updating the preset confidence level.

[0105] For example, updating the preset reliability can be done by lowering the preset reliability.

[0106] It should be noted that when the conditions for whether the next target parameter estimate passes the health assessment include a pre-set reliability, the updated pre-set reliability is used as a condition for whether the next target parameter estimate passes the health assessment. However, the updated pre-set reliability is not used as a condition for whether the current target parameter estimate passes the health assessment. The condition for whether the current target parameter estimate passes the health assessment remains the pre-set reliability before the update, or stability, or a combination of the pre-set reliability and stability before the update.

[0107] In some embodiments, when the fault level is a level 2 fault, the alert includes pushing a security reminder to the cloud server and / or the user.

[0108] For example, the safety alert could read, "The lidar calibration parameters are abnormal; it is recommended to schedule an inspection."

[0109] In some embodiments, when the fault level is three, the warning includes prompting the driver to take over the vehicle or stop safely.

[0110] For example, the vehicle's actuators can be controlled via the VCU to bring the vehicle to a safe stop. The actuators may be brake pads.

[0111] Understandably, a tiered early warning mechanism allows for corresponding response measures based on the severity of the fault, thereby maximizing driving safety without affecting the driving experience. For example, minor issues can be resolved by adjusting preset confidence levels, while serious issues require driver intervention or automatic safe stopping procedures to prevent potential accidents.

[0112] It is understood that the parameter calibration method for LiDAR provided in this application obtains point cloud data of the LiDAR at adjacent times under the current parameters and performs feature matching to obtain the actual observed value of the matching feature in the point cloud data at the current time. That is, this actual observed value is an actual value measured by the LiDAR under the current parameters. Then, based on the vehicle's motion information, the position of the matching feature in the first point cloud data at the current time is predicted under the condition that the current parameters of the LiDAR are completely accurate, and a predicted observed value can be obtained. For the same matching feature, if the difference between the predicted observed value and the actual observed value of the matching feature is within a preset deviation range, it indicates that the current parameters of the LiDAR are correct. If the difference between the predicted observed value and the actual observed value of the matching feature exceeds the preset deviation range, it indicates that the current parameters of the LiDAR have changed and need to be calibrated. The target parameter estimate of the LiDAR can be obtained by solving the minimum error between the predicted observed value and the actual observed value. The current parameters of the LiDAR are then calibrated using the target parameter estimate. The solution provided in this application, on the one hand, transforms the calibration method of LiDAR parameters from offline and periodic operations to an online and real-time process, enabling timely and automatic compensation for parameter drift, avoiding silent degradation of the perception system, and improving the safety and reliability of autonomous driving; on the other hand, it eliminates the need for manual labor or specific scenarios, enabling real-time calibration during normal vehicle operation, thereby effectively compensating for parameter drift caused by bumps, thermal expansion and contraction, and improving the accuracy and timeliness of LiDAR calibration; simultaneously, the matching features include natural features such as ground markings (e.g., arrows and zebra crossings), lamppost corners, building wall edges, and traffic sign edges. By utilizing rich natural features in the calibration process, the high convenience and automation of calibration can be demonstrated, significantly reducing the manpower, time, and site costs required by traditional calibration methods, thus removing obstacles to the large-scale deployment of autonomous driving.

[0113] The following examples illustrate possible implementation schemes for the cross-cloud resource processing method and the prediction model training method described in one or more of the above embodiments.

[0114] This embodiment belongs to the field of intelligent vehicle technology, specifically involving autonomous driving sensing technology, sensor calibration, fault diagnosis and predictive maintenance, and in particular a system and method for real-time online calibration of lidar during vehicle operation and early fault diagnosis based on calibration data.

[0115] In related technologies, on the one hand, LiDAR calibration suffers from timing lag. Specifically, LiDAR parameter calibration is typically performed offline, not in real-time, only before delivery or during after-sales maintenance. Due to factors like bumps, collisions, and thermal expansion and contraction during long-term vehicle use, the calibrated parameters will slowly change (e.g., temperature drift and mechanical drift), and these changes cannot be detected and corrected in a timely manner. This leads to late fault detection: In related technologies, the vehicle's autonomous driving system usually only reports errors when the LiDAR suffers severe failure (e.g., significant point cloud loss, dense noise), failing to provide early warnings of performance degradation (e.g., gradual decrease in LiDAR accuracy, slight field-of-view shift). On the other hand, it relies on manual labor and specific scenarios. Specifically, LiDAR calibration must be performed in a dedicated calibration workshop, relying on calibration boards and specialized tools, and is done manually, resulting in low efficiency and unsuitability for future large-scale mass production needs. Finally, perception performance suffers from silent degradation. Among them, calibration misalignment can lead to silent degradation of perception algorithm performance, manifested as decreased positioning accuracy and false / missed target detection, but the autonomous driving system does not perceive it, thus creating potential safety hazards.

[0116] This embodiment aims to address the aforementioned pain points by providing an online, automatic, and real-time LiDAR calibration and fault diagnosis solution. It can automatically complete calibration parameter verification, optimization, and early fault warning without manual intervention during normal vehicle operation, thereby improving the safety and reliability of the autonomous driving system.

[0117] In general, this embodiment adopts the following technical solution: using natural features that are stable in the daily driving environment of a vehicle (such as street lamp poles, traffic signs, building corners, etc.) as natural calibration references, by comparing the difference between the feature positions observed by the lidar (i.e., actual observed values) and the expected positions (i.e., predicted observed values), the optimal external parameter estimate of the lidar (i.e., target parameter estimate) is calculated in reverse optimization, and early fault diagnosis and predictive maintenance are achieved based on the change pattern of the external parameters.

[0118] Among them, the external parameters are used to represent the transformation relationship between the radar coordinate system and the vehicle coordinate system. They precisely define the position and angle of the lidar relative to a fixed coordinate system of the vehicle (usually with the rear axle center of the vehicle as the origin). They include two parts: position (Translation Vector [x, y, z]) and angle (Rotation Matrix R, including pitch, roll and yaw).

[0119] The parameter calibration system and method for the lidar in this embodiment are described below. (For reference...) Figure 3 .

[0120] Figure 3This is a schematic diagram of the structure of a parameter calibration system for a lidar provided in an embodiment of this application, as shown below. Figure 3 As shown, the parameter calibration system of the lidar includes a sensing device 301, a controller 302, and an execution interaction device 303.

[0121] In this embodiment, the sensing device 301 is located in the sensing layer; the controller 302 is located in the Autonomous Driving Control (ADC); and the execution interaction device 303 is located in the execution and interaction layer.

[0122] For example, the sensing device 301 may include a lidar 301A, an inertial measurement unit 301B, a global navigation satellite system 301C, and a wheel speed sensor 301D.

[0123] For example, the controller 302 may include a data processing module 302A, a feature extraction and matching module 302B, an online calibration calculation module 302C, a health assessment and diagnosis module 30D, a decision and execution module 302E, and a perception fusion algorithm module 302F.

[0124] For example, the interactive device 303 may include a telematics box 303A, a human-machine interface 303B, and an on-board controller 303C.

[0125] The data processing module 501 can receive data collected by the sensing device 301 and preprocess the data collected by the sensing device 301.

[0126] In this embodiment, the lidar 301A in the sensing device 301 is used to collect point cloud data of the vehicle's driving environment, and can output the data to the data processing module 302A via the vehicle Ethernet.

[0127] In this embodiment, the inertial measurement unit 301B in the sensing device 301 is used to collect vehicle motion data (i.e., vehicle motion information), including measuring triaxial angular velocity and acceleration. It can be output to the data processing module 302A through the Controller Area Network (CAN) bus or the CAN with Flexible Data-Rate (CAN FD) bus.

[0128] In this embodiment, the Global Navigation Satellite System 301C in the sensing device 301 is used to receive the absolute position (i.e., positioning data) provided by satellite signals, and can output it to the data processing module 302A via CAN bus or CAN FD bus.

[0129] In this embodiment, the wheel speed sensor 301D in the sensing device 301 is used to collect wheel speed signals. It can monitor wheel rotation through magnetoelectricity or Hall effect to calculate the travel distance. It can output to the data processing module 302A through CAN bus or CAN FD bus.

[0130] The data processing module 302A can perform the above-mentioned raw data acquisition, data synchronization and point cloud preprocessing.

[0131] In this embodiment, data synchronization is achieved by assigning a unified timestamp to all input data, ensuring that the instant the LiDAR 301A scans a frame of the environment corresponds strictly to the instant the vehicle attitude is recorded by the IMU, thus eliminating calculation errors caused by time asynchrony.

[0132] In this embodiment, point cloud preprocessing involves filtering, denoising, and removing motion distortion from the point cloud data uploaded by the LiDAR 301A to improve the quality of the point cloud data.

[0133] The feature extraction and matching module 302B can automatically extract stable and unique natural features from the point cloud data at the current moment, such as ground markings (arrows, zebra crossings), lamppost corners, building wall edges, and traffic sign edges. It then associates and matches the extracted features with reference features from adjacent frames (i.e., the previous moment) to achieve cross-frame tracking of the same feature, forming a feature motion trajectory.

[0134] It should be noted that the reference features of adjacent frames are the same as the stable and unique natural features extracted from the point cloud data at the current moment.

[0135] Among them, the online calibration calculation module 302C is the core of calibration parameter calculation. It can match the features extracted from corresponding adjacent point cloud frames based on the vehicle's own high-precision integrated navigation system data. By minimizing the matching error, it back-optimizes and calculates the optimal external parameter estimate (i.e., target parameter estimate) of the lidar relative to the vehicle coordinate system.

[0136] Specifically, it includes the following steps: Motion prediction: Based on the vehicle's own motion information provided by the high-precision integrated navigation system (i.e., by continuously acquiring the vehicle's own motion information multiple times), the vehicle's trajectory can be obtained. This allows for the prediction of the position (i.e., the predicted value, or the predicted observation value) of the feature points observed in the previous frame when the original lidar extrinsic parameters [R|t] are completely accurate. The vehicle's own motion information includes the vehicle's rotation matrix (R_vehicle) and translation vector (t_vehicle).

[0137] When the original lidar extrinsic parameters are completely accurate, it means that the current extrinsic parameters (i.e., current parameters) of the lidar are the completely accurate extrinsic parameters that were calibrated before the lidar left the factory.

[0138] Actual observation: Through feature matching, the actual observed coordinates (i.e., the observed value, or actual observation value) of the feature point in the current frame point cloud are obtained.

[0139] The optimization problem is constructed as follows: the difference between the predicted and observed values ​​is the reprojection error. The parameter calibration system of the lidar in this embodiment can construct a nonlinear least squares optimization problem, with the goal of finding a set of optimal lidar extrinsic parameters [R|t] to minimize the total error between the predicted and observed values ​​of all matching feature point pairs.

[0140] Solving for optimal extrinsic parameters: This problem is solved using an iterative optimization algorithm, and the optimal extrinsic parameter estimates under the current conditions are finally output.

[0141] The health assessment and diagnosis module 302D includes health assessment functions and fault diagnosis functions.

[0142] In this embodiment, the health assessment function is as follows: calculate the confidence level (e.g., matching error, number of feature points involved in matching) of the current calibration result (i.e., the optimal external parameter estimate) and compare its stability (e.g., parameter fluctuation variance) with historical data (i.e., historical target parameter estimates). If the confidence level of the current calibration result is less than the preset confidence level and the parameter fluctuation variance of the current calibration result and the historical data is within the preset range, then the current calibration result passes the health assessment; otherwise, the current calibration result fails the health assessment.

[0143] If the calibration result fails the health assessment, fault diagnosis (i.e., fault prediction) can be performed on the LiDAR 301A.

[0144] In this embodiment, the fault diagnosis function is to establish a diagnostic rule base and associate the external parameter change patterns with potential physical faults.

[0145] For example, if a parameter in a certain axis (such as the Z-axis pitch angle) undergoes a step change, it may indicate that the mounting bracket has been impacted or loosened, indicating that the laser radar has experienced a level three fault.

[0146] For example, if the calculated current extrinsic parameters of the lidar show a slow but continuous linear drift, it may indicate that the support is slowly deforming due to material fatigue, suggesting that the lidar has experienced a level one failure.

[0147] For example, if the calibration confidence of the optimal external parameter estimate is consistently too low, it may indicate that the internal mirror of the lidar is dirty, icy, or the scanning mechanism is degraded, indicating that the lidar 301A has experienced a level 2 fault.

[0148] Among them, the decision-making and execution module 302E can execute corresponding strategies based on the diagnostic results output by the health assessment and diagnosis module 302D, forming a closed loop.

[0149] If the calibration result passes the health assessment (i.e., the calibration parameters change normally), the calibration parameters in the domain controller will be automatically updated without manual intervention. That is, the calibration result is output to the perception fusion algorithm module 302F, and the perception fusion algorithm module 302F updates silently.

[0150] If the calibration result fails the health assessment, different warnings will be issued based on the fault level. That is, a level 1 fault corresponds to a level 1 warning, a level 2 fault corresponds to a level 2 warning, and a level 3 fault corresponds to a level 3 warning.

[0151] For example, a Level 1 warning (performance degradation) notifies the perception fusion algorithm module 302F, which can then weight the results (i.e., the preset reliability) to reduce the preset reliability.

[0152] For example, a level 2 early warning (requiring maintenance) sends a reminder to the cloud server 303D and the user's mobile phone application (APP) via the remote information processing box 303A in the interactive device 303: "The lidar calibration parameters are abnormal; it is recommended to schedule an inspection." The reminder sent to the user's mobile phone application also needs to be implemented through the cloud server 303D.

[0153] For example, a level-three warning (immediate safe stop): If a sharp change in parameters or an extremely abnormal confidence level is detected, indicating a major safety hazard, the vehicle's minimum risk strategy is triggered. This can be achieved by prompting the driver 50 to take over the vehicle through the human-machine interface 303B in the interactive device 303, or by controlling the actuator 60 in the vehicle through the on-board controller 703 in the interactive device 303 to achieve a safe stop. The actuator 60 can be a brake pad.

[0154] Compared to related technologies, this embodiment offers several advantages: First, it features online real-time calibration, transforming it from an offline, periodic operation into an online, real-time process that can instantly and automatically compensate for parameter drift. Second, it is automated and seamless, requiring no human intervention or specific calibration sites and utilizing natural environments, thus adapting to large-scale automated driving vehicle operations. Third, it provides early fault warnings, enabling predictive maintenance by intelligently analyzing calibration parameter change patterns before sensor performance significantly degrades or completely fails. Fourth, it enhances safety by preventing silent degradation of perception performance due to calibration inaccuracies, providing continuous and reliable external parameter assurance for the autonomous driving system, and strengthening overall system safety redundancy. Fifth, it reduces costs by minimizing unexpected downtime and vehicle return-to-factory repairs due to sensor failures, thereby lowering maintenance costs for autonomous vehicles.

[0155] Based on the above description, the core of this embodiment is a system and method that enables autonomous vehicles to self-calibrate their "LiDAR" through online self-calibration during daily driving, and to perform predictive diagnosis of the radar based on the "historical record of calibration deviation", thereby automatically making different levels of safety responses.

[0156] Among them, online self-calibration: using natural road features (such as streetlights and signs) as reference objects, the LiDAR is automatically calibrated in real time while the vehicle is driving normally, without the need for manual intervention or a dedicated calibration site.

[0157] Among them, predictive diagnosis: based on the historical trends and patterns of calibration parameters (external parameters), rather than just single results, to achieve early fault diagnosis and health prediction of lidar.

[0158] The system architecture protects the entire system hardware and modular design that implements the above methods, including specific combinations and collaborative relationships of data processing modules, feature extraction and matching modules, online calibration calculation modules, health assessment and diagnosis modules, decision and execution modules, and perception fusion algorithm modules.

[0159] It should be noted that although the steps of the method in this application are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps; or steps from different embodiments may be combined into a new technical solution.

[0160] Based on the same inventive concept as the foregoing embodiments, this application provides a parameter calibration device for a lidar. The various modules and units included in this device can be implemented by a processor; of course, they can also be implemented by specific logic circuits. In the implementation process, the processor can be an AI acceleration engine (such as an NPU), GPU, central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP), or field programmable gate array (FPGA), etc.

[0161] Figure 4 This is a schematic diagram of the structure of a parameter calibration device for a lidar provided in an embodiment of this application, as shown below. Figure 4 As shown, the parameter calibration device 70 for lidar includes an acquisition module 701, a determination module 702, a calculation module 703, and a calibration module 704.

[0162] Among them, the acquisition module 701 is configured to acquire the first point cloud data collected by the vehicle in the driving environment at the previous moment and the second point cloud data collected at the current moment based on the current parameters of the lidar.

[0163] The determination module 702 is configured to determine the actual observed value of the matching feature in the second point cloud data based on the matching feature between the first point cloud data and the second point cloud data; and to predict the matching feature in the first point cloud data based on the vehicle's motion information to determine the predicted observed value of the matching feature at the current time.

[0164] The calculation module 703 is configured to perform minimum error calculation on the predicted and actual observations when the difference between the predicted and actual observations exceeds a preset deviation range, thereby obtaining the target parameter estimate of the lidar.

[0165] The calibration module 704 is configured to calibrate the current parameters of the lidar using the target parameter estimate.

[0166] In some embodiments, the calibration module 704 is further configured to: assess the health of the target parameter estimate, determine the health assessment result of the target parameter estimate; and, if the health assessment result indicates that the target parameter estimate passes the health assessment, calibrate the current parameters of the lidar using the target parameter estimate.

[0167] In some embodiments, the calibration module 704 is further configured to: determine the confidence level of the target parameter estimate; if the confidence level of the target parameter estimate is greater than or equal to a preset confidence level, determine that the health assessment result indicates that the target parameter estimate passes the health assessment; if the confidence level of the target parameter estimate is less than the preset confidence level, determine that the health assessment result indicates that the target parameter estimate fails the health assessment.

[0168] In some embodiments, the calibration module 704 is further configured to: determine the difference between the target parameter estimate and the historical target parameter estimate; if the difference is within a preset range, determine that the health assessment result indicates that the target parameter estimate has passed the health assessment; if the difference is outside the set range, determine that the health assessment result indicates that the target parameter estimate has not passed the health assessment.

[0169] In some embodiments, the parameter calibration device 70 of the lidar further includes a fault prediction module; wherein the fault prediction module is configured to: when the health assessment result indicates that the estimated value of the target parameter fails the health assessment, perform fault prediction on the lidar based on at least one of the following judgment strategies; wherein the judgment strategies include: whether multiple current parameters obtained within a first preset time period have undergone linear drift; whether the confidence level of multiple target parameter estimates obtained within a second preset time period is less than a preset confidence level; and whether at least one axial parameter in the target parameter estimates has undergone a step change.

[0170] In some embodiments, the fault prediction module is further configured to: determine that the lidar has experienced a level 1 fault when multiple current parameters obtained within a first preset time period exhibit linear drift; or determine that the lidar has experienced a level 2 fault when the confidence levels of multiple target parameter estimates obtained within a second preset time period are all less than a preset confidence level; or determine that the lidar has experienced a level 3 fault when at least one axial parameter in the target parameter estimates undergoes a step change; wherein the level 1 fault is less dangerous than the level 2 fault, and the level 2 fault is less dangerous than the level 3 fault.

[0171] In some embodiments, the parameter calibration device 70 for the lidar further includes an early warning module; wherein the early warning module is configured to: issue an early warning based on the lidar's fault level when it is determined that the lidar has malfunctioned; wherein: when the fault level is a level 1 fault, the early warning includes updating the preset confidence level; when the fault level is a level 2 fault, the early warning includes pushing a safety reminder to the cloud server and / or the user; when the fault level is a level 3 fault, the early warning includes prompting the driver to take over the vehicle or stop the vehicle safely.

[0172] It should be noted that the parameter calibration device 70 for LiDAR provided in this application embodiment can be applied to an autonomous driving domain controller.

[0173] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0174] It should be noted that the module division in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical units, or have two or more units integrated into one unit. The integrated units can be implemented in hardware, as software functional units, or a combination of software and hardware.

[0175] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0176] This application provides a vehicle, Figure 5 This is a schematic diagram of the vehicle structure provided in the embodiments of this application, such as... Figure 5 As shown, the vehicle 80 includes a memory 801 and a processor 802. The memory 801 stores a computer program that can run on the processor 802. When the processor 802 executes the program, it implements the steps in the method provided in the above embodiments.

[0177] It should be noted that the memory 801 is configured to store instructions and applications executable by the processor 802, and can also cache data to be processed or already processed in the processor 802 and various modules in the vehicle 80 (e.g., image data, audio data, voice communication data and video communication data), which can be implemented by flash memory or random access memory (RAM).

[0178] This application also provides a computer-readable storage medium for storing computer programs.

[0179] Optionally, the computer-readable storage medium can be applied to the electronic device in the embodiments of this application, and the computer program causes the processor or electronic device to perform the various methods of the embodiments of this application, which will not be described in detail here for the sake of brevity.

[0180] This application also provides a computer program product, including computer program instructions.

[0181] Optionally, the computer program product can be applied to the electronic device in the embodiments of this application, and the computer program instructions cause the processor or electronic device to execute the various methods in the embodiments of this application. For the sake of brevity, they will not be described in detail here.

[0182] This application also provides a computer program.

[0183] Optionally, the computer program can be applied to the vehicle in the embodiments of this application. When the computer program runs on the processor or the vehicle, it causes the processor or the vehicle to execute the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0184] It should be noted that the descriptions of the vehicles, storage media, computer program products, and computer program embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the vehicle, storage media, computer program products, and computer program embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0185] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0186] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0187] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0188] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0189] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0190] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0191] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0192] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause the vehicle to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0193] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0194] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0195] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0196] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A parameter calibration method for a lidar, characterized in that, The method includes: Based on the current parameters of the lidar, acquire the first point cloud data collected by the vehicle in the driving environment at the previous moment and the second point cloud data collected at the current moment. Based on the matching features between the first point cloud data and the second point cloud data, determine the actual observed value of the matching features in the second point cloud data; Based on the vehicle's motion information, the matching features in the first point cloud data are predicted to determine the predicted observation value of the matching features at the current moment; If the difference between the predicted observation and the actual observation exceeds a preset deviation range, the predicted observation and the actual observation are solved for minimum error to obtain the target parameter estimate of the lidar. The current parameters of the lidar are calibrated using the estimated target parameters.

2. The method according to claim 1, characterized in that, The step of calibrating the current parameters of the lidar using the estimated target parameters includes: The health status of the estimated target parameter values ​​is assessed to determine the health assessment result of the estimated target parameter values. If the health assessment result indicates that the target parameter estimate passes the health assessment, the current parameters of the lidar are calibrated using the target parameter estimate.

3. The method according to claim 2, characterized in that, The process of assessing the health of the estimated target parameter values ​​and determining the health assessment result of the estimated target parameter values ​​includes: Determine the confidence level of the estimated target parameter; If the confidence level of the target parameter estimate is greater than or equal to a preset confidence level, the health assessment result indicates that the target parameter estimate passes the health assessment. If the confidence level of the target parameter estimate is less than the preset confidence level, the health assessment result indicates that the target parameter estimate has failed the health assessment.

4. The method according to claim 2, characterized in that, The process of assessing the health of the estimated target parameter values ​​and determining the health assessment result of the estimated target parameter values ​​includes: Determine the difference between the estimated target parameter value and the historical target parameter estimate; If the difference is within a preset range, the health assessment result indicates that the target parameter estimate passes the health assessment. If the difference is outside the set range, the health assessment result indicates that the target parameter estimate has failed the health assessment.

5. The method according to any one of claims 2 to 4, characterized in that, The method further includes: If the health assessment result indicates that the estimated value of the target parameter has failed the health assessment, the lidar shall be subject to fault prediction based on at least one of the following judgment strategies; The judgment strategy includes: Based on whether the multiple current parameters obtained within the first preset time period have undergone linear drift; Based on whether the confidence level of multiple estimated values ​​of the target parameters obtained within the second preset time period is less than a preset confidence level; Whether at least one axial parameter in the estimated target parameter values ​​undergoes a step change.

6. The method according to claim 5, characterized in that, The fault prediction of the lidar includes: If multiple current parameters obtained within the first preset time period exhibit linear drift, it is determined that the lidar has experienced a level one fault. or, If the confidence level of multiple target parameter estimates obtained within the second preset time period is less than the preset confidence level, it is determined that the lidar has experienced a level two fault. or, If at least one axial parameter in the estimated target parameter values ​​undergoes a step change, the lidar is determined to have experienced a level three fault. The level of danger of the first-level fault is less than that of the second-level fault, and the level of danger of the second-level fault is less than that of the third-level fault.

7. The method according to claim 6, characterized in that, The method further includes: If a fault is determined to have occurred in the lidar, an early warning is issued based on the fault level of the lidar; wherein: When the fault level is the first-level fault, the warning includes updating the preset confidence level; When the fault level is the Level 2 fault, the warning includes pushing a security alert to the cloud server and / or the user; When the fault level is the third level, the warning includes prompting the driver to take over the vehicle or stop the vehicle safely.

8. A parameter calibration device for a lidar, characterized in that, The parameter calibration device includes an acquisition module, a determination module, a calculation module, and a calibration module, wherein: The acquisition module is configured to acquire the first point cloud data collected by the vehicle in the driving environment at the previous moment and the second point cloud data collected at the current moment, based on the current parameters of the lidar. The determining module is configured to determine the actual observed value of the matching feature in the second point cloud data based on the matching feature between the first point cloud data and the second point cloud data; and to predict the matching feature in the first point cloud data based on the vehicle's motion information to determine the predicted observed value of the matching feature at the current moment. The calculation module is configured to perform a minimum error solution on the predicted observation value and the actual observation value when the difference between the predicted observation value and the actual observation value exceeds a preset deviation range, so as to obtain the target parameter estimate value of the lidar. The calibration module is configured to calibrate the current parameters of the lidar using the target parameter estimate.

9. A vehicle comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the method steps as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images