A method and system for online self-calibration of a lidar and camera without induction

Abstract

The present application relates to a kind of laser radar and camera online self-calibration method and system without induction, the method includes: S1.in the process of vehicle driving, through multi-source trigger mechanism acquisition laser radar point cloud data and camera image data, simultaneously obtain vehicle motion state information and automatic driving perception system output result, the multi-source trigger mechanism includes time trigger, event trigger and cloud data mining trigger parameter degradation model;S2.the pre-processing and feature extraction of the multi-source data collected, obtain laser radar point cloud feature and camera image feature;S3.based on geometric constraint, motion consistency and perception result fusion, constructs multi-modal verification model, and carries out multi-dimensional verification to calibration parameter.The present application not only reduces the artificial maintenance cost, through automatic means maintenance vehicle state to achieve sensor calibration parameter degradation self-detection and self-calibration without induction, and improves the reliability and attendance rate of automatic driving vehicle.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method and system for seamless online self-calibration of lidar and cameras. Background Technology

[0002] With the rapid development of autonomous driving technology, multi-sensor fusion has become a key technology for improving perception accuracy and system reliability. LiDAR and cameras, as core sensors in autonomous driving systems, directly impact the performance of downstream tasks such as target detection, semantic segmentation, and path planning due to their calibration accuracy. LiDAR provides precise 3D spatial information, while cameras provide rich texture and color information; the two complement each other, but this requires high-precision calibration parameters.

[0003] In practical engineering applications, various typical scenarios lead to the degradation of sensor calibration parameters. For example, during long-term vehicle operation, factors such as road bumps and engine vibration may cause slight displacement or loosening of sensor mounting brackets. Temperature changes cause material expansion and contraction, leading to drift in the relative position of the sensors. Furthermore, after a vehicle collision repair, reinstalling sensors may not guarantee complete consistency with the original calibration parameters. In summary, the calibration parameters of sensors on autonomous vehicles gradually degrade over long-term use. If not detected and updated in a timely manner, this will lead to a decline in the performance of the perception system and even safety hazards. Moreover, if sensor parameters change and recalibration is required, traditional methods typically require the vehicle to be taken to a repair shop for calibration using specialized equipment, which is technically challenging and time-consuming (usually 2-4 hours) and requires professional personnel. Therefore, there is a need for a seamless online self-calibration method and system that can automatically monitor the calibration parameter status during normal vehicle operation and complete calibration updates during idle periods. This would allow for timely detection and correction of calibration parameter deviations, ensuring the safety and reliability of the autonomous driving system while reducing maintenance costs and technical barriers.

[0004] In existing autonomous driving systems, sensor calibration primarily employs offline methods. Specifically, this involves setting up calibration boards or objects (such as checkerboard patterns, AprilTag markers, or calibration frames with specific geometric shapes) in a dedicated calibration area. Professional personnel manually adjust the sensor positions and collect multiple sets of calibration data. Then, calibration algorithms (such as Zhang Zhengyou's calibration method or hand-eye calibration algorithms) are used to calculate the camera's intrinsic parameters and the extrinsic parameters of the LiDAR and camera. Finally, the calibration parameters are written into the system configuration file, completing the calibration process. This process is typically performed at the end of the vehicle production line (EOL calibration) or at after-sales service points (after-sales calibration). The entire calibration process usually takes 2-4 hours and requires specialized equipment and personnel. Vehicles must be parked and moved to a dedicated calibration area, making calibration impossible during routine operations. However, this approach requires manual intervention and has a high technical threshold because it must be performed by professionals using specialized equipment in a dedicated calibration area, preventing automation, increasing operation and maintenance costs, and limiting calibration frequency (usually once every few months). Secondly, continuous monitoring is impossible because parameters are only acquired during calibration. During vehicle operation, minute displacements or vibrations in the sensors can cause parameter failures, which the system cannot detect in time until obvious perceptual errors occur, potentially leading to serious consequences. Furthermore, it relies on specific environments, requiring specific lighting conditions and calibration objects, resulting in poor adaptability and an inability to perform calibration in various road conditions and weather conditions. Finally, calibration involves vehicle downtime and high time costs, as the process takes 2-4 hours, impacting vehicle operational efficiency. This is particularly disadvantageous for scenarios requiring continuous operation, such as Robotaxi fleets. Additionally, vehicles must be taken to repair shops or production lines, making recalibration costly for daily user use. Summary of the Invention

[0005] In view of the above problems, the present invention provides a method and system for non-sensory online self-calibration of lidar and camera, which not only reduces the cost of manual maintenance and achieves self-detection and non-sensory self-calibration of sensor calibration parameter degradation through automated means of maintaining vehicle status, but also improves the reliability and uptime of autonomous vehicles.

[0006] To achieve the above and other related objectives, the present invention provides the following technical solution: A method for seamless online self-calibration of lidar and camera, the method comprising: S1. During vehicle operation, LiDAR point cloud data and camera image data are collected through a multi-source triggering mechanism, while vehicle motion state information and the output results of the autonomous driving perception system are obtained. The multi-source triggering mechanism includes a parameter degradation model triggered by time, event, and cloud data mining. S2. Preprocess and extract features from the collected multi-source data to obtain lidar point cloud features and camera image features; S3. Based on the fusion of geometric constraints, motion consistency, and perception results, a multimodal verification model is constructed to verify the calibration parameters in multiple dimensions; S4. Determine whether calibration is needed based on the triggering conditions. If any of the multi-source triggering conditions are met, mark it as needing calibration and accumulate data samples. S5. Perform calibration parameter update calculations during system idle periods or within a delayed power-off window after the user locks the vehicle. It supports calculation interruption and breakpoint resume functions, and supports dual modes of vehicle-side calculation and cloud-based calculation. S6. Update calibration parameters only when the autonomous driving system is inactive, and verify their reasonableness.

[0007] Furthermore, in step S1, the time trigger is triggered when the time since the last calibration exceeds a preset time threshold, or when the distance traveled since the last calibration exceeds a preset mileage threshold, or when data is collected once every fixed period. The event trigger is triggered when the autonomous driving system reports calibration-related fault codes or abnormal events, or when maintenance operations affecting sensor positions are detected through after-sales cloud data, on-board diagnostic systems, or maintenance records, or when collision events are detected through collision sensors, airbag deployment records, or accident reports, or when software version updates or sensor hardware replacements affect calibration parameters.

[0008] Furthermore, in step S2, the feature extraction includes extracting edge feature points, planar feature points, and corner features from the lidar point cloud, extracting ordinary feature points or deep learning feature descriptors from the camera image, and establishing a preliminary correspondence between the point cloud and image pixels.

[0009] Furthermore, in step S3, the multimodal verification model includes a geometric constraint verification submodule, a motion consistency verification submodule, and a perception fusion verification submodule. The geometric constraint verification submodule verifies the geometric consistency between the lidar point cloud and the camera image through reprojection error and epipolar constraint. The motion consistency verification submodule verifies the stability of the calibration parameters over time by utilizing the relative pose changes between consecutive frames. The perception fusion verification submodule compares the perception results generated independently based on different sensors to detect the consistency of the target detection box and the semantic segmentation results.

[0010] Furthermore, in step S4, the trigger condition is determined by checking whether the time interval or driving mileage since the last calibration exceeds a preset threshold, receiving event information from autonomous driving system fault reports, after-sales cloud data, and collision sensor channels. If any trigger condition is met, it is marked as needing calibration and high-quality data samples are started to be accumulated.

[0011] Furthermore, in step S5, the system idle period determination conditions include CPU / GPU utilization rate being lower than a preset threshold, the autonomous driving system not being in an active state or an emergency decision-making state, the vehicle being in a low-speed driving or stationary state, the dedicated calculation time window allocated by the background task scheduler, and the time window between the user locking the car and the vehicle power-off. The calculation mode for delayed power-off after the user locks the car is as follows: if there is a calibration calculation requirement and the vehicle is stationary, and the user performs a locking operation, the system delays the complete power-off of the vehicle. After the user locks the car, the system maintains the necessary power supply to the computing units and continues to execute the calibration calculation task. After the calibration calculation is completed, the system normally executes the vehicle power-off process. If the calibration... If the calculation is not completed or fails within the preset timeout period, the system will abandon the calculation and proceed with the vehicle power-off process normally. During the calculation, the battery level is monitored to ensure that the calculation does not consume excessive power and a maximum calculation time limit is set. It also includes a cloud calibration calculation mode, in which the collected calibration-related data is uploaded to the cloud server through the vehicle network. The cloud server uses computing resources to perform calibration optimization calculations and adopts optimization algorithms and large-scale data samples. After the cloud calculation is completed, the optimized calibration parameters are sent to the vehicle through the network link by sending configuration files. After receiving the new parameters, the vehicle performs reasonableness checks and safety verifications, and stores them in the local parameter storage area after confirming that they are correct.

[0012] Furthermore, in step S5, the breakpoint continuation function immediately interrupts the calibration calculation when the autonomous driving system is activated, the CPU / GPU utilization rate exceeds the safety threshold, the vehicle enters high-speed driving, or an emergency task request is received. When interrupted, the number of completed iterations, the current optimization results, the intermediate variables of the optimization algorithm, and the index of the remaining unprocessed data samples are saved. When the system enters the idle state again, the calculation state is loaded from the saved breakpoint and the optimization continues until all calculations are completed or the convergence condition is met.

[0013] Furthermore, in step S6, the inactive state parameter is updated to confirm that the vehicle is stationary, the autonomous driving function is turned off, and there are no pending activation requests before updating the calibration parameters. The newly calculated calibration parameters are written into the configuration file and the version number and update timestamp are updated, and the parameter change log is recorded. Before the next activation of the autonomous driving system, the new parameters are used for pre-check, and if the verification fails, it will automatically revert to the previous version.

[0014] To achieve the above and other related objectives, the present invention also provides a system for seamless online self-calibration of LiDAR and camera, used to implement the aforementioned method for seamless online self-calibration of LiDAR and camera. The system includes a data acquisition module, a feature extraction module, a multi-modal verification module, a trigger judgment module, a calibration decision module, a parameter management module, and a cloud communication module. The data acquisition module is used to acquire LiDAR, camera, and vehicle status data through a multi-source triggering mechanism, wherein the multi-source triggering mechanism includes time-triggered, event-triggered, and cloud data mining-triggered parameter degradation models. The feature extraction module is used to perform multi-source... The data undergoes preprocessing and feature extraction. The multimodal verification module is used to build a verification model based on geometric constraints, motion consistency, and perception fusion. The trigger judgment module is used to determine whether calibration is needed based on multi-source trigger conditions. The calibration decision module is used to perform vehicle-side calibration calculations or cloud-based calibration calculations during system idle periods or after the user locks the vehicle and delays power-off, and supports interruption and breakpoint resumption. The parameter management module is used to update calibration parameters and perform rationality verification only when the autonomous driving is inactive. The cloud communication module is used to upload calibration data to the cloud server and receive configuration files sent from the cloud via network link.

[0015] To achieve the above and other related objectives, the present invention also provides an autonomous driving vehicle equipped with the aforementioned system for seamless online self-calibration of lidar and camera. The system is communicatively connected to the vehicle's autonomous driving controller, power management system, and body control module, providing real-time calibration parameter status information and automatically updating sensor extrinsic parameters when the autonomous driving is inactive. The system can delay the complete power-off of the vehicle after the user locks the vehicle in order to complete the calibration calculation task.

[0016] The present invention has the following positive effects: 1. This invention reduces operation and maintenance costs and improves the reliability and uptime of autonomous driving by using multi-source triggered data acquisition and intelligent road segment determination. It achieves automated data acquisition without human intervention through three modes: time-triggered, event-triggered, and cloud-based model-triggered. It innovatively introduces a road segment determination mechanism based on a global path. The system automatically identifies road segments that meet calibration requirements (rich geometric features, stable lighting, and moderate traffic) and enters a data acquisition preparation state in advance when approaching the target road segment. After entering the designated road segment, it collects a series of frames of data according to predetermined rules (e.g., 100 frames covering a circular road segment to ensure data closure) and monitors three levels of constraints (motion state, environmental conditions, and system state) in real time. Formal data acquisition is only performed when all constraints are simultaneously satisfied. Compared with traditional methods, this intelligent data acquisition strategy not only reduces manual maintenance costs but also achieves self-detection of sensor calibration parameter degradation and non-sensory self-calibration through automated vehicle state maintenance, thus improving the reliability and uptime of autonomous vehicles.

[0017] 2. This invention improves the accuracy and robustness of verification through multimodal fusion verification. By combining verification information from three dimensions—geometric constraints, motion consistency, and sensory fusion—the accuracy and robustness of calibration parameter verification are significantly improved. Utilizing the parallel processing architecture of the multimodal verification module, the three sub-modules independently compute and then perform weighted fusion, avoiding the limitations of a single verification method. Experiments show that compared to a single verification method, multimodal fusion verification reduces both the false alarm rate and the false negative rate, providing a reliable basis for subsequent calibration decisions.

[0018] 3. This invention supports dual-mode operation (vehicle-side and cloud-side) through intelligent resource scheduling and breakpoint resumption, innovatively utilizing a time window after vehicle locking to ensure real-time performance and computational accuracy. By performing calibration calculations during system idle periods and supporting calculation interruption and breakpoint resumption functions, it avoids performance interference of the calibration algorithm on the main autonomous driving system. Utilizing multi-dimensional idle judgment conditions (CPU / GPU utilization, autonomous driving status, vehicle speed, and time window) of the calibration decision module, it ensures efficient execution of the calibration task without affecting safety-critical tasks. When important tasks such as autonomous driving need to be activated, the calibration task can be immediately interrupted and computing power released, saving the current calculation state; calculation can resume from the breakpoint after the system becomes idle again, avoiding redundant calculations and wasted resources. Furthermore, this invention innovatively utilizes the time window after the user locks the vehicle: when there is a calibration calculation requirement and the vehicle is stationary, if the user performs a locking operation, the system can delay the vehicle from fully powering down, continue calibration calculations after locking, and then power down normally after the calculation is completed or timeout, making full use of fragmented time to improve calibration efficiency while monitoring battery power to ensure safety. This post-lock calibration mode is particularly suitable for daily commuting scenarios. Calibration is automatically completed after the user parks and locks the vehicle, eliminating the need for additional waiting time. Simultaneously, relying on a closed-loop data link, it supports cloud-based calibration calculations. Leveraging the powerful computing capabilities of the cloud, more complex optimization algorithms and larger-scale data samples can be employed to improve calibration accuracy. Configuration files are distributed to the vehicle via network links, enabling fleet-level parameter management and version control. In actual testing, the peak CPU utilization during vehicle-side calibration calculations did not exceed 15%, preventing delays in other intelligent driving modules due to computing power contention and ensuring the normal operation and safety of the autonomous driving system. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a flowchart of the system idle time period determination and vehicle-side cloud dual-mode calibration process of the present invention. Detailed Implementation

[0020] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0021] Example 1: As Figure 1 As shown, a method for seamless online self-calibration of a lidar and camera is provided, the method comprising: S1. During vehicle operation, LiDAR point cloud data and camera image data are collected through a multi-source triggering mechanism, while vehicle motion state information and the output results of the autonomous driving perception system are obtained. The multi-source triggering mechanism includes a parameter degradation model triggered by time, event, and cloud data mining. S2. Preprocess and extract features from the collected multi-source data to obtain lidar point cloud features and camera image features; S3. Based on the fusion of geometric constraints, motion consistency, and perception results, a multimodal verification model is constructed to verify the calibration parameters in multiple dimensions; S4. Determine whether calibration is needed based on the triggering conditions. If any of the multi-source triggering conditions are met, mark it as needing calibration and accumulate data samples. S5. Perform calibration parameter update calculations during system idle periods or within a delayed power-off window after the user locks the vehicle. It supports calculation interruption and breakpoint resume functions, and supports dual modes of vehicle-side calculation and cloud-based calculation. S6. Update calibration parameters only when the autonomous driving system is inactive, and verify their reasonableness.

[0022] In this embodiment, in step S1, the time trigger is triggered when the time since the last calibration exceeds a preset time threshold, or when the distance since the last calibration exceeds a preset mileage threshold, or when data is collected once every fixed period. The event trigger is triggered when the autonomous driving system reports calibration-related fault codes or abnormal events, or when maintenance operations affecting sensor positions are detected through after-sales cloud data, on-board diagnostic systems, or maintenance records, or when collision events are detected through collision sensors, airbag deployment records, or accident reports, or when software version updates or sensor hardware replacements affect calibration parameters.

[0023] In this embodiment, in step S2, the feature extraction includes extracting edge feature points, planar feature points and corner features from the lidar point cloud, extracting ordinary feature points or deep learning feature descriptors from the camera image, and establishing a preliminary correspondence between the point cloud and image pixels.

[0024] In this embodiment, in step S3, the multimodal verification model includes a geometric constraint verification submodule, a motion consistency verification submodule, and a perception fusion verification submodule. The geometric constraint verification submodule verifies the geometric consistency between the lidar point cloud and the camera image through reprojection error and epipolar constraint. The motion consistency verification submodule verifies the stability of the calibration parameters over time by using the relative pose changes between consecutive frames. The perception fusion verification submodule compares the perception results generated independently based on different sensors to detect the consistency of the target detection box and the semantic segmentation results.

[0025] In this embodiment, in step S4, the trigger condition is determined by checking whether the time interval or driving mileage since the last calibration exceeds a preset threshold, receiving event information from autonomous driving system fault reports, after-sales cloud data, and collision sensor channels. If any trigger condition is met, it is marked as needing calibration and high-quality data samples are started to be accumulated.

[0026] In this embodiment, in step S5, the system idle time determination conditions include CPU / GPU utilization rate being lower than a preset threshold, the autonomous driving system not being in an active state or an emergency decision-making state, the vehicle being in a low-speed driving or stationary state, a dedicated calculation time window allocated by the background task scheduler, and a time window between the user locking the car and the vehicle power-off. The calculation mode for delayed power-off after the user locks the car is that if there is a calibration calculation requirement and the vehicle is stationary, and the user performs a locking operation, the system delays the complete power-off of the vehicle. After the user locks the car, the system maintains the necessary power supply to the computing units and continues to execute the calibration calculation task. After the calibration calculation is completed, the system normally executes the vehicle power-off process. If the calibration calculation is not completed, the system will proceed as scheduled. If the calibration calculation is not completed or fails within the preset timeout period, the system will abandon the calculation and proceed with the vehicle power-off process normally. During the calculation, the battery level is monitored to ensure that the calculation does not consume excessive power and a maximum calculation time limit is set. It also includes a cloud calibration calculation mode, in which the collected calibration-related data is uploaded to the cloud server through the vehicle network. The cloud server uses computing resources to perform calibration optimization calculations and adopts optimization algorithms and large-scale data samples. After the cloud calculation is completed, the optimized calibration parameters are sent to the vehicle through the network link by sending configuration files. After receiving the new parameters, the vehicle performs reasonableness checks and safety verifications, and stores them in the local parameter storage area after confirming that they are correct.

[0027] In this embodiment, the data acquisition module is responsible for automatically collecting multi-source sensor data during vehicle operation through a multi-source triggering mechanism. This module supports three triggering modes: a time-triggered mode that triggers data acquisition based on a preset time interval (e.g., 90 days), a mileage threshold (e.g., 5000 kilometers), or a fixed period (e.g., once per quarter); an event-triggered mode that triggers when events such as autonomous driving system fault codes, maintenance records, collision events, software version updates, or sensor hardware replacements are detected; and a cloud-based data mining triggering mode that receives parameter degradation model prediction results from a cloud server and triggers when the deviation of the prediction calibration parameters exceeds a threshold. The types of data collected include LiDAR point cloud data, camera image data, vehicle motion state information, and output results from the autonomous driving perception system (object detection boxes, semantic segmentation results, etc.). After timestamp alignment and coordinate system transformation, the collected data is stored in a local circular buffer (the capacity of which can store incomplete data (reduced frequency after the system startup time of the most recent 24 hours) or a small amount of complete data).

[0028] The feature extraction module preprocesses and extracts features from the acquired raw sensor data. For LiDAR point cloud processing, voxel grid filtering is first used for downsampling, followed by a statistical outlier removal algorithm to filter out noise points, and then a plane fitting algorithm to remove ground points. Based on this, curvature calculation is used to extract edge feature points, flatness calculation is used to extract planar feature points, and a corner detection algorithm is used to extract corner features. LiDAR data feature extraction can also be performed using segmentation models. By inferring single-frame point cloud data through the model, the type attributes of each point are obtained. The point cloud is then projected through a virtual camera to obtain a pseudo-grayscale image, and finally, point cloud features are obtained using computer graphics algorithms. For camera image processing, distortion correction is first performed based on camera intrinsic parameters, histogram equalization is used to enhance image contrast, and then ORB feature points, SIFT feature points, and other features are extracted and feature descriptors are calculated. Finally, by projecting the LiDAR point cloud onto the camera image plane, a preliminary correspondence between the point cloud and image pixels is established using depth information and pixel coordinates, and epipolar constraints are used to filter valid matching pairs.

[0029] The multimodal verification module constructs a multi-dimensional verification model based on geometric constraints, motion consistency, and perception fusion to perform multi-dimensional verification on the collected data and evaluate the effectiveness of the current calibration parameters. This module contains three parallel sub-modules. (1) The geometric constraint verification sub-module calculates the reprojection error (the Euclidean distance between the LiDAR point cloud and the actual pixel position after being projected onto the image plane according to the current extrinsic parameters, requiring the root mean square error RMSE to be less than 3 pixels), checks the epipolar constraint (the distance from the corresponding point to the epipolar line is less than 1 pixel), and calculates the alignment between the point cloud and the image edge (edge ​​detection extracts the image edge, and the average distance from the projected point cloud to the edge is less than 0.2 meters). (2) The motion consistency verification sub-module uses the relative pose change between consecutive frames to verify the stability of the calibration parameters in the time series, calculates the relative transformation of the LiDAR point cloud in adjacent frames, calculates the relative transformation of the camera in adjacent frames through visual odometry, compares the relative pose difference obtained by the two methods, and uses an exponentially weighted moving average filter to smooth the pose estimation results. (3) The perception fusion verification submodule compares the perception results generated independently by different sensors, calculates the IoU (Intersection over Union) of the LiDAR target detection box and the camera target detection box (required to be greater than 0.6), compares the consistency of the semantic segmentation results (measured using the Dice coefficient, required to be greater than 0.7), and detects conflicts in the multi-sensor perception results. Based on the verification results from the three dimensions, a weighted fusion strategy is used to calculate the comprehensive confidence score: Score = w1 × S geometry +w2×S motion +w3×S perception Where w1, w2, and w3 are weighting coefficients. When the overall confidence score is lower than the warning threshold (e.g., 0.6), it is marked as needing calibration; when it is lower than the emergency threshold (e.g., 0.4), the calibration process is triggered immediately.

[0030] The trigger judgment module determines whether calibration is required based on multiple trigger conditions. If any trigger condition is met, it is marked as requiring calibration and the data accumulation process is started. (1) Time trigger conditions include the time threshold (e.g., 90 days) since the last calibration, or the mileage threshold (e.g., 5000 kilometers) since the last calibration, or data collection and verification at fixed intervals (e.g., every few degrees). (2) Event trigger conditions include the autonomous driving system reporting calibration-related fault codes or abnormal events, the detection of maintenance operations that may affect the sensor position (e.g., sensor disassembly and reassembly, bracket replacement, etc.) through after-sales cloud data, vehicle diagnostic system, or maintenance records, the detection of collision events through collision sensors, airbag deployment records, or accident reports, and events that may affect calibration parameters such as software version updates and sensor hardware replacements. (3) Cloud model trigger conditions are the receipt of parameter degradation model prediction results from the cloud server. When the deviation of the current calibration parameters exceeds the safety threshold, it is triggered. When any trigger condition is met, the trigger judgment module outputs a calibration request signal, and the system enters the data accumulation stage, prioritizing the collection of high-quality data samples.

[0031] The calibration decision module is responsible for scheduling calibration calculation tasks, supports dual-mode calculation on both the vehicle and cloud, and introduces a calculation mode with delayed power-off after locking the vehicle.

[0032] In vehicle-side calibration calculation mode, the calibration decision module continuously monitors the system's idle state. Idle conditions include: CPU utilization, GPU utilization, the autonomous driving system not being activated or in an emergency decision-making state, the vehicle being in a low-speed driving or stationary state, and a dedicated calculation time window (such as a specific time period) allocated by the background task scheduler. Once the idle conditions are met, the system retrieves accumulated data samples from local storage and executes a joint optimization algorithm to calculate calibration parameters.

[0033] The breakpoint-based continuation mechanism handles interruptions during computation. When the autonomous driving system is activated, CPU / GPU utilization exceeds a safety threshold, the vehicle enters high-speed driving, or an emergency task request is received, the calibration computation is immediately interrupted. During the interruption, the number of completed iterations, the current optimization results, intermediate variables of the optimization algorithm, and the index of remaining unprocessed data samples are saved. When the system returns to an idle state, the computation state is loaded from the saved breakpoint and optimization continues until all computations are completed or convergence conditions are met. The breakpoint state is saved to non-volatile memory to ensure it is not lost after the vehicle loses power.

[0034] The delayed power-down calculation mode after locking the vehicle is triggered when calibration calculations are required and the vehicle receives a power-down request. If the user locks the vehicle, the system sends a delayed power-down request to the Body Control Module (BCM) to maintain power to necessary computing units (CPU, GPU, memory, and storage devices) and continue executing the calibration calculation task. After the calibration calculation is completed, the system executes the vehicle power-down process normally. If the calibration calculation is not completed within the preset timeout period or fails, the system abandons the current calculation and executes the vehicle power-down process normally, while saving the calculation results as data for resuming calculations. Battery power is monitored in real time during the calculation process to ensure that the calculation does not excessively consume power.

[0035] In cloud-based calibration computation mode, the collected calibration-related data (point cloud, images, motion status, etc.) is uploaded to the cloud server via the vehicle-to-everything (V2X) network. Data is compressed and encoded before upload to reduce transmission bandwidth. The cloud server utilizes computing resources to perform calibration optimization calculations, employing more complex optimization algorithms and larger-scale data samples (such as longer data periods or more data frames). After cloud computation is complete, the optimized calibration parameters are distributed to the vehicle via a network link using a configuration file. Upon receiving the new parameters, the vehicle performs rationality checks and security verifications, and stores them in its local parameter storage area after confirming their accuracy.

[0036] The system can dynamically select the computing mode based on the actual situation. The vehicle-side mode is suitable for scenarios with high real-time requirements and poor network conditions, while the cloud-side mode is suitable for scenarios requiring high-precision computing and large data volumes. Alternatively, the system can determine whether to use the cloud-side computing mode or the vehicle-side computing mode, or both, through a configuration file.

[0037] The parameter management module is responsible for storing, versioning, and controlling the updates of calibration parameters, and strictly enforces the inactive state update mechanism.

[0038] For parameter storage, calibration parameters are stored in non-volatile memory in the form of configuration files. The configuration files contain information such as sensor intrinsic and extrinsic parameters, parameter version numbers, update timestamps, confidence scores, and trigger event types. A dual-backup storage mechanism is employed, with one copy of the parameters stored in the primary storage area and another in the backup storage area to prevent parameter loss due to a single point of failure.

[0039] For version management, the version number is incremented each time parameters are updated, and a parameter change log is recorded. The parameter change log includes information such as old parameters, new parameters, triggering events, calculation time, and confidence scores. The system retains the parameter history for the most recent n versions and supports parameter rollback functionality.

[0040] The inactive state update mechanism requires that the autonomous driving system be confirmed to be inactive before updating calibration parameters. Inactive state is defined as follows: the vehicle is stationary (speed 0, in P gear or with the handbrake engaged) and the autonomous driving function is disabled. If the autonomous driving system is active or about to be activated, the system will delay parameter updates until the above conditions are met.

[0041] The parameter update process first checks the reasonableness of the newly calculated calibration parameters. This reasonableness check includes: orthogonality check, magnitude range check, and change magnitude check (the difference between the new and old parameters should be within a reasonable range). After confirming that the autonomous driving system is inactive, the new parameters are written to the configuration file in the main storage area, the version number and update timestamp are updated, and the parameter change log is recorded. Then, cross-validation is performed, using the new parameters to calculate the reprojection error on a reserved test dataset. If the validation passes, the new parameters are copied to the backup storage area and marked as usable. Before the next activation of the autonomous driving system, the system uses the new parameters for pre-checking. If the validation fails, it automatically reverts to the previous version of the calibration parameters; if the validation passes, it waits for the next activation of the autonomous driving system for use.

[0042] In terms of security, atomic operations are used during parameter updates to ensure that no parameter corruption or inconsistencies occur. If an anomaly occurs during the update process (such as a power outage or system crash), the system can automatically revert to the previous parameter version.

[0043] The cloud communication module is responsible for data communication between the vehicle and the cloud server. For data uploading, calibration data (point cloud, images, motion status, verification results, etc.) collected by the vehicle is uploaded to the cloud server via the vehicle network. Before uploading, the data is compressed, de-identified, de-encrypted, and encrypted to ensure data transmission security and compliance. It also supports breakpoint resume functionality, allowing uploading to continue after network interruption. For parameter distribution, it receives configuration files distributed via the network link after cloud computation, performs integrity verification and digital signature verification to confirm the data source is reliable and has not been tampered with. Regarding cloud collaboration, it supports collaborative work with the cloud server. Calibration data collected by the vehicle is uploaded to the cloud for analysis and management via the vehicle network. The cloud can aggregate calibration data from multiple vehicles, establish calibration parameter change models, identify common problems, and push updated calibration algorithms or parameter optimization suggestions to vehicles, forming a data closed loop. This cloud collaboration mechanism further enhances the intelligence and adaptability of the calibration system, enabling fleet-level calibration parameter management and optimization.

[0044] In this embodiment, in step S5, the breakpoint continuation function immediately interrupts the calibration calculation when the autonomous driving system is activated, the CPU / GPU utilization rate exceeds the safety threshold, the vehicle enters high-speed driving, or an emergency task request is received. When interrupted, the number of completed iterations, the current optimization result, the intermediate variables of the optimization algorithm, and the index of the remaining unprocessed data samples are saved. When the system enters the idle state again, the calculation state is loaded from the saved breakpoint and the optimization continues until all calculations are completed or the convergence condition is reached.

[0045] In this embodiment, in step S6, the inactive state parameter is updated to confirm that the vehicle is stationary, the autonomous driving function is turned off, and there are no pending activation requests before updating the calibration parameters. The newly calculated calibration parameters are written into the configuration file and the version number and update timestamp are updated, and the parameter change log is recorded. Before the next activation of the autonomous driving system, the new parameters are used for pre-check, and if the verification fails, it will automatically revert to the previous version.

[0046] In this embodiment, the present invention provides a system for seamless online self-calibration of LiDAR and camera, used to implement the aforementioned method for seamless online self-calibration of LiDAR and camera. The system includes a data acquisition module, a feature extraction module, a multimodal verification module, a trigger judgment module, a calibration decision module, a parameter management module, and a cloud communication module. The data acquisition module is used to acquire LiDAR, camera, and vehicle status data through a multi-source triggering mechanism, which includes time-triggered, event-triggered, and cloud data mining-triggered parameter degradation models. The feature extraction module is used to preprocess and extract features from the multi-source data. The multimodal verification module is used to construct a verification model based on geometric constraints, motion consistency, and perception fusion. The trigger judgment module is used to determine whether calibration is required based on the multi-source triggering conditions. The calibration decision module is used to perform vehicle-side calibration calculations during system idle periods or a delayed power-off window after the user locks the vehicle, or to perform calibration calculations in the cloud, supporting interruption and breakpoint resumption. The parameter management module is used to update calibration parameters only when the autonomous driving system is inactive and to perform rationality verification. The cloud communication module is used to upload calibration data to a cloud server and receive configuration files sent from the cloud via a network link.

[0047] In this embodiment, the present invention provides an autonomous vehicle equipped with the aforementioned system for seamless online self-calibration of lidar and camera. The system is communicatively connected to the vehicle's autonomous driving controller, power management system, and body control module, providing real-time calibration parameter status information and automatically updating sensor extrinsic parameters when the autonomous driving is inactive. The system can delay the complete power-off of the vehicle after the user locks the vehicle in order to complete the calibration calculation task.

[0048] In this embodiment, as Figure 2As shown, the entire system's workflow comprises four stages, tightly interconnected through data and control flows. The first stage is data acquisition and storage. The data acquisition module automatically collects multi-source sensor data based on multi-source triggering conditions (time-triggered, event-triggered, and cloud model-triggered). The feature extraction module preprocesses and extracts features from the data. The processed data is stored in a local circular buffer or uploaded to the cloud. During acquisition, three levels of constraints (motion state, environmental conditions, and system state) are monitored in real time. Formal acquisition is only executed when all constraints are simultaneously satisfied. The second stage is trigger judgment and data accumulation. The trigger judgment module continuously monitors time-triggered and event-triggered conditions. When any triggering condition is met, it is marked as requiring calibration. The system begins accumulating high-quality calibration data samples. The multimodal verification module performs parallel verification across three dimensions, calculating a comprehensive confidence score to provide a basis for calibration decisions. The third stage is the calibration calculation stage. The system can choose between vehicle-side or cloud-side modes. In vehicle-side mode, the calibration decision module performs calibration calculations during system idle periods or within a delayed power-off window after the user locks the vehicle. It supports calculation interruptions and resume calculations from breakpoints. In cloud-side mode, the cloud communication module uploads data to the cloud. After the cloud completes the calculation, it sends the configuration file down via the network link. The calculation mode can be dynamically selected according to the actual situation. The fourth stage is the parameter update and verification stage. The parameter management module checks the autonomous driving system status to confirm that it is inactive. It performs a rationality check on the newly calculated calibration parameters (orthogonality, magnitude range, and variation range) before updating the parameters and recording the version information. A pre-check verification is performed before the next autonomous driving activation. If the verification fails, it automatically reverts to the previous version. The modules communicate with each other through data buses and message middleware to form a complete data loop. The system adopts a modular design, and each module can operate and be upgraded independently, providing good scalability.

[0049] Example 2: Based on the method and system for non-intrusive online calibration of lidar and camera in Example 1, the present invention will be further described and explained below.

[0050] Step S1: Data Acquisition and Storage The data acquisition module automatically collects data through a multi-source triggering mechanism and performs high-quality data acquisition when specific scenario conditions are met. Time-triggered modes include time interval-based triggering, mileage-based triggering, and periodic inspection triggering, while event-triggered modes include fault triggering, maintenance triggering, and collision triggering. When any triggering condition is met, the system marks it as needing calibration and enters the data acquisition preparation state.

[0051] The system determines whether there are road segments that meet the requirements for calibration data collection by using the global path planned for autonomous driving. The calibration data collection road segments must meet conditions such as rich road geometry features (including straight sections, curved sections, and slope changes), stable ambient lighting, moderate traffic flow, and good positioning signals. When the vehicle travels to a preset distance (e.g., 500 meters) from the target road segment, the system enters the data collection preparation state, preloads the calibration data collection module, and checks the sensor status to ensure that the data collection equipment is working properly.

[0052] Upon entering the designated road segment, the system collects a series of frames of data according to predetermined rules. Typical acquisition strategies include collecting 100 frames of data to cover a circular road segment to ensure data closure, or collecting data from a continuous 500-meter straight road segment to provide stable geometric constraints. During the acquisition process, multi-level constraints are monitored in real time. The first layer is motion state constraints, requiring the vehicle speed to be kept below 60 km / h to ensure data quality and avoid ambiguity caused by high-speed motion, vehicle acceleration to be less than 0.2g to ensure smooth motion, and lateral acceleration not exceeding 0.3 m / s² to avoid sensor data distortion caused by sharp turns. The second layer is environmental constraints, requiring a moderate number of obstacles in the scene (neither too sparse to cause insufficient features nor too dense to cause matching difficulties), good weather conditions (no rain, snow, fog, or other severe weather), and stable lighting conditions (no strong backlighting or sudden shadow changes). The third layer is system state constraints, requiring good positioning (more than 8 GPS satellites, positioning accuracy better than 0.05 meters), normal IMU data (no abnormal drift or failure), and normal sensor operation (no fault codes for LiDAR and camera). The system will only perform formal data acquisition when all hierarchical constraints are met simultaneously.

[0053] The collected data includes LiDAR point clouds, camera images, vehicle motion status (IMU, wheel speedometer, GPS), output results from the autonomous driving perception system, and trigger event information. After timestamp alignment and coordinate system transformation, the collected data undergoes preprocessing and feature extraction by the feature extraction module. The processed data is stored locally, while important data may be optionally uploaded to a cloud server via a cloud communication module. Preferably, an adaptive sampling strategy is employed, dynamically adjusting the sampling frequency based on the confidence score. When the score approaches a warning threshold, the sampling density is increased to obtain more validation data.

[0054] Step S2: Feature Extraction and Validation The feature extraction module and the multimodal verification module work together to complete the entire processing flow from raw data to verification results. In the feature extraction stage, the LiDAR point cloud is first preprocessed, including ground point filtering, outlier removal, and downsampling. Then, feature information such as edges, planes, and corners is extracted from the point cloud. At the same time, distortion correction is performed on the camera image, and feature points and descriptors are extracted from the image. Finally, the correspondence between the point cloud and image pixels is established, providing basic data for subsequent verification.

[0055] In the multimodal verification phase, the system performs verification in parallel across three dimensions. Geometric constraint verification checks the geometric consistency between the point cloud and the image using reprojection error and epipolar constraints; motion consistency verification verifies the stability of calibration parameters over time by utilizing relative pose changes between consecutive frames; and perceptual fusion verification compares the consistency of target detection boxes and semantic segmentation results based on perception results generated independently by different sensors. The verification results from these three dimensions are then weighted to obtain a comprehensive confidence score, which is used to evaluate the effectiveness of the current calibration parameters and provide a basis for subsequent calibration decisions.

[0056] Step S3: Triggering condition judgment and data accumulation The trigger judgment module continuously monitors trigger conditions, including two types: time-triggered and event-triggered. For time-triggered events, the system checks whether the time interval since the last calibration exceeds a threshold, and whether the mileage since the last calibration exceeds a threshold. If either condition is met, it is marked as requiring calibration. For event-triggered events, the system receives fault reports from the autonomous driving system, obtains maintenance information from after-sales cloud data and the OBD system, and obtains collision information from collision sensors and accident reports. If any of these events is detected, it is immediately marked as requiring calibration.

[0057] Once a system is marked as requiring calibration, it begins accumulating high-quality data samples, prioritizing data that meets quality requirements, such as data from scenarios involving straight-line driving, stable speed, and good lighting. After accumulating a sufficient number of data samples, the system enters a pre-calibration state, waiting for an appropriate time to perform calibration calculations.

[0058] Step S4: Calibration calculation (vehicle-side or cloud-based dual mode) Once the system is in a calibration-ready state, calibration calculations can be performed either on the vehicle or in the cloud. In vehicle-side calibration calculation mode, the calibration decision module first monitors the system status to determine if idle conditions are met, including CPU / GPU utilization below a threshold, the autonomous driving system not activated or not in an emergency state, the vehicle traveling at low speed or stationary, being within the time window allocated by the background task scheduler, and the time window between when the user locks the vehicle and before the vehicle power is off. After the idle conditions are met, the system retrieves accumulated data samples from local storage, executes a joint optimization algorithm to calculate calibration parameters, and outputs the optimized rotation matrix and translation vector.

[0059] The vehicle-side computing supports a breakpoint-resume mechanism. If autonomous driving activation, excessive resource consumption, high-speed driving, or emergency tasks are detected, the system immediately interrupts the calculation and saves the completed iterations, current optimization results, intermediate variables, and remaining data indexes. Once the system is idle again, it loads the state from the breakpoint and continues the calculation, avoiding redundant calculations and wasting resources. Furthermore, the system innovatively supports a post-lock calculation mode. When calibration calculations are required and the vehicle is stationary, if the user locks the vehicle, the system can delay the complete power-off of the vehicle, maintaining power to necessary computing units and continuing the calibration calculation task after the user locks the vehicle. After the calibration calculation is completed, the system normally executes the vehicle power-off process. If the calibration calculation is not completed within the preset timeout period or fails, the system abandons the current calculation and normally executes the vehicle power-off process. In this mode, the system monitors the battery level to ensure that the calculation process does not excessively consume power, and sets a maximum calculation time limit to avoid affecting the user's next vehicle use.

[0060] In cloud-based calibration calculation mode, the cloud communication module uploads calibration data to the cloud server via the vehicle network. The cloud server utilizes its powerful computing capabilities to perform calibration optimization calculations. After the calculations are completed, the configuration file is sent to the vehicle via the network link. The vehicle receives the new parameters and performs preliminary verification. The system can dynamically select the calculation mode according to the actual situation. The vehicle-side mode is suitable for scenarios with high real-time requirements and poor network conditions, while the cloud mode is suitable for scenarios requiring high-precision calculations and utilizing large-scale data. Preferably, a priority queue is used to manage calibration tasks to ensure that high-priority urgent calibration requests can be processed in a timely manner.

[0061] Step S5: Update and verify inactive state parameters The parameter management module is responsible for controlling the update of calibration parameters and strictly enforces the inactive state update mechanism. Before updating calibration parameters, it must be confirmed that the autonomous driving system is inactive, including that the vehicle is stationary (in P gear or with the handbrake engaged), the autonomous driving function is turned off, the vehicle speed is 0, and there are no pending autonomous driving activation requests. If the autonomous driving system is active or about to be activated, the system will delay parameter updates until the above conditions are met.

[0062] The parameter update process includes verifying the reasonableness of newly calculated calibration parameters (orthogonality, magnitude range, variation amplitude, etc.), confirming that the autonomous driving system is in an inactive state, writing the new parameters to the configuration file or parameter storage area, updating the parameter version number and update timestamp, recording the parameter change log (old parameters, new parameters, trigger events, calculation time, etc.), and optionally uploading the new parameters to the cloud server. Before the next activation of the autonomous driving system, the system uses the new parameters for pre-checking. If the verification fails, it automatically reverts to the previous version of the calibration parameters; if the verification passes, the new parameters are marked as usable, waiting to be used during the next activation of the autonomous driving system. This inactive state parameter update mechanism completely avoids the safety risks that may arise from real-time parameter switching, ensuring that the calibration parameters used by the autonomous driving system during activation are stable and reliable.

[0063] In summary, this invention not only reduces manual maintenance costs and achieves self-detection and non-sensory self-calibration of sensor calibration parameter degradation through automated means of maintaining vehicle status, but also improves the reliability and uptime of autonomous vehicles.

[0064] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for seamless online self-calibration of lidar and camera, characterized in that, The method includes: S1. During vehicle operation, LiDAR point cloud data and camera image data are collected through a multi-source triggering mechanism, while vehicle motion state information and the output results of the autonomous driving perception system are obtained. The multi-source triggering mechanism includes a parameter degradation model triggered by time, event, and cloud data mining. S2. Preprocess and extract features from the collected multi-source data to obtain lidar point cloud features and camera image features; S3. Based on the fusion of geometric constraints, motion consistency, and perception results, a multimodal verification model is constructed to verify the calibration parameters in multiple dimensions; S4. Determine whether calibration is needed based on the triggering conditions. If any of the multi-source triggering conditions are met, mark it as needing calibration and accumulate data samples. S5. Perform calibration parameter update calculations during system idle periods or within a delayed power-off window after the user locks the vehicle. It supports calculation interruption and breakpoint resume functions, and supports dual modes of vehicle-side calculation and cloud-based calculation. S6. Update calibration parameters only when the autonomous driving system is inactive, and verify their reasonableness.

2. The method for seamless online self-calibration of lidar and camera according to claim 1, characterized in that, In step S1, the time trigger is triggered when the time since the last calibration exceeds a preset time threshold, or when the distance traveled since the last calibration exceeds a preset mileage threshold, or when data is collected once every fixed period. The event trigger is triggered when the autonomous driving system reports calibration-related fault codes or abnormal events, or when maintenance operations affecting sensor positions are detected through after-sales cloud data, on-board diagnostic systems, or maintenance records, or when collision events are detected through collision sensors, airbag deployment records, or accident reports, or when software version updates or sensor hardware replacements affect calibration parameters.

3. The method for seamless online self-calibration of lidar and camera according to claim 1, characterized in that, In step S2, the feature extraction includes extracting edge feature points, planar feature points and corner features from the lidar point cloud, extracting ordinary feature points or deep learning feature descriptors from the camera image, and establishing a preliminary correspondence between the point cloud and image pixels.

4. The method for seamless online self-calibration of lidar and camera according to claim 1, characterized in that, In step S3, the multimodal verification model includes a geometric constraint verification submodule, a motion consistency verification submodule, and a perception fusion verification submodule. The geometric constraint verification submodule verifies the geometric consistency between the lidar point cloud and the camera image through reprojection error and epipolar constraint. The motion consistency verification submodule verifies the stability of the calibration parameters over time by using the relative pose changes between consecutive frames. The perception fusion verification submodule compares the perception results generated independently based on different sensors to detect the consistency of the target detection box and the semantic segmentation results.

5. The method for seamless online self-calibration of lidar and camera according to claim 1, characterized in that, In step S4, the trigger condition is determined by checking whether the time interval or driving mileage since the last calibration exceeds a preset threshold, receiving event information from autonomous driving system fault reports, after-sales cloud data, and collision sensor channels. If any trigger condition is met, it is marked as needing calibration and high-quality data samples are started to be accumulated.

6. The method for seamless online self-calibration of lidar and camera according to claim 1, characterized in that, In step S5, the system idle period determination conditions include CPU / GPU utilization rate being lower than a preset threshold, the autonomous driving system not being in an active state or an emergency decision-making state, the vehicle being in a low-speed driving or stationary state, a dedicated calculation time window allocated by the background task scheduler, and a time window between the user locking the car and the vehicle power-off. The calculation mode for delayed power-off after the user locks the car is as follows: if there is a calibration calculation requirement and the vehicle is stationary, and the user performs a locking operation, the system delays the complete power-off of the vehicle. After the user locks the car, the system maintains the necessary power supply to the computing units and continues to execute the calibration calculation task. After the calibration calculation is completed, the system normally executes the vehicle power-off process. If the calibration calculation... If the calculation is not completed or fails within the preset timeout period, the system will abandon the calculation and proceed with the vehicle power-off process normally. During the calculation, the battery level is monitored to ensure that the calculation process does not consume excessive power and a maximum calculation time limit is set. It also includes a cloud calibration calculation mode, in which the collected calibration-related data is uploaded to the cloud server through the vehicle network. The cloud server uses computing resources to perform calibration optimization calculations and adopts optimization algorithms and large-scale data samples. After the cloud calculation is completed, the optimized calibration parameters are sent to the vehicle through the network link by sending configuration files. After receiving the new parameters, the vehicle performs reasonableness checks and safety verifications, and stores them in the local parameter storage area after confirming that they are correct.

7. The method for seamless online self-calibration of lidar and camera according to claim 1, characterized in that, In step S5, the breakpoint continuation function immediately interrupts the calibration calculation when the autonomous driving system is activated, the CPU / GPU utilization rate exceeds the safety threshold, the vehicle enters high-speed driving, or an emergency task request is received. When interrupted, the number of completed iterations, the current optimization results, intermediate variables of the optimization algorithm, and the index of the remaining unprocessed data samples are saved. When the system enters the idle state again, the calculation state is loaded from the saved breakpoint and the optimization continues until all calculations are completed or the convergence condition is met.

8. The method for seamless online self-calibration of lidar and camera according to claim 1, characterized in that, In step S6, the inactive state parameter is updated by confirming that the vehicle is stationary, the autonomous driving function is turned off, and there are no pending activation requests before updating the calibration parameters. The newly calculated calibration parameters are written into the configuration file and the version number and update timestamp are updated, and the parameter change log is recorded. Before the next activation of the autonomous driving system, the new parameters are used for pre-check, and if the verification fails, it will automatically revert to the previous version.

9. A system for seamless online self-calibration of lidar and camera, characterized in that, The system for implementing the non-sensory online self-calibration method for LiDAR and camera as described in any one of claims 1-8 includes a data acquisition module, a feature extraction module, a multimodal verification module, a trigger judgment module, a calibration decision module, a parameter management module, and a cloud communication module. The data acquisition module is used to acquire LiDAR, camera, and vehicle status data through a multi-source triggering mechanism, which includes time-triggered, event-triggered, and cloud data mining-triggered parameter degradation models. The feature extraction module is used to preprocess and extract features from the multi-source data. The multimodal verification module is used to construct a verification model based on geometric constraints, motion consistency, and perception fusion. The trigger judgment module is used to determine whether calibration is required based on multi-source triggering conditions. The calibration decision module is used to perform vehicle-side calibration calculations during system idle periods or a delayed power-off window after the user locks the vehicle, or to perform calibration calculations in the cloud, supporting interruption and breakpoint resumption. The parameter management module is used to update calibration parameters and perform rationality verification only when the autonomous driving system is inactive. The cloud communication module is used to upload calibration data to a cloud server and receive configuration files sent from the cloud via a network link.

10. An autonomous vehicle, characterized in that, The system is equipped with the lidar and camera non-sensory online self-calibration system as described in claim 9. The system is communicatively connected to the vehicle autonomous driving controller, power management system and body control module, provides calibration parameter status information in real time and automatically updates sensor extrinsic parameters when the autonomous driving is not inactive. The system can delay the complete power-off of the vehicle after the user locks the vehicle in order to complete the calibration calculation task.

Images