Method, system for automatically calibrating autonomous driving sensors with internal calibration source

Abstract

The application discloses a method and system for automatically calibrating automatic driving sensors by using internal calibration sources. The method comprises: obtaining response data of at least two sensors on a vehicle when respective internal calibration sources are enabled, and constructing a high-dimensional response matrix representing the response differences of different sensors to internal calibration signals; based on the high-dimensional response matrix and historical interaction response data between sensors, a multi-sensor micro-error prediction model based on a graph neural network is constructed and trained; during vehicle driving, real-time signals of each sensor are collected and input into the prediction model to obtain a real-time error correction matrix, based on which dynamic calibration parameters are calculated to compensate the sensor output in real time; the compensated actual output data is obtained, the residual error between the multi-sensor fusion consistency reference is calculated, and the prediction model is updated online. The application can effectively identify and eliminate the micro-error accumulation caused by the interaction interference of multi-sensors, and improve the multi-sensor fusion accuracy and system reliability.

Description

Technical Field

[0001] This application relates to the field of calibration technology, specifically a method and system for automatically calibrating autonomous driving sensors using an internal calibration source. Background Technology

[0002] With the development of autonomous driving technology, vehicles are generally equipped with multiple sensors such as LiDAR, millimeter-wave radar, and cameras to achieve accurate perception of the surrounding environment through multi-sensor fusion. In the process of multi-sensor fusion, calibration accuracy is a key prerequisite for ensuring perception reliability and system safety.

[0003] Most existing automatic calibration technologies utilize internal calibration sources to automatically calibrate single sensors, adjusting the sensor output through internal standard signals, thus avoiding the complexity of external calibration boards and manual operation. However, these technologies are primarily geared towards single-sensor scenarios and fail to adequately consider the interactive interference issues under multi-sensor collaborative working conditions.

[0004] Specifically, although the internal calibration sources of each sensor are configured within their respective sensor systems, their calibration signals must pass through the sensor's main signal path to act on the detection unit, and this path is not inherently completely closed. When multiple types of sensors simultaneously activate their respective internal calibration sources, the calibration signals may cause weak crosstalk to other sensors through the following pathways: the calibration signals of radio frequency sensors leak electromagnetic radiation through power lines, sensor gaps, etc., and are captured by the receiving channels of other sensors; the calibration pulses of lidar generate stray light in the optical cavity or through the exit lens, creating optical path crosstalk to adjacent cameras; the transient current changes when the calibration source is activated cause voltage fluctuations on the shared power network, indirectly causing the reference level of other sensors to drift; and the weak vibrations generated by the moving parts of some sensors during the calibration process are transmitted through the vehicle structure and affect other sensing units.

[0005] The calibration deviation caused by such interference in a single instance is extremely small, far below the noise threshold of each sensor, and difficult to detect in real time. However, during long-term, high-frequency collaborative operation of multiple sensors, these micro-errors will continue to accumulate and gradually propagate, eventually leading to a gradual amplification of the calibration deviation and severely reducing the accuracy and consistency of multi-sensor fusion data.

[0006] In summary, existing technologies lack targeted solutions to address the cumulative effect of micro-errors under multi-sensor interaction interference, and improvements are urgently needed to enhance the collaborative accuracy and overall reliability of multi-sensor systems. Summary of the Invention

[0007] To address the technical problems mentioned in the background section, the purpose of this application is to provide a method, system storage medium, and program product for automatically calibrating autonomous driving sensors using an internal calibration source.

[0008] According to a first aspect of this application, a method for automatically calibrating autonomous driving sensors using an internal calibration source is provided, comprising the following steps:

[0009] S1. Obtain response data of at least two sensors on the vehicle when their respective internal calibration sources are enabled, and construct a high-dimensional response matrix to characterize the differences in the response of different sensors to the internal calibration signal.

[0010] S2, Based on the high-dimensional response matrix and the historical response data reflecting the interactive interference between the at least two sensors, construct and train a multi-sensor micro-error prediction model based on a graph neural network;

[0011] S3. During vehicle operation, real-time signals from the internal calibration sources of each sensor and sensor response data are collected and input into the multi-sensor micro-error prediction model to obtain the real-time error correction matrix corresponding to the current state. Based on the real-time error correction matrix, the dynamic calibration parameters of each sensor are calculated, and real-time compensation is performed based on the output of the corresponding sensor according to the dynamic calibration parameters.

[0012] S4. Obtain the actual output data after sensor compensation, calculate the residual between it and the multi-sensor fusion consistency benchmark, and use the residual to update the multi-sensor micro-error prediction model online.

[0013] According to a second aspect of this application, a system for automatically calibrating autonomous driving sensors using an internal calibration source is provided, the system comprising:

[0014] A high-dimensional response matrix construction module is used to acquire response data of at least two sensors on the vehicle when their respective internal calibration sources are enabled, and to construct a high-dimensional response matrix to characterize the differences in the response of different sensors to the internal calibration signal.

[0015] The model building and training module is used to build and train a multi-sensor micro-error prediction model based on graph neural networks, based on the high-dimensional response matrix and historical response data reflecting interactive interference between the at least two sensors.

[0016] The real-time compensation module is used to collect real-time signals and sensor response data from the internal calibration sources of each sensor during vehicle operation, input them into the multi-sensor micro-error prediction model to obtain the real-time error correction matrix corresponding to the current state, calculate the dynamic calibration parameters of each sensor based on the real-time error correction matrix, and perform real-time compensation based on the output of the corresponding sensor based on the dynamic calibration parameters.

[0017] The online update module is used to acquire the actual output data after sensor compensation, calculate the residual between it and the multi-sensor fusion consistency benchmark, and use the residual to update the multi-sensor micro-error prediction model online.

[0018] According to a third aspect of this application, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the preceding claims.

[0019] According to a fourth aspect of this application, a program product is provided, comprising a computer program that, when executed by a processor, implements the method as described in any of the preceding claims.

[0020] This invention constructs a high-dimensional matrix of multi-sensor responses and combines it with a micro-error prediction model based on graph neural networks to achieve real-time identification, dynamic compensation, and online adaptive correction of micro-errors caused by multi-sensor interaction interference. This can effectively eliminate the accumulation of micro-errors, improve the accuracy and consistency of multi-sensor fusion data, and enhance the calibration stability and overall reliability of autonomous driving systems in long-term operation and complex environments. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0022] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0023] Figure 1 A flowchart illustrating a method for automatically calibrating autonomous driving sensors using an internal calibration source, provided in an embodiment of this application;

[0024] Figure 2 This is a schematic diagram of the structure of the multi-sensor micro-error prediction model provided in the embodiments of this application;

[0025] Figure 3 This application provides a schematic diagram of the structure of a system for automatically calibrating autonomous driving sensors using an internal calibration source, as shown in the embodiments of this application.

[0026] Figure 4 This is a schematic diagram of the structure of the real-time compensation module 30 provided in the embodiments of this application;

[0027] Figure 5 This is a schematic diagram of the structure of the online update module 40 provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0029] In the multi-sensor systems of autonomous vehicles, various sensors such as LiDAR, millimeter-wave radar, and cameras are typically equipped with their own internal calibration sources to generate known standard signals for self-calibration. Although these calibration sources are located within their respective sensor systems, their calibration signals must pass through the sensor's main signal path to reach the detection unit; this path is not inherently fully enclosed. Therefore, when multiple sensors simultaneously activate their internal calibration sources, the actual signal received by each sensor's detection unit is a superposition of its own calibration signal and stray, crosstalk, or coupling signals from other sensors.

[0030] In response to the above technical issues, such as Figure 1 As shown in the figure, this application discloses a method for automatically calibrating autonomous driving sensors using an internal calibration source, the method comprising the following steps:

[0031] S1. Obtain response data of at least two sensors on the vehicle when their respective internal calibration sources are enabled, and construct a high-dimensional response matrix to characterize the differences in the response of different sensors to the internal calibration signal.

[0032] During the system self-test phase upon vehicle power-on or during periodically triggered online calibration moments while driving, at least two different types of sensors on the vehicle simultaneously activate their respective internal calibration sources. The response output data of each sensor in this state is synchronously collected, such as the range-Doppler spectrum response returned by millimeter-wave radar, the point cloud intensity distribution returned by lidar, and the pixel grayscale response of the camera under internal reference light illumination. It is understandable that the response data collected by each sensor at this time already includes weak interference components from other sensors introduced through electromagnetic radiation leakage, optical crosstalk, power coupling, or vibration transmission, and different sensors inherently differ in their responses to the same or similar calibration conditions.

[0033] Furthermore, the response data of each sensor in the same calibration event are time-aligned and feature-extracted, and organized according to sensor type and response feature dimension to construct a high-dimensional response matrix. Each row of this high-dimensional response matrix corresponds to a sensor, and each column corresponds to a response feature dimension of the sensor to the internal calibration signal, such as the response amplitude at a certain frequency point, the reflection intensity at a certain angle, the grayscale mean of a certain color channel, etc.

[0034] S2, Based on the high-dimensional response matrix and the historical response data reflecting the interaction interference between the at least two sensors, a multi-sensor micro-error prediction model is constructed and trained; wherein, the multi-sensor micro-error prediction model is based on a graph neural network;

[0035] The aforementioned high-dimensional response matrix characterizes the response differences of multiple sensors when their internal calibration sources are simultaneously activated, including the one-time superposition effect of interactive interference. However, the response matrix from a single acquisition only reflects the static interference state under a specific condition and cannot characterize the dynamic evolution process of micro-errors accumulating gradually during long-term collaborative operation of multiple sensors. In actual driving scenarios, factors such as the vehicle's operating environment, sensor temperature, power supply voltage, and device aging are all slowly changing. The intensity and pattern of the resulting interactive interference also drift over time, making micro-errors not constant but exhibiting complex characteristics of accumulation, propagation, and fluctuation. To eliminate this dynamic accumulation of micro-errors, it is necessary to construct a predictive model capable of predicting micro-errors caused by multi-sensor interactive interference in the current state.

[0036] Therefore, this step constructs and trains a multi-sensor micro-error prediction model based on a graph neural network, using the aforementioned high-dimensional response matrix and historical response data reflecting interactive interference between the at least two sensors. The high-dimensional response matrix provides the basic response characteristics of each sensor under interactive interference conditions, serving as the static input representation of the model. The historical response data includes multi-sensor response sequences recorded during vehicle operation under different conditions and time periods, along with corresponding known error corrections, providing the temporal supervision signals required for model training. By combining these two data points for training, the model learns the mapping relationship between current sensor response data and the corresponding error correction matrix, and models the temporal and spatial variations of micro-errors.

[0037] It should be understood that, due to the inherent graph structure characteristics of multi-sensor interference problems, this invention employs graph neural networks to construct a multi-sensor micro-error prediction model. Each sensor can be considered a node in the graph structure, and the interference relationships between different sensors (such as electromagnetic coupling strength, optical crosstalk, vibration transmission sensitivity, etc.) can be considered as edges connecting the nodes. Compared to traditional fully connected neural networks or convolutional neural networks, graph neural networks can explicitly model these irregular topological relationships between nodes. They can not only extract features from individual nodes but also aggregate information from adjacent nodes to capture the propagation path and coupling strength of interference, thereby more accurately predicting the micro-errors caused by interference.

[0038] As an example, such as Figure 2 As shown, the multi-sensor micro-error prediction model includes a spatial feature extraction network, a temporal feature extraction network, a state-aware network, and an error generation network. The specific structure and function of each sub-network are described in detail below:

[0039] A spatial feature extraction network is used to construct a graph structure reflecting the interaction and interference relationship between the input sensor response data, and to extract spatial coupling features based on the graph structure.

[0040] Specifically, this network uses a graph convolutional network as its core. After receiving real-time or historical sensor response data, it constructs a graph structure based on the preset topological relationships between the sensors (such as the sensor's installation location on the vehicle, the degree of overlap in operating frequency bands, and the spatial proximity of signal coverage areas). In this graph structure, each sensor corresponds to a graph node, and sensor pairs with interactive interference relationships are connected by edges.

[0041] By performing multi-layer graph convolution operations on the graph—that is, aggregating the feature information of the nodes of each node in each layer and performing nonlinear transformations—the spatial feature extraction network can model the spatial pattern of interference signals propagating from the source node to the affected node along the connections. For example, if millimeter-wave radar and lidar are adjacent in installation location and their operating frequency bands partially overlap, the connection weight between them is high, and the response features of the radar node will have more information transmitted to the lidar node after the convolution operation. Finally, the network outputs the spatial coupling feature vector of each sensor node, which encodes the comprehensive spatial information of the interference from other sensors to that sensor at the current moment.

[0042] A temporal feature extraction network is used to encode the temporal information in the input sensor response data and extract the temporal evolution features of micro-errors accumulating over time.

[0043] Specifically, the micro-errors caused by interactive interference are not isolated deviations generated independently in each calibration, but rather a continuous process that gradually accumulates over multiple calibration cycles. The small deviations remaining in the previous cycle become the initial bias conditions for the calibration in the next cycle, thus forming a time-dependent error accumulation.

[0044] To capture this temporal evolution pattern, a temporal feature extraction network is designed with a recurrent neural network or its variants (such as gated recurrent units or long short-term memory networks) as its main structure. This network receives sensor response data sequences within multiple consecutive calibration periods or preset time windows. Through the gating mechanism (update gate and reset gate) within the gated recurrent unit, the network can selectively remember or forget historical state information over time: for continuously increasing micro-error trends, the network maintains long-term memory to track its cumulative trajectory; for instantaneous fluctuations, the network can suppress their interference with the long-term state through the reset gate.

[0045] After time-step encoding of the input sequence, the temporal feature extraction network outputs the temporal evolution features of each sensor node. These features reflect the current accumulation stage of the micro-error (such as a rapid growth period, a slow stabilization period, or a periodic fluctuation period) and its rate of change.

[0046] A state-aware network is used to extract features from the input real-time signal and generate state features that reflect the current working state of the sensor.

[0047] Specifically, the real-time signal refers to the real-time auxiliary parameters of the internal calibration sources of each sensor, which are collected synchronously with the sensor response data in step S3. These parameters include, but are not limited to, the current transmit power, operating temperature, supply voltage, and reference frequency deviation of the calibration source. It should be understood that although the aforementioned real-time signal does not directly contain interactive interference information, it can indicate the current physical operating state of each sensor. For example, an increase in the sensor chip temperature may cause an overall shift in its response curve; a slight decrease in the transmit power of the calibration source may reduce the signal-to-noise ratio of the calibration signal; and fluctuations in the supply voltage may cause a drift in the reference level. These state changes will all affect the degree of deviation of the same interactive interference quantity in the actual sensor output; that is, the gain of the interference varies depending on the state.

[0048] State-aware networks encode multidimensional real-time signals into compact state feature vectors through fully connected layers or lightweight convolutional networks.

[0049] An error generation network is used to fuse the spatial coupling features, the temporal evolution features, and the state features to generate the real-time error correction matrix.

[0050] Specifically, the network receives output features from the three sub-networks mentioned above. A feature concatenation layer concatenates spatial coupling features, temporal evolution features, and state features along the feature dimension to form a comprehensive feature representation. This comprehensive feature is then input into a fusion mapping network composed of several stacked fully connected layers. Through multi-layer nonlinear transformations, features from different sources and with different semantics fully interact and fuse within the deep network.

[0051] The spatial coupling feature provides the spatial distribution of interference between sensors at the current moment, the temporal evolution feature indicates the accumulation trend of micro-errors on the time axis, and the state feature adjusts the magnitude and direction of the correction amount according to the current operating conditions of the sensors. The fusion of these three features allows the error generation network to comprehensively consider three dimensions: which sensors are interfering (spatial), the extent to which the interference has accumulated (temporal), and how the correction amount should be adjusted in the current state (state). The final output is a real-time error correction matrix corresponding to the number of sensors and the response dimension. It should be understood that each element of this real-time error correction matrix corresponds to the predicted micro-error correction amount generated by multi-sensor interaction interference in a certain response dimension of a particular sensor, used for the calculation of subsequent dynamic calibration parameters.

[0052] S3. During vehicle operation, real-time signals from the internal calibration sources of each sensor and sensor response data are collected and input into the multi-sensor micro-error prediction model to obtain the real-time error correction matrix corresponding to the current state. Based on the real-time error correction matrix, the dynamic calibration parameters of each sensor are calculated, and the output of the corresponding sensor is compensated in real time based on the dynamic calibration parameters.

[0053] After offline training of the multi-sensor micro-error prediction model and deployment to the onboard computing platform, the online real-time calibration and compensation phase begins. During actual vehicle operation, multiple sensors continuously collaborate, and the patterns and intensity of interactive interference dynamically change with factors such as the driving environment, sensor operating temperature, power supply status, and device aging. Therefore, it is necessary to perform online calibration periodically or under specific triggering conditions to dynamically track and eliminate the continuous accumulation of micro-errors.

[0054] Specifically, when the preset calibration cycle is reached, or when the vehicle ECU detects a significant change in environmental conditions (such as a sudden rise in temperature, increased vibration, or power supply voltage fluctuations exceeding the normal range) and triggers a calibration request, the following operations are performed simultaneously:

[0055] Through the diagnostic interfaces or independent monitoring modules of each sensor, real-time signals from the internal calibration sources of each sensor are acquired, including but not limited to auxiliary parameters characterizing the calibration source's operating status such as current transmit power, actual operating temperature, reference voltage value, and frequency offset. Simultaneously, the response output data of each sensor to its internal calibration signals under this state is obtained, such as the range-Doppler response of millimeter-wave radar, the point cloud intensity and angular distribution of lidar, and the pixel grayscale values ​​of cameras under internal reference light sources. It is understandable that these real-time acquired response data also contain interactive interference components from other sensors, such as electromagnetic leakage, optical crosstalk, power coupling, or vibration transmission.

[0056] After data acquisition, the real-time signals and sensor response data are preprocessed, including timestamp alignment of each sensor data, outlier removal, and data format normalization. The data is organized into the same data structure used during model training in step S2 and then input into the already trained multi-sensor micro-error prediction model. The multi-sensor micro-error prediction model then performs the following forward inference:

[0057] The spatial feature extraction network extracts the spatial coupling features of the interaction interference between sensors at the current moment from the sensor response data. The temporal feature extraction network encodes the response data sequence of multiple consecutive periods to obtain the temporal evolution features of micro-error accumulation. The state awareness network extracts state features reflecting the current working state of each sensor from the real-time signal. The error generation network fuses the above three types of features and outputs the real-time error correction matrix corresponding to the current state. ,in For the number of sensors, For each sensor's response feature dimension, matrix elements Indicates sensor In the The amount of prediction micro-error correction caused by multi-sensor interaction interference in each response feature dimension.

[0058] It should be understood that the dimension of the error correction matrix corresponds to the number of sensors and the response characteristic dimension of each sensor. The value of each element in the matrix represents the amount of prediction micro-error correction caused by multi-sensor interaction interference in a certain response dimension of a certain sensor under the current operating conditions.

[0059] However, the real-time error correction matrix represents the amount of deviation that the model predicts each sensor should compensate for under ideal calibration conditions, and it cannot be directly equated to the final dynamic calibration parameters required for each sensor. This is because different sensors have their own independent physical response characteristics, and their current operating state (such as effective detection distance, signal attenuation, linearity, etc.) will significantly affect the effect of the same correction amount in actual compensation. If the values ​​in the real-time error correction matrix are applied as fixed coefficients to each sensor, it may lead to overcompensation for some sensors and undercompensation for others, thus introducing new fusion inconsistencies.

[0060] In some embodiments, the dynamic calibration parameters of each sensor are calculated based on the real-time error correction matrix, including:

[0061] S31, obtain the current operating status parameters of each sensor, including the effective detection distance, signal attenuation coefficient and response linearity of the sensor, and adaptively weight and adjust the error correction component corresponding to the sensor in the real-time error correction matrix based on the current operating status parameters to generate the initial calibration offset of each sensor.

[0062] The effective detection distance refers to the actual effective sensing range of the sensor under the current environment and operating conditions, denoted as . , indicating sensor The maximum distance at which reliable detection can be maintained under current environmental conditions. The signal attenuation coefficient reflects the energy attenuation characteristics of the sensor's transmitted signal along the propagation path, denoted as... The closer the value is to 1, the smaller the attenuation and the better the signal quality. Response linearity characterizes the degree to which the sensor output signal deviates from the ideal linear relationship with the actual physical quantity, denoted as . The closer the value is to 1, the better the linearity. It should be understood that all three types of operating state parameters mentioned above can be obtained in real time through the sensor's own state monitoring interface, an independent environmental perception module, or an on-board diagnostic system.

[0063] Based on this, the error correction components corresponding to the sensor in the real-time error correction matrix are adaptively weighted according to the current operating state parameters, as follows:

[0064] First, according to the sensor Effective detection distance and signal attenuation coefficient Calculate its current signal quality confidence factor. :

[0065]

[0066] in, For sensors The reference effective detection distance under ideal calibration conditions. This confidence factor comprehensively reflects the reliability of the sensor's current signal quality: when the effective detection distance is shortened due to environmental deterioration (such as rain, fog, dust, etc.), or when the signal attenuation coefficient increases due to device aging, the confidence factor decreases accordingly.

[0067] Secondly, based on response linearity Determine the linear compensation coefficient :

[0068]

[0069] in, This is a preset linearity penalty factor used to control the sensitivity of linearity to the effect of correction. When the sensor linearity is good ( )hour, The correction amount is almost unaffected; when the linearity is poor ( )hour, The amount of correction was moderately suppressed.

[0070] Then, for the real-time error correction matrix Medium sensor Corresponding error correction component vector The sensor is obtained by performing element-wise multiplication with the signal quality confidence factor and the linear compensation coefficient. Initial calibration offset vector :

[0071]

[0072] Apply the above calculations to all Each sensor yields its initial calibration offset set. .

[0073] It should be understood that through this adaptive weighting mechanism, for signals with high quality ( near , Approaching 1) Good linearity ( For sensors close to 1), their confidence factor and linear compensation coefficient All values ​​are close to 1, and the error correction component basically retains its original value; for sensors with degraded signal quality or significant nonlinearity, and If the value is less than 1, the weight of its correction component is appropriately reduced to avoid errors introduced by low-quality signals being incorrectly amplified and compensated. This effectively solves the overcompensation or undercompensation problems caused by differences in the operating states of various sensors in traditional fixed-coefficient compensation.

[0074] S32, the initial calibration offset of each sensor is fused with the current reference calibration parameter of the sensor to obtain the dynamic calibration parameter.

[0075] Reference calibration parameters refer to the set of fundamental parameters determined by the sensor under ideal calibration conditions or factory calibration conditions, denoted as , indicating sensor The baseline calibration parameter vector under conditions of no interactive interference. These baseline parameters can be stored in the non-volatile memory of the sensor firmware or in the onboard central calibration database.

[0076] The fusion operation can employ different strategies depending on the specific application requirements. One implementation method is to linearly superimpose the reference calibration parameters with the initial calibration offset:

[0077]

[0078] in, For sensors The dynamic calibration parameter vector. It should be understood that this method is simple to calculate, highly real-time, and suitable for scenarios where the sensor's operating state is relatively stable and the correction amount is small.

[0079] As another implementation approach, a recursive fusion strategy based on Kalman filtering can be adopted, using the benchmark calibration parameters as prior estimates and the initial calibration offset as the current observation, to smoothly approximate the optimal calibration parameters through a prediction-update iterative process.

[0080] Prediction steps:

[0081] Update steps:

[0082] in, Indicates the current calibration cycle number. These are the dynamic calibration parameters from the previous cycle. These are predicted values ​​based on historical conditions. The initial calibration offset calculated for the current cycle. The Kalman gain matrix, whose value is determined by the relative magnitudes of the process noise covariance and the observation noise covariance, is used to balance the reliability of historical estimates and current observations. It should be understood that this method can effectively suppress noise and jitter in single corrections, avoiding parameter jumps caused by abrupt changes in correction values, and is suitable for scenarios where sensor operating conditions fluctuate significantly and dynamic tracking is required.

[0083] After calculating the dynamic calibration parameters for each sensor, these parameters are written into the output processing pipeline of the corresponding sensor in real time or at a preset cycle to perform online compensation and correction on the original output data. Through this real-time compensation operation, the accumulation of micro-errors caused by multi-sensor interference can be effectively eliminated, ensuring high-precision consistency of the outputs of each sensor in both spatial and temporal dimensions.

[0084] S4. Obtain the actual output data after sensor compensation, calculate the residual between it and the multi-sensor fusion consistency benchmark, and use the residual to update the multi-sensor micro-error prediction model online.

[0085] After the aforementioned real-time compensation, the outputs of each sensor should theoretically tend to be consistent and accurate. However, during the long-term operation of autonomous vehicles, the interaction interference patterns of multiple sensors are not static. With factors such as sensor aging, vehicle replacement or the addition of new electronic equipment, and significant changes in the driving environment (e.g., moving from a temperate plain to a high-temperature plateau), the electromagnetic coupling strength, optical crosstalk characteristics, and spectral distribution of power supply coupling noise between sensors may all slowly but substantially drift. This drift leads to a gradual deviation between the error patterns learned by the multi-sensor micro-error prediction model trained on historical data and the current actual interaction interference characteristics, causing the compensation effect to gradually decrease and the residual deviation to continuously increase. Therefore, this invention further establishes an online update mechanism, using the compensated actual sensor data to continuously optimize the model, enabling it to automatically adapt to long-term changes in the interaction interference environment.

[0086] Specifically, within each calibration cycle, after real-time compensation of the outputs of each sensor is completed, the actual output data of each sensor after compensation is acquired. At this point, the outputs of each sensor have been corrected by the dynamic calibration parameters and should theoretically be consistent with each other. However, due to the existence of model prediction bias, there may still be slight residual deviations between the compensated outputs. To quantify this residual deviation, a reliable reference benchmark needs to be determined.

[0087] In some embodiments, the process includes acquiring the actual output data after sensor compensation, calculating the residual between the data and the multi-sensor fusion consistency benchmark, and using the residual to update the multi-sensor micro-error prediction model online, including:

[0088] S41, obtain the actual output data of each sensor after compensation, and determine the fusion result output by the multi-sensor fusion system at the same time as the multi-sensor fusion consistency benchmark. Calculate the deviation between the actual output of each sensor after compensation and the fusion result, and perform weighted fusion on the deviation based on the confidence weight of each sensor in the fusion process to obtain the weighted residual.

[0089] A multi-sensor fusion consistency benchmark refers to the comprehensive perception result of the same target or scene output by a fusion algorithm in a multi-sensor fusion system. This fusion algorithm can be a Kalman filter, extended Kalman filter, unscented Kalman filter, particle filter, or a deep learning-based fusion network. Its input is the compensated output data of each sensor, and its output is a unified estimate of the perceived target's state (such as position, velocity, category, etc.). Because this fusion result integrates redundant observation information from multiple sensors and effectively suppresses measurement noise and blind spots of individual sensors by utilizing the complementarity between them, it has higher accuracy and consistency than any single sensor under normal conditions, and can serve as a reliable reference benchmark for measuring the output deviation of each sensor after compensation.

[0090] Assuming the sensor is within the current calibration period... The actual output data after compensation is ,in This represents the output feature dimension. The fusion result of a multi-sensor fusion system for the same target is... Then the sensor The deviation vector between the compensated output and the fusion result for:

[0091]

[0092] However, different sensors have different confidence levels during the fusion process. The fusion algorithm can dynamically assign confidence weights based on the current sensing quality of each sensor. For example, a camera's confidence decreases at night or in backlight conditions, a LiDAR's confidence decreases in rainy or foggy weather, and a millimeter-wave radar has high confidence in measuring velocity when the target is moving radially but decreases in confidence when it is moving tangentially. Let the fusion algorithm output the sensor... The confidence weight at the current moment is And it satisfies the normalization condition. (in (where the number of sensors is [number]). The bias vector is then weighted and fused to obtain a weighted residual vector. :

[0093]

[0094] By weighting the biases of each sensor with confidence levels, the biases of high-confidence sensors contribute more significantly to the residual calculation, while the contributions of low-confidence sensors are suppressed. For example, if a camera's confidence level is low due to strong backlighting, the deviation between its current level and the fused result may be large. However, this deviation is likely due to a decrease in the camera's own perception quality, rather than a failure of the interaction interference model. Through confidence weighting, the contribution of this deviation to the weighted residual is attenuated, thereby improving the accuracy and robustness of the residual calculation and preventing abnormal biases caused by transient noise or adverse operating conditions of individual sensors from dominating the residual signal.

[0095] S42, perform a time-domain continuity test on the weighted residual. When the weighted residual exceeds the threshold and is in the same direction for a preset number of consecutive periods, the weighted residual is confirmed as an effective interactive interference drift. The confirmed effective weighted residual is used to update the multi-sensor micro-error prediction model online.

[0096] After obtaining the weighted residuals, they cannot be directly used for model updates. This is because the large residuals appearing within a single calibration period have multiple sources: they may indeed originate from substantial drift in the interactive interference pattern (i.e., new features the model needs to learn), or they may originate from instantaneous disturbances such as instantaneous electromagnetic pulse interference from sensors, severe bumps and impacts during vehicle operation, brief periods of strong light in the environment, or high-power radio signals. The latter type of residual is an invalid disturbance and does not represent a true change in the interactive interference pattern. If each residual is used indiscriminately for model parameter updates, erroneous gradient signals will be introduced, causing the model to misadjust its parameters under normal conditions, deviating from the learned correct error pattern, resulting in harmful updates, and ultimately reducing the model's prediction accuracy and stability.

[0097] Therefore, this step further performs a time-domain continuity test to distinguish between effective drift and invalid disturbances. The weighted residual vector calculated for the current calibration period is then used. Take its norm (e.g., L2 norm) or infinite norm As a residual amplitude indicator, it is compared with a preset threshold. A comparison is then made. Only when the residual amplitude exceeds the threshold is it preliminarily considered that interactive interference drift may exist. However, a single exceedance of the threshold is insufficient to confirm a valid drift, and its temporal continuity needs further verification.

[0098] Set up a time-domain continuity test window, with a window length denoted as . (For example (For 3 to 5 calibration cycles). Check the current cycle and previous consecutive cycles. The residual state for each cycle. For each cycle within the window. Define residual exceedance flag :

[0099]

[0100] Meanwhile, to ensure the consistency of the residual direction, the residual vector for each period within the window is... Calculate its dominant direction (this can be achieved through principal component analysis of the residual vector or a simple sign consistency judgment). Assume continuous... If the angle between the residual vectors within a period is less than a preset angle threshold (e.g., 30 degrees), then the directions are considered to be consistent.

[0101] When both of the following conditions are met: (1) The residual flag for all periods within the window is 1, that is... (2) If the dominant direction of the residual vector within the window remains consistent, then the current weighted residual is confirmed to be an effective interactive interference drift.

[0102] After confirmation, the effective weighted residual is used as the error signal to update the multi-sensor micro-error prediction model online. Specifically, online gradient descent can be employed, using the weighted residual as a core component of the loss function. Its gradient with respect to the parameters of each network layer is calculated, and the parameters are updated along the negative gradient direction with a preset learning rate. To preserve the effective knowledge learned by the model during the update process and avoid catastrophic forgetting, an update strategy combining a small learning rate and a momentum term can be used.

[0103]

[0104] in, For the first Model parameters updated over each cycle. For learning rate, The gradient of the loss function is calculated based on the current weighted residuals. This is the momentum coefficient, used to maintain the directional inertia of parameter updates and suppress the noise effects of single gradients.

[0105] Furthermore, for residuals that fail the temporal continuity test (i.e., residuals judged as invalid disturbances), updating the model parameters is prohibited. Only the residual data and the corresponding sensor response data are recorded in the vehicle's log storage. It should be understood that this data marked as invalid disturbances can be uploaded to the cloud data center during subsequent vehicle maintenance or OTA upgrades for offline batch model retraining and iterative optimization, enabling the model to cover more diverse operating scenarios in subsequent deployments.

[0106] Understandably, the above online update process enables the multi-sensor micro-error prediction model to have the ability to continuously learn: under normal circumstances, the time-domain continuity test effectively filters out instantaneous noise and occasional interference, ensuring the safety of model updates and avoiding harmful updates; when the interactive interference environment does undergo continuous and substantial changes, the model can quickly sense and complete parameter adjustments through the effective residuals of multiple consecutive cycles to adapt to the new interference mode.

[0107] Reference Figure 3 As shown in the illustration, this application also provides a system 200 for automatically calibrating autonomous driving sensors using an internal calibration source, the system comprising:

[0108] The high-dimensional response matrix construction module 10 is used to acquire response data of at least two sensors on the vehicle when their respective internal calibration sources are enabled, and to construct a high-dimensional response matrix to characterize the differences in the response of different sensors to the internal calibration signal.

[0109] The model building and training module 20 is used to build and train a multi-sensor micro-error prediction model based on graph neural networks, based on the high-dimensional response matrix and historical response data reflecting interactive interference between the at least two sensors.

[0110] The real-time compensation module 30 is used to collect real-time signals and sensor response data from the internal calibration sources of each sensor during vehicle operation, input them into the multi-sensor micro-error prediction model to obtain the real-time error correction matrix corresponding to the current state, calculate the dynamic calibration parameters of each sensor based on the real-time error correction matrix, and perform real-time compensation based on the output of the corresponding sensor based on the dynamic calibration parameters.

[0111] The online update module 40 is used to acquire the actual output data after sensor compensation, calculate the residual between it and the multi-sensor fusion consistency benchmark, and use the residual to update the multi-sensor micro-error prediction model online.

[0112] In some embodiments, the multi-sensor micro-error prediction model includes:

[0113] A spatial feature extraction network is used to construct a graph structure reflecting the interaction and interference relationship between the input sensor response data, and to extract spatial coupling features based on the graph structure.

[0114] A temporal feature extraction network is used to encode the temporal information in the input sensor response data and extract the temporal evolution features of micro-errors accumulating over time.

[0115] A state-aware network is used to extract features from the input real-time signal and generate state features that reflect the current working state of the sensor.

[0116] An error generation network is used to fuse the spatial coupling features, the temporal evolution features, and the state features to generate the real-time error correction matrix.

[0117] In some embodiments, refer to Figure 4 As shown, the real-time compensation module 30 includes:

[0118] The status parameter acquisition unit 301 is used to acquire the current working status parameters of each sensor, including the effective detection distance, signal attenuation coefficient and response linearity of the sensor.

[0119] The adaptive weighting unit 302 is used to adaptively weight and adjust the error correction component corresponding to the sensor in the real-time error correction matrix based on the current working state parameters, so as to generate the initial calibration offset of each sensor.

[0120] The fusion unit 303 is used to fuse the initial calibration offset of each sensor with the current reference calibration parameter of the sensor to obtain the dynamic calibration parameter.

[0121] In some embodiments, refer to Figure 5 As shown, the online update module 40 includes:

[0122] The weighted residual calculation unit 401 is used to obtain the actual output data of each sensor after compensation, and to determine the fusion result output by the multi-sensor fusion system at the same time as the multi-sensor fusion consistency benchmark. It calculates the deviation between the actual output of each sensor after compensation and the fusion result, and performs weighted fusion on the deviation based on the confidence weight of each sensor in the fusion process to obtain the weighted residual.

[0123] The validity confirmation unit 402 is used to perform a time-domain continuity test on the weighted residual. When the weighted residual exceeds the threshold and is in the same direction within a preset number of consecutive periods, the weighted residual is confirmed as a valid interactive interference drift.

[0124] The update execution unit 403 is used to update the multi-sensor micro-error prediction model online using the confirmed valid weighted residuals.

[0125] This application also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the preceding claims.

[0126] The computer-readable storage medium can be a non-volatile storage medium, such as a solid-state drive, embedded multimedia memory card, flash memory, read-only memory, etc. in an in-vehicle computing platform, or a removable storage medium, such as a universal serial bus flash drive, secure digital card, etc.

[0127] This application also provides a program product comprising a computer program that, when executed by a processor, implements the method as described in any of the preceding claims.

[0128] The software product can be provided as a software installation package, firmware upgrade package, or OTA remote upgrade package. For example, a basic version of the sensor calibration software is pre-installed in the vehicle at the factory. Subsequently, when the sensor configuration changes (e.g., adding a new sensor or changing the sensor model) or the calibration algorithm is trained on cloud data to obtain a new version, the system can receive the software product upgrade package from the server via the vehicle network. The central gateway in the vehicle then distributes the package to each domain controller or perception processing unit to complete the online upgrade of the calibration program. After the upgrade is completed, the processor executes the updated computer program, thus enabling the new version of the multi-sensor automatic calibration function.

[0129] The above description is the preferred embodiment of this application. It should be noted that, for those skilled in the art, various improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A method for automatically calibrating autonomous driving sensors using an internal calibration source, characterized in that, Includes the following steps: S1. Obtain response data of at least two sensors on the vehicle when their respective internal calibration sources are enabled, and construct a high-dimensional response matrix to characterize the differences in the response of different sensors to the internal calibration signal. S2, Based on the high-dimensional response matrix and the historical response data reflecting the interactive interference between the at least two sensors, construct and train a multi-sensor micro-error prediction model based on a graph neural network; S3. During vehicle operation, real-time signals from the internal calibration sources of each sensor and sensor response data are collected and input into the multi-sensor micro-error prediction model to obtain the real-time error correction matrix corresponding to the current state. Based on the real-time error correction matrix, the dynamic calibration parameters of each sensor are calculated, and real-time compensation is performed based on the output of the corresponding sensor according to the dynamic calibration parameters. S4. Obtain the actual output data after sensor compensation, calculate the residual between it and the multi-sensor fusion consistency benchmark, and use the residual to update the multi-sensor micro-error prediction model online. The multi-sensor micro-error prediction model includes: A spatial feature extraction network is used to construct a graph structure reflecting the interaction and interference relationship between the input sensor response data, and to extract spatial coupling features based on the graph structure. A temporal feature extraction network is used to encode the temporal information in the input sensor response data and extract the temporal evolution features of micro-errors accumulating over time. A state-aware network is used to extract features from the input real-time signal and generate state features that reflect the current working state of the sensor. An error generation network is used to fuse the spatial coupling features, the temporal evolution features, and the state features to generate the real-time error correction matrix; The dynamic calibration parameters of each sensor are calculated based on the real-time error correction matrix, including: S31, obtain the current operating status parameters of each sensor, including the effective detection distance, signal attenuation coefficient and response linearity of the sensor, and adaptively weight and adjust the error correction component corresponding to the sensor in the real-time error correction matrix based on the current operating status parameters to generate the initial calibration offset of each sensor. S32, the initial calibration offset of each sensor is fused with the current reference calibration parameter of the sensor to obtain the dynamic calibration parameter.

2. The method for automatically calibrating autonomous driving sensors using an internal calibration source according to claim 1, characterized in that, Acquire the actual output data after sensor compensation, calculate the residual between it and the multi-sensor fusion consistency benchmark, and use this residual to update the multi-sensor micro-error prediction model online, including: S41, obtain the actual output data of each sensor after compensation, and determine the fusion result output by the multi-sensor fusion system at the same time as the multi-sensor fusion consistency benchmark. Calculate the deviation between the actual output of each sensor after compensation and the fusion result, and perform weighted fusion on the deviation based on the confidence weight of each sensor in the fusion process to obtain the weighted residual. S42, perform a time-domain continuity test on the weighted residual. When the weighted residual exceeds the threshold and is in the same direction for a preset number of consecutive periods, the weighted residual is confirmed as an effective interactive interference drift. The confirmed effective weighted residual is used to update the multi-sensor micro-error prediction model online.

3. A system for automatically calibrating autonomous driving sensors using an internal calibration source, characterized in that, The system includes: A high-dimensional response matrix construction module is used to acquire response data of at least two sensors on the vehicle when their respective internal calibration sources are enabled, and to construct a high-dimensional response matrix to characterize the differences in the response of different sensors to the internal calibration signal. The model building and training module is used to build and train a multi-sensor micro-error prediction model based on graph neural networks, based on the high-dimensional response matrix and historical response data reflecting interactive interference between the at least two sensors. The real-time compensation module is used to collect real-time signals and sensor response data from the internal calibration sources of each sensor during vehicle operation, input them into the multi-sensor micro-error prediction model to obtain the real-time error correction matrix corresponding to the current state, calculate the dynamic calibration parameters of each sensor based on the real-time error correction matrix, and perform real-time compensation based on the output of the corresponding sensor based on the dynamic calibration parameters. The online update module is used to acquire the actual output data after sensor compensation, calculate the residual between it and the multi-sensor fusion consistency benchmark, and use the residual to update the multi-sensor micro-error prediction model online. The multi-sensor micro-error prediction model includes: A spatial feature extraction network is used to construct a graph structure reflecting the interaction and interference relationship between the input sensor response data, and to extract spatial coupling features based on the graph structure. A temporal feature extraction network is used to encode the temporal information in the input sensor response data and extract the temporal evolution features of micro-errors accumulating over time. A state-aware network is used to extract features from the input real-time signal and generate state features that reflect the current working state of the sensor. An error generation network is used to fuse the spatial coupling features, the temporal evolution features, and the state features to generate the real-time error correction matrix; The real-time compensation module includes: The status parameter acquisition unit is used to acquire the current operating status parameters of each sensor, including the effective detection distance, signal attenuation coefficient, and response linearity of the sensor. An adaptive weighting unit is used to adaptively weight and adjust the error correction component corresponding to the sensor in the real-time error correction matrix based on the current operating state parameters, so as to generate the initial calibration offset of each sensor. The fusion unit is used to fuse the initial calibration offset of each sensor with the current reference calibration parameters of the sensor to obtain the dynamic calibration parameters.

4. A system for automatically calibrating autonomous driving sensors using an internal calibration source according to claim 3, characterized in that, The online update module includes: The weighted residual calculation unit is used to obtain the actual output data of each sensor after compensation, and to determine the fusion result output by the multi-sensor fusion system at the same time as the multi-sensor fusion consistency benchmark. It calculates the deviation between the actual output of each sensor after compensation and the fusion result, and performs weighted fusion on the deviation based on the confidence weight of each sensor in the fusion process to obtain the weighted residual. The validity confirmation unit is used to perform a time-domain continuity test on the weighted residual. When the weighted residual exceeds the threshold and is in the same direction for a preset number of consecutive periods, the weighted residual is confirmed as a valid interactive interference drift. An update execution unit is used to update the multi-sensor micro-error prediction model online using the confirmed valid weighted residuals.

5. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-2.

6. A program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-2.

Images