IMU (Inertial Measurement Unit) online self-adaptive calibration method, calibration device and automatic driving system

Abstract

The invention discloses an IMU online adaptive calibration method and calibration device and an automatic driving system, and relates to the technical field of deep learning, automatic driving and navigation. Comprising the following steps: acquiring original measurement data from an IMU in real time; inputting the original measurement data into an error prediction model to output predicted IMU error parameters, wherein the error prediction model adopts a hybrid neural network architecture, is trained and updated based on an end-to-end framework containing a differentiable physical model, and is configured to execute offline training in an offline stage and execute adaptive updating in an online application stage; and correcting the original measurement data by using the IMU error parameter and executing dead reckoning to generate a calibrated motion state track. According to the invention, a static off-line calibration system is improved into a dynamic on-line adaptive system, so that the long-term robustness and reliability are remarkably improved. A differentiable physical model is introduced as a training bridge to realize an end-to-end physical model training normal form, so that the training precision can be improved and the training convergence can be accelerated.

Description

Technical Field

[0001] This application relates to the fields of deep learning, autonomous driving and navigation technology, specifically to an online adaptive calibration method for an IMU, a calibration device and an autonomous driving system. Background Technology

[0002] In the closed-loop control system of autonomous driving systems for agricultural vehicles, drones, and other applications, high-precision positioning and navigation are the cornerstones of autonomous operation. Among these, the IMU (Inertial Measurement Unit), with its high-frequency sampling and autonomous operation, has become the core sensor for achieving all-weather continuous positioning, attitude calculation, and vehicle dynamics control. However, the dynamic error parameters of low-cost IMUs are highly susceptible to interference from the harsh working conditions of farmland. Severe mechanical vibrations and temperature drift caused by prolonged operation can lead to real-time, nonlinear changes in these error parameters, and traditional, one-time offline calibration methods cannot effectively compensate for such dynamic drift.

[0003] This defect is particularly severe in scenarios where GNSS (Global Navigation Satellite System) signals are limited (such as orchards, greenhouses, or tall crop areas): once external signal correction is lost, the integration error based on the IMU will accumulate rapidly over time, which will not only cause a sharp drop in positioning accuracy, but also cause serious deviations in vehicle attitude and control calculations, becoming a major technical bottleneck restricting the fully automated operation of agricultural machinery.

[0004] Existing technologies primarily employ a filter-based multi-source fusion framework. These methods utilize Kalman filtering (KF) and its variants (such as Extended Kalman Filter (EKF) and Enhanced Extended Kalman Filter (EEKF)) as their core, incorporating external "true value" observations from GNSS, RF inertial navigation, etc., to correct the integration error of the IMU in real time. To further improve accuracy, existing research also introduces multiple constraints and adaptive mechanisms: for example, using the IMU array in conjunction with the ICP (Iterative Closest Point) algorithm to calibrate geometric installation errors; employing adaptive filtering algorithms such as Variational Bayesian Adaptive Kalman Filter (VBAKF) to dynamically estimate process noise and measurement noise parameters; and utilizing kinematic constraints such as Zero Velocity Update (ZUPT) as virtual measurements.

[0005] However, the aforementioned technical approaches have fundamental limitations: First, their core remains confined to the "fusion correction" paradigm, relying on external information to passively correct the state within the filter framework, making it difficult to accurately model and predict the complex nonlinear error mechanisms within the sensor (such as the cross-coupling effect of acceleration on the gyroscope). Second, after offline calibration or training, the model parameters are typically statically fixed during deployment. However, under the harsh conditions of farmland operations, factors such as continuous engine overheating and sensor aging can trigger dynamic error drifts unforeseen during training. Static models cannot adapt to such real-time, non-fixed environmental changes, leading to a significant degradation in calibration performance over time. Therefore, a technical solution with online adaptive capabilities is urgently needed to address these issues. Summary of the Invention

[0006] The purpose of this application is to provide an online adaptive calibration method, calibration device, and autonomous driving system for an IMU, which can realize online adaptive dynamic updating of the error prediction model. At the same time, by setting an end-to-end physical model training paradigm, the training accuracy can be improved and the training convergence can be accelerated.

[0007] To achieve the above objectives, a first aspect of this application provides an online adaptive calibration method for an IMU, the calibration method comprising: acquiring raw measurement data from the IMU in real time; inputting the raw measurement data into an error prediction model to output predicted IMU error parameters, wherein the error prediction model adopts a hybrid neural network architecture and is trained and updated based on an end-to-end framework including a differentiable physical model, and is configured to perform offline training in an offline phase and adaptive updates in an online application phase; and correcting the raw measurement data using the IMU error parameters, and performing dead reckoning using the corrected data to generate a calibrated motion trajectory.

[0008] This application also provides an online adaptive calibration device for an IMU, comprising: a data acquisition module for acquiring raw measurement data from the IMU in real time; an error prediction module for inputting the raw measurement data into an error prediction model to output predicted IMU error parameters, wherein the error prediction model adopts a hybrid neural network architecture and is trained and updated based on an end-to-end framework including a differentiable physical model, and is configured to perform offline training in an offline phase and adaptive updates in an online application phase; and a dead reckoning module for correcting the raw measurement data using the IMU error parameters and performing dead reckoning using the corrected data to generate a calibrated motion trajectory.

[0009] This application also provides an autonomous driving system, comprising: a sensor module for acquiring raw IMU measurement data and GPS positioning signals; and an onboard intelligent computing platform employing an asynchronous processing architecture, including a first processing unit for performing real-time inference tasks and a second processing unit for performing background learning tasks, wherein the processing frequency of the second processing unit is lower than that of the first processing unit, wherein the first processing unit is configured to perform: correction based on IMU error parameters and dead reckoning steps in the aforementioned IMU online adaptive calibration method; and the second processing unit is configured to perform: adaptive update steps in the aforementioned IMU online adaptive calibration method, and asynchronously transmit the updated model parameters to the first processing unit.

[0010] Through the above technical solution, this application improves the traditional static offline calibration system into a dynamic online adaptive system, achieving online adaptation to combat real-time drift and significantly improving the long-term robustness and reliability of the system. Furthermore, this invention fundamentally departs from the traditional fusion approach, introducing an innovative differentiable physical model as a training bridge to achieve an end-to-end physical model training paradigm, which can improve training accuracy and accelerate training convergence.

[0011] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0012] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings: Figure 1 A schematic flowchart of the IMU online adaptive calibration method according to this application is shown; Figure 2 A framework diagram of the offline training phase according to this application is shown; Figure 3 A flowchart of the offline training process according to this application is shown; Figure 4a This is a simulation diagram of the zero-bias instability of the accelerometer. Figure 4b This is a simulation diagram of the zero-bias instability of a gyroscope. Figure 5a A schematic diagram showing the comparison of the outputs of an ideal IMU and an IMU with errors on the X-axis for accelerometer data simulation. Figure 5b A schematic diagram showing the comparison of the outputs of an ideal IMU and an IMU with errors on the Y-axis for accelerometer data simulation; Figure 5c A schematic diagram showing the comparison of the Z-axis outputs of an ideal IMU and an IMU with errors in the simulation of acceleration data; Figure 5d A schematic diagram comparing the outputs of an ideal IMU and an IMU with errors on the X-axis for angular velocity data simulation; Figure 5e A schematic diagram comparing the outputs of an ideal IMU and an IMU with errors on the Y-axis for angular velocity data simulation. Figure 5f A schematic diagram comparing the Z-axis outputs of an ideal IMU and an IMU with errors for angular velocity data simulation. Figure 6 A comparison diagram of dead reckoning trajectories based on this application is shown; Figure 7 A schematic diagram of the structure of the IMU online adaptive calibration device according to this application is shown; Figure 8 A schematic diagram of the structure of an autonomous driving system according to this application is shown; Figure 9 A diagram of the vehicle hardware system architecture according to this application is shown; Figure 10 A diagram of the online application phase-asynchronous dual-core architecture according to this application is shown. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0014] It should be noted that if the embodiments of this application involve directional indications (such as up, down, left, right, front, back, etc.), these directional indications are only used to explain the relative positional relationships and movement of the components in a specific posture (as shown in the attached figures). If the specific posture changes, the directional indications will also change accordingly. Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" can explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0015] First, this application provides an online adaptive calibration method 100 for an IMU, configured to perform a real-time estimation task of IMU error parameters, such as... Figure 1 As shown in the flowchart, the calibration method 100 may include steps S110-S130.

[0016] Step S110: Obtain raw measurement data from the IMU in real time.

[0017] Step S120: Input the raw measurement data into the error prediction model to output the predicted IMU error parameters. The error prediction model employs a hybrid neural network architecture and is trained and updated based on an end-to-end framework containing a differentiable physical model. It is configured to perform offline training in the offline phase and adaptive updates in the online application phase.

[0018] Step S130: Correct the original measurement data using IMU error parameters, and perform dead reckoning using the corrected data to generate a calibrated motion trajectory.

[0019] In one embodiment, the specific process of constructing and training the error prediction model may include the following steps S210-S230.

[0020] The differentiable physical model module is a pure mathematical function written using only tensor operations within an automatic differentiation framework such as PyTorch. It is used in step S210 to match the frequency difference between the input data and the calibration data. This module essentially constructs a differentiable physical model layer embedded in the neural network computation graph. Through this design, this layer not only rigorously performs kinematic integral operations but also acts as a bridge for gradient propagation, enabling sparse GPS monitoring signals to cross time steps and be propagated back to the high-frequency IMU error prediction network. This effectively solves the frequency mismatch problem between the input data and the calibration data in step S210.

[0021] Hybrid neural networks consist of a base network module and an online adapter module. Their functions are: during the offline training phase, they are fully trained to learn the general physical characteristics of the error; and during the online application phase, they receive the raw measurement data from the IMU and output predicted IMU error parameters. p final Simultaneously, online adaptive updates are performed, as detailed in steps S220-S230.

[0022] Step S210: Construct an end-to-end training framework based on a differentiable physical model.

[0023] The end-to-end training framework can be used to match the frequency differences between input and calibration data. For example, if the input data is high-frequency IMU data and the calibration data is low-frequency GPS data, due to the frequency mismatch between the IMU (high frequency, such as 200Hz) and GPS (low frequency, such as 10Hz), a time-series recursive process is needed to calculate the IMU motion between two GPS observation times. Specifically, the differentiable physical model module can execute the following steps S211-S213: Step S211: Based on the ratio of the IMU sampling frequency to the GPS sampling frequency, determine the number N of IMU data frames contained between two adjacent GPS observations.

[0024] This step is to determine the number of recursive steps, that is, to calculate the number of IMU data frames N between two adjacent GPS observations based on the ratio of the sampling frequency of the IMU to that of the GPS, for example, N=20.

[0025] Step S212 involves performing a kinematic state integral recursion process with N time steps for the N frames of IMU data. This step aims to perform an integral recursion based on differentiable operations. During the recursion process with N time steps, the model strictly follows Newton's laws of motion and kinematic equations to reproduce the physical integration process from acceleration / angular velocity to position / velocity. For example, this differentiable kinematic state update can be achieved through tensor operations in an automatic differentiation framework. Specifically, the differentiable physical model module is implemented using discretized kinematic equations based on the first-order Euler method (or trapezoidal method). In an automatic differentiation framework (such as PyTorch), all variables (quaternions, velocity, position) are constructed as tensors with computable gradients.

[0026] In step S213, at each time step of the kinematic state integral recursion process, the current raw measurement data is corrected using the IMU error parameters, and the motion state is updated using a differentiable kinematic formula. At the same time, the calibrated global acceleration is output to construct the physical residual loss.

[0027] This step is for performing motion state updates and loss construction. That is, at each time step... k The following specific mathematical logic is executed: 1. Error Correction: First, the IMU's comprehensive error parameters at the current moment are predicted using a hybrid neural network. It should be noted that this invention employs residual learning to map the sensor's internal additive errors (such as zero bias) and multiplicative errors (such as scaling factor error and nonlinearity error) into a unified "equivalent additive correction quantity." Therefore, during calibration, the system subtracts the corresponding comprehensive error parameters predicted by the model from the sensor's output angular velocity and acceleration, thereby eliminating the influence of multiple sources of error such as zero bias drift, scaling factor deviation, and nonlinear distortion.

[0028] 2. Attitude Update: The carrier attitude is recursively updated using the calibrated angular velocity. The system uses quaternions to represent spatial attitude, following the principle of discretized kinematic integration (such as the first-order Picard integral method). Combining the IMU sampling time interval and the current angular velocity vector, the differential increment of the attitude quaternion is calculated and superimposed on the attitude at the previous moment. Subsequently, the updated quaternion vector is normalized, and its magnitude is constrained to a unit length to ensure the mathematical validity of the attitude representation.

[0029] 3. Coordinate Rotation and Global Acceleration Calculation: Based on the updated attitude quaternions, the rotation matrix for transforming from the sensor body coordinate system to the navigation global coordinate system is analytically calculated. Using this rotation matrix, the calibrated body acceleration vector is projected and transformed into the global coordinate system. Then, mathematical compensation is performed on the gravitational acceleration vector in the global coordinate system (i.e., removing the influence of the gravity component), thereby analytically determining the true acceleration of the carrier relative to the ground, i.e., the calibrated global acceleration.

[0030] 4. Velocity and Position Integration: Finally, a discretization integration operation is performed according to Newton's laws of motion to update the motion state. Specifically, the calibrated global acceleration is multiplied by the sampling time interval and accumulated to the velocity vector of the previous moment, thereby updating the current velocity. v k The current position is updated by multiplying the velocity from the previous moment (or the average velocity combined with the current acceleration) by the sampling time interval and accumulating the sum to the position vector from the previous moment. p k .

[0031] After iterating through the above four sub-steps for N time steps, the module outputs the final predicted position. p N and with GPS true value p GPS calculate L position The loss is also output, along with the calibrated global acceleration sequence for the entire process, used to compare with the true global acceleration. a GPS Combine calculation of physical residual loss L physics_mean .

[0032] In summary, the complete physical modeling process includes: 1. Perform kinematic recursion: For N frames of raw IMU measurement data within two adjacent GPS observation time intervals, perform a kinematic state integration recursion process with N time steps; 2. Establish end-to-end mapping: Using differentiable kinematics formulas, establish the mapping from IMU error parameters to the final predicted position. p N Differentiable mapping relationship; 3. Output intermediate physical quantities: Calculate and output the calibrated acceleration at each intermediate time step k in the recursive process. a corr,k ; 4. Constructing physical ground truth: By analyzing the GPS ground truth location... p GPS The data is smoothed, filtered, and numerically differentiated to obtain the global acceleration reference value. aGPS The Savitzky-Golay filter is preferably used for time-series positioning data based on GPS signals. p GPS Joint smoothing and numerical differentiation are performed to obtain a high signal-to-noise ratio global acceleration reference. a GPS ; 5. Construct a system that includes location loss. L position and physical residual loss L physics_mean Composite loss function L total Comprehensive constraint calibration accuracy; Through this physical modeling process, the system achieves end-to-end mapping from IMU error parameters to the final trajectory position, constructing a differentiable gradient backhaul channel.

[0033] This design integrates the physical model as a differentiable hierarchical seamless neural network architecture, which not only ensures the physical consistency of motion inference but also solves the problem that pure data-driven methods cannot effectively supervise training using sparse GPS signals and high-frequency IMU errors.

[0034] Step S220: Based on the basic network modules in the hybrid neural network model, perform offline training in the offline phase.

[0035] The basic network module is configured to extract the temporal features of the raw IMU measurement data. It can employ a deep neural network architecture with time-series processing capabilities, including but not limited to recurrent neural networks (RNNs), such as LSTM (Long Short-Term Memory) or GRU (Gated Recurrent Unit), transformer networks, temporal convolutional networks (TCNs), or state-space models (SSMs), to learn the general physical characteristics of IMU error. During the offline training phase, the input data is a sequence of raw IMU measurement data within a time window. Sufficient training can be performed on a PC or server to learn and characterize the general physical characteristics of IMU error and output the basic error prediction values. p baseAfter offline training is completed, its weights are deployed to the in-vehicle intelligent computing platform (or inference processing unit) and remain frozen in the subsequent online application stage (i.e., its parameters are no longer updated over time), and are used only as a static network for extracting general physical features of the IMU.

[0036] Offline training can be performed on a PC or server, as shown in the example below. Figure 2 The diagram shows the framework for the offline training phase. Specifically, the key implementation of this invention employs a strategy of "simulation pre-training + real-world fine-tuning" to execute the specific training process, which can be referred to as follows: Figure 3 The flowchart shown illustrates the offline training process of "simulation pre-training + real-world fine-tuning" to address the problem of obtaining the true value of real-world IMU errors. It should be noted that pre-training and basic training in this paper have essentially the same meaning and will not be specifically distinguished in the following descriptions. Specifically, it can include the following steps S221-S222: Step S221: Using the generated simulation dataset, perform basic training on the hybrid neural network module using a composite loss function to obtain a first training model that has learned the general error physical properties.

[0037] In this step, the hybrid neural network module needs to be fully pre-trained using a simulation dataset so that it learns the general physical laws governing IMU errors.

[0038] The simulation dataset includes paired IMU ground truth motion data and raw measurement simulation data. The IMU ground truth motion data includes ideal acceleration and ideal angular velocity. The raw measurement simulation data is generated by superimposing error characteristics onto the IMU ground truth motion data within a vehicle dynamics simulation environment, based on a defined IMU error model. For example, in professional vehicle dynamics simulation software (such as CarSim, MATLAB, Simulink, or their co-simulation environments), a complex ideal "ground truth trajectory" (including acceleration / deceleration, turning, and bumpy road conditions) can be set using a high-precision driver model or path-following control, and the ideal acceleration and angular velocity of this ground truth trajectory at each moment can be output.

[0039] Then, an IMU error model can be established to simulate one or more of the following error characteristics: bias instability, scale factor error, inter-axis non-orthogonality error (or installation error), Gaussian white noise, accelerometer nonlinearity error, and gyroscope sensitivity error to acceleration. This allows for the simulation of key sensor error characteristics; for example, a random walk model can be optionally used to simulate bias instability. Figure 4a The accelerometer shown and as Figure 4bThe simulation illustrates the zero-bias instability of a gyroscope. By using this error model, errors are superimposed on ideal acceleration and angular velocity to generate IMU data containing errors. It is important to emphasize that the simulation data in this embodiment is strictly constructed based on the datasheet specifications of typical low-cost MEMS IMUs (such as automotive-grade sensors). Specifically, when setting the IMU error model parameters, physical indicators such as noise density, Allan variance, and sensitivity tolerance of the real hardware are referenced. Based on this, to improve the adaptability (generalization ability) of the pre-trained model to different individual differences and harsh operating conditions, this invention adopts a domain randomization strategy. That is, within the typical range of the above physical indicators, parameters such as zero bias, scaling factor, nonlinearity, and noise intensity are randomly sampled to generate massive amounts of virtual training data covering various error characteristics and their corresponding ground truth labels, such as... Figures 5a-5c The acceleration data shown are located on the X, Y, and Z axes, respectively, and as shown in the figure. Figures 5d-5f The angular velocity data shown are located on the X, Y, and Z axes, respectively, which are used to fully pre-train the hybrid neural network.

[0040] You can refer to, for example Figure 6 The dead reckoning trajectory comparison chart shown illustrates the dead reckoning calibration effect of this invention during the offline training phase, using only the generated simulation dataset to pre-train the hybrid neural network. The chart simulates the vehicle's trajectory over 300 seconds. Wherein: the black solid line (true trajectory): represents the vehicle's actual movement path; the red dotted line (uncalibrated trajectory): represents the trajectory obtained by directly integrating the original measurement data, which includes zero bias, scaling factors, and nonlinear errors. As shown, without model intervention, accumulated errors cause the trajectory to deviate significantly from the true value; the green dashed line (pre-trained model calibration): represents the recalculated trajectory after correcting the original data using an error prediction model pre-trained only on simulation data. The comparison results show that the calibrated trajectory basically overlaps with the true trajectory, and the drift is significantly reduced. This verifies the effectiveness of the "simulation pre-training" strategy, proving that the model has successfully learned the general physical laws of sensor errors, providing a good initial parameter foundation for subsequent online fine-tuning and adaptation on real hardware.

[0041] Step S222: Using the collected real dataset, fine-tune the first training model using a composite loss function to obtain a second training model adapted to specific hardware noise characteristics. The amount of data in the real dataset should be smaller than that in the simulation dataset.

[0042] This step is to fine-tune the measured data. That is, by using small batches (e.g., tens of minutes) of IMU and RTK-GPS data collected on a real vehicle, the pre-trained hybrid neural network is fine-tuned so that the model can adapt to the unique noise characteristics of a specific hardware vehicle while retaining general physical knowledge.

[0043] In addition, to address position loss L position The problem of excessively long gradient chains leading to training instability (vanishing / exploding gradients) was addressed by introducing physical residual loss based on the PINN (Physics-Informed Neural Network) concept. L physics This constitutes a composite loss function. L total That is, the composite loss function. L total It is constructed by weighting the location loss and physical residual loss, as shown in the following formula: L total = λ 1 L position + λ 2 L physics_mean , in, L position The position loss, used to characterize the deviation between the predicted trajectory position derived from the differentiable physical model and the true GPS position, needs to be calculated at the final point of N time steps. It is obtained by comparing the predicted trajectory... p N GPS True Value p GPS The calculation yields, for example, MSE ( p N , p GPS ), used to constrain the accuracy of macroscopic trajectories.

[0044] in, L physics_mean The aggregated value of the physical residual loss is used to characterize the mean deviation between the calibrated global acceleration output by the differentiable physical model and the true global acceleration. For example, this can be achieved by comparing the calibrated acceleration... a corr,k With true global acceleration a GPS The calculation yielded the result. L physics_mean It is the physical residual loss calculated at each step (or multiple intermediate steps) of the N-step recursive process. Lphysics,k The mean or aggregate value, where, L physics,k Indicates the first k The instantaneous physical consistency error over a time step is calculated using the following formula: L physics,k =MSE( a corr,k , a GPS,k ), a GPS,k For the first k The reference true global acceleration (i.e., high-frequency true acceleration) corresponding to each time step is obtained from GPS positioning data through smoothing filtering and numerical differentiation. This loss provides N parallel gradient shortcuts for training the hybrid neural network model. During training, the hybrid neural network is forced to first learn the physical laws, thereby stabilizing the data obtained from GPS positioning data. L position The gradient vanishing problem caused by long chains.

[0045] in, λ 1 and λ 2 represents the weighting coefficient. These two hyperparameters are used to balance the physical dimensions and numerical magnitudes of different loss terms.

[0046] Step S230: Based on the online adapter module in the hybrid neural network model, perform adaptive updates during the online application phase.

[0047] The online adapter module can employ a lightweight multilayer perceptron, such as a small multilayer perceptron (MLP), to learn the residuals or real-time drift predicted by the base network. Its parameter count is extremely small, and the weights can be updated online, allowing for rapid updates at the processing unit. Its input can be raw IMU measurement data, the output of the base network, or both, and it outputs a real-time drift correction. p For example, its final error parameter. p final = p base + p .

[0048] Specifically, the online adaptive update is configured to update the parameters of the online adapter module in the error prediction model using physical residual loss, and may include the following steps S231-S234: Step S231: Obtain time-series positioning data based on GPS signals, and perform smoothing filtering and numerical differentiation processing on it to determine the true global acceleration.

[0049] Step S232: Process the raw measurement data using the error prediction model and the differentiable physical model to determine the calibrated global acceleration.

[0050] Step S233: Construct a physical residual loss function that includes the difference between the true global acceleration and the calibrated global acceleration.

[0051] Step S234: Freeze the parameters of the basic network modules in the hybrid neural network, calculate the gradient based on the physical residual loss function, and only update the parameters of the online adapter module.

[0052] Compared with the prior art, the present invention has the following significant advantages: 1. This invention reconstructs the underlying paradigm of IMU error calibration, breaking through the limitations of the linear assumptions of traditional filters: Existing mainstream technologies (such as EKF / VBAKF) essentially treat GPS as an external observation within the filter framework to passively correct the state, making it difficult to accurately model the complex and highly nonlinear error mechanisms within the IMU (such as the cross-coupling effect of acceleration on the gyroscope). This invention abandons the traditional "fusion correction" approach and innovates the "end-to-end differentiable physical model" training paradigm. By constructing the kinematic integral formula as a differentiable layer in the neural network, the problem of estimating non-differentiable error parameters is transformed into a differentiable trajectory prediction problem, allowing the position error gradient to pass through the long-period integral chain and propagate back to directly drive the network to learn the complex nonlinear error characteristics inside the sensor.

[0053] 2. A physically constrained gradient shortcut is constructed to solve the training problem of long sequence integrals: Addressing the "vanishing gradient" or "exploding gradient" problem faced by purely data-driven methods in long chain integrals, this invention introduces a physical residual loss based on the PINN concept in both offline training and online adaptation. This mechanism utilizes the acceleration constraints of intermediate layers to provide a direct gradient backpropagation shortcut for the neural network, forcing the network to output physically consistent results that conform to Newton's laws of motion. This design accelerates model convergence and ensures accurate calculation of high-dimensional IMU error parameters even in the absence of ground truth pose values, relying solely on sparse GPS position signals.

[0054] 3. Achieved Online Adaptation under Low Computing Power: Addressing the limitations of traditional static models in handling sensor aging and temperature drift, this invention proposes an online adaptive mechanism combining an asynchronous dual-core architecture with efficient parameter fine-tuning. Unlike traditional adaptive filters that only adjust noise parameters (Q / R array), this invention employs a strategy of "freezing the base network + updating only the lightweight adapter" to achieve real-time model parameter updates on low-computing-power edge computing devices (such as Raspberry Pi). This system can continuously utilize GPS information to combat dynamic environmental interference unseen during the training phase, improving the robustness and reliability of low-cost sensors throughout their entire lifecycle.

[0055] 4. A low-cost data solution with a closed-loop "simulation-real-world measurement" process has been established: This invention adopts a transfer learning strategy of "simulation pre-training + real-world fine-tuning," solving the problems of obtaining the true value of IMU internal errors in real-world scenarios and the high cost of high-precision calibration equipment (such as turntables and high-precision fiber optic inertial navigation systems). The system utilizes massive amounts of simulation data to solidify general physical error patterns and uses a small amount of real-world measurement data to align the noise characteristics of specific hardware, reducing the data acquisition threshold and implementation cost of high-precision inertial navigation algorithms.

[0056] On the other hand, this application also provides an IMU online adaptive calibration device 300, such as Figure 7 As shown in the structural diagram, it may include: Data acquisition module 310 is used to acquire raw measurement data from the IMU in real time; Error prediction module 320 is used to input raw measurement data into an error prediction model to output predicted IMU error parameters. The error prediction model is built on an end-to-end framework of a hybrid neural network model and a differentiable physical model, and is configured to perform offline training in the offline phase and adaptive updates in the online application phase. The dead reckoning module 330 is used to correct the original measurement data using IMU error parameters and to perform dead reckoning using the corrected data to generate a calibrated motion trajectory.

[0057] For details on the specific implementation method and beneficial effects of the IMU online adaptive calibration device 300, please refer to the above description of the IMU online adaptive calibration method 100, which will not be repeated here.

[0058] On the other hand, this application also provides an autonomous driving system 400, which can serve as an application carrier for the aforementioned IMU online adaptive calibration method, such as an agricultural vehicle or a laboratory vehicle. Figure 8 As shown in the structural diagram, the autonomous driving system 400 may include a sensor module 410 and an on-board intelligent computing platform 420.

[0059] The sensor module 410 is used to acquire raw measurement data from the IMU and GPS positioning signals. This may include, for example, a low-cost IMU and an RTK-GPS module kit (such as the ZED-F9P, which includes a base station and a rover). Both are securely mounted and connected to the same clock source to ensure accurate timestamp synchronization.

[0060] The in-vehicle intelligent computing platform 420 can adopt an asynchronous processing architecture, including a first processing unit 421 for performing real-time inference tasks and a second processing unit 422 for performing background learning tasks. It is important to emphasize that the processing frequency of the second processing unit should be lower than that of the first processing unit. Specifically, the first processing unit 421 is configured to perform the correction based on IMU error parameters and dead reckoning steps in the aforementioned IMU online adaptive calibration method 100; while the second processing unit 422 is configured to perform the adaptive update steps in the aforementioned IMU online adaptive calibration method 100 and asynchronously transmit the updated model parameters to the first processing unit.

[0061] For example, such as Figure 9 The in-vehicle hardware system architecture diagram shown depicts a core processing unit that can employ a dual Raspberry Pi 5 asynchronous architecture. The first processing unit 421 is a Raspberry Pi-A (also known as the inference unit), responsible for high-priority real-time tasks, dedicated to executing high-priority real-time inference tasks. The second processing unit 422 is a Raspberry Pi-B (also known as the adaptation unit), responsible for low-priority background learning tasks, dedicated to executing low-priority background online adaptive loops.

[0062] Specifically, such as Figure 10 The execution logic of the onboard asynchronous dual-core architecture diagram shown in the online application phase, the real-time inference process of the first processing unit 421 in the online application phase may include the following steps: a) receiving raw IMU measurement data in real time; b) performing forward propagation of the hybrid neural network module (frozen base network + current adapter), and outputting the final IMU error parameters. p final c) Use p final d) Calibrate the raw IMU measurement data and perform standard dead reckoning to output the calibrated motion state; d) Monitor the network port from the second processing unit asynchronously. Once updated adapter weights are received, the weights of the online adapter modules in the current model are dynamically replaced without interrupting inference.

[0063] The online adaptive process of the second processing unit 422 in the online application phase may include the following steps: a) real-time monitoring of the effectiveness of high-precision GPS signals, and when high-precision GPS signals are available, continuously collecting raw IMU measurement data and time-series positioning data based on GPS signals. p GPS And through the p GPS The data undergoes real-time smoothing filtering and numerical differentiation to obtain the true global acceleration. a GPSb) Process the IMU data within this time window using the current error prediction model and differentiable physics model to calculate the calibrated global acceleration. a corr c) Construct the physical residual loss function L physics =MSE( a corr , a GPS d) The gradient of the loss with respect to the parameters of the online adapter module is calculated using an automatic differentiation engine; and the parameters of the online adapter module (MLP) are updated only based on the calculated gradient, and the updated weights are sent to the first processing unit via the network.

[0064] In another embodiment, the second processing unit 422 is configured to generate updated online adapter module weights by: 1) acquiring raw IMU measurement data and time-series positioning data based on GPS positioning signals within a set time window; 2) performing smoothing filtering and numerical differentiation on the time-series positioning data to determine the true global acceleration; 3) correcting the raw measurement data using IMU error parameters predicted by the error prediction model, and calculating the corresponding calibrated global acceleration using a differentiable physical model; 4) calculating the gradient and updating the parameters of the online adapter module based on the physical residual loss between the calibrated global acceleration and the true global acceleration, thereby generating updated weights. The first processing unit 421 is configured to perform the following functions: 1) acquiring updated weights sent by the second processing unit through asynchronous listening while performing real-time inference tasks; and 2) dynamically replacing the current weights of the online adapter module in the error prediction model with the updated weights.

[0065] For example, the task of the inference unit Raspberry Pi-A is to run a high-priority, high-frequency (e.g., 200Hz) real-time loop. Within the loop, the following steps are performed: a) receiving raw measurement data from the IMU; b) performing forward inference; c) outputting calibrated data (for navigation or attitude control); d) asynchronously listening to the network port from Raspberry Pi-B and dynamically replacing the currently used adapter model once new adapter weights are received.

[0066] The Raspberry Pi-B adapter unit's task is to run a low-priority, low-frequency (e.g., 1Hz) background loop. Within the loop, it performs the following: a) checks if the RTK-GPS signal is valid; b) if the signal is valid, collects IMU and GPS data over a short sliding window; c) within that sliding window, it retrieves the true GPS location data. p GPS Perform real-time smoothing filtering and numerical differentiation to calculate the true global acceleration online. aGPS d) Calculate the physical residual loss on this data batch. L physics =MSE( a corr , a GPS e) Perform backpropagation using an automatic differentiation engine. This backpropagation is computationally inefficient because the base network is frozen, preventing gradient backpropagation; therefore, only the lightweight adapter needs to be updated, i.e., only the computation time is calculated. L physics The gradient path calculated in this way is short and does not involve long chain integrals; f) Send the new adapter weights to Raspberry Pi-A through the network.

[0067] In another embodiment, the second processing unit 422 may also be configured to: monitor the validity of the GPS positioning signal in real time; and when the GPS positioning signal is detected to be lost or unavailable, suspend the execution of gradient calculation based on physical residual loss and the generation of online adapter module weights. Correspondingly, the first processing unit 421 may also be configured to: during the period when the GPS positioning signal is lost or unavailable, continue to use the most recently updated valid online adapter module weights and continue to execute real-time inference tasks to maintain dead reckoning.

[0068] In other words, when GPS signals are lost, the autonomous driving system can pause the online adaptive process and use the last set of valid adapter parameters to continue supporting the real-time inference process for navigation, thereby ensuring the robustness of the system.

[0069] Compared with the prior art, the present invention has the following significant advantages: 1. This invention reconstructs the underlying paradigm of IMU error calibration, breaking through the limitations of the linear assumptions of traditional filters: Existing mainstream technologies (such as EKF / VBAKF) essentially treat GPS as an external observation within the filter framework to passively correct the state, making it difficult to accurately model the complex and highly nonlinear error mechanisms within the IMU (such as the cross-coupling effect of acceleration on the gyroscope). This invention abandons the traditional "fusion correction" approach and innovates the "end-to-end differentiable physical model" training paradigm. By constructing the kinematic integral formula as a differentiable layer in the neural network, the problem of estimating non-differentiable error parameters is transformed into a differentiable trajectory prediction problem, allowing the position error gradient to pass through the long-period integral chain and propagate back to directly drive the network to learn the complex nonlinear error characteristics inside the sensor.

[0070] 2. A physically constrained gradient shortcut is constructed to solve the training problem of long sequence integrals: Addressing the "vanishing gradient" or "exploding gradient" problem faced by purely data-driven methods in long chain integrals, this invention introduces a physical residual loss based on the PINN concept in both offline training and online adaptation. This mechanism utilizes the acceleration constraints of intermediate layers to provide a direct gradient backpropagation shortcut for the neural network, forcing the network to output physically consistent results that conform to Newton's laws of motion. This design accelerates model convergence and ensures accurate calculation of high-dimensional IMU error parameters even in the absence of ground truth pose values, relying solely on sparse GPS position signals.

[0071] 3. Achieved Online Adaptation under Low Computing Power: Addressing the limitations of traditional static models in handling sensor aging and temperature drift, this invention proposes an online adaptive mechanism combining an asynchronous dual-core architecture with efficient parameter fine-tuning. Unlike traditional adaptive filters that only adjust noise parameters (Q / R array), this invention employs a strategy of "freezing the base network + updating only the lightweight adapter" to achieve real-time model parameter updates on low-computing-power edge computing devices (such as Raspberry Pi). This system can continuously utilize GPS information to combat dynamic environmental interference unseen during the training phase, improving the robustness and reliability of low-cost sensors throughout their entire lifecycle.

[0072] 4. A low-cost data solution with a closed-loop "simulation-real-world measurement" process has been established: This invention adopts a transfer learning strategy of "simulation pre-training + real-world fine-tuning," solving the problems of obtaining the true value of IMU internal errors in real-world scenarios and the high cost of high-precision calibration equipment (such as turntables and high-precision fiber optic inertial navigation systems). The system utilizes massive amounts of simulation data to solidify general physical error patterns and uses a small amount of real-world measurement data to align the noise characteristics of specific hardware, reducing the data acquisition threshold and implementation cost of high-precision inertial navigation algorithms.

[0073] 5. Asynchronous computing architecture ensures real-time performance and adaptive capabilities: The dual-processor (dual Raspberry Pi) deployment scheme proposed in this invention decouples high-frequency real-time inference tasks from low-frequency background adaptive tasks. This architecture ensures that the online learning (adaptation) process does not preempt real-time inference computing resources, achieving advanced intelligent adaptive functions without affecting the system's real-time performance.

[0074] 6. Provides high-fidelity motion state, supporting multiple control tasks: The core value of this invention is providing a high-precision motion state observer. It can not only be used for navigation in GPS blind spots (dead reckoning), but its calibrated high-precision attitude angles and angular velocities are also key inputs to upper-level control systems. For example: for vehicle rollover control, it can provide a drift-free tilt angle input for rollover warning systems; for agricultural machinery attitude control, it can provide precise leveling data for large spray booms, ensuring accurate spraying measurements; for vehicle dynamic stability control, it can provide high-precision yaw rate input for anti-skid systems.

[0075] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0076] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An online adaptive calibration method for an IMU, characterized in that, The calibration method includes: Raw measurement data is acquired from the IMU in real time; The raw measurement data is input into an error prediction model to output predicted IMU error parameters. The error prediction model employs a hybrid neural network architecture and is trained and updated based on an end-to-end framework including a differentiable physical model. It is configured to perform offline training in the offline phase and adaptive updates in the online application phase. The original measurement data is corrected using the IMU error parameters, and dead reckoning is performed using the corrected data to generate a calibrated motion trajectory.

2. The calibration method according to claim 1, characterized in that, The end-to-end training framework is used to match the frequency difference between input data and calibration data, where the input data is IMU high-frequency data and the calibration data is GPS low-frequency data, and includes the following steps: Based on the ratio of the IMU sampling frequency to the GPS sampling frequency, the number of IMU data frames N contained in the time interval between two adjacent GPS observations is determined. For N frames of IMU data located within the time interval between two adjacent GPS observations, a kinematic state integral recursion process with N time steps is executed using a mathematical model based on differentiable operations; and In each iteration, the IMU error parameters are used to correct the current raw data, and the motion state is updated using a differentiable kinematic formula. At the same time, the calibrated global acceleration is output to construct the physical residual loss.

3. The calibration method according to claim 1, characterized in that, The offline training includes the following steps: Using the generated simulation dataset, the hybrid neural network module is subjected to basic training using a composite loss function to obtain a first training model that has learned the general error physical properties; and Using the collected real dataset, the first training model is fine-tuned using the composite loss function to obtain a second training model adapted to specific hardware noise characteristics, wherein the amount of data in the real dataset is smaller than that in the simulation dataset.

4. The calibration method according to claim 3, characterized in that, The simulation dataset includes: paired IMU ground truth motion data and raw measurement simulation data. The IMU ground truth motion data includes ideal acceleration and ideal angular velocity. The original measurement simulation data was generated by superimposing error characteristics onto the true motion data of the IMU in a vehicle dynamics simulation environment based on a set IMU error model. The IMU error model is used to simulate one or more of the following error characteristics: zero bias instability, scale factor error, inter-axis non-orthogonality error or installation error, Gaussian white noise, accelerometer nonlinearity error, and gyroscope sensitivity error to acceleration.

5. The calibration method according to claim 3, characterized in that, The composite loss function L total It is constructed by weighting the location loss and physical residual loss, as shown in the following formula: L total = λ 1 L position + λ 2 L physics_mean ， in, L position The position loss is used to characterize the deviation between the predicted trajectory position derived from the differentiable physical model and the true GPS position. L physics_mean The aggregated value of the physical residual loss is used to characterize the mean deviation between the calibrated global acceleration output by the differentiable physical model and the true global acceleration. λ 1 and λ 2 is the weighting coefficient, used to balance the physical dimensions and numerical magnitudes of different loss terms.

6. The calibration method according to claim 1, characterized in that, The hybrid neural network includes a basic network module and an online adapter module; The online adaptive update is configured to update the parameters of the online adapter module in the hybrid neural network using physical residual loss, including the following steps: Time-series positioning data based on GPS signals are acquired and then smoothed, filtered, and numerically differentiated to determine the true global acceleration. The error prediction model is used to predict the original measurement data to obtain IMU error parameters, and the IMU error parameters and the original measurement data are input into the differentiable physical model to calculate the calibrated global acceleration. Construct a physical residual loss function that includes the difference between the true global acceleration and the calibrated global acceleration; The parameters of the basic network modules in the hybrid neural network are frozen, the gradient is calculated based on the physical residual loss function, and the parameters of the online adapter module are updated only.

7. An online adaptive calibration device for an IMU, characterized in that, The calibration device includes: The data acquisition module is used to acquire raw measurement data from the IMU in real time; An error prediction module is used to input the raw measurement data into an error prediction model to output predicted IMU error parameters. The error prediction model employs a hybrid neural network architecture and is trained and updated based on an end-to-end framework including a differentiable physical model. It is configured to perform offline training in the offline phase and adaptive updates in the online application phase. The dead reckoning module is used to correct the original measurement data using the IMU error parameters, and to perform dead reckoning using the corrected data to generate a calibrated motion trajectory.

8. An autonomous driving system, characterized in that, The autonomous driving system includes: The sensor module is used to acquire raw measurement data from the IMU and GPS positioning signals; The in-vehicle intelligent computing platform adopts an asynchronous processing architecture, including a first processing unit for performing real-time inference tasks and a second processing unit for performing background learning tasks. The processing frequency of the second processing unit is lower than that of the first processing unit. The first processing unit is configured to perform the following steps in the IMU online adaptive calibration method according to any one of claims 1-6: correction based on IMU error parameters and dead reckoning. The second processing unit is configured to perform the adaptive update step in the IMU online adaptive calibration method according to any one of claims 1-6, and asynchronously transmit the updated model parameters to the first processing unit.

9. The autonomous driving system according to claim 8, characterized in that, The second processing unit is configured to generate updated online adapter module weights in the following manner: Acquire the raw IMU measurement data and time-series positioning data based on the GPS positioning signal within a set time window; The time series positioning data is subjected to smoothing filtering and numerical differentiation to determine the true global acceleration; The original measurement data are corrected using the IMU error parameters predicted by the error prediction model, and the calibrated global acceleration is calculated using the differentiable physical model. Based on the physical residual loss between the calibrated global acceleration and the true global acceleration, the gradient is calculated and the parameters of the online adapter module are updated, thereby generating updated weights; The first processing unit is configured as follows: While performing real-time inference tasks, the updated weights sent by the second processing unit are obtained through asynchronous listening. as well as The updated weights are used to dynamically replace the current weights of the online adapter module in the error prediction model.

10. The autonomous driving system according to claim 8, characterized in that, The second processing unit is further configured to: monitor the validity of the GPS positioning signal in real time; and when the GPS positioning signal is detected to be lost or unavailable, suspend the execution of gradient calculation based on physical residual loss and the generation of online adapter module weights; The first processing unit is also configured to: during the period when the GPS positioning signal is lost or unavailable, maintain the most recently updated online adapter module weights and continue to execute the real-time inference task to maintain dead reckoning.

Images