Vehicle-mounted inertial positioning method and device based on data and model combined driving

Abstract

The application relates to the technical field of navigation positioning, in particular to a vehicle-mounted inertial positioning method and device based on data and model joint driving, which comprises the following steps: performing data preprocessing on acceleration data and gyroscope data of a vehicle-mounted IMU, inputting the data into a three-dimensional velocity estimation network to obtain pseudo-measurement constraints and measurement noise scale factors, and calculating measurement noise covariance according to the two factors; inputting the preprocessed data into a process noise estimation network to obtain process noise scale factors and process noise uncertainty, and calculating process noise covariance according to the two factors; constructing a loss function for uncertainty estimation, and adjusting error penalty weights of the three-dimensional velocity estimation network and the noise estimation network; mechanically arranging the data, combining the pseudo-measurement constraints and the adaptive noise covariance, performing extended Kalman filtering, and performing vehicle-mounted inertial positioning according to the filtering result. Thus, the problem of decline of vehicle positioning precision in related technologies under complex environments is solved.

Description

A data- and model-driven method and device for vehicle inertial positioning. Technical Field

[0001] This application relates to the field of navigation and positioning technology, and in particular to a vehicle-mounted inertial positioning method and device based on data and model joint driving. Background Technology

[0002] High-precision vehicle positioning is a fundamental requirement for the development of intelligent driving and IoT technologies. GNSS (Global Navigation Satellite System), LiDAR, cameras, and inertial sensors are widely used for vehicle navigation, control, and decision-making. However, in challenging and complex urban environments, such as urban canyons, thin textures, and severe weather conditions, the positioning accuracy of GNSS and vision-based technologies can significantly decrease. The application of most devices is also typically limited by complex installation requirements and high costs. INS (Inertial Navigation System) can perform continuous autonomous positioning without any external infrastructure, making it a key component of intelligent vehicle navigation. However, INS errors accumulate over time, and the estimated position drifts rapidly. Addressing the limitations of INS is crucial to enabling vehicles to accurately execute dynamic movements with a high update rate in all scenarios, ensuring safe driving and avoiding extreme risks.

[0003] Noise in traditional Kalman filters used for vehicle localization is typically based on fixed empirical assumptions. However, noise characteristics often vary with environment and motion state, making it challenging to manually construct suitable noise models. This limits the accuracy of Kalman filter state estimation and leads to decreased vehicle localization accuracy in complex environments. Furthermore, pseudo-velocity measurements, such as odometer readings and NHC (Non-Holonomic Constraint), can provide additional constraints to the Kalman filter under GNSS rejection conditions, thereby improving the accuracy of state estimation. However, acquiring odometer data requires access to the vehicle's controller area network, which limits its application in some cases. The NHC assumption also becomes inaccurate when measuring complex motions such as sideslips and jumps. Summary of the Invention

[0004] This application provides a vehicle inertial positioning method and device based on data and model joint driving, in order to solve the problem of decreased vehicle positioning accuracy in complex environments.

[0005] The first aspect of this application provides a vehicle-mounted inertial positioning method based on data and model joint driving, comprising the following steps: preprocessing the acceleration data and gyroscope data from the vehicle-mounted IMU; inputting the preprocessed data into a three-dimensional velocity estimation network, which outputs the vehicle's three-dimensional velocity and measurement noise uncertainty, using the vehicle's three-dimensional velocity as a pseudo-measurement constraint and the measurement noise uncertainty as a measurement noise scaling factor, and calculating the measurement noise covariance based on the measurement noise scaling factor; inputting the preprocessed data into a process noise estimation network, which outputs the process noise scaling factor and process noise uncertainty, and calculating the process noise covariance based on the process noise scaling factor; constructing a loss function for uncertainty estimation, adjusting the error penalty weights of the three-dimensional velocity estimation network based on the loss function and measurement noise uncertainty, and adjusting the error penalty weights of the process noise estimation network based on the loss function and process noise uncertainty; mechanically arranging the acceleration data and gyroscope data from the vehicle-mounted IMU, performing extended Kalman filtering based on the mechanically arranged data, pseudo-measurement constraints, zero-velocity detection results, measurement noise covariance, and process noise covariance, and performing vehicle-mounted inertial positioning based on the filtering results.

[0006] Optionally, the acceleration and gyroscope data from the vehicle-mounted IMU are preprocessed, including: adding random Gaussian noise to the acceleration and gyroscope data; and segmenting the acceleration and gyroscope data using a sliding window.

[0007] Optionally, the 3D velocity estimation network includes an input block, a residual block, and an output block. The input block consists of convolutional layers, batch normalization layers, and max pooling layers. The convolutional layers are used for feature extraction, the batch normalization layers normalize the outputs of the convolutional layers, and the max pooling layers select the features with the maximum value in the pooling window. The residual block controls two convolutional layers, which are connected by a batch normalization layer and a residual layer. A self-attention module is fused after each residual block. The output block outputs the 3D velocity of the vehicle and the measurement noise uncertainty. The output block includes a fully connected layer and a batch normalization layer, followed by multiple dropout layers.

[0008] Optionally, the output block of the 3D velocity estimation network is set with a Tanh function. Replacing the Tanh function with a Sigmoid function allows for binary classification of vehicle states to obtain zero-speed detection results.

[0009] Optionally, the process noise estimation network includes two feature extraction modules, which have the same structure. Each feature extraction module includes a convolutional layer, a ReLU function, and a compression activation module. After the convolutional layer extracts features and activates them using the ReLU function, the compression activation module dynamically adjusts the weights of each channel.

[0010] Optionally, the compression activation module includes a global average pooling layer, a fully connected layer, and a sigmoid function activation. The global average pooling layer performs global aggregation of the spatial information of the channels of each original feature map. The globally aggregated information is activated by the sigmoid function through the fully connected layer. The channels of the original feature map are re-weighted based on the activation result before being output.

[0011] Optionally, the loss function for uncertainty estimation is:

[0012]

[0013] Where N is the length of the sliding window; y k f(x) serves as the reference truth value. k ) is the input x k The estimated value of the corresponding output; σ(x) k ) 2 It is model uncertainty.

[0014] Alternatively, the state vector of the extended Kalman filter is defined as follows:

[0015]

[0016] Where, p n Position in the navigation system; v n Speed ​​under navigation system; For posture; b ω For the gyroscope bias in the carrier system; b a Accelerometer bias under load system; The IMU mounting angle between the carrier system and the vehicle system; This serves as the IMU lever between the carrier system and the vehicle system.

[0017] The covariance matrix is ​​updated as follows:

[0018]

[0019] in, Let be the prior state covariance at time k; Let F be the posterior state covariance at time k-1; k-1 and G k-1 , , are the Jacobian matrices of the nonlinear function of state propagation; Q is the process noise covariance; v is the coordinate transformation matrix from the loading system to the navigation system at time k-1; k-1 p represents the velocity in the navigation system at time k-1. k-1dt represents the position in the navigation system at time k-1; g represents gravity; dt represents the time interval; (·×) denotes an antisymmetric matrix.

[0020] A second aspect of this application provides a vehicle-mounted inertial positioning device based on data and model joint driving, comprising: a processing module for preprocessing acceleration data and gyroscope data from an onboard IMU; a first calculation module for inputting the preprocessed data into a three-dimensional velocity estimation network, which outputs the vehicle's three-dimensional velocity and measurement noise uncertainty, using the vehicle's three-dimensional velocity as a pseudo-measurement constraint and the measurement noise uncertainty as a measurement noise scaling factor, and calculating the measurement noise covariance based on the measurement noise scaling factor; and a second calculation module for inputting the preprocessed data into a process noise estimation network, which outputs the process noise estimation network... The system includes a noise scaling factor and process noise uncertainty, and calculates the process noise covariance based on the process noise scaling factor. An adjustment module is used to construct a loss function for uncertainty estimation, and adjusts the error penalty weights of the 3D velocity estimation network based on the loss function and measurement noise uncertainty. A filtering module is used to mechanically arrange the acceleration and gyroscope data from the onboard IMU, perform extended Kalman filtering based on the mechanically arranged data, pseudo-measurement constraints, zero-speed detection results, measurement noise covariance, and process noise covariance, and perform onboard inertial positioning based on the filtering results.

[0021] A third aspect of this application provides a vehicle, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle inertial positioning method based on data and model joint drive as described in the first aspect.

[0022] Therefore, this application has the following beneficial effects:

[0023] This application's embodiments preprocess the acceleration and gyroscope data from the vehicle-mounted IMU, inputting them into a 3D velocity estimation network. The network outputs pseudo-measurement constraints and a measurement noise scaling factor, and the measurement noise covariance is calculated based on the measurement noise scaling factor. The preprocessed data is then input into a process noise estimation network, which outputs a process noise scaling factor and process noise uncertainty. The process noise covariance is calculated based on the process noise scaling factor. A loss function for uncertainty estimation is constructed, and the error penalty weights of the 3D velocity estimation network and the noise estimation network are adjusted. The acceleration and gyroscope data from the vehicle-mounted IMU are mechanically arranged and subjected to extended Kalman filtering. Vehicle inertial positioning is performed based on the filtering results. Deep learning is used to estimate the vehicle's 3D velocity and corresponding uncertainties. A data-driven noise covariance adapter is introduced to dynamically adjust the process noise covariance, achieving accurate and robust position estimation. This solves the problem of decreased vehicle positioning accuracy in complex environments associated with related technologies.

[0024] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0025] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0026] Figure 1 is a flowchart of a vehicle inertial positioning method based on data and model joint driving according to an embodiment of this application;

[0027] Figure 2 is a schematic diagram of a vehicle inertial positioning method based on data and model joint driving according to an embodiment of this application;

[0028] Figure 3 is a schematic diagram of the structure of a residual network based on an attention mechanism according to an embodiment of this application;

[0029] Figure 4 is a schematic diagram of a process noise estimation network structure provided according to an embodiment of this application;

[0030] Figure 5 is an example diagram of a vehicle-mounted inertial positioning device based on data and model joint drive according to an embodiment of this application;

[0031] Figure 6 is a structural schematic diagram of a vehicle provided according to an embodiment of this application. Detailed Implementation

[0032] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0033] The following describes, with reference to the accompanying drawings, an embodiment of the vehicle inertial positioning method and apparatus based on data and model joint driving of this application. Addressing the problem of decreased vehicle positioning accuracy in complex environments mentioned in the background art, this application provides a vehicle inertial positioning method based on data and model joint driving. In this method, acceleration data and gyroscope data from the vehicle IMU are preprocessed and input into a three-dimensional velocity estimation network. The network outputs pseudo-measurement constraints and a measurement noise scaling factor. The measurement noise covariance is calculated based on the measurement noise scaling factor. The preprocessed data is then input into a process noise estimation network, which outputs a process noise scaling factor and process noise uncertainty. The process noise covariance is calculated based on the process noise scaling factor. A loss function for uncertainty estimation is constructed, and the error penalty weights of the three-dimensional velocity estimation network and the noise estimation network are adjusted. The acceleration data and gyroscope data from the vehicle IMU are mechanically arranged and subjected to extended Kalman filtering. Vehicle inertial positioning is performed based on the filtering results. The vehicle's three-dimensional velocity and corresponding uncertainty are estimated based on deep learning. A data-driven noise covariance adapter is introduced to dynamically adjust the process noise covariance, achieving accurate and robust position estimation. This solves the problem of decreased vehicle positioning accuracy in complex environments.

[0034] Specifically, Figure 1 is a flowchart illustrating a vehicle inertial positioning method based on data and model joint driving provided in an embodiment of this application.

[0035] As shown in Figure 1, the vehicle inertial localization method based on data and model joint drive includes the following steps:

[0036] In step S101, the acceleration data and gyroscope data of the vehicle-mounted IMU are preprocessed.

[0037] It is understood that the embodiments of this application can collect acceleration data and gyroscope data through an on-board IMU, and perform data preprocessing on the acceleration data and gyroscope data. The preprocessing method will be described in detail below and will not be repeated here.

[0038] In this embodiment of the application, the acceleration data and gyroscope data of the vehicle-mounted IMU are preprocessed, including: adding random Gaussian noise to the acceleration data and gyroscope data; and using a sliding window to segment the acceleration data and gyroscope data.

[0039] The random Gaussian noise is typically set based on data provided by the IMU manufacturer and is not specifically limited here; the formula for segmenting the original data sequence using a sliding window is as follows: ω and a represent the raw 3D gyroscope data and 3D accelerometer data from the IMU, respectively; L is the length of the raw data sequence. and This is the processed data with added random Gaussian noise; N is the sliding window length; Num seg This represents the number of slices.

[0040] It is understood that the method for preprocessing the acceleration and gyroscope data of the vehicle-mounted IMU in this embodiment of the application is to add random Gaussian noise to the acceleration and gyroscope data, and to use a sliding window and the formula... The acceleration data and gyroscope data are separated.

[0041] In step S102, the preprocessed data is input into the three-dimensional velocity estimation network. The three-dimensional velocity estimation network outputs the vehicle's three-dimensional velocity and the measurement noise uncertainty. The vehicle's three-dimensional velocity is used as a pseudo-measurement constraint, and the measurement noise uncertainty is used as a measurement noise scaling factor. The measurement noise covariance is calculated based on the measurement noise scaling factor.

[0042] The three-dimensional velocity estimation network is a residual network designed based on an attention mechanism. Preprocessed data is input into the three-dimensional velocity estimation network, and the network outputs the vehicle's three-dimensional velocity and the formula for measurement noise uncertainty. and It is gyroscope and accelerometer data with Gaussian noise added, v c =[v for ,v lat ,v up ] T γ is the estimated three-dimensional vehicle velocity; F1(θ) is the uncertainty; F1(θ) is the designed attention-based residual network; the uncertainty is defined as the measurement noise scaling factor that adjusts the measurement noise covariance of the pseudo-measurement, and the formula is: R k Let k be the measurement noise covariance at time k; γ is the initial value; k It is the diagonal matrix of the uncertainty transformation of the velocity network output; 10 λ It is a constant parameter.

[0043] It is understood that, in the embodiments of this application, preprocessed acceleration and gyroscope data can be input into a three-dimensional velocity estimation network. The three-dimensional velocity estimation network outputs the vehicle's three-dimensional velocity and measurement noise uncertainty, using the vehicle's three-dimensional velocity as a pseudo-measurement constraint. The obtained measurement noise uncertainty is used as a scaling factor, and the measurement noise covariance can be calculated based on the measurement noise scaling factor. R k Let be the measurement noise covariance at time k.

[0044] In this embodiment, the 3D velocity estimation network includes an input block, a residual block, and an output block. The input block consists of convolutional layers, batch normalization layers, and max pooling layers. The convolutional layers are used for feature extraction, the batch normalization layers normalize the outputs of the convolutional layers, and the max pooling layers select the feature with the maximum value in the pooling window. The residual block controls two convolutional layers, which are connected by a batch normalization layer and a residual layer. A self-attention module is fused after each residual block. The output block outputs the 3D velocity of the vehicle and the measurement noise uncertainty. The output block includes a fully connected layer and a batch normalization layer, and multiple dropout layers are set after the fully connected layer and the batch normalization layer.

[0045] The Dropout layer can randomly discard a portion of neurons, which can prevent the model from overfitting and improve generalization ability.

[0046] It is understood that the 3D velocity estimation network in this embodiment includes an input block, a residual block, and an output block. The input block includes convolutional layers, batch normalization layers, and max pooling layers. Feature extraction is achieved through convolutional layers, and the output of the convolutional layers is normalized by batch normalization layers. Max pooling layers select the feature with the maximum value in the pooling window. The residual block controls two convolutional layers, which have batch normalization operations. An additional residual connection is added to ensure smooth information flow. A self-attention module is fused after each residual block. The output block outputs the 3D velocity of the vehicle and the measurement noise uncertainty. The output block includes fully connected layers and batch normalization layers. Multiple dropout layers are set after the fully connected layers and batch normalization layers. By randomly dropping some neurons, overfitting of the model can be prevented and generalization ability can be improved.

[0047] In this embodiment, the output block of the three-dimensional velocity estimation network is equipped with a Tanh function. The Tanh function is replaced with a Sigmoid function to perform binary classification of the vehicle state and obtain the zero-speed detection result.

[0048] Binary classification refers to the task of assigning a state to one of two mutually exclusive categories, which can distinguish between "vehicle moving" and "vehicle stationary".

[0049] It is understood that, in the embodiments of this application, the Tanh function set in the output block can be replaced with the Sigmoid function to classify the vehicle state using binary classification and perform zero-speed detection on the vehicle to obtain the zero-speed detection result.

[0050] In step S103, the preprocessed data is input into the process noise estimation network, which outputs the process noise scaling factor and the process noise uncertainty. The process noise covariance is calculated based on the process noise scaling factor.

[0051] The process of inputting the preprocessed data into the process noise estimation network, and the process noise estimation network outputting the process noise scaling factor and process noise uncertainty is as follows: ρ is the process noise scaling factor; μ is the process noise uncertainty; F2(θ) is the designed process noise estimation network; the structure of the process noise estimation network will be described in detail below, and will not be repeated here; the process noise covariance is calculated based on the process noise scaling factor, as shown below: Q k =Q·ρ k , Q k Let k be the adaptive process noise covariance at time k. ρ is the initial matrix; k Let be the diagonal matrix of the process noise scaling factor transformation at time k.

[0052] It is understood that this application embodiment designs a process noise estimation network, which can input preprocessed data into the process noise estimation network, and output the process noise scaling factor and process noise uncertainty through the process noise estimation network, based on Q. k =Q·ρ k The adaptive process noise covariance at time k is calculated.

[0053] In this embodiment, the process noise estimation network includes two feature extraction modules, which have the same structure. Each feature extraction module includes a convolutional layer, a ReLU function, and a compression activation module. After the convolutional layer extracts features and activates them using the ReLU function, the compression activation module dynamically adjusts the weights of each channel.

[0054] It is understood that the process noise estimation network in this application embodiment includes two feature extraction modules with the same structure. The feature extraction module includes a convolutional layer, a ReLU function and a compression activation module. In the convolutional layer, it is responsible for extracting local features from the preprocessed data. Then, the ReLU function is used to activate these features. Finally, the compression activation module dynamically adjusts the weights of each channel according to global information.

[0055] In this embodiment, the compression excitation module includes a global average pooling layer, a fully connected layer, and a sigmoid function activation. The global average pooling layer performs global aggregation of the spatial information of the channels of each original feature map. The globally aggregated information is activated by the sigmoid function through the fully connected layer. The channels of the original feature map are reweighted based on the activation result and then output.

[0056] It is understood that the compression excitation module in this application embodiment includes a global average pooling layer, a fully connected layer, and sigmoid function activation. First, the spatial information of each channel is globally aggregated through the global average pooling layer. Then, it is passed through two fully connected layers and activated by the sigmoid function. The results are used to reweight the channels of the original feature map before output.

[0057] In step S104, a loss function for uncertainty estimation is constructed, and the error penalty weights of the three-dimensional velocity estimation network are adjusted according to the loss function and measurement noise uncertainty. The error penalty weights of the process noise estimation network are also adjusted according to the loss function and process noise uncertainty.

[0058] The loss function for uncertainty estimation will be described in detail below and will not be repeated here.

[0059] It is understood that the embodiments of this application can construct a loss function for uncertainty estimation, use the loss function for uncertainty estimation and the error penalty weights of the network for measurement noise uncertainty and process noise uncertainty estimation, and dynamically adjust the error penalty weights to improve the accuracy and robustness of the network output under various motion states.

[0060] In this embodiment of the application, the loss function for uncertainty estimation is:

[0061]

[0062] Where N is the length of the sliding window; y k f(x) serves as the reference truth value. k ) is the input x k The estimated value of the corresponding output; σ(x) k ) 2 It is model uncertainty.

[0063] In step S105, the acceleration data and gyroscope data of the vehicle-mounted IMU are mechanically arranged, and extended Kalman filtering is performed based on the mechanically arranged data, pseudo-measurement constraints, zero-speed detection results, measurement noise covariance and process noise covariance. Vehicle-mounted inertial positioning is then performed based on the filtering results.

[0064] Mechanical orchestration refers to the process of converting the raw outputs of accelerometers and gyroscopes into position, velocity, and attitude; extended Kalman filtering will be described in detail below and will not be repeated here; pseudo-measurement constrains the vehicle's three-dimensional velocity, defined as follows: [v for ,v lat ,v up ] T For the estimated three-dimensional velocities of the vehicle mentioned above, n c The covariance of the noise is the estimated noise covariance mentioned above.

[0065] It is understood that the embodiments of this application can perform INS mechanical orchestration on the raw IMU data, combine the estimated pseudo measurement constraints, zero-speed detection results, measurement noise covariance and process noise covariance to perform extended Kalman filtering, and obtain the filtering results for vehicle inertial positioning.

[0066] In this embodiment, the state vector of the extended Kalman filter is defined as follows:

[0067]

[0068] Where, p n Position in the navigation system; v n Speed ​​under navigation system; For posture; b ω For the gyroscope bias in the carrier system; b a Accelerometer bias under load system; The IMU mounting angle between the carrier system and the vehicle system; This serves as the IMU lever between the carrier system and the vehicle system.

[0069] The covariance matrix is ​​updated as follows:

[0070]

[0071] in, Let be the prior state covariance at time k; Let F be the posterior state covariance at time k-1; k-1 and G k-1 , , are the Jacobian matrices of the nonlinear function of state propagation; Q is the process noise covariance; v is the coordinate transformation matrix from the loading system to the navigation system at time k-1; k-1 p represents the velocity in the navigation system at time k-1. k-1 dt represents the position in the navigation system at time k-1; g represents gravity; dt represents the time interval; (·×) denotes an antisymmetric matrix.

[0072] It should be noted that, during the update process, the vehicle speed calculated by INS is: in, This is the coordinate transformation matrix between the carrier system and the vehicle system, i.e., the IMU mounting angle; For lever arm; and Zero-mean Gaussian noise was used, and the vehicle's three-dimensional velocity was used as a pseudo-measurement constraint, defined as follows: [v for ,v lat ,v up ] T The three-dimensional velocities of the vehicle estimated in step S102; n c To measure noise, its covariance is the R estimated in step S102. k Based on the above formula, the measurement equation for a vehicle in motion is: n c The noise from the pseudo-measurement of the vehicle's three-dimensional velocity is represented by R; H1 is the Jacobian matrix of the nonlinear function of the measured values ​​under moving conditions; based on zero-speed detection, when the vehicle is stationary, the measurement equation is: H2=[θ 3×3 ,I 3×3 ,0 15×3 ], where n c0 H1 represents the noise from the pseudo-measurement of the vehicle's three-dimensional velocity; H2 is the Jacobian matrix of the nonlinear function of the measured value under stationary conditions.

[0073] It is understood that the embodiments of this application can use the above formula to expand the state vector of the Kalman filter and update the covariance matrix. During the update process, the vehicle measurement equation in the moving state or the stationary state is calculated, and the filtering result is obtained for vehicle inertial positioning.

[0074] The vehicle-mounted inertial positioning method based on data and model joint driving proposed in this application preprocesses the acceleration and gyroscope data from the vehicle-mounted IMU and inputs them into a three-dimensional velocity estimation network. The network outputs pseudo-measurement constraints and a measurement noise scaling factor, and the measurement noise covariance is calculated based on the measurement noise scaling factor. The preprocessed data is then input into a process noise estimation network, which outputs a process noise scaling factor and process noise uncertainty. The process noise covariance is calculated based on the process noise scaling factor. A loss function for uncertainty estimation is constructed, and the error penalty weights of the three-dimensional velocity estimation network and the noise estimation network are adjusted. The acceleration and gyroscope data from the vehicle-mounted IMU are mechanically arranged and subjected to extended Kalman filtering. Vehicle-mounted inertial positioning is performed based on the filtering results. The three-dimensional velocity of the vehicle and the corresponding uncertainty are estimated based on deep learning. A data-driven noise covariance adapter is introduced to dynamically adjust the process noise covariance, achieving accurate and robust position estimation.

[0075] The following specific example further describes the vehicle inertial positioning method based on joint data and model driving:

[0076] As shown in Figure 2, the vehicle inertial positioning method based on data and model joint driving provided in this embodiment specifically includes the following steps:

[0077] Step 1: Preprocess the raw acceleration and gyroscope data from the vehicle-mounted IMU, including adding Gaussian zero-mean noise and using a sliding window to segment the data sequence, as detailed below:

[0078] To improve the robustness of the neural network to noise in the input data during the training phase, random Gaussian noise is added to the raw observation data. The random Gaussian noise is typically set according to the data provided by the IMU manufacturer. Furthermore, since the features of IMU data at a single time step cannot be extracted, a sliding window is used to segment the raw data sequence to serve as input to the network in steps 2 and 3, as shown below:

[0079]

[0080] Where ω and a are the raw 3D gyroscope data and 3D accelerometer data of the IMU, respectively; L is the length of the raw data sequence. and This is the processed data with added random Gaussian noise; N is the sliding window length; Num seg This represents the number of slices.

[0081] Step 2: Based on the preprocessed data from Step 1, a residual network incorporating an attention mechanism is used to estimate the vehicle's 3D velocity and uncertainty through multi-task learning. The vehicle's 3D velocity is used as a pseudo-measurement constraint, and the uncertainty associated with the velocity output is used as the measurement noise covariance, as detailed below:

[0082] The inputs and outputs of the IMU data-driven 3D velocity estimation network are as follows:

[0083]

[0084] in, and It is gyroscope and accelerometer data with Gaussian noise added; v c =[v for ,v lat ,v up ] T γ is the predicted three-dimensional vehicle velocity; N is the uncertainty; F1(θ) is the length of the sliding window; F1(θ) is the designed attention-based residual network.

[0085] As shown in Figure 3, the attention-based residual network consists of three parts: an input block, a residual block, and an output block. The specific structural design is as follows:

[0086] The input block consists of a convolutional layer, a batch normalization layer, and a max pooling layer. The convolutional layer is used for initial local feature extraction, the batch normalization layer stabilizes the process by normalizing the output of the convolutional layer, and the max pooling layer retains the most important features by selecting the maximum value in the pooling window, which helps to reduce irrelevant features and noise.

[0087] Residual Blocks: The core of the network consists of residual blocks composed of four residual regions. A classic residual block consists of two convolutional layers with batch normalization and a residual connection. To optimize the structure, a self-attention module is fused after each residual block, avoiding a significant increase in computational overhead. This part of the network structure first extracts local features and then captures long-term dependencies in the global feature space based on the self-attention mechanism.

[0088] Output block: High-dimensional features are flattened and fed into a fully connected layer, then passed through a Tanh function for final prediction. Multiple dropout layers are set after the fully connected layers and batch normalization layers to prevent overfitting. The network output contains two vectors: the estimated 3D vehicle velocity and its uncertainty.

[0089] Uncertainty is defined as the measurement noise scaling factor that adjusts the measurement noise covariance of spurious measurements:

[0090] R k =R·γk (3)

[0091]

[0092] Among them, R k Let k be the measurement noise covariance at time k; γ is the initial value; k It is the diagonal matrix of the uncertainty transformation of the velocity network output; 10 λ It is a constant parameter.

[0093] Based on the velocity estimation network structure, the Tanh function in the output block is replaced with a Sigmoid function to perform binary classification of the vehicle state, thereby achieving zero-speed detection. Based on zero-speed detection, additional constraints can be applied in the subsequent adaptive Kalman filter when the vehicle is stationary.

[0094] Step 3: Based on the preprocessed data from Step 1, a convolutional network with a compressed activation mechanism is used to estimate the process noise covariance and its uncertainty through multi-task learning, as follows:

[0095] The input to the IMU data-driven process noise estimation network is preprocessed gyroscope and accelerometer data, and the network output is the process noise scaling factor.

[0096]

[0097] in, and The data consists of gyroscope and accelerometer data with Gaussian noise added; ρ is the process noise scaling factor; μ is the process noise uncertainty; N is the sliding window length; and F2(θ) is the designed process noise estimation network.

[0098] As shown in Figure 4, the process noise estimation network structure is as follows:

[0099] Two identical modules are used to extract features from the input data. In each module, a one-dimensional convolutional layer is first used to extract initial features and activated using the ReLU function. Then, a compression activation module dynamically adjusts the weights of each channel. In the compression activation module, spatial information for each channel is first globally aggregated using GAP, then passed through two fully connected layers activated by the sigmoid function. The result is used to reweight the channels of the original feature map before output. The process noise covariance is dynamically adjusted using a process noise scaling factor from the noise estimation network output, as shown below:

[0100] Q k =Q·ρ k (6)

[0101]

[0102] Among them, Q k Let k be the adaptive process noise covariance at time k. ρ is the initial matrix; k Let be the diagonal matrix of the process noise scaling factor transformation at time k.

[0103] Step 4: For the networks in Step 2 and Step 3, construct a loss function based on uncertainty estimation of sensor data noise, dynamically adjust the error penalty weights, and improve the accuracy and robustness of the network output under various motion states.

[0104] Further, step 4 specifically involves improving the model's estimation performance by using multi-task training in the data-driven module to estimate the uncertainties associated with each sensor and incorporate them into the training. Both the proposed velocity estimation network and process noise estimation network are optimized using a negative log-likelihood loss function. Maximizing the probability of the prediction distribution, when incorporating uncertainties, effectively captures the stochastic uncertainties present in the data.

[0105]

[0106] y k f(x) serves as the reference truth value. k ) is the input x k The estimated value of the corresponding output; σ(x) k ) 2 It is model uncertainty.

[0107] Specifically, the speed estimation network in step 2 has a y k For true three-dimensional velocity, f(x) k ) represents the output three-dimensional velocity. Since sensor data exhibits inconsistent noise under different motion states, uncertainty estimation can improve the accuracy of velocity estimation. For the process noise covariance estimation network in step 3, since the process noise covariance reflects the filter state, y is used when constructing the loss function of the process noise network. k For the actual displacement, f(x) k ) represents the displacement output by the adaptive Kalman filter.

[0108] By introducing uncertainty into the loss function, the proposed network can more effectively capture and quantify the noise characteristics of real-world environments, thereby enabling the neural Kalman filter system to adapt to state changes and improve localization performance.

[0109] Step 5: Perform INS mechanical orchestration based on the raw IMU data. Combine the pseudo-measurement constraints and measurement noise estimated in Step 2 with the process noise covariance estimated in Step 3 to construct an adaptive Kalman filter for vehicle inertial positioning, thereby improving the positioning accuracy and robustness under multiple motion states in GNSS denied environments.

[0110] Furthermore, step 5 specifically involves: using an extended Kalman filter for inertial positioning, with the state vector defined as follows:

[0111]

[0112] Where, p n Position in the navigation system; v n Speed ​​under navigation system; For posture; b ω For the gyroscope bias in the carrier system; b a Accelerometer bias under load system; The IMU mounting angle between the carrier system and the vehicle system; This refers to the IMU lever connecting the carrier system and the vehicle system.

[0113] In the prediction step, the covariance matrix is ​​updated as follows:

[0114]

[0115] in, Let be the prior state covariance at time k; Let F be the posterior state covariance at time k-1; k-1 and G k-1 , respectively, are the Jacobian matrices of the nonlinear function of state propagation; Q is the process noise covariance, dynamically estimated from step 3; v is the coordinate transformation matrix from the loading system to the navigation system at time k-1; k-1 p represents the velocity in the navigation system at time k-1. k-1 dt represents the position in the navigation system at time k-1; g represents gravity; dt represents the time interval; (·×) denotes an antisymmetric matrix.

[0116] In the update step, the vehicle speed calculated by INS is:

[0117]

[0118] in Install the IMU at an angle; For lever arm; and It is zero-mean Gaussian noise.

[0119] The vehicle's three-dimensional velocity is used as a pseudo-measurement constraint, defined as follows:

[0120]

[0121] Among them, [v for ,v lat ,v up ] T n represents the estimated three-dimensional velocity of the vehicle in step 2. c To measure the noise, its covariance is the R estimated in step 2. k .

[0122] Based on the above two formulas, the measurement equation for a vehicle in motion is:

[0123]

[0124] Where, n c The noise of the vehicle's three-dimensional velocity pseudo-measurement is R, with a covariance of R; H1 is the Jacobian matrix of the nonlinear function of the measured values ​​under the moving state.

[0125] Based on the zero-speed detection in step 2, when the vehicle is stationary, the measurement equation is as follows:

[0126]

[0127] H2 = [0 3×3 ,I 3×3 ,0 15×3 (18)

[0128] Where, n c0 H1 represents the noise from the pseudo-measurement of the vehicle's three-dimensional velocity; H2 is the Jacobian matrix of the nonlinear function of the measured value under stationary conditions.

[0129] Next, referring to the accompanying drawings, we describe the vehicle-mounted inertial positioning device based on data and model joint driving according to the embodiments of this application.

[0130] Figure 5 is a block diagram of an inertial positioning device based on data and model joint driving according to an embodiment of this application.

[0131] As shown in Figure 5, the vehicle-mounted inertial positioning device 10 based on data and model joint drive includes: a processing module 201, a first calculation module 202, a second calculation module 203, an adjustment module 204, and a filtering module 205.

[0132] The processing module 201 preprocesses the acceleration and gyroscope data from the onboard IMU. The first calculation module 202 inputs the preprocessed data into a three-dimensional velocity estimation network, which outputs the vehicle's three-dimensional velocity and measurement noise uncertainty. The vehicle's three-dimensional velocity is used as a pseudo-measurement constraint, and the measurement noise uncertainty is used as a measurement noise scaling factor. The measurement noise covariance is calculated based on the measurement noise scaling factor. The second calculation module 203 inputs the preprocessed data into a process noise estimation network, which outputs the process noise scaling factor and process noise uncertainty. The process noise covariance is calculated based on the process noise scaling factor; the adjustment module 204 is used to construct the loss function for uncertainty estimation, and adjust the error penalty weights of the three-dimensional velocity estimation network based on the loss function and measurement noise uncertainty; the filtering module 205 is used to mechanically arrange the acceleration data and gyroscope data of the vehicle IMU, perform extended Kalman filtering based on the mechanically arranged data, pseudo-measurement constraints, zero-speed detection results, measurement noise covariance and process noise covariance, and perform vehicle inertial positioning based on the filtering results.

[0133] In this embodiment, the processing module 201 is further configured to: add random Gaussian noise to the acceleration data and gyroscope data; and segment the acceleration data and gyroscope data using a sliding window.

[0134] In this embodiment, the 3D velocity estimation network includes an input block, a residual block, and an output block. The input block consists of convolutional layers, batch normalization layers, and max pooling layers. The convolutional layers are used for feature extraction, the batch normalization layers normalize the outputs of the convolutional layers, and the max pooling layers select the feature with the maximum value in the pooling window. The residual block controls two convolutional layers, which are connected by a batch normalization layer and a residual layer. A self-attention module is fused after each residual block. The output block outputs the 3D velocity of the vehicle and the measurement noise uncertainty. The output block includes a fully connected layer and a batch normalization layer, and multiple dropout layers are set after the fully connected layer and the batch normalization layer.

[0135] In this embodiment, the output block of the three-dimensional velocity estimation network is equipped with a Tanh function. The Tanh function is replaced with a Sigmoid function to perform binary classification of the vehicle state and obtain the zero-speed detection result.

[0136] In this embodiment, the process noise estimation network includes two feature extraction modules, which have the same structure. Each feature extraction module includes a convolutional layer, a ReLU function, and a compression activation module. After the convolutional layer extracts features and activates them using the ReLU function, the compression activation module dynamically adjusts the weights of each channel.

[0137] In this embodiment, the compression excitation module includes a global average pooling layer, a fully connected layer, and a sigmoid function activation. The global average pooling layer performs global aggregation of the spatial information of the channels of each original feature map. The globally aggregated information is activated by the sigmoid function through the fully connected layer. The channels of the original feature map are reweighted based on the activation result and then output.

[0138] In this embodiment of the application, the loss function for uncertainty estimation is:

[0139]

[0140] Where N is the length of the sliding window; y k f(x) serves as the reference truth value. k ) is the input x k The estimated value of the corresponding output; σ(x) k ) 2 It is model uncertainty.

[0141] In this embodiment, the state vector of the extended Kalman filter is defined as follows:

[0142]

[0143] Where, p n Position in the navigation system; v n Speed ​​under navigation system; For posture; b ω For the gyroscope bias in the carrier system; b a Accelerometer bias under load system; The IMU mounting angle between the carrier system and the vehicle system; This refers to the IMU lever connecting the carrier system and the vehicle system.

[0144] The covariance matrix is ​​updated as follows:

[0145]

[0146]

[0147] in, Let be the prior state covariance at time k; Let F be the posterior state covariance at time k-1; k-1 and G k-1 , , are the Jacobian matrices of the nonlinear function of state propagation; Q is the process noise covariance; v is the coordinate transformation matrix from the loading system to the navigation system at time k-1; k-1 p represents the velocity in the navigation system at time k-1. k-1 dt represents the position in the navigation system at time k-1; g represents gravity; dt represents the time interval; (·×) denotes an antisymmetric matrix.

[0148] It should be noted that the foregoing explanation of the embodiment of the vehicle inertial positioning method based on data and model joint drive also applies to the vehicle inertial positioning device based on data and model joint drive of this embodiment, and will not be repeated here.

[0149] The vehicle-mounted inertial positioning device based on data and model joint driving proposed in this application can achieve accurate and robust position estimation by preprocessing the acceleration data and gyroscope data from the vehicle-mounted IMU and inputting them into a three-dimensional velocity estimation network. The network outputs pseudo-measurement constraints and a measurement noise scaling factor, and the measurement noise covariance is calculated based on the measurement noise scaling factor. The preprocessed data is then input into a process noise estimation network, which outputs a process noise scaling factor and a process noise uncertainty. The process noise covariance is calculated based on the process noise scaling factor. A loss function for uncertainty estimation is constructed, and the error penalty weights of the three-dimensional velocity estimation network and the noise estimation network are adjusted. The acceleration data from the vehicle-mounted IMU and the gyroscope data are mechanically arranged and subjected to extended Kalman filtering. Vehicle-mounted inertial positioning is performed based on the filtering results. The three-dimensional velocity of the vehicle and the corresponding uncertainty are estimated based on deep learning. A data-driven noise covariance adapter is introduced to dynamically adjust the process noise covariance.

[0150] Figure 6 is a structural schematic diagram of a vehicle provided in an embodiment of this application. The vehicle may include:

[0151] The memory 301, the processor 302, and the computer program stored on the memory 301 and capable of running on the processor 302.

[0152] When the processor 302 executes the program, it implements the vehicle inertial positioning method based on data and model joint driving provided in the above embodiments.

[0153] Furthermore, the vehicle also includes:

[0154] Communication interface 303 is used for communication between memory 301 and processor 302.

[0155] The memory 301 is used to store computer programs that can run on the processor 302.

[0156] The memory 301 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0157] If the memory 301, processor 302, and communication interface 303 are implemented independently, they can be interconnected via a bus to communicate with each other. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, only one thick line is used in Figure 6, but this does not indicate that there is only one bus or one type of bus.

[0158] Optionally, in a specific implementation, if the memory 301, processor 302, and communication interface 303 are integrated on a single chip, then the memory 301, processor 302, and communication interface 303 can communicate with each other through an internal interface.

[0159] Processor 302 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of this application.

[0160] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0161] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0162] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0163] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (FPGAs), field-programmable gate arrays (FPGAs), etc.

[0164] Those skilled in the art will understand that all or part of the steps of the methods implementing the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0165] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A vehicle-mounted inertial positioning method based on data and model joint driving, characterized in that, Includes the following steps: Preprocess the acceleration and gyroscope data from the vehicle-mounted IMU; The preprocessed data is input into a three-dimensional velocity estimation network, which outputs the vehicle's three-dimensional velocity and measurement noise uncertainty. The vehicle's three-dimensional velocity is used as a pseudo-measurement constraint, and the measurement noise uncertainty is used as a measurement noise scaling factor. The measurement noise covariance is calculated based on the measurement noise scaling factor. The preprocessed data is input into the process noise estimation network, which outputs the process noise scaling factor and the process noise uncertainty. The process noise covariance is calculated based on the process noise scaling factor. Construct a loss function for uncertainty estimation, adjust the error penalty weights of the three-dimensional velocity estimation network according to the loss function and the measurement noise uncertainty, and adjust the error penalty weights of the process noise estimation network according to the loss function and the process noise uncertainty. The acceleration data and gyroscope data of the vehicle-mounted IMU are mechanically arranged. Based on the mechanically arranged data, the pseudo-measurement constraints, the zero-speed detection results, the measurement noise covariance, and the process noise covariance, extended Kalman filtering is performed. Vehicle-mounted inertial positioning is then performed based on the filtering results.

2. The vehicle inertial positioning method based on data and model joint driving according to claim 1, characterized in that, The data preprocessing of the acceleration data and gyroscope data from the vehicle-mounted IMU includes: adding random Gaussian noise to the acceleration data and the gyroscope data; and segmenting the acceleration data and the gyroscope data using a sliding window.

3. The vehicle inertial positioning method based on data and model joint driving according to claim 1, characterized in that, The 3D velocity estimation network includes an input block, a residual block, and an output block. The input block consists of convolutional layers, batch normalization layers, and max pooling layers. The convolutional layers are used for feature extraction, the batch normalization layers normalize the outputs of the convolutional layers, and the max pooling layers select the feature with the maximum value in the pooling window. The residual block controls two convolutional layers, which are connected by a batch normalization layer and a residual layer. A self-attention module is fused after each residual block. The output block outputs the 3D velocity of the vehicle and the measurement noise uncertainty. The output block includes a fully connected layer and a batch normalization layer, followed by multiple dropout layers.

4. The vehicle inertial positioning method based on data and model joint driving according to claim 3, characterized in that, The output block of the three-dimensional velocity estimation network is equipped with a Tanh function. Replacing the Tanh function with a Sigmoid function allows for binary classification of vehicle states to obtain zero-speed detection results.

5. The vehicle inertial positioning method based on data and model joint driving according to claim 1, characterized in that, The process noise estimation network includes two feature extraction modules, which have the same structure. Each feature extraction module includes a convolutional layer, a ReLU function, and a compression activation module. After the convolutional layer extracts features and is activated by the ReLU function, the compression activation module dynamically adjusts the weights of each channel.

6. The vehicle inertial positioning method based on data and model joint driving according to claim 5, characterized in that, The compression excitation module includes a global average pooling layer, a fully connected layer, and a sigmoid function activation. The global average pooling layer performs global aggregation of the spatial information of the channels of each original feature map. The globally aggregated information is then activated by the sigmoid function through the fully connected layer. Based on the activation result, the channels of the original feature map are reweighted and output.

7. The vehicle inertial positioning method based on data and model joint driving according to claim 1, characterized in that, The loss function for the uncertainty estimation is: in, The length of the sliding window; For reference truth value; It is input The corresponding estimated value of the output; It is model uncertainty.

8. The vehicle inertial positioning method based on data and model joint driving according to claim 1, characterized in that, The state vector of the extended Kalman filter is defined as follows: in, Position under the navigation system; Speed ​​under navigation system; As a posture; For the gyroscope bias in the carrier system; Accelerometer bias under load system; The IMU mounting angle between the carrier system and the vehicle system; The IMU linkage between the carrier system and the vehicle system; the covariance matrix is ​​updated as follows: in, for The prior state covariance at time t; for The posterior state covariance at time t; and These are the Jacobian matrices of the nonlinear functions of state propagation, respectively. For process noise covariance; for The coordinate transformation matrix from the time-carrying system to the navigation system; for Speed ​​under constant navigation system; for Location under the real-time navigation system; For gravity; For time intervals; This represents an antisymmetric matrix.

9. A vehicle-mounted inertial positioning device based on data and model joint driving, characterized in that, include: The processing module is used to preprocess the acceleration data and gyroscope data from the onboard IMU; The first calculation module is used to input the preprocessed data into a three-dimensional velocity estimation network. The three-dimensional velocity estimation network outputs the vehicle's three-dimensional velocity and the measurement noise uncertainty. The vehicle's three-dimensional velocity is used as a pseudo-measurement constraint, and the measurement noise uncertainty is used as a measurement noise scaling factor. The measurement noise covariance is calculated based on the measurement noise scaling factor. The second calculation module is used to input the preprocessed data into the process noise estimation network, the process noise estimation network outputs the process noise scaling factor and the process noise uncertainty, and calculates the process noise covariance based on the process noise scaling factor. An adjustment module is used to construct a loss function for uncertainty estimation, adjust the error penalty weights of the three-dimensional velocity estimation network according to the loss function and the measurement noise uncertainty, and adjust the error penalty weights of the process noise estimation network according to the loss function and the process noise uncertainty. The filtering module is used to mechanically arrange the acceleration data and gyroscope data of the vehicle-mounted IMU, perform extended Kalman filtering based on the mechanically arranged data, the pseudo measurement constraints, the zero-speed detection results, the measurement noise covariance and the process noise covariance, and perform vehicle-mounted inertial positioning based on the filtering results.

10. A vehicle, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the vehicle inertial positioning method based on data and model joint drive as described in any one of claims 1-8.

Images