An IMU gyroscope random error compensation method and device based on deep learning

Abstract

A kind of IMU gyroscope random error compensation method and device based on deep learning, comprising: obtaining IMU data;The IMU data is input into the random error compensation model constructed in advance, to obtain clean gyroscope data;Wherein, the random error model is based on the Gyro-Net network training obtained from IMU data;The present application extracts and selects IMU data features by IFESblock, can adjust feature flow by semi-dense network structure and carry out reuse to reach depth supervision and sufficient use of features, and utilizes relative loss and absolute loss function to train to reach attention local precision and global precision, with better performance in direction estimation.

Description

Technical Field

[0001] This invention relates to the field of robot navigation, specifically to a method and apparatus for compensating random errors of an IMU gyroscope based on deep learning. Background Technology

[0002] Heading estimation plays a crucial role in the overall state estimation of odometry for robots. When estimating the robot's trajectory, even small errors in direction can lead to significant errors in position estimation, ultimately causing the entire odometry system to fail. Inertial Measurement Units (IMUs) are commonly used for heading estimation to address these issues due to their advantages such as immunity to external visual interference, low power consumption, and high operating frequency. However, low-precision IMUs are susceptible to environmental temperature fluctuations, exhibiting random errors such as bias and random walk. Over extended periods, integration can lead to pose divergence, resulting in non-negligible cumulative errors and significantly reduced data reliability. Therefore, compensating for random errors in the gyroscope is essential to improve the accuracy of IMU heading estimation.

[0003] IMUs are typically used in conjunction with complementary sensors, enabling robotic systems to fuse features from multiple sensors for more robust pose estimation. For example, some methods incorporate more information by using cameras, LiDAR, and GNSS, fusing various data sources for pose estimation. Other methods use IMU data to obtain intermediate values ​​in the localization system, combine them with data from other sensors, and input them into an Extended Kalman Filter (EKF) for pose estimation. Because IMUs contain various errors, they are usually calibrated using the Allen variance method before use; however, this calibration cannot accurately estimate time-varying errors.

[0004] With the rise of deep learning, many data-driven methods have been applied to IMU information processing, which can be divided into supervised learning methods and unsupervised learning methods according to the type of training data. Most existing works extract relevant features from sensor signals and propagate them in real time using recurrent neural networks (RNNs), while recent works use five-layer convolutional neural networks (CNNs) for pose estimation. However, existing algorithms do not consider the intrinsic connection between accelerometers and gyroscopes, and they do not make full use of IMU information. Summary of the Invention

[0005] To address the problem of inaccurate heading estimation after long-term operation of inertial measurement units (IMUs) in existing technologies, this invention proposes a deep learning-based method for compensating for random errors in IMU gyroscopes, including:

[0006] Acquire IMU data;

[0007] The IMU data is input into a pre-built random error compensation model to obtain clean gyroscope data;

[0008] The random error model is obtained by training the Gyro-Net network based on IMU data.

[0009] Preferably, the construction of the random error compensation model includes:

[0010] Data augmentation and data padding are performed based on the IMU data to obtain processed data;

[0011] The processed data is then input into the Gyro-Net network to obtain clean gyroscope data;

[0012] The clean gyroscope data is integrated to obtain quaternions, which are then used to construct relative loss functions and absolute loss functions with the quaternions in the IMU data.

[0013] The Gyro-Net network is backpropagated based on the relative loss function and the absolute loss function. After iterative training, the loss function converges, and a trained random error compensation model is obtained.

[0014] Preferably, the step of inputting the processed data into the Gyro-Net network to obtain clean gyroscope data includes:

[0015] The IMU data was convolved using two sets of convolution kernels with different receptive fields to obtain features at different scales;

[0016] After global average pooling based on the features at different scales, the feature maps are embedded into the channels. After two fully connected layers and softmax calculation, the importance of each channel to heading estimation is obtained.

[0017] The importance of each channel to heading estimation is multiplied by the different scale features to obtain clean gyroscope data.

[0018] Preferably, the importance of each channel to the heading estimation is calculated using the following formula:

[0019] W att =σ(W fc1+fc2+av Head (W c1 (F)+W c2 (F)))

[0020] In the formula, W att The importance of each channel to heading estimation, W c1 For the processing of IMU information by grouped dilated convolution, W c2 For grouped dilated convolution processing of IMU information, σ is the softmax function, and W... fc1+fc2+av目 The convolutional layers are fused into groups, then processed through two fully connected layers and global average pooling. F is the output of the previous convolutional layer.

[0021] Preferably, the clean gyroscope data is calculated using the following formula:

[0022] O = W att *(F1+F2)=W att *(W conv1 (F)+W conv2 (F))

[0023] In the formula, W att The importance of each channel for heading estimation is represented by F1, the features obtained from the first convolutional process, and F2, the features obtained from the second convolutional process. W... conv1 (F) represents the features obtained from the first convolution process, W conv2 (F) represents the feature obtained from the second convolution, F is the output of the previous convolution, and O is the output value of a single IFESBlock.

[0024] Preferably, the step of integrating the clean gyroscope data to obtain quaternions and then constructing relative and absolute loss functions with the quaternions in the IMU data includes:

[0025] The increment of the quaternion is obtained by integrating the clean gyroscope count;

[0026] The loss function is calculated based on the incremental combination of the quaternion and the loss function formula.

[0027] The relative loss function is calculated based on the aforementioned loss function and the relative loss function calculation formula.

[0028] The absolute loss function is calculated based on the increment of the quaternion and the formula for calculating the absolute loss function.

[0029] Preferably, the loss function is calculated as follows:

[0030]

[0031] In the formula, L j Let δq be the loss of quaternions within j integration time points, M be the number of integration intervals of type j in the entire sequence, N be the length of the sequence, j be the number of sampling time points to be integrated, and δq be the value of δq. k,k+jThe quaternion increment for integrating over j sampling times, where k is the k-th sampling time, j is the number of sampling times to be integrated, and Ω -1 To convert from quaternions to angles.

[0032] Preferably, the relative loss function is calculated as follows:

[0033] L rl =L 16 +0.5*L 32

[0034] In the formula, L rl For relative angle loss, L 16 L represents the relative loss of quaternions over 16 sampling times. 32 The relative loss of quaternions over 32 sampling times.

[0035] Preferably, the absolute loss function is calculated as follows:

[0036]

[0037] In the formula, L al The loss is the absolute angle loss, where m is the number of integration intervals of type j in the entire sequence, N is the length of the sequence, and δq is the absolute angle loss. 0,64 δq represents the quaternion increment for the first 64 sampling times. 64*(m-1),64*m The sampling time ranges from the (m-1)th to the mth 64th sampling time, where k is the kth sampling time.

[0038] Based on the same inventive concept, this invention also proposes a deep learning-based IMU gyroscope random error device, comprising:

[0039] The acquisition module is used to acquire IMU data;

[0040] The model solving module is used to input the IMU data into a pre-built random error compensation model to obtain clean gyroscope data;

[0041] The random error model is obtained by training the Gyro-Net network based on IMU data.

[0042] Preferably, the apparatus further includes a model building module; the model building module includes:

[0043] The data processing submodule is used to perform data augmentation and data padding based on the IMU data to obtain processed data;

[0044] The network solving submodule is used to input the processed data into the Gyro-Net network to obtain clean gyroscope data;

[0045] The loss function calculation submodule is used to integrate the clean gyroscope data to obtain quaternions, and then construct relative loss function and absolute loss function with quaternions in the IMU data;

[0046] The model training submodule is used to backpropagate the Gyro-Net network based on the relative loss function and the absolute loss function. After iterative training, the loss function converges, and a trained random error compensation model is obtained.

[0047] Preferably, the network solving submodule is specifically the same as:

[0048] The IMU data was convolved using two sets of convolution kernels with different receptive fields to obtain features at different scales;

[0049] After global average pooling based on the features at different scales, the feature maps are embedded into the channels. After two fully connected layers and softmax calculation, the importance of each channel to heading estimation is obtained.

[0050] The importance of each channel to heading estimation is multiplied by the different scale features to obtain clean gyroscope data.

[0051] Preferably, the loss function calculation submodule is specifically used for:

[0052] The increment of the quaternion is obtained by integrating the clean gyroscope count;

[0053] The loss function is calculated based on the incremental combination of the quaternion and the loss function formula.

[0054] The relative loss function is calculated based on the aforementioned loss function and the relative loss function calculation formula.

[0055] The absolute loss function is calculated based on the increment of the quaternion and the formula for calculating the absolute loss function.

[0056] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0057] A method and apparatus for random error compensation of IMU gyroscopes based on deep learning includes: acquiring IMU data; inputting the IMU data into a pre-constructed random error compensation model to obtain clean gyroscope data; wherein the random error model is obtained by training a Gyro-Net network based on the IMU data; this invention extracts and selects IMU data features through IFESblock, and the semi-dense network structure can adjust the feature flow direction and reuse features to achieve deep supervision and full utilization of features, and uses relative loss and absolute loss functions for training to achieve focus on local accuracy and global accuracy, resulting in better performance in orientation estimation. Attached Figure Description

[0058] Figure 1 This is a flowchart of a deep learning-based random error compensation method for IMU gyroscopes according to the present invention.

[0059] Figure 2 This is a schematic diagram of the method steps of the present invention;

[0060] Figure 3 This is a schematic diagram showing the implementation details and loss function of Gyro-Net in this invention;

[0061] Figure 4 This is the relative heading error (AOE) diagram of the present invention;

[0062] Figure 5 This is the heading estimation diagram for the challenge sequences MH_04_difficult and V1_03_difficult of the present invention;

[0063] Among them, ①, the relative angle error of Denoise-IMU with respect to position; ②, the relative angle error of Gyro-Net with respect to position; ③, the relative angle error of OpenVINS with respect to position; ④, the relative angle error of OpenVINS_prop with respect to position; ⑤, the relative angle error of VINS-Mono with respect to position; and ⑥, the relative angle error of VINS-Mono with respect to position, i.e., VINS-Mono with loop closure detection. Detailed Implementation

[0064] To address the issue of inaccurate heading estimation after long-term operation of an Inertial Measurement Unit (IMU), we propose a learning-based method called Gyro-Net to estimate and compensate for random errors in the IMU's gyroscopes. The random error compensation system takes acceleration and gyroscope information from the IMU data as input and mainly consists of three parts: data preprocessing, a deep neural network, and a loss function. Finally, it obtains gyroscope data to compensate for random errors and then uses integration to obtain quaternions. To better understand this invention, the following description, in conjunction with the accompanying drawings and embodiments, further illustrates the invention.

[0065] Example 1:

[0066] A deep learning-based method for compensating random errors in an IMU gyroscope, the specific process of which is as follows: Figure 1 As shown, it includes:

[0067] Step 1, acquire IMU data;

[0068] Step 2: Input the IMU data into a pre-built random error compensation model to obtain clean gyroscope data;

[0069] The random error model is obtained by training the Gyro-Net network based on IMU data.

[0070] The construction of a random error compensation model includes:

[0071] The first step is to preprocess the IMU data. The data preprocessing stage requires two steps: data augmentation and data imputation. Gaussian noise is added to the IMU data to augment the data and avoid overfitting during training. Secondly, since the longer the IMU sensor operates, the larger the error, the value of the first sampling time is used to imput the IMU data and learn the characteristics of the IMU when the error is small.

[0072] The second step is to feed the data processed in the first step into the designed network for training, such as... Figure 2 As shown.

[0073] According to the measurement model of the gyroscope in the IMU

[0074]

[0075] In the formula, S g It is the scaling factor, M g S represents the misalignment error between shafts. ga The g-sensitivity represents the effect of the acceleration measurement on the gyroscope measurement, η is Gaussian white noise with a mean of 0, and b is the bias term. The true value of the acceleration in the body coordinate system. This represents the measurement value at the k-th sampling time in the body coordinate system. Let η be the true value at the k-th sampling time in the body coordinate system. g b represents the noise term of the gyroscope. g q represents the bias term of the gyroscope, k is the abbreviation for gyroscope, k is the kth sampling time, and b is the abbreviation for body coordinate system.

[0076] Due to S ga The existence of accelerometer readings indicates that gyroscope measurements are influenced by accelerometer readings. Therefore, an attention mechanism is introduced into IMU information processing, adjusting the weight of acceleration and gyroscope readings on the task using channel attention. Simultaneously, to extract richer IMU features, grouped dilated convolution is introduced, forming the IMU Feature Extraction & Selection Block (IFESBlock) module together with the aforementioned channel attention. This module extracts IMU features at different scales, considering the intrinsic relationship between acceleration and gyroscope readings, and calculates weights for each channel to achieve the optimal feature combination.

[0077] The formula for calculating attention in the IMU feature extraction and selection module:

[0078] W att =σ(W fc1+fc2+avg (W c1 (F)+W c2 (F)))

[0079] In the formula, W att The importance of each channel to heading estimation, W c1 For the processing of IMU information by grouped dilated convolution, W c2 For grouped dilated convolution processing of IMU information, σ is the softmax function, and W... fc1+fc2+avg The process involves grouped convolutional fusion followed by two fully connected layers and global average pooling.

[0080] The final output calculation formula of the IMU feature extraction and selection module is as follows:

[0081] O = W att *(F1+F2)=W att *(W conv1 (F)+W conv2 (F))

[0082] In the formula, W att The importance of each channel for heading estimation is represented by F1, the features obtained from the first convolutional process, and F2, the features obtained from the second convolutional process. W... conv1 F represents the features obtained from the first convolution process, and W represents the features obtained from the first convolution process. conv2 F represents the feature obtained from the second convolution, F represents the output of the previous convolution, and O represents the output value of a single IFESBlock.

[0083] The convolutional layer structure mentioned in this article is called the IMU Feature Compress Block (IFC Block), which is used to reduce the number of parameters in the network and shorten the training time. At the same time, to utilize features more effectively, a semi-dense network structure is designed as follows: Figure 3 As shown, the feature flow direction is adjusted, and feature reuse between different blocks is achieved before and after feature extraction and selection and before and after feature compression through skip connections, thereby realizing the effective use of IMU information.

[0084] To reduce the impact of accumulated error on the overall heading estimation, we designed an absolute angle loss function, which allows the entire network to focus on error accumulation over a longer period.

[0085] Finally, the preprocessed data is passed through the Gyro-Net network to obtain clean gyroscope data (CleanGyro). Quaternions are obtained through CleanGyro integration, and these are then used to construct relative and absolute loss functions with the quaternions in the ground truth. Backpropagation is performed on the network, and after 1800 training iterations, the loss functions converge, resulting in a random error compensation network model for the gyroscope. During the testing phase, acceleration and gyroscope data are input into the trained model, and random error compensation is performed to obtain clean gyroscope data.

[0086] The core of this algorithm lies in its well-designed network structure and full utilization of IMU characteristics, achieving excellent results on both high-speed drone movements and handheld devices. Simply using the output angular velocity can also effectively enhance the VIO method. In the future, we hope to design a more efficient method to combine network output and the VIO system for even better performance.

[0087] Step 1, acquire IMU data;

[0088] Step 2: Input the IMU data into a pre-built random error compensation model to obtain clean gyroscope data.

[0089] Example 2:

[0090] A deep learning-based IMU gyroscope random error device includes:

[0091] The acquisition module is used to acquire IMU data;

[0092] The model solving module is used to input the IMU data into a pre-built random error compensation model to obtain clean gyroscope data;

[0093] The random error model is obtained by training the Gyro-Net network based on IMU data.

[0094] It also includes a model building module; the model building module includes:

[0095] The data processing submodule is used to perform data augmentation and data padding based on the IMU data to obtain processed data;

[0096] The network solving submodule is used to input the processed data into the Gyro-Net network to obtain clean gyroscope data;

[0097] The loss function calculation submodule is used to integrate the clean gyroscope data to obtain quaternions, and then construct relative loss function and absolute loss function with quaternions in the IMU data;

[0098] The model training submodule is used to perform backpropagation on the deep neural network based on the relative loss function and the absolute loss function. After iterative training, the loss function converges, and a trained random error compensation model is obtained.

[0099] The network solving submodule is specifically the same as:

[0100] Inspired by ResNet and DenceNet, this invention proposes a novel network architecture. It consists of a data input module, an IFESBlock feature extraction and selection module, an IFCBlock feature compression module, and a data output head. When estimating heading, considering that both gyroscope data and acceleration data are needed, and that the two types of data contribute differently to the task, we designed the IFES Block. This module uses two sets of convolutions with different receptive fields to extract richer IMU features, assigning different weights to each channel and dynamically adjusting the features obtained from the two sets of different receptive fields.

[0101] Simultaneously, the IMU feature compression module IFCblock is used to connect two IFESblocks for feature compression. To prevent feature information loss during feature transfer between different blocks, skip connections and transition layers are used to reuse features. In this way, we construct a semi-dense network structure to adjust the flow of features, enhance feature transfer, utilize features more effectively, and improve the model's compactness. The data output header uses a 1*1 convolution to set the number of channels to 3, representing the gyroscope values ​​in the x, y, and z components after compensating for random errors.

[0102] The output of IFCBlock is connected to the next IFCBlock via a skip connection and to a second non-adjacent IFESBlock via a transition layer, forming a semi-dense network structure. This network structure facilitates feature reuse and results in a more compact model. The transition layer reduces the scale of the feature maps and the number of parameters. Furthermore, to enable the network to handle sharp points caused by rapid motion in the IMU signal, the ReLU activation function is used in the last layer of the transition layer. The network parameters corresponding to the transition layer are shown in Table 1 (transitionlayer). The semi-dense network structure allows some layers to receive additional supervision signals through shorter connections during backpropagation, achieving implicit deep supervision. This enhances the feature flow between different blocks, making it easier for the error function to propagate to earlier layers.

[0103] Inspired by SKNet, we designed IFESblock to extract acceleration and angular velocity features from the IMU. During the first IFESblock input, the six channels are assigned to the x, y, and z components of acceleration and gyroscope, respectively. Channel attention is then used to adjust the weighting of acceleration and gyroscope in the task, achieving dynamic feature reorganization in different scenarios. Specifically, this module first uses two sets of convolutional kernels with different receptive fields to convolve the IMU feature map to obtain features at different scales. Then, global average pooling is performed to embed the feature map into the channels. After two fully connected layers and softmax, the importance of each channel for heading estimation is obtained, calculated as follows:

[0104] W att =σ(W fc1+fc2+avg (W c1 (F)+W c2 (F)))

[0105] In the formula, W att The importance of each channel to heading estimation, W c1 For the processing of IMU information by grouped dilated convolution, W c2 For grouped dilated convolution processing of IMU information, σ is the softmax function, and W... fc1+fc2+avg The process involves grouped convolutional fusion followed by two fully connected layers and global average pooling.

[0106] Finally, the weights on each channel are multiplied by the feature maps obtained from the group convolutions, resulting in the final network output:

[0107] O = W att *(F1+F2)=W att *(W conv1 (F)+W conv2 (F))

[0108] In the formula, W att The importance of each channel for heading estimation is represented by F1, the features obtained from the first convolutional process, and F2, the features obtained from the second convolutional process. W... conv1 (F) represents the features obtained from the first convolution process, W conv2 (F) represents the feature obtained from the second convolution, F is the output of the previous convolution, and O is the output value of a single IFESBlock.

[0109] The specific parameters of this module in the network layer are shown in the table below:

[0110]

[0111] Table 1 Gyro-Net Network Parameter Settings

[0112] In this context, FC-setting represents the feature scaling transformation in the FC layer. 6-32-6 means that the first FC layer has an input of 6 and an output of 32, while the second FC layer has an input of 32 and an output of 6. The first FC layer expands layers with fewer channels to prevent information loss and compresses layers with more channels to reduce network parameters. The second FC layer predicts channel importance. 7-9 indicates that the sizes of the two sets of convolutions in IFESBlock are 7 and 9, respectively. conv represents the kernel size; Dia represents the dilation rate; FC-Setting represents the parameters of the FC layer in IFESBlock. In-Out represents the number of input and output channels indicated by the asterisk. AcFun represents the activation function.

[0113] The loss function calculation submodule is specifically used for:

[0114] Based on the measurement model of the gyroscope in the IMU, the random error compensation of the gyroscope is modeled as follows:

[0115]

[0116] In the formula, This represents the measurement value at the k-th sampling time in the body coordinate system. Let S be the estimated value at the k-th sampling time in the body coordinate system. g M g )^ represents the estimated values ​​of the scale factor and the inter-axis non-alignment error. This represents an estimate of the random error of the gyroscope. This represents an estimate of the random error of the gyroscope. These are estimates of the scaling factor and the inter-axis non-alignment error.

[0117] Among them, the random error of the gyroscope The estimate is obtained by neural network calculation:

[0118]

[0119] In the formula, f(.) represents a function defined by the neural network. This represents the measurements from the accelerometer and gyroscope at the k-th moment in the body coordinate system. This represents the measurements taken by the accelerometer and gyroscope at the kN-th time in the body coordinate system of the IMU.

[0120] To enable the network to focus on both local and global accuracy, we designed relative and absolute loss functions:

[0121] The relative loss function references the local loss of the design. To pair the estimated value with the true value, the... The increment of the quaternion over a sampling time period is calculated by integration as follows:

[0122]

[0123] In the formula, Ω(.) represents the conversion of angle to quaternion, parameter j represents the number of angular velocities in each integration, and δq k,k+j Let the quaternion increment be the integral over j sampling times. q is the inverse of the quaternion value at the k-th sampling time. k+j For the (k+j)th sampling time, ω is the quaternion value. k Let denot be the angular velocity value at the k-th sampling time, δt be the time difference between the k-th and (k+j)-th sampling times, k be the k-th sampling time, i be the initial time for calculating the increment of the quaternion within a sampling time interval, and j be the sampling time interval for calculating the increment of the quaternion.

[0124] The number of integrations per iteration is determined by the parameter j, and the loss function is defined as follows:

[0125]

[0126] In the formula, L j Let δq be the loss of quaternions within j integration time points, M be the number of integration intervals of type j in the entire sequence, N be the length of the sequence, j be the number of sampling time points to be integrated, and δq be the value of δq. k,k+j Let H(.) be the quaternion increment for integrating over j sampling times, where k is the k-th sampling time, j is the number of sampling times to be integrated, H(.) denotes the Huber function, and Ω is the quadrature increment for integrating over j sampling times. -1 (.) indicates the conversion from quaternions to angles, therefore the relative loss function is defined as follows:

[0127] L rl =L 16 +0.5*L 32

[0128] In the formula, L rl For relative angle loss, L 16 L represents the relative loss of quaternions over 16 sampling times. 32 The relative loss of quaternions over 32 sampling times.

[0129] Absolute Loss: To avoid the impact of accumulated errors caused by long-term system operation on accuracy, the specific mathematical expression of the loss function is as follows:

[0130]

[0131] In the formula, L al The loss is the absolute angle loss, where m is the number of integration intervals of type j in the entire sequence, N is the length of the sequence, and δq is the absolute angle loss.0,64 δq represents the quaternion increment for the first 64 sampling times. 64*(m-1),64*m The sampling time ranges from the (m-1)th to the mth 64th sampling time, where k is the kth sampling time.

[0132] The final loss function is defined as follows:

[0133] L all =α1L rl +α2L al

[0134] L all This represents the final loss function, where α1 represents the weights of the relative loss, and L... rl This represents the relative loss, where α2 represents the weight of the relative loss, and L... al This represents the relative loss, where the weight parameter defaults to 1.

[0135] Example 3:

[0136] This invention was conducted using two widely applicable datasets, EuRoC and TUM-VI. To ensure a fair comparison of each metric, the deep learning method was trained and tested on both datasets using the same training and test sets. EuRoC consisted of 11 sequences, acquired by MAV in two scenes using images and 200Hz IMU data. The training set included MH_01_easy, MH_03_medium, MH_05_difficult, V1_02_medium, V2_01_easy, and V2_03_difficult; the test set included MH_02_easy, MH_04_difficult, V2_02_medium, V1_03_difficult, and V1_01_easy. The TUM-VI dataset consisted of appropriately calibrated images and 200Hz IMU data acquired by a handheld device in different scenes. Due to occlusion of the motion capture system by the handheld device for a certain period, each device was unusable for 0.2 seconds, which needed to be removed. The training set consists of room1, room3, and room5, while the test set consists of room2, room4, and room6. The training duration for each dataset is the first 50 seconds of the sequence, with the remaining portion used for validation.

[0137] This invention is implemented using PyTorch 1.10.1, selecting the ADAM optimizer and setting the learning rate to 0.01. In the EuRoC dataset, formula L... all =α1L rl +α2L alWe used the default values; however, our designed absolute loss needs to consider the sequential relationship of the sequences. Therefore, for the TUM-VI dataset with missing ground truth values, we set the parameters to α1 = 1 and α2 = 0.2, and trained for 1800 epochs. All experiments were conducted using a GTX3090 device, each training sequence took approximately 33 seconds, and the GPU memory consumption was less than 6GB.

[0138] This invention calculates five indicators to evaluate the heading estimation results when quantitatively assessing the estimation results. ROE represents the relative error between the model-predicted angle and the actual value every 7 meters; AOE, ARE, AYE, and APE represent the absolute errors between the model-predicted Euler angles and the actual values. For all five indicators, a smaller value is better. The specific calculation formulas are as follows:

[0139]

[0140] In the formula, n represents the timestamp, and gn represents the displacement of the IMU corresponding to the corresponding timestamp. This represents the transpose of the true value of the rotational increment at the corresponding displacement. This represents the estimated value of the rotational increment under the corresponding displacement.

[0141]

[0142] In the formula, M is the length of the sequence, and log(.) is the logarithmic mapping of SO(3). Represents the transpose of the truth value of a rotation of a sequence. This represents the rotation estimate of the sequence, where n represents the timestamp.

[0143]

[0144] In the formula, M is the length of the sequence, and A roll The true value representing the roll angle. This represents the estimated roll angle, and n represents the timestamp.

[0145]

[0146] In the formula, M is the length of the sequence, and A yaw The true value representing the yaw angle. The value represents the estimated yaw angle, and n represents the timestamp.

[0147]

[0148] Where M is the length of the sequence, A pitch The true value representing the pitch angle. The pitch angle is the estimated value, and n represents the timestamp.

[0149] The results of the method proposed in this invention and other comparative methods on AOE and ROE are as follows: Figure 4 And as shown in the table below:

[0150]

[0151]

[0152] Table 2 shows the absolute heading error (AOE) on the heading / yaw scale for the test sequence, in deg.

[0153] We can see that our proposed Gyro-Net outperforms existing algorithms on the test data. Applying our results directly to Open-VINS, it achieves better accuracy in heading estimation than the original VIO method. (Bold text indicates the best results for the VIO system; unbold text indicates the best performance of the inertial method.)

[0154] Figure 4 It is clear from the data that on each sub-trajectory, Gyro-Net's ROE is lower than that of Denoise-IMU, and it is comparable to Open-VINS and VINS-Mono. Meanwhile, Open-VINS(prop) is the most stable among the compared VIO systems, with minimal error variation.

[0155] Since the roll and pitch angles become considerable in the VIO system, only the absolute error in the yaw direction was calculated when comparing with the VIO algorithm. Furthermore, to demonstrate that our proposed method is effective in estimating in all three directions, the best-performing Denoise-IMU and Gyro-Net were compared in terms of their Euler angle estimates.

[0156] The method proposed in this invention not only achieves accurate estimation results in yaw, but also performs effective random error estimation in roll and pitch directions, improving the overall heading estimation results and exceeding the baseline overall, as shown in the table below:

[0157]

[0158] Table 3. Estimated values ​​of Denoise-IMU and Gyro-Net in the roll / pitch / yaw directions on the test sequence.

[0159] To illustrate this more clearly, we statistically analyzed the motion characteristics of the test sequences and selected some challenging scenarios for testing, such as MH_04_difficult, which involves rapid motion in a dark environment, and V1_03_difficult, which involves significant changes in lighting and perspective, as well as motion blur. We observed the estimation performance of the two sequences in three Euler angle directions along the entire time axis. Figure 5 As shown. At the same time, in order to more clearly describe the transformation of the curve, the roll direction values ​​of the MH_04_difficult and V1_03_difficult sequences are changed to the range [0, 180]. It can be found that the method of the present invention, i.e. the orange line, is closer to the true value and has a smaller estimation error. The method of the present invention can estimate the heading with a small cumulative error on both sequences, especially at some inflection points where our method is clearly superior to

[17] . When the MH_04_difficult sequence is moving at high speed, flying at a high speed of 2.50m / s-2.86m / s between 0.73s-0.75s; and flying at a high speed of 2.51m / s-2.63m / s between 1.20s-1.21s, the method of the present invention achieves better local relative accuracy and global accuracy in heading estimation.

[0160] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0161] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0162] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0163] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0164] The above are merely embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of the claims of the present invention pending approval.

Claims

1. A method for compensating random errors in an IMU gyroscope based on deep learning, characterized in that, include: Acquire IMU data; The IMU data is input into a pre-built random error compensation model to obtain clean gyroscope data; The random error compensation model is obtained by training the Gyro-Net network based on IMU data. The construction of the random error compensation model includes: Data augmentation and data padding are performed based on the IMU data to obtain processed data; The processed data is then input into the Gyro-Net network to obtain clean gyroscope data; The clean gyroscope data is integrated to obtain quaternions, which are then used to construct relative loss functions and absolute loss functions with the quaternions in the IMU data. The Gyro-Net network is backpropagated based on the relative loss function and the absolute loss function. After iterative training, the loss function converges, and a trained random error compensation model is obtained. The process of inputting the processed data into the Gyro-Net network to obtain clean gyroscope data includes: The IMU data was convolved using two sets of convolution kernels with different receptive fields to obtain features at different scales; After global average pooling based on the features at different scales, the feature maps are embedded into the channels. After two fully connected layers and softmax calculation, the importance of each channel to heading estimation is obtained. The importance of each channel to heading estimation is multiplied by the different scale features to obtain clean gyroscope data.

2. The method according to claim 1, characterized in that, The importance of each channel to the heading estimation is calculated using the following formula: In the formula, The importance of each channel for heading estimation For the processing of IMU information by grouped dilated convolution, For the processing of IMU information by grouped dilated convolution, For the softmax function, After grouped convolutional fusion, the mixture undergoes two fully connected layers and global average pooling. This is the output of the previous convolutional layer.

3. The method according to claim 1, characterized in that, The clean gyroscope data is calculated using the following formula: In the formula, The importance of each channel for heading estimation The features obtained from the first convolution process, The features obtained from the second convolution process, The features obtained from the first convolution process are... The features obtained from the second convolution process, This is the output of the previous convolutional layer. This is the output value of the IFES Block, a single feature extraction and selection module.

4. The method according to claim 1, characterized in that, The step of integrating the clean gyroscope data to obtain quaternions and then constructing relative and absolute loss functions with the quaternions in the IMU data includes: The increment of the quaternion is obtained by integrating the clean gyroscope count; The loss function is calculated based on the incremental combination of the quaternion and the loss function formula. The relative loss function is calculated based on the aforementioned loss function and the relative loss function calculation formula. The absolute loss function is calculated based on the increment of the quaternion and the formula for calculating the absolute loss function.

5. The method according to claim 4, characterized in that, The loss function is calculated as follows: In the formula, Let be the loss of the quaternion over j integration time points. Let j be the number of integration intervals of type j contained in the entire sequence. The length of the sequence. The number of sampling times that need to be integrated. Let the quaternion increment be the integral over j sampling times. For the k-th sampling time, The number of sampling times that need to be integrated. To convert from quaternions to angles, This refers to the Huber function.

6. The method according to claim 4, characterized in that, The relative loss function is calculated as follows: In the formula, For relative angle loss, The relative loss of quaternions over 16 sampling times. The relative loss of quaternions over 32 sampling times.

7. The method according to claim 4, characterized in that, The absolute loss function is calculated as follows: In the formula, For absolute angle loss, Let j be the number of integration intervals of type j contained in the entire sequence. The length of the sequence. This represents the quaternion increment for the first 64 sampling times. For the (m-1)th 64th sampling time to the mth 64th sampling time, This refers to the Huber function.

8. A deep learning-based IMU gyroscope random error compensation device, characterized in that, include: The acquisition module is used to acquire IMU data; The model solving module is used to input the IMU data into a pre-built random error compensation model to obtain clean gyroscope data; The random error compensation model is obtained by training the Gyro-Net network based on IMU data. The device further includes a model building module; the model building module includes: The data processing submodule is used to perform data augmentation and data padding based on the IMU data to obtain processed data; The network solving submodule is used to input the processed data into the Gyro-Net network to obtain clean gyroscope data; The loss function calculation submodule is used to integrate the clean gyroscope data to obtain quaternions, and then construct relative loss function and absolute loss function with quaternions in the IMU data; The model training submodule is used to backpropagate the Gyro-Net network based on the relative loss function and the absolute loss function. After iterative training, the loss function converges, and a trained random error compensation model is obtained. The network solving submodule includes: The IMU data was convolved using two sets of convolution kernels with different receptive fields to obtain features at different scales; After global average pooling based on the features at different scales, the feature maps are embedded into the channels. After two fully connected layers and softmax calculation, the importance of each channel to heading estimation is obtained. The importance of each channel to heading estimation is multiplied by the different scale features to obtain clean gyroscope data.

9. The apparatus according to claim 8, characterized in that, The loss function calculation submodule is specifically used for: The increment of the quaternion is obtained by integrating the clean gyroscope count; The loss function is calculated based on the incremental combination of the quaternion and the loss function formula. The relative loss function is calculated based on the aforementioned loss function and the relative loss function calculation formula. The absolute loss function is calculated based on the increment of the quaternion and the formula for calculating the absolute loss function.

Images