A domain-adaptive inertial sensor denoising algorithm based on meta-learning

Abstract

This patent discloses a domain-adaptive inertial sensor denoising algorithm based on meta-learning. Since inertial sensors will exhibit different error characteristics in different application scenarios, previous denoising algorithms do not have the generalization ability for different scenarios. To address this problem, this algorithm can use the same denoising network to achieve excellent denoising effects on different inertial sensors in different environments through a meta-learning domain-adaptive method. The domain adaptation framework in the algorithm is used for denoising, and a specially designed reconstruction loss is used to improve domain adaptation and representation capabilities. In addition, in order to improve adaptability under limited data, the algorithm uses a meta-learning strategy to improve the generalization ability of the network, and experimental results prove the effectiveness of the algorithm's denoising.

Description

Technical Field

[0001] The present invention relates to the technical field of inertial sensor denoising, and in particular to a domain-adaptive inertial sensor denoising algorithm based on meta-learning. Background Art

[0002] Inertial measurement units (IMUs) play an important role in robotics. They consist of gyroscopes that measure angular velocity and accelerometers that can measure linear acceleration signals on three axes in space. Both angular velocity and acceleration are crucial for robots.

[0003] The rapid development of robotics technology has put forward higher requirements on inertial sensors. For example, the same inertial sensor may need to adapt to different robots, or different inertial sensors can be calibrated with the same parameters. Since robots usually use low-cost inertial sensors, the noise problem of inertial sensors is very serious. Its demand can be summarized as processing gyroscope noise from multiple fields (different inertial sensors and different robots). Although there are many methods for gyroscope denoising, from nonlinear filtering methods that use filters to denoise inertial sensor signals to neural network methods, they only perform well in a specific field or require large enough data to achieve generalization.

[0004] The existing deep learning-based gyro noise reduction algorithm does not have the generalization capability for multiple environments and multiple sensors. The main reasons are concentrated in the following two aspects:

[0005] 1. The representation ability of the denoising network structure is poor, and it is difficult to process inertial sensor data well. Inertial sensor data is long sequence, low-dimensional, and highly variable time series information. For an end-to-end network structure, using a single loss for supervised training will make it difficult for the network structure to represent inertial data well.

[0006] 2. The training mode does not consider multi-environment generalization. It only achieves good denoising effects for specific usage scenarios or inertial sensors: The error characteristics of inertial data collected by different usage environments and different inertial sensors are different. Such denoising methods are difficult to adapt to various usage scenarios and need to be recalibrated after entering a new usage environment. This problem is more prominent on low-cost inertial sensors. Summary of the invention

[0007] The purpose of the present invention is to provide a domain-adaptive inertial sensor denoising algorithm based on meta-learning, propose a new network structure and reconstruction loss, improve the network representation effect, and pave the way for generalization in different environments. At the same time, a meta-learning training scheme is adopted to further improve the generalization ability in multiple environments.

[0008] The present invention is implemented by the following technical solution, a domain adaptive inertial sensor denoising algorithm based on meta-learning, comprising the following steps:

[0009] Step 1: Establish a noise-free angular velocity model of the inertial sensor;

[0010] Step 2: Design an embedding module to project data with large distribution differences into a new similarity space;

[0011] Step 3: Design a generation module and denoising loss for denoising;

[0012] Step 4: Design a reconstruction module and reconstruction loss for better data representation capabilities;

[0013] Step 5: Design a small sample domain adaptation strategy for inertial sensor denoising.

[0014] Preferably, the noise-free angular velocity model of the inertial sensor is:

[0015] (1);

[0016] In formula (1) For parameters, parameters , is the angular velocity measured by the inertial sensor, is the gyro bias, gyro bias ,in is a time-varying variable, is the constant deviation.

[0017] Preferably, in step 2, the embedding module is designed as a simple three-layer perceptron, and the structure of any layer of the three-layer perceptron is:

[0018] (2);

[0019] In formula (2), Indicates j Input variables, α represents the output of the multi-layer perceptron, b Indicates bias, φ () is the activation function, As a parameter, in the algorithm, a three-layer perceptron is used to project the low-dimensional inertial sensor data into a high-dimensional space, so that the data distribution of different inertial sensors in different usage environments is similar.

[0020] Preferably, in step 3, an embedding module is designed that can project data with large distribution differences into a new similarity space, as follows:

[0021] The structure of the generation module is an expanded neural network structure:

[0022] (3);

[0023] in , is the acceleration information measured by the inertial sensor, Indicates the local window length of the previous measurement, which is the basis for correcting the current state. Defined by the dilated neural network, it can smoothly correct the data and extract multi-level time information. The dilated neural network samples the time series data at intervals in each layer. In the generator, we use a five-layer dilated neural network, and the dilation intervals of each layer are: 1, 4, 16, 64, 1, and the kernel dimension of each layer is 7;

[0024] The structure of the denoising loss is:

[0025] (4);

[0026] in yes The logarithmic mapping of is the Huber loss, where Defined as:

[0027] (5);

[0028] The parameters Used to reduce the frequency of inertial sensors, and We start with the identity matrix.

[0029] Preferably, in step 4, a reconstruction module and a reconstruction loss are designed to obtain better data representation capability;

[0030] The structure of the reconstruction module is an expanded neural network structure:

[0031] (6);

[0032] in, After adding noise ,

[0033] The structure of the reconstruction loss is:

[0034] (7);

[0035] This structure is the mean square error, which is the difference between the estimated value and the true value at each moment.

[0036] Preferably, in step 5, a small sample domain adaptive strategy for inertial sensor denoising is designed, as follows:

[0037] Using structure with two parameter functions and To characterize the overall structure , Represents the reconstruction process, To represent the angular velocity error generation process, we use represents the parameters of the embedding module, reconstruction module and generation module, so There are two parameters , There are two parameters , the structure is based on the noisy angular velocity sequence and acceleration sequence As input, there are two training methods in the framework: the embedding module is trained by meta-learning, and the generation module and reconstruction module are trained by gradient descent;

[0038] The training of the embedding module is: During the training process, there is an angular velocity sequence of length n , an angular acceleration sequence of length n and the real angular velocity sequence of length n , the goal is to train through small sample meta-learning , trained by gradient descent , meta-learning consists of two parts: an adaptation optimization phase and a meta-optimization phase, which use different inertial sensor datasets from the same distribution ,

[0039] In the optimization phase, when it is necessary to adapt to data from a different source, embed module parameters Updated to ,parameter Update with the following expression:

[0040] (8);

[0041] This is the gradient descent update method. It's on mission The updated module parameters in It is a super parameter. is the loss function, which includes two losses, one is feature extraction loss and the other is denoising loss.

[0042] ; (9)

[0043] in, is equation (7), is formula (4), Indicates super parameter;

[0044] In the meta-optimization phase, the model is trained using the remaining data pairs, and the model parameters are updated via gradient descent:

[0045] ; (10)

[0046] in, These are the parameters that have been updated during the adaptation and optimization phase. It is a super parameter. For sequence The loss function is Through meta-learning training, Training is done by gradient descent.

[0047] After several training cycle updates, the final updated model parameters are the meta-learning training model parameters;

[0048] The meta-learning strategy is as follows:

[0049] Input: Contains a collection of inertial sensor sequences from different inertial sensors and different usage environments ,Include , are randomly initialized parameters, is a super parameter;

[0050] Output: Embedding module parameters obtained through meta-learning , gradient descent learning ;

[0051] While loop;

[0052] From the inertial sensor sequence collection Medium sampling sequence block , the sampling sequence Divided into ,right Each data loop in:

[0053] ;

[0054] use Update the parameters:

[0055] ;

[0056] While loop ends:

[0057] [ ]= .

[0058] By adopting the above scheme, the present invention achieves the following beneficial effects:

[0059] 1. This scheme proposes a new Embedding-Generator denoising structure. The embedding module Embedding is used to re-characterize the output data of the IMU. The IMU data is reconstructed through the designed reconstruction module, which improves the characterization effect of Embedding and filters out the noise. Then, the generator module is used to regenerate noise-free IMU data from the representation of the Embedding output.

[0060] 2. In order to improve the generalization ability of the overall denoising network, the generalization ability of the Embedding representation processing must be improved. Therefore, the meta-learning algorithm is introduced to take advantage of the double-loop training to improve the generalization ability of the Embedding module in different environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is the algorithm framework in the embodiment of the present invention;

[0062] Figure 2 It is the meta-learning strategy in the embodiment of the present invention;

[0063] Figure 3 Direction estimation of different methods in the test sequence room4 in the embodiments of the present invention;

[0064] Figure 4 is the directional error of different methods in the test sequence room4 in the embodiment of the present invention;

[0065] Figure 5 Direction estimation of the test sequence V101easy by different methods in the embodiments of the present invention;

[0066] Figure 6 is the directional error of the test sequence V101easy by different methods in the embodiments of the present invention;

[0067] Figure 7 Two robots (the car in the upper left corner and the quadruped robot in the upper right corner), the trajectory (lower left corner) and the three-axis angular velocity of the gyroscope according to an embodiment of the present invention;

[0068] Figure 8 : The embodiments of the present invention are experimented on a real robot. DETAILED DESCRIPTION

[0069] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0070] The following will be combined with the drawings in the embodiments of the invention to clearly and completely describe the technical solutions in the embodiments of the invention. Obviously, the described embodiments are only part of the embodiments of the invention, not all of the embodiments. Based on the embodiments of the invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the invention.

[0071] like Figure 1-8 As shown, a domain adaptive inertial sensor denoising algorithm based on meta-learning is characterized by comprising the following steps:

[0072] Step 1: Establish a noise-free angular velocity model of the inertial sensor;

[0073] Step 2: Design an embedding module to project data with large distribution differences into a new similarity space;

[0074] Step 3: Design a generation module and denoising loss for denoising;

[0075] Step 4: Design a reconstruction module and reconstruction loss for better data representation capabilities;

[0076] Step 5: Design a small sample domain adaptation strategy for inertial sensor denoising.

[0077] The noise-free angular velocity model of the inertial sensor is:

[0078] (1);

[0079] In formula (1) For parameters, parameters , is the angular velocity measured by the inertial sensor, is the gyro bias, gyro bias ,in is a time-varying variable, is the constant deviation.

[0080] In step 2, the embedding module is designed as a simple three-layer perceptron. The structure of any layer of the three-layer perceptron is:

[0081] (2);

[0082] In formula (2), Indicates j Input variables, α represents the output of the multi-layer perceptron, b Indicates bias, φ () is the activation function, As a parameter, in the algorithm, a three-layer perceptron is used to project the low-dimensional inertial sensor data into a high-dimensional space, so that the data distribution of different inertial sensors in different usage environments is similar.

[0083] In step 3, we design an embedding module that can project data with large distribution differences into a new similarity space as follows:

[0084] The structure of the generation module is an expanded neural network structure:

[0085] (3);

[0086] in , is the acceleration information measured by the inertial sensor, Indicates the local window length of the previous measurement, which is the basis for correcting the current state. Defined by the dilated neural network, it can smoothly correct the data and extract multi-level time information. The dilated neural network samples the time series data at intervals in each layer. In the generator, we use a five-layer dilated neural network, and the dilation intervals of each layer are: 1, 4, 16, 64, 1, and the kernel dimension of each layer is 7;

[0087] The structure of the denoising loss is:

[0088] (4);

[0089] in yes The logarithmic mapping of is the Huber loss, where Defined as:

[0090] (5);

[0091] The parameters Used to reduce the frequency of inertial sensors, and We start with the identity matrix.

[0092] In step 4, a reconstruction module and reconstruction loss are designed to obtain better data representation capability;

[0093] The structure of the reconstruction module is an expanded neural network structure:

[0094] (6);

[0095] in, After adding noise ,

[0096] The structure of the reconstruction loss is:

[0097] (7);

[0098] This structure is the mean square error, which is the difference between the estimated value and the true value at each moment.

[0099] In step 5, a small sample domain adaptation strategy for inertial sensor denoising is designed as follows:

[0100] Using structure with two parameter functions and To characterize the overall structure , Represents the reconstruction process, To represent the angular velocity error generation process, we use represents the parameters of the embedding module, reconstruction module and generation module, so There are two parameters , There are two parameters , the structure is based on the noisy angular velocity sequence and acceleration sequence As input, there are two training methods in the framework: the embedding module is trained by meta-learning, and the generation module and reconstruction module are trained by gradient descent;

[0101] The training of the embedding module is: During the training process, there is an angular velocity sequence of length n , an angular acceleration sequence of length n and the real angular velocity sequence of length n , the goal is to train through small sample meta-learning , trained by gradient descent , meta-learning consists of two parts: an adaptation optimization phase and a meta-optimization phase, which use different inertial sensor datasets from the same distribution ,

[0102] In the optimization phase, when it is necessary to adapt to data from a different source, embed module parameters Updated to ,parameter Update with the following expression:

[0103] (8);

[0104] This is the gradient descent update method. It's on mission The updated module parameters in It is a super parameter. is the loss function, which includes two losses, one is feature extraction loss and the other is denoising loss.

[0105] ; (9)

[0106] in, is equation (7), is formula (4), Indicates super parameter;

[0107] In the meta-optimization phase, the model is trained using the remaining data pairs, and the model parameters are updated via gradient descent:

[0108] ; (10)

[0109] in, These are the parameters that have been updated during the adaptation and optimization phase. It is a super parameter. For sequence The loss function is Through meta-learning training, Training is done by gradient descent.

[0110] After several training cycle updates, the final updated model parameters are the meta-learning training model parameters;

[0111] The meta-learning strategy is as follows:

[0112] Input: Contains a collection of inertial sensor sequences from different inertial sensors and different usage environments ,Include , are randomly initialized parameters, is a super parameter;

[0113] Output: Embedding module parameters obtained through meta-learning , gradient descent learning ;

[0114] While loop;

[0115] From the inertial sensor sequence collection Medium sampling sequence block , the sampling sequence Divided into ,right Each data loop in:

[0116] ;

[0117] use Update the parameters:

[0118] ;

[0119] While loop ends:

[0120] [ ]= .

[0121] Example 1

[0122] Experimental setup:

[0123] To demonstrate the advancedness of the algorithm, we test it on the public datasets EuRoC and TUM-VI.

[0124] EuRoC: This dataset comes from a small aircraft equipped with an uncalibrated inertial sensor ADIS16448. The dataset has seven sequences, namely MH01easy, MH02easy, MH03medium, MH04difficult, MH05difficult, V101easy, V102medium, V103difficult, V201easy, V202mediumandV203difficult. The seven sequences represent the inertial sensor data collected in different areas, at different times and under different tasks.

[0125] TUM-VI: This dataset comes from a handheld instrument equipped with an inertial sensor BM1160. It contains inertial sensor data collected at different locations and in different motion modes. The comparison method is as follows:

[0126] DIGDL: This method uses deep learning to remove noise. This method is one of the best methods for inertial sensors at present;

[0127] FSDA-F: This method only uses the algorithm framework we proposed. It does not contain a meta-learning strategy;

[0128] FSDA: This method is the algorithm we proposed;

[0129] Evaluation method: We use relative direction error to evaluate:

[0130]

[0131] Results: In the open dataset experiment, we set MH01easy, MH02easy, MH03medium, MH05hard, V102medium, V201easy, V203hard (from EURoC) and room1, room3, room5 from TUM-VI as training sets and set MH04hard, V101easy, V103hard, V202medium, room2, room4 and room6 as test sets. In each training set, the data is divided into learning and validation parts. The learning part is used to optimize parameters and the validation part is used to evaluate the deviation. From Table I, we first consider the denoising performance at the end of the sequence. The average performance of our proposed method on EuRoC and TUM-VI is 37.7% and 18.6% better than DIGDL, respectively. However, this result cannot evaluate the whole process because the IMU sequence has a lot of jitter, so we also need to see the denoising performance of the entire IMU sequence. Figure 3 , 4 , 5, and 6 represent the overall processing results of room4 and v101easy. Obviously, FSDA and FSDA-F outperform DIGDL in all directions. The comparison between FSDA and FSDA-F shows that the meta-learning strategy is indeed effective.

[0132] Tested on a real robot:

[0133] IMU settings: We selected three IMUs with different precisions. From high precision to low precision, they are CERTUS-EVO from ADVANCED NAVIGATION, MTI600 series from XSENS, and MPU-9250 from TDK InvenSense:

[0134] The CERTUS-EVO features a gyroscope with 0.2° / hr bias instability, 6° / hr / √Hz noise density, and <0.03% nonlinearity, and an accelerometer with 8μg bias instability, 2μg / √Hz noise density, and <0.05% nonlinearity.

[0135] The MTI600 series features gyroscopes with 8° / hr bias instability, 0.007° / s / √Hz noise density, and 0.1%FS nonlinearity, and accelerometers with 15μg bias instability, 60μg / √Hz noise density, and 0.1%FS nonlinearity.

[0136] The MPU-9250 features a gyroscope with 0.01° / s / √Hz noise density and ±0.1% nonlinearity, and an accelerometer with 300µg / √Hz noise density and ±0.5% nonlinearity. With a dual antenna system, CERTUS-EVO can also provide high-precision orientation, which can be viewed as ground truth.

[0137] Robotic Platforms: The two robotic platforms are a car and a quadruped robot, set up as follows Figure 7 As shown in the figure. In the implementation, the car robot travels six times, each time for one circle. The quadruped robot travels twice, each time for a 3 / 4 circle. Due to page limitations, we choose the yaw angle that is most likely to accumulate errors to show the performance of our method. The performance is shown in the figure. Figure 8 As shown in Figure 2, the proposed algorithm works well on the real robot and IMU. Obviously, the divergence of DICDL is serious, which means that DICDL is not adaptable to the order of different fields. Our FSDA framework greatly reduces the error, and the performance is further improved by adopting the meta-learning strategy.

[0138] Table 1:

[0139] Relative orientation error (ROE) in terms of orientation (roll, pitch and yaw) of the test sequence

[0140]

[0141] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A domain-adaptive inertial sensor denoising algorithm based on meta-learning, characterized in that The following steps are involved: Step 1: Establish a noise-free angular velocity model of the inertial sensor; Step 2: Design an embedding module to project data with large distribution differences into a new similarity space; Step 3: Design a generation module and denoising loss for denoising; Step 4: Design a reconstruction module and reconstruction loss for better data representation capabilities; Step 5: Design a small sample domain adaptation strategy for inertial sensor denoising. The noise-free angular velocity model of the inertial sensor is: （1）； In formula (1) For parameters, parameters , is the angular velocity measured by the inertial sensor, is the gyro bias, gyro bias ,in is a time-varying variable, is the constant deviation, In step 2, the embedding module is designed as a simple three-layer perceptron. The structure of any layer of the three-layer perceptron is: （2）； In formula (2), Indicates j Input variables, α represents the output of the multi-layer perceptron, b Indicates bias, φ () is the activation function, As a parameter, in the algorithm, a three-layer perceptron is used to project the low-dimensional inertial sensor data into a high-dimensional space, so that the data distribution of different inertial sensors in different usage environments is similar. In step 3, we design an embedding module that can project data with large distribution differences into a new similarity space as follows: The structure of the generation module is an expanded neural network structure: (3)； in , is the acceleration information measured by the inertial sensor, Indicates the local window length of the previous measurement, which is the basis for correcting the current state. Defined by the dilated neural network, it can smoothly correct the data and extract multi-level time information. The dilated neural network samples the time series data at intervals in each layer. In the generator, we use a five-layer dilated neural network, and the dilation intervals of each layer are: 1, 4, 16, 64, 1, and the kernel dimension of each layer is 7; The structure of the denoising loss is: （4）； in yes The logarithmic mapping of is the Huber loss, where Defined as: （5）； The parameters Used to reduce the frequency of inertial sensors, and Initially, it is the identity matrix. In step 4, a reconstruction module and reconstruction loss are designed to obtain better data representation capability; The structure of the reconstruction module is an expanded neural network structure: (6)； in, After adding noise , The structure of the reconstruction loss is: （7）； This structure is the mean square error, which takes the difference between the estimated value and the true value at each moment. In step 5, a small sample domain adaptation strategy for inertial sensor denoising is designed as follows: Using structure with two parameter functions and To characterize the overall structure , Represents the reconstruction process, To represent the angular velocity error generation process, we use represents the parameters of the embedding module, reconstruction module and generation module, so There are two parameters , There are two parameters , the structure is based on the noisy angular velocity sequence and acceleration sequence As input, there are two training methods in the framework: the embedding module is trained by meta-learning, and the generation module and reconstruction module are trained by gradient descent; The training of the embedding module is: During the training process, there is an angular velocity sequence of length n , an angular acceleration sequence of length n and the real angular velocity sequence of length n , the goal is to train through small sample meta-learning , trained by gradient descent , meta-learning consists of two parts: an adaptation optimization phase and a meta-optimization phase, which use different inertial sensor datasets from the same distribution , In the adaptation optimization phase, when it is necessary to adapt to data from a different source, the module parameters are embedded Updated to ,parameter Update with the following expression: （8）； This is the gradient descent update method, where It's on mission The updated module parameters in It is a super parameter. is the loss function, which includes two losses, one is feature extraction loss and the other is denoising loss. ；（9） in, is equation (7), is formula (4), Indicates super parameter; In the meta-optimization phase, the model is trained using the remaining data pairs, and the model parameters are updated via gradient descent: ；（10） in, These are the parameters that have been updated during the adaptation and optimization phase. It is a super parameter. For sequence The loss function is Through meta-learning training, Training is done by gradient descent. After several training cycle updates, the final updated model parameters are the meta-learning training model parameters; The meta-learning strategy is as follows: Input: Contains a collection of inertial sensor sequences from different inertial sensors and different usage environments ,Include , are randomly initialized parameters, is a super parameter; Output: Embedding module parameters obtained through meta-learning , gradient descent learning ; While loop; From the inertial sensor sequence collection Medium sampling sequence block , the sampling sequence Divided into ,right Each data loop in: ； use Update the parameters: ； While loop ends: [ ]= 。

Images