Inertial sensing data denoising method based on self-supervised adversarial encoder-decoder

Abstract

The application provides an inertial sensing data denoising method based on a self-supervised adversarial encoder-decoder, comprising collecting inertial data of a human body, a vehicle, a robot and a UAV under various complex motions by a high-precision inertial measurement unit to establish a high-precision inertial data set; inputting original high-precision inertial data collected in the high-precision inertial data set into an encoder, continuously training the encoder-decoder by minimizing reconstruction loss until the input high-precision inertial data is reconstructed from the decoder end, and obtaining a latent representation of the high-precision inertial data from an intermediate network layer of the trained encoder-decoder; collecting inertial data of the human body, the vehicle, the robot and the UAV under various complex motions by a low-precision inertial measurement unit to establish a low-precision inertial data set; and performing adversarial training through a generator and a discriminator to finally generate inertial data more approximating to the latent representation of the high-precision inertial data. The application improves the sensor data quality and the ability to handle nonlinear problems.

Description

Technical Field

[0001] This invention relates to the field of inertial sensor signal processing technology, specifically to an inertial sensor data noise reduction method based on a self-supervised adversarial codec. Background Technology

[0002] Inertial sensors (including accelerometers and gyroscopes) are core components of modern motion tracking and attitude estimation systems, widely used in smartphones, drones, robots, and vehicle navigation. However, inertial sensor data is often affected by noise, including measurement noise, zero-point drift, and temperature drift. This noise can severely impact the measurement accuracy and stability of inertial sensors, thereby affecting the effectiveness of applications based on inertial sensor data.

[0003] Traditional inertial sensor data denoising methods mainly include various digital signal processing techniques such as low-pass filters and Kalman filters. While these methods can reduce noise to some extent, they also have some significant limitations. For example, low-pass filters may weaken the effective components of the signal, and Kalman filters require accurate noise models, which are often difficult to obtain in practical applications. Furthermore, traditional methods perform poorly when dealing with complex noise characteristics, especially when the noise is nonlinear and non-Gaussian.

[0004] In existing technologies, such as the patent application number 202010339248.6 entitled "A Time Delay Noise Reduction Method for Vehicle-Mounted MEMS Inertial Sensors," a time delay noise reduction method suitable for vehicle-mounted MEMS inertial sensors is proposed. This method utilizes the tapped delay method to establish a time delay multiple linear regression mathematical model (TD-MLR) and trains the model in a supervised manner. However, this method has several limitations. First, its applicability is relatively limited, only applicable to vehicle-mounted MEMS inertial sensors. Second, compared to deep neural network models, multiple linear regression mathematical models are less capable of handling nonlinear problems. Third, the supervised training method requires extensive pre-training data labeling.

[0005] For example, patent application number 202211600874.1, entitled "Signal Enhancement and Gesture Recognition Method and System for Low-Cost Inertial Sensors," utilizes the deterministic index of recursive graphs to evaluate the randomness of multi-resolution modal components obtained after empirical mode decomposition. Simultaneously, it employs wavelet threshold denoising to effectively control random errors in the modal components, thereby achieving signal denoising for low-cost inertial data. However, this method relies on wavelet basis selection, which is inflexible. Furthermore, it is limited to a fixed resolution; although wavelet transform has multi-resolution analysis capabilities, its resolution at each scale is fixed, which limits its ability to handle complex nonlinear and non-stationary signals.

[0006] For example, patent application number 202310384223.1, entitled "A Data Denoising Method and System for Inertial Measurement Units," uses a supervised deep learning model constructed with dilated convolutional neural networks and temporal convolutional neural networks for denoising. First, this method requires extensive manual annotation of the original data. Second, the design of the dilated convolutional neural network suffers from significant drawbacks. As the dilation rate increases, more and more "holes" appear between the convolutional kernels, leading to redundant computations in the convolution operation. This makes the network less accurate in capturing local features, and a large dilation rate can cause boundary effects, resulting in poor feature extraction at boundaries.

[0007] While the existing technologies mentioned above can achieve noise reduction in certain specific applications, they generally suffer from drawbacks such as narrow applicability, insufficient ability to handle complex nonlinear problems, and high dependence on data annotation. Summary of the Invention

[0008] In view of this, the present invention provides a method for denoising inertial sensing data based on a self-supervised adversarial codec, so as to at least solve the above-mentioned problems.

[0009] This invention provides an inertial sensing data denoising method based on a self-supervised adversarial codec, comprising: collecting inertial data of humans, vehicles, robots, and drones under various complex motions using a high-precision inertial measurement unit to establish a high-precision inertial dataset; inputting the original high-precision inertial data collected in the high-precision inertial dataset into an encoder, continuously training the encoder / decoder by minimizing the reconstruction loss until the input high-precision inertial data is reconstructed from the decoder, and obtaining the latent representation of the high-precision inertial data from the intermediate network layer of the trained encoder / decoder; collecting inertial data of humans, vehicles, robots, and drones under various complex motions using a low-precision inertial measurement unit to establish a low-precision inertial dataset; and performing adversarial training through a generator and a discriminator, specifically including the generator minimizing the reconstruction loss of the encoder / decoder, while the discriminator performs adversarial training by maximizing the adversarial loss between the latent representation of the high-precision inertial data and the latent representation of the low-precision inertial data generated by the generator, ultimately generating inertial data that more closely approximates the latent representation of the high-precision inertial data.

[0010] In one implementation, the formula for the reconstruction loss is as follows:

[0011]

[0012] Where x represents the high-precision inertial data reconstructed by the decoder. This represents the raw, high-precision inertial data input to the encoder.

[0013] In another implementation, the formula for the adversarial loss is as follows:

[0014]

[0015] Where D(z) represents the output of the discriminator for the input latent space sample z. This represents the expected value sampled from the true data distribution p(z). This represents the expected value sampled from the data distribution q(z) generated by the generator.

[0016] In another implementation, the generator minimizes the reconstruction loss of the codec, while the discriminator maximizes the adversarial loss between the latent representation of the high-precision inertial data and the latent representation of the low-precision inertial data generated by the generator; this should be understood as minimizing the loss function L. G L G The formula is as follows:

[0017] L G =L recon +λL adv

[0018] Where λ represents L recon and L adv The balance factor.

[0019] In another implementation, the high-precision inertial measurement unit is an Xsens MTi-610IMU, and the low-precision inertial measurement unit is an ST LSM6DS1.

[0020] Compared with the prior art, the beneficial effects of the present invention are:

[0021] (1) The model of this invention introduces an adversarial mechanism between the generator and the discriminator, which can fully take into account the influence of nonlinear factors such as sensor error, noise, motion mode and environment.

[0022] (2) This invention enables the rapid and efficient reconstruction of low-quality inertial data into high-quality data, providing the possibility of achieving high-precision positioning using low-cost sensors.

[0023] (3) This invention uses unsupervised learning, which does not require data labeling. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the sensor installation location according to an embodiment of the present invention.

[0025] Figure 2 This is a structural diagram of the encoder used in an embodiment of the present invention.

[0026] Figure 3This is a flowchart illustrating the overall process of the inertial sensing data denoising method based on a self-supervised adversarial codec according to an embodiment of the present invention. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] In recent years, with the rapid development of deep learning technology, data processing methods based on deep neural networks have shown great potential in signal processing and noise reduction. Adversarial Encoder-Decoder (AED) is a model that combines the advantages of encoders and decoders and generative adversarial networks (GANs). Encoders learn low-dimensional representations of input data by compressing and reconstructing it, while GANs generate more realistic data distributions through adversarial training between the discriminator and generator. This invention combines these two approaches to create an adversarial encoder that not only effectively captures the complex distribution characteristics of data but also enhances the robustness of the model, thus performing exceptionally well in noise reduction tasks. The inertial sensing data noise reduction method based on adversarial encoders leverages the powerful learning capabilities and adversarial training mechanism of adversarial encoders to better separate signals from noise, thereby significantly improving the quality of inertial sensing data. This method is not only applicable to traditional inertial navigation systems but also to inertial data processing in more complex dynamic environments, providing a new technical means to improve the accuracy of motion tracking and attitude estimation.

[0029] Specifically, see Figure 1 In this embodiment of the invention, the high-precision inertial measurement unit 1 (model Xsens MTi-610 IMU, costing tens of thousands of yuan) and the low-precision inertial measurement unit 2 (model ST LSM6DS1, costing about ten yuan) used for collecting all datasets are both installed at the toes of the human body. The sampling frequency of the device is 100Hz, which is used to measure the inertial data of the human foot.

[0030] See Figure 2 The codec used in this embodiment of the invention includes an encoder 21, an intermediate network layer 22, and a decoder 23. The generator used in this embodiment of the invention is... Figure 2 In section 24, the discriminator is... Figure 2 25.

[0031] See Figure 3The above is a flowchart of the overall process of the inertial sensing data denoising method based on a self-supervised adversarial codec according to an embodiment of the present invention. The steps of the embodiment of the present invention include:

[0032] Step 31: High-precision inertial data of human feet were collected using Xsens MTi-610 IMU for six actions: slow walking, normal walking, running, climbing stairs, descending stairs, and riding a horizontal elevator. Data for each action was collected from 6 volunteers for 30 minutes each, and a high-precision inertial dataset was constructed.

[0033] Step 32: Input the original high-precision inertial data collected in step 31 into the encoder, and continuously train the encoder and decoder by minimizing the reconstruction loss until the input high-precision inertial data is reconstructed from the decoder. The potential representation p(z) of the high-precision inertial data is obtained from the intermediate network layer of the trained encoder and decoder.

[0034] The formula for the reconstruction loss is as follows:

[0035]

[0036] Where x represents the high-precision inertial data reconstructed by the decoder. This represents the raw, high-precision inertial data input to the encoder.

[0037] Step 33: Collect low-precision inertial data of human feet under six actions: slow walking, normal walking, running, climbing stairs, descending stairs, and riding a horizontal elevator using ST LSM6DS1. Collect data for 30 minutes from 6 volunteers for each action to build a low-precision inertial dataset.

[0038] Step 34, the generator minimizes the reconstruction loss L of the codec. recon Simultaneously, the discriminator maximizes the adversarial loss L between the latent representation p(z) of the high-precision inertial data and the latent representation q(z) of the low-precision inertial data generated by the generator in step 32. adv Adversarial training is then conducted to ultimately generate inertial data that more closely approximates the latent representation p(z) of the high-precision inertial data in step 32.

[0039] The formula for the reconstruction loss is as follows:

[0040]

[0041] Where x represents the high-precision inertial data reconstructed by the decoder. This represents the raw, high-precision inertial data input to the encoder.

[0042] The formula for the adversarial loss is as follows:

[0043]

[0044] Where D(z) represents the output of the discriminator for the input latent space sample z. This represents the expected value sampled from the true data distribution p(z). This represents the expected value sampled from the data distribution q(z) generated by the generator.

[0045] In step 34, the generator minimizes the reconstruction loss L of the codec. recon Simultaneously, the discriminator maximizes the adversarial loss L between the latent representation p(z) of the high-precision inertial data and the latent representation q(z) of the low-precision inertial data generated by the generator in step 32. adv This should be understood as minimizing the loss function L. G L G The formula is as follows:

[0046] L G =L recon +λL adv

[0047] Where λ represents L recon and L adv The balance factor.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] (1) The model of this invention introduces an adversarial mechanism between the generator and the discriminator, which can fully take into account the influence of nonlinear factors such as sensor error, noise, motion mode and environment.

[0050] (2) This invention enables the rapid and efficient reconstruction of low-quality inertial data into high-quality data, providing the possibility of achieving high-precision positioning using low-cost sensors.

[0051] (3) This invention uses unsupervised learning, which does not require data labeling.

[0052] In summary, this invention employs an adversarial codec structure to reconstruct high-quality IMU data from low-quality IMU data, thereby improving the quality of sensor data from the source, reducing the impact of noise on inertial sensor data, and minimizing the influence of error sources on subsequent algorithms. It has the advantages of high applicability, strong ability to handle nonlinear problems, and no need for data labeling.

[0053] It should be noted that the present invention can be a method, system, apparatus, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the present invention.

[0054] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0055] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0056] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0057] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0058] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0059] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0060] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0061] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A method for denoising inertial sensing data based on a self-supervised adversarial codec, characterized in that, include: High-precision inertial measurement units are used to collect inertial data of human bodies, vehicles, robots, and drones under various complex motions to establish a high-precision inertial dataset. The original high-precision inertial data collected from the high-precision inertial dataset is input into the encoder. The encoder and decoder are continuously trained by minimizing the reconstruction loss until the input high-precision inertial data is reconstructed from the decoder. The potential representation of the high-precision inertial data is obtained from the intermediate network layer of the trained encoder and decoder. Low-precision inertial measurement units are used to collect inertial data of humans, vehicles, robots, and drones under various complex motions to establish a low-precision inertial dataset. Adversarial training is performed using a generator and a discriminator. Specifically, the generator minimizes the reconstruction loss of the encoder and decoder, while the discriminator performs adversarial training by maximizing the adversarial loss between the latent representation of the high-precision inertial data and the latent representation of the low-precision inertial data generated by the generator. Ultimately, this generates inertial data that more closely approximates the latent representation of the high-precision inertial data.

2. The method according to claim 1, characterized in that, The formula for the reconstruction loss is as follows: Where x represents the high-precision inertial data reconstructed by the decoder. This represents the raw, high-precision inertial data input to the encoder.

3. The method according to claim 2, characterized in that, The formula for the adversarial loss is as follows: Where D(z) represents the output of the discriminator for the input latent space sample z. This represents the expected value sampled from the true data distribution p(z). This represents the expected value sampled from the data distribution q(z) generated by the generator.

4. The method according to claim 3, characterized in that, The generator minimizes the reconstruction loss of the codec, while the discriminator maximizes the adversarial loss between the latent representation of the high-precision inertial data and the latent representation of the low-precision inertial data generated by the generator. This should be understood as minimizing the loss function L. G L G The formula is as follows: THE G =L recon +λL adv Where λ represents L recon and L adv The balance factor.

5. The method according to any one of claims 1-4, characterized in that, The high-precision inertial measurement unit is model Xsens MTi-610 IMU, and the low-precision inertial measurement unit is model ST LSM6DS1.

Images