Inertial sensor signal enhancement method and system based on zoom generative adversarial network

Abstract

The application discloses an inertial sensor signal enhancement method and system based on a zoom generative adversarial network, and relates to the field of inertial sensor signal enhancement. The method comprises the following steps: constructing an unsupervised zoom generative adversarial network; constructing a weakly supervised zoom generative adversarial network; the weakly supervised zoom generative adversarial network comprises an encoder and a decoder; when there is no similar semantic information between low-cost sensor signals and high-cost sensor signals, a trained high-cost signal generator in the unsupervised zoom generative adversarial network is used to enhance the low-cost sensor signals; when there is similar semantic information between low-cost sensor signals and high-cost sensor signals, a trained weakly supervised zoom generative adversarial network is used to enhance the low-cost sensor signals. The application can significantly enhance the quality of low-cost sensor signals.

Description

Technical Field

[0001] This invention relates to the field of inertial sensor signal enhancement technology, and in particular to an inertial sensor signal enhancement method and system based on zoom generative adversarial networks. Background Technology

[0002] In the future information-driven and intelligent society, ubiquitous sensor systems will become the cornerstone of efficient social operation. Intelligent robots, smart homes, and autonomous driving systems will all rely on networks composed of various sensors. Therefore, to some extent, the accuracy of sensors will determine the level of intelligence in future human society. As one of the most representative sensors, MEMS inertial sensors are widely used in aerospace, autonomous driving, human-computer interaction, navigation, and positioning fields due to their advantages such as small size, easy wearability, low power consumption, low cost, and ease of mass production. In fact, the unit price of common inertial sensors on the market can be as low as 0.2-0.5 yuan, thus possessing broad market prospects. However, while low-cost sensors can be mass-produced and are inexpensive, their signals often contain complex noise patterns and error drift. Summary of the Invention

[0003] Therefore, the purpose of this invention is to provide an inertial sensor signal enhancement method and system based on zoom generative adversarial networks, so as to achieve a significant enhancement of sensor signal quality at low cost.

[0004] To achieve the above objectives, the present invention provides the following solution:

[0005] A method for enhancing inertial sensor signals based on zoom-based generative adversarial networks, comprising:

[0006] An unsupervised zoom generative adversarial network is constructed; the unsupervised zoom generative adversarial network includes: a high-cost signal generator, a low-cost signal generator, and a discriminator; the high-cost signal generator is equipped with a zoom mechanism;

[0007] A weakly supervised zoom generative adversarial network is constructed; the weakly supervised zoom generative adversarial network includes an encoder and a decoder; multiple auxiliary tasks are set in the weakly supervised zoom generative adversarial network to constrain the semantic information consistency between low-cost sensor signals and high-cost sensor signals;

[0008] When there is no similar semantic information between low-cost sensor signals and high-cost sensor signals, the trained high-cost signal generator in the unsupervised zoom generative adversarial network is used to enhance the low-cost sensor signals.

[0009] When low-cost sensor signals and high-cost sensor signals have similar semantic information, a well-trained weakly supervised zoom generative adversarial network is used to enhance the low-cost sensor signals.

[0010] Optionally, the training process of the high-cost signal generator is as follows:

[0011] Obtain real low-cost sensor signal samples and real high-cost sensor signal samples;

[0012] The real low-cost sensor signal sample is input into the high-cost signal generator to obtain the first virtual high-cost sensor signal;

[0013] The first virtual high-cost sensor signal is input to the low-cost signal generator to obtain the first virtual low-cost sensor signal;

[0014] The high-cost signal generator is initially optimized based on the error between the real low-cost sensor signal sample and the first virtual low-cost sensor signal.

[0015] The real high-cost sensor signal sample is input into the low-cost signal generator to obtain a second virtual low-cost sensor signal.

[0016] The second virtual low-cost sensor signal is input into the initially optimized high-cost signal generator to obtain the second virtual high-cost sensor signal.

[0017] The discriminator distinguishes between the first virtual high-cost sensor signal and the second virtual high-cost sensor signal.

[0018] The high-cost signal generator, which was initially optimized, is further optimized based on the discrimination results of the discriminator.

[0019] Optionally, the zoom mechanism specifically includes:

[0020] Extract real low-cost sensor signal samples of different scales from the original signal stream; the real low-cost sensor signal samples of different scales include real low-cost sensor signal samples of intrinsic scale and real low-cost sensor signal samples of larger than intrinsic scale;

[0021] The intrinsic-scale real low-cost sensor signal sample is input into the high-cost signal generator.

[0022] Real low-cost sensor signal samples larger than the intrinsic scale are downsampled to the intrinsic scale and then input into different high-cost signal generators;

[0023] Optimize the high-cost signal generator based on the outputs of the two high-cost signal generators.

[0024] Optionally, during the training process of the weakly supervised zoom generative adversarial network, when the learning direction of the auxiliary task is consistent with that of the main task, the loss of the auxiliary task is amplified; when the learning direction of the auxiliary task is inconsistent with that of the main task, the loss of the auxiliary task is reduced to suppress the negative impact of the auxiliary task on the main task. The consistency of the learning direction is determined by the consistency loss. The consistency loss is the loss between the original features of the main task and the copied features of the auxiliary task. The copied features are obtained by copying the original features.

[0025] Optionally, the auxiliary task loss includes: auxiliary task loss of shallow features at the input end, auxiliary task loss of mid-level features at the input end, auxiliary task loss of deep features at the input end, auxiliary task loss of shallow features at the output end, auxiliary task loss of shallow features at the output end, and auxiliary task loss of shallow features at the output end; the input end is the encoder; the output end is the decoder; the shallow features are features close to the data end; the deep features are features far from the data end; and the mid-level features are features between the shallow features and the deep features.

[0026] Optionally, the total loss L of the auxiliary task Auxiliary The calculation formula is as follows:

[0027]

[0028] in, The auxiliary task loss is used for shallow features at the input end; The auxiliary task loss is used for the mid-level features at the input end; The auxiliary task loss is used for deep features at the input end; This is an auxiliary task loss for shallow features at the output end; The auxiliary task loss is used for the mid-level features at the output end; This is an auxiliary task loss for deep features at the output end; Consistency loss for shallow features at the input end; The consistency loss is calculated for the mid-level features at the input end. Consistency loss for deep features at the input end; This represents the consistency loss of shallow features at the output end. This represents the consistency loss of the mid-level features at the output end. This represents the consistency loss of deep features at the output end.

[0029] The present invention also provides an inertial sensor signal enhancement system based on a zoom-based generative adversarial network, comprising:

[0030] An unsupervised zoom generative adversarial network construction module is used to construct an unsupervised zoom generative adversarial network; the unsupervised zoom generative adversarial network includes: a high-cost signal generator, a low-cost signal generator, and a discriminator; the high-cost signal generator is equipped with a zoom mechanism;

[0031] A weakly supervised zoom generative adversarial network construction module is used to construct a weakly supervised zoom generative adversarial network; the weakly supervised zoom generative adversarial network includes an encoder and a decoder; multiple auxiliary tasks are set in the weakly supervised zoom generative adversarial network to constrain the semantic information consistency between low-cost sensor signals and high-cost sensor signals;

[0032] The first signal enhancement module is used to enhance the low-cost sensor signal by employing the trained high-cost signal generator in the unsupervised zoom generative adversarial network when there is no similar semantic information between the low-cost sensor signal and the high-cost sensor signal.

[0033] The second signal enhancement module is used to enhance the low-cost sensor signal by employing a trained weakly supervised zoom generative adversarial network when the low-cost sensor signal and the high-cost sensor signal have similar semantic information.

[0034] Optionally, it further includes: a high-cost signal generator training module, wherein the high-cost signal generator training module specifically includes:

[0035] The signal sample set acquisition unit is used to acquire real low-cost sensor signal samples and real high-cost sensor signal samples.

[0036] The first input unit is used to input the real low-cost sensor signal sample into the high-cost signal generator to obtain the first virtual high-cost sensor signal;

[0037] The second input unit is used to input the first virtual high-cost sensor signal to the low-cost signal generator to obtain the first virtual low-cost sensor signal.

[0038] The first optimization unit is used to perform initial optimization on the high-cost signal generator based on the error between the real low-cost sensor signal sample and the first virtual low-cost sensor signal.

[0039] The third input unit is used to input the real high-cost sensor signal sample into the low-cost signal generator to obtain the second virtual low-cost sensor signal.

[0040] The fourth input unit is used to input the second virtual low-cost sensor signal into the initially optimized high-cost signal generator to obtain the second virtual high-cost sensor signal.

[0041] The discrimination unit is used to discriminate the first virtual high-cost sensor signal and the second virtual high-cost sensor signal through the discriminator;

[0042] The second optimization unit is used to further optimize the initially optimized high-cost signal generator based on the discrimination result of the discriminator.

[0043] Optionally, during the training process of the weakly supervised zoom generative adversarial network, when the learning direction of the auxiliary task is consistent with that of the main task, the loss of the auxiliary task is amplified; when the learning direction of the auxiliary task is inconsistent with that of the main task, the loss of the auxiliary task is reduced to suppress the negative impact of the auxiliary task on the main task. The consistency of the learning direction is determined by the consistency loss. The consistency loss is the loss between the original features of the main task and the copied features of the auxiliary task. The copied features are obtained by copying the original features.

[0044] Optionally, the auxiliary task loss includes: auxiliary task loss of shallow features at the input end, auxiliary task loss of mid-level features at the input end, auxiliary task loss of deep features at the input end, auxiliary task loss of shallow features at the output end, auxiliary task loss of shallow features at the output end, and auxiliary task loss of shallow features at the output end; the input end is the encoder; the output end is the decoder; the shallow features are features close to the data end; the deep features are features far from the data end; and the mid-level features are features between the shallow features and the deep features.

[0045] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0046] This invention constructs an unsupervised zoom generative adversarial network (U-ZoomGAN) and a weakly-supervised zoom generative adversarial network (W-ZoomGAN). When there is no correspondence between low-cost and high-cost sensor signals, U-ZoomGAN is used to enhance signal quality; when low-cost and high-cost sensor signals have some similar semantic information, W-ZoomGAN is used. In the U-ZoomGAN architecture, this invention sets up a "zoom" mechanism to improve the generator's ability to migrate between different resolutions. In the W-ZoomGAN architecture, this invention sets up multiple auxiliary tasks to constrain the consistency of semantic information between low-cost and high-cost sensor signals, thereby improving the stability of the signal quality enhancement process and thus improving the signal quality enhancement effect. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of an inertial sensor signal enhancement method based on a zoom-based generative adversarial network provided in Embodiment 1 of the present invention.

[0049] Figure 2 The structural diagram of the unsupervised zoom generative adversarial network provided by this invention;

[0050] Figure 3 The structure diagram of the weakly supervised zoom generative adversarial network provided by this invention is shown. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] The purpose of this invention is to provide an inertial sensor signal enhancement method and system based on zoom generative adversarial networks, which generates high-cost sensor signals from low-cost sensor signals by constructing U-ZoomGAN and W-ZoomGAN networks.

[0053] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0054] Example 1

[0055] Embodiment 1 of the present invention provides a method for enhancing inertial sensor signals based on zoom-based generative adversarial networks, such as... Figure 1 As shown, the method includes the following steps:

[0056] S1: Construct an unsupervised zoom generative adversarial network; the unsupervised zoom generative adversarial network includes a high-cost signal generator, a low-cost signal generator based on a discriminator; the high-cost signal generator is equipped with a zoom mechanism. The training process of the high-cost signal generator is as follows: acquire real low-cost sensor signal samples and real high-cost sensor signal samples; input the real low-cost sensor signal samples into the high-cost signal generator to obtain a first virtual high-cost sensor signal; input the first virtual high-cost sensor signal into the low-cost signal generator to obtain a first virtual low-cost sensor signal; perform initial optimization of the high-cost signal generator based on the error between the real low-cost sensor signal samples and the first virtual low-cost sensor signal; input the real high-cost sensor signal samples into the low-cost signal generator to obtain a second virtual low-cost sensor signal; input the second virtual low-cost sensor signal into the initially optimized high-cost signal generator to obtain a second virtual high-cost sensor signal; discriminate the first virtual high-cost sensor signal and the second virtual high-cost sensor signal using the discriminator; further optimize the initially optimized high-cost signal generator based on the discrimination result of the discriminator.

[0057] S2: Construct a weakly supervised zoom generative adversarial network; the weakly supervised zoom generative adversarial network includes an encoder and a decoder; the weakly supervised zoom generative adversarial network sets up multiple auxiliary tasks to constrain the semantic information consistency between low-cost sensor signals and high-cost sensor signals.

[0058] S3: When there is no similar semantic information between low-cost sensor signals and high-cost sensor signals, the high-cost signal generator in the trained unsupervised zoom generative adversarial network is used to enhance the low-cost sensor signals.

[0059] S4: When low-cost sensor signals and high-cost sensor signals have similar semantic information, a trained weakly supervised zoom generative adversarial network is used to enhance the low-cost sensor signals.

[0060] Furthermore, steps S1 and S3 specifically include:

[0061] The biggest challenge in low-cost sensor signal enhancement lies in the inability to obtain strictly paired high-cost and low-cost sensor signals, thus preventing the use of end-to-end fully supervised training methods to construct mapping relationships between signals from sensors of different costs. To address this, this invention uses CycleGAN as a foundation and adds a "zoom mechanism" to focus on signal features at different scales, ultimately achieving signal enhancement based on low-cost inertial sensors.

[0062] like Figure 1As shown, firstly, this invention inputs low-cost sensor signals into a high-cost signal generator (GAN). I2S This yields the corresponding virtual high-cost sensor signal. To ensure the virtual high-cost sensor signal is as realistic as possible, this invention sets up a classifier to determine the authenticity of the virtual high-cost sensor signal, and a generator GAN... I2S During training, the generator will strive to produce sufficiently realistic, high-cost sensor signals to deceive the discriminator. The discriminator, in turn, will work to improve its ability to distinguish between real and virtual signals, thus minimizing its chances of being fooled. This adversarial mechanism enables basic low-cost sensor signal enhancement.

[0063] However, even the most basic generative adversarial network cannot guarantee the consistency between the generated signal and the original signal. In this case, the generated virtual high-cost sensor signal loses a significant amount of information from the original low-cost sensor signal. Therefore, this invention proposes a generator (GAN) that transforms high-cost sensor signals into low-cost sensor signals. S2I GAN I2S The generated virtual high-cost sensor signal is input into the GAN S2I This process generates a corresponding virtual low-cost sensor signal, which is derived from a real low-cost sensor signal. Therefore, this invention can impose a constraint term "cyclic consistency loss." During this process, GAN... I2S It is essential to ensure that the generated virtual high-cost sensor signals contain a sufficient amount of low-cost sensor signal information; otherwise, the subsequent generator GAN will be ineffective. S2I It will be impossible to reconstruct the signal from a low-cost sensor.

[0064] The above signal enhancement process can be summarized as: real low-cost sensor signal → GAN I2S →Virtual high-cost sensor signals→GAN S2I → Virtual low-cost sensor signal, which this invention refers to as the "low-cost sensor signal loop". In this process, the two generators have never encountered real high-cost sensor signals, and their feature extractors have never recorded the characteristics of real high-cost sensor signals. Therefore, the virtual high-cost sensor signals they generate are difficult to deceive the discriminator and show a significant difference from real high-cost sensor signals. To address this, this invention establishes a "high-cost sensor signal loop," whose signal generation process can be summarized as: Real high-cost sensor signal → GAN S2I →Virtual low-cost sensor signal→GAN I2S → Virtual high-cost sensor signal. In this process, the two generators can access real high-cost sensor signals and extract and record features from them, thereby generating a more realistic high-cost sensor signal.

[0065] To enhance the generator's ability to focus on features at different resolutions, this invention employs a "zoom" mechanism to improve its migration capability across different resolutions. For ease of distinction, this invention refers to the scale of the original input data as the intrinsic scale, and uses L to represent the length of the input data at that scale. To acquire signals at different scales, this invention extracts signals of length 1.2L and 2.0L from the original signal stream and downsamples them to the intrinsic scale, resulting in signals with a larger scale and lower resolution. These signals undergo the same data generation process as the signals at the intrinsic scale. Since the signals at the intrinsic scale are part of the larger-scale signals, this invention uses the relationships between different scales as constraints in the multi-scale signal generation process. Ultimately, the input low-cost signal can migrate to high-cost signals across multiple scales, including fluctuation details, trends, and overall morphology.

[0066] Furthermore, steps S2 and S4 specifically include:

[0067] The advantage of unsupervised signal enhancement algorithms lies in their lack of requirements regarding motion during data acquisition and the elimination of the need for matching low-cost and high-cost signals, thus offering broad applicability. However, by appropriately recording certain indicators during motion acquisition and incorporating this prior information as weakly supervised labels into the generator's training process, the quality of the generated signal can be further improved. Therefore, four types of information will be recorded during data acquisition as labels for four auxiliary tasks: motion classification, motion segmentation, motion pose angle prediction, and motion trajectory morphology prediction. The model will simultaneously complete these four tasks during signal generation, and adjustments will be made based on the performance to achieve better signal generation results. The specific settings for the four auxiliary tasks are as follows.

[0068] Motion classification task: During data collection, in addition to completely random movements, movements containing specific semantic information, such as writing numbers and letters, will also be collected. Each action will be assigned a category label, and the model will predict the motion category corresponding to the input signal based on the extracted features during signal generation.

[0069] Attitude change prediction task: A robotic arm will collect partial motion data to obtain the spatial attitude angle changes of the sensors during motion. When using this signal as training data, the model will predict the spatial attitude angle changes corresponding to the input data based on the extracted features during signal generation.

[0070] Trajectory morphology prediction task: During motion data acquisition, optical motion capture equipment will be used to obtain precise motion trajectories, and these trajectories will be simulated using geometric models. During signal generation, the model will predict the geometric parameters corresponding to the input signal based on the extracted features.

[0071] Three auxiliary tasks force the generator to extract richer features from the input signal, thereby improving the quality of signal enhancement. Of course, other auxiliary tasks can be added, such as motion segmentation point prediction: In all the collected motion data, this invention will segment a portion of the motion data and manually label moments of drastic changes in motion direction as motion segmentation points. The model will then predict these segmentation points in the input signal based on the extracted features during signal generation.

[0072] Considering the differences between the auxiliary task and the main generation task, and the fact that their relied-upon features are not entirely consistent, the auxiliary task can interfere with the learning direction of the generation task. To address this, this invention designs a learning direction representation method, which can determine the importance of the auxiliary task in the entire model. When the learning direction of the auxiliary task is consistent with that of the generation task, the auxiliary task will be given a larger weight to better assist the generation task in extracting features from the input data. When there is a significant difference between the learning directions of the auxiliary task and the main task, the contribution of the auxiliary task in this round of training will be suppressed or even set to zero to prevent it from negatively impacting the feature extraction process of the generation task. The framework of the learning direction representation method is as follows: Figure 3 As shown:

[0073] In W-ZoomGAN, a generator is used to convert between low-cost and high-cost sensor signals. Features closer to the data source in the generator are generally called shallow features, features farther from the data source are called deep features, and features in between are referred to as mid-level features. During signal generation, this invention sets up auxiliary tasks for different feature layers, such as displacement prediction based on shallow features, pose calculation based on mid-level features, and semantic recognition based on deep features. The loss for the auxiliary task based on shallow features from the input (encoder) is... The auxiliary task loss based on mid-layer features from the input (encoder) is: The auxiliary task loss based on deep features from the input (encoder) is The auxiliary task loss based on shallow features from the output (decoder) is The auxiliary task loss based on the mid-layer features at the output end (decoder) is The auxiliary task loss based on deep features at the output end (decoder) is These auxiliary tasks force the generator in W-ZoomGAN to focus on and maintain multiple semantic information during signal generation, thereby improving signal enhancement quality. However, the auxiliary tasks and the main task are different, and the features they rely on are also different. Therefore, in the later stages of training, the feature extraction process of the auxiliary tasks may interfere with the feature extraction process of the main task. To address this, this invention copies the features relied upon by the three auxiliary tasks; the main task relies on the original features, and the auxiliary tasks rely on the copied features. This invention uses consistency constraints on the original and copied features to maintain the promoting effect of the auxiliary tasks on the main task. The consistency loss of the shallow features at the input end (encoder) is denoted as... The consistency loss of the layer features in the input (encoder) is denoted as... The consistency loss of deep features at the input end (encoder) is denoted as The consistency loss of shallow features at the output (decoder) is denoted as... The consistency loss of the layer features in the output (decoder) is denoted as... The consistency loss of deep features at the output (decoder) is denoted as... Total loss of auxiliary tasks L Auxiliary The calculation method is as follows:

[0074]

[0075] In the early stages of training, the main task (generation task) and the auxiliary task have high consistency, and the features relied upon by the auxiliary task and the main task are also highly consistent. Therefore, the feature consistency loss is negligible. The value is relatively small, and at this point, the auxiliary task mainly serves to facilitate the main task, so the total loss of the auxiliary task is L. Auxiliary The difference is significant. Later in training, the learning directions of the main task (generation task) and the auxiliary task diverge, and the features they rely on gradually become inconsistent, leading to a loss of feature consistency. The value is relatively large. At this point, the auxiliary task has more of an interfering effect on the main task, so the total loss of the auxiliary task is L. Auxiliary The effect is relatively small. In summary, this approach can evaluate the learning direction of the auxiliary task and the main task in real time during training. When the two learning directions are consistent (high feature consistency and low feature consistency loss), the loss of the auxiliary task is amplified, thereby enhancing the role of the auxiliary task. When the two learning directions are inconsistent (low feature consistency and high feature consistency loss), the loss of the auxiliary task is reduced, thereby suppressing the role of the auxiliary task and reducing its impact on the generation task.

[0076] The inertial sensor signal enhancement method provided by the present invention has the following advantages:

[0077] 1. An unsupervised zoom generative adversarial network (U-ZoomGAN) was designed, in which the designed zoom constraint mechanism can force the generator to learn the features of high-cost sensor signals at different scales.

[0078] 2. A weakly-supervised zoom generative adversarial network (W-ZoomGAN) was designed. In the W-ZoomGAN architecture, multiple auxiliary tasks are set up to constrain the consistency of semantic information between low-cost and high-cost signals, thereby improving the stability of the signal quality enhancement process and ultimately enhancing the signal quality enhancement effect.

[0079] 3. A learning direction representation mechanism was designed, which can prevent auxiliary tasks from interfering with the main task, thereby better leveraging the role of auxiliary tasks in enhancing the main task.

[0080] Example 2

[0081] In order to implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, an inertial sensor signal enhancement system based on zoom generative adversarial network is provided below.

[0082] The system includes:

[0083] An unsupervised zoom generative adversarial network construction module is used to construct an unsupervised zoom generative adversarial network; the unsupervised zoom generative adversarial network includes: a high-cost signal generator, a low-cost signal generator, and a discriminator.

[0084] A weakly supervised zoom generative adversarial network construction module is used to construct a weakly supervised zoom generative adversarial network; the weakly supervised zoom generative adversarial network includes an encoder and a decoder; multiple auxiliary tasks are set in the weakly supervised zoom generative adversarial network to constrain the semantic information consistency between low-cost sensor signals and high-cost sensor signals.

[0085] The first signal enhancement module is used to enhance the low-cost sensor signal by employing the trained high-cost signal generator in the unsupervised zoom generative adversarial network when there is no similar semantic information between the low-cost sensor signal and the high-cost sensor signal.

[0086] The second signal enhancement module is used to enhance the low-cost sensor signal by employing a trained weakly supervised zoom generative adversarial network when the low-cost sensor signal and the high-cost sensor signal have similar semantic information.

[0087] It also includes: a high-cost signal generator training module, which specifically includes:

[0088] The signal sample set acquisition unit is used to acquire real low-cost sensor signal samples and real high-cost sensor signal samples.

[0089] The first input unit is used to input the real low-cost sensor signal sample into the high-cost signal generator to obtain the first virtual high-cost sensor signal;

[0090] The second input unit is used to input the first virtual high-cost sensor signal to the low-cost signal generator to obtain the first virtual low-cost sensor signal.

[0091] The first optimization unit is used to perform initial optimization on the high-cost signal generator based on the error between the real low-cost sensor signal sample and the first virtual low-cost sensor signal.

[0092] The third input unit is used to input the real high-cost sensor signal sample into the low-cost signal generator to obtain the second virtual low-cost sensor signal.

[0093] The fourth input unit is used to input the second virtual low-cost sensor signal into the initially optimized high-cost signal generator to obtain the second virtual high-cost sensor signal.

[0094] The discrimination unit is used to discriminate the first virtual high-cost sensor signal and the second virtual high-cost sensor signal through the discriminator;

[0095] The second optimization unit is used to further optimize the initially optimized high-cost signal generator based on the discrimination result of the discriminator.

[0096] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0097] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for enhancing inertial sensor signals based on zoom-based generative adversarial networks, characterized in that, include: Constructing an unsupervised zoom generative adversarial network; The unsupervised zoom-based generative adversarial network includes: a high-cost signal generator, a low-cost signal generator, and a discriminator; the high-cost signal generator incorporates a zoom mechanism; the zoom mechanism specifically includes: extracting real low-cost sensor signal samples of different scales from the original signal stream; the real low-cost sensor signal samples of different scales include real low-cost sensor signal samples at the intrinsic scale and real low-cost sensor signal samples larger than the intrinsic scale; inputting the real low-cost sensor signal samples at the intrinsic scale into the high-cost signal generator, and downsampling the real low-cost sensor signal samples larger than the intrinsic scale to the intrinsic scale and then inputting them into different high-cost signal generators; optimizing the high-cost signal generator based on the output results of the two high-cost signal generators; A weakly supervised zoom-based generative adversarial network (GAN) is constructed, comprising an encoder and a decoder. Multiple auxiliary tasks are set within the GAN to constrain the semantic information consistency between low-cost and high-cost sensor signals. During the training process, when the learning direction of the auxiliary task aligns with that of the main task, the auxiliary task loss is amplified; when the learning direction of the auxiliary task is inconsistent with that of the main task, the auxiliary task loss is reduced to suppress the negative impact of the auxiliary task on the main task. The consistency of learning direction is determined using a consistency loss, which is the loss between the original features of the main task and the copied features of the auxiliary task. The copied features are obtained by copying the original features. When there is no similar semantic information between low-cost sensor signals and high-cost sensor signals, the trained high-cost signal generator in the unsupervised zoom generative adversarial network is used to enhance the low-cost sensor signals. When low-cost sensor signals and high-cost sensor signals have similar semantic information, a trained weakly supervised zoom generative adversarial network is used to enhance the low-cost sensor signals.

2. The inertial sensor signal enhancement method based on zoom-based generative adversarial networks according to claim 1, characterized in that, The training process of the high-cost signal generator is as follows: Obtain real low-cost sensor signal samples and real high-cost sensor signal samples; The real low-cost sensor signal sample is input into the high-cost signal generator to obtain the first virtual high-cost sensor signal; The first virtual high-cost sensor signal is input to the low-cost signal generator to obtain the first virtual low-cost sensor signal; The high-cost signal generator is initially optimized based on the error between the real low-cost sensor signal sample and the first virtual low-cost sensor signal. The real high-cost sensor signal sample is input into the low-cost signal generator to obtain a second virtual low-cost sensor signal. The second virtual low-cost sensor signal is input into the initially optimized high-cost signal generator to obtain the second virtual high-cost sensor signal. The discriminator distinguishes between the first virtual high-cost sensor signal and the second virtual high-cost sensor signal. The high-cost signal generator, which was initially optimized, is further optimized based on the discrimination results of the discriminator.

3. The inertial sensor signal enhancement method based on zoom-based generative adversarial networks according to claim 1, characterized in that, The auxiliary task loss includes: auxiliary task loss of shallow features at the input end, auxiliary task loss of mid-level features at the input end, auxiliary task loss of deep features at the input end, auxiliary task loss of shallow features at the output end, auxiliary task loss of mid-level features at the output end, and auxiliary task loss of deep features at the output end; the input end is the encoder; the output end is the decoder; the shallow features are features close to the data end; the deep features are features far from the data end; and the mid-level features are features between the shallow features and the deep features.

4. The inertial sensor signal enhancement method based on zoom-based generative adversarial networks according to claim 3, characterized in that, The total loss of the auxiliary task The calculation formula is as follows: in, The auxiliary task loss is used for shallow features at the input end; The auxiliary task loss is used for the mid-level features at the input end; The auxiliary task loss is used for deep features at the input end; This is an auxiliary task loss for shallow features at the output end; The auxiliary task loss is used for the mid-level features at the output end; This is an auxiliary task loss for deep features at the output end; Consistency loss for shallow features at the input end; The consistency loss is calculated for the mid-level features at the input end. Consistency loss for deep features at the input end; This represents the consistency loss of shallow features at the output end. This represents the consistency loss of the mid-level features at the output end. This represents the consistency loss of deep features at the output end.

5. An inertial sensor signal enhancement system based on zoom-based generative adversarial networks, characterized in that, include: An unsupervised zoom generative adversarial network building module is used to construct unsupervised zoom generative adversarial networks; The unsupervised zoom-based generative adversarial network includes: a high-cost signal generator, a low-cost signal generator, and a discriminator; the high-cost signal generator incorporates a zoom mechanism; the zoom mechanism specifically includes: extracting real low-cost sensor signal samples of different scales from the original signal stream; the real low-cost sensor signal samples of different scales include real low-cost sensor signal samples at the intrinsic scale and real low-cost sensor signal samples larger than the intrinsic scale; inputting the real low-cost sensor signal samples at the intrinsic scale into the high-cost signal generator, and downsampling the real low-cost sensor signal samples larger than the intrinsic scale to the intrinsic scale and then inputting them into different high-cost signal generators; optimizing the high-cost signal generator based on the output results of the two high-cost signal generators; A weakly supervised zoom-based generative adversarial network (GAN) construction module is used to construct a weakly supervised zoom-based GAN. The weakly supervised zoom-based GAN includes an encoder and a decoder. Multiple auxiliary tasks are set in the weakly supervised zoom-based GAN to constrain the semantic information consistency between low-cost and high-cost sensor signals. During the training process of the weakly supervised zoom-based GAN, when the learning direction of the auxiliary task is consistent with that of the main task, the loss of the auxiliary task is amplified; when the learning direction of the auxiliary task is inconsistent with that of the main task, the loss of the auxiliary task is reduced to suppress the negative impact of the auxiliary task on the main task. The consistency of the learning direction is determined by a consistency loss, which is the loss between the original features of the main task and the copied features of the auxiliary task. The copied features are obtained by copying the original features. The first signal enhancement module is used to enhance the low-cost sensor signal by employing the trained high-cost signal generator in the unsupervised zoom generative adversarial network when there is no similar semantic information between the low-cost sensor signal and the high-cost sensor signal. The second signal enhancement module is used to enhance the low-cost sensor signal by employing a trained weakly supervised zoom generative adversarial network when the low-cost sensor signal and the high-cost sensor signal have similar semantic information.

6. The inertial sensor signal enhancement system based on zoom-based generative adversarial networks according to claim 5, characterized in that, Also includes: A high-cost signal generator training module, specifically comprising: The signal sample set acquisition unit is used to acquire real low-cost sensor signal samples and real high-cost sensor signal samples. The first input unit is used to input the real low-cost sensor signal sample into the high-cost signal generator to obtain the first virtual high-cost sensor signal; The second input unit is used to input the first virtual high-cost sensor signal to the low-cost signal generator to obtain the first virtual low-cost sensor signal. The first optimization unit is used to perform initial optimization on the high-cost signal generator based on the error between the real low-cost sensor signal sample and the first virtual low-cost sensor signal. The third input unit is used to input the real high-cost sensor signal sample into the low-cost signal generator to obtain the second virtual low-cost sensor signal. The fourth input unit is used to input the second virtual low-cost sensor signal into the initially optimized high-cost signal generator to obtain the second virtual high-cost sensor signal. The discrimination unit is used to discriminate the first virtual high-cost sensor signal and the second virtual high-cost sensor signal through the discriminator; The second optimization unit is used to further optimize the initially optimized high-cost signal generator based on the discrimination result of the discriminator.

7. The inertial sensor signal enhancement system based on zoom-based generative adversarial networks according to claim 5, characterized in that, The auxiliary task loss includes: auxiliary task loss of shallow features at the input end, auxiliary task loss of mid-level features at the input end, auxiliary task loss of deep features at the input end, auxiliary task loss of shallow features at the output end, auxiliary task loss of mid-level features at the output end, and auxiliary task loss of deep features at the output end; the input end is the encoder; the output end is the decoder; the shallow features are features close to the data end; the deep features are features far from the data end; and the mid-level features are features between the shallow features and the deep features.

Images