Method and system for improving precision of low-end sensing data

Abstract

The invention discloses a method and a system for improving the data precision of a low-end sensor, which relate to the field of sensors and comprise three stages. In the first stage, three random noise adding strategies are adopted to generate shielding data to form a training data pair, and the method comprises the following steps: (1) the proportion of replacement data to original time series data is random; (2) the position of the replacement data in the original time sequence data is random; and (3) randomly generating replacement data, wherein the mean value, variance, time sequence length, spectral characteristics and the like of the replacement data are random. And then training the pre-trained BERT model by using the training data. In the second stage, data of a high-precision sensor is used as annotation data, the annotation data and an original low-end sensor form a training data pair, and the BERT model is further finely adjusted. And in the third stage, the trained BERT model is used in production and life to form a system to remove noise in low-end sensing data, and the precision is improved. And the noise reducer is plug-and-play and has practicability.

Description

Technical Field

[0001] The present invention relates to the technical field of sensor data denoising, and in particular to a method for improving the accuracy of low-end sensor data by using BERT (Bidirectional Encoder Representation from Transformers) and MLM (Masked Language Model). Background Art

[0002] Sensors play an important role in production and life. However, the raw sensor data is noisy, especially for low-cost sensors, the noise is so loud that it drowns out the real data, so efficient preprocessing or denoising is required before further use in the application. Traditional sensor denoising methods are mostly hand-crafted and may face problems such as sensor modeling errors and generalization problems. Some recent works have used deep neural networks (DNNs) to solve this problem and achieved good results. However, the deep learning methods reported so far are all based on supervised learning and require sufficient and accurate annotations. In practical applications, it is often difficult to obtain these annotated data. Therefore, self-supervised learning is used to remove noise from sensor data and improve the accuracy of low-end sensor data. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention proposes to use BERT (Bidirectional Encoder Representation from Transformers) and MLM (Masked Language Model) to improve the accuracy of low-end sensor data.

[0004] The present invention provides a method and system for improving the accuracy of low-end sensor data, and the method includes the following three stages: Phase S1: Step S11: Collect a large amount of unlabeled raw sensor time series data; Step S12: Three random noise adding strategies are adopted to replace 5%-40% of the continuous time series data in the original time series data. The three random noise adding strategies include: (1) The proportion of replacement data to the original time series data is random, ranging from about 2% to 40%; (2) The replacement data is randomly located in the original time series data; (3) The replacement data is randomly generated, and its mean, variance, time series length, and spectral characteristics are all random; Step S13: The original time series data and the noise-added time series data form a data pair, thereby forming a training data set; Step S14: training the pre-trained BERT with the training data set, recovering the original data therein for unsupervised training, and learning the expressions in the sensor data; Phase S2: Step S21: simultaneously measuring the physical quantity using the low-end sensor to be denoised and the high-precision sensor; Step S22: low-end sensor sequence data and high-precision sensor time series data form data pairs to form a training data set; Step S23: fine-tuning the pre-trained model in a supervised manner; Phase S3: Connect the model that can improve the accuracy of low-end sensor data after training to the sensor backend, input the original time series data, and output the denoised time series data, forming a system for improving the accuracy of low-end sensor data.

[0005] The system consists of the following three parts: Part P1: Original low-end sensor; Part P2: Computer that runs the trained model, including processor, memory, and network modules; Part P3: Follow-up steps using high-precision sensor data. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 This is a flow chart of improving the accuracy of low-end sensor data based on BERT and MLM in this embodiment. DETAILED DESCRIPTION

[0007] In order to facilitate the understanding of the present application, the present application will be described more comprehensively with reference to the relevant drawings. The drawings show the preferred implementation of the present application. However, before describing the steps of the present invention in detail, it is necessary to briefly describe the important tool (BERT model) of the present invention in combination with the use case.

[0008] BERT stands for Bidirectional Encoder Representation from Transformers. It is a bidirectional encoder representation based on Transformer. It is a huge pre-trained language model with more than 100M pre-trained parameters. The BERT model uses an English corpus of more than 3.3 billion words in the pre-training stage and uses a training method for predicting masked words. It randomly masks a word when a sentence is input into the model, and then uses other words in the sentence to predict the word. After multiple rounds of training convergence, the BERT model has semantic knowledge in its huge parameters. Specifically, it can predict the masked words in the sentence based on the semantics in the sentence and the knowledge in the model parameters. For example: Intel, based in Santa Clara, Calif., will invest more than $1 billion in India over the next five years, with $800 million going to expand its research and development center in Bangalore, the company's chairman Craig R. Barrett said.

[0009] It can be seen that the only word related to the word India in this sentence is the word Bangalore (a city in India). It has been verified that after masking 'India', BERT can capture such a long-distance semantic relationship and successfully predict India, which provides a powerful tool support for the method of the present invention. The BERT model has a vocabulary of size 30522. When the BERT model predicts a word, it actually outputs 30522 numbers through the Bert For Masked LM function to quantify the possibility that these 30522 words may fill in the position of the word to be predicted. For example, in the above example, the prediction of "India" means that the number corresponding to the word "India" is the largest among the 30522 numbers.

[0010] Figure 1This is a method and system for improving the accuracy of low-end sensor data based on BERT and MLM. The data 101 collected by the low-end sensor 100, after three randomization processes 306, constitutes a masked data set 307, which is sent to the pre-trained BERT number 300 for rough training 303. Among them, the processed data in the masked data set 307 is used as the training input, the original data 101 is used as the label, and the difference between the output and the label 302 is adjusted in the BERT model and the Linear+softmax model added to the back end of BERT. Then in the second stage S2, the data 101 collected by the low-end sensor 100 is used as the training input, and the data 201 collected synchronously by the high-end sensor 200 is used as the label, and is sent to the trained BERT for fine-tuning training 304. In the third stage S3, the data 101 collected by the low-end sensor 100 is directly sent to the trained BERT model according to the step 305, and high-precision sensor data 400 is output.

[0011] One approach to improving the accuracy of low-end sensor data consists of the following three stages: Phase S1: Step S11: Collect a large amount of unlabeled raw sensor time series data; Step S12: Three random noise adding strategies are adopted to replace 5%-40% of the continuous time series data in the original time series data. The three random noise adding strategies include: (1) The proportion of replacement data to the original time series data is random, ranging from about 2% to 40%; (2) The replacement data is randomly located in the original time series data; (3) The replacement data is randomly generated, and its mean, variance, time series length, and spectral characteristics are all random; Step S13: The original time series data and the noise-added time series data form a data pair, thereby forming a training data set; Step S14: training the pre-trained BERT with the training data set, recovering the original data therein for unsupervised training, and learning the expressions in the sensor data; Phase S2: Step S21: simultaneously measuring the physical quantity using the low-end sensor to be denoised and the high-precision sensor; Step S22: low-end sensor sequence data and high-precision sensor time series data form data pairs to form a training data set; Step S23: Fine-tune the pre-trained model in a supervised manner.

[0012] Phase S3: Connect the model that can improve the accuracy of low-end sensor data after training to the sensor backend, input the original time series data, and output the denoised time series data, forming a system for improving the accuracy of low-end sensor data.

[0013] A system for improving the accuracy of low-end sensor data consists of three parts: Part P1: Original low-end sensor; Part P2: Computer that runs the trained model, including processor, memory, and network modules; Part P3: Follow-up steps using high-precision sensor data.

Claims

1. A method and system for improving the accuracy of low-end sensor data, characterized in that: The method consists of the following three stages: Phase S1: Step S11: Collect a large amount of unlabeled raw sensor time series data; Step S12: Three random noise adding strategies are adopted to replace 5%-40% of the continuous time series data in the original time series data. The three random noise adding strategies include: (1) The proportion of replacement data to the original time series data is random, ranging from about 2% to 40%; (2) The replacement data is randomly located in the original time series data; (3) The replacement data is randomly generated, and its mean, variance, time series length, and spectral characteristics are all random; Step S13: The original time series data and the noise-added time series data form a data pair, thereby forming a training data set; Step S14: training the pre-trained BERT with the training data set, recovering the original data therein for unsupervised training, and learning the expressions in the sensor data; Phase S2: Step S21: simultaneously measuring the physical quantity using the low-end sensor to be denoised and the high-precision sensor; Step S22: low-end sensor sequence data and high-precision sensor time series data form data pairs to form a training data set; Step S23: fine-tuning the pre-trained model in a supervised manner; Phase S3: Connect the model that can improve the accuracy of low-end sensor data after training to the sensor backend, input the original time series data, and output the denoised time series data, forming a system for improving the accuracy of low-end sensor data.

2. The method according to claim 1, characterized in that The system includes the following three parts: Part P1: Original low-end sensor; Part P2: Computer that runs the trained model, including processor, memory, and network modules; Part P3: Follow-up steps using high-precision sensor data.

Images