Time series data filling and restoring method based on machine learning

Abstract

The invention relates to the technical field of computer time series data analysis and prediction, in particular to a time series data filling and restoring method based on machine learning. The method includes: filling the missing value by using a domain-based median and mean value filling method; estimating a true value of an expected sampling moment through a linear rule; detecting wave crestsand wave troughs of the time sequence, and smoothing abnormal values; taking hundreds of thousands of collected real data as samples, designing and generating time sequence characteristics, taking real results as labels, and training a machine learning model based on an XGBoost (Extreme Gradient Boost) for predicting a large number of unknown data. According to the method, the problems of multiplemissing values, large volatility, error accumulation and the like of specific time sequence data are solved, and the accuracy of data filling and restoring is effectively improved; moreover, the complexity of a machine learning model is well controlled, the filling and restoration of hundreds of millions of data records can be completed within an hour level, and the method has a high practical value.

Description

technical field [0001] The invention relates to the technical field of computer time series data analysis and prediction, in particular to a method for filling and restoring time series data based on machine learning. Background technique [0002] At present, information technology is widely used in all walks of life and continuously generates various related data, and data collection and mining technology is also emerging, providing strong support for management decisions in related industries and improving economic and social benefits. [0003] Data acquisition is the process of collecting, identifying, and selecting data from data sources. Data collection can be divided into real-time collection and interval collection. Real-time acquisition refers to the acquisition of data during its existence. Interval collection refers to the collection of data at equally spaced time points. Ideal real-time collection can retain the original data to the greatest extent, thus provid...

AI Technical Summary

Problems solved by technology

[0006] 1. Simply adopting the mean filling method, data association-based and density-based filling methods will cause serious bias in data restoration due to data volatility and affect the availability of collected data;

[0007] 2. The data restoration method based on deep learning in the industry is prone to model overfitting, time-consuming training and prediction, and it is difficult to be practical for filling and restoring large-scale data

Images