Abnormal time series data value processing method based on adaptive threshold stationary wavelet transform

Abstract

The invention discloses an abnormal time series data value processing method based on adaptive threshold stationary wavelet transform. The invention mainly aims to overcome the problems that abnormal value false detection probability is large and efficiency is low in the prior art. According to a scheme in the invention, to-be-processed time series data f(n) containing abnormal values are obtained from time series data with abnormal values to be processed, and m-layer stationary wavelet transformation is carried out, so wavelet reconstruction detail coefficients Dj(n) of all layers and an m-layer reconstruction approximation coefficient sequence Am(n) are obtained; the reconstruction detail coefficient of each layer are summarized to obtain a reconstruction detail coefficient and a sequence D(n); an abnormal value detection threshold corresponding to each element in the D(n) is calculated; each element in the D(n) is judged according to the threshold to obtain a detected reconstruction detail coefficient and a sequence D'(n); and the D'(n) and the Am(n) are added to obtain data f'(n) after abnormal value processing. The method can accurately detect and process abnormal values in a large amount of time series data, and can be used for cleaning abnormal data in the time series data.

Description

technical field [0001] The invention belongs to the technical field of signal processing, and in particular relates to a method for processing abnormal values ​​of time series data, which can be used for cleaning abnormal data in time series data with a large amount of data. Background technique [0002] With the advent of the era of big data, due to the influence of accidental or inevitable factors in the massive data in various application fields, there are often some data that deviate from the vast majority of data. The mechanism by which these data are generated is different from most other data, and is often referred to as outlier data or outliers. If the abnormal data is mixed into the subsequent data analysis and processing, it will have a very adverse impact on the subsequent data analysis results, and even produce wrong data analysis results. Therefore, abnormal data must be detected from a large number of data samples and cleaned and repaired. [0003] In recent ...

AI Technical Summary

Problems solved by technology

[0006] (1) This method builds the abnormal data detection threshold based on the global statistical characteristics of the wavelet detail coefficients. When dealing with time series data with complex stationarity, the abnormal data detection threshold is affected by the non-stationary data segment and the stationary data segment at the same time. Some abnormal data are smaller than the detection threshold in the stable data segment, resulting in missed detection; some normal data are larger than the detection threshold in the processing of non-stationary data segment, and false detection occurs. These two phenomena eventually lead to a decline in detection performance

[0007] (2) Since this method uses the traditional discrete wavelet transform, when detecting abnormal data in the original data based on the wavelet detail coefficients of each layer, the wavelet detail coefficients of multiple data p

Images