Method and device for detecting abnormal subsequence in data sequences

Abstract

A method for detecting abnormal subsequences in data sequence includes constructing a hierarchical data structure of a target subsequence, each node in a bottommost layer of the data structure storing corresponding data of the target subsequence, and each node in a layer above the bottommost layer storing values based on data stored in corresponding nodes in a lower layer next to the layer above the bottommost layer; determining a second number of neighbors of the target subsequence based on the data structure of the target subsequence and of the first number of reference subsequences constructed in advance, the second number of neighbors having minimum Euclidean distances from the target subsequence; determining a third number of neighbors of each reference subsequence in the second number of reference subsequences, which have minimum Euclidean distances from each reference subsequence and determining whether the target subsequence is an abnormal subsequence.

Description

technical field [0001] The present invention relates to abnormal data detection, and more particularly to a method and device for detecting abnormal subsequences in a data sequence. Background technique [0002] In scenarios such as the Internet of Things (IOT) or Smart Earth, through a certain data generation mechanism, data can be continuously generated over time to form a time data sequence. For example, in a scenario where a detector is used to detect air pollutants, the detector continuously outputs data over time, thereby forming a time data sequence reflecting the air pollution level at each moment. In time data series, some data may deviate greatly from other data, such data can be called abnormal data. Since abnormal data can reflect certain problems in the data generation mechanism or some important states of the objects associated with the data, it is very important to detect abnormal data in time data series. [0003] Currently, various methods have been propos...

AI Technical Summary

Problems solved by technology

In these traditional methods, when detecting whether some data in a time data sequence is abnormal, all the data of the sequence need to be used, and the entire time data sequence needs to be scanned multiple times during the detection period, which makes the calculation of the detection operation very large. and it takes a long time

In addition, since the distribution density of all data in the time data series (especially the time data series generated in a long period of time) often varies greatly in the mapping space, if some data are detected based on the distribution density of all data Whether it is abnormal, the normal data whose distribution density is quite different from other data may be identified as abnormal data, making the result inaccurate

Moreover, traditional methods can only perform offline (non-real-time) detection of time series data, rather than online (real-time) detection, which is unacceptable for some scenarios that hope to obtain detection results as soon as possible

Images