Subsequence anomaly detection algorithm for time series data

Through the subsequence exception detection algorithm of time series data, the SAX character and PAA dimensionality reduction technology are used to solve the problem of complexity in abnormal pattern search in time series data, and automatic detection and efficient recognition of abnormal subsequences of different lengths are realized.

CN118916805APending Publication Date: 2024-11-08CHINA TELECOM CLOUD TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410536611.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-04-30
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The prior art is difficult to deal with exception pattern search in time series data, especially the need to know the length of the exception pattern in advance, which is unrealistic in practical problems, and patterns with similar exception structures but different lengths may exist simultaneously, increasing complexity .

Method used

A subsequence exception detection algorithm for time-series data is proposed. By generating SAX characters, sorting the number of occurrences of each character in the syntax rules, the exception score is calculated, and anomaly subs of different lengths is detected through PAA dimensionality reduction and symbolic representation. Sequence, syntax compression and cropping techniques are used to speed up the detection process.

Benefits of technology

It can automatically discover abnormal subsequences of different lengths, improve detection efficiency and accuracy, draw density curves according to the frequency of occurrence of strings, and identify the most likely abnormal subsequences.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916805A_ABST
    Figure CN118916805A_ABST
Patent Text Reader

Abstract

The invention discloses a time series data sub-sequence anomaly detection algorithm, relates to the technical field of time series data anomaly detection, and can detect abnormal sub-sequences with different lengths. In a real production environment, abnormal subsequences with different lengths may occur, and the lengths of the abnormal subsequences need to be specified in previous methods; when the abnormal score of each sub-sequence is calculated, sorting can be performed according to the occurrence frequency of character strings in the grammar rule, the most abnormal sub-sequence is calculated preferentially, and the detection efficiency is improved; a density curve can be drawn according to the occurrence frequency of a character string after syntactic compression, and the sub-sequence with the lowest density is most likely to be abnormal.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of time series data anomaly detection, and specifically relates to a subsequence anomaly detection algorithm for time series data. Background Art

[0002] Time series data appear in many fields such as environmental science, finance, astrophysics, neuroscience, engineering, etc. In the past decade, discovering frequently occurring abnormal patterns in time series data has attracted the attention of researchers. Existing algorithms usually require anomalies to have distinctive features and known lengths, which may be impractical for many practical problems.

[0003] At present, there are several challenges in the research work on time series anomaly detection at home and abroad.

[0004] Challenge 1: Searching for unusual patterns in time series data is often intractable.

[0005] Challenge 2: Most algorithms require prior knowledge of the length of the anomaly pattern as input, which is often an impractical requirement.

[0006] Challenge 3: Patterns with similar anomaly structures but different lengths may coexist in the time series, increasing the complexity. Summary of the invention

[0007] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a subsequence anomaly detection algorithm for time series data.

[0008] The subsequence anomaly detection algorithm for time series data includes the following steps:

[0009] Generate SAX characters stage:

[0010] A grammar is formed from repeated phrases in a sequence; each repetition generates a rule in the grammar, and the repeated subsequence is replaced by a non-terminal symbol, after which the generated grammar is trimmed;

[0011] Anomaly detection stage: sort the number of occurrences of each character in the grammatical rules, and then calculate the anomaly score of each character.

[0012] Furthermore, the method for generating SAX characters is:

[0013] Normalize the original time series;

[0014] Dimensionality reduction was performed using the PAA method;

[0015] Symbolically, the mean of PAA is mapped to the corresponding letters and finally discretized into a string.

[0016] Furthermore, the specific method of tailoring the generated grammar is as follows:

[0017] If the same SAX string appears continuously, the consecutive SAX characters are trimmed and only the first SAX character is retained.

[0018] Furthermore, the specific method for calculating the abnormality score of each character is as follows:

[0019] Anomaly detection is performed based on the grammatical rules obtained in the grammar compression phase, and different methods are used to process external sorting and internal loops.

[0020] Furthermore, the specific processing method for external sorting is:

[0021] External sorting uses the information obtained from the hierarchical grammatical structure to sort the subsequences in ascending order of the frequency of use of the corresponding rules.

[0022] Furthermore, the specific processing method for the internal loop is:

[0023] The inner loop selects candidate subsequences through the grammatical rules selected by the outer loop, and then accesses them in random order to calculate the anomaly score.

[0024] Furthermore, the specific method of standardizing the original time series is to convert the original time series into a series with a mean of 0 and a standard deviation of 1.

[0025] Furthermore, the PAA method is used to reduce the dimension as follows:

[0026] The original time series is divided into several segments of equal length, and then the subsequences of each segment are replaced.

[0027] Furthermore, the specific method of dimensionality reduction by the PAA method is as follows:

[0028] First, divide the original time series into w subsequences of equal length, and then replace each subsequence with its mean.

[0029] Furthermore, W is a preset value.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] This application can detect abnormal subsequences of different lengths. In a real production environment, abnormal subsequences of different lengths may appear. The previous methods all need to specify the length of the abnormal subsequence;

[0032] When calculating the anomaly score of each subsequence, the strings can be sorted according to the number of times they appear in the grammatical rules, and the most abnormal subsequences can be calculated first to speed up the detection efficiency.

[0033] After grammatical compression, the subsequence can be plotted into a density curve according to the frequency of occurrence of the string, and the one with the lowest density is most likely to be abnormal. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The PAA dimension reduction formula and specific schematic diagram of the present invention;

[0035] Figure 2 A schematic diagram for symbolically representing the algorithm of the present invention;

[0036] Figure 3 This is a schematic diagram of the original time series of the present invention being discretized into a string;

[0037] Figure 4 A schematic diagram of cutting SAX characters of the present invention;

[0038] Figure 5 It is a schematic diagram of the detection effect of the present invention. DETAILED DESCRIPTION

[0039] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] See also Figure 1-Figure 5 This application provides a subsequence anomaly detection algorithm for time series data. The time series anomaly detection algorithm based on syntax compression has the characteristics of automatically discovering abnormal subsequences of different lengths, high accuracy, and high detection efficiency. The algorithm flow is as follows Figure 1 As shown; the algorithm mainly includes the following steps:

[0041] SAX stage, generating SAX characters;

[0042] In the grammar compression phase, a grammar is formed based on repeated phrases in the sequence; each repetition generates a rule in the grammar, and the repeated subsequence is replaced by a non-terminal symbol, thus generating a more concise representation of the entire sequence;

[0043] In the anomaly detection phase, the number of occurrences of each character in the grammatical rules is used to sort them, and then the anomaly score of each character is calculated;

[0044] Among them, the SAX stage is mainly divided into three steps:

[0045] (1) Standardize the original time series and transform it into a series with a mean of 0 and a standard deviation of 1;

[0046] (2) Dimensionality reduction is performed using the PAA method; that is, the original time series is first divided into w subsequences of equal length, and then the mean of each subsequence is used to replace the subsequence, such as Figure 1 As shown; here w is a preset value.

[0047] (3) Symbolic representation: the mean of PAA is mapped to the corresponding letters and finally discretized into a string. The mapping method is the mapping table preset by the administrator; Figure 2 As shown;

[0048] Step 2: Syntax compression stage;

[0049] A grammar is formed from repeated phrases in a sequence; each repetition produces a rule in the grammar, and the repeated subsequence is replaced by a nonterminal symbol, resulting in a more concise representation of the entire sequence, such as Figure 3 As shown;

[0050] After generating the grammar, the generated grammar is trimmed;

[0051] The rule of trimming is that if the same SAX string appears continuously, the consecutive SAX characters are trimmed and only the first SAX character is retained, such as Figure 4 As shown;

[0052] Step 3: Anomaly detection phase;

[0053] Anomaly detection is performed based on the grammatical rules obtained in the grammar compression phase. External sorting uses the information obtained from the hierarchical grammatical structure to sort the subsequences in ascending order of the frequency of use of the corresponding rules;

[0054] The inner loop selects candidate subsequences through the grammatical rules selected by the outer loop, and then accesses them in random order to calculate the anomaly score.

[0055] In the system interface, users only need to upload the data set, set the model-related parameters, and finally visualize the results of subsequence anomaly detection. Finally, this method is used to detect subsequence anomalies. The detection effect is as follows: Figure 5 As shown, the point with the highest value indicates that the corresponding subsequence is an abnormal subsequence.

[0056] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A subsequence anomaly detection algorithm for time series data, characterized by: The method specifically comprises the following steps: Generate SAX characters stage: A grammar is formed from repeated phrases in a sequence; each repetition generates a rule in the grammar, and the repeated subsequence is replaced by a non-terminal symbol, after which the generated grammar is trimmed; Anomaly detection stage: sort the number of occurrences of each character in the grammatical rules, and then calculate the anomaly score of each character.

2. The subsequence anomaly detection algorithm for time series data according to claim 1, characterized in that: The method to generate SAX characters is: Normalize the original time series; Dimensionality reduction was performed using the PAA method; Symbolically, the mean of PAA is mapped to the corresponding letters and finally discretized into a string.

3. The subsequence anomaly detection algorithm for time series data according to claim 2, characterized in that: The specific way to tailor the generated grammar is as follows: If the same SAX string appears continuously, the consecutive SAX characters are trimmed and only the first SAX character is retained.

4. The subsequence anomaly detection algorithm for time series data according to claim 1, characterized in that: The specific way to calculate the anomaly score of each character is: Anomaly detection is performed based on the grammatical rules obtained in the grammar compression phase, and different methods are used to process external sorting and internal loops.

5. The subsequence anomaly detection algorithm for time series data according to claim 4, characterized in that: The specific processing method for external sorting is: External sorting uses the information obtained from the hierarchical grammatical structure to sort the subsequences in ascending order of the frequency of use of the corresponding rules.

6. The subsequence anomaly detection algorithm for time series data according to claim 5, characterized in that: The specific processing method for the internal loop is: The inner loop selects candidate subsequences through the grammatical rules selected by the outer loop, and then accesses them in random order to calculate the anomaly score.

7. The subsequence anomaly detection algorithm for time series data according to claim 2, characterized in that: The specific method of standardizing the original time series is to convert the original time series into a series with a mean of 0 and a standard deviation of 1.

8. The subsequence anomaly detection algorithm for time series data according to claim 1, characterized in that: The way to reduce dimension using PAA method is: The original time series is divided into several segments of equal length, and then the subsequences of each segment are replaced.

9. The subsequence anomaly detection algorithm for time series data according to claim 8, characterized in that: The specific method of dimensionality reduction by PAA method is as follows: First, divide the original time series into w subsequences of equal length, and then replace each subsequence with its mean.

10. The subsequence anomaly detection algorithm for time series data according to claim 9, characterized in that: W is a preset value.