Time sequence anomaly detection method and system based on grammar compression

Abstract

The invention relates to the technical field of time sequence analysis, and discloses a syntax compression-based time sequence anomaly detection method and system.The method comprises the following steps: discretizing an input original time sequence to determine a symbolic representation sequence, and if repeated continuous symbol segments exist in the symbolic representation sequence, judging whether the symbolic representation sequence is abnormal or not; if yes, replacing the continuous symbol segment of the symbol representation sequence with an intermediate representation symbol having a syntax mapping relationship with the continuous symbol segment for compression to obtain a compressed syntax representation sequence corresponding to the symbol representation sequence, and de-duplicating all symbols to obtain a non-repeated compressed representation sequence; and calculating the occurrence frequency of the symbols in the non-repeated compression representation sequence to determine the abnormal score of the symbols, and taking the sub-sequence corresponding to the symbols with the abnormal score higher than a preset abnormal threshold as an abnormal sub-sequence. According to the method, the continuous time sequence data is converted into the symbol representation sequence, grammar compression and deduplication are further performed, and the anomaly score is calculated through the symbol occurrence frequency, so that the accuracy of time sequence anomaly detection is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of time series analysis, and in particular to a time series anomaly detection method and system based on grammar compression. Background Art

[0002] As an important form of data, time series data is widely used in many fields such as environmental science, finance, astrophysics, neuroscience, and engineering. For example, equipment operation data in industrial production, stock price trends in financial markets, and temperature changes in environmental monitoring are all typical time series data.

[0003] Taking the time series data collected by the device as an example, a certain device may experience short-term abnormal vibration during normal operation. In the actual operating environment, the length of the abnormal pattern caused by the equipment failure is often uncertain. However, the existing anomaly detection algorithm sets the length of the abnormal pattern to a fixed value. Regardless of whether the actual abnormal length is longer or shorter, only abnormal information of a fixed length can be detected, resulting in inaccurate detection results of the abnormal time series. Summary of the invention

[0004] In view of this, the present invention provides a time series anomaly detection method and system based on syntax compression to solve the problem of how to improve the detection accuracy of time series anomalies.

[0005] In a first aspect, the present invention provides a time series anomaly detection method based on syntax compression, the method comprising: discretizing an input original time series, determining a symbol representation sequence corresponding to the original time series, wherein the symbol representation sequence comprises a plurality of continuous symbols, each symbol mapping a subsequence of a preset length in the original time series;

[0006] If there are repeated continuous symbol segments in the symbol representation sequence, the continuous symbol segments are replaced with intermediate representation symbols having a syntax mapping relationship therewith, and the symbol representation sequence is compressed to obtain a compressed syntax representation sequence corresponding to the symbol representation sequence;

[0007] De-duplicating all symbols in the compressed syntax representation sequence to obtain a non-repetitive compressed representation sequence;

[0008] Calculating the frequency of occurrence of each symbol in the non-repetitive compressed representation sequence, and determining a symbol anomaly score according to the frequency;

[0009] The subsequences corresponding to the symbols whose symbol anomaly scores are higher than the preset anomaly threshold are taken as abnormal subsequences.

[0010] The time series anomaly detection method based on syntax compression provided in this embodiment discretizes the input original time series, determines the symbol representation sequence corresponding to the original time series, and simplifies the representation of the original data by dimensionality reduction symbolization, thereby reducing the complexity of subsequent processing; if there are repeated continuous symbol segments in the symbol representation sequence, the continuous symbol segments are replaced by intermediate representation symbols with which there is a syntax mapping relationship, and the symbol representation sequence is compressed to obtain a compressed syntax representation sequence corresponding to the symbol representation sequence, and compression is achieved based on repeated segments in the symbol sequence, thereby determining regular changes in the time series and simplifying the recognition difficulty of the time series; the frequency of occurrence of each symbol in the non-repeating compressed representation sequence is calculated, and the symbol anomaly score is determined by the frequency, and the unique symbol in the sequence is extracted to provide a basis for subsequent frequency statistics, and finally the abnormal subsequence is determined according to the anomaly score; the present invention adaptively identifies repeated patterns through a syntax compression method, does not need to preset a fixed length, avoids the requirement of the prior art to pre-set a fixed abnormal pattern length, and cannot adapt to actual changes. Similar patterns are mapped to the same symbol through symbolic representation and compression, which simplifies the pattern recognition process and reduces data complexity, thereby improving the accuracy of time series anomaly detection.

[0011] In an optional implementation, the discretization processing of the input original time series to determine the symbol representation sequence corresponding to the original time series specifically includes:

[0012] Calculate the standard deviation of each item of data in the original time series based on the standard deviation algorithm to obtain a time series after the standard deviation is calculated;

[0013] Based on the segmented average approximation processing algorithm, the time series after the standard deviation is calculated is divided into a plurality of subsequences of the same preset length, and the standard deviation mean corresponding to each subsequence is determined;

[0014] Determining the symbol mapped to each subsequence according to the standard deviation mean corresponding to each subsequence and a preset symbol representation interval;

[0015] A symbol representation sequence corresponding to the original time series is generated based on the symbols mapped to each subsequence.

[0016] The embodiment of the present invention can highlight the relative change characteristics of time series data through standardization processing. The segmented average approximation processing algorithm divides the time series data into subsequences of equal length, thereby ensuring the consistency of input and output variables. The symbol mapped to each subsequence is further determined according to the standard deviation mean corresponding to each subsequence and the preset symbol representation interval, thereby achieving dimensionality reduction and eliminating the appearance of complex data, making the pattern characteristics of the original time series easier to identify. The symbols determined by each subsequence are organized according to the temporal relationship of the original data to form a complete symbolic representation sequence, which provides a basic data structure for subsequent pattern recognition and anomaly detection, retaining the main characteristics of the time series and greatly reducing the processing complexity of the time series data.

[0017] In an optional implementation, if there are repeated continuous symbol segments in the symbol representation sequence, the continuous symbol segments are replaced with intermediate representation symbols having a syntax mapping relationship therewith, and the symbol representation sequence is compressed to obtain a compressed syntax representation sequence corresponding to the symbol representation sequence, specifically including:

[0018] Traversing the symbol representation sequence to obtain repeated continuous symbol segments, and determining positions of the repeated continuous symbol segments;

[0019] The position of the repeated continuous symbol segment is replaced by an intermediate representation symbol having a syntax mapping relationship therewith, and the traversal is stopped until there is no repeated continuous symbol segment in the symbol representation sequence;

[0020] The traversed symbol representation sequence is used to represent the compressed grammar representation sequence.

[0021] The embodiment of the present invention traverses the repeated fields in the sequence to provide a basis for subsequent compression replacement, and applies syntax mapping rules to compress the repeated symbol segments into a more concise representation, thereby retaining the structural characteristics of the sequence and reducing the amount of data, providing a more concise data representation for subsequent anomaly detection and improving the efficiency of anomaly detection.

[0022] In an optional implementation, removing duplicate symbols from the compressed syntax representation sequence to obtain a non-repetitive compressed representation sequence specifically includes:

[0023] Traversing each symbol in the compressed grammar representation sequence; during the traversal process, determining whether the current symbol is the same as any other symbol in the compressed grammar representation sequence, if the current symbol is the same, retaining the current symbol and its position, and clearing any other symbol and position that is the same as the current symbol, until the last symbol is traversed, then stopping the traversal;

[0024] The traversed compressed grammar representation sequence is used to represent a non-repetitive compressed representation sequence.

[0025] The embodiment of the present invention retains the first occurrence of each symbol in the compressed syntax representation sequence through deduplication processing, eliminates duplicate information to make the sequence more concise, and provides a non-redundant data basis for subsequent frequency statistics. Through the deduplication operation, the distribution characteristics of different symbols in the time series are more accurately reflected without the interference of complex original data, thereby improving the accuracy of anomaly detection.

[0026] In an optional implementation, calculating the frequency of occurrence of each symbol in the non-repetitive compressed representation sequence and determining the symbol anomaly score according to the frequency specifically includes:

[0027] Comparing the non-repeating compressed representation sequence with each character in the compressed grammatical representation sequence to determine the number of identical characters;

[0028] Calculate the proportion of identical characters in all characters based on the number of identical characters;

[0029] Determine the frequency of occurrence of each symbol through the proportion;

[0030] According to the frequency of occurrence of each symbol and a preset frequency-abnormality score mapping relationship, a symbol abnormality score corresponding to each symbol is determined; wherein the frequency of occurrence of each symbol is negatively correlated with the symbol abnormality score.

[0031] In an embodiment of the present invention, each character in the non-repetitive compressed representation sequence is compared with the compressed grammatical representation sequence, and the proportion of identical characters in all characters is calculated based on the number of identical characters. The regularity of pattern occurrence is reflected by character frequency, and low-frequency patterns correspond to high anomaly scores, so that the anomaly detection result is more explanatory, and there is no need to pre-set a fixed anomaly length, which reduces the complexity of data processing and improves the accuracy of anomaly detection.

[0032] In an optional embodiment, the method further includes:

[0033] Obtaining a time series coordinate graph corresponding to the original time series, wherein the abscissa of the time series coordinate graph represents time and the ordinate represents sequence value;

[0034] The corresponding abnormal position of the abnormal subsequence is marked on the sequence coordinate graph according to the time and sequence value.

[0035] The embodiment of the present invention intuitively displays the anomaly detection results in a visual manner to facilitate understanding of the abnormal patterns in the time series. The anomalies are marked by frequency, and abnormal areas of different degrees are highlighted, which can make the anomaly detection results more intuitive.

[0036] In a second aspect, the present invention provides a time series anomaly detection system based on syntax compression, the system comprising:

[0037] A sequence symbolization module, used for discretizing the input original time series and determining a symbol representation sequence corresponding to the original time series, wherein the symbol representation sequence includes a plurality of continuous symbols, each symbol mapping a subsequence of a preset length in the original time series;

[0038] A syntax compression module, configured to, if there are repeated continuous symbol segments in the symbol representation sequence, replace the continuous symbol segments with intermediate representation symbols having a syntax mapping relationship therewith, compress the symbol representation sequence, and obtain a compressed syntax representation sequence corresponding to the symbol representation sequence;

[0039] A symbol deduplication module, used for deduplicating all symbols in the compressed syntax representation sequence to obtain a non-repetitive compressed representation sequence;

[0040] A frequency analysis module, used to calculate the frequency of occurrence of each symbol in the non-repetitive compressed representation sequence, and determine the symbol anomaly score according to the frequency;

[0041] The anomaly detection module is used to take the subsequence corresponding to the symbol whose symbol anomaly score is higher than the preset anomaly threshold as an abnormal subsequence.

[0042] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the time series anomaly detection method based on grammatical compression of the above-mentioned first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0043] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the time series anomaly detection method based on grammatical compression of the above-mentioned first aspect or any corresponding embodiment thereof.

[0044] In a fifth aspect, the present invention provides a computer program product, comprising computer instructions, wherein the computer instructions are used to enable a computer to execute the time series anomaly detection method based on grammatical compression of the above-mentioned first aspect or any corresponding embodiment thereof. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 is a flow chart of a method for detecting anomalies in a time series based on syntax compression according to an embodiment of the present invention;

[0047] Figure 2 is a flow chart of another method for detecting anomalies in a time series based on syntax compression according to an embodiment of the present invention;

[0048] Figure 3 is a schematic diagram of syntax compression according to a method for detecting anomalies in a time series based on syntax compression according to an embodiment of the present invention;

[0049] Figure 4 is a schematic diagram of symbol deduplication according to a time series anomaly detection method based on syntax compression according to an embodiment of the present invention;

[0050] Figure 5 is a flow chart of another method for detecting anomalies in a time series based on syntax compression according to an embodiment of the present invention;

[0051] Figure 6 is a schematic diagram of anomaly density detection of a time series anomaly detection method based on syntax compression according to an embodiment of the present invention;

[0052] Figure 7 is a structural block diagram of a time series anomaly detection system based on syntax compression according to an embodiment of the present invention;

[0053] Figure 8 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0055] Discovering frequently occurring abnormal patterns in time series data has attracted the attention of researchers. Existing algorithms usually require anomalies to have distinctive features and known lengths, which cannot adapt to actual changing environments and scenarios.

[0056] Taking the time series data collected by the device as an example, a certain device may experience short-term abnormal vibration during normal operation. In the actual operating environment, the length of the abnormal pattern caused by the equipment failure is often uncertain. However, the existing anomaly detection algorithm sets the length of the abnormal pattern to a fixed value. Regardless of whether the actual abnormal length is longer or shorter, only abnormal information of a fixed length can be detected, resulting in inaccurate detection results of the abnormal time series.

[0057] Therefore, the search for abnormal patterns in time series data in the prior art is usually difficult to handle. Most algorithms require knowing the length of the abnormal pattern in advance as input, which is an unrealistic requirement. Patterns with similar abnormal structures but different lengths may exist in the time series at the same time, which increases the complexity and leads to inaccurate time series anomaly detection results.

[0058] The embodiment of the present invention provides a time series anomaly detection method based on syntax compression, which is applied to equipment monitoring scenarios, time series data analysis, dynamic time data monitoring scenarios, and other situations involving time series. By converting continuous time series data into a symbol representation sequence, the processing process is simplified, repeated continuous symbol segments are identified and replaced with intermediate representation symbols, and there is no need to pre-set the abnormal pattern length. The abnormality score is calculated by the frequency of symbol occurrence, where a symbol sequence that appears at a low frequency is more likely to be an abnormal sequence, thereby overcoming the limitation of the existing method that requires a preset abnormal pattern length, and effectively improving the accuracy of anomaly detection.

[0059] According to an embodiment of the present invention, an embodiment of a time series anomaly detection method based on syntax compression is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0060] In this embodiment, a time series anomaly detection method based on syntax compression is provided, which can be used for the above-mentioned computer. Figure 1 is a flowchart of a time series anomaly detection method based on syntax compression according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0061] Step S101 , discretize the input original time series to determine a symbol representation sequence corresponding to the original time series, wherein the symbol representation sequence includes a plurality of continuous symbols, and each symbol maps a subsequence of a preset length in the original time series.

[0062] It should be noted that discretization refers to the process of converting continuous time series data into discrete symbol sequences. Symbol representation sequence refers to the use of a series of symbols to represent the data features of the original time series. Subsequences of preset length refer to data segments obtained by splitting the original time series into fixed lengths.

[0063] Specifically, a fixed-size window can be used to slide on the original time series, the data can be divided into subsequences of equal length and symbols can be assigned according to different numerical intervals to obtain a symbolic representation sequence, or the numerical values ​​of each subsequence can be segmented and averaged and then mapped to symbols to obtain a symbolic representation sequence. The representation of the original data can be simplified by symbolizing the dimensionality reduction, reducing the complexity of subsequent processing while retaining the main features of the original data.

[0064] Step S102: If there are repeated continuous symbol segments in the symbol representation sequence, the continuous symbol segments are replaced with intermediate representation symbols having a syntax mapping relationship therewith, and the symbol representation sequence is compressed to obtain a compressed syntax representation sequence corresponding to the symbol representation sequence.

[0065] It should be noted that repeated continuous symbol segments refer to the same symbol combination that appears continuously in the symbol representation sequence. The syntax mapping relationship refers to the corresponding rules for replacing repeated symbol segments with new symbols. The compressed syntax representation sequence refers to a shorter symbol sequence obtained after compression.

[0066] Specifically, the symbol representation sequence can be compressed by iteratively searching for the most frequently occurring symbol and replacing it, or dynamically constructing the mapping relationship representation of each layer of symbols based on grammatical rules. Compression is achieved based on repeated segments in the symbol sequence, thereby determining regular changes in the time series.

[0067] Exemplarily, the symbol sequence is abcabcabc, it can be determined that abc is a repeated symbol segment, and a new symbol X can be introduced to represent abc, then the compressed sequence becomes XXX, and the corresponding mapping relationship rule is X->abc.

[0068] Step S103: remove duplicate symbols from the compressed syntax representation sequence to obtain a non-repetitive compressed representation sequence.

[0069] It should be noted that deduplication of all symbols refers to deleting repeated symbols in the compressed syntax representation sequence and retaining only one occurrence of each symbol. The non-repetitive compressed representation sequence refers to the symbol sequence after deduplication processing.

[0070] Specifically, the sequence can be converted into a data structure, and the number and position of the symbol in the data structure can be recorded using hashes. Duplicate characters can be deleted based on the number and position of the symbol, and only the first-appearing characters can be retained to obtain the original order of the symbols after deduplication, thus forming a non-repeating compressed representation sequence. By extracting the unique symbol in the sequence, a basis can be provided for subsequent frequency statistics.

[0071] Exemplarily, the compressed grammatical representation sequence is XXXYZ, where X appears repeatedly in the sequence. After deduplication, a non-repeating compressed representation sequence XYZ is obtained, where XXXYZ corresponds to the original time series data [1,2,3,1,2,3,1,2,3,4,5,6,7].

[0072] Step S104, calculating the frequency of occurrence of each symbol in the non-repetitive compressed representation sequence, and determining the symbol abnormality score according to the frequency.

[0073] It should be noted that the frequency of symbol occurrence refers to the ratio of the number of times each symbol appears in the original compressed syntax representation sequence to the total number of symbols. The symbol anomaly score is a quantitative value of the degree of anomaly calculated based on the frequency.

[0074] Specifically, the number of occurrences of each symbol in the original sequence is calculated, and the relative frequency is calculated; the anomaly score is calculated based on the relative frequency according to the preset frequency feature calculation rules, or the anomaly score is calculated based on the preset mapping relationship between the frequency and the anomaly score. By quantifying the degree of anomaly of the symbol, it is convenient to identify patterns that do not conform to the normal distribution law in the future.

[0075] For example, if symbol X appears 3 times in the original sequence and the total length is 5, its frequency is 0.6. If the frequency threshold is set to 0.5, the frequency greater than the threshold can be converted into the abnormal score of symbol X according to the preset frequency feature calculation rule.

[0076] Step S105 : taking the subsequences corresponding to the symbols whose symbol anomaly scores are higher than the preset anomaly threshold as abnormal subsequences.

[0077] Specifically, a preset anomaly threshold is set, the anomaly score of each symbol is compared with the anomaly threshold, symbols with anomaly scores higher than the threshold are screened out, and the abnormal symbols are mapped back to the original subsequence according to the grammatical mapping relationship, the time window position corresponding to the abnormal symbol is located, and the original data segment corresponding to the window position is extracted, and finally the start and end time of the abnormal subsequence and the specific value marking the abnormal subsequence are obtained.

[0078] For example, the anomaly score (0.6) of symbol X exceeds the threshold (0.5), and the grammatical mapping relationship of X is searched: X->abc, and the subsequences [1,2,3,1,2,3,1,2,3] corresponding to multiple abc consecutive segments in the original sequence are located and marked as abnormal subsequences.

[0079] The time series anomaly detection method based on syntax compression provided in this embodiment discretizes the input original time series, determines the symbol representation sequence corresponding to the original time series, and simplifies the representation of the original data by dimensionality reduction symbolization, thereby reducing the complexity of subsequent processing; if there are repeated continuous symbol segments in the symbol representation sequence, the continuous symbol segments are replaced by intermediate representation symbols with which there is a syntax mapping relationship, and the symbol representation sequence is compressed to obtain a compressed syntax representation sequence corresponding to the symbol representation sequence, and compression is achieved based on repeated segments in the symbol sequence, thereby determining regular changes in the time series and simplifying the recognition difficulty of the time series; the frequency of occurrence of each symbol in the non-repeating compressed representation sequence is calculated, and the symbol anomaly score is determined by the frequency, and the unique symbol in the sequence is extracted to provide a basis for subsequent frequency statistics, and finally the abnormal subsequence is determined according to the anomaly score; the present invention adaptively identifies repeated patterns through a syntax compression method, does not need to preset a fixed length, avoids the requirement of the prior art to pre-set a fixed abnormal pattern length, and cannot adapt to actual changes, and maps similar patterns to the same symbol through symbolic representation and compression, simplifies the pattern recognition process, reduces data complexity, and improves the accuracy of time series anomaly detection.

[0080] In this embodiment, a time series anomaly detection method based on syntax compression is provided, which can be used for the above-mentioned computer. Figure 2 is a flowchart of a time series anomaly detection method based on syntax compression according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0081] Step S201, discretize the input original time series to determine a symbol representation sequence corresponding to the original time series, wherein the symbol representation sequence includes a plurality of continuous symbols, and each symbol maps a subsequence of a preset length in the original time series.

[0082] Specifically, the above step S201 includes:

[0083] Step S2011, calculating the standard deviation of each item of data in the original time series based on the standard deviation algorithm to obtain the time series after the standard deviation calculation;

[0084] Specifically, the standard deviation corresponding to each data item in the original time series is calculated by using wavelet transform standardization method, Z-score standardization method, etc., so as to determine the time series after the standard deviation calculation.

[0085] For example, the original time series is [10, 20, 15, 30, 25], and the standardized series is calculated to be [-1.41, 0, -0.71, 1.41, 0.71], where the mean of the data is 0 and the corresponding standard deviation is 1. This provides a unified data basis for the subsequent segmented average processing.

[0086] Step S2012, based on the segmented average approximation processing algorithm, divide the time series after the standard deviation is calculated into multiple subsequences of the same preset length, and determine the standard deviation mean corresponding to each subsequence;

[0087] It should be noted that the piecewise average approximation algorithm (PAA algorithm) refers to dividing the standardized time series into multiple subsequences of fixed length and calculating the average value of each segment to represent the characteristics of the data segment. The standard deviation mean refers to the arithmetic mean of each subsequence.

[0088] Specifically, the preset length w is determined as the subsequence length; then the standardized sequence is divided into equal parts according to the length w; the arithmetic mean is calculated for each subsequence; and finally a mean sequence after dimensionality reduction is obtained. For example, w is 2, and the standardized sequence [-1.41, 0, -0.71, 1.41, 0.71] is processed by the segmented approximation algorithm to obtain a mean sequence after dimensionality reduction of [-0.75, 0.35, 1.06].

[0089] Step S2013, determining the symbol mapped to each subsequence according to the standard deviation mean corresponding to each subsequence and a preset symbol representation interval;

[0090] Step S2014: Generate a symbol representation sequence corresponding to the original time series based on the symbols mapped to each subsequence.

[0091] It should be noted that the preset symbol representation interval refers to the numerical range division of mapping the standard deviation mean to discrete symbols. The standard deviation mean refers to the average value calculated for each subsequence. The mapped symbol refers to the letter or other symbol used to represent a specific numerical interval.

[0092] For example, if -0.5 to 0.5 is the first symbol representation interval corresponding to symbol a, 0.5 to 2 is the second symbol representation interval corresponding to symbol b, and -1 to -0.5 is the third symbol representation interval corresponding to symbol c, then symbol matching is performed according to the mean sequence after dimensionality reduction. For example, the mean sequence after dimensionality reduction is [-0.75, 0.35, 1.06], and the corresponding symbol representation sequence is [c, b, a].

[0093] The embodiment of the present invention can highlight the relative change characteristics of time series data through standardization processing. The piecewise average approximation processing algorithm divides the time series data into subsequences of equal length, thereby ensuring the consistency of input and output variables. The symbol mapped to each subsequence is further determined according to the standard deviation mean corresponding to each subsequence and the preset symbol representation interval, thereby achieving dimensionality reduction and eliminating the appearance of complex data, making the pattern characteristics of the original time series easier to identify. The symbols determined by each subsequence are organized according to the temporal relationship of the original data to form a complete symbolic representation sequence, which provides a basic data structure for subsequent pattern recognition and anomaly detection, retaining the main characteristics of the time series and greatly reducing the processing complexity of the time series data.

[0094] Step S202: If there are repeated continuous symbol segments in the symbol representation sequence, the continuous symbol segments are replaced with intermediate representation symbols having a syntax mapping relationship therewith, and the symbol representation sequence is compressed to obtain a compressed syntax representation sequence corresponding to the symbol representation sequence.

[0095] Specifically, the above step S202 includes:

[0096] Step S2021, traversing the symbol representation sequence to obtain repeated continuous symbol segments, and determining the positions of the repeated continuous symbol segments;

[0097] The position of the symbol segment refers to the starting and ending position of the repeated symbol combination in the sequence. Traversal refers to the process of sequentially scanning the entire symbol representation sequence.

[0098] Specifically, determine the minimum repetition length threshold; start from the beginning of the sequence and gradually increase the scanning window length; search for substrings identical to the current window content in the remaining sequence; and record all found repetitive substrings and their position information.

[0099] Step S2022, replace the position of the repeated continuous symbol segment with an intermediate representation symbol that has a syntax mapping relationship with it, and stop traversing until there is no repeated continuous symbol segment in the symbol representation sequence; wherein the traversed symbol representation sequence is used to represent the compressed syntax representation sequence.

[0100] Further, the following two methods can be adopted:

[0101] Method 1: Start replacing from the longest repeated segment, and gradually replace the shorter repeated segments until the length of the repeated segment is 0.

[0102] Method 2: Replace one repeated segment each time and update the corresponding position of the repeated segment until there is no repeated segment.

[0103] For example, refer to Figure 3 , where S->abcdbc in block a represents a symbol representation sequence, and its corresponding syntax mapping rule is: A->bc, then the final sequence obtained after S is grammatically compressed is S->aAdA, and the compression rules corresponding to blocks b, c, and d refer to block a, which will not be repeated in this embodiment, and the type of syntax mapping rules is not limited to one.

[0104] The embodiment of the present invention provides a basis for subsequent compression and replacement by discovering repeated patterns in the sequence, and applies syntax mapping rules to compress repeated symbol segments into a more concise representation, thereby retaining the structural characteristics of the sequence and reducing the amount of data, providing a more concise data representation for subsequent anomaly detection and improving the efficiency of anomaly detection.

[0105] Step S203: remove duplicate symbols from the compressed syntax representation sequence to obtain a non-repetitive compressed representation sequence.

[0106] Specifically, the above step S203 includes: traversing each symbol in the compressed syntax representation sequence; during the traversal process, determining whether the current symbol is the same as any other symbol in the compressed syntax representation sequence, if the same, retaining the current symbol and its position, and clearing any other symbol and position that is the same as the current symbol, until the last symbol is traversed, and then stopping the traversal.

[0107] The traversed compressed grammar representation sequence is used to represent a non-repetitive compressed representation sequence.

[0108] Furthermore, a data structure for storing non-repeating symbols is initialized; each symbol is traversed starting from the start position of the sequence; for each current symbol, it is checked whether it has appeared in the result set; if it has not appeared, it is added to the result set and the position information is retained; if it has appeared, the symbol is skipped and the next one is processed; a sequence containing only the first appearing symbol is obtained.

[0109] For example, refer to Figure 4 , where the compressed syntax representation sequence corresponding to S1 is aAdA, and the non-repetitive compressed representation sequence corresponding to S1 after deduplication is aAd.

[0110] The embodiment of the present invention retains the first occurrence of each symbol in the compressed syntax representation sequence through deduplication processing, eliminates duplicate information, makes the sequence more concise, and provides a non-redundant data basis for subsequent frequency statistics. Through the deduplication operation, the distribution characteristics of different symbols in the time series are more accurately reflected without the interference of complex original data, thereby improving the accuracy of anomaly detection.

[0111] Step S204, calculate the frequency of occurrence of each symbol in the non-repetitive compressed representation sequence, and determine the symbol anomaly score based on the frequency. Figure 1 Step S104 of the illustrated embodiment will not be described in detail here.

[0112] Step S205: The subsequences corresponding to the symbols whose symbol anomaly scores are higher than the preset anomaly threshold are taken as abnormal subsequences. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.

[0113] In this embodiment, a time series anomaly detection method based on syntax compression is provided, which can be used for the above-mentioned computer. Figure 5 is a flowchart of a time series anomaly detection method based on syntax compression according to an embodiment of the present invention. Figure 5 As shown, the process includes the following steps:

[0114] Step S501, discretize the input original time series to determine the symbol representation sequence corresponding to the original time series, wherein the symbol representation sequence includes a plurality of continuous symbols, each symbol mapping a subsequence of a preset length in the original time series. For details, please refer to Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0115] Step S502: If there are repeated continuous symbol segments in the symbol representation sequence, the continuous symbol segments are replaced with intermediate representation symbols that have a syntax mapping relationship with the symbol representation sequence, and the symbol representation sequence is compressed to obtain a compressed syntax representation sequence corresponding to the symbol representation sequence. Figure 1 Step S102 of the illustrated embodiment will not be described in detail here.

[0116] Step S503: remove all duplicate symbols in the compressed syntax representation sequence to obtain a non-repetitive compressed representation sequence. Figure 1 Step S103 of the illustrated embodiment will not be described in detail here.

[0117] Step S504: calculate the frequency of occurrence of each symbol in the non-repetitive compressed representation sequence, and determine the symbol abnormality score according to the frequency.

[0118] Specifically, the above step S504 includes:

[0119] Step S5041, compare each character in the non-repetitive compressed representation sequence with the compressed grammatical representation sequence to determine the number of identical characters.

[0120] Specifically, each character in the non-repeating compressed representation sequence is traversed, a match is found in the compressed syntax representation sequence, and the number of matches is counted as the number of identical characters. The code implementation logic is a nested loop traversal. Specifically, the outer loop sorts each character in the non-repeating compressed representation sequence, and the inner nested loop obtains each character to calculate the anomaly score.

[0121] Step S5042, calculating the proportion of identical characters in all characters according to the number of identical characters.

[0122] Step S5043, determining the frequency of occurrence of each symbol by proportion.

[0123] For example, the non-repetitive compressed representation sequence is aAd, the compressed syntax representation sequence is aAdA, and the statistical results are: character a appears once, character A appears twice, and character d appears once. The total length of the sequence and the proportion of the number of occurrences of each character are calculated to obtain the percentage result as the frequency, character a accounts for 1 / 4 = 0.25, character A accounts for 2 / 4 = 0.5, and character d accounts for 1 / 4 = 0.25.

[0124] Step S5044, determining a symbol anomaly score corresponding to each symbol according to the frequency of occurrence of each symbol and a preset frequency-anomaly score mapping relationship; wherein the frequency of occurrence of each symbol is negatively correlated with the symbol anomaly score.

[0125] It should be noted that the frequency-abnormality score mapping relationship refers to the corresponding rule for converting the frequency of character occurrence into the abnormality score. Negative correlation means that the higher the frequency, the lower the abnormality score.

[0126] For example, when the frequency is above 0.5, the corresponding abnormality score is 20, and when the frequency is below 0.5, the corresponding abnormality score is 80. Therefore, the above character a, character A, and character d correspond to 80, 20, and 80 points respectively.

[0127] In an embodiment of the present invention, each character in a non-repeating compressed representation sequence is compared with that in a compressed grammatical representation sequence, and the proportion of identical characters in all characters is calculated based on the number of identical characters. The regularity of pattern occurrence is reflected by character frequency, and low-frequency patterns correspond to high anomaly scores, so that anomaly detection results are more explanatory, and there is no need to pre-set a fixed anomaly length, which reduces the complexity of data processing and improves the accuracy of anomaly detection.

[0128] Step S505: The subsequences corresponding to the symbols whose symbol anomaly scores are higher than the preset anomaly threshold are taken as abnormal subsequences. Figure 1 Step S105 of the illustrated embodiment will not be described in detail here.

[0129] Step S506, obtaining a time series coordinate graph corresponding to the original time series, wherein the abscissa of the time series coordinate graph represents time and the ordinate represents the sequence value.

[0130] Specifically, determine the time range and value range of the time series; establish a coordinate system of appropriate proportions; and plot the original time series data into the coordinate system in chronological order. For the original time series, the horizontal axis ranges from 1 to 5000 to represent the time point, and the vertical axis ranges from 0 to 100 to represent the value corresponding to each time point.

[0131] Step S507: Mark the corresponding abnormal position of the abnormal subsequence on the sequence coordinate graph according to the time and sequence value.

[0132] For example, if the two subsequences [10, 20] corresponding to the character C are identified as abnormal, the area corresponding to the time point 1-2 of the subsequence [10, 20] is marked in red on the coordinate graph, or an abnormal description label is added.

[0133] The embodiment of the present invention intuitively displays the anomaly detection results in a visual manner to facilitate understanding of the abnormal patterns in the time series. The anomalies are marked by frequency, which highlights abnormal areas of different degrees, thereby improving the intuitiveness of the anomaly detection results.

[0134] In addition, in the actual application process, the time series anomaly detection method based on syntax compression provided by the embodiment of the present invention is encapsulated in the anomaly detection system in the form of an algorithm. In the system interface, the user only needs to upload the data set, then set the model-related parameters, and finally visualize the results of the subsequence anomaly detection. Finally, the method is used to perform subsequence anomaly detection, and the detection effect is as follows: Figure 6 As shown, the point with the highest value indicates that the corresponding subsequence is an abnormal subsequence.

[0135] Therefore, compared with the nearest neighbor-based method, density-based method and model-based method in the prior art, which all require setting the length of the abnormal subsequence to perform anomaly detection, the embodiment of the present invention can judge the probability of anomaly of each character by the frequency of its occurrence through grammatical compression, and can give priority to calculating its anomaly score during the anomaly detection process, thereby greatly improving the efficiency and accuracy of detection.

[0136] In this embodiment, a time series anomaly detection system based on syntax compression is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the system described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0137] This embodiment provides a time series anomaly detection system based on syntax compression, such as Figure 7 As shown, including:

[0138] The sequence symbolization module 701 is used to discretize the input original time series and determine the symbol representation sequence corresponding to the original time series, wherein the symbol representation sequence includes a plurality of continuous symbols, each symbol mapping a subsequence of a preset length in the original time series;

[0139] A syntax compression module 702 is configured to, if there are repeated continuous symbol segments in the symbol representation sequence, replace the continuous symbol segments with intermediate representation symbols having a syntax mapping relationship therewith, compress the symbol representation sequence, and obtain a compressed syntax representation sequence corresponding to the symbol representation sequence;

[0140] A symbol deduplication module 703 is used to dedupe all symbols in the compressed syntax representation sequence to obtain a non-repetitive compressed representation sequence;

[0141] A frequency analysis module 704 is used to calculate the frequency of occurrence of each symbol in the non-repetitive compressed representation sequence and determine the symbol anomaly score according to the frequency;

[0142] The anomaly detection module 705 is configured to take the subsequences corresponding to the symbols whose symbol anomaly scores are higher than a preset anomaly threshold as abnormal subsequences.

[0143] In some optional implementations, the sequence symbolization module 701 specifically includes:

[0144] A standard deviation calculation unit is used to calculate the standard deviation of each data item in the original time series based on the standard deviation algorithm to obtain the time series after the standard deviation calculation;

[0145] A segmented average processing unit, used to divide the time series after the standard deviation is calculated into a plurality of subsequences of the same preset length based on a segmented average approximate processing algorithm, and determine the standard deviation mean corresponding to each subsequence;

[0146] A symbol mapping unit, used to determine the symbol mapped to each subsequence according to the standard deviation mean corresponding to each subsequence and a preset symbol representation interval;

[0147] The symbol sequence generating unit is used to generate a symbol representation sequence corresponding to the original time series based on the symbols mapped to each subsequence.

[0148] In some optional implementations, the grammar compression module 702 specifically includes:

[0149] A continuous symbol detection unit, used for traversing the symbol representation sequence to obtain repeated continuous symbol segments and determining the positions of the repeated continuous symbol segments;

[0150] A syntax compression unit, used for replacing the position of repeated continuous symbol segments with intermediate representation symbols having a syntax mapping relationship therewith, and stopping traversal when there are no repeated continuous symbol segments in the symbol representation sequence;

[0151] The traversed symbol representation sequence is used to represent the compressed grammar representation sequence.

[0152] In some optional implementations, the symbol deduplication module 703 specifically includes:

[0153] A repeated symbol clearing unit is used to traverse each symbol in the compressed syntax representation sequence; during the traversal process, it is determined whether the current symbol is the same as any other symbol in the compressed syntax representation sequence. If they are the same, the current symbol and its position are retained, and any other symbol and position that are the same as the current symbol are cleared, until the last symbol is traversed, and then the traversal is stopped;

[0154] The traversed compressed grammar representation sequence is used to represent a non-repetitive compressed representation sequence.

[0155] In some optional implementations, the frequency analysis module 704 specifically includes:

[0156] A character comparison unit, used to compare each character in the non-repeating compressed representation sequence with the compressed grammar representation sequence to determine the number of identical characters;

[0157] A character ratio calculation unit, used to calculate the ratio of identical characters in all characters according to the number of identical characters;

[0158] A frequency statistics unit is used to determine the frequency of occurrence of each symbol by proportion;

[0159] The anomaly score calculation unit is used to determine the symbol anomaly score corresponding to each symbol according to the frequency of occurrence of each symbol and a preset frequency-anomaly score mapping relationship; wherein the frequency of occurrence of each symbol is negatively correlated with the symbol anomaly score.

[0160] In some optional implementations, the above system further includes:

[0161] A sequence coordinate graph acquisition module is used to obtain a time series coordinate graph corresponding to the original time series, wherein the abscissa of the time series coordinate graph represents time and the ordinate represents the sequence value;

[0162] The abnormal marking module is used to mark the corresponding abnormal position of the abnormal subsequence on the sequence coordinate diagram according to time and sequence value.

[0163] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0164] The time series anomaly detection system based on syntax compression in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0165] The embodiment of the present invention also provides a computer device having the above Figure 7 The time series anomaly detection system based on grammar compression is shown.

[0166] See also Figure 8 , Figure 8 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 8 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 8 A processor 10 is taken as an example.

[0167] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0168] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0169] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0170] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0171] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0172] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0173] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0174] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the defined scope.

Claims

1. A time series anomaly detection method based on syntax compression, characterized in that: The method comprises: Discretize the input original time series to determine a symbol representation sequence corresponding to the original time series, wherein the symbol representation sequence includes a plurality of continuous symbols, each symbol mapping a subsequence of a preset length in the original time series; If there are repeated continuous symbol segments in the symbol representation sequence, the continuous symbol segments are replaced with intermediate representation symbols having a syntax mapping relationship therewith, and the symbol representation sequence is compressed to obtain a compressed syntax representation sequence corresponding to the symbol representation sequence; De-duplicating all symbols in the compressed syntax representation sequence to obtain a non-repetitive compressed representation sequence; Calculating the frequency of occurrence of each symbol in the non-repetitive compressed representation sequence, and determining a symbol anomaly score according to the frequency; The subsequences corresponding to the symbols whose symbol anomaly scores are higher than the preset anomaly threshold are taken as abnormal subsequences.

2. The method according to claim 1, characterized in that The discretization processing of the input original time series to determine the symbol representation sequence corresponding to the original time series specifically includes: Calculate the standard deviation of each item of data in the original time series based on the standard deviation algorithm to obtain a time series after the standard deviation is calculated; Based on the segmented average approximation processing algorithm, the time series after the standard deviation is calculated is divided into a plurality of subsequences of the same preset length, and the standard deviation mean corresponding to each subsequence is determined; Determining the symbol mapped to each subsequence according to the standard deviation mean corresponding to each subsequence and a preset symbol representation interval; A symbol representation sequence corresponding to the original time series is generated based on the symbols mapped to each subsequence.

3. The method according to claim 2, characterized in that If there are repeated continuous symbol segments in the symbol representation sequence, the continuous symbol segments are replaced with intermediate representation symbols having a syntax mapping relationship therewith, and the symbol representation sequence is compressed to obtain a compressed syntax representation sequence corresponding to the symbol representation sequence, specifically including: Traversing the symbol representation sequence to obtain repeated continuous symbol segments, and determining positions of the repeated continuous symbol segments; The position of the repeated continuous symbol segment is replaced by an intermediate representation symbol having a syntax mapping relationship therewith, and the traversal is stopped until there is no repeated continuous symbol segment in the symbol representation sequence; The traversed symbol representation sequence is used to represent the compressed grammar representation sequence.

4. The method according to claim 3, characterized in that The step of removing duplicate symbols from the compressed syntax representation sequence to obtain a non-repetitive compressed representation sequence specifically includes: Traversing each symbol in the compressed grammar representation sequence; during the traversal process, determining whether the current symbol is the same as any other symbol in the compressed grammar representation sequence, if the current symbol is the same, retaining the current symbol and its position, and clearing any other symbol and position that is the same as the current symbol, until the last symbol is traversed, then stopping the traversal; The traversed compressed grammar representation sequence is used to represent a non-repetitive compressed representation sequence.

5. The method according to claim 4, characterized in that The calculating the frequency of occurrence of each symbol in the non-repetitive compressed representation sequence and determining the symbol anomaly score according to the frequency specifically includes: Compare the non-repeating compressed representation sequence with each character in the compressed grammatical representation sequence to determine the number of identical characters; Calculate the proportion of identical characters in all characters based on the number of identical characters; Determine the frequency of occurrence of each symbol through the proportion; According to the frequency of occurrence of each symbol and a preset frequency-abnormality score mapping relationship, a symbol abnormality score corresponding to each symbol is determined; wherein the frequency of occurrence of each symbol is negatively correlated with the symbol abnormality score.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtaining a time series coordinate graph corresponding to the original time series, wherein the abscissa of the time series coordinate graph represents time and the ordinate represents sequence value; The corresponding abnormal position of the abnormal subsequence is marked on the sequence coordinate graph according to the time and sequence value.

7. A time series anomaly detection system based on syntax compression, characterized in that: The system comprises: A sequence symbolization module, used for discretizing the input original time series and determining a symbol representation sequence corresponding to the original time series, wherein the symbol representation sequence includes a plurality of continuous symbols, each symbol mapping a subsequence of a preset length in the original time series; A syntax compression module, configured to, if there are repeated continuous symbol segments in the symbol representation sequence, replace the continuous symbol segments with intermediate representation symbols having a syntax mapping relationship therewith, compress the symbol representation sequence, and obtain a compressed syntax representation sequence corresponding to the symbol representation sequence; A symbol deduplication module, used for deduplicating all symbols in the compressed syntax representation sequence to obtain a non-repetitive compressed representation sequence; A frequency analysis module, used to calculate the frequency of occurrence of each symbol in the non-repetitive compressed representation sequence, and determine the symbol anomaly score according to the frequency; The anomaly detection module is used to take the subsequence corresponding to the symbol whose symbol anomaly score is higher than the preset anomaly threshold as an abnormal subsequence.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the time series anomaly detection method based on grammatical compression according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the time series anomaly detection method based on grammar compression according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are used to enable a computer to execute the time series anomaly detection method based on grammar compression according to any one of claims 1 to 6.

Images