Multi-scale graph structure learning-based multivariate time series anomaly detection method and system

Abstract

The invention discloses a multivariate time sequence anomaly detection method based on multi-scale graph structure learning, and the method comprises the steps: firstly, decomposing a time sequence into time blocks with different sizes according to the size of a time scale; secondly, by introducing a shared embedding vector and a node embedding vector, learning spatial correlation between time sequences under different time scales by using a graph structure learning formula to obtain a multi-scale graph structure, and based on the learned graph structure, extracting spatial features by using a graph neural network; secondly, respectively extracting global and local attention representations under different time scales by utilizing global and local attention mechanisms; then, aggregating global and local attention representations under different time scales; and finally, based on the thought that the correlation difference between the normal points in the global and local view angles is small and the difference between the abnormal points in the global and local view angles is large, the anomaly is identified by comparing the global attention representation and the local attention representation.

Description

Technical Field

[0001] The present invention belongs to the technical field of anomaly detection in multivariate time series data, and more specifically, relates to a method and system for multivariate time series anomaly detection based on multi-scale graph structure learning. Background Art

[0002] With the rapid development of big data technology, the application of multivariate time series data is becoming more and more extensive, especially in multiple fields such as cloud computing, Internet of Things, finance, and intelligent manufacturing. Anomaly detection of multivariate time series data is one of the important tasks to ensure the stability and performance of the system.

[0003] Currently, the mainstream methods for multivariate time series anomaly detection mainly tend to the field of deep learning. The first method is based on the encoder architecture of Transformer. This method mainly uses Transformer to extract the temporal correlation within the multivariate time series, then predicts the future values of the multivariate time series. Finally, anomalies are detected by judging whether the difference between the predicted value and the true value exceeds the anomaly threshold. The second method is based on the method of VAE (Variational Autoencoder). This method first uses Transformer to extract the temporal correlation of the multivariate time series, then uses VAE to construct a new multivariate time series. Finally, anomalies are detected by comparing whether the distribution error between the original sequence and the true sequence exceeds the anomaly threshold. The third method is based on the method of spatio-temporal graph neural network. This method uses graph neural network to extract the spatio-temporal features of the multivariate time series and predicts a new multivariate time series. Anomalies are detected by calculating whether the difference between the predicted sequence and the true sequence exceeds the anomaly threshold.

[0004] However, the above several existing methods for multivariate time series anomaly detection all have some non-negligible defects:

[0005] First, the method based on the encoder architecture of Transformer only considers the global temporal correlation of the multivariate time series while ignoring the local temporal correlation of the multivariate time series. Therefore, the extraction of the temporal features of the multivariate time series is not perfect enough.

[0006] Second, the method based on VAE only considers the problem of the temporal correlation of the multivariate time series while ignoring the spatial correlation of the multivariate time series. Therefore, the detailed features of the multivariate time series cannot be extracted.

[0007] Third, the method based on the spatio-temporal graph neural network only considers the spatio-temporal features of the multivariate time series under a single time scale while ignoring the multi-scale information. Therefore, the accuracy of anomaly detection is not high. Summary of the Invention

[0008] In view of the above defects or improvement requirements of the prior art, the present invention provides a multi-variate time series anomaly detection method and system based on multi-scale graph structure learning, aiming to solve the technical problems that the existing methods based on the encoder architecture of transformers only consider the global time correlation of multi-variate time series while ignoring the local time correlation of multi-variate time series, resulting in imperfect extraction of the time features of multi-variate time series; the existing methods based on VAE only consider the time correlation of multi-variate time series while ignoring the spatial correlation of multi-variate time series, so they cannot extract the detailed features of multi-variate time series; and the existing methods based on spatio-temporal graph neural networks only consider the spatio-temporal features of multi-variate time series at a single time scale while ignoring multi-scale information, resulting in low accuracy of anomaly detection.

[0009] To achieve the above object, according to one aspect of the present invention, a multi-variate time series anomaly detection method based on multi-scale graph structure learning is provided, including the following steps:

[0010] (1) Obtain a multi-variate time series, and preprocess the obtained multi-variate time series to obtain a preprocessed multi-variate time series.

[0011] (2) Input the preprocessed multi-variate time series obtained in step (1) into a pre-trained anomaly detection model to obtain a corresponding anomaly detection result.

[0012] Preferably, in step (1), the obtained multi-variate time series is preprocessed by a normalization method, and the normalization formula is as follows.

[0013]

[0014] Where x represents each time in the multi-variate time series, represents the time after preprocessing, and min(x) and max(x) represent the minimum time and the maximum time in the multi-variate time series.

[0015] Preferably, the anomaly detection model includes an adaptive multi-scale module, a global and local attention representation contrast module, and an anomaly detection module.

[0016] The structure of the adaptive multi-scale module is:

[0017] The first layer is a multi-scale segmentation layer of the multi-variate time series, and its input is the preprocessed multi-variate time series Its dimension is T×d, where T is the length of the multivariate time series and d is the dimension of the multivariate time series. This multivariate time series multi-scale segmentation layer divides the input multivariate time series into P1 time blocks respectively according to the segmentation scale S1. Its output is the first time block set, whose dimension is P1×S1×d, and divides the input multivariate time series into P2 time blocks respectively according to the segmentation scale S2. Its output is the second time block set, whose dimension is P2×S2×d, where P1 = T / S1 and P2 = T / S2.

[0018] The second layer is the multi-scale graph structure learning layer, which generates the first adjacency matrix A respectively according to the first time block set and the second time block set output by the first layer S1 and the second adjacency matrix A S2 and outputs it, whose dimension is d×d.

[0019] The third layer is the spatial feature capture layer of the first time block set. Its input is the first time block set output by the first layer and the first adjacency matrix A output by the second layer S1 , and this layer identifies the spatial correlation between the first time blocks output by the first layer according to the output A S1 of the second layer, and uses the graph attention neural network to perform feature weighted aggregation on the first time blocks output by the first layer based on the obtained spatial correlation, so as to output a feature representation with a dimension of P1×3×d.

[0020] The fourth layer is the global time feature capture layer. Its input is the feature representation with a dimension of P1×3×d output by the third layer. This global time feature capture layer uses the global attention mechanism to obtain a global attention representation with a dimension of T×256 from this feature representation and outputs it.

[0021] The fifth layer is the local time feature capture layer. Its input is the feature representation with a dimension of P1×3×d output by the third layer. This local time feature capture layer uses the local attention mechanism to obtain a local attention representation with a dimension of T×256 from the feature representation output by the third layer and outputs it.

[0022] The sixth layer is the spatial feature capture layer of the second time block set. Its input is the second time block set output by the first layer and the second adjacency matrix A output by the second layer S2 , and this layer identifies the spatial correlation of the second time block set output by the first layer according to the second adjacency matrix A S2 output by the second layer, and based on the obtained spatial correlation, uses the graph attention neural network to perform feature weighted aggregation on the second time blocks output by the first layer, so as to output a feature representation with a dimension of P2×5×d.

[0023] The seventh layer is the global time feature capture layer, whose input is the feature representation with a dimension of P2×5×d output by the sixth layer. This global time feature capture layer uses the global attention mechanism to obtain a global attention representation with a dimension of T×256 from this feature representation and outputs it.

[0024] The eighth layer is the local time feature capture layer, whose input is the feature representation with a dimension of P2×5×d output by the sixth layer. This local time feature capture layer uses the local attention mechanism to obtain a local attention feature with a dimension of T×256 from this feature representation and outputs it.

[0025] The ninth layer is the global multi-scale representation aggregation layer, whose input is the global attention representation with a dimension of T×256 output by the fourth layer and the global attention representation with a dimension of T×256 output by the seventh layer. This global multi-scale representation aggregation layer aggregates the outputs of the fourth layer and the seventh layer to obtain the global attention representation and outputs it.

[0026] The tenth layer is the local multi-scale representation aggregation layer, whose input is the local attention representation with a dimension of T×256 output by the fifth layer and the local attention representation with a dimension of T×256 output by the eighth layer. This local multi-scale representation aggregation layer aggregates the outputs of the fifth layer and the eighth layer to obtain the local attention representation and outputs it.

[0027] Preferably, the multi-scale graph structure learning layer generates corresponding adjacency matrices for the first time block set and the second time block set output by the first layer and outputs them. Specifically, first, the node embedding vectors the first-scale embedding and the second-scale embedding are obtained, where N is the dimension of the multivariate time series and d e is the embedding dimension;

[0028] Then, the shared embedding vector N nodes is respectively dot-multiplied with the first-scale embedding vector and the second-scale embedding vector to respectively obtain the first feature embedding vector N S1 and the second feature embedding vector N S2 .

[0029]

[0030] After that, based on the first feature embedding vector N S1 and the second feature embedding vector N S2 the first adjacency matrix A corresponding to the first time block set S1 and the second adjacency matrix A corresponding to the second time block set S2, where the first adjacency matrix A is obtained S1 Specifically, the following formula is adopted:

[0031]

[0032]

[0033] where is a trainable parameter matrix;

[0034] The formula for obtaining the second adjacency matrix A S2 is basically the same as the above formula, except that S1 in the formula is replaced by S2.

[0035] Preferably, the process of the global time feature capture layer obtaining the global attention representation is specifically as follows. First, an embedding operation is performed on the feature representation with dimensions P1×3×d output by the third layer using the segmentation scale S1 to obtain the embedded sequence Then, the global time features are obtained from the embedded sequence using the multi-head attention mechanism. Finally, the obtained global time features are upsampled to obtain the global attention representation with dimensions T×256.

[0036] The process of the local time feature capture layer obtaining the local attention representation is specifically as follows. First, an embedding operation is performed on the feature representation with dimensions P1×3×d output by the third layer using the segmentation scale S1 to obtain the embedded sequence Then, the local time features are obtained from the embedded sequence using the multi-head attention mechanism. Finally, as Figure 2 shown, the obtained local time features are upsampled to obtain the local attention representation with dimensions T×256.

[0037] Preferably, for the global and local attention representation comparison module, its input is the global attention representation with dimensions T×256 output by the ninth layer and the local attention representation with dimensions T×256 output by the tenth layer in the adaptive multi-scale module. The global and local attention representation comparison module compares the global attention representation and the local attention representation to obtain an anomaly score with dimensions T×1 and outputs it;

[0038] The calculation formula for the anomaly score AnomalyScore is as follows:

[0039] AnomalyScore(X)=∑KL(X[[ID=##]] inter ,Stopgrad(X intra ))+KL(X intra ,Stopgrad(X inter ))

[0040] Among them, X represents the preprocessed multivariate time series which is the input of the first layer in the adaptive multi-scale module, KL() represents the KL divergence, and X inter represents the global attention representation which is the output of the 9th layer in the adaptive multi-scale module, and X intra represents the local attention representation which is the output of the 10th layer in the adaptive multi-scale module, and Stopgrad() represents stopping gradient descent.

[0041] Preferably, the input of the anomaly detection module is the anomaly score with the dimension of T×1 output by the global and local attention representation contrast module. The anomaly detection module identifies whether there is an anomaly in the preprocessed multivariate time series which is the input of the first layer in the adaptive multi-scale module by judging whether the anomaly score exceeds the anomaly threshold;

[0042] Among them, if the anomaly score exceeds the anomaly threshold, it is considered that there is an anomaly in the time series.

[0043]

[0044] Preferably, the anomaly detection model is obtained through the following steps:

[0045] (2-1) Obtain the open-source datasets SMD, MSL, SWaT, and PSM, merge all the datasets, and divide the merged dataset into a training set and a test set according to the ratio of 8:2.

[0046] (2-2) Preprocess the training set obtained in step (2-1) to obtain the preprocessed training set.

[0047] (2-3) For each sample in the preprocessed training set obtained in step (2-2), input the sample into the first layer of the adaptive multi-scale module to obtain the corresponding first set of time blocks of the sample; its dimension is P1×3×d, where P1 is the total number of time blocks in the first set of time blocks, and d is the dimension of the multivariate time series.

[0048] (2-4) For each sample in the preprocessed training set obtained in step (z-2), input the sample into the second layer of the adaptive multi-scale module to obtain the corresponding first adjacency matrix of the sample, and its dimension is d×d.

[0049] (2-5) For each sample in the preprocessed training set obtained in step (2-2), input the corresponding first set of time blocks of the sample obtained in step (2-3) and the corresponding first adjacency matrix of the sample obtained in step (2-4) into the third layer of the adaptive multi-scale module to extract the corresponding feature representation of the sample, and its dimension is P1×3×d.

[0050] (2-6) For each sample in the training set preprocessed in step (2-2), input the feature representation corresponding to this sample obtained in step (2-5) into the fourth layer of the adaptive multi-scale module to extract the global attention representation corresponding to this sample, whose dimension is T×256.

[0051] (2-7) For each sample in the training set preprocessed in step (2-2), input the feature representation corresponding to this sample obtained in step (3) into the fifth layer of the adaptive multi-scale module to extract the local attention representation corresponding to this sample, whose dimension is T×256.

[0052] (2-8) For each sample in the training set preprocessed in step (2-2), input this sample into the first layer of the adaptive multi-scale module to obtain the second set of time blocks, whose output dimension is P2×5×d, where P2 is the size of the time block.

[0053] (2-9) For each sample in the training set preprocessed in step (2-2), input this sample into the second layer of the adaptive multi-scale module to obtain the second adjacency matrix corresponding to this sample, whose dimension is d×d.

[0054] (2-10) For each sample in the training set preprocessed in step (2-2), input the second set of time blocks corresponding to this sample obtained in step (2-8) and the second adjacency matrix corresponding to this sample obtained in step (2-9) into the sixth layer of the adaptive multi-scale module to extract the feature representation corresponding to this sample, whose dimension is P2×3×d.

[0055] (2-11) For each sample in the training set preprocessed in step (2-2), input the feature representation corresponding to this sample obtained in step (2-10) into the seventh layer of the adaptive multi-scale module to extract the global attention representation corresponding to this sample, whose dimension is T×256.

[0056] (2-12) For each sample in the training set preprocessed in step (2-2), input the feature representation corresponding to this sample obtained in step (2-10) into the eighth layer of the adaptive multi-scale module to extract the local attention representation corresponding to this sample, whose dimension is T×256.

[0057] (2-13) For each sample in the training set preprocessed in step (2-2), input the global attention representation corresponding to this sample obtained in step (2-6) and the global attention representation corresponding to this sample obtained in step (2-11) into the ninth layer of the adaptive multi-scale module for aggregation to obtain the global attention representation, whose output dimension is T×256.

[0058] (2-14) For each sample in the training set preprocessed in step (2-2), the local attention representation corresponding to this sample obtained in step (2-7) and the local attention representation corresponding to this sample obtained in step (2-12) are input into the 10th layer of the adaptive multi-scale module for aggregation to obtain the local attention feature corresponding to this sample, and its output dimension is T×256.

[0059] (2-15) For each sample in the training set preprocessed in step (2-2), according to the global attention representation corresponding to this sample obtained in step (2-13) and the local attention representation corresponding to this sample obtained in step (2-14), and using the loss function based on KL divergence to calculate the global representation loss L corresponding to this sample inter ;

[0060] (2-16) For each sample in the training set preprocessed in step (2-2), according to the global attention representation corresponding to this sample obtained in step (2-13) and the local attention representation corresponding to this sample obtained in step (2-14), and using the loss function based on KL divergence to calculate the local representation loss L corresponding to this sample intra :

[0061] (2-17) For each sample in the training set preprocessed in step (2-2), obtain the loss function corresponding to this sample according to the global representation loss corresponding to this sample obtained in step (2-15) and the local representation loss corresponding to this sample obtained in step (2-16). According to this loss function and using the gradient descent method to iteratively train the anomaly detection model until the anomaly detection model reaches the preset number of iterations, thereby obtaining a preliminarily trained anomaly detection model.

[0062] (2-18) Use the test set obtained in step (2-1) to test the anomaly detection model preliminarily trained in step (2-17) until the obtained detection accuracy reaches the optimum, thereby obtaining a finally trained anomaly detection model.

[0063] Preferably, the global representation loss L inter is equal to;

[0064] L inter =∑KL(X inter , Stopgrad(X intra )) + KL(Stopgrad(X intra ), X inter )

[0065] where Stopgrad() represents stopping gradient descent, and X interdenotes the global attention representation corresponding to this sample obtained in step (2-13), X intra denotes the local attention representation corresponding to this sample obtained in step (2-14), and KL() represents the KL loss function;

[0066] Local representation loss L intra is equal to:

[0067] L intra = ∑KL(X intra , Stopgrad(X inter )) + KL(Stopgrad(X inter ), X intra )

[0068] The loss function is:

[0069]

[0070] where len() represents the length calculation function.

[0071] According to another aspect of the present invention, a multivariate time series anomaly detection system based on multi-scale graph structure learning is provided, including:

[0072] The first module is used to obtain a multivariate time series, and preprocess the obtained multivariate time series to obtain a preprocessed multivariate time series.

[0073] The second module is used to input the preprocessed multivariate time series obtained by the first module into a pre-trained anomaly detection model to obtain corresponding anomaly detection results.

[0074] Generally speaking, compared with the prior art by the above technical solutions conceived by the present invention, the following beneficial effects can be achieved:

[0075] (1) Since the present invention adopts steps (2-7) and (2-12), it extracts the local attention representation of the multivariate time series, thus solving the technical problem that the existing transformer-based method is not perfect enough in time feature extraction due to only considering global time correlation;

[0076] (2) Since the present invention adopts steps (2-5) and (2-10), it extracts the spatial features between multivariate time series, thus solving the technical problem that the existing VAE-based method cannot extract detailed features of multivariate time series due to only considering time features;

[0077] (3) Since the present invention adopts steps (2-4) to (2-5), and steps (2-9) to (2-10), it extracts the spatial correlation between multiple time-scale multivariate time series, thus solving the technical problem that the accuracy of anomaly detection is not high due to the fact that the spatio-temporal graph neural network method only considers the spatio-temporal features of multivariate time series under a single time scale. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 is a framework diagram of the multivariate time series anomaly detection method based on multi-scale graph structure learning of the present invention;

[0079] Figure 2 is a schematic diagram of the upsampling process of the present invention;

[0080] Figure 3 is a flow chart of the multivariate time series anomaly detection method based on multi-scale graph structure learning of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0081] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0082] The present invention proposes a multivariate time series anomaly detection method based on multi-scale graph structure learning, which simultaneously considers the temporal correlation and spatial correlation of multivariate time series at different time scales; first, this method decomposes the time series into time blocks of different sizes according to the size of the time scale; secondly, by introducing a shared embedding vector and a node embedding vector, the spatial correlation between time series at different time scales is learned using the graph structure learning formula to obtain a multi-scale graph structure, and based on the learned graph structure, a graph neural network is used to extract spatial features; then, global and local attention mechanisms are used to extract the global and local attention representations at different time scales respectively; after that, the global and local attention representations at different time scales are aggregated; finally, based on the idea that the correlation difference between normal points in the global and local perspectives is small, and the difference between abnormal points in the global and local perspectives is large, anomalies are identified by comparing the global and local attention representations.

[0083] As Figure 1 and Figure 3 shown, the present invention provides a multivariate time series anomaly detection method based on multi-scale graph structure learning, including the following steps:

[0084] (1) Obtain a multivariate time series, and preprocess the obtained multivariate time series to obtain a preprocessed multivariate time series;

[0085] Specifically, in this step, the obtained multivariate time series is preprocessed by using a normalization method, that is, each time in the multivariate time series is linearly transformed according to its minimum and maximum values and compressed into a specified range (usually 0 to 1). The normalization formula is as follows.

[0086]

[0087] Where x represents each time in the multivariate time series, represents this time after preprocessing, and min(x) and max(x) represent the minimum time and the maximum time in the multivariate time series.

[0088] (2) Input the preprocessed multivariate time series obtained in step (1) into a pre-trained anomaly detection model to obtain corresponding anomaly detection results.

[0089] Such as Figure 1 shown, the anomaly detection model of the present invention includes an adaptive multi-scale module, a global and local attention feature comparison module, and an anomaly detection module.

[0090] The structure of the adaptive multi-scale module is:

[0091] The first layer is a multivariate time series multi-scale segmentation layer, and its input is the preprocessed multivariate time series Its dimension is T×d, where T is the length of the multivariate time series and d is the dimension of the multivariate time series. The multivariate time series multi-scale segmentation layer divides the input multivariate time series into P1 time blocks respectively according to the segmentation scale S1 (where the value of S1 is 3), and its output is the first time block set, and its dimension is P1×S1×d, and divides the input multivariate time series into P2 time blocks respectively according to the segmentation scale S2 (where the value of S2 is 5), and its output is the second time block set, and its dimension is P2×S2×d, where P1 = T / S1 and P2 = T / S2.

[0092] The second layer is a multi-scale graph structure learning layer, which respectively generates a first adjacency matrix A S1 and a second adjacency matrix A S2 and outputs them, and its dimension is d×d.

[0093] Specifically, in the process that the multi-scale graph structure learning layer respectively generates corresponding adjacency matrices for the first time block set and the second time block set output by the first layer and outputs them, first, obtain the node embedding vector First-scale embedding and second-scale embedding where N is the dimension of the multivariate time series, and d e is the embedding dimension;

[0094] Then, perform dot product operations on the shared embedding vector N nodes with the first-scale embedding vector and the second-scale embedding vector respectively to obtain the first feature embedding vector N S1 and the second feature embedding vector N S2 .

[0095]

[0096] After that, based on the first feature embedding vector N S1 and the second feature embedding vector N S2 obtain the first adjacency matrix A corresponding to the first set of time blocks S1 and the second adjacency matrix A corresponding to the second set of time blocks S2 , where obtaining the first adjacency matrix A S1 specifically uses the following formula:

[0097]

[0098]

[0099] where, is the trainable parameter matrix;

[0100] The formula for obtaining the second adjacency matrix A S2 is basically the same as the above formula, except that S1 in the formula is replaced by S2.

[0101] The third layer is the spatial feature capture layer for the first set of time blocks. Its input is the first set of time blocks output by the first layer and the first adjacency matrix A output by the second layer S1 . This layer identifies the spatial correlations between the first time blocks output by the first layer based on the output A of the second layer S1 and performs feature weighted aggregation on the first time blocks output by the first layer using a graph attention neural network based on the obtained spatial correlations to output a feature representation with a dimension of P1×3×d.

[0102] The fourth layer is the global time feature capture layer. Its input is the feature representation with a dimension of P1×3×d output by the third layer. This global time feature capture layer uses a global attention mechanism to obtain a global attention representation with a dimension of T×256 from this feature representation and outputs it.

[0103] Specifically, first, an embedding operation is performed on the feature representation with dimensions P1×3×d output by the third layer using the segmentation scale S1 to obtain the embedded sequence Then, the global time features are obtained from the embedded sequence using the multi-head attention mechanism. Finally, as Figure 2 shown, upsampling processing is performed on the obtained global time features to obtain the global attention representation with dimensions T×256.

[0104] The fifth layer is the local time feature capture layer. Its input is the feature representation with dimensions P1×3×d output by the third layer. This local time feature capture layer uses the local attention mechanism to obtain the local attention representation with dimensions T×256 from the feature representation output by the third layer and outputs it.

[0105] Specifically, first, an embedding operation is performed on the feature representation with dimensions P1×3×d output by the third layer using the segmentation scale S1 to obtain the embedded sequence Then, the local time features are obtained from the embedded sequence using the multi-head attention mechanism. Finally, as Figure 2 shown, upsampling processing is performed on the obtained local time features to obtain the local attention representation with dimensions T×256.

[0106] The sixth layer is the spatial feature capture layer of the second time block set. Its input is the second time block set output by the first layer and the second adjacency matrix A output by the second layer S2 , and this layer identifies the spatial correlation of the second time block set output by the first layer according to the second adjacency matrix A output by the second layer S2 , and based on the obtained spatial correlation, and uses the graph attention neural network to perform feature weighted aggregation on the second time block output by the first layer to output the feature representation with dimensions P2×5×d.

[0107] The seventh layer is the global time feature capture layer. Its input is the feature representation with dimensions P2×5×d output by the sixth layer. This global time feature capture layer uses the global attention mechanism to obtain the global attention representation with dimensions T×256 from this feature representation and outputs it.

[0108] The eighth layer is the local time feature capture layer. Its input is the feature representation with dimensions P2×5×d output by the sixth layer. This local time feature capture layer uses the local attention mechanism to obtain the local attention feature with dimensions T×256 from this feature representation and outputs it.

[0109] The 9th layer is the global multi-scale representation aggregation layer. Its inputs are the global attention representations with dimensions of T×256 output by the 4th layer and the global attention representations with dimensions of T×256 output by the 7th layer. This global multi-scale representation aggregation layer aggregates the outputs of the 4th layer and the 7th layer to obtain the global attention representation and outputs it.

[0110] The advantage of this step is that by processing data at different scales, it can effectively capture detailed information at different levels, thereby improving the model's ability to recognize complex features, enhancing the model's generalization ability, and reducing information loss that may be caused by single-scale methods.

[0111] The 10th layer is the local multi-scale representation aggregation layer. Its inputs are the local attention representations with dimensions of T×256 output by the 5th layer and the local attention representations with dimensions of T×256 output by the 8th layer. This local multi-scale representation aggregation layer aggregates the outputs of the 5th layer and the 8th layer to obtain the local attention representation and outputs it.

[0112] The second module is the global and local attention representation comparison module. Its inputs are the global attention representations with dimensions of T×256 output by the 9th layer in the adaptive multi-scale module and the local attention representations with dimensions of T×256 output by the 10th layer. This module compares the global attention representation and the local attention representation to obtain an anomaly score with dimensions of T×1 and outputs it.

[0113] Specifically, since normal points have strong correlations from different perspectives and the correlations of abnormal points vary greatly from different perspectives, the representation differences of normal points from different perspectives are smaller than those of abnormal points. The present invention obtains the anomaly score AnomalyScore by comparing the global attention representation and the local attention representation from different perspectives. The calculation formula is as follows: AnomalyScore(X) = ∑KL(X inter , Stopgrad(X intra )) + KL(X intra , Stopgrad(X inter ))

[0114] where X represents the preprocessed multivariate time series input to the first layer in the adaptive multi-scale module, KL() represents the KL divergence, X inter represents the global attention representation output by the 9th layer in the adaptive multi-scale module, X intra represents the local attention representation output by the 10th layer in the adaptive multi-scale module, and Stopgrad() represents stopping gradient descent.

[0115] The input of the anomaly detection module is the anomaly score of dimension T×1 output by the global and local attention representation contrast module. This module identifies whether the preprocessed multivariate time series of the first-layer input in the adaptive multi-scale module is abnormal by judging whether the anomaly score exceeds the anomaly threshold.

[0116] Specifically, the present invention uses an anomaly threshold τ (with a value of 0.7) to determine whether the multivariate time series of the first-layer input in the adaptive multi-scale module is abnormal. If the anomaly score exceeds the anomaly threshold, it is considered that the time series is abnormal.

[0117]

[0118] The anomaly detection model of the present invention is obtained through the following steps of training:

[0119] (2-1) Obtain the open-source datasets SMD, MSL, SWaT, and PSM, merge all the datasets, and divide the merged dataset into a training set and a test set according to a ratio of 8:2.

[0120] The advantages of this step are as follows. First, different datasets contain different features and data patterns. By training on multiple datasets simultaneously, the model can learn more types of data distributions, thereby improving the generalization ability of the model. Second, when the model is trained on multiple datasets, it can adapt to different data features and changes. This means that even when encountering unseen datasets or data with large variations, the model can handle them better.

[0121] (2-2) Preprocess the training set obtained in step (2-1) to obtain the preprocessed training set.

[0122] Specifically, the data preprocessing process in this step is exactly the same as step (1) above and will not be elaborated here.

[0123] (2-3) For each sample in the training set preprocessed in step (2-2), input the sample into the first layer of the adaptive multi-scale module to obtain the corresponding first set of time blocks for the sample; its dimension is P1×3×d, where P1 is the total number of time blocks in the first set of time blocks, and d is the dimension of the multivariate time series.

[0124] (2-4) For each sample in the training set preprocessed in step (2-2), input the sample into the second layer of the adaptive multi-scale module to obtain the corresponding first adjacency matrix for the sample, and its dimension is d×d.

[0125] (2-5) For each sample in the training set preprocessed in step (2-2), input the first time block set corresponding to this sample obtained in step (2-3) and the first adjacency matrix corresponding to this sample obtained in step (2-4) into the third layer of the adaptive multi-scale module to extract the feature representation corresponding to this sample, whose dimension is P1×3×d.

[0126] (2-6) For each sample in the training set preprocessed in step (2-2), input the feature representation corresponding to this sample obtained in step (2-5) into the fourth layer of the adaptive multi-scale module to extract the global attention representation corresponding to this sample, whose dimension is T×256.

[0127] (2-7) For each sample in the training set preprocessed in step (2-2), input the feature representation corresponding to this sample obtained in step (3) into the fifth layer of the adaptive multi-scale module to extract the local attention representation corresponding to this sample, whose dimension is T×256.

[0128] (2-8) For each sample in the training set preprocessed in step (2-2), input this sample into the first layer of the adaptive multi-scale module to obtain a second time block set, whose output dimension is P2×5×d, where P2 is the size of the time block.

[0129] (2-9) For each sample in the training set preprocessed in step (2-2), input this sample into the second layer of the adaptive multi-scale module to obtain the second adjacency matrix corresponding to this sample, whose dimension is d×d.

[0130] The advantages of steps (2-4) and (2-9) are that the spatial correlation between multivariate time series is captured by the multi-scale graph structure learning module and modeled into a multi-scale graph structure, so that the multi-scale spatial dependence relationship between multivariate time series can be extracted, and the difference between the global and local attention representations of multivariate time series can be enlarged, thereby improving the anomaly detection accuracy.

[0131] (2-10) For each sample in the training set preprocessed in step (2-2), input the second time block set corresponding to this sample obtained in step (2-8) and the second adjacency matrix corresponding to this sample obtained in step (2-9) into the sixth layer of the adaptive multi-scale module to extract the feature representation corresponding to this sample, whose dimension is P2×3×d.

[0132] (2-11) For each sample in the training set preprocessed in step (2-2), input the feature representation corresponding to this sample obtained in step (2-10) into the seventh layer of the adaptive multi-scale module to extract the global attention representation corresponding to this sample, whose dimension is T×256.

[0133] (2-12) For each sample in the training set preprocessed in step (2-2), the feature representation corresponding to this sample obtained in step (2-10) is input into the 8th layer of the adaptive multi-scale module to extract the local attention representation corresponding to this sample, with a dimension of T×256.

[0134] (2-13) For each sample in the training set preprocessed in step (2-2), the global attention representation corresponding to this sample obtained in step (2-6) and the global attention representation corresponding to this sample obtained in step (2-11) are input into the 9th layer of the adaptive multi-scale module for aggregation to obtain the global attention representation, with an output dimension of T×256.

[0135] (2-14) For each sample in the training set preprocessed in step (2-2), the local attention representation corresponding to this sample obtained in step (2-7) and the local attention representation corresponding to this sample obtained in step (2-12) are input into the 10th layer of the adaptive multi-scale module for aggregation to obtain the local attention feature corresponding to this sample, with an output dimension of T×256.

[0136] The advantages of steps (2-13) and (2-14) are that by processing data at different scales, it is possible to effectively capture detailed information at different levels, thereby improving the model's ability to recognize complex features, enhancing the model's generalization ability, and reducing information loss that may be caused by single-scale methods.

[0137] (2-15) For each sample in the training set preprocessed in step (2-2), based on the global attention representation corresponding to this sample obtained in step (2-13) and the local attention representation corresponding to this sample obtained in step (2-14), a loss function based on KL divergence is used to calculate the global representation loss corresponding to this sample;

[0138] L inter =∑KL(X inter , Stopgrad(X intra )) + KL(Stopgrad(X intra ), X inter )

[0139] where Stopgrad() represents stopping gradient descent, X inter represents the global attention representation corresponding to this sample obtained in step (2-13), X intra represents the local attention representation corresponding to this sample obtained in step (2-14), and KL() represents the KL loss function.

[0140] (2-16) For each sample in the training set preprocessed in step (2-2), according to the global attention representation corresponding to this sample obtained in step (2-13) and the local attention representation corresponding to this sample obtained in step (2-14), and using a loss function based on KL divergence to calculate the local representation loss corresponding to this sample:

[0141] L intra = ∑KL(X intra , Stopgrad(X inter )) + KL(Stopgrad(X inter ), X intra )

[0142] (2-17) For each sample in the training set preprocessed in step (2-2), obtain the loss function corresponding to this sample according to the global representation loss corresponding to this sample obtained in step (2-15) and the local representation loss corresponding to this sample obtained in step (2-16). According to this loss function, and using the gradient descent method to iteratively train the anomaly detection model until the anomaly detection model reaches the preset number of iterations (in the present invention, it is 100 times), so as to obtain a preliminarily trained anomaly detection model.

[0143] Specifically, the loss function in this step is:

[0144]

[0145] ]]where len() represents the length calculation function.

[0146] (2-18) Use the test set obtained in step (2-1) to test the anomaly detection model preliminarily trained in step (2-17) until the obtained detection accuracy reaches the optimal value, so as to obtain a finally trained anomaly detection model.

[0147] Experimental results

[0148] The experiments of the present invention were implemented using CUDA 11.7 and Pytorch 2.0.0. All experiments of the present invention were implemented on an NVIDIA GeForce RTX 3090 24GB GPU. For the dataset, the present invention set the sliding window of the multivariate time series to 125. In MAGAT, the number of layers of the GAT module is 2. For Transformer, the model contains three encoding layers, where the hidden layer dimension d model is 256, the number of heads of the multi-head attention is 4, and the hidden dimension of the fully connected layer is 128. The time resolution size is set to {5, 7}. In the training part, the learning rate of the present invention is set to 10 ―4, the anomaly threshold is set to 0.7, and all datasets are trained for 3 rounds.

[0149] The proposed model is experimented on multiple datasets (SMD, MSL, SWaT, and PSM) and compared with other methods to demonstrate its effectiveness. The results are shown in Table 1. It can be seen that the proposed model achieves the best results under various evaluation metrics compared with other methods: OmniAnomaly does not consider the correlation between sequences; GDN, MTAD-GAT, and InterFusion consider the intra- and inter-sequence correlations and achieve a certain improvement in accuracy. However, these methods only model the correlation from a local perspective; AnomalyTrans and GLAD propose the global and local correlation differences to identify anomalies because the correlation differences between normal and abnormal points are different under different perspectives; Dcdetector further models the global and local correlations of multivariate time series and detects anomalies by comparing the global and local attention representations; compared with previous methods, the detection accuracies of AnomalyTrans, GLAD, and Dcdetector have been significantly improved. However, these three methods do not consider the correlation between modeled sequences at different time resolutions. In contrast, the proposed model considers both the intra-sequence correlation and the inter-sequence correlation at different time resolutions, thus achieving better results.

[0150] Table 1 Overall results on real-world multivariate datasets (P, R, and F1 are precision, recall, and F1-score, respectively, and the best results are shown in bold)

[0151]

[0152] It is easy for those skilled in the art to understand that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multivariate time series anomaly detection method based on multi-scale graph structure learning, characterized in that It includes the following steps: (1) Obtain a multivariate time series, and preprocess the obtained multivariate time series to obtain a preprocessed multivariate time series. (2) Input the preprocessed multivariate time series obtained in step (1) into a pre-trained anomaly detection model to obtain corresponding anomaly detection results.

2. The multivariate time series anomaly detection method based on multi-scale graph structure learning according to claim 1, wherein, Step (1) preprocesses the obtained multivariate time series by using a normalization method, and the normalization formula is as follows. where x represents each time in the multivariate time series, represents the time after preprocessing, and min(x) and max(x) represent the minimum time and the maximum time in the multivariate time series.

3. The multivariate time series anomaly detection method based on multi-scale graph structure learning according to claim 1 or 2, characterized in that The anomaly detection model includes an adaptive multi-scale module, a global and local attention representation contrast module, and an anomaly detection module. The structure of the adaptive multi-scale module is: The first layer is a multivariate time series multi-scale segmentation layer, and its input is the preprocessed multivariate time series The dimension is T×d, where T is the length of the multivariate time series and d is the dimension of the multivariate time series. This multivariate time series multi-scale segmentation layer divides the input multivariate time series into P1 time blocks respectively according to the segmentation scale S1. Its output is the first time block set, and its dimension is P1×S1×d. It also divides the input multivariate time series into P2 time blocks respectively according to the segmentation scale S2. Its output is the second time block set, and its dimension is P2×S2×d, where P1 = T / S1 and P2 = T / S2. The second layer is a multi-scale graph structure learning layer, which generates a first adjacency matrix A according to the first set of time blocks and the second set of time blocks output by the first layer respectively S1 and a second adjacency matrix A S2 and outputs it, and its dimension is d×d. The third layer is the spatial feature capture layer of the first set of time blocks, and its input is the first set of time blocks output by the first layer and the first adjacency matrix A output by the second layer. S1 , and this layer identifies the spatial correlations between the first time blocks output by the first layer based on the output A of the second layer. S1 Then, based on the obtained spatial correlations, it uses a graph attention neural network to perform feature weighted aggregation on the first time blocks output by the first layer, so as to output a feature representation with a dimension of P1×3×d. The fourth layer is a global time feature capture layer, and its input is a feature representation with a dimension of P1×3×d output by the third layer. This global time feature capture layer uses a global attention mechanism to obtain a global attention representation with a dimension of T×256 from this feature representation and outputs it. The fifth layer is a local time feature capture layer, and its input is a feature representation with a dimension of P1×3×d output by the third layer. This local time feature capture layer uses a local attention mechanism to obtain a local attention representation with a dimension of T×256 from the feature representation output by the third layer and outputs it. The sixth layer is the spatial feature capture layer of the second set of time blocks. Its input is the second set of time blocks output by the first layer and the second adjacency matrix A output by the second layer S2 , and this layer is based on the second adjacency matrix A output by the second layer S2 to identify the spatial correlation of the second set of time blocks output by the first layer, and based on the obtained spatial correlation, use the graph attention neural network to perform feature weighted aggregation on the second time blocks output by the first layer to output a feature representation with a dimension of P2×5×d. The seventh layer is a global time feature capture layer, and its input is a feature representation with a dimension of P2×5×d output by the sixth layer. This global time feature capture layer uses a global attention mechanism to obtain a global attention representation with a dimension of T×256 from this feature representation and outputs it. The eighth layer is a local time feature capture layer, and its input is a feature representation with a dimension of P2×5×d output by the sixth layer. This local time feature capture layer uses a local attention mechanism to obtain a local attention feature with a dimension of T×256 from this feature representation and outputs it. The ninth layer is the global multi-scale representation aggregation layer. Its inputs are the global attention representations with dimensions of T×256 output by the fourth layer and the global attention representations with dimensions of T×256 output by the seventh layer. This global multi-scale representation aggregation layer aggregates the outputs of the fourth layer and the seventh layer to obtain the global attention representation and outputs it. The 10th layer is a local multi-scale representation aggregation layer, whose input is the local attention representation with a dimension of T×256 output by the 5th layer and the local attention representation with a dimension of T×256 output by the 8th layer. This local multi-scale representation aggregation layer aggregates the outputs of the 5th layer and the 8th layer to obtain the local attention representation and outputs it.

4. The multivariate time series anomaly detection method based on multi-scale graph structure learning according to any one of claims 1 to 3, characterized in that, The multi-scale graph structure learning layer generates corresponding adjacency matrices for the first set of time blocks and the second set of time blocks output by the first layer respectively and outputs them. Specifically, first, obtain the node embedding vectors First-scale embedding and second-scale embedding where N is the dimension of the multivariate time series and d e is the embedding dimension; Then, for the shared embedding vector N nodes perform dot product operations with the first-scale embedding vector and the second-scale embedding vector respectively to obtain the first feature embedding vector N S1 and the second feature embedding vector N S2 . Subsequently, based on the first feature embedding vector N S1 and the second feature embedding vector N S2 obtain the first adjacency matrix A corresponding to the first set of time blocks S1 and the second adjacency matrix A corresponding to the second set of time blocks S2 , where the first adjacency matrix A is obtained S1 specifically using the following formula: Among them, is a trainable parameter matrix; Obtain the second adjacency matrix A S2 The formula is basically the same as the above formula, except that S1 in the formula is replaced by S2.

5. The multivariate time series anomaly detection method based on multi-scale graph structure learning according to claim 4, characterized in that The process of the global time feature capture layer obtaining the global attention representation is specifically as follows. First, an embedding operation is performed on the feature representation with dimensions of P1×3×d output by the third layer using the segmentation scale S1 to obtain the embedded sequence. Then, the global time features are obtained from the embedded sequence using the multi-head attention mechanism. Finally, the obtained global time features are upsampled to obtain the global attention representation with dimensions of T×256. The process of the local time feature capture layer obtaining the local attention representation is specifically as follows. First, an embedding operation is performed on the feature representation with a dimension of P1×3×d output by the third layer using the segmentation scale S1 to obtain the embedded sequence. Then, the local time features are obtained from the embedded sequence using the multi-head attention mechanism. Finally, as shown in Figure 2, the obtained local time features are upsampled to obtain the local attention representation with a dimension of T×256.

6. The multivariate time series anomaly detection method based on multi-scale graph structure learning according to claim 5, characterized in that The global and local attention representation contrast module, whose input is the global attention representation with a dimension of T×256 output by the ninth layer and the local attention representation with a dimension of T×256 output by the tenth layer in the adaptive multi-scale module. The global and local attention representation contrast module compares the global attention representation and the local attention representation to obtain an anomaly score with a dimension of T×1 and outputs it; The calculation formula of the anomaly score AnomalyScore is as follows: AnomalyScore(X)=∑KL(X inter ,Stopgrad(X intra ))+KL(X intra ,Stopgrad(X inter )) Among them, X represents the preprocessed multivariate time series that is the input of the first layer in the adaptive multi-scale module, KL() represents the KL divergence, and X inter represents the global attention representation output by the 9th layer in the adaptive multi-scale module, and X intra represents the local attention representation output by the 10th layer in the adaptive multi-scale module, and Stopgrad() represents stopping gradient descent.

7. The multivariate time series anomaly detection method based on multi-scale graph structure learning according to claim 6, characterized in that The input of the anomaly detection module is the anomaly score with a dimension of T×1 output by the global and local attention representation contrast module. The anomaly detection module determines whether the multivariate time series input to the first layer in the adaptive multi-scale module is abnormal by judging whether the anomaly score exceeds the anomaly threshold; Among them, if the anomaly score exceeds the anomaly threshold, it is considered that the time series is abnormal.

8. The multivariate time series anomaly detection method based on multi-scale graph structure learning according to claim 7, characterized in that The anomaly detection model is obtained through the following steps: (2-1) Obtain the open-source datasets SMD, MSL, SWaT, and PSM, merge all the datasets, and divide the merged dataset into a training set and a test set according to a ratio of 8:

2. (2-2) Preprocess the training set obtained in step (2-1) to obtain the preprocessed training set. (2-3) For each sample in the training set preprocessed in step (2-2), input the sample into the first layer of the adaptive multi-scale module to obtain the corresponding first set of time blocks for the sample; its dimension is P1×3×d, where P1 is the total number of time blocks in the first set of time blocks, and d is the dimension of the multivariate time series. (2-4) For each sample in the training set preprocessed in step (2-2), input the sample into the second layer of the adaptive multi-scale module to obtain the corresponding first adjacency matrix for the sample, with its dimension being d×d. (2-5) For each sample in the training set preprocessed in step (2-2), input the corresponding first set of time blocks for the sample obtained in step (2-3) and the corresponding first adjacency matrix for the sample obtained in step (2-4) into the third layer of the adaptive multi-scale module to extract the corresponding feature representation for the sample, with its dimension being P1×3×d. (2-6) For each sample in the training set preprocessed in step (2-2), input the corresponding feature representation for the sample obtained in step (2-5) into the fourth layer of the adaptive multi-scale module to extract the corresponding global attention representation for the sample, with its dimension being T×256. (2-7) For each sample in the training set preprocessed in step (2-2), input the corresponding feature representation for the sample obtained in step (3) into the fifth layer of the adaptive multi-scale module to extract the corresponding local attention representation for the sample, with its dimension being T×256. (2-8) For each sample in the training set preprocessed in step (2-2), input the sample into the first layer of the adaptive multi-scale module to obtain the second set of time blocks, with its output dimension being P2×5×d, where P2 is the size of the time block. (2-9) For each sample in the training set preprocessed in step (2-2), input the sample into the second layer of the adaptive multi-scale module to obtain the corresponding second adjacency matrix for the sample, with its dimension being d×d. (2-10) For each sample in the training set preprocessed in step (2-2), input the corresponding second set of time blocks for the sample obtained in step (2-8) and the corresponding second adjacency matrix for the sample obtained in step (2-9) into the sixth layer of the adaptive multi-scale module to extract the corresponding feature representation for the sample, with its dimension being P2×3×d. (2-11) For each sample in the training set preprocessed in step (2-2), input the corresponding feature representation for the sample obtained in step (2-10) into the seventh layer of the adaptive multi-scale module to extract the corresponding global attention representation for the sample, with its dimension being T×256. (2-12)For each sample in the training set preprocessed in step (2-2), input the feature representation corresponding to this sample obtained in step (2-10) into the 8th layer of the adaptive multi-scale module to extract the local attention representation corresponding to this sample, whose dimension is T×256. (2-13)For each sample in the training set preprocessed in step (2-2), input the global attention representation corresponding to this sample obtained in step (2-6) and the global attention representation corresponding to this sample obtained in step (2-11) into the 9th layer of the adaptive multi-scale module for aggregation to obtain the global attention representation, whose output dimension is T×256. (2-14)For each sample in the training set preprocessed in step (2-2), input the local attention representation corresponding to this sample obtained in step (2-7) and the local attention representation corresponding to this sample obtained in step (2-12) into the 10th layer of the adaptive multi-scale module for aggregation to obtain the local attention feature corresponding to this sample, whose output dimension is T×256. For each sample in the training set preprocessed in step (2-15), according to the global attention representation corresponding to this sample obtained in step (2-13) and the local attention representation corresponding to this sample obtained in step (2-14), and use a loss function based on KL divergence to calculate the global representation loss L corresponding to this sample inter ; (2-16) For each sample in the training set preprocessed in step (2-2), based on the global attention representation corresponding to this sample obtained in step (2-13) and the local attention representation corresponding to this sample obtained in step (2-14), and use a loss function based on KL divergence to calculate the local representation loss L corresponding to this sample intra : (2-17)For each sample in the training set preprocessed in step (2-2), obtain the loss function corresponding to this sample according to the global representation loss corresponding to this sample obtained in step (2-15) and the local representation loss corresponding to this sample obtained in step (2-16). Iteratively train the anomaly detection model according to this loss function and using the gradient descent method until the anomaly detection model reaches the preset number of iterations, thereby obtaining a preliminarily trained anomaly detection model. (2-18)Use the test set obtained in step (2-1) to test the anomaly detection model preliminarily trained in step (2-17) until the obtained detection accuracy reaches the optimum, thereby obtaining a finally trained anomaly detection model.

9. The multi-variate time series anomaly detection method based on multi-scale graph structure learning according to claim 8, characterized in that Global representation loss L inter is equal to; L inter = ∑KL(X inter , Stopgrad(X intra )) + KL(Stopgrad(X intra ), X inter ) Among them, Stopgrad() represents stopping gradient descent, and X inter represents the global attention representation corresponding to this sample obtained in step (2-13), and X intra represents the local attention representation corresponding to this sample obtained in step (2-14), and KL() represents the KL loss function; Local representation loss L intra is equal to: L intra = ∑KL(X intra , Stopgrad(X inter )) + KL(Stopgrad(X inter ), X intra ) The loss function is: where len() represents the length calculation function.

10. A multivariate time series anomaly detection system based on multi-scale graph structure learning, characterized in that, It includes: The first module is used to obtain a multi-variate time series, and preprocess the obtained multi-variate time series to obtain a preprocessed multi-variate time series. The second module is used to input the preprocessed multi-variate time series obtained by the first module into a pre-trained anomaly detection model to obtain corresponding anomaly detection results.

Images