Multi-element time series anomaly detection method and system based on mask graph neural network model

Abstract

The application discloses a multi-element time sequence anomaly detection method and system based on a mask graph neural network model, and the method comprises the following steps: performing a data preprocessing operation on a multi-element time sequence; inputting the preprocessed multi-element time sequence; performing a proportional mask operation on time sequence data in a sliding window; establishing a graph structure by using the similarity of each node embedded representation vector; aggregating neighbor node information for each graph node according to the time sequence in the sliding window and by using the current graph structure, learning a feature representation of the sliding window, and predicting the value of the next timestamp by using the feature; performing generative adversarial training on the predicted value and the real value, calculating the generator loss and the discriminator loss, and updating the model; calculating the anomaly score of each node by the difference between the predicted value and the real value, judging the time point with an anomaly score higher than a threshold value as an abnormal time point, and obtaining an anomaly analysis result. The application is helpful for accurately detecting the anomaly of multi-element time variable data.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data mining, and particularly relates to a multivariate time series anomaly detection method and system based on a masked graph neural network model. BACKGROUND

[0002] In order to ensure the normal operation of a network system, operation and maintenance personnel need to monitor a large amount of data at any time. These data come from a large number of interrelated monitoring devices, which are constantly generated as the system operates, forming a multivariate time series. Time series anomaly detection is a bottom-layer core work of intelligent operation and maintenance, and its goal is to analyze the changes of the time series and find out abnormal outliers or columns from a large amount of data, that is, abnormal conditions of system hardware and software services. With the large-scale and complexification of service systems, the application of the Internet of Things and sensors is gradually expanding, and it is particularly important to find faults and ensure safety through monitoring.

[0003] Unsupervised anomaly detection has a wider range of applications because, in general, the application environment often lacks sufficient anomaly labels, and the changes of anomalies are irregular. Intuitively, a threshold can be set, and data outside the range can be identified as abnormal data, thereby completing the task without the aid of labeled data. The current common unsupervised anomaly detection methods include the following: statistical learning-based methods, such as principal component analysis, distance-based clustering, and density-based clustering, but these methods require prior knowledge about anomalies; machine learning-based methods, such as random forests, isolation forests, and single classification support vector machines, are also applied to anomaly detection, but this kind of method is relatively simple in fitting the distribution of abnormal data and is not sufficient to accurately detect anomalies in multivariate time data; in addition, deep learning-based methods have the disadvantage that the complexity of the network structure will lead to large computational load and lack of real-time performance.

[0004] Therefore, it is necessary to design a new multivariate time series anomaly detection method to solve the above problems. SUMMARY

[0005] The main purpose of the application is to provide a multivariate time series anomaly detection method and system based on a masked graph neural network model, which helps to accurately detect anomalies in multivariate time variable data.

[0006] To achieve the above purpose, the application provides a multivariate time series anomaly detection method based on a masked graph neural network model, which comprises the following steps:

[0007] Step S1, performing data preprocessing operation on the multivariate time series;

[0008] Step S2, input the pre-processed multivariate time series, and cut a fixed length, i.e. a sliding window, from the time sequence of each sensor in time sequence, and slide at a fixed step to obtain a final data set, each sensor is a graph node, and the corresponding time sequence is a node feature; the time sequence data in the sliding window is subjected to proportional masking operation;

[0009] Step S3, initializing a node embedded representation vector for each sensor, and updating, a graph structure is established by using the similarity of each node embedded representation vector;

[0010] Step S4, according to the time sequence in the sliding window and by using the current graph structure, neighbor node information is aggregated for each graph node, a feature representation of the sliding window is learned, and the feature is used to predict the value of the next time stamp;

[0011] Step S5, the predicted value and the true value are subjected to generative adversarial training, a generator loss and a discriminator loss are calculated, and the model is updated;

[0012] Step S6, in the test stage, an anomaly score of each node is calculated according to the difference between the predicted value and the true value, the highest score is selected as a threshold, time points with an anomaly score higher than the threshold are determined as abnormal time points, and a final anomaly analysis result is obtained.

[0013] Further improvement of the application is that step S1 comprises the following steps:

[0014] The missing values are processed, and the missing parts are filled with 0;

[0015] The time sequence is subjected to sampling operation, and a group of data is selected every fixed time period;

[0016] The values of the entire time sequence of each sensor are subjected to normalization processing, and the data is compressed to 0-1.

[0017] Further improvement of the application is that step S2 comprises the following steps:

[0018] The fixed length, i.e. the sliding window, of each sensor is cut in time sequence, and slides at a fixed step until the end of time to obtain a processed data set;

[0019] The maximum sequence length of the continuous mask is specified to ensure the continuity of the time sequence; the maximum continuous mask sequence length is half of the total number of masked time points;

[0020] The time sequence in the sliding window is subjected to proportional random masking, and the value of the sensor at the masked time point is set to 0; if the continuous mask sequence length exceeds the maximum mask sequence length, the mask operation is performed with the maximum mask sequence length.

[0021] The further improvement of the application is that step S3 comprises the following steps: initializing a node-embedded representation vector for each sensor;

[0022] The node-embedded representation vector is updated by using an encoder, and the randomly initialized node-embedded representation vector is converted into a dense vector with the same dimension, and the encoder parameters are updated;

[0023] In order to control the number of neighbor nodes to improve the operation efficiency and avoid overfitting, the maximum number of neighbor nodes is set to control the number of edges;

[0024] The graph structure is constructed according to the cosine similarity of the node-embedded representation vector.

[0025] The further improvement of the application is that step S4 comprises the following steps:

[0026] According to the time series in the sliding window and the current graph structure, the neighbor node information of each graph node is aggregated by using an attention mechanism, and the feature representation of the sliding window is learned;

[0027] The feature representation of the sliding window is input into a multi-layer fully connected network to predict the value of the next timestamp of each graph node.

[0028] The further improvement of the application is that step S5 comprises the following steps:

[0029] The generator is an attention mechanism-based graph model, the generator loss is calculated, and the generator is updated;

[0030] The discriminator is a GAN model based on the discriminator, the real value of the next timestamp is spliced after the time series in the sliding window to form a real time series, the predicted value of the next timestamp is spliced after the time series in the sliding window to form a predicted time series, the real time series and the predicted time series are input into the discriminator, the discriminator loss is calculated, and the discriminator is updated.

[0031] The further improvement of the application is that step S6 comprises the following steps:

[0032] The time series in the sliding window is not subjected to a mask operation, and the predicted value of each node is calculated;

[0033] The anomaly score of each node is calculated by the difference between the predicted value and the real value;

[0034] In order to reduce the influence of the relative size of different sequences, when calculating the anomaly score, the application uses a smoothing function: subtracting the average value of each sequence and dividing by the standard deviation, and selecting the highest score as the threshold;

[0035] The time point with the abnormal score higher than the threshold value is determined as an abnormal time point, and a final abnormal analysis result is obtained.

[0036] To achieve the above purposes, the application further provides a multi-element time series anomaly detection system based on a mask graph neural network model, which executes the method of any one of the preceding.

[0037] The application has the following advantages: the application uses an unsupervised method to solve the problems of difficult data labeling and imbalance; the combination of time series mask operation and graph structure enhances the learning ability of the graph model, which is helpful for locating abnormal nodes and nodes affected by them; the application has good flexibility and scalability, and by adjusting the network parameters more suitable for a certain working environment, the prediction accuracy or operation efficiency is further improved; experiments prove that this model has better performance than the most advanced anomaly detection technology in anomaly detection. The results on a rich data set also prove that the anomaly detection method provided by the application has universality. BRIEF DESCRIPTION OF DRAWINGS

[0038] Fig. 1 The application provides an overall flowchart;

[0039] Fig. 2 The application provides a mask operation flowchart;

[0040] Fig. 3 The application provides a graph structure construction flowchart;

[0041] Fig. 4 The application provides an anomaly detection flowchart. DETAILED DESCRIPTION

[0042] In order to make the purposes, technical solutions and advantages of the application clearer, the application will be described in detail below with reference to the drawings and specific embodiments.

[0043] It should be emphasized that in the description of the application, various formulas and constraints are distinguished by consistent labels before and after, but different labels may also be used to mark the same formula and / or constraint. The purpose of this setting is to more clearly illustrate the features of the application.

[0044] The application discloses a multi-element time series anomaly detection method based on a mask graph neural network model, which mainly includes the following steps:

[0045] Step S1, data preprocessing operation is performed on the multi-element time series;

[0046] Step S2, input the pre-processed multivariate time series, and cut a fixed length, i.e., a sliding window, from the time sequence of each sensor in time sequence, and slide at a fixed step to obtain a final data set, each sensor being a graph node and the corresponding time sequence being a node feature; and performing proportional masking operation on the time sequence data in the sliding window.

[0047] Step S3, initializing a node embedding representation vector for each sensor and updating, and establishing a graph structure by using the similarity of each node embedding representation vector.

[0048] Step S4, aggregating neighbor node information for each graph node according to the time sequence in the sliding window and using the current graph structure to learn a feature representation of the sliding window, and predicting a value of a next time stamp by using the feature.

[0049] Step S5, performing generative adversarial training on the predicted value and the real value, calculating a generator loss and a discriminator loss, and updating the model.

[0050] Step S6, calculating an anomaly score of each node by the difference between the predicted value and the real value in the test stage, selecting the highest score as a threshold, and judging time points with an anomaly score higher than the threshold as abnormal time points to obtain a final anomaly analysis result.

[0051] The method of the application will be described in detail below with reference to the accompanying drawings. Figs. 1 to 4

[0052] In step S1, further comprising:

[0053] Firstly, the data is processed for missing values, and the missing parts are filled with 0.

[0054] Secondly, under the premise of not affecting the abnormal detection effect, in order to save training resources, the application performs sampling operation on the time sequence, selects a group of data every fixed time period, and uses the median of the value of each sensor to represent the feature of the time period.

[0055] Finally, the values of the entire time sequence of each sensor are normalized to compress the data to 0-1 to speed up the model training convergence speed.

[0056] Step S2, input the pre-processed multivariate time series, and cut a fixed length, i.e., a sliding window, from the time sequence of each sensor in time sequence, and slide at a fixed step to obtain a final data set, each sensor being a graph node and the corresponding time sequence being a node feature; and performing proportional masking operation on the time sequence data in the sliding window.

[0057] The specific method of performing proportional masking operation on the time sequence data in the sliding window comprises:

[0058] ​Firstly, a fixed length of each sensor is intercepted in time sequence, i.e. a sliding window, and is slid with a fixed step until the end of time to obtain a processed data set. In the embodiment, the multivariate time variable can be expressed as X={xt}t∈T, n is the number of sensors. The sliding window size is set as w, and the data in the continuous w time stamps of each sensor is selected, such as and is slid with a fixed step until the end of time to record the data in the sliding window and the data corresponding to the next time stamp to obtain a processed data set.

[0059] Secondly, the maximum sequence length of the continuous mask is specified to ensure the continuity of the time sequence. The maximum continuous mask sequence length is half of the total number of shielded time points.

[0060] Finally, the time sequence in the sliding window is randomly masked in proportion, and the value of the sensor at the masked time point is set to 0. If the continuous mask sequence length exceeds the maximum mask sequence length, the mask operation is performed with the maximum mask sequence length. In the embodiment, some parts of the time sequence in the sliding window are randomly removed in proportion. For example, when 20% of the mask is taken for the sliding window with a length of 20, four time nodes are randomly set to 0 by the present application. The goal of the present application is to generate prediction values under the mask condition, therefore, in order not to damage the continuity of the time sequence in a large range, the present application will specify the maximum sequence length of the mask operation. Generally, the maximum continuous mask length will not exceed half of the mask rate. That is, the longest continuous mask length in the above example is 2.

[0061] Step S3, initializing a node embedding representation vector for each sensor and updating, and establishing a graph structure by using the similarity of each node embedding representation vector. The steps of establishing the graph structure include:

[0062] initializing a node embedding representation vector for each sensor;

[0063] updating the node embedding representation vector by using the encoder to convert the randomly initialized node embedding representation vector into a dense vector with the same dimension, and keeping the encoder parameters updated;

[0064] In order to control the number of neighbor nodes to improve the operation efficiency and avoid overfitting, the maximum number of neighbor nodes is set to control the number of edges;

[0065] constructing a graph structure according to the cosine similarity of the node embedding representation vector.

[0066] In this embodiment, the node embedding representation vector is constantly updated as the training proceeds, and the graph model most suitable for prediction is trained using the node embedding representation vector. The present application uses the node embedding representation vector as a tool to represent each time series feature, which is mapped to a high-dimensional vector. These vectors are randomly initialized, and the present application hopes that these embeddings can represent the features between nodes as accurately as possible, so it updates them using an encoder to convert the randomly initialized node embedding representation vector into a dense vector of the same dimension. The encoder will keep updating its parameters during model training, and the vector after encoding by the encoder is the new node embedding representation vector. The node embedding representation vector is denoted as V i ∈R d , and the graph structure is updated using the cosine similarity between them. V1, V2 are the embedding representation vectors of nodes v1, v2, and the node similarity R n (v1, v2) is:

[0067] In order to control the number of neighbor nodes to improve the operation efficiency, while avoiding overfitting, the present application adopts the strategy of setting the maximum number of neighbor nodes in advance to control the number of edges. For example, when the maximum number of neighbor nodes is 20, the similarity of other nodes with node A is sorted from large to small, and then the top 20 nodes are selected to establish a relationship with node A.

[0068] Step S4, according to the time series in the sliding window and using the current graph structure, aggregate neighbor node information for each graph node to predict the value of the next timestamp. The step of predicting the value of the next timestamp includes:

[0069] According to the time series in the sliding window and the current graph structure, use the attention mechanism to aggregate neighbor node information for each graph node to learn the feature representation of the sliding window;

[0070] Input the feature representation of the sliding window into a multi-layer fully connected network to predict the value of the next timestamp for each graph node.

[0071] In this embodiment, to aggregate information about nodes and their relevance to their neighbors, the attention coefficient a i，j between graph nodes needs to be calculated. The attention coefficient represents the importance of the features of node j to node i. In order to calculate a i，j , first consider the influence of both nodes and calculate the attention value ∈ i，j between them:

[0072]

[0073] Where LeakyReLU is a non-linear activation function, and 'a' is the learning coefficient vector of the attention mechanism. Let W represent a connection, where W is a trainable linear transformation matrix with shared weights for each node. This is achieved by embedding the node representation vector V... i Features of the current moment Combining these methods allows for a more comprehensive calculation of the attention factor. When aggregating neighbor information, it is necessary to normalize the attention of all neighbors for each node. The normalized attention weight is the aggregation coefficient α. i，j :

[0074]

[0075] The aggregation representation of each node is obtained through the aggregation coefficient.

[0076]

[0077] The aggregated representations of all nodes are then input into a multi-layer fully connected network to obtain the predicted value for each node. The entire prediction process can be represented as G():

[0078] Step S5 involves performing generative adversarial training on the predicted and true values, calculating the generator loss and discriminator loss, and updating the model. Methods for generative adversarial training include:

[0079] The generator is a graph model based on an attention mechanism. The generator loss is calculated and the generator is updated.

[0080] The discriminator-based GAN model concatenates the true value of the next timestamp after the time series within the sliding window to form the true time series, and concatenates the predicted value of the next timestamp after the time series within the sliding window to form the predicted time series. The true time series and the predicted time series are input into the discriminator, the discriminator loss is calculated, and the discriminator is updated.

[0081] In a specific embodiment, the focus of the graph structure model of this invention is to learn a normal time series data distribution, so it needs to be trained to make the predicted value of the next timestamp as close as possible to the true value. That is, when training in an adversarial manner, the model needs to generate as much fake data as possible to confuse the discriminator. The input obtained through the time window is represented as: {x1, ..., x...} t-1}, then x should be obtained through a graph-based model G(). t :

[0082]

[0083] The generator is updated according to the generator loss.

[0084] The present application uses a basic GAN model as the discriminator. The discriminator needs to judge the data generated by the model and the real data. In order to consider deeper correlations, the present application inputs time series into the discriminator. The real time series is composed of the real value of the next timestamp after the time series in the sliding window, and the predicted time series is composed of the predicted value of the next timestamp after the time series in the sliding window. The real time series and the predicted time series are input into the discriminator. The discriminator is represented as D(), and x t is real data, is predicted data, and the input can be represented as:

[0085]

[0086] The discriminator loss is as follows:

[0087]

[0088] The discriminator is updated according to the discriminator loss.

[0089] Step S6, the test phase calculates the anomaly score of each node through the difference between the predicted value and the real value, selects the highest score as the threshold, judges the time point with an anomaly score higher than the threshold as an abnormal time point, and obtains the final anomaly analysis result. The method of anomaly detection in the test phase includes:

[0090] The predicted value of each node is calculated without mask operation on the time series in the sliding window;

[0091] The anomaly score of each node is calculated through the difference between the predicted value and the real value;

[0092] In order to reduce the influence of the relative size of different sequences, when calculating the anomaly score, the present application uses a smoothing function: subtract the average value of each sequence and divide by the standard deviation, and selects the highest score as the threshold;

[0093] The time point with an anomaly score higher than the threshold is judged as an abnormal time point, and the final anomaly analysis result is obtained.

[0094] For specific embodiments, in the test phase, the predicted value is calculated without mask operation, and the anomaly score of each node is calculated through the difference between the predicted value and the real value. The index with the highest anomaly score is selected as the basis for judging the abnormality of the time point. In order to reduce the influence of the relative size of different sequences, the present application uses a smoothing function: subtract the average value of each sequence and divide by the standard deviation:

[0095]

[0096] ADS t = max{loss n} n∈N

[0097] The time point where the anomaly score exceeds the threshold value is regarded as an abnormal time point, and a final anomaly analysis result is obtained.

[0098] Based on the above inventive concept, the application further provides a multi-element time series anomaly detection system based on a mask graph neural network model, comprising at least one computing device, the computing device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the computer program is loaded into the processor, the multi-element time series anomaly detection method based on the mask graph neural network model described above can be implemented.

[0099] The application adopts an unsupervised method to solve the problems of difficult data labeling and imbalance; in combination with time series mask operation and graph structure, the learning ability of the graph model is enhanced, which is helpful for locating abnormal nodes and nodes affected thereby; the application has good flexibility and expansibility, and by adjusting network parameters more suitable for a certain working environment, the prediction accuracy or operation efficiency is further improved; experiments prove that the model has better performance than the most advanced anomaly detection technology in anomaly detection. The results on a rich data set also prove that the anomaly detection method provided by the application has universality.

[0100] The above examples are only used to illustrate the technical solutions of the application and not to limit it, although the application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the application.

Claims

1. A multivariate time series anomaly detection method based on a masked graph neural network model, characterized in that, The method includes the following steps: Step S1: Perform data preprocessing on the multivariate time series; Step S2: Input the preprocessed multivariate time series, extract a fixed length (i.e., a sliding window) for each sensor according to the time sequence, and slide it with a fixed step size to obtain the final dataset. Each sensor is a graph node, and the corresponding time series is a node feature. Perform a proportional masking operation on the timing data within the sliding window; Step S3: Initialize a node embedded representation vector for each sensor, and update the node embedded representation vector using the encoder. Transform the randomly initialized node embedded representation vector into a dense vector with the same dimension, while keeping the encoder parameters updated. In order to control the number of neighboring nodes to improve computational efficiency and avoid overfitting, set the maximum number of neighboring nodes to control the number of edges. A graph structure is built using the cosine similarity of the embedded representation vectors of each node; Step S4: Based on the time series within the sliding window and the current graph structure, use an attention mechanism to aggregate neighbor node information for each graph node, learn the feature representation of the sliding window, and input the feature representation of the sliding window into a multi-layer fully connected network to predict the value of the next timestamp for each graph node. Step S5: Perform generative adversarial training on the predicted and true values, calculate the generator loss and discriminator loss, and update the model. The generator is a graph model based on an attention mechanism. Calculate the generator loss and update the generator. The discriminator is a GAN model based on a GAN. The true time series is formed by concatenating the true value of the next timestamp after the time series within the sliding window, and the predicted time series is formed by concatenating the predicted value of the next timestamp after the time series within the sliding window. Input the true time series and the predicted time series into the discriminator, calculate the discriminator loss, and update the discriminator. Step S6: During the testing phase, the anomaly score of each node is calculated based on the difference between the predicted value and the actual value. The highest score is selected as the threshold, and the time points with anomaly scores higher than the threshold are identified as abnormal time points, thus obtaining the final anomaly analysis results.

2. The method according to claim 1, characterized in that: Step S1 includes the following steps: Handle missing values ​​by filling in 0 for the missing parts; The time series is sampled, and a set of data is selected at fixed time intervals. The values ​​of each sensor over the entire time series are normalized, compressing the data to between 0 and 1.

3. The method according to claim 2, characterized in that: Step S2 includes the following steps: For each sensor, a fixed length, i.e. a sliding window, is extracted according to the time sequence and slid in fixed steps until the end of the time, to obtain the processed dataset; Specify the maximum sequence length of the continuous mask to ensure the continuity of the time series; the maximum continuous mask sequence length is half the total number of masked time points; The time series within the sliding window is randomly masked proportionally, and the sensor value at the masked time point is set to 0; if the length of the continuous mask sequence exceeds the maximum mask sequence length, the masking operation is performed with the maximum mask sequence length.

4. The method according to claim 1, characterized in that: Step S6 includes the following steps: For time series within the sliding window, no masking operation is performed; the predicted value for each node is calculated. The anomaly score for each node is calculated based on the difference between the predicted and actual values. To reduce the influence of the relative size of different sequences, this invention uses a smoothing function when calculating outlier scores: subtract the mean of each sequence and divide by the standard deviation, then select the highest score as the threshold. The time points where the abnormal score is higher than the threshold are identified as abnormal time points, and the final anomaly analysis results are obtained.

5. A multivariate time series anomaly detection system based on a masked graph neural network model, characterized in that, Perform the method according to any one of claims 1-4.

Images