Multi-period time sequence anomaly detection method based on space-time diagram neural network

Through the multi-period time series anomaly detection method based on the spatio-temporal graph neural network, the problem of high false alarm rate and insufficient timeliness of time series anomaly detection in industrial scenarios is solved. By constructing a double-residual spatio-temporal graph attention network model and multi-period prediction results The fusion algorithm achieves more accurate and efficient abnormal detection, and is suitable for fault analysis and monitoring of complex systems.

CN118939699APending Publication Date: 2024-11-12HOHAI UNIV

Patent Information

Application Number
CN202310524917.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-10
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing time series anomaly detection methods have problems with high false positive rates and insufficient timeliness in industrial scenarios, especially in multivariate time series and complex systems, it is difficult to capture the relationship between different time series, and traditional models ignore the system. The periodicity of operation leads to false positives.

Method used

Using a multi-period time series anomaly detection method based on a spatio-temporal graph neural network, the multi-period double-residual space-time graph attention network model is constructed to construct a coordinated device entity graph model, and the data is standardized and outlier adjustment is adjusted. The multi-period double-residual space-time graph attention network model is constructed, combined with parallelism Spatiotemporal feature extraction model and long-time series prediction model, using multi-period prediction results fusion algorithm to improve detection accuracy.

Benefits of technology

Effectively handles the missing or abnormal information of the device entity data, improves the timeliness and accuracy of system abnormality detection, reduces false alarms, improves data analysis efficiency and model performance, and is suitable for grid fault analysis and network equipment operation status monitor.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118939699A_ABST
    Figure CN118939699A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-period time sequence anomaly detection method based on a space-time diagram neural network, and the method comprises the steps: building an abnormal value adjustment algorithm based on density distribution, and effectively processing the missing or anomaly of key information of equipment entity data; a multi-period-based double-residual space-time diagram attention network model is constructed, and comprises a parallel space-time feature extraction model based on a residual network and a long-time sequence prediction model based on a double-residual network, so that the timeliness and accuracy of system anomaly detection are improved, and a user is effectively assisted in system maintenance; a multi-period prediction result fusion algorithm is provided, an initial value is set, and a component weight is adaptively learned, so that more accurate prediction is realized. According to the invention, the professional requirements of business lines such as scheduling, equipment, operation and maintenance can be met, and the method is of great significance to the maintenance and construction of the system. The method comprises the following steps: S1, constructing a graph model for mutually cooperative equipment entities, collecting a multivariable time sequence of each equipment entity, and executing data standardization and abnormal value adjustment based on density distribution on each independent time sequence; s2, constructing a multi-period-based double-residual space-time diagram attention network model, and inputting the preprocessed multivariable time sequence data to respectively obtain multi-component prediction results based on different historical cycle data; and S3, performing a multi-period prediction result fusion algorithm on the multi-component model prediction results of the input different period data to obtain a final prediction result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of electronic information technology, and in particular to an anomaly detection method for multi-period time series based on a spatiotemporal graph neural network. Background Art

[0002] Time series anomaly detection is a very important field in computer science and data mining. In industrial scenarios, the ability to detect anomalies is essential, as undetected faults can lead to serious damage. Equipment entities are prone to failure, which means that effective anomaly detection can improve system availability and reliability. Undetected abnormal states directly affect production efficiency and cause increased operating and maintenance costs. Therefore, many studies on this topic can be found in multiple real industrial scenarios such as power systems, network equipment, automobiles, manufacturing, energy, or industrial sensor networks.

[0003] In recent years, thanks to the development of Industry 4.0 and the Internet of Things, technology has provided companies with more efficient and reliable monitoring systems. In this way, industrial equipment entities are equipped with multiple sensors, which form a multi-sensor system. These systems make data collection simpler, so now more quantity and quality data can be obtained. As a result, machine learning techniques for time series anomaly detection have proliferated. However, industrial equipment entities are complex and often use a large number of sensors, and many of the execution actions in these equipment entities consist of multiple events, which makes it more difficult to detect anomalies. Because it is necessary to consider in which part of the time series each event occurs. Therefore, there may be different behaviors in a time series, which can be regarded as context-specific anomalies in a single action. And known faults can be modeled and detected, unforeseen problems are not defined, so there are many more difficult to detect anomalies that can only rely on the experience of professionals to judge whether the system is operating normally.

[0004] However, in the existing anomaly detection solutions, the high false alarm rate and lack of timeliness still exist. Therefore, users have an increasing demand for improving system stability monitoring and early warning of anomalies. When a system fails, the speed and accuracy of fault analysis not only affects the user experience and interests, but also the operating performance and service level of the system operator. Therefore, timely determining the fault location, minimizing the fault repair time, and restoring services are crucial to reducing user waiting time, improving service reliability and user satisfaction.

[0005] The present invention uses two indicators commonly used in the technical field when merging the multi-period prediction results: accuracy and Pearson correlation coefficient, which are defined as follows:

[0006] (1) Accuracy: The present invention uses the root mean square error to measure the accuracy of the prediction algorithm. This value represents the difference between the algorithm prediction result and the actual value. The higher the accuracy, the better the algorithm prediction effect. The formula is as follows:

[0007]

[0008] Among them, A represents the accuracy rate after the introduction of the root mean square error, MSE() represents the root mean square error, and yi represents the actual value at the i-th moment. represents the predicted value at the i-th moment, and n represents the number of time points in the sample.

[0009] (2) Pearson correlation coefficient: This indicator is defined by the following formula:

[0010]

[0011] Among them, r is the Pearson correlation coefficient, x i is the input eigenvalue, is the average of the input features, y i is the characteristic value of the predicted daily data, is the average of the characteristic values ​​of the predicted daily data, and m1 is the number of input time values.

[0012] In summary, in order to overcome the above-mentioned deficiencies in the prior art, namely:

[0013] 1. With the development of technology and production relations, the equipment entities in various network systems are large in number, diverse in types, widely distributed, and have complex relationships. Therefore, data often lacks or is abnormal due to equipment failures, transmission data loss, etc., and relevant data need to be identified, supplemented, and adjusted.

[0014] 2. The univariate time series anomaly detection algorithm can detect anomalies in a single indicator, but it does not clearly capture the relationship between different time series when judging whether the entire system is operating normally, which will inevitably lead to false alarms.

[0015] 3. Traditional models only use recent data for analysis and judgment, ignoring the periodicity of system operation. As a result, they will mistake periodic changes in operating status as system operation abnormalities, resulting in false alarms and losses of manpower and material resources.

[0016] In summary, those skilled in the art are in urgent need of solving the above technical problems in distribution network fault judgment. Summary of the invention

[0017] The technical problem to be solved by the present invention is to provide an anomaly detection method based on time series.

[0018] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0019] The present invention provides a multi-period time series anomaly detection method based on a spatiotemporal graph neural network, comprising the following steps:

[0020] Step S1: construct a graph model of the cooperating equipment entities, collect the multivariate time series of each equipment entity and perform data standardization and density distribution-based outlier adjustment on each individual time series;

[0021] Step S2: construct a dual residual spatiotemporal graph attention network model based on multiple periods, input the preprocessed multivariate time series data to obtain multi-component prediction results based on different historical period data;

[0022] Step S3: The prediction results of the multi-component model with different period data are inputted using a multi-period prediction result fusion algorithm to obtain the final prediction result.

[0023] In the method, the multi-period dual residual spatiotemporal graph attention network model in step S2 further includes:

[0024] Step S21: construct a parallel spatiotemporal feature extraction model based on residual network to learn the complex dependencies of multivariate time series in feature and time dimensions;

[0025] Step S22: Build a long time series prediction model based on a dual residual network, use gating to highlight important information, and then globally learn the entity status of node devices according to the system's topological structure to enhance features, and deepen the network through dual residuals to improve prediction accuracy.

[0026] As a preferred solution, the outlier adjustment algorithm based on density distribution is specifically as follows:

[0027]

[0028] Among them, x i is the data at the i-th time point in a time series, M() is the operation expression for taking the median, R is the tolerance distance for outlier detection, x′ is the data with the largest value in the top 25% of the time series data x, |x′| is the number of data in x′, and r is the value of the 75th% of the time series data x in ascending order.

[0029] As a preferred solution, the dual residual spatiotemporal graph attention network model based on multiple periods uses a multi-component structure as the model structure for learning different periodic time patterns. The model consists of three independent component structures with the same structure, which input recent data, daily cycle data, and weekly cycle data respectively. Each independent component structure stacks the same number of spatiotemporal blocks, which enables the model to obtain the same level of spatiotemporal features and effectively fuse the component prediction results of the other two component structures in the later fusion process to obtain the final prediction result. Each independent component consists of two parts: a parallel spatiotemporal feature extraction model based on a residual network and a long time series prediction model based on a dual residual network.

[0030] Recent data is a common historical data used by almost all prediction models. However, the physical devices in the system often have different strong periodicities in time patterns. This periodic data is very helpful in improving prediction accuracy. Because the weather and user habits will make the physical device data of the previous day provide a good reference value for the current prediction, and considering that there will be some differences in the operating status of the physical devices compared to weekdays and weekends, in order to fill this gap, it is necessary to input the physical device data of the same day of each week into the prediction model to improve the prediction accuracy. Therefore, this patent selects the physical device data near the time point of the prediction day as recent data, the physical device data one day before the prediction day as daily cycle data, and the physical device data 7 days before the prediction day as weekly cycle data.

[0031] In this model, t0, t p and S represent the current time, the prediction window size, and the number of entity data per day, respectively. Take the time series of the same length before the current time in the recent data, daily cycle data, and weekly cycle data as the input of each independent component, denoted by T r , T d , T w The following is a detailed description of the three periods of data input into the model:

[0032] (1) Recent data: refers to historical data around the current time of the forecast date. Since operational anomalies of entities generally have precursors, recent data will provide important reference information for anomaly detection and forecast results.

[0033] (2) Daily cycle data: refers to the historical data of the corresponding time period one day before the current time of the forecast day. Because the operation of an entity may show consistent or similar operating status in the same period of time, this patent selects this period of data as part of the joint forecast to capture the similar characteristics of the daily cycle data and the forecast day.

[0034] (3) Weekly cycle data: refers to the historical data of the corresponding time period seven days before the current time of the forecast day. Because the operating status of the entity will change significantly between working days and weekends, selecting data of similar time periods can effectively improve the forecast accuracy. In particular, if the weekly cycle data and the forecast day data do not belong to the same time type, the data of the same time type closest to the weekly cycle data is selected.

[0035] As a preferred solution, the parallel spatiotemporal feature extraction model based on the residual network includes a temporal convolutional network layer, a parallel feature-oriented graph attention network, a time-oriented informer network, and a residual convolution. First, the preprocessed multivariate time series data is input into the temporal convolutional network to extract the high-level features of each time series; the output of the temporal convolutional network is processed separately and simultaneously by the parallel feature-oriented graph attention network and the time-oriented informer network to capture sufficient and rich spatiotemporal features; the output of the temporal convolutional network and the output of the parallel layer are spliced ​​and fused and input into the first residual network. The feature-oriented graph attention network captures the causal relationship between multiple features, regards each univariate time series as a separate feature, and attempts to explicitly model the correlation between different features. At the same time, the time-oriented informer layer emphasizes the dependency in the time dimension and models the time dependency within each time series. The complex dependencies of multivariate time series in the feature and time dimensions are learned through this parallel network layer. The residual network is connected to the fusion layer of the parallel layer and the temporal attention layer. Ensure that this model can not only capture dynamic spatiotemporal features similar to other emerging networks, but also deepen them. The graph attention network and the temporal convolution layer provide dynamic capture capabilities and use residual convolution to effectively prevent network degradation and gradient vanishing. In addition, the graph attention network can alleviate the effect of over-smoothing during network deepening, while the graph neural convolution network cannot alleviate this problem. In order to improve the medium- and long-term prediction capabilities of the model, the coordinated use of the attention mechanism and time series prediction can extract effective spatiotemporal features in a large receptive field. The attention mechanism filters out redundant information in different data, allowing the model to better expand the receptive field. Due to the simple stacking method, there is irreversible information loss during the network deepening process, resulting in network degradation and gradient disappearance. In addition, due to a single feature, the graph neural network may encounter the problem of over-smoothing. When the residual structure is simply superimposed, the residual does not allow the network to continue to deepen, because the residual structure largely lacks the richness of derived features. In order to solve these problems, the parallel spatiotemporal feature extraction model based on the residual network proposed in this patent enables the network to resist the information loss of deepening and enhance the features, capturing sufficient and rich spatiotemporal features.

[0036] As a preferred solution, the long time series prediction model based on the dual residual network includes a gated recurrent unit (GRU) layer, a global graph attention layer and a residual convolution. The residual convolution of the model in step S22 and the residual in step S21 together constitute a double residual convolution. The first convolution is connected to the residual network after the fusion layer, and the second residual is connected to the output of the entire independent component. The input of the GRU is the output of the first residual in step S22, which highlights important information from the long time series and solves the long dependency problem in the long time series. Then the result of the GRU is input into the global graph attention layer to learn the topological structure of the entire system. According to the collaborative relationship between the various device entities of the system, the state information of each entity is updated to reduce the judgment errors caused by the possible misreporting and omission of data in the single device entity data. Finally, the output of the global graph attention network is connected to the second residual network.

[0037] As a preferred solution, the multi-period prediction result fusion algorithm is as follows:

[0038]

[0039] Among them, F r 、F d 、F w are the prediction results obtained using the recent data, daily cycle data and weekly cycle data models, respectively. C is the fusion matrix, W r , W d , W w F r 、F d 、F w The fusion weight parameter matrix corresponding to the prediction result, Y is the final fusion prediction result, α∈{r, d, w}, and the initial values ​​of the fusion weight parameter matrix of the prediction results of the three models are calculated respectively, A α is the historical accuracy of the model’s prediction results, R α is the Pearson similarity between the historical data and the predicted data used by the model, J is and W α A uniformly sized matrix filled with all ones.

[0040] In general, the beneficial technical effects of the present invention are as follows: the present invention provides a multi-period time series anomaly detection method based on a spatiotemporal graph neural network, establishes an outlier adjustment algorithm based on density distribution, and effectively handles the key information missing or anomaly of the device entity data; constructs a multi-period dual residual spatiotemporal graph attention network model, including a parallel spatiotemporal feature extraction model based on a residual network and a long time series prediction model based on a dual residual network, which improves the timeliness and accuracy of system anomaly detection and effectively assists users in system maintenance; proposes a multi-period prediction result fusion algorithm, sets initial values ​​and adaptively learns component weights to achieve more accurate predictions. Its advantages over the prior art are:

[0041] (1) The present invention proposes an outlier adjustment algorithm based on density distribution. Outliers may have a significant impact on the accuracy and efficiency of the model. The present invention can better improve data quality, identify potential problems in the data, reduce erroneous analysis and decision-making caused by outliers, and improve the efficiency of data analysis, thereby improving the accuracy and performance of the model.

[0042] (2) This paper proposes a dual-residual spatiotemporal graph attention network model based on multiple periods. Compared with the traditional model, the network can resist the deepening of information loss and enhance features, capturing sufficient and rich spatiotemporal features. The input of multivariate historical periodic data increases the generalization ability of the model, fully considers the periodicity of the system operation status, and reduces false positives caused by periodic changes.

[0043] (3) The present invention proposes a multi-period prediction result fusion algorithm. When the prediction time has a high degree of match with the historical data used, it is necessary to highlight the prediction results of the corresponding model. The algorithm first determines the initial value of the fusion matrix and adaptively learns the component weights from the data, achieving more accurate predictions and faster training speeds compared to randomly initialized matrices.

[0044] (4) The present invention has broad application prospects in various network-based systems such as power grid fault analysis and network equipment operation status monitoring.

[0045] The specific embodiments of the present invention are further described below in conjunction with the accompanying drawings. Additional aspects and advantages of the present invention will be partially given in the following description, which will become apparent from the following description or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic flow chart of a method for detecting anomalies in multi-period time series based on a spatiotemporal graph neural network according to the technical solution of the present invention is shown;

[0047] Figure 2A model framework diagram of a multi-period time series anomaly detection method based on a spatiotemporal graph neural network according to the technical solution of the present invention is shown; DETAILED DESCRIPTION

[0048] The embodiments of the present invention are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar concepts, objects, elements, etc. or concepts, objects, elements, etc. with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be interpreted as limiting the present invention.

[0049] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the field to which the present invention belongs and related fields. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0050] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0051] The present invention provides a multi-period time series anomaly detection method based on spatiotemporal graph neural network, such as Figure 1 As shown, it includes the following steps S1 to S3:

[0052] Step S1: construct a graph model of the cooperating equipment entities, collect the multivariate time series of each equipment entity and perform data standardization and density distribution-based outlier adjustment on each individual time series.

[0053] Time series anomaly detection is an important research topic in data mining and has a wide range of applications in the industry. Efficient and accurate anomaly detection can help users continuously monitor their key indicators and issue alerts to potential events in a timely manner. In practical applications, univariate time series anomaly detection algorithms can detect anomalies in a single indicator, but there may be problems in judging whether the entire system is operating normally, so multiple time series indicators need to be collected to reflect the operating status of the system. When two related indicators show changes in the same trend, the system is still in a healthy state at this time, so a sudden change in an indicator does not necessarily mean that the system is faulty. In contrast, when changes that are inconsistent with the trend occur, it indicates that the system is abnormal at this time. Therefore, the correlation between different time series must be considered in the multivariate time series anomaly detection system.

[0054] In complex and mutually cooperative equipment entities, a typical graph structure is often presented. Therefore, each entity in the system is regarded as a node of the graph model, and the link lines of the equipment entities are regarded as the edges of the graph model. The graph neural network model of the system is constructed, and the parameter information of each equipment entity in the network structure at different states is obtained. The system presenting a typical graph structure is recorded as a graph model G = (V, E, A), where V∈R N It represents the vertex set, that is, the number N of device entities in the system; E represents the edge set, that is, the link lines of the device entities in the system; A represents the adjacency matrix.

[0055] The input of multivariate time series anomaly detection for each device entity in this patent is x∈R pxk Represented as, where p is the maximum length of the timestamp and k is the number of features in the input. For long time series, a fixed-length input is generated by a sliding window of length n. The task of multivariate time series anomaly detection is to produce an output vector y∈R n , where y i ∈[0, 1] represents the probability of an abnormality occurring at the i-th timestamp, 0 represents normal operation and 1 represents an abnormality.

[0056] The physical data of the equipment in the system is missing or abnormal due to problems with the communication transmission channel such as the network provided by the operator, hardware failure of the device, outdated program version, terminal encryption configuration error and other data collection equipment problems, as well as low collection frequency of key data such as load. In order to improve the robustness of the model, data standardization and outlier adjustment need to be performed on each individual time series.

[0057] (1) Data normalization: Use the maximum and minimum values ​​in the training data to normalize the time series:

[0058]

[0059] Where x is multivariate time series data, is the data after data standardization, max(X train ) and min(X train ) are the maximum and minimum values ​​of the training set, respectively.

[0060] (2) Outlier adjustment: Outliers may have a significant impact on the accuracy and efficiency of the model. In order to better improve data quality and identify potential problems in the data, this patent proposes an outlier adjustment algorithm based on density distribution to reduce erroneous analysis and decision-making caused by outliers, and also improve the efficiency of data analysis, thereby improving the accuracy and performance of the model. The algorithm is as follows:

[0061]

[0062] Among them, x i is the data at the i-th time point in a time series, M() is the operation expression for taking the median, R is the tolerance distance for outlier detection, x′ is the data with the largest value in the top 25% of the time series data x, |x′| is the number of data in x′, and r is the value of the 75th% of the time series data x in ascending order.

[0063] Step S2: Construct a dual residual spatiotemporal graph attention network model based on multiple periods, input the preprocessed multivariate time series data to obtain multi-component prediction results based on different historical period data.

[0064] Recent data is a common historical data used by almost all prediction models. However, the physical devices in the system often have different strong periodicities in time patterns. This periodic data is very helpful in improving prediction accuracy. Because the weather and user habits will make the physical device data of the previous day provide a good reference value for the current prediction, and considering that there will be some differences in the operating status of the physical devices compared to weekdays and weekends, in order to fill this gap, it is necessary to input the physical device data of the same day of each week into the prediction model to improve the prediction accuracy. Therefore, this patent selects the physical device data near the time point of the prediction day as recent data, the physical device data one day before the prediction day as daily cycle data, and the physical device data 7 days before the prediction day as weekly cycle data.

[0065] In this model, t0, t p and S represent the current time, the prediction window size, and the number of entity data per day, respectively. Take the time series of the same length before the current time in the recent data, daily cycle data, and weekly cycle data as the input of each independent component, denoted by T r , T d , T w The following is a detailed description of the three periods of data input into the model:

[0066] (1) Recent data: refers to historical data around the current time of the forecast date. Since operational anomalies of entities generally have precursors, recent data will provide important reference information for anomaly detection and forecast results.

[0067] (2) Daily cycle data: refers to the historical data of the corresponding time period one day before the current time of the forecast day. Because the operation of an entity may show consistent or similar operating status in the same period of time, this patent selects this period of data as part of the joint forecast to capture the similar characteristics of the daily cycle data and the forecast day.

[0068] (3) Weekly cycle data: refers to the historical data of the corresponding time period seven days before the current time of the forecast day. Because the operating status of the entity will change significantly between working days and weekends, selecting data of similar time periods can effectively improve the forecast accuracy. In particular, if the weekly cycle data and the forecast day data do not belong to the same time type, the data of the same time type closest to the weekly cycle data is selected.

[0069] The dual residual spatiotemporal graph attention network model based on multiple periods proposed in this patent uses a multi-component structure as the model structure for learning different periodic time patterns. The model consists of three independent component structures with the same structure, which input recent data, daily cycle data, and weekly cycle data respectively. Each independent component structure stacks the same number of spatiotemporal blocks, which enables the model to obtain the same level of spatiotemporal features and effectively fuse the component prediction results of the other two component structures in the later fusion process to obtain the final prediction result. Each independent component consists of two parts: a parallel spatiotemporal feature extraction model based on a residual network and a long time series prediction model based on a dual residual network. The step S2 further includes:

[0070] Step S21: Build a parallel spatiotemporal feature extraction model based on residual network to learn the complex dependencies of multivariate time series in feature and time dimensions.

[0071] Due to the simple stacking method, there is irreversible information loss during the network deepening process, resulting in network degradation and gradient disappearance. In addition, due to the single feature, graph neural networks may encounter the problem of over-smoothing. When the residual structure is simply superimposed, the residual does not allow the network to continue to deepen, because the residual structure lacks the richness of derived features to a large extent. In order to solve these problems, the parallel spatiotemporal feature extraction model based on the residual network proposed in this patent enables the network to resist the deepening information loss and enhance the features, capturing sufficient and rich spatiotemporal features.

[0072] The parallel spatiotemporal feature extraction model based on residual network includes a temporal convolutional network layer, a parallel feature-oriented graph attention network, a time-oriented informer network, and a residual convolution. First, the preprocessed multivariate time series data is input into the temporal convolutional network to extract the high-level features of each time series; the output of the temporal convolutional network is processed separately and simultaneously by the parallel feature-oriented graph attention network and the time-oriented informer network to capture sufficient and rich spatiotemporal features; the output of the temporal convolutional network and the output of the parallel layer are spliced ​​and fused and input into the first residual network. The feature-oriented graph attention network captures the causal relationship between multiple features, treats each univariate time series as a separate feature, and attempts to explicitly model the correlation between different features. At the same time, the time-oriented informer layer emphasizes the dependency in the time dimension and models the time dependency within each time series. The complex dependencies of multivariate time series in the feature and time dimensions are learned through this parallel network layer. The residual network is connected to the fusion layer of the parallel layer and the temporal attention layer. Ensure that this model can not only capture dynamic spatiotemporal features similar to other emerging networks, but also deepen them. The graph attention network and temporal convolutional layers provide dynamic capture capabilities and use residual convolutions to effectively prevent network degradation and gradient vanishing. In addition, the graph attention network can alleviate the effects of over-smoothing during network deepening, which cannot be alleviated by the graph neural convolutional network. In order to improve the medium- and long-term prediction capabilities of the model, the coordinated use of the attention mechanism and time series prediction can extract effective spatiotemporal features in a large receptive field. The attention mechanism filters out redundant information in different data, allowing the model to better expand the receptive field.

[0073] The following is a detailed introduction to each network model in this model:

[0074] 1. Temporal Convolutional Network (TCN)

[0075] TCN is a sequence modeling method based on convolutional neural networks, which is used for tasks such as prediction and classification of time series data. Compared with traditional recurrent neural networks (RNNs) and long short-term memory networks (LSTMs), TCN can achieve similar time series modeling capabilities without introducing a cyclic structure, while having better parallelism and computational efficiency.

[0076] The TCN model is mainly composed of a series of one-dimensional convolutional layers and pooling layers, in which each convolutional layer uses the same convolution kernel size and jump step size, but uses different numbers of convolution kernels to capture patterns in different time ranges in the input sequence. Therefore, the TCN model can model the input sequence at different time scales. Its main features are:

[0077] (1) The convolutions in the architecture are causal, which means that there is no information “leaking” from the future to the past;

[0078] (2) The architecture can take a sequence of arbitrary length and map it to an output sequence of the same length

[0079] (3) Emphasize the use of a combination of very deep networks and dilated convolutions to build very long effective history sizes.

[0080] 2. Parallel feature-oriented graph attention network and time-oriented informer network layer

[0081] This parallel network layer is used to learn the complex dependencies of multivariate time series in both feature and time dimensions.

[0082] (1) Feature-oriented Graph Attention Network

[0083] This patent uses a feature-oriented graph attention network to capture the causal relationship between multiple features, treating the multivariate time series as a complete graph, where each node represents a feature and each edge represents the relationship between two corresponding features. In this way, the relationship between adjacent nodes can be carefully captured by the graph attention operation.

[0084] Graph Attention Network (GAT) is a neural network model for graph data modeling. Compared with the traditional graph convolutional neural network (GCN), GAT uses an attention mechanism for information interaction between nodes, which can adaptively model the importance between nodes, thereby better learning the information in the graph model. GAT is a spatial domain convolutional network and does not need to be converted to the spectral domain for convolution. Furthermore, it does not rely significantly on the graph structure like GCN, but relies on the size of the graph signal value at each point and the distance between points to complete the convolution process. Therefore, it can alleviate some of the over-smoothing problems caused by the deepening of GCN, and help improve the accuracy of predicting peaks and valleys. And the attention weights of GAT change in real time according to data changes, which directly adds dynamic spatial correlation to the network.

[0085] The output of each node calculated by GAT is expressed as follows:

[0086]

[0087] Among them, v j represents the jth node in the graph model, h i is node v i The output of is represented by, σ represents the sigmoid activation function; α ij is the measure of node vj For node v i The contribution of attention score, node v j is node v i One of the adjacent nodes of node v; L represents node v i The number of neighboring nodes.

[0088] Note the fraction α ij It can be calculated by the following equation:

[0089]

[0090]

[0091] in, is the concatenation of the two node representations, w is the column vector of learnable parameters, where m is the dimension of the feature vector of each node, and LeakyReLU is the nonlinear activation function.

[0092] (2) Time-oriented Informer Network

[0093] This patent uses a time-oriented Informer network to emphasize dependencies in the time dimension and model the time dependencies within each time series.

[0094] The Transformer model proposed by Google in 2017 is far superior to the traditional RNN model in processing long time series problems. The advantage of the Transformer model is that the signal propagation path is short, avoiding the complex loop structure of the traditional RNN series network. However, the model consumes too much GPU resources and server storage resources, and requires a lot of hardware costs to be invested in model training, so it is not particularly practical for long time series prediction tasks in the real world. Therefore, a better structure is needed to improve the transformer model to improve its computing, memory and architectural efficiency, while maintaining a high prediction ability. The problems that need to be solved are: (1) secondary calculation of the self-attention mechanism; (2) memory bottleneck of long input stacking layers; (3) a sharp drop in the speed of predicting long-term outputs.

[0095] In order to solve the above problems, Informer is proposed based on the Transformer model. The following points are made to significantly improve the model in long time series prediction tasks.

[0096] (1) ProbSparse Self-attention is proposed, which can filter out all the most important parts of the query and greatly reduce the time and space complexity of the calculation.

[0097] (2) A self-attention distilling operation is proposed to reduce the dimension and the number of network parameters through convolution and pooling operations to solve the problem of privileged attention scores in the J layer and greatly reduce the overall space complexity to O(Llog(L)), which is helpful for receiving long sequence inputs.

[0098] (3) A Generation Style decoder is proposed, which can obtain all prediction results in just one step, thus improving the speed of long sequence prediction. Only one forward step is required to obtain the long sequence output, while avoiding the cumulative error propagation in the inference stage.

[0099] The current method for calculating the self-attention value is to calculate the weighted value of a query based on the input triple (query, key, value). However, this method results in excessive computation and potential numerical stability issues in the LSE operation. Therefore, the informer model uses an approximate method for measuring query sparsity and proposes a maximum mean measurement:

[0100]

[0101] Step S22: Build a long time series prediction model based on a dual residual network, use gating to highlight important information, and then globally learn the entity status of node devices according to the system's topological structure to enhance features, and deepen the network through dual residuals to improve prediction accuracy.

[0102] The long time series prediction model based on the double residual network includes a gated recurrent unit (GRU) layer, a global graph attention layer and a residual convolution. The residual convolution of the model in step S22 and the residual in step S21 together constitute a double residual convolution. The first convolution is connected to the residual network after the fusion layer, and the second residual is connected to the output of the entire independent component. The input of the GRU is the output of the first residual in step S22, which highlights important information from the long time series and solves the long dependency problem in the long time series. Then the result of the GRU is input into the global graph attention layer to learn the topological structure of the entire system. According to the collaborative relationship between the various device entities of the system, the state information of each entity is updated to reduce the judgment errors caused by the possible misreporting and omission of data in the single device entity data. Finally, the output of the global graph attention network is connected to the second residual network.

[0103] The following is a detailed introduction to each network model in this model:

[0104] 1. Gated Recurrent Unit (GRU) layer

[0105] The RNN model will have the problem of gradient vanishing when the sequence is too long, resulting in the parameters only capturing local relationships and failing to learn long-term associations. The memory ability of the words in the front of the sequence is weak. LSTM uses memory channels to alleviate the problem of gradient vanishing and failure to capture long-term associations in the RNN model to a certain extent. However, the number of LSTM parameters is too large. The GRU model ensures that fewer parameters are used to alleviate the problem of gradient vanishing and reduce the risk of overfitting.

[0106] Three gate functions are introduced in LSTM: input gate, forget gate and output gate to control input value, memory value and output value. In the GRU model, there are only two gates: update gate and reset gate. The reset gate is used to control whether the model ignores past input, and the update gate is used to control whether the model ignores current input. Through these two gating mechanisms, the GRU model can adaptively choose to retain or forget historical information, so as to have better performance when processing long sequence data. The specific formula is as follows:

[0107] z t =σ(W z ·[h t-1 , x t ])

[0108] r t =σ(W r ·[h t-1 , x t ])

[0109]

[0110]

[0111] Among them, z t and r t Represent the update gate and reset gate respectively, x t is the input at time t, h t-1 It is the hidden layer state at time t-1, which contains the relevant information of the previous node. It contains the input x at time t t The hidden layer state.

[0112] 2. Residual Network (ResNet)

[0113] Increasing the width and depth of the network can greatly improve the performance of the network. Deep networks generally perform better than shallow networks. For example, if there is a deep network A and a shallow network B, as long as all the network parameters of B are transferred to the first few layers of A, and the layers behind A are just an equivalent mapping, the performance of A can at least be the same as that of B. Because even if we, we can achieve the same effect as the B network. In short, compared with shallow networks, the performance of deeper networks should not be worse. However, in fact, deeper networks are more difficult to train.

[0114] The core idea of ​​the residual network is to make the nonlinear layer satisfy H(x, w h ), and then introduce a short connection directly from the input to the output of the nonlinear layer, so that the entire mapping becomes:

[0115] y=H(x,w h )+x

[0116] Residual is an operation in network construction. Any network that uses this operation can be called a residual network. The characteristics of this network are that it is easy to optimize and can improve accuracy by increasing the depth. The residual block inside it uses skip connections to alleviate the gradient vanishing problem caused by increasing the depth in deep neural networks.

[0117] Step S3: The multi-component model prediction results of input data of different periods are fused with a multi-period prediction result fusion algorithm to obtain the final prediction result.

[0118] The prediction results obtained by the dual residual spatiotemporal graph attention network model using recent data, daily cycle data, and weekly cycle data are independent of each other. Simply taking the average of the three results cannot directly improve the prediction accuracy. When the prediction time has a high degree of match with the historical data used, the prediction results of the corresponding model need to be highlighted. For example, the historical data at 8:00 am on a Sunday is very similar to the daily cycle data on Saturday and the weekly cycle data on last Sunday. In the fusion result, the model output of the daily cycle data and the weekly cycle data needs to be given a higher weight; if there is no obvious periodic pattern at some time points, it is necessary to give the recent data model a higher weight. Therefore, this patent proposes a multi-time period prediction result fusion algorithm to improve the prediction accuracy. In the fusion process, a tensor is used to define the fusion matrix W C , the matrix can adaptively learn component weights from data to achieve more accurate predictions. The conventional practice is to randomly initialize the fusion matrix. The multi-period prediction result fusion algorithm used in the present invention first determines the initial value of the fusion matrix to improve the training speed and prediction accuracy, so that the prediction result of the fusion matrix with the initial value after autonomous learning in the matrix is ​​better than that of the randomly initialized matrix. The specific algorithm is as follows:

[0119]

[0120] Among them, F r 、F d 、F w are the prediction results obtained using the recent data, daily cycle data and weekly cycle data models, respectively. C is the fusion matrix, W r , W d , W w F r 、F d 、F w The fusion weight parameter matrix corresponding to the prediction result, Y is the final fusion prediction result, α∈{r, d, w}, and the initial values ​​of the fusion weight parameter matrix of the prediction results of the three models are calculated respectively, A α is the historical accuracy of the model’s prediction results, R α is the Pearson similarity between the historical data and the predicted data used by the model, J is and W α A uniformly sized matrix filled with all ones.

[0121] In general, it can be understood from the above technical solutions of the present invention and its specific implementation methods that the beneficial technical effects of the present invention are as follows: the present invention provides a multi-period time series anomaly detection method based on a spatiotemporal graph neural network, establishes an outlier adjustment algorithm based on density distribution, and effectively handles the key information missing or anomaly of the device entity data; constructs a multi-period dual residual spatiotemporal graph attention network model, including a parallel spatiotemporal feature extraction model based on a residual network and a long time series prediction model based on a dual residual network, which improves the timeliness and accuracy of system anomaly detection and effectively assists users in system maintenance; proposes a multi-period prediction result fusion algorithm, sets initial values ​​and adaptively learns component weights to achieve more accurate predictions. Its advantages over the prior art are:

[0122] (1) The present invention proposes an outlier adjustment algorithm based on density distribution. Outliers may have a significant impact on the accuracy and efficiency of the model. The present invention can better improve data quality, identify potential problems in the data, reduce erroneous analysis and decision-making caused by outliers, and improve the efficiency of data analysis, thereby improving the accuracy and performance of the model.

[0123] (2) This paper proposes a dual-residual spatiotemporal graph attention network model based on multiple periods. Compared with the traditional model, the network can resist the deepening of information loss and enhance features, capturing sufficient and rich spatiotemporal features. The input of multivariate historical periodic data increases the generalization ability of the model, fully considers the periodicity of the system operation status, and reduces false positives caused by periodic changes.

[0124] (3) The present invention proposes a multi-period prediction result fusion algorithm. When the prediction time has a high degree of match with the historical data used, it is necessary to highlight the prediction results of the corresponding model. The algorithm first determines the initial value of the fusion matrix and adaptively learns the component weights from the data, achieving more accurate predictions and faster training speeds compared to randomly initialized matrices.

[0125] (4) The present invention has broad application prospects in various network-based systems such as power grid fault analysis and network equipment operation status monitoring.

[0126] The above is only part of the embodiments of the present invention. It should be noted that, except for the technical features described in the specification, all other technical features are known to those skilled in the art. For those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention.

Claims

1. A multi-period time series anomaly detection method based on spatiotemporal graph neural network, characterized in that: The steps include: Step S1: construct a graph model of the cooperating equipment entities, collect the multivariate time series of each equipment entity and perform data standardization and density distribution-based outlier adjustment on each individual time series; Step S2: construct a dual residual spatiotemporal graph attention network model based on multiple periods, input the preprocessed multivariate time series data to obtain multi-component prediction results based on different historical period data; Step S3: The prediction results of the multi-component model with different period data are inputted using a multi-period prediction result fusion algorithm to obtain the final prediction result.

2. The method for detecting anomalies in multi-period time series based on spatiotemporal graph neural network according to claim 1 is characterized in that: The formula of the outlier adjustment algorithm based on density distribution is as follows: Among them, x i is the data at the i-th time point in a time series, M() is the operation expression for taking the median, R is the tolerance distance for outlier detection, x′ is the data with the largest value in the top 25% of the time series data x, |x′| is the number of data in x′, and r is the value of the 75th% of the time series data x in ascending order.

3. The method for detecting anomalies in multi-period time series based on spatiotemporal graph neural network according to claim 1 is characterized in that: The outlier adjustment algorithm based on density distribution includes: The multi-period dual residual spatiotemporal graph attention network model uses a multi-component structure as the model structure for learning different periodic time patterns. The model consists of three independent component structures with the same structure, which input recent data, daily periodic data, and weekly periodic data respectively. Each independent component structure stacks the same number of spatiotemporal blocks, which enables the model to obtain the same level of spatiotemporal features and effectively fuse the component prediction results of the other two component structures in the later fusion process to obtain the final prediction result. Each independent component consists of two parts: a parallel spatiotemporal feature extraction model based on a residual network and a long time series prediction model based on a dual residual network. Recent data is a common historical data used by almost all prediction models. However, the physical devices in the system often have different strong periodicities in time patterns. This periodic data is very helpful in improving prediction accuracy. Because the weather and user habits will make the physical device data of the previous day provide a good reference value for the current prediction, and considering that there will be some differences in the operating status of the physical devices compared to weekdays and weekends, in order to fill this gap, it is necessary to input the physical device data of the same day of each week into the prediction model to improve the prediction accuracy. Therefore, this patent selects the physical device data near the time point of the prediction day as recent data, the physical device data one day before the prediction day as daily cycle data, and the physical device data 7 days before the prediction day as weekly cycle data. In this model, t0, t p and S represent the current time, the prediction window size, and the number of entity data per day, respectively. Take the time series of the same length before the current time in the recent data, daily cycle data, and weekly cycle data as the input of each independent component, denoted by T r , T d , T w The following is a detailed description of the three periods of data input into the model: (1) Recent data: refers to historical data around the current time of the forecast date. Since operational anomalies of entities generally have precursors, recent data will provide important reference information for anomaly detection and forecast results. (2) Daily cycle data: refers to the historical data of the corresponding time period one day before the current time of the forecast day. Because the operation of an entity may show consistent or similar operating status in the same period of time, this patent selects this period of data as part of the joint forecast to capture the similar characteristics of the daily cycle data and the forecast day. (3) Weekly cycle data: refers to the historical data of the corresponding time period seven days before the current time of the forecast day. Because the operating status of the entity will change significantly between working days and weekends, selecting data of similar time periods can effectively improve the forecast accuracy. In particular, if the weekly cycle data and the forecast day data do not belong to the same time type, the data of the same time type closest to the weekly cycle data is selected.

4. The method for detecting anomalies in multi-period time series based on spatiotemporal graph neural network according to claim 3 is characterized in that: A parallel spatiotemporal feature extraction model based on residual networks, including: The parallel spatiotemporal feature extraction model based on residual network includes a temporal convolutional network layer, a parallel feature-oriented graph attention network, a time-oriented informer network, and a residual convolution. First, the preprocessed multivariate time series data is input into the temporal convolutional network to extract the high-level features of each time series; the output of the temporal convolutional network is processed separately and simultaneously by the parallel feature-oriented graph attention network and the time-oriented informer network to capture sufficient and rich spatiotemporal features; the output of the temporal convolutional network and the output of the parallel layer are spliced ​​and fused and input into the first residual network. The feature-oriented graph attention network captures the causal relationship between multiple features, treats each univariate time series as a separate feature, and attempts to explicitly model the correlation between different features. At the same time, the time-oriented informer layer emphasizes the dependency in the time dimension and models the time dependency within each time series. The complex dependencies of multivariate time series in the feature and time dimensions are learned through this parallel network layer. The residual network is connected to the fusion layer of the parallel layer and the temporal attention layer. Ensure that this model can not only capture dynamic spatiotemporal features similar to other emerging networks, but also deepen them. The graph attention network and the temporal convolution layer provide dynamic capture capabilities and use residual convolution to effectively prevent network degradation and gradient vanishing. In addition, the graph attention network can alleviate the effect of over-smoothing during network deepening, while the graph neural convolution network cannot alleviate this problem. In order to improve the medium- and long-term prediction capabilities of the model, the coordinated use of the attention mechanism and time series prediction can extract effective spatiotemporal features in a large receptive field. The attention mechanism filters out redundant information in different data, allowing the model to better expand the receptive field. Due to the simple stacking method, there is irreversible information loss during the network deepening process, resulting in network degradation and gradient disappearance. In addition, due to a single feature, the graph neural network may encounter the problem of over-smoothing. When the residual structure is simply superimposed, the residual does not allow the network to continue to deepen, because the residual structure largely lacks the richness of derived features. In order to solve these problems, the parallel spatiotemporal feature extraction model based on the residual network proposed in this patent enables the network to resist the information loss of deepening and enhance the features, capturing sufficient and rich spatiotemporal features.

5. The method for detecting anomalies in multi-period time series based on spatiotemporal graph neural network according to claim 3 is characterized in that: Long time series prediction model based on dual residual network, including: The long time series prediction model based on the double residual network includes a gated recurrent unit (GRU) layer, a global graph attention layer and a residual convolution. The residual convolution of the model in step S22 and the residual in step S21 together constitute a double residual convolution. The first convolution is connected to the residual network after the fusion layer, and the second residual is connected to the output of the entire independent component. The input of the GRU is the output of the first residual in step S22, which highlights important information from the long time series and solves the long dependency problem in the long time series. Then the result of the GRU is input into the global graph attention layer to learn the topological structure of the entire system. According to the collaborative relationship between the various device entities of the system, the state information of each entity is updated to reduce the judgment errors caused by the possible misreporting and omission of data in the single device entity data. Finally, the output of the global graph attention network is connected to the second residual network.

6. The method for detecting anomalies in multi-period time series based on spatiotemporal graph neural network according to claim 1 is characterized in that: The multi-period prediction result fusion algorithm has the following formula: Among them, F r 、F d 、F w are the prediction results obtained using the recent data, daily cycle data and weekly cycle data models, respectively. C is the fusion matrix, W r , W d , W w F r 、F d 、F w The fusion weight parameter matrix corresponding to the prediction result, Y is the final fused prediction result, α∈{r, d, w}, and the initial values ​​of the fusion weight parameter matrix of the prediction results of the three models are calculated respectively, A α is the historical accuracy of the model’s prediction results, R α is the Pearson similarity between the historical data and the predicted data used by the model, J is and W α A uniformly sized matrix filled with all ones.

Citation Information

Patent Citations

  • Multi-dimensional time sequence prediction method based on self-attention mechanism and graph convolutional network

    CN114818515A

  • Transform anomaly detection method based on spatio-temporal characteristics

    CN115618196A

Cited By

  • Time sequence prediction method and device based on period embedding and multi-scale features

    CN119622319A

  • Border gateway protocol anomaly detection method and device and computer equipment

    CN119788578A

  • A method, apparatus, and computer device for detecting anomalies in a border gateway protocol.

    CN119788578B

  • Protein interaction prediction method based on hybrid graph attention neural network

    CN119889427A

  • Protein interaction prediction method based on hybrid graph attention neural network

    CN119889427B