Industrial Internet of Things anomaly detection method based on space-time variation auto-encoder
Through the spatial and temporal variational autoencoder and supervised comparison learning method, the time series data spatial information of industrial IoT networks is extracted, which solves the problem of large resource requirements and overfitting of models, improves detection accuracy and stability, and enhances the system Security.
Patent Information
- Application Number
- CN202411091403.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-11-22
AI Technical Summary
The existing industrial IoT network anomaly detection methods have problems such as large resource requirements, model overfitting and unknown characteristics, which affect detection accuracy and system security.
Using a method based on spatiotemporal variational autoencoder, the spatial information of time series data is extracted through the Fast-DTW multi-head graph attention network, combined with the TCN-VAE model of supervised comparison learning, the reconstruction loss, KL divergence and supervised comparison loss are used. Perform training, calculate the fuzzy entropy weighted sum to perform abnormal detection.
It improves the accuracy and stability of abnormal detection of industrial IoT networks, reduces resource requirements, avoids model overfitting, and enhances the security of the system.
Smart Images

Figure CN119004328A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer technology and relates to an industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder. Background Art
[0002] The Industrial Internet of Things is considered an emerging technology due to its great advantages in bringing ubiquitous intelligent connectivity and automation into industrial production processes. The Industrial Internet of Things can build powerful industrial systems and provide diversified industrial services by combining with a wide variety of sensors. However, usually, a large number of Industrial Internet of Things devices with limited computing and storage capabilities are widely distributed in locations with poor security protection and use diverse function-oriented communication protocols. The limitations of the above situation have weakened the security of Industrial Internet of Things devices to a certain extent. The network layer is designed to transmit network data packets between heterogeneous networks transmitted by various Industrial Internet of Things devices. Monitoring the performance indicators of the Industrial Internet of Things network is helpful for anomaly detection and troubleshooting. In order to reduce the cost of system accidents, the first task is to detect anomalies on these indicators. Since the collected measurement data belongs to time series data, an efficient and accurate time series anomaly detection algorithm is required.
[0003] Because the parallel calculation and superposition of multi-head graph attention networks result in a large amount of calculation and occupy a large amount of memory, it is limited in the industrial Internet of Things scenario, affecting its effect in practical applications. Secondly, if appropriate regularization is not applied, the model based on the variational autoencoder will suffer from overfitting problems. In the process of generating anomaly scores, the sum of the differences on each feature is usually used, and the sum has equal weight for each feature. Obviously, this is not reasonable. In some extreme cases, the abnormality of a sample can be judged by the generation errors of several category features.
[0004] In general, existing IIoT network anomaly detection methods generally have problems such as large resource requirements, model overfitting, and feature importance. Summary of the invention
[0005] In view of this, the purpose of the present invention is to provide an industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder, aiming to solve the problems of large resource requirements, model overfitting, and feature importance in training, while utilizing the correlation between industrial Internet of Things network features to effectively improve the accuracy of anomaly detection.
[0006] In order to achieve the above object, the present invention provides the following technical solutions:
[0007] An industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder comprises the following steps:
[0008] S1. Extracting key performance indicator time series data from the IIoT network;
[0009] S2, capturing spatial information in time series data through a multi-head graph attention network correlation extraction model based on Fast-DTW;
[0010] S3. Use the TCN-VAE anomaly detection model based on supervised contrastive learning to learn the normal features of the data, and use spatiotemporal information to improve the anomaly detection model's ability to learn data features;
[0011] S4. Construct the supervised contrast loss, combine the reconstruction loss and KL divergence to define the total loss function, and train the TCN-VAE anomaly detection model based on the total loss function;
[0012] S5. Calculate the anomaly score by weighted summation of fuzzy entropy, and judge the network status according to the anomaly score to complete the detection.
[0013] Further, in step S1, the extracted key performance indicator time series data is multivariate time series data, and the multivariate time series data includes N univariate time series of T timestamps, which are expressed as:
[0014]
[0015] Among them, at timestamp t (t≤T), Represents the value of the i-th sensor at time t.
[0016] Further, in step S2, the following steps are included:
[0017] S21. Multivariate time series data Scale to the interval [0,1];
[0018] S22, the original multivariate time series data X generates the adjacency matrix A through the Fast-WDT module;
[0019] S23. Input the scaled multivariate time series data X′ and the adjacency matrix A into the multi-head attention network to fuse the correlation information between the nodes and output the hidden representation of the time series.
[0020] Further, in step S21, the time series data is normalized using the maximum and minimum values in the training data:
[0021]
[0022] Among them, x' represents the scaled time series, max(X) and min(X) are the maximum and minimum values of the corresponding time series in the training set, respectively.
[0023] Further, in step S22, in the Fast-DTW module, each feature in the scaled data is regarded as a separate node, and the adjacency matrix is output where a ij represents the edge from node i to node j, where
[0024] First, the original multivariate time series data X is coarse-grained, that is, the data extraction operation is performed iteratively multiple times to obtain multiple fine-grained data points, and the multiple fine-grained data points are averaged to obtain coarse-grained data;
[0025] Then, we apply the DTW algorithm to the coarse-grained data, restore the output of the DTW algorithm, and obtain the similarity distance of the two time series data:
[0026] D i,j =Fast_DTW(x i ,x j )
[0027] Where D i,j represents the distance between the time series of the i-th variable and the time series of the j-th variable, and Fast_DTW(·) means using Fast-DTW to calculate the DTW distance between two time series;
[0028] D i,j The larger the variable x is, the i and x j The smaller the correlation between them; set the correlation judgment threshold θ, if the D between two features i,j is higher than the discrimination threshold, then the variable x i and x j Connection, that is:
[0029]
[0030] The sparsity of the graph can be controlled by controlling the size of the threshold θ.
[0031] Further, in step S23, the normalized multivariate time series data X′ and the adjacency matrix A are input into the multi-head graph attention network for spatial correlation fusion, and the graph attention layer GAT calculates the output representation of each node as follows:
[0032]
[0033] where h i represents the output representation of node i, which is consistent with the input x i have the same shape; σ represents the sigmoid activation function; α ijis the attention weight used to measure the contribution of node j to node i, where node j is one of the neighboring nodes of node i; L represents the number of neighboring nodes of node i;
[0034] Attention weight α ij Calculated by the following formula:
[0035]
[0036]
[0037] in, represents the concatenation of two node representations, ω∈R 2m is a column vector of learnable parameters, where m is the dimension of the feature vector of each node, (·) T represents the transpose of the vector, LeakyReLU() is a nonlinear activation function; e ij The attention score representing the contribution of node j to node i, e ij′ Represents the attention score of node j''s contribution to node i, and softmax() is a normalized exponential function;
[0038] After cascading the feature vectors calculated by the K-head attention mechanism, the corresponding output feature vector is expressed as:
[0039]
[0040] Where Π represents vector concatenation; K represents the number of attention heads; is the attention weight calculated by the kth attention mechanism head; w k is the parameter matrix of the linear transformation of the input vector, h j is the output of the graph attention layer of the neighboring node j of node i;
[0041] Using the average rule, the output of the last layer calculated by the multi-head attention mechanism is expressed as:
[0042]
[0043] Further, in step S3, the process of reconstructing the data using the TCN-VAE anomaly detection model based on supervised contrastive learning includes the following steps:
[0044] S31. Establish a TCN-VAE anomaly detection model, wherein the TCN-VAE anomaly detection model includes at least one encoder, a decoder, a random perturbation module and a discriminator, the encoder and the decoder are both composed of a temporal convolutional network TCN, and the discriminator is composed of a multi-layer perceptron.
[0045] S32, the time series of the fusion space is sent to the time convolution network TCN encoder to obtain the hidden representation of the time series, and a TCN network is formed by stacking multiple residual blocks, which is expressed as:
[0046]
[0047] Where Z represents the hidden representation of the final output, A hidden representation representing the fused feature correlation information of the multi-head graph attention output;
[0048] S33, input the potential representation Z into the TCN decoder for data reconstruction, and obtain the reconstructed data
[0049] Further, in step S4, the total loss function includes reconstruction loss, KL divergence loss and supervised contrast loss, which is expressed as:
[0050]
[0051] in, is the reconstruction loss, is the KL divergence loss, is the supervised contrast loss, λ1 and λ2 are the weights of KL divergence loss and supervised contrast loss, respectively.
[0052] Based on the reconstructed data And the data after integrating spatial information Calculate the reconstruction loss and use the mean square error to represent the reconstruction loss:
[0053]
[0054] Where N is the total number of dimensions, T is the total time length, is the reconstructed data of dimension i at time t, is the original data of dimension i at time t.
[0055] The KL divergence loss is used to measure the distance between the Gaussian distribution learned by the encoder and the true Gaussian distribution, which is expressed as:
[0056]
[0057] where q θ (Z|H) is the posterior distribution of the encoder-generated input data H, is the standard normal distribution, D KL [·] denotes KL divergence loss.
[0058] The potential representations generated by the TCN encoder in the same batch are marked as positive samples, random perturbations are added to the potential representations to generate negative samples, and the positive and negative samples are used to generate supervised contrastive learning losses:
[0059]
[0060] Where Z i is the potential representation generated at the anchor point, Z p is the potential representation of the same batch except the anchor point, Z k is the potential representation after random perturbation, p(i) represents the sample set from the same class as the currently processed sample in the batch, |p(i)| represents the number of samples in the set, and K(i) represents the probability distribution of the sample set with Z k A collection of different negative pairs, τ is the temperature coefficient.
[0061] Further, in step S5, in the detection stage, the fuzzy entropy of different features is first calculated:
[0062]
[0063] in is the fuzzy entropy of the i-th feature at time point t, FuzzyEn(·) is the fuzzy entropy calculation function, r is the similarity tolerance, and N is the sequence length;
[0064] The weight of each feature is normalized to [0,1], and the sum of all values is 1, as shown in the following formula:
[0065]
[0066] The error on each feature dimension is weighted summed to obtain the anomaly score:
[0067]
[0068] In the detection stage, the detection rule is that if the anomaly score of a timestamp is greater than the anomaly threshold, the timestamp is marked as "abnormal", otherwise it is marked as "normal", where the anomaly threshold is set using the streaming peak over-threshold method.
[0069] The beneficial effects of the present invention are:
[0070] The present invention adopts the TCN-VAE model based on supervised contrastive learning to perform anomaly detection on the industrial Internet of Things network, which solves the problems of large resource requirements, easy overfitting of the model, and feature importance. The generator and encoder are replaced with a temporal convolutional network, which fully considers the feature correlation information and time information in the industrial Internet of Things, improves the detection accuracy and stability, and thus enhances the security of the industrial Internet of Things.
[0071] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0072] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:
[0073] Figure 1 The overall architecture diagram of the industrial Internet of Things anomaly detection method of the present invention is shown;
[0074] Figure 2 It is a structural schematic diagram of the TCN-VAE anomaly detection model based on supervised contrastive learning of the present invention;
[0075] Figure 3 Schematic diagram of the iterative process of training the anomaly detection model of the present invention. DETAILED DESCRIPTION
[0076] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and the features in the embodiments can be combined with each other without conflict.
[0077] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.
[0078] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.
[0079] See also Figure 1 to Figure 3 , which is an anomaly detection method for industrial Internet of Things based on spatiotemporal variational autoencoders.
[0080] Example
[0081] like Figure 1 The overall architecture diagram of the industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder of the present invention is shown, which includes the following steps:
[0082] S1: Extract the time series data of key performance indicators on the network in the industrial Internet of Things and build training sets and test sets;
[0083] S2: Design a multi-head graph attention network correlation extraction model based on Fast-DTW to capture the spatial information existing in the industrial Internet of Things network dataset and improve the accuracy of anomaly detection;
[0084] S3: Since the autoencoder-based model is prone to overfitting and cannot capture the long-term temporal dependencies in the time series, the TCN-VAE anomaly detection model based on supervised contrastive learning is used to learn the normal features of the data, and the spatiotemporal information is used to improve the learning ability of the anomaly detection model for data features;
[0085] S4: Define the supervised contrast loss, and then define the total loss function by combining the reconstruction loss and KL divergence. Input the training set into the anomaly detection model for training to learn the data distribution of normal data.
[0086] S5: Design a fuzzy entropy weighted sum anomaly score to judge the network status, and then complete the anomaly detection of the industrial Internet of Things network.
[0087] In step S1 of this embodiment, the extracted key performance indicator time series data is multivariate time series data, which is composed of N univariate time series with T timestamps, that is, At timestamp t (t≤T), in represents the value of the i-th sensor at time t. In the detection stage, the goal of the present invention is to detect a multivariate time series with a length of T' from N univariate time series. The output of the final model is where y i ∈{0,1}, when y i =1 indicates the observation result at time t is abnormal.
[0088] In step S2 of this embodiment, a multi-head graph attention network correlation extraction model based on Fast-DTW is designed to capture multivariate time series data on the industrial Internet of Things network. The spatial information existing in the data includes the following steps:
[0089] S21. Multivariate time series data Scale to the interval [0,1]. In this embodiment, specifically, the maximum and minimum values in the training data are used to normalize the time series data:
[0090]
[0091] Among them, x' represents the scaled time series, max(X) and min(X) are the maximum and minimum values of the corresponding time series in the training set, respectively.
[0092] S22, the original multivariate time series data X generates an adjacency matrix A through the Fast-WDT (Fast Dynamic Time Warping) module. In this embodiment, specifically, the original data is input into Fast-DTW, its relevance is measured, and each feature in the original data is regarded as a separate node, and the adjacency matrix is output where a ij Represents the edge from node i to node j. Fast-DTW first coarsens the original sequence, that is, the data extraction is performed multiple times in iterations of 1 / 8→1 / 4→1 / 2→1 / 1, and then the corresponding multiple fine-grained data points are averaged to obtain coarse-grained data. Then the DTW algorithm is applied to the coarse-grained data, and finally the coarse-grained data is restored to obtain the similarity distance of the two time series.
[0093] D i,j =Fast_DTW(x i ,x j )
[0094] Where D i,jrepresents the distance between the time series of the i-th variable and the time series of the j-th variable, and Fast_DTW(·) represents the use of Fast-DTW to calculate the DTW distance between the two time series. i,j The larger the variable x is, the i and x j The smaller the correlation between them. Set the correlation discrimination threshold θ. If the DTW between two features is higher than the discrimination threshold, the variable x i and x j Connect. That is:
[0095]
[0096] By controlling the size of the threshold θ, the sparsity of the graph can be controlled, ensuring the computing speed and low resource usage while ensuring the accuracy of anomaly detection.
[0097] S23, the scaled multivariate time series data X' and the adjacency matrix A are input into the multi-head attention network to fuse the correlation information between the nodes, and the hidden representation of the time series is output. In this embodiment, specifically, after the graph structure is generated, the normalized multivariate time series data X' and the adjacency matrix A are input into the multi-head graph attention network for spatial correlation fusion, and the graph attention layer GAT calculates the output representation of each node as follows:
[0098]
[0099] where h i represents the output representation of node i, which is consistent with the input x i have the same shape; σ represents the sigmoid activation function; α ij is the attention weight used to measure the contribution of node j to node i, where node j is one of the neighboring nodes of node i; L represents the number of neighboring nodes of node i.
[0100] Attention weight α ij It can be calculated by the following equation:
[0101]
[0102]
[0103] here represents the concatenation of two node representations, ω∈R 2m is a column vector of learnable parameters, where m is the dimension of the feature vector of each node, LeakyReLU() is a nonlinear activation function, and e ij The attention score representing the contribution of node j to node i, e ij′The attention score representing the contribution of node j' to node i, and softmax() is a normalization function.
[0104] The process of applying the multi-head attention mechanism is to cascade the feature vectors calculated by the K-head attention mechanism, and the corresponding output feature vector is expressed as:
[0105]
[0106] Where Π represents vector concatenation; K represents the use of K attention heads to calculate the attention score; is the attention weight calculated by the kth attention mechanism head; w k is the parameter matrix of the linear transformation of the input vector. Therefore, using the average rule, the output of the last layer calculated by the multi-head attention mechanism is expressed as:
[0107]
[0108] In step S3 of this embodiment, the TCN-VAE anomaly detection model based on supervised contrastive learning is used to reconstruct the data.
[0109] S31. First, in this embodiment, the design is as follows Figure 2 The anomaly detection model of TCN-VAE based on supervised contrastive learning shown in the figure consists of an encoder, a decoder, a random perturbation module and a discriminator, where the encoder and decoder are composed of TCN modules, and the discriminator is composed of a multi-layer perceptron.
[0110] S32. Then, the time series of the fused space is sent to the time convolution network TCN encoder to obtain the hidden representation of the time series, and a TCN network is formed by stacking multiple residual blocks, which is expressed as:
[0111]
[0112] Where Z represents the hidden representation of the final output, Represents the time series data of the fused feature correlation information of the multi-head graph attention output. h' n The output data of the nth feature dimension representing the correlation of fused features.
[0113] S33, input the potential representation Z into the TCN decoder for data reconstruction, and obtain the reconstructed data It is expressed as:
[0114]
[0115] In step S4 of this embodiment, a loss function is constructed for the above-mentioned anomaly detection model to train it, specifically including defining a supervised contrast loss, and then combining the reconstruction loss and the KL divergence to define a total loss function.
[0116] S41. Based on the reconstructed data And the data after integrating spatial information Calculate the reconstruction loss, and calculate the divergence loss by measuring the KL divergence between the potential representation Z and the normal distribution. In this embodiment, specifically, the overall loss function is set to consist of three parts: reconstruction loss, KL divergence loss, and supervised contrast loss. The reconstruction loss is used to measure the distance between the input data and the generated data, and the mean square error is used to represent the reconstruction loss:
[0117]
[0118] Where N is the total number of dimensions, T is the total time length, is the reconstructed data of dimension i at time t, is the original data of dimension i at time t.
[0119] The KL divergence loss is used to measure the distance between the Gaussian distribution learned by the encoder and the true Gaussian distribution.
[0120]
[0121] where q θ (Z|H) is the posterior distribution of the encoder-generated input data H, is a standard normal distribution
[0122] S42. The latent representation generated by the TCN encoder in a batch is labeled as a positive sample, random perturbations are added to the latent representation to generate negative samples, and supervised contrastive learning loss is generated using these two types of data samples. In this embodiment, specifically, for each anchor point, the SupCon loss first calculates the similarity score between it and all other similar ones, and then weighted sums these scores. Since anomaly detection is usually unsupervised learning, there is no label information available. Therefore, this embodiment performs pseudo-labeling on the latent representation, that is, the latent representation of the same batch of time series generated by the VAE encoder is labeled as a positive sample Z, and then the original generated latent representation is randomly perturbed with noise, and the disturbed latent representation is regarded as a negative sample.
[0123]
[0124] Where Z i is the potential representation generated at the anchor point, Z p is the potential representation of the same batch except the anchor point, Z kis the potential representation after random perturbation, p(i) represents the sample set from the same class as the currently processed sample in the batch, |p(i)| represents the number of samples in the set, and K(i) represents the probability distribution of the sample set with Z k A collection of different negative pairs. τ is the temperature coefficient, which is an adjustable coefficient. The smaller it is, the more attention is paid to distinguishing the sample from the most similar samples.
[0125] S43, the total loss function is obtained by weighted summation of reconstruction loss, divergence loss, and supervised contrast loss. The total loss function is shown as follows:
[0126]
[0127] Where λ1 and λ2 are the weights of KL divergence loss and supervised contrast loss respectively. The loss function is used to update the parameters of the anomaly detection model. The training set and the total loss function are used to optimize the parameters of TCN-VAE.
[0128] In step S5 of this embodiment, the network status is judged by designing a fuzzy entropy weighted sum anomaly score, thereby completing anomaly detection of the industrial Internet of Things network. Specifically, Figure 3 The anomaly detection model training iterative steps shown in the figure first calculates the fuzzy entropy of different features in the detection stage.
[0129]
[0130] in is the fuzzy entropy of the i-th feature at time point t, FuzzyEn(·) is the fuzzy entropy calculation function, r is the similarity tolerance, and N is the sequence length. Assuming that each feature has a uniformly distributed value range, a feature with the greatest uncertainty will provide more abnormal possibilities. Therefore, we should pay more attention to this feature and give it a larger weight in the anomaly score. Normalize the weight of each feature to [0,1], and the sum of all values is 1. As shown in the formula:
[0131]
[0132] The different importance of different variables to anomaly detection is obtained from the fuzzy entropy. The error on each feature dimension is weighted and summed to obtain the anomaly score as follows:
[0133]
[0134] In the detection phase, the detection rule is that if the anomaly score of a timestamp is greater than the defined anomaly threshold, the timestamp is marked as "abnormal", otherwise it is marked as "normal". The streaming peak over-threshold method is applied. This is a method based on extreme value theory for setting thresholds, which can automatically select appropriate thresholds for time series. When the anomaly score is greater than the threshold, the data is detected as abnormal and an alarm is issued.
[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.
Claims
1. An industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder, characterized by: It includes the following steps: S1. Extracting key performance indicator time series data from the IIoT network; S2, capturing spatial information in time series data through a multi-head graph attention network correlation extraction model based on Fast-DTW; S3. Use the TCN-VAE anomaly detection model based on supervised contrastive learning to learn the normal features of the data, and use spatiotemporal information to improve the anomaly detection model's ability to learn data features; S4. Construct the supervised contrast loss, combine the reconstruction loss and KL divergence to define the total loss function, and train the TCN-VAE anomaly detection model based on the total loss function; S5. Calculate the anomaly score by weighted summation of fuzzy entropy, and judge the network status according to the anomaly score to complete the detection.
2. The industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder according to claim 1 is characterized in that: In step S1, the extracted key performance indicator time series data is multivariate time series data, which includes N univariate time series of T timestamps, expressed as: Where, at timestamp t, t≤T, Represents the value of the i-th sensor at time t.
3. The industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder according to claim 1 is characterized in that: In step S2, the following steps are included: S21. Multivariate time series data Scale to the interval [0,1]; S22, the original multivariate time series data X generates the adjacency matrix A through the Fast-WDT module; S23. Input the scaled multivariate time series data X′ and the adjacency matrix A into the multi-head attention network to fuse the correlation information between the nodes and output the hidden representation of the time series.
4. The industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder according to claim 3 is characterized in that: In step S21, the time series data is normalized using the maximum and minimum values in the training data: Among them, x' represents the scaled time series, max(X) and min(X) are the maximum and minimum values of the corresponding time series in the training set, respectively.
5. The industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder according to claim 3 is characterized in that: In step S22, in the Fast-DTW module, each feature in the scaled data is treated as a separate node, and the adjacency matrix is output where a ij represents the edge from node i to node j, where First, the original multivariate time series data X is coarse-grained, that is, the data extraction operation is performed iteratively multiple times to obtain multiple fine-grained data points, and the multiple fine-grained data points are averaged to obtain coarse-grained data; Then, we apply the DTW algorithm to the coarse-grained data, restore the output of the DTW algorithm, and obtain the similarity distance of the two time series data: D i,j =Fast_DTW(x i ,x j ) Where D i,j represents the distance between the time series of the i-th variable and the time series of the j-th variable, and Fast_DTW(·) means using Fast-DTW to calculate the DTW distance between two time series; D i,j The larger the variable x is, the i and x j The smaller the correlation between them; set the correlation judgment threshold θ, if the D between two features i,j is higher than the discrimination threshold, then the variable x i and x j Connection, that is: The sparsity of the graph can be controlled by controlling the size of the threshold θ.
6. The industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder according to claim 3 is characterized in that: In step S23, the normalized multivariate time series data X′ and the adjacency matrix A are input into the multi-head graph attention network for spatial correlation fusion. The graph attention layer GAT calculates the output representation of each node as follows: where h i represents the output representation of node i, which is consistent with the input x i have the same shape; σ represents the sigmoid activation function; α ij is the attention weight used to measure the contribution of node j to node i, where node j is one of the neighboring nodes of node i; L represents the number of neighboring nodes of node i; Attention weight α ij Calculated by the following formula: in, represents the concatenation of two node representations, ω∈R 2m is a column vector of learnable parameters, where m is the dimension of the feature vector of each node, (·) T represents the transpose of the vector, LeakyReLU() is a nonlinear activation function; e ij The attention score representing the contribution of node j to node i, e ij′ Represents the attention score of node j''s contribution to node i, and softmax() is a normalized exponential function; After cascading the feature vectors calculated by the K-head attention mechanism, the corresponding output feature vector is expressed as: Where Π represents vector concatenation; K represents the number of attention heads; is the attention weight calculated by the kth attention mechanism head; w k is the parameter matrix of the linear transformation of the input vector, h j is the output of the graph attention layer of the neighboring node j of node i; Using the average rule, the output of the last layer calculated by the multi-head attention mechanism is expressed as:
7. The industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder according to claim 1 is characterized in that: In step S3, the process of reconstructing the data using the TCN-VAE anomaly detection model based on supervised contrastive learning includes the following steps: S31. Establish a TCN-VAE anomaly detection model, wherein the TCN-VAE anomaly detection model includes at least one encoder, one decoder, a random perturbation module and a discriminator, the encoder and the decoder are both composed of a temporal convolutional network TCN, and the discriminator is composed of a multi-layer perceptron; S32, the time series of the fusion space is sent to the time convolution network TCN encoder to obtain the hidden representation of the time series, and a TCN network is formed by stacking multiple residual blocks, which is expressed as: Where Z represents the hidden representation of the final output, A hidden representation representing the fused feature correlation information of the multi-head graph attention output; S33, input the potential representation Z into the TCN decoder for data reconstruction, and obtain the reconstructed data 8. The industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder according to claim 1 is characterized in that: In step S4, the total loss function includes reconstruction loss, KL divergence loss and supervised contrast loss, which is expressed as: in, is the reconstruction loss, is the KL divergence loss, is the supervised contrast loss, λ1 and λ2 are the weights of KL divergence loss and supervised contrast loss respectively; Based on the reconstructed data And the data after integrating spatial information Calculate the reconstruction loss and use the mean square error to represent the reconstruction loss: Where N is the total number of dimensions, T is the total time length, is the reconstructed data of dimension i at time t, is the original data of dimension i at time t; The KL divergence loss is used to measure the distance between the Gaussian distribution learned by the encoder and the true Gaussian distribution, which is expressed as: where q θ (Z|H) is the posterior distribution of the encoder-generated input data H, is the standard normal distribution, D KL [·] represents KL divergence; The potential representations generated by the TCN encoder in the same batch are marked as positive samples, random perturbations are added to the potential representations to generate negative samples, and the positive and negative samples are used to generate supervised contrastive learning losses: Where Z i is the potential representation generated at the anchor point, Z p is the potential representation of the same batch except the anchor point, Z k is the potential representation after random perturbation, p(i) represents the sample set from the same class as the currently processed sample in the batch, |p(i)| represents the number of samples in the set, and K(i) represents the probability distribution of the sample set with Z k A collection of different negative pairs, τ is the temperature coefficient.
9. The industrial Internet of Things anomaly detection method based on spatiotemporal variational autoencoder according to claim 1 is characterized in that: In step S5, during the detection phase, the fuzzy entropy of different features is first calculated: in is the fuzzy entropy of the i-th feature at time point t, FuzzyEn(·) is the fuzzy entropy calculation function, r is the similarity tolerance, and N is the sequence length; The weight of each feature is normalized to [0,1], and the sum of all values is 1, as shown in the following formula: The error on each feature dimension is weighted summed to obtain the anomaly score: In the detection stage, the detection rule is that if the anomaly score of a timestamp is greater than the anomaly threshold, the timestamp is marked as "abnormal", otherwise it is marked as "normal". The anomaly threshold is set using the streaming peak over-threshold method.
Citation Information
Patent Citations
Abnormity detection method based on heterogeneous information network element learning architecture
CN114239712A
Anomaly detection device, anomaly detection method, anomaly detection program, and base model
JP2022121058A
Cited By
Abnormal traffic detection network training method, abnormal traffic detection method and device
CN120389910A
Abnormal traffic detection network training method, abnormal traffic detection method and device
CN120389910B
