Industrial Internet of Things system anomaly detection method based on multivariable time series data

Abstract

The invention discloses an industrial Internet of Things system anomaly detection method based on multivariable time series data. The method comprises the following steps: S1, dividing original multivariable time series data into an original training set and an original test set; s2, performing normalization processing on the original training set and the original test set, and performing time window segmentation to obtain a new training set and a new test set; s3, a GTVform model is constructed; s4, the GTVformer model trains the data in the new training set in two stages, and the two Transformer models are trained in a confrontation mode to reconstruct normal input window data; and S5, predicting the data in the new test set by using the trained GTVformer model to obtain a marking result. By combining the time-variable attention mechanism and the adversarial training method, the accuracy of multivariable time series data anomaly detection is remarkably improved, and meanwhile, the robustness of the model is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of multivariate time series anomaly detection, and in particular to an industrial Internet of Things system anomaly detection method based on multivariate time series data. Background Art

[0002] With the rapid development of Industrial Internet of Things (IIoT) and intelligent manufacturing technology, a large amount of multivariate time series data is generated and collected. These data usually come from industrial equipment, sensors, monitoring systems, etc. By analyzing these multivariate time series data, functions such as equipment failure prediction, production process optimization, and safety monitoring can be achieved. Anomaly detection, as an important task in time series data analysis, can help timely discover potential equipment failures, performance problems, or security threats, and therefore plays a vital role in industrial applications.

[0003] In the prior art, multivariate time series anomaly detection methods are usually based on two types of strategies: reconstruction-based models (such as autoencoders) and prediction-based models (such as regression models). Reconstruction-based models detect anomalies by learning a compressed representation of normal time series data, reconstructing the original data with this representation, and calculating the reconstruction error. However, such methods often ignore the complex dependencies between multiple variables and are easily interfered by noise. Prediction-based models judge anomalies by predicting the future behavior of time series data and calculating the prediction error. Although such methods can learn the time dependency of time series data, they usually have difficulty in handling nonlinear dependencies between multiple variables and are very sensitive to noise. In order to meet these challenges, deep learning methods have been widely used in multivariate time series anomaly detection in recent years. However, such methods still face the problem of being unable to effectively capture complex temporal and spatial dependencies at the same time, and it is difficult to avoid false positives or negatives when abnormal data is scarce. Summary of the invention

[0004] The present invention provides an industrial Internet of Things system anomaly detection method based on multivariate time series data, which can effectively improve detection accuracy and robustness and can solve at least one of the above-mentioned technical problems.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solution: an industrial Internet of Things system anomaly detection method based on multivariate time series data, comprising the following steps: S1. Divide the original multivariate time series data into an original training set and an original test set, wherein the original training set contains only normal time series data, and the original test set contains abnormal time series data; S2, normalizing the original training set and the original test set respectively, and then performing time window segmentation on each of them to obtain a new training set and a new test set; S3. Combine the time-variable attention mechanism module with the adversarial training mechanism module to construct the GTVformer model. S4, the GTVformer model trains the data in the new training set in two stages, and trains two Transformer models in an adversarial manner to reconstruct the normal input window data W. In the two Transformer models, the Transformer1 model tries to deceive the Transformer2 model, and the Transformer2 model learns to distinguish whether the data is the real input data from the input window data W or the reconstructed data from the Transformer1 model; S5. Use the trained GTVformer model to predict the data in the new test set, combine the output results of the two Transformer models to give the anomaly score score at each time point, and mark the sample points according to the pre-defined anomaly threshold λ to obtain the marking result y; S6. According to the marking result y, determine whether there is an abnormality at each time point.

[0006] Furthermore, in S2, the minimum-maximum normalization method is selected to normalize the original training set and the original test set according to the following formula (1): (1) Wherein, x represents the normalized data set, X represents the original training set or the original test set, and max(X) and min(X) represent the maximum value and minimum value of X, respectively.

[0007] Furthermore, in S2, the normalized data set x is segmented into time windows, and the data set x containing multiple variables is x={x1, x2, ..., x T}, the data set x is segmented by equidistant sampling and converted into multiple sliding windows of length k. The time interval between sliding windows is 1, and a time window W is defined. The time window W contains a time series data from time point t-k+1 to t, that is, W={x t-k+1 , ..., x t}, transform the dataset x into a series of time windows W = {W1, W2, ..., W T} is used as the training input of the model to form the input window data W.

[0008] Furthermore, in S3, the GTVformer model consists of a shared encoder Encoder and two decoders Decoder1 and Decoder2. The encoder Encoder and decoder Decoder1 constitute the Transformer1 model, and the encoder Encoder and decoder Decoder2 constitute the Transformer2 model. The time-variable attention mechanism module and the adversarial training mechanism module both act on the Transformer1 model and the Transformer2 model. The time-variable attention mechanism module is used to mine the correlation between the time dimension and the variable dimension in multivariate time series data and fully integrate the information. The adversarial training mechanism module is used to further amplify the difference between normal data and abnormal data.

[0009] Furthermore, in S3, the construction of the GTVformer model further includes: S31, the time-variable attention mechanism module in the shared encoder is used to model and analyze the original input data, and the time-variable attention mechanism module is used to deeply mine the feature vectors of the time dimension and the variable dimension; S32, in the training stage, the output of the encoder Encoder is used as the input of the decoder Decoder1 and the decoder Decoder2, and the time-variable attention mechanism modules in the two decoders Decoder1 and Decoder2 further mine the information of the input in the time dimension and the variable dimension, and pass the mined feature vector to the MLP model for long-term prediction; S33, training is performed in two stages based on two Transformer models; S34. In the test phase, the output results of the Transformer1 model and the Transformer2 model are combined to give an abnormal score at each time point, and a predicted label for each time point is given according to the pre-defined abnormal threshold. .

[0010] Furthermore, in S4, the training is divided into two stages: Stage 1: Transformer training In the first stage, the goal is to train each Transformer model to reconstruct the input data. The input window data W is compressed into the latent space by the encoder and then reconstructed by each decoder. The training objectives of this stage are the following equations (2) and (3): (2) (3) in, and They represent the training targets of the Transformer1 model and the Transformer2 model respectively, Trans is the abbreviation of Transformer, and W is the input window data; Stage 2: Adversarial Training In the second stage, the goal is to train the Transformer2 model to distinguish between real data and reconstructed data from the Transformer1 model, and at the same time train the Transformer1 model to deceive the Transformer2 model. The data from the Transformer1 model is compressed into the latent space again through the encoder and then reconstructed by the Transformer2 model. In adversarial training, the goal of the Transformer1 model is to minimize the difference between the input window data W and the output of the Transformer2 model, while the goal of the Transformer2 model is to maximize the difference. The training purpose of the Transformer1 model is to determine whether it has successfully deceived the Transformer2 model, while the training purpose of the Transformer2 model is to try to distinguish the data reconstructed by the Transformer1 model from the real data. The training goal of this stage is the following formula (4): (4) The corresponding loss functions are as follows (5) and (6): (5) (6).

[0011] Furthermore, in S4, both Transformer models have dual tasks, the task of the Transformer1 model is to minimize the reconstruction error of the input window data W in stage 1, and to minimize the difference between the input window data W and the reconstructed output of the Transformer2 model in stage 2, and the task of the Transformer2 model is to minimize the reconstruction error of the input window data W in stage 1, and to maximize the reconstruction error of the data reconstructed by the Transformer1 model in stage 2; The dual training objective of each Transformer model is expressed by combining the following equations (7) and (8), and the weight ratios of different parts are adjusted over time: (7) (8) Here, n represents the number of training rounds.

[0012] Furthermore, in S5, the anomaly score score is calculated by the following formula (9): (9) in: Represents the window data that has not been seen, that is, the test window data; t represents the timestamp; α+β=1 is used to represent the trade-off between parameterized false positives and true positives. If α is greater than β, the number of true positives is reduced while the number of false positives is reduced. On the contrary, if α is less than β, the number of true positives is increased, but the number of false positives is also increased.

[0013] Furthermore, in S5, the marking result y at each time point is calculated by the following formula (10): (10) Among them, λ is the pre-defined abnormal threshold; When the tag value of the tag result y is 0, it indicates that the system is normal at the current moment. When the tag value of the tag result y is 1, it indicates that there is an abnormality in the system at the current moment.

[0014] Furthermore, in S6, since the threshold value can be set in different ways, if If a fixed threshold is exceeded, timestamp t is marked as an anomaly.

[0015] The beneficial effects of the present invention are embodied in: 1. Efficient modeling of nonlinear time series dependencies: By introducing the time variable attention mechanism to capture the complex dependencies between the time dimension and the variable dimension in multivariate time series data, it can effectively model the complex nonlinear dependencies between the time dimension and the variable dimension and improve the expressiveness of the model.

[0016] 2. Effectively enhance the ability to identify anomalies: By using adversarial training strategies to amplify the difference between normal data and abnormal data, the risk of abnormal points being masked by normal points is effectively reduced, the recognition rate of abnormal points is improved, and the accuracy of anomaly detection is improved.

[0017] 3. Strong robustness: This method can work effectively in noisy environments, reduce false alarm rates, and has strong generalization capabilities, and can show good adaptability and stability in different industrial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0019] Figure 1 It is an overall flow chart of the detection method of an embodiment of the present invention.

[0020] Figure 2 It is a network overall architecture diagram of the GTVformer model of an embodiment of the present invention.

[0021] Figure 3 4 is a diagram of the detection phase architecture of an embodiment of the present invention.

[0022] Figure 4 It is a schematic diagram of the structure of the inter-variable attention mechanism module of an embodiment of the present invention.

[0023] Figure 5 It is a comparison chart of experimental value results under F1 of the GTVformer model of an embodiment of the present invention and other multiple benchmark models.

[0024] Figure 6 It is a structural block diagram of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0025] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. In the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0026] It should be noted that the meaning of "and / or" in the full text includes three parallel solutions. Taking "A and / or B" as an example, it includes solution A, solution B, or solutions that satisfy both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0027] See also Figure 1 The embodiment of the present invention provides an industrial Internet of Things system anomaly detection method based on multivariate time series data, comprising the following steps: S1. Divide the original multivariate time series data into an original training set and an original test set, wherein the original training set contains only normal time series data, and the original test set contains abnormal time series data; S2, normalizing the original training set and the original test set respectively, and then performing time window segmentation on each of them to obtain a new training set and a new test set; S3, combining the time-variable attention mechanism module with the adversarial training mechanism module to construct the GTVformer model; S4, the GTVformer model trains the data in the new training set in two stages, and trains two Transformer models in an adversarial manner to reconstruct the normal input window data W. In the two Transformer models, the Transformer1 model tries to deceive the Transformer2 model, and the Transformer2 model learns to distinguish whether the data is the real input data from the input window data W or the reconstructed data from the Transformer1 model; S5. Use the trained GTVformer model to predict the data in the new test set, combine the output results of the two Transformer models to give the anomaly score score at each time point, and mark the sample points according to the pre-defined anomaly threshold λ to obtain the marking result y; S6. According to the marking result y, determine whether there is an abnormality at each time point.

[0028] In order to solve the problems of complex nonlinear dependencies and scarce anomalies in anomaly detection of multivariate time series data, the present invention proposes a multivariate time series anomaly detection method combining adversarial training and time-variable attention mechanism, and constructs a GTVformer model including an adversarial training module and a time-variable attention mechanism module (TVA module). On the one hand, the GTVformer model uses the time-variable attention mechanism to explore the correlation between the time dimension and the variable dimension in multivariate time series data, fully integrates information, and enables the model to understand the complex relationship between data more comprehensively; on the other hand, the adversarial training mechanism is used to amplify the difference between normal data and abnormal data, thereby improving the accuracy of detection. At the same time, the GTVformer model of the present invention can show excellent performance and wide application potential in multiple complex industrial scenarios, and has strong robustness and robustness.

[0029] In this embodiment, in S2, the minimum-maximum normalization method is selected to normalize the original training set and the original test set according to the following formula (1): (1) Wherein, x represents the normalized data set, X represents the original training set or the original test set, and max(X) and min(X) represent the maximum value and minimum value of X, respectively.

[0030] In this embodiment, in S2, the normalized data set x is segmented into time windows, and the data set x containing multiple variables is x={x1, x2, ..., x T}, the data set x is segmented by equidistant sampling and converted into multiple sliding windows of length k. The time interval between sliding windows is 1, and a time window W is defined. The time window W contains a time series data from time point t-k+1 to t, that is, W={x t-k+1 , ..., x t}, transform the dataset x into a series of time windows W = {W1, W2, ..., W T} is used as the training input of the model to form the input window data W.

[0031] The multivariate time series anomaly detection problem can be described as follows: given a time series dataset x={x1, x2, ..., x T}, where each data point represents For observations of multiple variables, the goal is to detect whether there are anomalies in the time series without labels. The core of unsupervised time series anomaly detection is to determine whether a certain time point x t Whether it deviates from the normal mode. The basis for judging the abnormality is to calculate the difference between the time point and the normal data, that is, the abnormality score, and compare the score with the set threshold to determine x. t Is it an exception? In order to capture the dependency between the current time point and the previous time point, a time window W is defined, which contains a period of time series data from time point t-k+1 to t, that is, W={x t-k+1 , ..., x t In this way, the original time series dataset x can be converted into a series of time windows W = {W1, W2, ..., W T} as the training input of the model. In the anomaly detection model, given an unseen window value (where t>T), the goal of the model is to predict the label based on the anomaly score of the window , used to indicate whether the window is abnormal. If an abnormality is detected, ,otherwise During the training process, W is used to represent the training input window data, and the unseen window In summary, the core of the problem is to use time-windowed multivariate time series to determine whether the current window is abnormal based on its anomaly score.

[0032] See also Figure 2-Figure 4In this embodiment, in S3, the GTVformer model is composed of a shared encoder Encoder and two decoders Decoder1 and Decoder2. The encoder Encoder and decoder Decoder1 constitute the Transformer1 model, and the encoder Encoder and decoder Decoder2 constitute the Transformer2 model. The time-variable attention mechanism module and the adversarial training mechanism module both act on the Transformer1 model and the Transformer2 model. The time-variable attention mechanism module is used to mine the correlation between the time dimension and the variable dimension in multivariate time series data and fully integrate the information. The adversarial training mechanism module is used to further amplify the difference between normal data and abnormal data.

[0033] In the present application, the time-variable attention mechanism module, namely the TVA module, is designed to obtain the potential relationship of the input data from two dimensions: time and variable. Through the parallel structure, the TVA module realizes the mining of the time dependency and space (variable) dependency of the input features respectively. Unlike the traditional multi-layer stacking structure, the parallel structure of the TVA module enables the Transformer model to capture the relationship between time and variables more efficiently and centrally. Then, the Transformer model effectively integrates time information and variable information through feature addition operations and layer normalization steps. The overall framework of the GTVformer also adopts a parallel structure to further improve the effect of feature mining and modeling.

[0034] The TVA module consists of two main components: time attention and variable attention. The main function of time attention is to mine the contextual information of data from the time dimension, while the main function of variable attention is to mine the internal implicit relationship between different variables. The information mined by these two components is integrated through a parallel structure. Figure 4 The detailed composition of the TVA module, temporal attention, and variable attention is shown.

[0035] For input window data , where k is the window size and d is the variable dimension, as the input to the time attention and variable attention in the TVA module, the time attention and variable attention will process these two feature vectors as follows: 1. Time and attention: Temporal attention consists of three main components: multi-head attention, residual connection, and normalization. First, multi-head attention is used to mine the input feature vector The time dimension and obtain the processed feature vector The specific processing process is as follows:

[0036]

[0037] Among them, Q represents Query, K represents Key, V represents Value, FC (.) represents the fully connected layer, softmax (.) represents the normalized exponential function, and K T represents K after the last two-dimensional transposition ( ).

[0038] Then, the output of the temporal attention module is obtained through residual connection and normalization. :

[0039]

[0040]

[0041] Where U and σ represent the statistics of the feature vector, g is the gain, b is the bias, ε is a small value to prevent division by zero, L represents the length of the input time series, i represents the time step position in the input sequence, and W tai W ta The value at time step i, W i Represents the value of W at time step i.

[0042] 2. Variable attention. Different from time attention, variable attention mainly uses a multi-head attention mechanism to mine the dependencies between variables from the dimension d of the variables. Through the mining of variable attention, the GTVformer model can effectively analyze the correlation between different variables and conduct information interaction. The output of the variable attention module is :

[0043]

[0044] Among them, d k Is to make The result is consistent with a distribution with an expected value of 1 and a variance of 1. In particular, in the above formula, Q and K T The corresponding matrix shapes are and .

[0045] Based on the above temporal attention method and variable attention method, and are integrated. Then, the output The TVA module is obtained by the following formula:

[0046] Among them, FLN(.) stands for layer normalization.

[0047] Similar to the anomaly detection method based on autoencoders (AE), anomaly detection based on generative adversarial networks (GAN) is also trained with normal data. After training, the discriminator is used as an anomaly detector. If the input data deviates from the data distribution learned during training, the discriminator will regard it as false data generated by the generator and judge it as abnormal. However, the training process of GAN often encounters problems of mode collapse and non-convergence, mainly due to the imbalance of the capabilities of the generator and the discriminator.

[0048] Therefore, in order to solve the above problems, this application proposes a detection method based on the GTVformer model, and introduces the Transformer architecture in a two-stage adversarial training framework. Compared with the traditional AE architecture, the Transformer model is superior in capturing timing information and global dependencies. On the one hand, the Transformer model can more accurately identify whether there are anomalies in the input data, and perform high-quality reconstruction when there are no anomalies, overcoming the inherent limitations of AE. On the other hand, the Transformer model provides higher stability in adversarial training, effectively solving the common mode collapse and non-convergence problems of GAN.

[0049] In the adversarial training mechanism module, the two decoders play the roles of the generator and the discriminator respectively. Through the adversarial game between the generator and the discriminator, the GTVformer model strengthens the ability to identify abnormal data and improves its robustness. Specifically, adversarial training helps the GTVformer model better distinguish normal samples from abnormal samples by amplifying the difference between abnormal data and normal data, further improving the detection performance.

[0050] See also Figure 2-Figure 3 In this embodiment, in S3, the construction of the GTVformer model further includes: S31, the time-variable attention mechanism module in the shared encoder is used to model and analyze the original input data, and the time-variable attention mechanism module is used to deeply mine the feature vectors of the time dimension and the variable dimension; S32, in the training stage, the output of the encoder Encoder is used as the input of the decoder Decoder1 and the decoder Decoder2, and the time-variable attention mechanism modules in the two decoders Decoder1 and Decoder2 further mine the information of the input in the time dimension and the variable dimension, and pass the mined feature vector to the MLP model for long-term prediction; S33, training is performed in two stages based on two Transformer models; S34. In the test phase, the output results of the Transformer1 model and the Transformer2 model are combined to give an abnormal score at each time point, and a predicted label for each time point is given according to the pre-defined abnormal threshold. .

[0051] It can be seen from the above embodiments that when =1, it means that there is an abnormality in the system at the current moment. =0, it means the system is normal at the current moment.

[0052] See also Figure 2-Figure 3 In this embodiment, in S4, the training is divided into two stages: Stage 1: Transformer training In the first stage, the goal is to train each Transformer model to reconstruct the input data. The input window data W is compressed into the latent space by the encoder and then reconstructed by each decoder. The training objectives of this stage are the following equations (2) and (3): (2) (3) in, and They represent the training targets of the Transformer1 model and the Transformer2 model respectively, Trans is the abbreviation of Transformer, and W is the input window data; Stage 2: Adversarial Training In the second stage, the goal is to train the Transformer2 model to distinguish between real data and reconstructed data from the Transformer1 model, and at the same time train the Transformer1 model to deceive the Transformer2 model. The data from the Transformer1 model is compressed into the latent space again through the encoder and then reconstructed by the Transformer2 model. In adversarial training, the goal of the Transformer1 model is to minimize the difference between the input window data W and the output of the Transformer2 model, while the goal of the Transformer2 model is to maximize the difference. The training purpose of the Transformer1 model is to determine whether it has successfully deceived the Transformer2 model, while the training purpose of the Transformer2 model is to try to distinguish the data reconstructed by the Transformer1 model from the real data. The training goal of this stage is the following formula (4): (4) The corresponding loss functions are as follows (5) and (6): (5) (6).

[0053] In the above embodiment, both Transformer models have dual tasks. The task of the Transformer1 model is to minimize the reconstruction error of the input window data W in stage 1 and minimize the difference between the input window data W and the reconstructed output of the Transformer2 model in stage 2. The task of the Transformer2 model is to minimize the reconstruction error of the input window data W in stage 1 and maximize the reconstruction error of the data reconstructed by the Transformer1 model in stage 2. The dual training objective of each Transformer model is expressed by combining the following equations (7) and (8), and the weight ratios of different parts are adjusted over time: (7) (8) Here, n represents the number of training rounds.

[0054] It should be noted that the Transformer2 model does not work strictly like the discriminator in GAN, because if its input is the original data, then the losses in formulas (2) and (3) will work, and when its input is the reconstructed data, the targets in formulas (4) and (6) will work.

[0055] In this embodiment, in S5, the anomaly score score is calculated by the following formula (9): (9) in: Represents the unseen window, i.e., the test window data; t represents the timestamp; α+β=1 is used to represent the trade-off between parameterized false positives and true positives. If α is greater than β, the number of true positives is reduced while the number of false positives is reduced. On the contrary, if α is less than β, the number of true positives is increased, but the number of false positives is also increased.

[0056] In the above embodiment, the marking result y at each time point is calculated by the following formula (10): (10) Among them, λ is the pre-defined abnormal threshold; When the tag value of the tag result y is 0, it indicates that the system is normal at the current moment. When the tag value of the tag result y is 1, it indicates that there is an abnormality in the system at the current moment.

[0057] Based on the above embodiment, in S6, since the threshold value can be set in different ways, if If the value exceeds the fixed threshold, the timestamp t is marked as an anomaly. As in extreme value theory, the same anomaly detection model will produce different prediction performance under different anomaly thresholds. Therefore, a grid search is applied to all possible anomaly thresholds to search for the theoretically optimal F1-score and give the result.

[0058] The above design is applied in practice, and multiple repeated experiments are carried out based on multiple indicators (Precision, Recall, F1-score), and parameter adjustment is optimized. The method of the GTVformer model is used to demonstrate the superior performance of the design of the present invention in multivariate time series anomaly detection. The present invention selects four real public data sets (MSL, SMAP, PSM and SWaT) to compare the model GTVformer of the present invention with benchmark models such as IF, GDN, OmniAnomaly, USAD, MTAD-GAT, CAE-M, BeatGAN, and InterFusion.

[0059] like Figure 5As shown in the figure, in the MSL dataset, in terms of the F1-score index, the model GTVformer of the present invention is significantly better than other benchmark models, with an F1-score of 90.23%, which is 1.81%, 3.16% and 3.61% higher than GDN, OmniAnomaly and InterFusion respectively, indicating that the design model of the present invention has higher robustness and accuracy in data anomaly detection. In the SMAP dataset, in terms of the F1-score index, the model GTVformer of the present invention reaches 92.76%, which is 7.62% and 3.62% higher than OmniAnomaly and InterFusion respectively, and greatly surpasses the traditional method IF (increased by 37.23%), showing a strong adaptability to complex data environments. In the PSM dataset, in terms of the F1-score index, the model GTVformer of the present invention reaches 97.80%, which is 12.22% and 16.97% higher than InterFusion and OmniAnomaly respectively, and is significantly better than other traditional methods, showing its superiority in industrial server monitoring scenarios. In the SWaT dataset, in terms of the F1-score indicator, the GTVformer model of the present invention reached 87.07%, which was 2.76% and 4.06% higher than BeatGAN and InterFusion respectively, demonstrating the reliability of anomaly detection in industrial control systems.

[0060] In summary, the GTVformer model designed in the present invention shows significant performance advantages on multiple data sets, and performs well in both precision and recall of anomaly detection, further proving its application potential and stability in complex time series anomaly detection.

[0061] An embodiment of the present invention also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned industrial Internet of Things system anomaly detection method based on multivariate time series data.

[0062] See also Figure 6 An embodiment of the present invention further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the above-mentioned industrial Internet of Things system anomaly detection method based on multivariate time series data.

[0063] An embodiment of the present invention also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the steps of the above-mentioned industrial Internet of Things system anomaly detection method based on multivariate time series data.

[0064] It is understandable that the system, device and storage medium provided in the embodiments of the present invention correspond to the method provided in the embodiments of the present invention, and the explanation, examples and beneficial effects of the relevant contents can refer to the corresponding parts in the above-mentioned industrial Internet of Things system anomaly detection method based on multivariate time series data.

[0065] It should be noted that those skilled in the art can understand that all or part of the steps implemented in the embodiments of the present invention can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using hardware, it can be implemented in whole or in part in the form of purchased standard parts or modified parts. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0066] It should be understood that the examples and implementation modes described herein are for illustrative purposes only and are not intended to limit the present invention. Those skilled in the art may make various modifications or changes based on the examples and implementation modes. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An anomaly detection method for industrial Internet of Things systems based on multivariate time series data, characterized in that: The following steps are involved: S1. Divide the original multivariate time series data into an original training set and an original test set, wherein the original training set contains only normal time series data, and the original test set contains abnormal time series data; S2, normalizing the original training set and the original test set respectively, and then performing time window segmentation on each of them to obtain a new training set and a new test set; S3, combining the time-variable attention mechanism module with the adversarial training mechanism module to construct the GTVformer model; S4, the GTVformer model trains the data in the new training set in two stages, and trains two Transformer models in an adversarial manner to reconstruct the normal input window data W. In the two Transformer models, the Transformer1 model tries to deceive the Transformer2 model, and the Transformer2 model learns to distinguish whether the data is the real input data from the input window data W or the reconstructed data from the Transformer1 model; S5. Use the trained GTVformer model to predict the data in the new test set, combine the output results of the two Transformer models to give the anomaly score score at each time point, and mark the sample points according to the pre-defined anomaly threshold λ to obtain the marking result y; S6. According to the marking result y, determine whether there is an abnormality at each time point.

2. The method for detecting anomalies in an industrial Internet of Things system based on multivariate time series data according to claim 1, characterized in that: In S2, the minimum-maximum normalization method is selected to normalize the original training set and the original test set according to the following formula (1): （1） Wherein, x represents the normalized data set, X represents the original training set or the original test set, and max(X) and min(X) represent the maximum value and minimum value of X, respectively.

3. The method for detecting anomalies in an industrial Internet of Things system based on multivariate time series data according to claim 1, characterized in that: In S2, the normalized data set x is segmented into time windows, and the data set x containing multiple variables is x={x1, x2, ..., x T }, the data set x is segmented by equidistant sampling and converted into multiple sliding windows of length k. The time interval between sliding windows is 1, and a time window W is defined. The time window W contains a time series data from time point t-k+1 to t, that is, W={x t-k+1 , ..., x t }, transform the dataset x into a series of time windows W = {W1, W2, ..., W T } is used as the training input of the model to form the input window data W.

4. The method for detecting anomalies in an industrial Internet of Things system based on multivariate time series data according to claim 1, characterized in that: In S3, the GTVformer model consists of a shared encoder Encoder and two decoders Decoder1 and Decoder2. The encoder Encoder and decoder Decoder1 constitute the Transformer1 model, and the encoder Encoder and decoder Decoder2 constitute the Transformer2 model. The time-variable attention mechanism module and the adversarial training mechanism module both act on the Transformer1 model and the Transformer2 model. The time-variable attention mechanism module is used to mine the correlation between the time dimension and the variable dimension in multivariate time series data and fully integrate the information. The adversarial training mechanism module is used to further amplify the difference between normal data and abnormal data.

5. The method for detecting anomalies in an industrial Internet of Things system based on multivariate time series data according to claim 4, characterized in that: In S3, the construction of the GTVformer model further includes: S31, the time-variable attention mechanism module in the shared encoder is used to model and analyze the original input data, and the time-variable attention mechanism module is used to deeply mine the feature vectors of the time dimension and the variable dimension; S32, in the training stage, the output of the encoder Encoder is used as the input of the decoder Decoder1 and the decoder Decoder2, and the time-variable attention mechanism modules in the two decoders Decoder1 and Decoder2 further mine the information of the input in the time dimension and the variable dimension, and pass the mined feature vector to the MLP model for long-term prediction; S33, training is performed in two stages based on two Transformer models; S34. In the test phase, the output results of the Transformer1 model and the Transformer2 model are combined to give an abnormal score at each time point, and a predicted label for each time point is given according to the pre-defined abnormal threshold. .

6. The method for detecting anomalies in an industrial Internet of Things system based on multivariate time series data according to claim 1, characterized in that: In S4, the training is divided into two stages: Stage 1: Transformer training In the first stage, the goal is to train each Transformer model to reconstruct the input data. The input window data W is compressed into the latent space by the encoder and then reconstructed by each decoder. The training objectives of this stage are the following equations (2) and (3): （2） （3） in, and They represent the training targets of the Transformer1 model and the Transformer2 model respectively, Trans is the abbreviation of Transformer, and W is the input window data; Stage 2: Adversarial Training In the second stage, the goal is to train the Transformer2 model to distinguish between real data and reconstructed data from the Transformer1 model, and at the same time train the Transformer1 model to deceive the Transformer2 model. The data from the Transformer1 model is compressed into the latent space again through the encoder and then reconstructed by the Transformer2 model. In adversarial training, the goal of the Transformer1 model is to minimize the difference between the input window data W and the output of the Transformer2 model, while the goal of the Transformer2 model is to maximize the difference. The training purpose of the Transformer1 model is to determine whether it has successfully deceived the Transformer2 model, while the training purpose of the Transformer2 model is to try to distinguish the data reconstructed by the Transformer1 model from the real data. The training goal of this stage is the following formula (4): （4） The corresponding loss functions are as follows (5) and (6): （5） （6）。 7. The method for detecting anomalies in an industrial Internet of Things system based on multivariate time series data according to claim 6, characterized in that: In S4, both Transformer models have dual tasks. The task of the Transformer1 model is to minimize the reconstruction error of the input window data W in stage 1 and minimize the difference between the input window data W and the reconstructed output of the Transformer2 model in stage 2. The task of the Transformer2 model is to minimize the reconstruction error of the input window data W in stage 1 and maximize the reconstruction error of the data reconstructed by the Transformer1 model in stage 2. The dual training objective of each Transformer model is expressed by combining the following equations (7) and (8), and the weight ratios of different parts are adjusted over time: （7） （8） Here, n represents the number of training rounds.

8. The method for detecting anomalies in an industrial Internet of Things system based on multivariate time series data according to claim 1, characterized in that: In S5, the anomaly score score is calculated by the following formula (9): （9） in: Represents the unseen window, i.e., the test window data; t represents the timestamp; α+β=1 is used to represent the trade-off between parameterized false positives and true positives. If α is greater than β, the number of true positives is reduced while the number of false positives is reduced. On the contrary, if α is less than β, the number of true positives is increased, but the number of false positives is also increased.

9. The method for detecting anomalies in an industrial Internet of Things system based on multivariate time series data according to claim 1, characterized in that: In S5, the marking result y at each time point is calculated by the following formula (10): （10） Among them, λ is the pre-defined abnormal threshold; When the tag value of the tag result y is 0, it indicates that the system is normal at the current moment. When the tag value of the tag result y is 1, it indicates that there is an abnormality in the system at the current moment.

10. The method for detecting anomalies in an industrial Internet of Things system based on multivariate time series data according to claim 8, characterized in that: In S6, since the threshold value can be set in different ways, if If a fixed threshold is exceeded, timestamp t is marked as an anomaly.

Images