Anomaly detection method for multi-dimensional time series data of Internet of Things

Abstract

The invention relates to an anomaly detection method for multi-dimensional time series data of the Internet of Things, and belongs to the technical field of anomaly detection of the Internet of Things. Time features and measurement features of the multi-dimensional time series data of the Internet of Things are deeply extracted and fused, and a generative adversarial network is formed by combining a data reconstruction model; according to the method, generative adversarial is carried out continuously and iteratively, so that the data reconstruction capability of the model is improved, the reconstruction error of the model for normal data is small, the reconstruction error of the model for abnormal data is large, and the abnormality is distinguished accurately through the abnormality score. The method is suitable for data monitoring detection of multiple sensors or controllers in the Internet of Things environment, multi-dimensional time series data in the real Internet of Things environment can be received, and timely early warning is carried out on the part which possibly has abnormity in the data through model detection, so that potential problems or losses caused by data abnormity are avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of anomaly detection in the Internet of Things, and in particular to an anomaly detection method for multi-dimensional time series data in the Internet of Things. Background Art

[0002] With the rapid development of the Internet of Things, the scope of its application is expanding day by day, playing a vital role in the fields of smart cities, smart transportation, smart energy, smart finance, smart agriculture, smart manufacturing, smart construction, smart environmental protection and smart medical care. The Internet of Things includes IoT devices such as sensors, controllers, and control computers. The data produced, collected or monitored by these devices reflect the changes in the real physical world and the status of the devices themselves in real time, and the real-time monitoring and precise control of data have important strategic value. In the real environment, due to factors such as equipment failure, operational errors, omissions or network attacks, the collected IoT data will have data patterns that do not conform to normal expectations or working conditions, such as abnormal peaks and troughs, and long-term oscillations and other behaviors that do not conform to normal patterns. There is a significant difference between this type of data and normal observation data, so it is usually called anomaly. Anomaly detection is a technology that identifies anomalies in data. It can find data patterns that do not conform to normal conditions in the data, and promptly warn and respond to them to avoid personal safety problems or serious economic losses caused by anomalies. For example: by performing abnormal detection on indicators such as hydrogen, carbon monoxide, and combustion compounds collected by the forest fire "electronic nose" sensor, the signs of forest fires can be discovered and early warnings can be issued in a timely manner; by performing abnormal detection on indicators such as pH value, chlorophyll, dissolved oxygen, and heavy metals in water quality sensors, changes in water pollution can be discovered in a timely manner; by performing abnormal detection on sensors and controllers in water treatment plants, equipment failures can be discovered in a timely manner to avoid personal safety issues and economic losses caused by errors in the water treatment process. Therefore, abnormal detection of IoT data has important application significance.

[0003] IoT data is usually collected by sensors and has obvious time characteristics and multivariate characteristics (also called metric characteristics). Therefore, this type of data is also called multidimensional time series data. Due to the unknown nature of anomalies, the differences between anomalies in different fields are very different, and there is a lack of labeled data, multidimensional time series data anomaly detection is usually an unsupervised method, which has better adaptability and detection effect than supervised learning methods. In terms of model complexity, traditional machine learning methods such as threshold detection and density analysis cannot cope with complex and dynamically changing data anomalies, and there are internal connections between multidimensional time series data, such as time connections and metric connections. It is difficult for traditional methods to extract these key features, resulting in poor anomaly detection effect. Therefore, unsupervised deep learning methods have become the mainstream of multidimensional time series data anomaly detection. Deep learning methods can be divided into data reconstruction-based and data prediction-based. The data reconstruction-based method learns the data distribution characteristics of multidimensional time series data to reconstruct the data, and determines the anomaly by comparing the difference between the reconstructed data and the original data. In this way, the model will try to narrow the difference between normal data and its reconstructed data, while abnormal data will have a greater difference between the reconstructed data and the original data due to different data distribution characteristics, thereby indicating anomalies. Data prediction methods predict IoT data and compare prediction errors to determine anomalies. However, in a complex IoT environment, some data features such as metric features are difficult to accurately predict. Therefore, anomaly detection methods based on data reconstruction are an important research direction for anomaly detection in multidimensional time series data because of their simple ideas and excellent results.

[0004] At present, the anomaly detection model based on data reconstruction has problems in the process of data feature extraction, fusion and data reconstruction. Multidimensional time series data has obvious time characteristics, and these data come from different sensors, so there are also metric characteristics. Existing methods usually only consider time characteristics or metric characteristics in feature extraction, ignoring the implicit connection between time characteristics and metric characteristics. The lack of feature extraction fusion directly leads to poor data reconstruction effect; in the data reconstruction process, existing methods use basic network structures such as autoencoder AE, variational autoencoder VAE or generative adversarial network GAN to reconstruct data, and usually only use a single network structure for data reconstruction. This single network structure has natural limitations. For example, the AE structure learns the local features of time series data and may not be able to reconstruct unknown data well; although the VAE structure adds variational latent variables and has a certain ability to deal with unknown anomalies, the reconstruction result may be more vague than AE, ignoring the local variable characteristics; GAN has problems such as training difficulties, gradient disappearance and mode collapse, and the effect is unstable in anomaly detection work. Therefore, these problems lead to a single data reconstruction model that may only have good data reconstruction capabilities for some multidimensional time series data, but it is difficult to reconstruct data well for other data, resulting in unstable data reconstruction results. Both of these problems directly affect the reconstruction results of multidimensional time series data, and data reconstruction is the core of this type of method. Therefore, poor or unstable data reconstruction results will directly affect the data anomaly detection results. Summary of the invention

[0005] In view of the above problems, the present invention provides an anomaly detection method for multi-dimensional time series data of the Internet of Things, which deeply extracts and fuses the time features and metric features of the multi-dimensional time series data of the Internet of Things, and combines the data reconstruction model to form a generative adversarial network. By continuously iteratively performing generative adversarial networks, the data reconstruction capability of the model is improved, so that the reconstruction error of the model for normal data is small, while the reconstruction error for abnormal data is relatively large, and then the anomaly score is used to accurately distinguish the anomaly. The present invention is suitable for data monitoring and detection of multiple sensors or controllers in the Internet of Things environment, and can accept multi-dimensional time series data in a real Internet of Things environment, and give timely warnings of parts that may have anomalies in the data through model detection, avoiding potential problems or losses caused by data anomalies. It solves the problems of insufficient data feature extraction and fusion, unstable reconstruction effect of a single network, and poor detection effect in the existing anomaly detection method based on data reconstruction.

[0006] The present invention provides an anomaly detection method for multi-dimensional time series data of the Internet of Things, comprising:

[0007] Step S1, obtaining multi-dimensional time series observation data of multiple sensors or controllers in the Internet of Things environment, the expression is:

[0008]

[0009] Among them, X represents multidimensional time series observation data, N represents the number of metric features of observed data at each moment, Represents the observation value at time t, t = 1, 2, 3…T, T represents the total number of moments, that is, the length of the time series.

[0010] The multi-dimensional time series observation data is divided into sliding windows to obtain multiple time series with context information, expressed as:

[0011]

[0012] Among them, W t is the time series with context information at time t, w represents the length of the sliding window sequence.

[0013] Based on multiple time series with context information, a multivariate time window sequence W = {W1, W2, ..., W T};

[0014] Step S2, establishing a dual encoder architecture based on a time dimension encoder Time-Encoder and a metric dimension encoder Metric-Encoder;

[0015] The multivariate time window sequence is input into a dual encoder architecture, and a coding matrix after deep fusion of the features of the observation data at each moment is obtained based on a multi-head attention mechanism and a cross-attention mechanism;

[0016] Preferably, the specific steps of obtaining the encoding matrix after the feature depth fusion of the observation data at each moment include:

[0017] Establish a dual encoder architecture based on the time dimension encoder and the metric dimension encoder;

[0018] The multivariate time window sequence is input into the dual encoder architecture, and based on the multi-head attention mechanism, the time dimension multi-head attention matrix and the metric dimension multi-head attention matrix are obtained, which are expressed as:

[0019]

[0020] Among them, Q z is the query vector of the time dimension or the measurement dimension, K z is the time dimension or measurement dimension key vector, V zis the time dimension or measurement dimension value vector, is the weight matrix of the query vector in the time dimension or measurement dimension, is the weight matrix of the key vector of the time dimension or measurement dimension, is the weight matrix of the time dimension or measurement dimension value vector, Attention z is the attention matrix of the time dimension or the measurement dimension; softmax(·) is the activation function, K z T is the transpose of the time dimension or measure dimension key vector, The dimension of the query vector for the time dimension or the metric dimension is to avoid the value obtained by the dot product operation being too large. MultiHeadAttention z (W) is the multi-head attention matrix in the time dimension or measurement dimension, is the attention matrix of the head in the time dimension or measurement dimension, concat(·) is the concatenation function of the matrix in the time dimension or measurement dimension, and Liner(·) is the linear function.

[0021] The time dimension multi-head attention matrix and the metric dimension multi-head attention matrix are passed through two residual connection layers and normalized Add&LayerNorm layers respectively to obtain the time dimension feature encoding matrix and the metric dimension feature encoding matrix, which are expressed as:

[0022] C z =LayerNorm(FeedForward(First_Layer z )+First_Layer z )

[0023] Among them, C z is the feature encoding matrix of the time dimension or measurement dimension, First_Layer z It is the first residual connection layer and normalization (Add&LayerNorm) layer of the time dimension or measurement dimension. FeedForward(·) is a feedforward neural network used to further extract and process features. LayerNorm(·) is a layer normalization function used to standardize the output of each layer of the neural network and enhance the stability of training.

[0024] Preferably, the expressions of the residual connection layer and the normalized Add&LayerNorm layer are:

[0025] First_Layer z =LayerNorm(MultiHeadAttention z (W)+W)

[0026] Among them, First_Layer z is the first residual connection layer and normalization (Add&LayerNorm) layer of the time dimension or measurement dimension, C z is the feature encoding matrix of the time dimension or the metric dimension, FeedForward(·) is a feedforward neural network used to further extract and process features, and LayerNorm(·) is a layer normalization function used to standardize the output of each layer of the neural network to enhance the stability of training.

[0027] The time dimension feature encoding matrix and the metric dimension feature encoding matrix are deeply fused through a cross attention mechanism to obtain a time dimension cross attention matrix and a metric dimension cross attention matrix;

[0028] The time dimension cross attention matrix and the metric dimension cross attention matrix are concatenated and then passed through two residual connections and layer normalization Add&LayerNorm layers for residual calculation to obtain a coding matrix after feature deep fusion;

[0029] Preferably, the time dimension encoder Time-Encoder and the metric dimension encoder Metric-Encoder have the same structure;

[0030] Preferably, the expression of the time dimension cross attention matrix is:

[0031]

[0032] Among them, Q time is the time dimension query vector, K metric is the metric dimension key vector, V metric is the measurement dimension value vector, Attention time is the temporal cross attention matrix.

[0033] Step S3, constructing a data reconstruction model using the autoencoder AE, the variational autoencoder VAE and the GAN discriminator;

[0034] Using the encoding matrix after deep fusion of the features of the observation data at each moment as a training sample, and using the training sample to perform generative adversarial training on the data reconstruction model to obtain an updated data reconstruction model;

[0035] Preferably, the specific steps of obtaining the updated data to reconstruct the model include:

[0036] Step S31, let t=t+1, when t=1, it indicates the initial time of history;

[0037] Obtain the encoding matrix after deep fusion of the features of the observation data at historical time t;

[0038] Step S32: Input the encoding matrix after the feature deep fusion of the historical time t observation data into the historical time t data reconstruction model D t , based on the historical moment t automatic encoder AE, the encoding matrix after the feature deep fusion of the historical moment t observation data is transformed into a linear layer to obtain the historical moment t hidden space one;

[0039] Step S33, reconstructing the hidden space 1 at historical moment t through the decoder at historical moment t to obtain the reconstructed data 1 at historical moment t;

[0040] Step S34, based on the variational autoencoder VAE at historical time t, the encoding matrix after the feature depth fusion of the observation data at historical time t is subjected to linear layer transformation to obtain the hidden layer space 2 at historical time t;

[0041] Step S35, reconstructing the hidden space 2 at historical moment t through the decoder at historical moment t to obtain the reconstructed data 2 at historical moment t;

[0042] Step S36, inputting the historical moment t observation data, the historical moment t reconstruction data 1 and the historical moment t reconstruction data 2 into the GAN discriminator, respectively comparing the historical moment t reconstruction data 1 and the historical moment t reconstruction data 2 with the real data, and obtaining the historical moment t identification result based on the similarity with the real data;

[0043] Step S37, calculating the loss function based on the identification result at the historical time t to obtain the loss gradient;

[0044] The data reconstruction model is updated based on the loss gradient to obtain an updated data reconstruction model; the updated data reconstruction model is used as the data reconstruction model for the next historical moment;

[0045] Step S38, determine whether t is greater than or equal to T, where T is the total number of historical moments. If so, stop training and obtain updated data to reconstruct the model; if not, set t=t+1 and return to step S32.

[0046] Furthermore, the specific steps of calculating the loss function based on the identification result at the historical moment t in step S37 to obtain the loss gradient and updating the data reconstruction model based on the loss gradient to obtain the updated data reconstruction model include:

[0047] Based on the identification results at the historical time t, the autoencoder AE loss function, the variational autoencoder VAE loss function, the encoder loss function and the discriminator loss function are calculated respectively, and the corresponding loss gradients are obtained respectively;

[0048] Based on the corresponding loss gradients, the autoencoder AE, variational autoencoder VAE, encoder, and discriminator in the data reconstruction model are updated respectively to obtain an updated data reconstruction model.

[0049] Furthermore, the autoencoder AE loss function expression is:

[0050] Loss(AE)=||W′ AE -W||2

[0051] Among them, W′ AE represents the reconstructed data 1, Loss(·) represents the loss function, and W is the real data;

[0052] The expression of the variational autoencoder VAE loss function is:

[0053] Loss(VAE)=||W′ VAE -W||2+D KL (q||p)

[0054] Among them, W′ VAE To reconstruct data 2, D KL represents the KL divergence of the data distribution, p represents the standard normal distribution, and q represents the model simulation distribution;

[0055] The expression of the encoder loss function is:

[0056] Loss(Encoder)=0.5*Loss(AE)+0.5*Loss(VAE)

[0057] Among them, Encoder is the encoder.

[0058] The discriminator GAN loss function expression is:

[0059] Loss(GAN)=log(Dis(W))+log(1-Dis(W′ AE ))+log(1-Dis(W′ VAE ))

[0060] Among them, Dis(·) represents the probability that the discriminator identifies the data as true. During training, the model hopes that the loss function of the discriminator is gradient-ascending, proving that the discriminator can distinguish between real data and reconstructed data well.

[0061] The present invention continuously iterates and updates in this way, and AE and VAE can learn the characteristics and distribution of normal data. When faced with normal data, the reconstruction error is small, and when faced with abnormal data, the data with large errors will be reconstructed. At the same time, because the discriminator of GAN has learned the distribution pattern of the original normal data, the abnormal data can also be correctly evaluated by the discriminator during detection. In this way, by combining multiple different data reconstruction models, abnormal data can be identified more accurately, avoiding the inability to deal with all anomalies due to the limitations of a single model.

[0062] Step S4: Select the data to be tested of the Internet of Things; reconstruct the model based on the updated data to obtain the abnormal score of the data to be tested;

[0063] Anomaly detection of the data to be tested is achieved based on the anomaly score.

[0064] Preferably, the specific steps of implementing anomaly detection of the data to be tested based on the anomaly score in step S4 include:

[0065] Input the current test data into the updated data reconstruction model, and obtain reconstruction data 3 and corresponding reconstruction error 1, reconstruction data 4 and corresponding reconstruction error 2, identification result and corresponding identification error respectively;

[0066] Based on the reconstruction error 1 and the reconstruction error 2 at the current moment, determine the abnormal probability 1 and the abnormal probability 2 at the current moment, combine the reconstruction error 1 and the abnormal probability 1 at the current moment to obtain the abnormal score of the reconstruction data 1 at the current moment; combine the reconstruction error 2 and the abnormal probability 2 at the current moment to obtain the abnormal score of the reconstruction data 2 at the current moment;

[0067] Determine the identification anomaly probability at the moment based on the identification error at the current moment; combine the identification error at the current moment with the identification anomaly probability to obtain the anomaly score of the identification result at the current moment;

[0068] Obtaining an anomaly score of the reconstructed data at the current moment based on the anomaly score of the reconstructed data one at the current moment and the anomaly score of the reconstructed data two;

[0069] The final anomaly score at the current moment is obtained by weighted summing the anomaly score of the reconstructed data at the current moment and the anomaly score of the identification result at the current moment;

[0070] Preset anomaly score threshold δ;

[0071] Determine whether the final abnormality score at the current moment is greater than the abnormality score threshold, and obtain a determination result;

[0072] If the judgment result is yes, it means that the data to be tested at the current moment is abnormal, and if no, it means that the data to be tested at the current moment is normal.

[0073] Preferably, the final anomaly score is expressed as:

[0074] Anomaly Score=α*Score(Reconstruction)+(1-α)*Score(Discriminination)

[0075] Among them, Anomaly Score is the final anomaly score, α is the weighted summation coefficient, Score(·) is the harmonic function, Reconstruction is the anomaly score of the reconstructed data, and Discrimination is the anomaly score of the identification result.

[0076] Preferably, the judgment result expression is:

[0077]

[0078] Among them, result is the judgment result. When the judgment result is 1, it indicates an abnormality.

[0079] Compared with the prior art, the present invention has at least the following beneficial effects:

[0080] (1) The present invention uses transformers to effectively extract temporal and metric features through the attention mechanism, and deeply cross-integrates the two features through cross-attention, which can accurately capture the implicit connection between temporal features and metric features;

[0081] (2) Generative adversarial network composed of multiple data reconstruction models of the present invention. The network is a generative adversarial network composed of multiple data reconstruction models jointly composed of AE, VAE and GAN. Among them, AE focuses on the local features of the data, VAE focuses on the global distribution of the data, AE and VAE jointly constitute the generator of the generative adversarial network, and the discriminator of GAN is used to identify whether the data is original or generated by the generator. Continuously generate adversarial to improve the data reconstruction ability and identification ability of the model, thereby improving the stability of data reconstruction and the abnormality detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The drawings are only for the purpose of illustrating particular embodiments and are not to be construed as limiting the invention.

[0083] Figure 1 A schematic diagram of the process of anomaly detection for multi-dimensional time series data of the Internet of Things in an embodiment of the present invention;

[0084] Figure 2 A schematic diagram of updating a data reconstruction model in an embodiment of the present invention;

[0085] Figure 3It is a schematic diagram of the structure diagram of the feature extraction and fusion module in an embodiment of the present invention. DETAILED DESCRIPTION

[0086] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein, and therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0087] A specific embodiment of the present invention, as Figure 1-3 , discloses an anomaly detection method for multi-dimensional time series data of the Internet of Things. In order to illustrate the effectiveness of the method proposed by the present invention, the above technical solution of the present invention is described in detail through a specific embodiment below. The specific implementation steps are as follows:

[0088] The present invention provides an anomaly detection method for multi-dimensional time series data of the Internet of Things, comprising:

[0089] Step S1, obtaining multi-dimensional time series observation data of multiple sensors or controllers in the Internet of Things environment, the expression is:

[0090]

[0091] Among them, X represents multidimensional time series observation data, N represents the number of metric features of observed data at each moment, Represents the observation value at time t, t = 1, 2, 3…T, T represents the total number of moments, that is, the length of the time series.

[0092] The multi-dimensional time series observation data is divided into sliding windows to obtain multiple time series with context information, expressed as:

[0093]

[0094] Among them, W t is the time series with context information at time t, w represents the length of the sliding window sequence.

[0095] Based on multiple time series with context information, a multivariate time window sequence W = {W1, W2, ..., W T};

[0096] Step S2, establishing a dual encoder architecture based on a time dimension encoder Time-Encoder and a metric dimension encoder Metric-Encoder;

[0097] The multivariate time window sequence is input into a dual encoder architecture, and a coding matrix after deep fusion of the features of the observation data at each moment is obtained based on a multi-head attention mechanism and a cross-attention mechanism;

[0098] Preferably, the specific steps of obtaining the encoding matrix after the feature depth fusion of the observation data at each moment include:

[0099] Establish a dual encoder architecture based on the time dimension encoder and the metric dimension encoder;

[0100] The multivariate time window sequence is input into the dual encoder architecture, and based on the multi-head attention mechanism, the time dimension multi-head attention matrix and the metric dimension multi-head attention matrix are obtained, which are expressed as:

[0101]

[0102] Among them, Q z is the query vector of the time dimension or the measurement dimension, K z is the time dimension or measurement dimension key vector, V z is the time dimension or measurement dimension value vector, is the weight matrix of the query vector in the time dimension or measurement dimension, is the weight matrix of the key vector of the time dimension or measurement dimension, is the weight matrix of the time dimension or measurement dimension value vector, Attention z is the attention matrix of the time dimension or the measurement dimension; softmax(·) is the activation function, K z T is the transpose of the time dimension or measure dimension key vector, The dimension of the query vector for the time dimension or the metric dimension is to avoid the value obtained by the dot product operation being too large. MultiHeadAttention z (W) is the multi-head attention matrix in the time dimension or measurement dimension, is the attention matrix of the head in the time dimension or measurement dimension, concat(·) is the concatenation function of the matrix in the time dimension or measurement dimension, and Liner(·) is the linear function.

[0103] The time dimension multi-head attention matrix and the metric dimension multi-head attention matrix are passed through two residual connection layers and normalized Add&LayerNorm layers respectively to obtain the time dimension feature encoding matrix and the metric dimension feature encoding matrix, which are expressed as:

[0104] C z=LayerNorm(FeedForward(First_Layer z )+First_Layer z )

[0105] Among them, C z is the feature encoding matrix of the time dimension or measurement dimension, First_Layer z It is the first residual connection layer and normalization (Add&LayerNorm) layer of the time dimension or measurement dimension. FeedForward(·) is a feedforward neural network used to further extract and process features. LayerNorm(·) is a layer normalization function used to standardize the output of each layer of the neural network and enhance the stability of training.

[0106] Preferably, the expressions of the residual connection layer and the normalized Add&LayerNorm layer are:

[0107] First_Layer z =LayerNorm(MultiHeadAttention z (W)+W)

[0108] Among them, First_Layer z is the first residual connection layer and normalization (Add&LayerNorm) layer of the time dimension or measurement dimension, C z is the feature encoding matrix of the time dimension or the metric dimension, FeedForward(·) is a feedforward neural network used to further extract and process features, and LayerNorm(·) is a layer normalization function used to standardize the output of each layer of the neural network to enhance the stability of training.

[0109] The time dimension feature encoding matrix and the metric dimension feature encoding matrix are deeply fused through a cross attention mechanism to obtain a time dimension cross attention matrix and a metric dimension cross attention matrix;

[0110] The time dimension cross attention matrix and the metric dimension cross attention matrix are concatenated and then passed through two residual connections and layer normalization Add&LayerNorm layers for residual calculation to obtain a coding matrix after feature deep fusion;

[0111] Preferably, the time dimension encoder Time-Encoder and the metric dimension encoder Metric-Encoder have the same structure;

[0112] Preferably, the expression of the time dimension cross attention matrix is:

[0113]

[0114] Among them, Q time is the time dimension query vector, K metric is the metric dimension key vector, V metric is the measurement dimension value vector, Attention time is the temporal cross attention matrix.

[0115] Step S3, constructing a data reconstruction model using the autoencoder AE, the variational autoencoder VAE and the GAN discriminator;

[0116] Using the encoding matrix after deep fusion of the features of the observation data at each moment as a training sample, and using the training sample to perform generative adversarial training on the data reconstruction model to obtain an updated data reconstruction model;

[0117] Preferably, the specific steps of obtaining the updated data to reconstruct the model include:

[0118] Step S31, let t=t+1, when t=1, it indicates the initial time of history;

[0119] Obtain the encoding matrix after deep fusion of the features of the observation data at historical time t;

[0120] Step S32: Input the encoding matrix after the feature deep fusion of the historical time t observation data into the historical time t data reconstruction model D t , based on the historical moment t automatic encoder AE, the encoding matrix after the feature deep fusion of the historical moment t observation data is transformed into a linear layer to obtain the historical moment t hidden space one;

[0121] Step S33, reconstructing the hidden space 1 at historical moment t through the decoder at historical moment t to obtain the reconstructed data 1 at historical moment t;

[0122] Step S34, based on the variational autoencoder VAE at historical time t, the encoding matrix after the feature depth fusion of the observation data at historical time t is subjected to linear layer transformation to obtain the hidden layer space 2 at historical time t;

[0123] Step S35, reconstructing the hidden space 2 at historical moment t through the decoder at historical moment t to obtain the reconstructed data 2 at historical moment t;

[0124] Step S36, inputting the historical moment t observation data, the historical moment t reconstruction data 1 and the historical moment t reconstruction data 2 into the GAN discriminator, respectively comparing the historical moment t reconstruction data 1 and the historical moment t reconstruction data 2 with the real data, and obtaining the historical moment t identification result based on the similarity with the real data;

[0125] Step S37, calculating the loss function based on the identification result at the historical time t to obtain the loss gradient;

[0126] The data reconstruction model is updated based on the loss gradient to obtain an updated data reconstruction model; the updated data reconstruction model is used as the data reconstruction model for the next historical moment;

[0127] Step S38, determine whether t is greater than or equal to T, where T is the total number of historical moments. If so, stop training and obtain updated data to reconstruct the model; if not, set t=t+1 and return to step S32.

[0128] Furthermore, the specific steps of calculating the loss function based on the identification result at the historical moment t in step S37 to obtain the loss gradient and updating the data reconstruction model based on the loss gradient to obtain the updated data reconstruction model include:

[0129] Based on the identification results at the historical time t, the autoencoder AE loss function, the variational autoencoder VAE loss function, the encoder loss function and the discriminator loss function are calculated respectively, and the corresponding loss gradients are obtained respectively;

[0130] Based on the corresponding loss gradients, the autoencoder AE, variational autoencoder VAE, encoder, and discriminator in the data reconstruction model are updated respectively to obtain an updated data reconstruction model.

[0131] Furthermore, the autoencoder AE loss function expression is:

[0132] Loss(AE)=||W′ AE -W||2

[0133] Among them, W′ AE represents the reconstructed data 1, Loss(·) represents the loss function, and W is the real data;

[0134] The expression of the variational autoencoder VAE loss function is:

[0135] Loss(VAE)=||W′ VAE -W||2+D KL (q||p)

[0136] Among them, W ′ VAE To reconstruct data 2, D KL represents the KL divergence of the data distribution, p represents the standard normal distribution, and q represents the model simulation distribution;

[0137] The expression of the encoder loss function is:

[0138] Loss(Encoder)=0.5*Loss(AE)+0.5*Loss(VAE)

[0139] Among them, Encoder is the encoder.

[0140] The discriminator GAN loss function expression is:

[0141] Loss(GAN)=log(Dis(W))+log(1-Dis(W′ AE ))+log(1-Dis(W′ VAE ))

[0142] Among them, Dis(·) represents the probability that the discriminator identifies the data as true. During training, the model hopes that the loss function of the discriminator is gradient-ascending, proving that the discriminator can distinguish between real data and reconstructed data well.

[0143] The present invention continuously iterates and updates in this way, and AE and VAE can learn the characteristics and distribution of normal data. When faced with normal data, the reconstruction error is small, and when faced with abnormal data, the data with large errors will be reconstructed. At the same time, because the discriminator of GAN has learned the distribution pattern of the original normal data, the abnormal data can also be correctly evaluated by the discriminator during detection. In this way, by combining multiple different data reconstruction models, abnormal data can be identified more accurately, avoiding the inability to deal with all anomalies due to the limitations of a single model.

[0144] Step S4: Select the data to be tested of the Internet of Things; reconstruct the model based on the updated data to obtain the abnormal score of the data to be tested;

[0145] Anomaly detection of the data to be tested is achieved based on the anomaly score.

[0146] Preferably, the specific steps of implementing anomaly detection of the data to be tested based on the anomaly score in step S4 include:

[0147] Input the current test data into the updated data reconstruction model, and obtain reconstruction data 3 and corresponding reconstruction error 1, reconstruction data 4 and corresponding reconstruction error 2, identification result and corresponding identification error respectively;

[0148] Based on the reconstruction error 1 and the reconstruction error 2 at the current moment, determine the abnormal probability 1 and the abnormal probability 2 at the current moment, combine the reconstruction error 1 and the abnormal probability 1 at the current moment to obtain the abnormal score of the reconstruction data 1 at the current moment; combine the reconstruction error 2 and the abnormal probability 2 at the current moment to obtain the abnormal score of the reconstruction data 2 at the current moment;

[0149] Determine the identification anomaly probability at the moment based on the identification error at the current moment; combine the identification error at the current moment with the identification anomaly probability to obtain the anomaly score of the identification result at the current moment;

[0150] Obtaining an anomaly score of the reconstructed data at the current moment based on the anomaly score of the reconstructed data one at the current moment and the anomaly score of the reconstructed data two;

[0151] The final anomaly score at the current moment is obtained by weighted summing the anomaly score of the reconstructed data at the current moment and the anomaly score of the identification result at the current moment;

[0152] Preset anomaly score threshold δ;

[0153] Determine whether the final abnormality score at the current moment is greater than the abnormality score threshold, and obtain a determination result;

[0154] If the judgment result is yes, it means that the data to be tested at the current moment is abnormal, and if no, it means that the data to be tested at the current moment is normal.

[0155] Preferably, the final anomaly score is expressed as:

[0156] Anomaly Score=α*Score(Reconstruction)+(1-α)*Score(Discriminination)

[0157] Among them, Anomaly Score is the final anomaly score, α is the weighted summation coefficient, Score(·) is the harmonic function, Reconstruction is the anomaly score of the reconstructed data, and Discrimination is the anomaly score of the identification result.

[0158] Preferably, the judgment result expression is:

[0159]

[0160] Among them, result is the judgment result. When the judgment result is 1, it indicates an abnormality.

[0161] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for detecting anomalies in multidimensional time series data of the Internet of Things, characterized in that: include: Step S1, obtaining multidimensional time series observation data of multiple sensors or controllers in an Internet of Things environment and performing sliding window division to obtain multiple time series with context information; Obtain a multivariate time window series based on multiple time series with context information; Step S2, establishing a dual encoder architecture based on a time dimension encoder and a metric dimension encoder; The multivariate time window sequence is input into a dual encoder architecture, and a coding matrix after deep fusion of the features of the observation data at each moment is obtained based on a multi-head attention mechanism and a cross-attention mechanism; Step S3, constructing a data reconstruction model using an autoencoder, a variational autoencoder, and a GAN discriminator; Using the encoding matrix after deep fusion of the features of the observation data at each moment as a training sample, and using the training sample to perform generative adversarial training on the data reconstruction model to obtain an updated data reconstruction model; Step S4: Select the data to be tested of the Internet of Things; reconstruct the model based on the updated data to obtain the abnormal score of the data to be tested; Anomaly detection of the data to be tested is achieved based on the anomaly score.

2. The anomaly detection method for multidimensional time series data of the Internet of Things according to claim 1 is characterized in that: The specific steps of obtaining the encoding matrix after the feature depth fusion of the observation data at each moment in step S2 include: Establish a dual encoder architecture based on the time dimension encoder and the metric dimension encoder; Input the multivariate time window sequence into a dual encoder architecture, and obtain a time dimension multi-head attention matrix and a metric dimension multi-head attention matrix based on a multi-head attention mechanism; The time dimension multi-head attention matrix and the metric dimension multi-head attention matrix are passed through two residual connection layers and a normalization layer respectively to obtain the time dimension feature encoding matrix and the metric dimension feature encoding matrix; The time dimension feature encoding matrix and the metric dimension feature encoding matrix are deeply fused through a cross attention mechanism to obtain a time dimension cross attention matrix and a metric dimension cross attention matrix; The time dimension cross-attention matrix and the metric dimension cross-attention matrix are concatenated and then passed through two residual connections and a layer normalization layer to perform residual calculation to obtain a coding matrix after deep feature fusion.

3. The anomaly detection method for multidimensional time series data of the Internet of Things according to claim 1 is characterized in that: The specific steps of obtaining the updated data and reconstructing the model in step S3 include: Step S31, let t=t+1, when t=1, it indicates the initial time of history; Obtain the encoding matrix after deep fusion of the features of the observation data at historical time t; Step S32: Input the encoding matrix after deep fusion of the feature of the historical time t observation data into the historical time t data reconstruction model D t , based on the historical moment t automatic encoder AE, the encoding matrix after the feature deep fusion of the historical moment t observation data is transformed into a linear layer to obtain the historical moment t hidden space one; Step S33, reconstructing the hidden space 1 at historical moment t through the decoder at historical moment t to obtain the reconstructed data 1 at historical moment t; Step S34, based on the variational autoencoder VAE at historical time t, the encoding matrix after the feature depth fusion of the observation data at historical time t is subjected to linear layer transformation to obtain the hidden layer space 2 at historical time t; Step S35, reconstructing the hidden space 2 at historical moment t through the decoder at historical moment t to obtain the reconstructed data 2 at historical moment t; Step S36, inputting the historical moment t observation data, the historical moment t reconstruction data 1 and the historical moment t reconstruction data 2 into the GAN discriminator, respectively comparing the historical moment t reconstruction data 1 and the historical moment t reconstruction data 2 with the real data, and obtaining the historical moment t identification result based on the similarity with the real data; Step S37, calculating the loss function based on the identification result at the historical time t to obtain the loss gradient; The data reconstruction model is updated based on the loss gradient to obtain an updated data reconstruction model; the updated data reconstruction model is used as the data reconstruction model for the next historical moment; Step S38, determine whether t is greater than or equal to T, where T is the total number of historical moments. If so, stop training and obtain updated data to reconstruct the model; if not, set t=t+1 and return to step S32.

4. The anomaly detection method for multidimensional time series data of the Internet of Things according to claim 1 is characterized in that: The specific steps of implementing anomaly detection of the data to be tested based on the anomaly score in step S4 include: Input the current test data into the updated data reconstruction model, and obtain reconstruction data 3 and corresponding reconstruction error 1, reconstruction data 4 and corresponding reconstruction error 2, identification result and corresponding identification error respectively; Based on the reconstruction error 1 and the reconstruction error 2 at the current moment, determine the abnormal probability 1 and the abnormal probability 2 at the current moment, combine the reconstruction error 1 and the abnormal probability 1 at the current moment to obtain the abnormal score of the reconstruction data 1 at the current moment; combine the reconstruction error 2 and the abnormal probability 2 at the current moment to obtain the abnormal score of the reconstruction data 2 at the current moment; Determine the identification anomaly probability at the moment based on the identification error at the current moment; combine the identification error at the current moment with the identification anomaly probability to obtain the anomaly score of the identification result at the current moment; Obtaining a reconstruction anomaly score at the current moment based on the anomaly score of the reconstruction data one at the current moment and the anomaly score of the reconstruction data two; The current moment reconstruction anomaly score and the current moment identification result anomaly score are weighted summed to obtain the current moment final anomaly score; Preset anomaly score threshold; It is determined whether the final abnormality score at the current moment is greater than the abnormality score threshold. If so, it indicates that the data to be tested at the current moment is abnormal. If not, it indicates that the data to be tested at the current moment is normal.

5. The anomaly detection method for multidimensional time series data of the Internet of Things according to claim 1 is characterized in that: The multi-head attention matrix of the time dimension and the multi-head attention matrix of the measurement dimension are obtained, and the expression is: Among them, MultiHeadAttention z (W) is the multi-head attention matrix in the time dimension or measurement dimension, is the attention matrix of the head in the time dimension or measurement dimension, concat(·) is the concatenation function of the matrix in the time dimension or measurement dimension, Liner(·) is the linear function, Q z is the query vector of the time dimension or the measurement dimension, K z is the time dimension or measurement dimension key vector, V z is the time dimension or measurement dimension value vector, Attention z (·) is the attention function of the time dimension or the measurement dimension; softmax(·) is the activation function, K z T is the transpose of the time dimension or measure dimension key vector, The dimension of the query vector for the time dimension or the metric dimension to avoid the dot product value being too large.

6. The anomaly detection method for multidimensional time series data of the Internet of Things according to claim 2 is characterized in that: The time dimension feature coding matrix and the measurement dimension feature coding matrix are expressed as follows: C z =LayerNorm(FeedForward(First_Layer z )+First_Layer z ) Among them, C z is the feature encoding matrix of the time dimension or measurement dimension, First_Layer z is the first residual connection layer and normalization layer of the time dimension or measurement dimension, FeedForward(·) is a feedforward neural network, and LayerNorm(·) is a layer normalization function.

7. The anomaly detection method for multidimensional time series data of the Internet of Things according to claim 1 is characterized in that: The specific steps of calculating the loss function based on the identification result at the historical time t in step S37 to obtain the loss gradient include: Based on the identification results at historical time t, the autoencoder AE loss function, the variational autoencoder VAE loss function, the encoder loss function and the discriminator loss function are calculated respectively, and the corresponding loss gradients are obtained respectively.

8. The anomaly detection method for multi-dimensional time series data of the Internet of Things according to claim 7 is characterized in that: The automatic encoder AE loss function expression is: Loss(AE)=||W′ AE |-W||2 Among them, W′ AE represents the reconstructed data, Loss(·) represents the loss function, and W is the real data.

9. The anomaly detection method for multi-dimensional time series data of the Internet of Things according to claim 7 is characterized in that: The discriminator GAN loss function expression is: Loss(GAN) = log(Dis(W)) + log(1 - Dis(W' AE )) + log(1 - Dis(W' VAE )) where, Dis(·) represents the probability that the discriminator identifies the truth, W′ VAE To reconstruct data 2.

10. The anomaly detection method for multi-dimensional time series data of the Internet of Things according to claim 4 is characterized in that: The expression of the final anomaly score is: Anomaly Score=α*Score(Reconstruction)+(1-α)*Score(Discriminination) Among them, Anomaly Score is the final anomaly score, α is the weighted summation coefficient, Score(·) is the harmonic function, Reconstruction is the anomaly score of the reconstructed data, and Discrimination is the anomaly score of the identification result.

Images