An anomaly detection method for multidimensional time series data in the Internet of Things

Abstract

This invention relates to an anomaly detection method for multidimensional time series data in the Internet of Things (IoT), belonging to the field of IoT anomaly detection technology. This invention deeply extracts and fuses the temporal and metric features of multidimensional time series data in IoT, and combines this with a data reconstruction model to form a generative adversarial network (GAN). Through continuous iterative generative adversarial processing, the model's data reconstruction capability is improved, resulting in smaller reconstruction errors for normal data and relatively larger reconstruction errors for abnormal data. Anomalies are then accurately identified through anomaly scoring. This invention is applicable to data monitoring and detection of multiple sensors or controllers in IoT environments. It can accept multidimensional time series data from real IoT environments and provide timely warnings of potentially anomalies through model detection, avoiding potential problems or losses caused by data anomalies.

Description

Technical Field

[0001] This invention relates to the field of IoT anomaly detection technology, specifically to an anomaly detection method for multidimensional time series data in the Internet of Things. Background Technology

[0002] With the rapid development of the Internet of Things (IoT), its applications are expanding, playing a crucial role in smart cities, smart transportation, smart energy, smart finance, smart agriculture, smart manufacturing, smart construction, smart environmental protection, and smart healthcare. The IoT comprises devices such as sensors, controllers, and control computers. The data generated, collected, or monitored by these devices reflects real-time changes in the physical world and the status of the devices themselves, possessing significant strategic value for real-time data monitoring and precise control. However, in the real world, due to factors such as equipment failure, operational errors, omissions, or cyberattacks, the collected IoT data may exhibit patterns that deviate from normal expectations or operating conditions. For example, it may show abnormal peaks and troughs, or prolonged oscillations. Such data differs significantly from normal observation data and is therefore often referred to as anomalies. Anomaly detection is a technology that identifies anomalies in data, discovering data patterns that do not conform to normal conditions, providing timely warnings and responses, and preventing personal safety issues or serious economic losses caused by anomalies. For example, by detecting anomalies in indicators such as hydrogen, carbon monoxide, and combustion compounds collected by "electronic nose" sensors for forest fires, early signs of wildfires can be detected and warnings issued promptly. Similarly, by detecting anomalies in indicators such as pH, chlorophyll, dissolved oxygen, and heavy metals in water quality sensors, changes in water pollution can be detected in a timely manner. Furthermore, by detecting anomalies in sensors and controllers in water treatment plants, equipment malfunctions can be identified promptly, preventing personal safety issues and economic losses caused by errors in the water treatment process. Therefore, anomaly detection using IoT data has significant practical applications.

[0003] IoT data is typically collected by sensors and exhibits distinct temporal and multivariate characteristics (also known as metric characteristics), hence it is also called multidimensional time series data. Due to the unknown nature of anomalies, the significant differences between anomalies across different domains, and the lack of labeled data, anomaly detection in multidimensional time series data is usually achieved using unsupervised methods, which offer better adaptability and detection performance compared to supervised learning methods. In terms of model complexity, traditional machine learning methods based on threshold detection and density analysis cannot handle complex and dynamically changing data anomalies. Furthermore, multidimensional time series data contain internal relationships, such as temporal and metric relationships, which traditional methods struggle to extract, leading to poor anomaly detection results. Therefore, unsupervised deep learning methods have become the mainstream approach for anomaly detection in multidimensional time series data. Deep learning methods can be further divided into data reconstruction-based and data prediction-based methods. Data reconstruction-based methods learn the data distribution characteristics of multidimensional time series data to reconstruct the data, identifying anomalies by comparing the reconstructed data with the original data. In this approach, the model strives to minimize the difference between normal data and its reconstructed data. However, due to different data distribution characteristics, the reconstructed data of anomalous data will show a greater difference from the original data, thus indicating anomalies. Data prediction methods identify anomalies by predicting IoT data and comparing the prediction errors. However, in complex IoT environments, some data features, such as measurement features, are difficult to predict accurately. Therefore, anomaly detection methods based on data reconstruction are an important research direction for anomaly detection in multidimensional time series data due to their simple approach and excellent performance.

[0004] Current anomaly detection models based on data reconstruction suffer from problems in feature extraction and fusion, as well as data reconstruction. Multidimensional time-series data exhibits significant temporal characteristics and, originating from various sensors, also possesses metric features. Existing methods typically consider only temporal or metric features in feature extraction, neglecting the implicit relationships between them. This lack of feature extraction and fusion directly leads to poor data reconstruction results. Furthermore, in the data reconstruction process, existing methods use basic network structures such as autoencoders (AEs), variational autoencoders (VAEs), or generative adversarial networks (GANs), often employing only a single network structure. This single-structure approach has inherent limitations. For example, AEs learn local features of time-series data and may not effectively reconstruct unknown data; while VAEs incorporate variational latent variables, providing some ability to handle unknown anomalies, the reconstruction results may be more ambiguous compared to AEs, neglecting local variable features; GANs suffer from training difficulties, vanishing gradients, and mode collapse, resulting in unstable performance in anomaly detection. Therefore, these issues mean that a single data reconstruction model may only have good data reconstruction capabilities for some multidimensional time series data, but struggle to reconstruct data well for others, resulting in unstable data reconstruction results. Both of these problems directly affect the reconstruction results of multidimensional time series data, and since data reconstruction is the core of this type of method, poor or unstable data reconstruction results will directly impact the data anomaly detection results. Summary of the Invention

[0005] In view of the above problems, this invention provides an anomaly detection method for multidimensional time series data in the Internet of Things (IoT). It deeply extracts and fuses the temporal and metric features of the IoT multidimensional time series data, and combines this with a data reconstruction model to form a generative adversarial network (GAN). Through iterative generative adversarial processing, the model's data reconstruction capability is improved, resulting in smaller reconstruction errors for normal data and relatively larger reconstruction errors for abnormal data. Anomaly scores are then used to accurately identify anomalies. This invention is applicable to data monitoring and detection using multiple sensors or controllers in IoT environments. It can accept multidimensional time series data from real IoT environments and provides timely warnings of potentially anomalies through model detection, avoiding potential problems or losses caused by data anomalies. It solves the problems of insufficient data feature extraction and fusion, and unstable reconstruction and poor detection performance of single networks in existing anomaly detection methods based on data reconstruction.

[0006] This invention provides an anomaly detection method for multidimensional time series data in the Internet of Things, comprising:

[0007] Step S1: Obtain multi-dimensional time series observation data from multiple sensors or controllers in an IoT environment. The expression is:

[0008]

[0009] in, ; N This represents the number of metric features in the observed data at each time step. Indicates in t The observed value at time, t =1,2,3… T , T This represents the total number of moments, i.e., the length of the time series.

[0010] By dividing the multidimensional time series observation data into sliding window segments, multiple time series with contextual information are obtained, expressed as:

[0011]

[0012] in, for t Time series with contextual information at every moment , This indicates the length of the sliding window sequence.

[0013] Multivariate time window sequences are obtained from multiple time series with contextual information. ;

[0014] Step S2: Establish a dual encoder architecture based on the time-dimensional encoder (Time-Encoder) and the metric-dimensional encoder (Metric-Encoder);

[0015] The multivariate time window sequence is input into the dual encoder architecture, and based on the multi-head attention mechanism and the cross attention mechanism, the feature deep fusion encoding matrix of the observation data at each time point is obtained;

[0016] Preferably, the specific steps for obtaining the coding matrix after feature deep fusion of the observation data at each time point include:

[0017] A dual encoder architecture is established based on a time-dimensional encoder and a metric-dimensional encoder.

[0018] The multivariate time window sequence is input into a dual encoder architecture. Based on a multi-head attention mechanism, the time-dimensional multi-head attention matrix and the metric-dimensional multi-head attention matrix are obtained, expressed as follows:

[0019]

[0020]

[0021]

[0022] in, Query vectors for time or metric dimensions. A key vector for either the time dimension or the metric dimension. A vector of values ​​in either the time or metric dimension. To query the weight matrix of a vector in either the time or metric dimension, The weight matrix is ​​the key vector for either the time dimension or the metric dimension. A weight matrix for a value vector of either the time dimension or the metric dimension. Attention matrices can be defined in either the time or metric dimension. For activation function, This is the transpose of the key vectors in either the time or metric dimension. To query the dimensions of a vector for either the time or metric dimension, the purpose is to avoid obtaining excessively large values ​​from the dot product operation. For multi-head attention matrices, either in the time dimension or the metric dimension, The attention matrix of the head attention unit, which is defined by either the time dimension or the metric dimension. This is a function for concatenating time-dimension or metric-dimension matrices. It is a linear function.

[0023] The time-dimension multi-head attention matrix and the metric-dimension multi-head attention matrix are respectively passed through two residual connection layers and a normalization Add&LayerNorm layer to obtain the time-dimension feature encoding matrix and the metric-dimension feature encoding matrix, as expressed in:

[0024]

[0025] in, For the feature encoding matrix of the time dimension or the metric dimension, For the first residual connection layer and normalization (Add&LayerNorm) layer in the time dimension or metric dimension, It is a feedforward neural network used for further feature extraction and processing. It is a layer normalization function used to standardize the output of each layer of the neural network, thereby enhancing the stability of training.

[0026] Preferably, the expressions for the residual connection layer and the normalized Add&LayerNorm layer are as follows:

[0027]

[0028] in, For the first residual connection layer and normalization (Add&LayerNorm) layer in the time dimension or metric dimension, For the feature encoding matrix of the time dimension or the metric dimension, It is a feedforward neural network used for further feature extraction and processing. It is a layer normalization function used to standardize the output of each layer of the neural network, thereby enhancing the stability of training.

[0029] The time dimension feature encoding matrix and the metric dimension feature encoding matrix are deeply fused through a cross-attention mechanism to obtain a time dimension cross-attention matrix and a metric dimension cross-attention matrix.

[0030] The time-dimensional cross-attention matrix and the metric-dimensional cross-attention matrix are concatenated, and then the residuals are calculated through two residual connections and layer normalization Add&LayerNorm layers to obtain the encoding matrix after feature deep fusion.

[0031] Preferably, the Time-Encoder and the Metric-Encoder have the same structure;

[0032] Preferably, the expression for the time-dimension cross-attention matrix is:

[0033]

[0034] in, For querying vectors in the time dimension, For the metric dimension key vector, Measure dimension value vector, This is the time-crossing attention matrix.

[0035] Step S3: Construct a data reconstruction model using an autoencoder (AE), a variational autoencoder (VAE), and a GAN discriminator;

[0036] The encoding matrix obtained by deep fusion of the features of the observation data at each time point is used as a training sample. The training sample is used to perform generative adversarial training on the data reconstruction model to obtain an updated data reconstruction model.

[0037] Preferably, the specific steps for obtaining the updated data to reconstruct the model include:

[0038] Step S31: Obtain historical moments t The encoding matrix after deep fusion of features from the observation data; when t When =1, it represents the initial moment of history;

[0039] Step S32: Record the historical moments tThe encoding matrix obtained by deep fusion of features from the observation data is input into historical time points. t Data Reconstruction Model D t Based on historical moments t Automatic encoder (AE) for historical moments t The historical moments are obtained by performing a linear layer transformation on the coding matrix after deep feature fusion of the observed data. t Hidden space one;

[0040] Step S33: Through historical moments t Decoder for historical moments t The hidden space is reconstructed to obtain historical moments. t Reconstructing Data 1;

[0041] Step S34, based on historical moments t Variational autoencoder (VAE) for historical moments t The historical moments are obtained by performing a linear layer transformation on the coding matrix after deep feature fusion of the observed data. t Hidden Space Two;

[0042] Step S35: Through historical moments t Decoder for historical moments t The hidden space was reconstructed to obtain historical moments. t Reconstructing Data 2;

[0043] Step S36: Record historical moments t Observational data, historical moments t Reconstructing data and historical moments t The reconstructed data is input into the GAN discriminator to convert historical moments. t Reconstructing data and historical moments t The reconstructed data is compared with the real data, and the similarity is used to obtain historical moments. t Identification results;

[0044] Step S37, Based on historical moments t The loss function is calculated based on the identification results to obtain the loss gradient;

[0045] The data reconstruction model is updated based on the loss gradient to obtain an updated data reconstruction model; the updated data reconstruction model is then used as the data reconstruction model for the next historical moment.

[0046] Step S38, Judgment t If the value is greater than or equal to T, where T is the total number of historical moments, then stop training and reconstruct the model using updated data; otherwise, let... t = t +1, return to step S31.

[0047] Furthermore, step S37, based on historical moments... t The loss function is calculated based on the identification results to obtain the loss gradient; the specific steps for updating the data reconstruction model based on the loss gradient to obtain the updated data reconstruction model include:

[0048] Based on historical moments t The discrimination results are used to calculate the loss function of the autoencoder (AE), the variational autoencoder (VAE), the encoder, and the discriminator, respectively, and the corresponding loss gradients are obtained.

[0049] Based on the corresponding loss gradient, the autoencoder AE, variational autoencoder VAE, encoder, and discriminator in the data reconstruction model are updated respectively to obtain the updated data reconstruction model.

[0050] Furthermore, the expression for the autoencoder AE loss function is as follows:

[0051]

[0052] in, This indicates that data one has been reconstructed. (•) represents the loss function. This is real data;

[0053] The expression for the loss function of the variational autoencoder (VAE) is as follows:

[0054]

[0055] in, To reconstruct data two, The KL divergence represents the data distribution, where p represents the standard normal distribution and q represents the model-simulated distribution.

[0056] The expression for the encoder loss function is:

[0057]

[0058] in, For encoder.

[0059] The loss function expression for the discriminator GAN is as follows:

[0060]

[0061] Where Dis(·) represents the probability that the discriminator identifies the data as true. During training, the model expects the loss function of the discriminator to be gradient-ascending, proving that the discriminator can distinguish between real data and reconstructed data well.

[0062] This invention iterates and updates in this way, allowing the AE and VAE to learn the characteristics and distribution of normal data. This results in small reconstruction errors when dealing with normal data, but larger errors when dealing with anomalous data. Simultaneously, the GAN's discriminator, having learned the distribution patterns of the original normal data, can correctly evaluate anomalous data during detection. By combining multiple different data reconstruction models, anomalous data can be identified more accurately, avoiding the limitations of a single model that cannot handle all anomalies.

[0063] Step S4: Select the IoT test data; reconstruct the model based on the updated data to obtain the anomaly score of the test data;

[0064] Anomaly detection of the test data is achieved based on the anomaly score.

[0065] Preferably, the specific steps of anomaly detection in the test data based on the anomaly score in step S4 include:

[0066] Input the test data at the current moment into the updated data reconstruction model to obtain the reconstructed data three and the corresponding reconstruction error one, the reconstructed data four and the corresponding reconstruction error two, the identification result and the corresponding identification error;

[0067] Based on the reconstruction error 1 and reconstruction error 2 at the current moment, the anomaly probability 1 and anomaly probability 2 at that moment are determined. The reconstruction error 1 and anomaly probability 1 at the current moment are combined to obtain the anomaly score of the reconstructed data 1 at the current moment. The reconstruction error 2 and anomaly probability 2 at the current moment are combined to obtain the anomaly score of the reconstructed data 2 at the current moment.

[0068] The identification error at the current moment is used to determine the identification anomaly probability at that moment; the identification error at the current moment and the identification anomaly probability are combined to obtain the anomaly score of the identification result at the current moment.

[0069] The abnormal score of the reconstructed data at the current moment is obtained based on the abnormal scores of reconstructed data one and reconstructed data two at the current moment.

[0070] The final anomaly score at the current moment is obtained by weighted summing the anomaly score of the reconstructed data at the current moment and the anomaly score of the identification result at the current moment;

[0071] Preset abnormal score threshold ;

[0072] Determine whether the final abnormal score at the current moment is greater than the abnormal score threshold to obtain the determination result;

[0073] If the judgment result is yes, it means that the data to be tested at the current moment is abnormal; if no, it means that the data to be tested at the current moment is normal.

[0074] Preferably, the expression for the final anomaly score is:

[0075]

[0076] in, For the final abnormal score, These are the weighted summation coefficients. For harmonic functions, To reconstruct the outlier scores of the data, To identify abnormal scores in the results.

[0077] Preferably, the expression for the judgment result is:

[0078]

[0079] in, The result is used to determine whether the condition is abnormal.

[0080] Compared with the prior art, the present invention has at least the following beneficial effects:

[0081] (1) The present invention uses transformer to effectively extract time and measurement features through attention mechanism, and enables deep cross-fusion of the two features through cross attention, which can accurately capture the implicit relationship between time features and measurement features;

[0082] (2) The generative adversarial network (GAN) constructed by the multi-data reconstruction model of the present invention. The network is a multi-data reconstruction model GAN ​​composed of AE, VAE and GAN. Among them, AE focuses on the local features of the data, VAE focuses on the global distribution of the data, AE and VAE together constitute the generator of the GAN, and the discriminator of GAN is used to distinguish whether the data is original or generated by the generator. The adversarial network is continuously generated to improve the data reconstruction ability and discrimination ability of the model, thereby improving the stability of data reconstruction and the anomaly detection effect. Attached Figure Description

[0083] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0084] Figure 1 This is a schematic diagram of the anomaly detection process for multidimensional time series data of the Internet of Things in an embodiment of the present invention;

[0085] Figure 2 This is a schematic diagram of the data reconstruction model in an embodiment of the present invention;

[0086] Figure 3 This is a schematic diagram of the structure of the feature extraction and fusion module in an embodiment of the present invention. Detailed Implementation

[0087] To better understand the above-described objectives, features, and advantages of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other. Furthermore, the present invention can be implemented in other ways different from those described herein; therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0088] A specific embodiment of the present invention, such as Figure 1-3 This invention discloses an anomaly detection method for multidimensional time series data in the Internet of Things. To illustrate the effectiveness of the proposed method, a specific embodiment is provided below for detailed explanation of the above technical solution. The specific implementation steps are as follows:

[0089] This invention provides an anomaly detection method for multidimensional time series data in the Internet of Things, comprising:

[0090] Step S1: Obtain multi-dimensional time series observation data from multiple sensors or controllers in an IoT environment. The expression is:

[0091]

[0092] in, ; N This represents the number of metric features in the observed data at each time step. Indicates in t The observed value at time, t =1,2,3… T , T This represents the total number of moments, i.e., the length of the time series.

[0093] By dividing the multidimensional time series observation data into sliding window segments, multiple time series with contextual information are obtained, expressed as:

[0094]

[0095] in, for t Time series with contextual information at every moment , This indicates the length of the sliding window sequence.

[0096] Multivariate time window sequences are obtained from multiple time series with contextual information. ;

[0097] Step S2: Establish a dual encoder architecture based on the time-dimensional encoder (Time-Encoder) and the metric-dimensional encoder (Metric-Encoder);

[0098] The multivariate time window sequence is input into the dual encoder architecture, and based on the multi-head attention mechanism and the cross attention mechanism, the feature deep fusion encoding matrix of the observation data at each time point is obtained;

[0099] Preferably, the specific steps for obtaining the coding matrix after feature deep fusion of the observation data at each time point include:

[0100] A dual encoder architecture is established based on a time-dimensional encoder and a metric-dimensional encoder.

[0101] The multivariate time window sequence is input into a dual encoder architecture. Based on a multi-head attention mechanism, the time-dimensional multi-head attention matrix and the metric-dimensional multi-head attention matrix are obtained, expressed as follows:

[0102]

[0103]

[0104]

[0105] in, Query vectors for time or metric dimensions. A key vector for either the time dimension or the metric dimension. A vector of values ​​in either the time or metric dimension. To query the weight matrix of a vector in either the time or metric dimension, The weight matrix is ​​the key vector for either the time dimension or the metric dimension. A weight matrix for a value vector of either the time dimension or the metric dimension. Attention matrices can be defined in either the time or metric dimension. For activation function, This is the transpose of the key vectors in either the time or metric dimension. To query the dimensions of a vector for either the time or metric dimension, the purpose is to avoid obtaining excessively large values ​​from the dot product operation. For multi-head attention matrices, either in the time dimension or the metric dimension, The attention matrix of the head attention unit, which is defined by either the time dimension or the metric dimension. This is a function for concatenating time-dimension or metric-dimension matrices. It is a linear function.

[0106] The time-dimension multi-head attention matrix and the metric-dimension multi-head attention matrix are respectively passed through two residual connection layers and a normalization Add&LayerNorm layer to obtain the time-dimension feature encoding matrix and the metric-dimension feature encoding matrix, as expressed in:

[0107]

[0108] in, For the feature encoding matrix of the time dimension or the metric dimension, For the first residual connection layer and normalization (Add&LayerNorm) layer in the time dimension or metric dimension, It is a feedforward neural network used for further feature extraction and processing. It is a layer normalization function used to standardize the output of each layer of the neural network, thereby enhancing the stability of training.

[0109] Preferably, the expressions for the residual connection layer and the normalized Add&LayerNorm layer are as follows:

[0110]

[0111] in, For the first residual connection layer and normalization (Add&LayerNorm) layer in the time dimension or metric dimension, For the feature encoding matrix of the time dimension or the metric dimension, It is a feedforward neural network used for further feature extraction and processing. It is a layer normalization function used to standardize the output of each layer of the neural network, thereby enhancing the stability of training.

[0112] The time dimension feature encoding matrix and the metric dimension feature encoding matrix are deeply fused through a cross-attention mechanism to obtain a time dimension cross-attention matrix and a metric dimension cross-attention matrix.

[0113] The time-dimensional cross-attention matrix and the metric-dimensional cross-attention matrix are concatenated, and then the residuals are calculated through two residual connections and layer normalization Add&LayerNorm layers to obtain the encoding matrix after feature deep fusion.

[0114] Preferably, the Time-Encoder and the Metric-Encoder have the same structure;

[0115] Preferably, the expression for the time-dimension cross-attention matrix is:

[0116]

[0117] in, For querying vectors in the time dimension, For the metric dimension key vector, Measure dimension value vector, This is the time-crossing attention matrix.

[0118] Step S3: Construct a data reconstruction model using an autoencoder (AE), a variational autoencoder (VAE), and a GAN discriminator;

[0119] The encoding matrix obtained by deep fusion of the features of the observation data at each time point is used as a training sample. The training sample is used to perform generative adversarial training on the data reconstruction model to obtain an updated data reconstruction model.

[0120] Preferably, the specific steps for obtaining the updated data to reconstruct the model include:

[0121] Step S31: Obtain historical moments t The encoding matrix after deep fusion of features from the observation data; when t When =1, it represents the initial moment of history;

[0122] Step S32: Record the historical moments t The encoding matrix obtained by deep fusion of features from the observation data is input into historical time points. t Data Reconstruction Model D t Based on historical moments t Automatic encoder (AE) for historical moments t The historical moments are obtained by performing a linear layer transformation on the coding matrix after deep feature fusion of the observed data. t Hidden space one;

[0123] Step S33: Through historical moments t Decoder for historical moments t The hidden space is reconstructed to obtain historical moments. t Reconstructing Data 1;

[0124] Step S34, based on historical moments t Variational autoencoder (VAE) for historical moments t The historical moments are obtained by performing a linear layer transformation on the coding matrix after deep feature fusion of the observed data. t Hidden Space Two;

[0125] Step S35: Through historical moments t Decoder for historical moments t The hidden space was reconstructed to obtain historical moments. t Reconstructing Data 2;

[0126] Step S36: Record historical momentst Observational data, historical moments t Reconstructing data and historical moments t The reconstructed data is input into the GAN discriminator to convert historical moments. t Reconstructing data and historical moments t The reconstructed data is compared with the real data, and the similarity is used to obtain historical moments. t Identification results;

[0127] Step S37, Based on historical moments t The loss function is calculated based on the identification results to obtain the loss gradient;

[0128] The data reconstruction model is updated based on the loss gradient to obtain an updated data reconstruction model; the updated data reconstruction model is then used as the data reconstruction model for the next historical moment.

[0129] Step S38, Judgment t If the value is greater than or equal to T, where T is the total number of historical moments, then stop training and reconstruct the model using updated data; otherwise, let... t = t +1, return to step S31.

[0130] Furthermore, step S37, based on historical moments... t The loss function is calculated based on the identification results to obtain the loss gradient; the specific steps for updating the data reconstruction model based on the loss gradient to obtain the updated data reconstruction model include:

[0131] Based on historical moments t The discrimination results are used to calculate the loss function of the autoencoder (AE), the variational autoencoder (VAE), the encoder, and the discriminator, respectively, and the corresponding loss gradients are obtained.

[0132] Based on the corresponding loss gradient, the autoencoder AE, variational autoencoder VAE, encoder, and discriminator in the data reconstruction model are updated respectively to obtain the updated data reconstruction model.

[0133] Furthermore, the expression for the autoencoder AE loss function is as follows:

[0134]

[0135] in, This indicates that data one has been reconstructed. (•) represents the loss function. This is real data;

[0136] The expression for the loss function of the variational autoencoder (VAE) is as follows:

[0137]

[0138] in, To reconstruct data two, The KL divergence represents the data distribution, where p represents the standard normal distribution and q represents the model-simulated distribution.

[0139] The expression for the encoder loss function is:

[0140]

[0141] in, For encoder.

[0142] The loss function expression for the discriminator GAN is as follows:

[0143]

[0144] Where Dis(·) represents the probability that the discriminator identifies the data as true. During training, the model expects the loss function of the discriminator to be gradient-ascending, proving that the discriminator can distinguish between real data and reconstructed data well.

[0145] This invention iterates and updates in this way, allowing the AE and VAE to learn the characteristics and distribution of normal data. This results in small reconstruction errors when dealing with normal data, but larger errors when dealing with anomalous data. Simultaneously, the GAN's discriminator, having learned the distribution patterns of the original normal data, can correctly evaluate anomalous data during detection. By combining multiple different data reconstruction models, anomalous data can be identified more accurately, avoiding the limitations of a single model that cannot handle all anomalies.

[0146] Step S4: Select the IoT test data; reconstruct the model based on the updated data to obtain the anomaly score of the test data;

[0147] Anomaly detection of the test data is achieved based on the anomaly score.

[0148] Preferably, the specific steps of anomaly detection in the test data based on the anomaly score in step S4 include:

[0149] Input the test data at the current moment into the updated data reconstruction model to obtain the reconstructed data three and the corresponding reconstruction error one, the reconstructed data four and the corresponding reconstruction error two, the identification result and the corresponding identification error;

[0150] Based on the reconstruction error 1 and reconstruction error 2 at the current moment, the anomaly probability 1 and anomaly probability 2 at that moment are determined. The reconstruction error 1 and anomaly probability 1 at the current moment are combined to obtain the anomaly score of the reconstructed data 1 at the current moment. The reconstruction error 2 and anomaly probability 2 at the current moment are combined to obtain the anomaly score of the reconstructed data 2 at the current moment.

[0151] The identification error at the current moment is used to determine the identification anomaly probability at that moment; the identification error at the current moment and the identification anomaly probability are combined to obtain the anomaly score of the identification result at the current moment.

[0152] The abnormal score of the reconstructed data at the current moment is obtained based on the abnormal scores of reconstructed data one and reconstructed data two at the current moment.

[0153] The final anomaly score at the current moment is obtained by weighted summing the anomaly score of the reconstructed data at the current moment and the anomaly score of the identification result at the current moment;

[0154] Preset abnormal score threshold ;

[0155] Determine whether the final anomaly score at the current moment is greater than the anomaly score threshold to obtain the determination result;

[0156] If the judgment result is yes, it means that the data to be tested at the current moment is abnormal; if no, it means that the data to be tested at the current moment is normal.

[0157] Preferably, the expression for the final anomaly score is:

[0158]

[0159] in, For the final abnormal score, These are the weighted summation coefficients. For harmonic functions, To reconstruct the outlier scores of the data, To identify abnormal scores in the results.

[0160] Preferably, the expression for the judgment result is:

[0161]

[0162] in, The result is used to determine whether the condition is abnormal.

[0163] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An anomaly detection method for multidimensional time series data in the Internet of Things, characterized in that, include: Step S1: Acquire multi-dimensional time series observation data from multiple sensors in an IoT environment and divide the data into sliding windows to obtain multiple time series with contextual information; A multivariate time window series is obtained based on multiple time series with contextual information; Step S2: Establish a dual encoder architecture based on the time dimension encoder and the metric dimension encoder; The multivariate time window sequence is input into a dual encoder architecture. Based on multi-head attention and cross-attention mechanisms, a deep feature fusion encoding matrix of the observed data at each time step is obtained, including: A dual encoder architecture is established based on a time-dimensional encoder and a metric-dimensional encoder. The multivariate time window sequence is input into the dual encoder architecture, and based on the multi-head attention mechanism, the time dimension multi-head attention matrix and the metric dimension multi-head attention matrix are obtained. The time-dimensional multi-head attention matrix and the metric-dimensional multi-head attention matrix are respectively passed through two residual connection layers and a normalization layer to obtain the time-dimensional feature encoding matrix and the metric-dimensional feature encoding matrix; The time dimension feature encoding matrix and the metric dimension feature encoding matrix are deeply fused through a cross-attention mechanism to obtain a time dimension cross-attention matrix and a metric dimension cross-attention matrix. The time-dimensional cross-attention matrix and the metric-dimensional cross-attention matrix are concatenated, and then the residuals are calculated through two residual connections and a layer normalization layer to obtain the coding matrix after feature deep fusion. Step S3: Construct a data reconstruction model using an autoencoder, variational autoencoder, and GAN discriminator. Use the encoding matrix obtained by deep fusion of features from the observation data at each time step as training samples. Use the training samples to perform generative adversarial training on the data reconstruction model to obtain an updated data reconstruction model. This includes: Step S31: Obtaining historical time steps. t The encoding matrix after deep fusion of features from the observation data; when t When =1, it represents the initial moment of history; Step S32: Record the historical moments t The encoding matrix obtained by deep fusion of features from the observation data is input into historical time points. t Data Reconstruction Model D t Based on historical moments t Automatic encoder (AE) for historical moments t The historical moments are obtained by performing a linear layer transformation on the coding matrix after deep feature fusion of the observed data. t Hidden space one; Step S33: Through historical moments t Decoder for historical moments t The hidden space is reconstructed to obtain historical moments. t Reconstructing Data 1; Step S34, based on historical moments t Variational autoencoder (VAE) for historical moments t The historical moments are obtained by performing a linear layer transformation on the coding matrix after deep feature fusion of the observed data. t Hidden Space Two; Step S35: Through historical moments t Decoder for historical moments t The hidden space was reconstructed to obtain historical moments. t Reconstructing Data 2; Step S36: Record historical moments t Observational data, historical moments t Reconstructing data and historical moments t The reconstructed data is input into the GAN discriminator to convert historical moments. t Reconstructing data and historical moments t The reconstructed data is compared with the real data, and the similarity is used to obtain historical moments. t Identification results; Step S37, Based on historical moments t The loss function is calculated based on the identification results to obtain the loss gradient; The data reconstruction model is updated based on the loss gradient to obtain an updated data reconstruction model; the updated data reconstruction model is then used as the data reconstruction model for the next historical moment. Step S38, Judgment t If the value is greater than or equal to T, where T is the total number of historical moments, then stop training and reconstruct the model using updated data; otherwise, let... t = t +1, return to step S31; Step S4: Select IoT data to be tested; reconstruct the model based on the updated data to obtain the anomaly score of the data to be tested; and implement anomaly detection of the data to be tested based on the anomaly score.

2. The anomaly detection method for multidimensional time series data in the Internet of Things according to claim 1, characterized in that, Step S4, which describes the specific steps for detecting anomalies in the test data based on the anomaly score, includes: Input the test data at the current moment into the updated data reconstruction model to obtain the reconstructed data three and the corresponding reconstruction error one, the reconstructed data four and the corresponding reconstruction error two, the identification result and the corresponding identification error; Based on the reconstruction error 1 and reconstruction error 2 at the current moment, the anomaly probability 1 and anomaly probability 2 at that moment are determined. The reconstruction error 1 and anomaly probability 1 at the current moment are combined to obtain the anomaly score of the reconstructed data 1 at the current moment. The reconstruction error 2 and anomaly probability 2 at the current moment are combined to obtain the anomaly score of the reconstructed data 2 at the current moment. The identification error at the current moment is used to determine the identification anomaly probability at that moment; the identification error at the current moment and the identification anomaly probability are combined to obtain the anomaly score of the identification result at the current moment. The anomaly score for reconstruction at the current moment is obtained based on the anomaly scores of reconstructed data one and reconstructed data two at the current moment. The final anomaly score at the current moment is obtained by weighted summing the anomaly score of the reconstruction at the current moment and the anomaly score of the identification result at the current moment; Preset abnormal score threshold; Determine whether the final abnormal score at the current moment is greater than the abnormal score threshold. If yes, it means that the data to be tested at the current moment is abnormal; otherwise, it means that the data to be tested at the current moment is normal.

3. The anomaly detection method for multidimensional time series data of the Internet of Things according to claim 1, characterized in that, The multi-head attention matrices for the time dimension and the metric dimension are obtained as follows: in, For multi-head attention matrices, either in the time dimension or the metric dimension, The attention matrix of the head attention unit, which is defined by either the time dimension or the metric dimension. This is a function for concatenating time-dimension or metric-dimension matrices. It is a linear function. Query vectors for time or metric dimensions. A key vector for either the time dimension or the metric dimension. A vector of values ​​in either the time or metric dimension. Attention functions for either the time dimension or the metric dimension; For activation function, This is the transpose of the key vectors in either the time or metric dimension. The purpose of querying the dimension of a vector for either the time dimension or the metric dimension is to avoid the dot product operation yielding an excessively large value.

4. The anomaly detection method for multidimensional time series data in the Internet of Things according to claim 2, characterized in that, The expressions for the time dimension feature encoding matrix and the metric dimension feature encoding matrix are: in, For the feature encoding matrix of the time dimension or the metric dimension, For the first residual connection layer and normalization layer in either the time or metric dimension, It is a feedforward neural network. It is a layer normalization function.

5. The anomaly detection method for multidimensional time series data in the Internet of Things according to claim 1, characterized in that, Step S37, based on historical moments t The specific steps for calculating the loss function and obtaining the loss gradient from the identification results include: Based on historical moments t The discrimination results are used to calculate the loss function of the autoencoder (AE), the variational autoencoder (VAE), the encoder, and the discriminator, respectively, and the corresponding loss gradients are obtained.

6. The anomaly detection method for multidimensional time series data in the Internet of Things according to claim 5, characterized in that, The expression for the autoencoder AE loss function is: in, This indicates that data one has been reconstructed. (•) represents the loss function. This is real data.

7. The anomaly detection method for multidimensional time series data in the Internet of Things according to claim 5, characterized in that, The discriminator GAN loss function expression is: Where Dis(·) represents the probability that the discriminator identifies it as true. To reconstruct data two.

8. The anomaly detection method for multidimensional time series data in the Internet of Things according to claim 2, characterized in that, The expression for the final anomaly score is: in, For the final abnormal score, These are the weighted summation coefficients. For harmonic functions, To reconstruct the outlier scores of the data, To identify abnormal scores in the results.

Images