Anomaly Localization Method for IoT Sensing Cloud Data Centers Based on Data Augmentation
By constructing a data-augmented anomaly detection model and utilizing an LSTM network with data augmentation techniques and attention mechanisms, the challenge of anomaly detection in multivariate time-series data in IoT and cloud data centers was solved, achieving higher detection accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIHUA UNIV
- Filing Date
- 2023-03-03
- Publication Date
- 2026-05-26
AI Technical Summary
Existing anomaly detection methods face problems of high false positive rates and insufficient detection accuracy in IoT and cloud data centers, especially when dealing with non-stationary, nonlinear, and high-noise multivariate time series data, making it difficult to effectively detect anomalies.
An anomaly detection model is constructed using a data augmentation-based approach, including a data augmentation-neural conversion layer, a ConvLSTM encoding layer, and an attention-based autoregressive LSTM network layer. New training samples are generated through data augmentation techniques, and feature extraction and dimensionality reduction are performed by combining a convolutional long short-term memory network and an attention mechanism to improve the robustness and accuracy of the model.
It improves the accuracy and robustness of anomaly detection in multivariate time series data, reduces false positives, and enhances the model's detection performance in complex IoT environments.
Smart Images
Figure CN116167008B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of cloud data, and in particular to a method for anomaly localization in IoT-based cloud data centers based on data augmentation. Background Technology
[0002] In 1991, Professor Kevin Ashton of MIT first proposed the concept of the Internet of Things. In 1999, MIT established the "Auto-ID Center" and proposed that "everything can be interconnected through the network", which clarified the basic meaning of the Internet of Things [1]. The early Internet of Things was a logistics network based on radio frequency identification (RFID) technology. With the development of technology and applications, the connotation of the Internet of Things has changed significantly. The Internet of Things is an interconnected network based on information carriers such as the Internet and traditional telecommunications networks. Through information sensing devices, according to the agreed protocol, any object is connected to the network. Objects exchange and communicate information through information transmission media to achieve intelligent identification, positioning, tracking and supervision. Its application areas mainly include transportation and logistics, industrial manufacturing, health care, smart environment (home, office, factory), etc., and have a very broad market prospect. The Internet of Things has the following characteristics: ① Overall perception. Various types of sensors are deployed on the Internet of Things. Each sensor is an information source. The information content and information format captured by different types of sensors are different. The data obtained by the sensors are real-time. Environmental information is collected periodically at a certain frequency and the data is constantly updated. ② Reliable transmission: The Internet remains a crucial foundation and core of IoT technology. By integrating with the Internet through various wired and wireless networks, information about objects can be transmitted accurately and in real time. ③ Intelligent processing: IoT combines sensors with intelligent processing, utilizing cloud computing, pattern recognition, and other intelligent technologies to expand its application areas. Meaningful data is analyzed, processed, and analyzed from the massive amounts of information obtained from sensors to adapt to the different needs of different users and discover new application areas and patterns. However, with the continuous development and application of big data technology, the amount of data collected in real time from IoT is growing exponentially. Furthermore, a small portion of this data contains anomalies or even noise interference; unless processed, it cannot yield valuable insights.
[0003] Data augmentation is a technique that generates new training samples from existing training samples (adding data representing different features). It is widely used for image data, but less so for multivariate time-series anomaly detection. Data augmentation can increase training data, improve model generalization ability, avoid sample imbalance, prevent overfitting in neural networks, and enhance model robustness. This has a significant impact on anomaly detection for IoT data in cloud data centers.
[0004] In the early stages of IoT development, researchers had limited access to real-time data, and the time-series data they analyzed was primarily univariate. Time-series anomalies in monitored objects could be detected quickly and accurately. However, with the rapid growth of IoT and big data, massive amounts of non-stationary, nonlinear, and noisy real-time data are being collected, mostly stored in cloud data centers as multivariate time series (cloud data centers are internet-based computing methods that provide shared hardware and software resources and information to computers and other devices on demand). This presents researchers with significant challenges. They must not only preprocess the massive amounts of real-time data stored in cloud data centers but also perform anomaly detection on the processed multivariate time series (anomalies are often characterized by randomness and scarcity). The purpose of detection is to monitor the state of monitored objects, thereby determining whether the cloud data center is operating normally and whether there are any anomalies in the IoT that require intervention. Therefore, anomaly detection of the collected real-time multivariate time series is currently a hot research topic in order to improve the reliability of IoT and cloud data centers.
[0005] In recent years, researchers both domestically and internationally have made considerable progress in multivariate time series anomaly detection. Based on whether or not neural networks are used, time series anomaly detection algorithms can be mainly divided into the following two categories:
[0006] 1. Traditional time series anomaly detection methods. The 3-sigma[2] method is used when the selected data follows a normal distribution, and outliers exceeding 3 times the standard deviation can be considered outliers. The ARIMA[3] integrated moving average autoregressive model learns a period of historical data and then predicts, and judges whether it is an anomaly by comparing the deviation between the true value and the predicted value. SH-ESD[4] is a seasonal anomaly detection method released by Twitter. Histogram-based Outlier Score (HBOS)[5] assumes that each dimension of the dataset is independent of each other, and then divides each dimension into intervals. The higher the density of the intervals, the lower the anomaly score. The Principal Component Analysis (PCA)[6] method uses the weight of the feature values to calculate the distance difference between the feature vectors corresponding to the feature values of the sample points, thereby calculating the degree of deviation of the data value from that direction. It identifies anomalies based on the cumulative degree of deviation in each direction.
[0007] 2. Anomaly detection methods based on deep learning. The Local Neural Transformations (LNT) [7] method uses data augmentation techniques. It integrates local neural transformations on the basis of CPC architecture and detects anomalies based on the proposed dynamic determination contrast loss. The core idea of Neural Transformation Learning for Anomaly Detection (NeuTraL AD) [8] is to embed the transformed data into a semantic space so that the transformed data is still similar to its untransformed form, and different transformations are easy to distinguish. Then, this learnable transformation is used for anomaly detection. The Multi-Scale Convolutional Recurrent Encoder-Decode (MSCRED) [9] model reconstructs the signature matrix through a convolutional encoder and decoder, and further utilizes the residual signature matrix to detect anomalies. The OmniAnomaly
[10] model learns latent representations while considering time dependence and randomness to capture normal patterns in multivariate time series. And patterns with low reconstruction probability are regarded as anomalies. Deep Transformer Networks for Anomaly Detection (TranAD)
[11] is a typical example of combining Transformer for anomaly detection. It uses attention mechanisms to learn time trends and combines model-independent meta-learning to ensure its performance even on limited datasets, and still judges anomalies by reconstruction error. Multivariate Anomaly Detection with GAN (MAD-GAN)
[12] uses LSTM-RNN as the basic model for GAN learning to analyze complex multivariate dependencies between multiple data streams, and uses generators and discriminators trained by GAN to detect anomalies. This model adopts a new anomaly score (combining the discrimination result of each test sample and the reconstruction residual).
[0008] Due to the interaction between the temporal and spatial characteristics of IoT time-series data stored in cloud data centers in real time, as well as the randomness and scarcity of anomalies themselves, the false positive rate may increase, and the lack of anomaly labels makes anomalies difficult to detect. Therefore, existing anomaly detection methods still have the following shortcomings: ① Due to the complexity and high dynamism of IoT monitoring objects, the non-stationarity, nonlinearity, and high noise characteristics of the collected real-time data increase the difficulty of extracting correlation information between different temporal and spatial characteristics of the time series, leading to an increased false positive rate in anomaly detection. ② Faced with high-dimensional, massive amounts of data, existing deep learning-based detection methods can only improve the accuracy of anomaly detection for specific scenarios, which is far from meeting the requirements of complex IoT. Summary of the Invention
[0009] The purpose of this invention is to design an anomaly localization method for IoT sensing cloud data centers based on data augmentation in order to solve the above problems.
[0010] The present invention achieves the above objectives through the following technical solutions:
[0011] Anomaly localization methods for IoT-sensing cloud data centers based on data augmentation include:
[0012] S1. Obtain the training dataset ,in This represents the maximum length of a multivariate time series. The number of feature dimensions in a multivariate time series;
[0013] S2. Construct an anomaly detection model, which includes a data augmentation-neural conversion layer, a ConvLSTM encoding layer, and an attention-based autoregressive LSTM network layer. The data augmentation-neural conversion layer is a residual network that includes a residual block.
[0014] S3. Import the training dataset into the anomaly detection model for optimized training;
[0015] S4. Obtain the raw data from the cloud data center in real time and import it into the optimized anomaly detection model to obtain the predicted value of the raw data;
[0016] S5. Determine the anomalies in the raw data of the cloud data center based on the predicted values and the actual values of the raw data.
[0017] The beneficial effects of this invention are as follows: This technology addresses the problem of anomaly detection in real-time time series data generated by monitored objects in the Internet of Things (IoT). It employs data augmentation techniques to design an unsupervised end-to-end anomaly detection method based on neural transformation. Its advantages are mainly reflected in: ① Utilizing data augmentation—neural transformation and convolutional long short-term memory networks—to mine different aspects of the temporal and spatial characteristics of multivariate time series, increasing the amount of training data and reducing false positive detections caused by non-stationary and nonlinear real-time IoT data. ② Employing an attention-based autoregressive LSTM network to reduce the dimensionality of the data and extract features again, fully capturing information beneficial to prediction from the data, and improving the robustness of the model's anomaly detection. Attached Figure Description
[0018] Figure 1 This is a schematic diagram of the anomaly localization method for IoT sensing cloud data centers based on data augmentation according to the present invention;
[0019] Figure 2 This is a flowchart illustrating the anomaly localization method for IoT sensing cloud data centers based on data augmentation, as described in this invention.
[0020] Figure 3 This is a comparison chart of the F1 scores of the present invention and all baseline methods on four datasets;
[0021] Figure 4 This is a comparison chart of the running time of the present invention and all baseline methods on four datasets;
[0022] The corresponding figure labels are:
[0023] Figure 1 In the diagram, 'a' is the data augmentation-neural conversion layer, 'b' is the ConvLSTM encoding layer, and 'c' is the attention-based autoregressive LSTM network layer. Detailed Implementation
[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0025] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0026] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.
[0027] In the description of this invention, it should be understood that the terms "upper," "lower," "inner," "outer," "left," "right," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product of this invention is in use, or the orientation or positional relationship commonly understood by those skilled in the art. They are only used to facilitate the description of this invention and to simplify the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.
[0028] Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be interpreted as indicating or implying relative importance.
[0029] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, terms such as "set" and "connection" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0030] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0031] Anomaly localization methods for IoT-sensing cloud data centers based on data augmentation include:
[0032] S1. Obtain the training dataset ,in This represents the maximum length of a multivariate time series. This represents the number of feature dimensions of a multivariate time series.
[0033] S2. Construct an anomaly detection model, which includes a data augmentation-neural conversion layer, a ConvLSTM encoding layer, and an attention-based autoregressive LSTM network layer. The data augmentation-neural conversion layer is a residual network that includes a residual block.
[0034] The residual block is a 2D convolution with two concatenated kernels of 3×3. Each 2D convolution includes an instance normalization layer and a ReLU activation layer, which are respectively processed using Batch Normalization and ReLU activation functions.
[0035] The ConvLSTM encoding layer consists of two ConvLSTM units, each using 64 filters and a 1×3 convolutional kernel, with the ReLU activation function.
[0036] The attention-based autoregressive LSTM network consists of a first self-attention layer, an LSTM network, and a second self-attention layer.
[0037] S0. Normalize the imported anomaly detection model data using the mean-variance normalization method, and then apply the normalization to the data based on a preset sliding window. The normalized data is then input into the anomaly detection model, where... Indicates the number of subsequence partitions. S0 represents the dimension of the subsequence, with S0 located between S1 and S3. A normalization approach is used to improve the accuracy and convergence speed of the anomaly detection model. This normalizes dimensions with different dimensions in multivariate time series data to the same dimension, reducing the impact of differences in scale, features, and distribution on the model.
[0038] S3. Import the training dataset into the anomaly detection model for optimization training, specifically including:
[0039] S31. The imported anomaly detection model data passes through a data augmentation-neural transformation layer. During the neural transformation, three transformations are performed to obtain three different views. , represented as ,in This represents the data after it has been processed by the sliding window; It is a stack of residual blocks from a 2D convolutional layer. This represents the transformed view generated through neural transformation; each view... These represent different aspects of the time series data, and they are all different from each other. This allows us to distinguish these different features, which not only increases the representation of different aspects of the data features, but also increases the amount of data available for training.
[0040] S32, Transfer different views The data is concatenated and integrated together and then fed into the ConvLSTM encoding layer, such as... Figure 1 As shown in b, the view is separated and temporal and spatial features are extracted to obtain the latent representation. , represented as ;
[0041] S33, Latent Representation The predicted value is obtained through an attention-based autoregressive LSTM network layer. ,in Indicates inclusion and all previous context information, Then it means the first The predicted value for the next time stamp;
[0042] S34. Use the mean squared error between the predicted value and the true value as the loss function of the anomaly detection model;
[0043] S35. Employ an attention mechanism to calculate data weights, extract useful information and perform dimensionality reduction based on the calculated weights, as shown below. ,in This indicates that potential variables that provide useful information have been considered. This represents the data input into the attention. Represents the weight matrix. This refers to the weighted probability of each point. This step further enables the anomaly detection model to focus on useful information and suppress useless interference, thereby improving the accuracy and robustness of anomaly detection. It allows the values predicted by the attention-based autoregressive LSTM network layer to better approximate the true values, thus achieving our final anomaly detection goal.
[0044] S36. Based on the useful information of the loss function and weights of the anomaly detection model, dimensionality reduction is used to guide the training of the anomaly detection model.
[0045] S4. Obtain the raw data from the cloud data center in real time and import it into the optimized anomaly detection model to obtain the predicted value of the raw data;
[0046] S5. Determine anomalies in the raw data of the cloud data center based on the predicted values and the actual values of the raw data, specifically including:
[0047] S51. The error between the predicted value and the true value of the original data is used as the outlier score, expressed as follows: ;
[0048] S52. When the anomaly score of each data point is greater than a preset threshold τ, that data point is considered an anomaly. The threshold τ is expressed as: ,in These are the outlier scores during the verification period. It is set to maximize the F1 score during validation.
[0049] The F1 score is expressed as ,in , TP, FN, and FP represent correctly predicting a positive example (true example), incorrectly predicting a negative example (false negative example), and incorrectly predicting a positive example (false positive example), respectively.
[0050] Model performance metrics
[0051] The model performance was compared using several key performance metrics based on the confusion matrix for classification: precision, recall, and F1 score. Since the data collected from the Internet of Things is acquired in real time, we also recorded the running time of all models to verify their performance.
[0052]
[0053] Precision refers to the prediction result; it represents the probability that a sample predicted as positive is actually positive. The expression is:
[0054]
[0055] Recall refers to the original sample; it represents the probability that a sample that is actually positive will be predicted as positive. The expression is:
[0056]
[0057] F1 considers both precision and recall, aiming to maximize both while achieving a balance. The F1 score expression is:
[0058]
[0059] Running Time directly records the time taken by the model from start to finish, in seconds.
[0060] Model comparison results
[0061] from Figure 3 , Figure 4 As shown in Tables 1 and 2, compared with existing models, the experimental results of this model on four real datasets are as follows:
[0062] ① As shown in Table 1, our model outperforms state-of-the-art methods on four publicly available datasets: SWAT, SKAB, MSL, and SMAP, with relative improvements in F1 scores of 9.28%, 0.32%, 12.95%, and 10.15%, respectively. Our model achieves better precision than state-of-the-art methods on three of the four datasets (excluding the SWAT dataset) and better recall than state-of-the-art methods on two of the other datasets (excluding the SWAT and MSL datasets). While these two metrics are not optimal on some datasets, they are not the worst either. The excellent robustness of our model is reflected in the fact that, except for the SMAP dataset (where our model still achieves the best F1, precision, and recall), all metrics remain above 0.9, a feat unmatched by any baseline method.
[0063] ②From Table 2 and Figure 4 As we can see, our model performs well in terms of runtime, slightly longer than traditional methods, but in the middle range compared to deep learning methods. Although our model takes longer to run on high-dimensional time series datasets than on low-dimensional time series datasets, compared to the time-optimal model on each dataset, our model's runtime growth rate on high-dimensional data is only slightly higher than that on low-dimensional data.
[0064] like Figure 3As shown, our model, after fully incorporating the spatiotemporal characteristics of multivariate time series, achieves a higher F1 score than all state-of-the-art methods, demonstrating the robustness and generalization of our model.
[0065] Table 1. Performance comparison of this technique with 6 anomaly detection methods on 4 datasets.
[0066]
[0067] Table 2. Comparison of Running Time of this technique with 6 anomaly detection methods on 4 datasets.
[0068]
[0069] The technical solutions of the present invention are not limited to the specific embodiments described above. Any technical modifications made in accordance with the technical solutions of the present invention fall within the protection scope of the present invention.
Claims
1. A method for anomaly localization in IoT-based sensing cloud data centers based on data augmentation, characterized in that: include: S1. Obtain the training dataset , where n represents the maximum length of the multivariate time series, and T represents the number of feature dimensions of the multivariate time series; S2. Construct an anomaly detection model, which includes a data augmentation-neural conversion layer, a ConvLSTM encoding layer, and an attention-based autoregressive LSTM network layer. The data augmentation-neural conversion layer is a residual network that includes a residual block. The attention-based autoregressive LSTM network layer includes a first self-attention layer, an LSTM network, and a second self-attention layer. S3. Import the training dataset into the anomaly detection model for optimization training; specifically including: S31. Imported data from the anomaly detection model is processed through a data augmentation-neural conversion layer to obtain different transformed views; S32. Integrate the different views together and feed them into the ConvLSTM encoding layer, separate the views and extract temporal and spatial features to obtain the latent representation; S33. The latent representation is predicted through an attention-based autoregressive LSTM network layer; S34. Use the mean squared error between the predicted value and the true value as the loss function of the anomaly detection model; S35. Employ an attention mechanism to calculate data weights, extract useful information and perform dimensionality reduction based on the calculated weights, as shown below. ,in This indicates that potential variables that provide useful information have been considered. This represents the data input into the attention. Represents the weight matrix. These are the weighted probabilities of each point; S36. Based on the useful information of the loss function and weights of the anomaly detection model, dimensionality reduction is used to guide the training of the anomaly detection model. S4. Obtain the raw data from the cloud data center in real time and import it into the optimized anomaly detection model to obtain the predicted value of the raw data; S5. Determine anomalies in the raw data of the cloud data center based on the predicted values and the actual values of the raw data; specifically including: S51. The error between the predicted value and the true value of the original data is used as the outlier score, expressed as follows: ; S52. When the anomaly score of each data point is greater than a preset threshold τ, that data point is considered an anomaly. The threshold τ is expressed as: ,in These are the outlier scores during the verification period. It is set to maximize the F1 score during validation.
2. The anomaly localization method for IoT sensing cloud data centers based on data augmentation according to claim 1, characterized in that, The residual block is a 2D convolution with two concatenated kernels of size 3×3. Each 2D convolution includes an instance normalization layer and a ReLU activation layer, which use batch normalization and ReLU activation functions respectively. The ConvLSTM encoding layer consists of two ConvLSTM units. Each ConvLSTM unit uses 64 filters and a 1×3 kernel, and the ReLU activation function is used.
3. The anomaly localization method for IoT sensing cloud data centers based on data augmentation according to claim 1, characterized in that, Between S1 and S3, there is also S0, which uses the mean-variance normalization method to normalize the data imported into the anomaly detection model, and performs data normalization according to a preset sliding window. The normalized data is then input into the anomaly detection model, where... Indicates the number of subsequence partitions. This represents the dimension of the subsequence.
4. The anomaly localization method for IoT sensing cloud data centers based on data augmentation according to claim 3, characterized in that, In S31, the data is processed through a data augmentation-neural transformation layer to obtain different transformed view representations. ,in This represents the data after it has been processed by the sliding window; It is a stack of residual blocks from a 2D convolutional layer. This represents a transformed view generated through neural transformation.
5. The anomaly localization method for IoT sensing cloud data centers based on data augmentation according to claim 1, characterized in that, The F1 score is expressed as ,in , TP, FN, and FP represent correctly predicting a true example, incorrectly predicting a false negative example, and incorrectly predicting a false positive example, respectively.