Adaptive anomaly detection method and system for multi-hazard IoT time series

Abstract

The present invention discloses a multi-disaster Internet of Things time series adaptive anomaly detection method and system, which relates to the field of Internet of Things. The method includes S1, constructing an anomaly detection model; S2, obtaining a training data set; S3, training an anomaly detection model; S4, obtaining data to be detected; S5, analyzing the reconstructed structure of the data to be detected; S6, analyzing anomaly scores; the emergency disaster warning system includes an acquisition unit, a storage unit, a computing unit, and an early warning unit; it integrates a multi-scale time convolutional network and an adaptive spectrum feature module, deeply explores the characteristics of time series data in the time domain and frequency domain, and introduces a gated memory mechanism to accurately capture and enhance the normal time-frequency features in the data, thereby improving recognition capabilities; the addition of a radial basis function layer significantly improves the model's ability to detect subtle anomalies, enabling the model to more accurately detect tiny abnormal changes.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things, and in particular to a method and system for adaptive anomaly detection of multi-disaster Internet of Things time series. Background Art

[0002] IoT sensor technology has become a crucial technical underpinning of modern disaster early warning systems. In disaster-prone areas, we have deployed large-scale, multifunctional intelligent sensor networks. These networks, equipped with multiple sensor types at each monitoring point, continuously collect real-time monitoring information on a variety of disasters, including mudslides, forest fires, flash floods, and earthquakes. Due to the frequent occurrence of multiple disasters, our monitoring points are widely distributed, covering multiple potential disaster areas. The data collected by these sensor networks is transmitted to data centers and aggregated into complex, multivariate time series data. However, simply collecting data is not enough to address the complexity and uncertainty of disasters. The key lies in efficiently and accurately analyzing this data to identify potential anomalies that could indicate disasters.

[0003] In the IoT environment, most data is presented in the form of time series. The temporal dependencies of data become key features for identifying anomalies, profoundly impacting data prediction and analysis. Therefore, the core problem of anomaly detection in IoT scenarios can often be attributed to time series anomaly detection: identifying abnormal points or patterns in time series data that significantly deviate from the normal pattern.

[0004] Traditional anomaly detection methods mostly rely on empirical judgment and static threshold settings, which not only limits detection accuracy but can also lead to false positives and missed alerts, impacting the timeliness and effectiveness of emergency response. Consequently, traditional technologies are increasingly unable to adapt to modern needs. Deep learning, a key component of machine learning, has gained significant attention in many industries in recent years, particularly in time series analysis and anomaly detection, where it has been widely applied. For example, by utilizing deep learning methods to extract features from time series data, the accuracy of time series prediction and anomaly detection can be effectively improved. Common deep learning models used for time series anomaly detection include those based on recurrent neural networks (RNNs), convolutional neural networks (CNNs), and graph neural networks (GNNs). The Gated Recurrent Unit (GRU) is a special type of RNN. Existing techniques have proposed an unsupervised method combining the GRU with an attention mechanism for anomaly detection in multivariate time series. The GRU captures deep information in the time series, while the attention mechanism dynamically adjusts feature weights, jointly improving the accuracy and efficiency of anomaly detection. The EdgeConvFormer model integrates Time2vec embedding, dynamic graph CNN, and the Transformer to extract global and local spatiotemporal information from time series, improving the accuracy of anomaly detection in complex time series. A variant of the GNN, the Graph Convolutional Network (GCN), is an unsupervised method that combines GCNs. MTGFlow, an unsupervised method that combines GCNs, leverages GCNs to capture variable relationships and captures the complex interdependencies and sparse nature of different entities through dynamic graph structure learning and entity-aware normalization.

[0005] Patent CN202411396506.9, "A Method for Detecting Hydrological Time Series Anomalies Based on Spatiotemporal Features," combines a time-domain convolutional network (TCN) and a graph convolutional neural network (GCN) to effectively address the difficulties of detecting and learning spatiotemporal features in multi-site hydrological time series data. Patent CN202410538278.8, "A Method and System for Detecting Time Series Anomalies Based on a Time-Frequency Masked Autoencoder," eliminates potential anomalous temporal patterns and time points, preventing deep autoencoders from being misled by abnormal data and thus more accurately detecting anomalies in time series. Patent CN202410421523.7, "A Method for Detecting Multivariate Time Series Anomalies Based on Time-Frequency Dual-Stream Graph Interaction," utilizes a graph attention mechanism to construct a relationship matrix and extracts features through spectral graph convolution, ultimately achieving more accurate anomaly detection. Patent CN202410041936.2, "A Method for Unsupervised Time Series Anomaly Detection Based on Multidimensional Feature Fusion," achieves more accurate anomaly detection by combining a multidimensional feature extraction network and fusion strategy with self-supervised training. Patent CN202310823653.9, “A Sensor Data Anomaly Detection Method Based on Adaptive Graph Attention Network,” proposes a method based on GAT and GRU to detect anomalies in sensor data. Patent CN202311440486.6, “A Electricity Meter Anomaly Detection Method Based on Dual-Memory Augmented Autoencoders,” utilizes a dual-memory augmented autoencoder to build an anomaly detection model.

[0006] However, IoT time series data in disaster-prone outdoor harsh natural environments are complex and high-dimensional, so we need to consider how to further improve the accuracy of model anomaly detection; sensor data is easily affected by temperature fluctuations, extreme weather, and unstable power supply, so how to effectively improve the detection performance of the model under noise interference; and in view of the dynamic and multi-anomaly concurrency characteristics of natural disaster emergency response systems, how to continuously optimize the anomaly detection system through statistical analysis, feature learning and other technologies to improve its robustness and accuracy, ensure that research and practical applications can be closely integrated, and thus effectively guarantee the high availability of natural disaster emergency services. These are all difficult. Summary of the Invention

[0007] The purpose of the present invention is to design a multi-disaster Internet of Things time series adaptive anomaly detection method and system to solve the above problems.

[0008] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0009] The adaptive anomaly detection method for multi-hazard IoT time series includes:

[0010] S1. Construct the initial anomaly detection model. The anomaly detection model includes a coding embedding layer, a multi-scale time convolutional network, an adaptive spectrum feature module, a fusion layer, a gate control memory mechanism, a radial basis function layer RBF and a decoder. The output of the coding embedding layer is used as the input of the multi-scale time convolutional network and the adaptive spectrum feature module respectively. The output of the multi-scale time convolutional network and the adaptive spectrum feature module are both used as the input of the fusion layer. The output of the fusion layer is used as the input of the gate control memory mechanism. The output of the gate control memory mechanism is used as the input of the radial basis function layer RBF. The output of the gate control memory mechanism and the output of the radial basis function layer RBF are both used as the input of the decoder. The coding embedding layer is used to positionally encode and embed the input data. The multi-scale time convolutional network is used to capture time domain features of different scales in the time series. The adaptive spectrum feature module is used to perform high-frequency denoising from the frequency domain perspective and extract the frequency domain features of the periodicity of the time series. The fusion layer is used to perform weighted fusion of the time domain features and the frequency domain features to obtain the time-frequency features as the query vector q i The gate-controlled memory mechanism is used to enhance the extracted time-frequency features; the radial basis function layer RBF is used to extract similarity features; the decoder is used to process and transform the features to obtain the reconstructed sequence; the original time series S is defined as a collection of subsequences , where N represents the total number of subsequences, and subsequence Is an observation vector sequence, the observation vector sequence is expressed as ; Encoding embedding layer pair S i The encoded sequence X i The vector corresponding to the nth dimension in ,in, , , L, d and They represent the subsequence length, original data dimension and observation vector at time t, t∈L, represents the set of real numbers;

[0011] S2, obtain training data set;

[0012] S3. Import the training data set into the initial anomaly detection model, and train and optimize it to obtain the optimized anomaly detection model;

[0013] S4. Obtain the data to be detected;

[0014] S5. Analyze the data to be detected using the optimized anomaly detection model to obtain a reconstruction result of the data to be detected;

[0015] S6. Analyze the anomaly score of the data to be detected based on the reconstruction result.

[0016] Multi-hazard IoT time series adaptive anomaly detection system, including:

[0017] Storage; a computer program is stored in the storage;

[0018] Executor; when the executor executes the computer program stored in the memory, the above-mentioned multi-hazard disaster Internet of Things time series adaptive anomaly detection method is implemented.

[0019] Emergency disaster warning system, including:

[0020] Multiple collection units; the collection unit is used to collect monitoring data of each area in real time;

[0021] A storage unit; the storage unit stores a computer program;

[0022] A computing unit; the computing unit is in communication with each acquisition unit and the storage unit; when the computing unit executes the computer program stored in the storage unit to analyze the monitoring data, the above-mentioned multi-hazard disaster IoT time series adaptive anomaly detection method is implemented to obtain an anomaly score;

[0023] The early warning unit is connected to the computing unit in communication. The early warning unit calculates the abnormal event level according to the abnormal score output by the computing unit, generates and sends early warning information.

[0024] The beneficial effects of the present invention are: (1) It solves the challenge of anomaly detection in multivariate time series data of emergency disasters. (2) It uses a multi-scale time convolutional network module to capture the complex long-term and short-term dependencies in time series data, overcoming the conflicts caused by long-term and short-term dependencies in the data. (3) It uses an adaptive spectrum feature module to perform high-frequency denoising from the frequency domain perspective to reduce the impact of high-frequency noise on model performance and extract potential periodic features in the time series through global circular convolution. (4) It introduces a gated memory mechanism to capture the prototype features of normal patterns in the data to strengthen the extracted time-frequency features and thus improve the generalization ability of the model. (5) It introduces a radial basis function layer (RBF) and combines the similarity score obtained from this layer with the reconstruction error to solve the problem that most unsupervised models have difficulty detecting subtle anomalies in complex data sets, thereby improving the model's ability to detect subtle anomalies. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is the overall architecture diagram of the multi-disaster IoT time series adaptive anomaly detection method of the present invention;

[0026] Figure 2 Schematic diagram of the structure of the multi-scale temporal convolutional network of the present invention;

[0027] Figure 3 Schematic diagram of the structure of the adaptive spectrum feature module of the present invention;

[0028] Figure 4 This is a model training flow chart of the adaptive anomaly detection method for multi-disaster IoT time series according to the present invention;

[0029] Figure 5 This is a system architecture diagram of the present invention applied to a multi-hazard emergency disaster large-scale disaster early warning system;

[0030] Figure 6 This is a system flow chart of the present invention applied to a large-scale disaster monitoring and early warning system with multiple disaster types and locations. DETAILED DESCRIPTION

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present invention. It should be understood that the described embodiments are only a portion of the embodiments of the present invention, not all of them. Generally, the components of the embodiments of the present invention described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

[0032] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0033] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0034] In the description of the present invention, it should be understood that, furthermore, the terms “first”, “second”, etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0035] In the description of the present invention, it should also be noted that, unless otherwise expressly specified or limited, terms such as "disposed" and "connected" should be understood in a broad sense. For example, "connected" can mean a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can also mean internal communication between two components. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0036] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0037] Figure 1The blue line in the bottom box is the analyzed anomaly score, and the red line is the set threshold.

[0038] The adaptive anomaly detection method for multi-hazard IoT time series includes:

[0039] S1. Build an initial anomaly detection model, such as Figure 1 As shown in the figure, the anomaly detection model includes a coding embedding layer, a multi-scale time convolutional network, an adaptive spectrum feature module, a fusion layer, a gate control memory mechanism, a radial basis function layer RBF and a decoder. The output of the coding embedding layer is used as the input of the multi-scale time convolutional network and the adaptive spectrum feature module respectively. The output of the multi-scale time convolutional network and the adaptive spectrum feature module are both used as the input of the fusion layer. The output of the fusion layer is used as the input of the gate control memory mechanism. The output of the gate control memory mechanism is used as the input of the radial basis function layer RBF. The output of the gate control memory mechanism and the output of the radial basis function layer RBF are both used as the input of the decoder; the coding embedding layer is used to positionally encode and embed the input data; the multi-scale time convolutional network is used to capture time domain features of different scales in the time series; the adaptive spectrum feature module is used to perform high-frequency denoising from the frequency domain perspective and extract the frequency domain features of the periodicity of the time series; the fusion layer is used to perform weighted fusion of the time domain features and the frequency domain features to obtain the time-frequency features as the query vector q i The gate-controlled memory mechanism is used to enhance the extracted time-frequency features; the radial basis function layer RBF is used to extract similarity features; the decoder is used to process and transform the features to obtain the reconstructed sequence; the original time series S is defined as a collection of subsequences , where N represents the total number of subsequences, and subsequence Is an observation vector sequence, the observation vector sequence is expressed as ; Encoding embedding layer pair S i The encoded sequence X i The vector corresponding to the nth dimension in ,in, , , L, d and They represent the subsequence length, original data dimension and observation vector at time t, t∈L, represents the set of real numbers;

[0040] like Figure 2 As shown in Figure 2, the multi-scale temporal convolutional network includes K convolution kernels of different sizes and an average pooling layer. The output of the encoding embedding layer is used as the input of the K convolution kernels, and the output of the K convolution kernels is used as the input of the average pooling layer. One-dimensional convolution kernels of different sizes are used to act on each dimension of the encoded sequence to extract its features. These convolutions can be expanded to increase their receptive field without increasing the kernel size. The encoded sequence X i The vector corresponding to the nth dimension in , the convolution kernel vector Feature extraction is expressed as: ,in Indicates the use of the kth convolution kernel in Output at any moment, , is the weight of the kth convolution kernel at position j, w k is the size of the kth convolution kernel; average pooling combines the features extracted by each convolution kernel into a unified output, expressed as , Responsible for aggregating the outputs of different convolution scales into a unified vector. The output of the multi-scale convolutional network is , By X i The convolution kernels of different sizes are used in this embodiment to flexibly cover various time scale requirements. The final output of the multi-scale time convolution network is By X i The convolution results of each dimension are combined to effectively capture the temporal features of different scales and better extract their long-term and short-term dependencies. Indicates that all sizes are A collection of real matrices .

[0041] like Figure 3 As shown in the figure, the adaptive spectrum feature module performs high-frequency denoising and extracts the periodic features of the time series from the perspective of the frequency domain, specifically including:

[0042] ①For the embedded sequence X i , along the spatial dimension, the frequency domain data F is obtained by fast Fourier transform i , expressed as: , where F[∙] represents a one-dimensional FFT operation, Represents the length of the frequency domain sequence after transformation, represents a set of complex numbers. Due to the implementation of FFT and the characteristics of time series data, L and May vary. i Each time series channel of is transformed separately to obtain the frequency domain representation F that integrates all channel information i , which encapsulates the spectral characteristics of the original time series;

[0043] ② According to the frequency domain data F i Analyze the power spectrum P i Identify the main frequency components, expressed as: , and adaptively filter according to the preset threshold to obtain the frequency domain data after adaptive filtering , this application uses a trainable threshold To achieve this, the threshold It will be dynamically adjusted according to the spectral characteristics of the data, so the adaptive threshold is set as: ,in represents point-by-point multiplication in the frequency domain, is a binary mask where the power is above the threshold The frequencies of the time series are retained, while other frequencies are filtered out. By adaptively setting the frequency threshold, not only the key information in the sequence is retained, but also high-frequency noise is effectively removed. This adaptive frequency selection allows ASFM filtering to adapt to the characteristics of each time series dataset, thereby improving the overall performance of the model when processing a wide range of data environments.

[0044] ③ Use two sets of learnable filters to extract the frequency domain data F i and frequency domain data Learn features and get features and features , expressed as: 、 , W G and W L denote global and local filters respectively; It is expressed as a point-by-point multiplication in the frequency domain, which is equivalent to the circular convolution operation in the time domain;

[0045] ④ Features and features Fusion is performed to obtain comprehensive spectrum characteristics , expressed as: ,Circular convolution is used to capture the periodic features in time series data because ,it can cover a wide perceptual range of the entire sequence.

[0046] ⑤Use inverse fast Fourier transform to transform the fused spectrum features Convert back to the time domain and get the frequency domain features of the original sequence , expressed as ;

[0047] An adaptive spectral feature module (ASFM) based on Fourier transform processing is used to reduce the impact of high-frequency noise on model performance and extract the periodic features of time series through global circular convolution.

[0048] In order to integrate the information of time domain and frequency domain, the fusion layer fuses the time domain features by weighted and frequency domain characteristics , we get the time-frequency characteristics of the multivariate time series, that is, the query vector , is a separate query vector at time t. This fusion strategy enables us to utilize both time domain features and frequency domain features, thereby improving the model's ability to understand time series data. Weight Set to 10 -2 , the fusion process is expressed as: .

[0049] The gate-controlled memory mechanism includes a memory module, a retrieval module, and a splicing layer. The memory module consists of M memory items. Each memory item is trained to capture the normal pattern in the original time sequence, and the retrieval module is used to i Retrieve the memory items of the normal model stored in the memory module , the concatenation layer converts the query vector and memory items Splice on the feature dimension to generate a new query vector , is the dimension of each memory item;

[0050] Define the memory attention score based on the query vector To incrementally update the memory unit, it is expressed as: ;

[0051] τ is the temperature parameter of the SoftMax function, and its value is set to 10 -1 , The probability distribution is calculated. In order to make the training of the memory module more flexible, the memory module adopts the update gate To control the range of new normal patterns that the memory module should obtain from the query, it is expressed as:

[0052] ;

[0053] ;

[0054] in, and represents a linear projection, and Represent sigmoid activation and element-wise multiplication respectively.

[0055] The update of memory items is only performed during the training phase. Once the training process is completed, the memory items will no longer be updated;

[0056] The retrieval module generates updated queries by retrieving features of the normal model stored in the memory module , which is then used as the input of the subsequent modules. The retrieval process first defines the conditional query attention score on each memory item , which is computed by applying softmax on the dot product between each query and the memory item: ;

[0057] Then, use For memory item m c Perform weighted summation to obtain the retrieved memory items , expressed as: ;

[0058] The query vector and the retrieved memory item Splice on the feature dimension to generate a new query vector This new query vector not only contains the original query information but also incorporates the normal pattern information of the original time series retrieved from the memory item, thereby reducing the abnormal characteristics and making the reconstruction of abnormal data closer to normal samples. Although this feature increases the difficulty of reconstructing anomalies, it also prompts the model to learn more refined feature representations during training. This feature representation enables the model to more effectively distinguish between normal and abnormal data, thereby improving the model's generalization performance.

[0059] The radial basis function layer RBF solves the problem of unsupervised models in detecting subtle anomalies in complex data sets. The radial basis function layer RBF can calculate the similarity score between the data point and a set of learnable centers. This score is particularly effective for anomaly detection tasks. Since anomalies usually deviate from the normal pattern, their similarity scores with the learnable centers are often low, which directly reflects the degree of abnormality of the data point. This similarity score is a useful supplement to the reconstruction error, especially when detecting subtle anomalies that may be overlooked by the reconstruction error. Its detection ability is more prominent. Apply the radial basis function layer RBF to the updated query , by calculating each query vector to a set of learnable centers Z, where , , represents an H-dimensional real vector, Represents a set of H center vectors, where H is the total number of center points in the radial basis function layer RBF, which is randomly initialized at the beginning of training and updated during training. ,in The similarity is expressed as: The parameter γ controls the width of the function, determining how the distance between the query vector and the learnable center is converted into different weights. During the initial training phase of the model, the parameter γ is randomly initialized and adjusted and optimized as training progresses. The exponential property of γ ensures that the scale parameter is always positive.

[0060] The output R of the final radial basis function layer RBF i As the input of the decoder composed of two fully connected layers, the final reconstruction result of the model is recorded as Through this process, the model can fully utilize the similarity features extracted by the radial basis function layer RBF, as well as the further processing and transformation of the features by the decoder, thereby achieving accurate reconstruction of the input data.

[0061] S2, obtain training data set;

[0062] S3, the training data set is imported into the initial anomaly detection model, and the training and optimization are performed to obtain the optimized anomaly detection model; Figure 4 As shown in Figure 2, the training process specifically includes:

[0063] (1) The encoding embedding layer encodes and embeds the training data into positions, and then inputs them into the multi-scale temporal convolutional network and the adaptive spectrum feature module to capture the time domain features and frequency domain features respectively.

[0064] (2) The fusion layer performs weighted fusion of time domain features and frequency domain features to obtain time-frequency features as the query vector q i ;

[0065] (3) Using gated memory mechanism to retrieve and query vectors Related memory items , and concatenate the two to get a new query vector ;

[0066] (4) Radial Basis Function (RBF) analysis query vector Similarity R to a set of learnable centers i ;

[0067] (5) Output R of radial basis function layer RBF i and query vector As the input of the decoder, the reconstructed sequence is obtained ;

[0068] (6) Optimize the model by reducing the reconstruction loss during training. Expressed as: , dense W iThe matrix causes the model to pay indiscriminate attention to all memory items, even if some memory items are not relevant to the current input data. This will cause outliers to have a high correlation with multiple memory items, making it difficult for the model to distinguish normal patterns from abnormal patterns. i yes The matrix representation of and , in order to solve the above problems, entropy loss is introduced As an auxiliary loss function, used to i Matrix sparse regularization, entropy loss Expressed as: ;

[0069] (7) According to the reconstruction loss and entropy loss Analyze the final loss value , expressed as: , where β is the weighting coefficient, which is 10 in this embodiment. -2 ;

[0070] (8) Input the parameters of the anomaly detection model into the Adam optimizer to obtain the gradient of the model parameters, and then update the parameters of the entire anomaly detection model through back propagation;

[0071] (9) Determine whether the number of training times is less than the preset number. If so, end the training and use the current anomaly detection model as the optimized anomaly detection model.

[0072] The anomaly detection model of the present invention is trained in two stages. The main purpose of the first stage of training is to use K-means clustering to initialize each memory item and set it to the approximate normal prototype pattern of the original sequence data. Specifically, the training is carried out through the self-supervised task of reconstructing the input. The trained time-frequency feature extraction module generates query q for 10% of the randomly sampled training data. i Then, we apply K-means clustering algorithm to q i Clustering is performed and each centroid is assigned as the initial value of the memory item. In the second stage, these initialized memory items are mainly used to train the entire model on the anomaly detection task.

[0073] The performance of the models was compared using several key performance metrics based on the confusion matrix: precision, recall, and F1-score. For these three metrics, higher values indicate better performance.

[0074] Precision (Pre) refers to the proportion of samples predicted correctly based on the prediction results. There are two types of results: positive examples (TP) or negative examples (FP). It can be expressed as: ;

[0075] Recall (Rec) refers to the ratio of the number of positive samples that are predicted to be correct to the total number of positive samples. The number of positive samples that are predicted to be correct (TP) or incorrect (FN) is expressed as: ;

[0076] The F1 value is the harmonic mean of precision and recall, and the calculation formula is: .

[0077] S4. Obtain the data to be detected;

[0078] S5. Analyze the data to be detected using the optimized anomaly detection model to obtain a reconstruction result of the data to be detected;

[0079] S6. Analyze the anomaly score of the data to be detected based on the reconstruction result, expressed as: ;in, is the anomaly score at each time point, Measure the dissimilarity, where H is the number of center points in the radial basis function layer RBF, Represents an L-dimensional real vector. The anomaly score improves the detection performance and integrates the normalized radial basis function layer RBF similarity score, the latent space deviation, and the reconstruction error of the original sequence. The normalization process uses the MinMax method to ensure that the scores are comparable. The radial basis function layer RBF similarity score reflects the data point The degree of alignment with the learned center, high similarity indicates normal behavior, while low similarity indicates abnormality. This score is obtained by the radial basis function layer RBF output of all learnable centers. The latent space deviation is defined as each query vector Its most recent memory item Distance in latent space. Since each memory item represents a prototype of a normal pattern, the abnormal latent space deviation will be greater than the normal time point; the reconstruction error is the actual data S i Instead of reconstructing the value The squared difference between the two. This anomaly score allows the model to more comprehensively assess the degree of anomaly in the data, thereby improving detection performance. Points with anomaly scores exceeding a certain threshold are considered anomalies.

[0080] To address the long-term conflict between local and global features in time series, this paper designs a multi-scale temporal convolutional network (MTCN) to capture fine-grained temporal dependencies. The MTCN utilizes one-dimensional convolution kernels of varying sizes to capture temporal features at different scales within the time series. These convolutions can be expanded to increase their receptive field without increasing the kernel size. To extract periodic features from the time series and mitigate the impact of noise on model performance, an adaptive spectral feature module is used to perform high-frequency denoising and feature extraction in the frequency domain. To address the difficulty of most unsupervised models in detecting subtle anomalies in complex datasets, a layer of radial basis functions (RBFs) is introduced. The similarity scores obtained from this layer are combined with the reconstruction error to improve the detection of subtle anomalies. Finally, a gated memory mechanism is introduced to enhance the previously extracted time-frequency features by capturing prototype features of normal patterns in the data, further improving the model's generalization and making it more widely applicable to new data.

[0081] Experimental comparison results: The experimental results of the proposed method on five public datasets are shown in Table 1. The five public datasets are MSL dataset, SWaT dataset, WADI dataset, PSM dataset and AstrosetHigh dataset. Compared with 13 baseline time series anomaly detection methods, the method achieved the best results and improved the average F1-score by 19.21% compared with all the compared methods. The specific compared methods are as follows: Method 1 is Deep Support Vector Data Description (DeepSvDD); Method 2 is Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection (DAGMM); Method 3 is Multivariate Anomaly Detection for Time Series Datawith Generative Adversarial Networks (MAD-GAN); Method 4 is Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network (Robust Anomaly Detection for Multivariate Time Series through Stochastic Recurrent Neural Network, Method 5 is Unsupervised Anomaly Detection on Multivariate Time (USAD); Method 6 is Deep Semi-Supervised Anomaly Detection (DeepSAD); Method 7 is Graph Neural Network-Based Anomaly Detection in Multivariate Time Series (GDN); Method 8 is Time series anomaly detection with association discrepancy (Anomaly transformer);Method 9 is Deep Transformer Networks for Anomaly Detection in Multivariate Time Series Data (TranAD); Method 10 is TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis (TimesNet); Method 11 is Memory-guided Transformer for Multivariate Time Series Anomaly Detection (MEMTO); Method 12 is Dual Attention Contrastive Representation Learning for Time Series Anomaly Detection (DCdetector); Method 13 is Self-Supervised Spatial-Temporal Normality Learning for Time Series Anomaly Detection (STEN).

[0082] Multi-hazard IoT time series adaptive anomaly detection system, including:

[0083] Storage; a computer program is stored in the storage;

[0084] Executor; when the executor executes the computer program stored in the memory, the above-mentioned multi-hazard disaster Internet of Things time series adaptive anomaly detection method is implemented.

[0085] Emergency disaster warning system, including:

[0086] Multiple collection units; the collection unit is used to collect monitoring data of each area in real time;

[0087] A storage unit; the storage unit stores a computer program;

[0088] A computing unit; the computing unit is in communication with each acquisition unit and the storage unit; when the computing unit executes the computer program stored in the storage unit to analyze the monitoring data, the above-mentioned multi-hazard disaster IoT time series adaptive anomaly detection method is implemented to obtain an anomaly score;

[0089] The early warning unit is connected to the computing unit in communication. The early warning unit calculates the abnormal event level according to the abnormal score output by the computing unit, generates and sends early warning information.

[0090] This method is applied to the large-scale monitoring and early warning system of multiple disaster types and disaster points in the field of emergency disasters, such as Figure 5 As shown in the figure, the monitoring areas of four typical disasters (mudslide, forest fire, flood, and earthquake) are displayed. The system implementation process is shown in Figure 6 , the detailed system implementation plan is as follows:

[0091] 1. Deploy an adaptive disaster anomaly warning system: Deploy the anomaly detection model of this method in the data center and ensure communication connection between the data center and the sensors at the monitoring site;

[0092] 2. Historical data preprocessing, using historical data to train model parameters:

[0093] (1) Standardize the format of historical data on debris flows, fires, floods, and earthquakes;

[0094] (2) Fusion of historical data from multiple dimensions into multivariate time series data.

[0095] (3) Using this historical multivariate time series data as the input of the anomaly detection model to train the anomaly detection model;

[0096] 3. Real-time monitoring data preprocessing:

[0097] (1) Collect on-site real-time monitoring data from each area, send it to the data center, and standardize the format of the real-time monitoring data;

[0098] (2) Preprocess the real-time collected data and fuse the monitoring data of multiple dimensions into multivariate time series data;

[0099] (3) Using real-time multivariate time series data as input to the trained anomaly detection model for calculation;

[0100] 4. Real-time monitoring data anomaly detection:

[0101] Use the anomaly score to calculate the abnormal event level, generate and output warning information (ID, monitoring point, time, risk level, etc.);

[0102] 5. Repeat 3-4 to perform real-time disaster anomaly event detection and iteratively update model parameters online.

[0103] Table 1 Performance comparison of the proposed method and 13 baseline methods on 5 datasets

[0104]

[0105] The technical solution of the present invention is not limited to the above-mentioned specific embodiments. Any technical variations made according to the technical solution of the present invention fall within the protection scope of the present invention.

Claims

1. A multi-disaster IoT time series adaptive anomaly detection method, characterized by: include: S1. Construct an initial anomaly detection model. The anomaly detection model includes a coding embedding layer, a multi-scale temporal convolutional network, an adaptive spectrum feature module, a fusion layer, a gate-controlled memory mechanism, a radial basis function layer (RBF), and a decoder. The output of the coding embedding layer is used as the input of the multi-scale temporal convolutional network and the adaptive spectrum feature module, respectively. The output of the multi-scale temporal convolutional network and the adaptive spectrum feature module are both used as the input of the fusion layer. The output of the fusion layer is used as the input of the gate-controlled memory mechanism. The output of the gate-controlled memory mechanism is used as the input of the radial basis function layer (RBF). The output of the gate-controlled memory mechanism and the output of the radial basis function layer (RBF) are both used as the input of the decoder. The coding embedding layer is used to perform position encoding and embedding on the input data. The multi-scale temporal convolutional network is used to capture time domain features of different scales in the time series. The adaptive spectrum feature module is used to perform high-frequency denoising and extract the frequency domain features of the time series periodicity from the perspective of the frequency domain; The fusion layer is used to perform weighted fusion of time domain features and frequency domain features to obtain time-frequency features as the query vector q i ; The gate-controlled memory mechanism is used to enhance the extracted time-frequency features; the radial basis function layer RBF is used to extract similarity features; The decoder is used to process and transform the features to obtain the reconstructed sequence; the original time series S is defined as a collection of subsequences , where N represents the total number of subsequences, and subsequence Is an observation vector sequence, the observation vector sequence is expressed as ; Encoding embedding layer pair S i The encoded sequence X i The vector corresponding to the nth dimension in ,in, , , L, d and They represent the subsequence length, original data dimension and observation vector at time t, t∈L, represents the set of real numbers; S2, obtain training data set; S3. Import the training data set into the initial anomaly detection model, and train and optimize it to obtain the optimized anomaly detection model; S4. Obtain the data to be detected; S5. Analyze the data to be detected using the optimized anomaly detection model to obtain a reconstruction result of the data to be detected; S6. Analyze the anomaly score of the data to be detected based on the reconstruction result.

2. The multi-hazard IoT time series adaptive anomaly detection method according to claim 1 is characterized in that: The multi-scale temporal convolutional network consists of K convolution kernels of different sizes and an average pooling layer. The output of the encoding embedding layer is used as the input of the K convolution kernels, and the output of the K convolution kernels is used as the input of the average pooling layer. The convolution kernel vector Feature extraction is expressed as: ,in, Represents the output of the k-th convolution kernel at time t, , is the weight of the kth convolution kernel at position j, w k is the size of the kth convolution kernel; average pooling combines the features extracted by each convolution kernel into a unified output, expressed as , Responsible for aggregating the outputs of different convolution scales into a unified vector. The output of the multi-scale convolutional network is , By X i The result of convolution on each dimension is combined, where Indicates that all sizes are A collection of real matrices .

3. The multi-hazard IoT time series adaptive anomaly detection method according to claim 1 is characterized in that: The adaptive spectrum feature module performs high-frequency denoising and extracts the periodic features of the time series from a frequency domain perspective, specifically including: ①For the embedded sequence X i , along the spatial dimension, the frequency domain data F is obtained by fast Fourier transform i ; ② According to the frequency domain data F i Analyze the power spectrum P i , and adaptively filter according to the preset threshold to obtain the frequency domain data after adaptive filtering ; ③ Use two sets of learnable filters to extract the frequency domain data F i and frequency domain data Learn features and get features and features ; ④ Features and features Fusion is performed to obtain comprehensive spectrum characteristics ; ⑤Use inverse fast Fourier transform to transform the fused spectrum features Convert back to the time domain and get the frequency domain features of the original sequence , expressed as .

4. The multi-hazard IoT time series adaptive anomaly detection method according to claim 1 is characterized in that: The gate-controlled memory mechanism includes a memory module, a retrieval module, and a splicing layer. The memory module consists of M memory items. Each memory item is trained to capture the normal pattern in the original time sequence, and the retrieval module is used to i Retrieve the memory items of the normal model stored in the memory module , the concatenation layer converts the query vector and memory items Splice on the feature dimension to generate a new query vector , is the dimension of each memory item; The retrieval module is used to i Retrieve the memory items of the normal model stored in the memory module Specifically: Define the conditional query attention score on each memory item , calculated by applying a softmax function on the dot product between each query and the memory item, and then using the attention score For memory item m c Perform weighted summation to obtain the retrieved memory items , attention score Expressed as: ; Memory item Expressed as: , where τ is the temperature parameter of the softmax function, The probability distribution is calculated. represents the exponential function; When training a memory item, the memory attention score is defined based on the query vector To incrementally update the memory module, it is expressed as: , the memory module uses an update gate To control the range of new normal patterns that the memory module should obtain from the query, it is expressed as: ； ； in, and represents a linear projection, and Represent sigmoid activation and element-wise multiplication respectively.

5. The multi-hazard IoT time series adaptive anomaly detection method according to claim 1 is characterized in that: Radial basis function layer RBF analyzes all query vectors Similarity relative to each learning center Z , For each query vector The similarity to each learning center Z is expressed as: ,in, , , represents an H-dimensional real vector, Represents a set of H center vectors, where H is the total number of center points in the radial basis function layer RBF. is a subsequence, and the parameter γ is responsible for regulating the function width.

6. The multi-hazard IoT time series adaptive anomaly detection method according to claim 1 is characterized in that: The optimized anomaly detection model after training optimization specifically includes: (1) The encoding embedding layer encodes and embeds the training data into positions, and then inputs them into the multi-scale temporal convolutional network and the adaptive spectrum feature module to capture the time domain features and frequency domain features respectively. (2) The fusion layer performs weighted fusion of time domain features and frequency domain features to obtain time-frequency features as the query vector q i ; (3) Using gated memory mechanism to retrieve and query vectors Related memory items , and concatenate the two to get a new query vector ; (4) Radial Basis Function (RBF) analysis query vector Similarity R to a set of learnable centers i ; (5) Output R of radial basis function layer RBF i and query vector As the input of the decoder, the reconstructed sequence is obtained ; (6) Analyze reconstruction loss and entropy loss ; (7) According to the reconstruction loss and entropy loss Analyze the final loss value ; (8) Input the parameters of the anomaly detection model into the Adam optimizer to obtain the gradient of the model parameters, and then update the parameters of the entire anomaly detection model through back propagation; (9) Determine whether the number of training times is less than the preset number. If so, end the training and use the current anomaly detection model as the optimized anomaly detection model.

7. The multi-hazard IoT time series adaptive anomaly detection method according to claim 6 is characterized in that: Reconstruction loss Expressed as: , entropy loss Expressed as: Final loss value Expressed as: , where β is the weighting coefficient.

8. The multi-hazard IoT time series adaptive anomaly detection method according to claim 5 is characterized in that: In S6, the anomaly score Score is expressed as: ; is the anomaly score at each time point, Measure the dissimilarity, where H is the total number of center points in the radial basis function layer RBF, represents an L-dimensional real vector.

9. Multi-disaster IoT time series adaptive anomaly detection system, characterized by: include: Storage; The memory has a computer program stored therein; Actuator; When the executor executes the computer program stored in the memory, the method for adaptive anomaly detection of time series of multi-hazard Internet of Things according to any one of claims 1 to 8 is implemented.

10. Emergency disaster warning system, characterized by: include: Multiple acquisition units; The acquisition unit is used to collect monitoring data of each area in real time; storage units; The storage unit stores a computer program; Computational unit; The computing unit is in communication with each acquisition unit and storage unit; When the computing unit executes the computer program stored in the storage unit to analyze the monitoring data, the method for adaptive anomaly detection of time series of multi-hazard Internet of Things disasters according to any one of claims 1 to 8 is implemented to obtain an anomaly score; Early Warning Unit; The early warning unit is connected to the calculation unit for communication. The early warning unit calculates the abnormal event level according to the abnormal score output by the calculation unit, and generates and sends early warning information.

Images