A time series anomaly detection method based on incremental learning

Abstract

The present invention relates to a time series anomaly detection method based on incremental learning, which belongs to the field of anomaly detection technology and solves the problem of inaccurate anomaly detection in the prior art. The method includes: constructing an anomaly detection model for extracting the spatiotemporal dependency features of input data and reconstructing samples based on the spatiotemporal dependency features; preheating the anomaly detection model based on a small batch of stock samples; constructing a dynamic memory pool for storing old samples and corresponding spatiotemporal dependency features; acquiring newly added normal time series data to obtain newly added samples; performing incremental learning training on the anomaly detection model based on the newly added samples and the dynamic memory pool and updating the dynamic memory pool to obtain a trained anomaly detection model; inputting the time series data to be detected into the trained anomaly detection model, and calculating an anomaly score based on the reconstruction error and feature distance of the anomaly detection model output result to obtain an anomaly detection result. Accurate anomaly detection is achieved.

Description

Technical Field

[0001] The present invention relates to the field of anomaly detection technology, and in particular to a time series anomaly detection method based on incremental learning. Background Art

[0002] As a key research area in data mining, time series anomaly detection has become a continuously active research area across various research communities due to its critical role in decision support systems. Time series anomaly detection aims to identify anomalous points or fragments in a data stream that deviate from the expected pattern. It is widely used in fields such as education, finance, healthcare, manufacturing, and transportation.

[0003] In the education sector, abnormal classroom behavior detection can be performed based on time series data of students' head and hand gestures, etc. In the manufacturing sector, anomaly detection can be performed based on time series data of temperature, pressure, vibration, power consumption, etc. detected by sensors in real time.

[0004] In these scenarios, it is often difficult to obtain complete data at one time, and traditional offline training methods based on complete data sets are difficult to apply, resulting in inaccurate detection results of the trained anomaly detection model. Summary of the Invention

[0005] In view of the above analysis, the embodiment of the present invention aims to provide a time series anomaly detection method based on incremental learning to solve the problem of inaccurate detection results of existing anomaly detection models.

[0006] In one aspect, an embodiment of the present invention provides a time series anomaly detection method based on incremental learning, comprising the following steps:

[0007] Constructing an anomaly detection model, which is used to extract spatiotemporal dependency features of input data and reconstruct samples based on the spatiotemporal dependency features; pre-training the anomaly detection model based on a small batch of stock samples;

[0008] Constructing a dynamic memory pool, wherein the dynamic memory pool is used to store old samples and corresponding spatiotemporal dependency features;

[0009] Acquire newly added normal time series data to obtain newly added samples;

[0010] Performing incremental learning training on the anomaly detection model based on the newly added samples and the dynamic memory pool and updating the dynamic memory pool to obtain a trained anomaly detection model;

[0011] The time series data to be detected is input into the trained anomaly detection model, and an anomaly score is calculated based on the reconstruction error and feature distance of the output result of the anomaly detection model to obtain the anomaly detection result.

[0012] Based on the further improvement of the above method, the anomaly detection model includes:

[0013] Series decomposition module, used to decompose input data into seasonal components and trend components;

[0014] A spatiotemporal encoder module is used to perform spatiotemporal feature encoding on the seasonal component and the trend component respectively to obtain spatiotemporal dependency features corresponding to the seasonal component and the trend component;

[0015] The reconstruction module is used to reconstruct samples based on the spatiotemporal dependency characteristics corresponding to the seasonal component and the trend component.

[0016] Based on the further improvement of the above method, the sequence decomposition module decomposes the input data into seasonal components and trend components in the following way:

[0017] According to the formula , Get trend component ;

[0018] in, represents the normalized data of the input data, Represents normalized data The first element of express The tth element of express The first element in express The tth element of represents weighted weight, T represents time step;

[0019] According to the formula Get seasonal components;

[0020] The seasonal component is enhanced in the frequency domain to obtain a frequency domain enhanced seasonal component.

[0021] Based on the further improvement of the above method, the spatiotemporal encoder includes:

[0022] A block module is used to block the seasonal component and the trend component respectively, and extract the feature representation of each block to obtain the seasonal component and the trend component represented by the block;

[0023] The time modeling module is used to perform time convolution on the seasonal component and trend component represented by the blocks, respectively, to obtain the time-dependent features corresponding to the seasonal component and trend component:

[0024] The spatial modeling module is used to model the spatial dependency relationship between variable dimensions for the time dependency characteristics corresponding to the seasonal component and the trend component, respectively, to obtain the spatiotemporal dependency characteristics corresponding to the seasonal component and the trend component.

[0025] Based on the further improvement of the above method, the temporal modeling module performs temporal convolution on the seasonal components represented by the blocks in the following manner to obtain the temporal dependency features corresponding to the seasonal components:

[0026] Perform group convolution on the seasonal components represented by the blocks, and the number of groups is the product of the first dimension and the second dimension of the seasonal components represented by the blocks;

[0027] Two-layer point convolution is used to perform two-layer point convolution on the output features of the grouped convolution to extract the time-dependent features corresponding to the seasonal components.

[0028] Based on the further improvement of the above method, the temporal modeling module performs temporal convolution on the trend component represented by the blocks in the following manner to obtain the time-dependent features corresponding to the trend component:

[0029] Deform the trend component represented by the blocks;

[0030] Perform group convolution on the trend component represented by the deformed block, and the number of groups is the product of the first dimension and the second dimension of the trend component represented by the deformed block;

[0031] Two-layer point convolution is used to perform two-layer point convolution on the output features of the grouped convolution to extract the time-dependent features corresponding to the trend component.

[0032] Based on the further improvement of the above method, the spatial modeling module includes:

[0033] A first graph convolutional network is configured to use graph convolution to model the spatial dependency relationship between variable dimensions of the time-dependent features corresponding to the seasonal components, thereby obtaining spatiotemporal dependency features of the seasonal components; the number of nodes in the first graph convolutional network is equal to the second dimension of the time-dependent features of the seasonal components, and the input feature of each node is the feature of the third dimension corresponding to the node;

[0034] The second graph convolutional network is used to use graph convolution to model the spatial dependency relationship between variable dimensions of the time-dependent features corresponding to the trend component, so as to obtain the spatiotemporal dependency features of the trend component; the number of nodes in the second graph convolutional network is the second dimension of the time-dependent features of the trend component, and the input feature of each node is the feature of the third dimension corresponding to the node.

[0035] Based on the further improvement of the above method, the first graph convolutional network adopts a multi-layer residual structure; the spatial dependency relationship of the time-dependent features corresponding to the seasonal components is modeled in the following way:

[0036] ;

[0037] ;

[0038] in, Represents the features of the first-layer graph convolution output of the first graph convolutional network, represents the time-dependent characteristics of the seasonal component, Represents the first graph convolutional network The features of the layer graph convolution output, Represents the first graph convolutional network The features of the layer graph convolution output, represents a graph convolutional encoder;

[0039] Based on the further improvement of the above method, the following formula is used to calculate the training loss :

[0040] ;

[0041] in, represents the reconstruction error, Represents the playback error, represents the distillation error, and Represents the weight coefficient.

[0042] Based on the further improvement of the above method, the following formula is used to calculate the anomaly score:

[0043] ;

[0044] in, Indicates the data to be detected. represents the output of the anomaly detection model, represents the spatiotemporal dependence characteristics of the seasonal component of the data to be detected, Represents the spatiotemporal dependence characteristics of the trend component of the data to be detected, Represents the spatiotemporal dependency characteristics of the seasonal component of the sample in the dynamic memory pool that is closest to the spatiotemporal dependency characteristics of the seasonal component of the data to be detected, Represents the spatiotemporal dependency characteristics of the trend component of the sample in the dynamic memory pool that is closest to the spatiotemporal dependency characteristics of the trend component of the data to be detected, Represents the L2 norm of the matrix.

[0045] Compared with the existing technology, the present invention constructs an anomaly detection model, first preheats the anomaly detection model based on a small batch of existing samples, and then gradually uses the newly generated data as new samples. Based on the new samples and the dynamic memory pool, the model is continuously updated using incremental learning, thereby adapting to the scenario where it is difficult to obtain complete data of time series data at one time and new data needs to be continuously incorporated for training. By constructing a dynamic memory pool to store old samples, old samples are also extracted from the dynamic memory pool for training during training, thereby achieving continuous memory of knowledge to avoid catastrophic forgetting, making the trained anomaly detection model more robust. The anomaly detection model extracts the spatiotemporal dependency features of the input data. During detection, the anomaly score is calculated based on the reconstruction error and feature distance of the output result of the anomaly detection model to obtain the anomaly detection result of the time series data to be detected, thereby achieving robust online anomaly judgment in the distribution drift scenario, and the detection result is more accurate.

[0046] In the present invention, the above-mentioned technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of the present invention will be described in the following description, and some advantages will become apparent from the description or be learned through practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. Like reference symbols denote like components throughout the accompanying drawings.

[0048] Figure 1 This is a flowchart of a time series anomaly detection method based on incremental learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, and are not used to limit the scope of the present invention.

[0050] A specific embodiment of the present invention discloses a time series anomaly detection method based on incremental learning, such as Figure 1 As shown, the following steps are included:

[0051] S1. Build an anomaly detection model, which is used to extract spatiotemporal dependency features of input data and reconstruct samples based on the spatiotemporal dependency features; preheat training the anomaly detection model based on a small batch of stock samples;

[0052] S2. Construct a dynamic memory pool, which is used to store old samples and corresponding spatiotemporal dependency features;

[0053] S3. Acquire newly added normal time series data to obtain newly added samples;

[0054] S4. Perform incremental learning training on the anomaly detection model based on the newly added samples and the dynamic memory pool and update the dynamic memory pool to obtain a trained anomaly detection model;

[0055] S5. Input the time series data to be detected into the trained anomaly detection model, calculate the anomaly score based on the reconstruction error and feature distance of the output result of the anomaly detection model, and obtain the anomaly detection result.

[0056] Compared with the existing technology, the time series anomaly detection method based on incremental learning provided in this embodiment constructs an anomaly detection model, first preheats the anomaly detection model based on a small batch of existing samples, then gradually uses newly generated data as new samples, and continuously updates the model using incremental learning based on the new samples and dynamic memory pool. This adapts to scenarios where it is difficult to obtain complete data for time series data at one time and new data needs to be continuously incorporated for training. By constructing a dynamic memory pool to store old samples and extracting old samples from the dynamic memory pool for training during training, continuous knowledge memory is achieved to avoid catastrophic forgetting, making the trained anomaly detection model more robust. The anomaly detection model extracts the spatiotemporal dependency features of the input data. During detection, the anomaly score is calculated based on the reconstruction error and feature distance of the anomaly detection model output result to obtain the anomaly detection result of the time series data to be detected, thereby achieving robust online anomaly judgment in the distribution drift scenario and more accurate detection results.

[0057] Time series data, for example, can be industrial time series data such as temperature, pressure, vibration, power consumption, etc. collected by sensors of industrial equipment, or time series data of students' classroom behaviors such as head posture and hand posture.

[0058] When implemented, the data for one sample is represented as ,in, represents the time step of the sample, Represents the data at each time step The variable dimension size. Variable dimensions include temperature dimension and pressure dimension. It should be noted that the sample data used for training are all normal data.

[0059] Specifically, the constructed anomaly detection model includes:

[0060] Series decomposition module, used to decompose input data into seasonal components and trend components;

[0061] A spatiotemporal encoder module is used to perform spatiotemporal feature encoding on the seasonal component and the trend component respectively to obtain spatiotemporal dependency features corresponding to the seasonal component and the trend component;

[0062] The reconstruction module is used to reconstruct samples based on the spatiotemporal dependency characteristics corresponding to the seasonal component and the trend component.

[0063] During implementation, in order to deal with the data drift problem, the anomaly detection model also includes a data preprocessing module before the sequence decomposition module to normalize the input data.

[0064] During implementation, in an unstable environment, data drift causes statistical quantities such as mean and variance to change. Therefore, the present invention first uses window normalization on the input data, and smoothes out the unstable information within the window through the instance-level normalization method to obtain The specific normalization calculation formula is as follows:

[0065] ;

[0066] ;

[0067] ;

[0068] in, represents the mean of the input data sequence, Describes the variance of the input data series.

[0069] In order to capture the long-term temporal dependency and nonlinear intra-periodic temporal dependency of time series, anomaly detection models decompose the input data into seasonal components and trend components through sequence decomposition.

[0070] Specifically, the sequence decomposition module decomposes the input data into seasonal components and trend components in the following way:

[0071] According to the formula , Get trend component ;

[0072] in, represents the normalized data of the input data, Represents normalized data The first element of express The tth element of express The first element in express The tth element of represents weighted weight, T represents time step;

[0073] According to the formula Get seasonal components;

[0074] The seasonal component is enhanced in the frequency domain to obtain a frequency domain enhanced seasonal component.

[0075] When data drifts, non-stationary changes may occur in one or more components of the sequence. Decomposing the sequence helps to reduce the impact of drift. When distribution shift occurs in a component, features can be extracted from other stable components. The sequence decomposition algorithm decomposes the normalized data into seasonal components. and trend component , the most recent data points have a large weight, while the older data have a small weight and are smoothed out, so as to quickly respond to changes in the potential trend of the time series, facilitate accurate capture of the characteristics of the sequence, and improve the accuracy of detection.

[0076] According to the independence assumption, the various parts of the sequence are independent of each other and do not affect each other. Therefore, the subsequent parts have an impact on the trend component. and seasonal components The two parts are processed separately to obtain the trend characteristics and seasonal characteristics of the series.

[0077] Different frequencies are correlated with different types of anomalies. The seasonal component reflects the periodicity in time series data, making it easier to capture abnormal patterns such as seasonality in time series data and learn important frequency features. This helps subsequent modules learn more representative seasonal representations and model the essential characteristics of seasonal quantities. Therefore, frequency feature extraction methods are applied to the seasonal component to enhance the correlation between frequency features and anomalies.

[0078] Specifically, the frequency domain enhanced seasonal component obtained by performing frequency domain enhancement on the seasonal component includes:

[0079] Perform convolution operation on each dimension of seasonal component;

[0080] The frequency domain features of the seasonal component after convolution are extracted based on fast Fourier transform to obtain the frequency domain enhanced seasonal component.

[0081] When implementing, seasonal components Perform group convolution, each variable dimension corresponds to a group, and the number of convolution kernels for each group is , thereby increasing the feature dimension And maintain multidimensional variables through grouping Independence between them, the seasonal component after convolution is reshaped to obtain the embedded representation .

[0082] During implementation, the frequency feature extraction of seasonal components is achieved through complex linear layer weighting, and the calculation formula is as follows:

[0083] ;

[0084] in represents the bias term of the linear layer, represents the inverse fast Fourier transform, represents the fast Fourier transform, represents the frequency characteristics of the seasonal component, represents the weight matrix.

[0085] right Again, the feature dimension is adjusted by group convolution Fusion is performed to obtain the frequency domain enhanced seasonal quantity .

[0086] In order to extract the local and global features of the seasonal component and the trend component, the spatiotemporal dependency features corresponding to the seasonal component and the trend component are extracted through a spatiotemporal encoder, thereby improving the accuracy of reconstruction and further improving the accuracy of detection.

[0087] Specifically, the spatiotemporal encoder includes:

[0088] A block module is used to block the seasonal component and the trend component respectively, and extract the feature representation of each block to obtain the seasonal component and the trend component represented by the block;

[0089] The time modeling module is used to perform time convolution on the seasonal component and trend component represented by the blocks to obtain the time-dependent features corresponding to the seasonal component and trend component:

[0090] The spatial modeling module is used to model the spatial dependency relationship between variable dimensions for the time dependency characteristics corresponding to the seasonal component and the trend component, respectively, to obtain the spatiotemporal dependency characteristics corresponding to the seasonal component and the trend component.

[0091] During implementation, in order to extract local information of the sequence, the seasonal component and trend component Use patch technology to divide the blocks and get and ,in is the size of a patch block, is the number of patches.

[0092] Each block passes Mapping to embedding dimension , apply activation function after seasonal quantity embedding To perform nonlinear transformation, we can get the seasonal component and trend component represented by blocks. , the calculation formula is as follows:

[0093] ;

[0094] ;

[0095] in Indicates batch normalization within each block to maintain distribution, is the activation function.

[0096] Specifically, the temporal modeling module performs temporal convolution on the seasonal components represented by the blocks in the following way to obtain the temporal dependency features corresponding to the seasonal components:

[0097] Perform group convolution on the seasonal components represented by the blocks, and the number of groups is the product of the first dimension and the second dimension of the seasonal components represented by the blocks;

[0098] Two-layer point convolution is used to perform two-layer point convolution on the output features of the grouped convolution to extract the time-dependent features corresponding to the seasonal components.

[0099] When implemented, the seasonal component is more focused on the characteristics within the subsequence to emphasize the repeated seasonal characteristics. When implemented, the seasonal component represented by the block The first and second dimensions are combined to get , use group convolution to do depth convolution. Among them, the size of the first dimension is , the size of the second dimension is , the size of the third dimension is The number of input channels and output channels of grouped convolution is equal to , the number of groups is , each input channel corresponds to a filter, so the variables between channels and blocks are not mixed, but the time dependency within the block is captured, and the distribution is normalized between sequence blocks to obtain the captured Dimension dependency characteristics .

[0100] Since the second dimension N and the third dimension D of the seasonal component are both derived from the time channel, in order to extract the complete time dependency, two point convolutions are used to mix different time channel features of the seasonal component. When implementing, the convolution kernel size of each layer of point convolution is , the number of point convolution groups is , that is, the channels of the variables are not mixed, and gelu is used for activation between the two point convolutions. The input channel of the first layer of point convolution is , the output channel is , the input channel of the second layer of point convolution is , the output channel is , r represents a hyperparameter. The number of convolution kernels in the first layer of point convolution is greater than that in the second layer of point convolution. The number of convolution channels increases first and then decreases, that is, first through a The point convolution layer increases the number of channels to capture more feature interactions, and then passes another The point convolution layer reduces the number of channels to achieve dimensionality reduction or feature selection, achieving The learning of the inter-block information of each channel is used to obtain the time-dependent features corresponding to the seasonal components , in order to facilitate subsequent processing, the time-dependent characteristics corresponding to the seasonal components are transformed into .

[0101] Specifically, the temporal modeling module performs temporal convolution on the trend components represented by the blocks in the following way to obtain the time-dependent features corresponding to the trend components:

[0102] Deform the trend component represented by the blocks;

[0103] Perform group convolution on the trend component represented by the deformed block, and the number of groups is the product of the first dimension and the second dimension of the trend component represented by the deformed block;

[0104] Two-layer point convolution is used to perform two-layer point convolution on the output features of the grouped convolution to extract the time-dependent features corresponding to the trend component.

[0105] The trend component captures the time dependency of the entire sequence and extracts features from the sequence obtained after block division to emphasize the relatively stable trend feature and more effectively capture the pattern dependency between blocks. Transform, Transformed into .

[0106] Perform group convolution on the trend components represented by the deformed blocks, merge the first dimension and the second dimension of the trend components represented by the deformed blocks, and obtain Use grouped convolution to perform depth convolution, where the number of input channels and output channels of grouped convolution is equal to , the number of groups is , each input channel corresponds to a filter, so the different channel layers (a total of D channel layers) between variable channels and tokens in a block will not be mixed, but the time dependency of the entire sequence is captured, and the distribution is normalized between different channels in the block to obtain the captured Dimension dependency characteristics .

[0107] Similarly, since the second and third dimensions of the trend component are derived from the time channel, in order to extract the complete time dependency, two point convolutions are used to mix different time channel features of the trend component. When implementing, the convolution kernel size of each layer of point convolution is , the number of point convolution groups is , that is, the channels of the variables are not mixed, and gelu is used for activation between the two point convolutions. The input channel of the first layer of point convolution is , the output channel is , the input channel of the second layer of point convolution is , the output channel is , r represents a hyperparameter. The number of convolution kernels in the first layer of point convolution is greater than that in the second layer of point convolution. The number of convolution channels increases first and then decreases, that is, first through a The point convolution layer increases the number of channels to capture more feature interactions, and then passes another The point convolution layer reduces the number of channels to achieve dimensionality reduction or feature selection, realizes the learning of the block information of each D-dimensional channel, and obtains the time-dependent features corresponding to the trend component , in order to facilitate subsequent processing, the time-dependent characteristics corresponding to the trend component are transformed into .

[0108] After modeling the temporal dependency, the spatial dependency between variable dimensions is modeled through the spatial modeling module.

[0109] Specifically, the spatial modeling module includes:

[0110] A first graph convolutional network is configured to use graph convolution to model the spatial dependency relationship between variable dimensions of the time-dependent features corresponding to the seasonal components, thereby obtaining spatiotemporal dependency features of the seasonal components; the number of nodes in the first graph convolutional network is equal to the second dimension of the time-dependent features of the seasonal components, and the input feature of each node is the feature of the third dimension corresponding to the node;

[0111] The second graph convolutional network is used for graph convolution to model the spatial dependency relationship between variable dimensions of the time-dependent features corresponding to the trend component, so as to obtain the spatiotemporal dependency features of the trend component; the number of nodes in the second graph convolutional network is the second dimension of the time-dependent features of the trend component, and the input feature of each node is the feature of the third dimension corresponding to the node.

[0112] Specifically, the first graph convolutional network adopts a multi-layer residual structure; the spatial dependency relationship of the time-dependent features corresponding to the seasonal components is modeled in the following way:

[0113] ;

[0114] ;

[0115] in, Represents the features of the first-layer graph convolution output of the first graph convolutional network, represents the time-dependent characteristics of the seasonal component, Represents the first graph convolutional network The features of the layer graph convolution output, Represents the first graph convolutional network The features of the layer graph convolution output, represents a graph convolutional encoder, for example, a Mixhop graph convolutional encoder can be used.

[0116] For example, the number of nodes in each layer of graph convolution in the first graph convolutional network is , the input feature length of each node is dimension, the output feature length is also dimension.

[0117] For the first layer of graph convolution in the first graph convolution network, the time-dependent features corresponding to each block are input into the graph convolution layer, and the obtained features are used as the layer features of the first layer of graph convolution output. The layer features are concatenated together to obtain the output features of the first layer of graph convolution.

[0118] For the first graph convolutional network Layer graph convolution, the output features of the previous layer of graph convolution are input into the first The generated features are used as intermediate features, and these features are connected with the output features of the previous layer of graph convolution to obtain the first Output features of the layer graph convolution.

[0119] After L layers of graph convolution encoding, the third dimension of the output feature is changed from Dimensional mapping back Dimension, obtain the spatiotemporal dependence characteristics of seasonal components .

[0120] Similarly, the second graph convolutional network adopts a multi-layer residual structure; the spatiotemporal dependency features of the trend component are extracted in the following way:

[0121] ;

[0122] ;

[0123] in, Represents the features of the first layer of graph convolution output of the second graph convolutional network, represents the time-dependent characteristics of the trend component, Represents the second graph convolutional network The features of the layer graph convolution output, Represents the second graph convolutional network The features of the layer graph convolution output, Represents a graph convolutional encoder.

[0124] The number of nodes in each layer of graph convolution in the second graph convolution network is , the input feature length of each node is dimensional features, the output feature length is also The different channel layers (a total of D channel layers) of the tokens in the block are input into the second graph convolutional network to obtain the spatiotemporal dependency features corresponding to the different channel layers of the tokens in the block, thereby obtaining the spatiotemporal dependency features corresponding to the trend component.

[0125] For the first layer of graph convolution of the second graph convolution network, the features of each channel layer of the token in the block are input into this layer of graph convolution, and the obtained features are used as a layer of features output by the first layer of graph convolution. The D layer features output by the first layer of graph convolution are spliced ​​together to obtain the output features of the first layer of graph convolution.

[0126] For the second graph convolutional network Layer graph convolution, the output features of the previous layer of graph convolution are input into the first The generated features are used as intermediate features, and these intermediate features are connected with the output features of the previous layer of graph convolution for residual connection to obtain the first Output features of the layer graph convolution.

[0127] will pass After the convolutional encoding of the layer graph, the first dimension of the output feature is changed from Dimensional mapping back The temporal and spatial dependence characteristics of the trend component are obtained by .

[0128] During implementation, the reconstruction module uses the spatiotemporal dependency characteristics corresponding to the seasonal component and the trend component to reconstruct samples:

[0129] ;

[0130] in, Indicates splicing, represents a linear transformation, represents a scalar parameter, Represents the bias vector. Used to control the scaling of features after linear transformation, Used to add a fixed offset to the features after linear transformation to enhance the expressiveness of the model.

[0131] When implemented, the spatiotemporal dependence characteristics of the seasonal component are and the spatiotemporal dependence of the trend component After splicing, the sample reconstruction result is obtained through linear transformation and inverse normalization.

[0132] During implementation, the anomaly detection model can be preheated and trained based on a small batch of stock samples. The loss of preheat training is the reconstruction loss of the sample, that is, and error.

[0133] In order to maintain continuous memory of learning, a dynamic memory pool is constructed to store old samples and corresponding spatiotemporal dependency features. The capacity of the dynamic memory pool is . Combine the spatiotemporal dependency characteristics of small batch sample data and the corresponding seasonal components and the spatiotemporal dependence of the trend component Stored in the dynamic memory pool.

[0134] When there is new data, the new normal time series data is obtained to obtain new samples, and some old samples are randomly selected from the dynamic memory pool to form the current training samples, and the anomaly detection model is incrementally trained.

[0135] When implemented, the training loss is calculated using the following formula :

[0136] ;

[0137] in, represents the reconstruction error, Represents the playback error, represents the distillation error, and Represents the weight coefficient.

[0138] The reconstruction error is the reconstruction result of the training sample With real data It should be noted that only new samples are used to calculate the reconstruction error.

[0139] The features in the dynamic memory pool represent past normal concepts. To prevent forgetting, old data is replayed and used in training. The replay error is used to measure the reconstruction error of old data in the new model.

[0140] Specifically, the playback error is calculated using the following formula:

[0141] ;

[0142] in, represents the real data of the old samples, Represents the reconstruction result of the old sample by the current anomaly detection model, Represents the L2 norm of the matrix.

[0143] The distillation error is used to measure the difference between the features extracted by the new model and the features extracted by the old model. Specifically, the distillation error is calculated using the following formula:

[0144] ;

[0145] in, represents the spatiotemporal dependency characteristics of the seasonal component extracted by the current anomaly detection model on old samples, Represents the spatiotemporal dependency features of the seasonal component extracted from the old samples stored in the dynamic memory pool when the model was last trained; Represents the spatiotemporal dependency characteristics of the trend component extracted by the current anomaly detection model on old samples, Represents the spatiotemporal dependency characteristics of the trend component extracted from the old samples stored in the dynamic memory pool when the model was last trained. Represents the L2 norm of the matrix.

[0146] The degree of model change is reflected by the degree of gradient updates. When the gradient gap exceeds a set threshold, the memory pool is updated. During implementation, if the gradient of the training loss exceeds a preset first threshold, newly added samples from the current training set are randomly selected and their raw data, along with the corresponding spatiotemporal dependency characteristics of the seasonal component and the trend component, are stored in the dynamic memory pool. If the number of samples in the dynamic memory pool exceeds m, some of them are randomly deleted.

[0147] This constrains the modeling of both the new and old models on normal features, thus avoiding the problem of reconstructing the objective function itself that is irrelevant to the field of anomaly detection. The continuously and randomly updated memory pool also prevents the problem of unlabeled anomalies being mixed into the training data. Data is always read randomly, preventing the situation where anomaly concepts are persistently used in training.

[0148] If the gradient of the current training loss is less than the preset second threshold, the training is terminated and a trained anomaly detection model is obtained.

[0149] In the inference stage, the time series data to be detected is input into the trained anomaly detection model, and the anomaly score is calculated based on the reconstruction error and feature distance of the output result of the anomaly detection model to obtain the anomaly detection result.

[0150] Specifically, the anomaly score is calculated using the following formula:

[0151] ;

[0152] in, Indicates the data to be detected. represents the output of the anomaly detection model, represents the spatiotemporal dependence characteristics of the seasonal component of the data to be detected, Represents the spatiotemporal dependence characteristics of the trend component of the data to be detected, Represents the spatiotemporal dependency characteristics of the seasonal component of the sample in the dynamic memory pool that is closest to the spatiotemporal dependency characteristics of the seasonal component of the data to be detected, Represents the spatiotemporal dependency characteristics of the trend component of the sample in the dynamic memory pool that is closest to the spatiotemporal dependency characteristics of the trend component of the data to be detected, Represents the L2 norm of the matrix.

[0153] During implementation, the trend and seasonal component features of the data to be tested are compared with those of the normal concepts in the memory pool. The feature with the smallest difference is selected to reflect the feature distance error. The anomaly score of the data to be tested is calculated based on the reconstruction error and the feature distance error. If the anomaly score exceeds a preset threshold, an anomaly is determined, thus achieving fast and accurate anomaly detection for time series data. In the inference phase, the minimum distance between the current data block and the historical concept is calculated, and the anomaly score is constructed based on the time series reconstruction error. This enables robust online anomaly determination in scenarios with distribution drift, thus achieving incremental update anomaly detection and good adaptability to drift.

[0154] During implementation, when the detection false alarm rate exceeds the preset third threshold, the model continues to be incrementally trained based on the normal data of the false alarm, thereby realizing online anomaly detection under incremental learning.

[0155] Those skilled in the art will appreciate that all or part of the process steps of the above-described embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, such as a magnetic disk, an optical disk, a read-only memory, or a random access memory.

[0156] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by any technician familiar with this technical field within the technical scope disclosed by the present invention should be covered by the scope of protection of the present invention.

Claims

1. A time series anomaly detection method based on incremental learning, characterized in that: The following steps are involved: Constructing an anomaly detection model, wherein the anomaly detection model is used to extract spatiotemporal dependency features of input data and perform sample reconstruction based on the spatiotemporal dependency features; Preheat training the anomaly detection model based on a small batch of stock samples; Constructing a dynamic memory pool, wherein the dynamic memory pool is used to store old samples and corresponding spatiotemporal dependency features; Acquire newly added normal time series data to obtain newly added samples; Performing incremental learning training on the anomaly detection model based on the newly added samples and the dynamic memory pool and updating the dynamic memory pool to obtain a trained anomaly detection model; Input the time series data to be detected into the trained anomaly detection model, calculate the anomaly score based on the reconstruction error and feature distance of the output result of the anomaly detection model, and obtain the anomaly detection result; The anomaly detection model includes: Series decomposition module, used to decompose input data into seasonal components and trend components; A spatiotemporal encoder module is used to perform spatiotemporal feature encoding on the seasonal component and the trend component respectively to obtain spatiotemporal dependency features corresponding to the seasonal component and the trend component; The reconstruction module is used to reconstruct samples based on the spatiotemporal dependency characteristics corresponding to the seasonal component and the trend component; The spatiotemporal encoder includes: A block module is used to block the seasonal component and the trend component respectively, and extract the feature representation of each block to obtain the seasonal component and the trend component represented by the block; The time modeling module is used to perform time convolution on the seasonal component and trend component represented by the blocks, respectively, to obtain the time-dependent features corresponding to the seasonal component and trend component: The spatial modeling module is used to model the spatial dependency relationship between variable dimensions for the time dependency characteristics corresponding to the seasonal component and the trend component, respectively, to obtain the spatiotemporal dependency characteristics corresponding to the seasonal component and the trend component; The spatial modeling module includes: A first graph convolutional network is configured to use graph convolution to model the spatial dependency relationship between variable dimensions of the time-dependent features corresponding to the seasonal components, thereby obtaining spatiotemporal dependency features of the seasonal components; the number of nodes in the first graph convolutional network is equal to the second dimension of the time-dependent features of the seasonal components, and the input feature of each node is the feature of the third dimension corresponding to the node; A second graph convolutional network is configured to use graph convolution to model the spatial dependency relationship between variable dimensions of the time-dependent features corresponding to the trend component, thereby obtaining the spatiotemporal dependency features of the trend component. The number of nodes in the second graph convolutional network is equal to the second dimension of the time-dependent features of the trend component, and the input feature of each node is the feature of the third dimension corresponding to the node. The first graph convolutional network adopts a multi-layer residual structure; the spatial dependency relationship of the time-dependent features corresponding to the seasonal components is modeled in the following way: ； ； in, Represents the features of the first-layer graph convolution output of the first graph convolutional network, represents the time-dependent characteristics of the seasonal component, Represents the first graph convolutional network The features of the layer graph convolution output, Represents the first graph convolutional network The features of the layer graph convolution output, Represents a graph convolutional encoder.

2. The time series anomaly detection method based on incremental learning according to claim 1 is characterized in that The series decomposition module decomposes the input data into seasonal components and trend components in the following way: According to the formula , Get trend component ; in, represents the normalized data of the input data, Represents normalized data The first element of express The tth element of express The first element in express The tth element of represents weighted weight, T represents time step; According to the formula Get seasonal components; The seasonal component is enhanced in the frequency domain to obtain a frequency domain enhanced seasonal component.

3. The time series anomaly detection method based on incremental learning according to claim 1, characterized in that The temporal modeling module performs temporal convolution on the seasonal components represented by the blocks in the following manner to obtain the temporal dependency features corresponding to the seasonal components: Perform group convolution on the seasonal components represented by the blocks, and the number of groups is the product of the first dimension and the second dimension of the seasonal components represented by the blocks; Two-layer point convolution is used to perform two-layer point convolution on the output features of the grouped convolution to extract the time-dependent features corresponding to the seasonal components.

4. The time series anomaly detection method based on incremental learning according to claim 1, characterized in that The temporal modeling module performs temporal convolution on the trend components represented by the blocks in the following manner to obtain the time-dependent features corresponding to the trend components: Deform the trend component represented by the blocks; Perform group convolution on the trend component represented by the deformed block, and the number of groups is the product of the first dimension and the second dimension of the trend component represented by the deformed block; Two-layer point convolution is used to perform two-layer point convolution on the output features of the grouped convolution to extract the time-dependent features corresponding to the trend component.

5. The time series anomaly detection method based on incremental learning according to claim 1, characterized in that The training loss is calculated using the following formula : ； in, represents the reconstruction error, Represents the playback error, represents the distillation error, and Represents the weight coefficient.

6. The time series anomaly detection method based on incremental learning according to claim 1, characterized in that: The anomaly score is calculated using the following formula: ； in, Indicates the data to be detected. represents the output of the anomaly detection model, represents the spatiotemporal dependence characteristics of the seasonal component of the data to be detected, Represents the spatiotemporal dependence characteristics of the trend component of the data to be detected, Represents the spatiotemporal dependency characteristics of the seasonal component of the sample in the dynamic memory pool that is closest to the spatiotemporal dependency characteristics of the seasonal component of the data to be detected, Represents the spatiotemporal dependency characteristics of the trend component of the sample in the dynamic memory pool that is closest to the spatiotemporal dependency characteristics of the trend component of the data to be detected, Represents the L2 norm of the matrix.

Images