Abnormality detection model training method, anomaly detection method and electronic equipment

Abstract

The invention provides an anomaly detection model training method, an anomaly detection method and electronic equipment. According to the method, the feature extraction module and the preset mask generation strategy are introduced, so that the problem that high-precision real-time anomaly detection cannot be realized on edge equipment due to too high calculation complexity of a traditional diffusion model in the cloud environment is effectively solved. Specifically, the model can quickly generate reconstructed multivariate time sequence data within a small number of iterations by extracting global features in a feature extraction module and combining a multi-layer denoising process, so that the calculation burden and the reasoning delay are greatly reduced. At the same time, the adopted mask strategy further improves the sensitivity and robustness of the model to anomalies, and ensures a high-precision anomaly detection result.

Description

Technical Field

[0001] This invention relates to the technical field of time series data processing, and specifically to a training method for an anomaly detection model, an anomaly detection method, and an electronic device. Background Technology

[0002] The cloud environment refers to a technical environment that enables unified scheduling and management of distributed computing resources through a collaborative computing architecture across multiple cloud service providers or cross-regional cloud data centers.

[0003] In existing technologies, traditional diffusion models in cloud environments suffer from high computational complexity, making it impossible for edge devices to achieve high-precision real-time anomaly detection. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned technical deficiencies and provide a training method for an anomaly detection model, an anomaly detection method, and an electronic device to solve the technical problem in the related technology that the high computational complexity of traditional diffusion models in cloud environments prevents edge devices from achieving high-precision real-time anomaly detection.

[0005] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for training an anomaly detection model, comprising: Obtain raw multivariate time series data; The anomaly detection model is trained based on the original multivariate time series data and a preset mask generation strategy to obtain the anomaly detection model. The time series anomaly detection model includes a feature extraction module and an anomaly detection backbone module. The feature extraction module extracts corresponding global features based on the original multivariate time series data, and the preset mask generation strategy generates mask data based on the global features. The anomaly detection backbone module performs multi-layer denoising based on the global features and the mask data, outputting reconstructed multivariate time series data.

[0006] Further, the step of training the anomaly detection model based on the original multivariate time series data and a preset mask generation strategy to obtain the anomaly detection model includes: Based on the original multivariate time series data, a preset preprocessing is performed to obtain trend component data and seasonal component data; The trend component data and seasonal component data are respectively input into the feature extraction module for feature extraction to obtain global features; For the global features, a preset mask generation strategy is executed to obtain at least one type of mask data; Using the mask data of at least one of the above and the global features as input, the data is fed into the anomaly detection backbone module. Under the control of a preset loss function, joint training is performed on the feature extraction module and the anomaly detection backbone module to obtain an anomaly detection model.

[0007] Furthermore, the feature extraction module includes a time-series feature extraction unit, which is used to process the seasonal component data, including: The seasonal component data is projected into a deep feature space through an embedding layer; Perform a short-time Fourier transform on the projected features to identify the target frequency and its corresponding period length; For each identified target frequency and its period length, the following processing is performed: the one-dimensional sequence is reshaped into a two-dimensional tensor according to the period length, where the row dimension represents the number of periods and the column dimension represents the time point within the period; The two-dimensional tensor is input into a hierarchical convolutional feature extraction subunit that includes multiple convolutions, and the short-term dependencies within the period and the long-term change patterns across periods are captured through hierarchical convolution operations. Reshape the feature tensor output by the hierarchical convolutional feature extraction subunit back into a one-dimensional form; An amplitude-based adaptive aggregation method is adopted to weight and fuse the one-dimensional representations corresponding to each significant frequency to generate a time-series feature representation, wherein the weighting coefficients are dynamically determined by the amplitude values ​​of the corresponding significant frequencies.

[0008] Furthermore, the feature extraction module further includes a correlation feature extraction unit, which is used to process the temporal feature representation, including: The temporal feature representation is projected onto the deep feature space through an embedding layer; The projected features were analyzed using short-time Fourier transform to identify the target frequency and its corresponding period length. For each target frequency and its corresponding period length, the following processing is performed: First, the one-dimensional sequence is converted into a two-dimensional tensor through a reshaping operation. The dimension of the two-dimensional tensor is determined by the period length and the frequency dimension. Then, graph convolution feature extraction is performed on the two-dimensional tensor to obtain node features. Finally, multilayer perceptron processing is performed on the node features. Reshape the processed node features back into a one-dimensional form; An amplitude-based adaptive aggregation method is used to weight and merge the one-dimensional representations corresponding to each frequency to generate a correlation feature representation, where the weights are determined by the amplitude values ​​of the corresponding significant frequencies.

[0009] Furthermore, the feature extraction module also includes a feature fusion unit, which is used to fuse the correlation feature representation with the trend component data to obtain the global feature.

[0010] Further, the step of executing a preset mask generation strategy for the global features to obtain at least one type of mask data includes: Based on the local outlier factor algorithm, the global features are analyzed to identify outliers and mark them as forced mask regions. A preset raster masking strategy is applied along the time dimension to generate at least three complementary mask instances by masking data at fixed intervals; wherein the masking windows of each mask instance are staggered along the time axis, and the window that is masked in any raster masking strategy remains unmasked in other raster masking strategies. The forced mask region is combined with the shielding region of the raster mask instance to generate the final mask data.

[0011] Furthermore, the anomaly detection backbone module is a diffusion model based on a stacked Transformer encoder; the preset loss function includes a diffusion loss term, a temporal feature loss term, and a correlation feature loss term; wherein, the diffusion loss term is used to calculate the L2 norm distance between the reconstructed multivariate time series data output by the anomaly detection backbone module and the unmasked region in the global features; the temporal feature loss term is used to constrain the reconstruction error between the temporal feature representation output by the feature extraction module and the seasonal component data; the correlation feature loss term is used to constrain the similarity between the adaptive adjacency matrix generated by the feature extraction module and the correlation matrix of the true variables.

[0012] Furthermore, the method also includes: The anomaly detection model is used as the teacher model; Construct a student model; wherein the model structure of the student model is the same as that of the anomaly detection model, and the number of sampling steps and parameters of the student model is less than that of the teacher model; Iterative distillation training is performed; during the distillation training process, the number of sampling steps of the student model is gradually reduced, and the output distribution of the teacher model is used as a supervision signal to optimize the student model. After training, the student model was used as a lightweight anomaly detection model.

[0013] Secondly, the present invention provides an anomaly detection method, comprising: Receive multivariate time series data to be detected; The multivariate time series data is input into the above-mentioned anomaly detection model or the above-mentioned lightweight anomaly detection model for detection to obtain the reconstructed multivariate time series data. Based on the multivariate time series data to be detected and the reconstructed multivariate time series data, error data is calculated; Based on the error data, outliers in the multivariate time series data to be detected are identified.

[0014] Thirdly, the present invention provides an electronic device, comprising: a memory, and one or more processors communicatively connected to the memory; the memory stores instructions executable by the one or more processors, the instructions being executed by the one or more processors to cause the one or more processors to implement the method described above.

[0015] Beneficial effects: This invention effectively solves the problem that traditional diffusion models in cloud environments cannot achieve high-precision real-time anomaly detection on edge devices due to excessive computational complexity by introducing a feature extraction module and a preset mask generation strategy. Specifically, by extracting global features in the feature extraction module and combining it with a multi-layer denoising process, the model can quickly generate reconstructed multivariate time-series data in fewer iterations, thereby significantly reducing computational burden and inference latency. At the same time, the masking strategy further improves the model's sensitivity and robustness to anomalies, ensuring high-precision anomaly detection results. Attached Figure Description

[0016] Figure 1 This is a scenario example diagram illustrating a training method for an anomaly detection model provided in an embodiment of the present invention; Figure 2 This is a flowchart of a training method for an anomaly detection model provided in an embodiment of the present invention; Figure 3 This is an architecture diagram of the feature extraction module provided in an embodiment of the present invention; Figure 4 This is an architecture diagram of the anomaly detection backbone module provided in an embodiment of the present invention; Figure 5 This is an architecture diagram of the temporal feature extraction unit provided in an embodiment of the present invention; Figure 6 This is an architecture diagram of the relevance feature extraction unit provided in an embodiment of the present invention; Figure 7 This is a scenario example diagram of an anomaly detection method provided in an embodiment of the present invention; Figure 8 This is a schematic diagram of the training process provided in an embodiment of the present invention. Detailed Implementation

[0017] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0018] In related technologies, with the rapid development of cloud computing and edge computing, the demand for processing diverse time-series data in cloud environments is constantly increasing. Especially in the field of real-time anomaly detection, traditional diffusion models have great theoretical potential due to their excellent generation capabilities and efficient anomaly detection performance. However, in practical applications, especially on resource-constrained edge devices, how to meet real-time requirements while ensuring high accuracy has become a major challenge.

[0019] In the current cloud environment, edge devices typically face challenges such as limited computing resources, memory constraints, and high real-time requirements. To process and analyze large amounts of multivariate time-series data, diffusion models, as powerful generative models, generate high-quality samples through multiple iterations and perform anomaly detection via denoising. However, traditional diffusion models usually require multiple iterations to generate stable outputs, and this high computational complexity makes their application on edge devices extremely difficult. In particular, when processing multivariate time-series data, due to its high temporal sequence and complex correlations, traditional models often fail to perform effective anomaly detection in real-time environments.

[0020] The key issue is that traditional diffusion models have excessively high computational complexity, requiring not only strong computing power but also ample memory resources. Therefore, even in cloud environments, the computing power and memory resources of edge devices often cannot support such high-load computational tasks, resulting in compromised accuracy and efficiency in real-time anomaly detection.

[0021] To address the aforementioned technical issues, this embodiment provides a training method for an anomaly detection model. The method can be executed by a central server or a distributed computing system within a cloud-based collaborative computing platform. The central server or distributed computing system can be responsible for executing large-scale model training tasks. During training, the central server utilizes multivariate time-series data collected from edge devices, leveraging more powerful computing resources to perform complex feature extraction and train the core anomaly detection module. By executing training in the cloud, high model accuracy can be ensured, while simultaneously optimizing the model structure, ensuring that the training process is not affected by the resource limitations of edge devices.

[0022] like Figure 1 and Figure 2 As shown, the method may include: Step S12: Obtain the raw multivariate time series data.

[0023] In this embodiment, the acquisition action can be that an executing entity (e.g., a central server) receives raw multivariate time-series data uploaded by a user or sensor device, which can be used as training data. Specifically, the user or sensor device can upload the generated multivariate time-series data to the central server via the Internet. The multivariate time-series data can be transmitted to the central server via API interfaces, file uploads, or real-time streaming. The raw multivariate time-series data can include multiple feature values ​​at different time points, such as temperature, humidity, flow rate, cost, sales volume, etc. It is understood that if it is a sensor-based monitoring system, the sensors can periodically record relevant multivariate data and upload this data to the central server in real time or in batches. For example, environmental monitoring sensors collect data such as temperature, humidity, and PM2.5 concentration.

[0024] Uploaded data can be in various formats, such as CSV, JSON, XML, or other standard time-series data formats. To facilitate subsequent processing, the central server can perform format conversion and standardization on the data. Understandably, after receiving the raw multivariate time-series data, the central server can store it in a database or distributed storage system. The raw multivariate time-series data can be structured (tabular data) or unstructured (text, sensor records, etc.), but it must be sorted according to a certain timestamp to ensure that the data can be arranged in a time series so that the model can learn the temporal characteristics of the data.

[0025] Step S14: Train the anomaly detection model to be trained based on the original multivariate time series data and a preset mask generation strategy to obtain the anomaly detection model; wherein, the time series anomaly detection model to be trained includes a feature extraction module and an anomaly detection backbone module; the feature extraction module is used to extract corresponding global features based on the original multivariate time series data, and the preset mask generation strategy is used to generate mask data based on the global features; the anomaly detection backbone module performs multi-layer denoising based on the global features and the mask data, and outputs reconstructed multivariate time series data.

[0026] In this embodiment, after the executing entity receives the original multivariate time series data, it can perform preprocessing on the original multivariate time series data (e.g., removing missing values, handling outliers, deleting duplicate data, normalizing data format, etc.) to improve data quality.

[0027] In this embodiment, after the executing entity receives the original multivariate time series data, it can also perform standardization or normalization processing on the original multivariate time series data so that data of different dimensions can be analyzed and trained on the same scale.

[0028] In this embodiment, after the execution entity receives the original multivariate time series data, it can also perform data segmentation on the original multivariate time series data. Specifically, the training data can be segmented into multiple small time periods (e.g., using a sliding window method). Each time period corresponds to a set of time series data as a sample input to the model.

[0029] In this embodiment, once the original multivariate time series data has been processed through the above steps, the central server can transfer it to the subsequent anomaly detection model to be trained.

[0030] In this embodiment, the input data for the feature extraction module can be the raw multivariate time series data processed as described above. Specifically, the feature extraction module may include: a time-series feature extraction unit, a correlation feature extraction unit, and a feature fusion unit. The time-series feature extraction unit processes the seasonal component data to obtain a time-series feature representation. The correlation feature extraction unit processes the time-series feature representation to obtain a correlation feature representation. The feature fusion unit fuses the correlation feature representation with the trend component data to obtain global features.

[0031] More specifically, in the feature extraction module, the temporal feature extraction unit can be configured to process the decomposed seasonal component data. The specific data processing steps include: first, projecting the input data into a deep feature space through an embedding layer to enhance the expressive power of the features; then applying a short-time Fourier transform (STFT) to perform spectral analysis on the projected features to identify significant dominant frequencies and their corresponding period lengths; for each identified significant period, performing a one-dimensional to two-dimensional tensor reshaping operation to transform the original sequence into a matrix structure with the number of periods as rows and time points within the period as columns, thus making the implicit temporal pattern explicit; inputting the reshaped two-dimensional tensor into a hierarchical convolution processing module, which can use multi-level convolution kernels to extract features in both row and column directions, thereby simultaneously capturing local detail features within the period and long-term evolution patterns across periods; finally, restoring the two-dimensional features output by the convolution to a one-dimensional representation, and dynamically weighting and aggregating them based on the amplitude values ​​of each frequency component to generate a temporal feature representation characterizing periodic dynamics.

[0032] In the feature extraction module, the correlation feature extraction unit can be configured to operate on the temporal feature representation, aiming to uncover the dynamic correlation between multivariate variables. Its specific data processing includes: in the initial stage, the unit performs deep feature mapping through an embedding layer, followed by reusing short-time Fourier transform to identify frequency structures; for each significant frequency period, the one-dimensional to two-dimensional transformation process is repeatedly performed to form a node feature matrix; the dependency weights between variables are modeled through an adaptive adjacency matrix generation mechanism—specifically, by using two sets of trainable parameter matrices to calculate the potential correlation strength between nodes; the transformed two-dimensional tensor is processed using a graph convolution method based on a multi-order neighborhood aggregation mechanism, simultaneously fusing feature information from directly adjacent nodes and higher-order indirectly related nodes; after nonlinear transformation of the aggregated features by a multilayer perceptron, the processing result is reconstructed back into a one-dimensional sequence form; finally, weights are assigned according to frequency amplitude to achieve adaptive fusion of correlation features at different scales, outputting a correlation feature representation that characterizes the complex interaction patterns between variables.

[0033] In the feature extraction module, the feature fusion unit is configured to receive the feature representation generated by the correlation feature extraction unit and perform multimodal fusion with the trend component data separated in the preprocessing stage. The specific data processing includes: using a weighted concatenation mechanism to combine the two types of features—the correlation feature representation mainly carries the dynamic correlation information between variables, while the trend component represents the long-term evolution direction of the data. The fusion process can introduce learnable gating weight coefficients to automatically adjust the contribution ratio of the two types of features, ensuring that the fusion result retains both spatial correlation and temporal trend. The final generated global feature representation serves as the input to the anomaly detection backbone module. Its dimensionality is significantly lower than the original input data, but its information density is greatly improved, laying the foundation for accurate reconstruction of the subsequent lightweight diffusion model.

[0034] On the one hand, by projecting the original multivariate time series data into a deep feature space, the temporal feature extraction unit enhances the data's expressive power, making potential temporal patterns and periodic changes more significant. Through deep feature mapping, the system can better capture complex patterns in the data. On the other hand, using Short-Time Fourier Transform (STFT) for spectral analysis of the input data can identify significant periodic changes. Through one-dimensional to two-dimensional tensor reshaping operations, implicit temporal patterns are made explicit, allowing periodic dynamic features to be presented more intuitively. Furthermore, hierarchical convolution processing utilizes multi-level convolutional kernels to extract features in different directions, thereby simultaneously capturing local details within a period and long-term evolutionary patterns across periods. This multi-scale feature extraction helps enhance the modeling ability of complex temporal patterns.

[0035] In this embodiment, the anomaly detection backbone module can be a diffusion model based on stacked Transformer encoders.

[0036] In this embodiment, the anomaly detection backbone module can adopt a lightweight convolutional diffusion architecture.

[0037] In this embodiment, the anomaly detection backbone module can employ a latent space diffusion model. That is, the original features are compressed into a low-dimensional latent space using a variational autoencoder (VAE) to perform diffusion.

[0038] In this embodiment, the anomaly detection backbone module can employ a generative adversarial network (GAN).

[0039] In this embodiment, the anomaly detection backbone module can be a self-encoder.

[0040] In this embodiment, the anomaly detection backbone module can employ a recurrent neural network.

[0041] This implementation effectively addresses the problem that traditional diffusion models in cloud environments cannot achieve high-precision real-time anomaly detection on edge devices due to excessive computational complexity. Specifically, by extracting global features in the feature extraction module and combining it with a multi-layer denoising process, the model can quickly generate reconstructed multivariate time-series data in fewer iterations, significantly reducing computational burden and inference latency. Simultaneously, the adopted masking strategy further enhances the model's sensitivity and robustness to anomalies, ensuring high-precision anomaly detection results.

[0042] In some implementations, the step of training the anomaly detection model based on the original multivariate time series data and a preset mask generation strategy to obtain the anomaly detection model includes: Step S142: Based on the original multivariate time series data, perform preset preprocessing to obtain trend component data and seasonal component data.

[0043] In this embodiment, a series of preprocessing operations can be performed on the original multivariate time series data to improve data quality. These operations include data cleaning such as denoising, normalization, and standardization. Then, the cleaned original multivariate time series data is decomposed into trend component data and seasonal component data. Specifically, seasonal decomposition or trend extraction algorithms can be used to decompose the original multivariate time series data into trend component data and seasonal component data. The trend component data reflects the long-term changing trend in the original multivariate time series data, such as a long-term pattern of growth or decline. The seasonal component data reflects the periodic fluctuations in the original multivariate time series data.

[0044] In one specific implementation, the preprocessing process can employ a time series decomposition mechanism to structure the original multivariate time series data. Specifically, firstly, fixed-length subsequence samples are obtained through sliding window sampling, and the data within each time window is normalized using the z-score standardization method. Then, a trend-seasonal decomposition model is applied, where the trend component is calculated using a sliding window averaging algorithm, the window size is dynamically set according to the data sampling frequency, and the seasonal component is obtained by subtracting the trend component from the original multivariate time series data to obtain the residual.

[0045] Step S144: Input the trend component data and seasonal component data into the feature extraction module for feature extraction to obtain global features.

[0046] In this embodiment, the seasonal component is input into the time-series feature extraction unit. After being mapped to the deep feature space through the embedding layer, the significant frequency components are identified by the short-time Fourier transform. Based on the period length corresponding to each frequency, the one-dimensional sequence is reshaped into a two-dimensional tensor (the rows and columns represent the number of periods and the time points within the period, respectively). The local features within the period and the cross-period evolution law are captured by the hierarchical convolution operation. Finally, the time-series feature representation is generated by weighted aggregation according to the amplitude value.

[0047] In this embodiment, the temporal feature representation is input to the correlation feature extraction unit. After feature mapping is performed again, the dependency relationship between variables is modeled through a dynamic adjacency matrix. Multi-order graph convolution is applied to fuse the information of nearest and far-distance nodes. The data is then processed by a multilayer perceptron and aggregated according to frequency amplitude to generate the correlation feature representation.

[0048] In this embodiment, the correlation feature representation and the trend component are fused through a weighted concatenation mechanism. The trend component retains the long-term trend information of the original dimension. The contribution ratio of the two types of features is adjusted by a learnable gating coefficient, and the global feature representation with compressed dimension is output.

[0049] Step S146: For the global features, execute a preset mask generation strategy to obtain at least one mask data.

[0050] In this embodiment, the preset mask generation strategy may include multiple mask generation strategies, and one mask generation strategy corresponds to one type of mask data.

[0051] In this embodiment, the preset mask generation strategy can be a mask generation strategy based on anomalies, a mask generation strategy with random masking, a mask generation strategy with dynamic masking, and so on.

[0052] In one specific implementation, the feature space is first analyzed using the local outlier factor algorithm to locate potential anomalies and mark them as forced masking regions. Then, a raster masking mechanism is applied along the time dimension to generate at least three complementary mask instances—each instance masks the data window at fixed intervals, ensuring that at any given time point, it is unmasked in at least one instance. Finally, the forced masking regions and the raster masking regions are logically merged to form a combined masked dataset covering both anomaly-sensitive and pattern-preserving regions. This strategy effectively balances the needs of information preservation and anomaly simulation.

[0053] Step S148: Using the mask data of at least one of the above and the global features as input, input them into the anomaly detection backbone module. Under the control of a preset loss function, perform joint training on the feature extraction module and the anomaly detection backbone module to obtain an anomaly detection model.

[0054] In this embodiment, the joint training process can simultaneously optimize the trainable parameters of the feature extraction module and the anomaly detection backbone module through an end-to-end backpropagation mechanism. The specific training process can be as follows: After inputting the mask data and global features into the anomaly detection backbone module, the backbone module performs multi-layer diffusion denoising operations and outputs reconstructed multivariate time-series data. A preset composite loss function value is calculated, which includes three core constraint terms (diffusion loss term + temporal feature loss term + correlation feature loss term). Gradient backpropagation is performed based on the total loss value, simultaneously transmitting the error signal to the Transformer encoding layer of the anomaly detection backbone module and the convolutional / graph neural network layer of the feature extraction module. During parameter updates, an adaptive learning rate scheduler can be used, automatically terminating training when the validation set loss decrease rate is lower than a preset value for multiple consecutive training cycles, outputting an anomaly detection model that can be deployed to cloud edge devices.

[0055] In this embodiment, the anomaly detection model can be further trained by distillation, that is, the current anomaly detection model is used as the teacher model, and the student model is obtained by distillation training.

[0056] The beneficial effects of this implementation are as follows: Firstly, by preprocessing the original multivariate time series data and decomposing it into trend component data and seasonal component data, it helps to clearly identify long-term trends and periodic changes in the data, thereby avoiding the influence of noise or irrelevant components on the training of the anomaly detection model. This results in more accurate training data, which helps improve the detection accuracy of the anomaly detection model. Secondly, the feature extraction module, by processing the trend component and seasonal component data separately, can extract key temporal features from the data, and through the generation of global features, ensures the diversity and comprehensiveness of features. This method can capture the temporal patterns and dynamic changes of data at different levels, enhancing the model's adaptability to complex data. Thirdly, by executing a preset mask generation strategy, the model can dynamically filter and focus on key parts of the data during training, reducing the interference of noise and irrelevant information. This strategy not only helps the model focus on anomaly patterns in the data but also improves its robustness to different data distributions and change patterns, enhancing the model's generalization ability for anomaly detection tasks. Finally, by inputting both the mask data and global features into the anomaly detection backbone module and jointly training them under the control of a preset loss function, the model can self-adjust during training, optimizing the collaborative work between the feature extraction module and the anomaly detection backbone module, thereby significantly improving the overall model performance. This joint training method ensures the efficiency and accuracy of the anomaly detection model in complex environments.

[0057] like Figure 3 and Figure 5 As shown, in some embodiments, the feature extraction module includes a time-series feature extraction unit, which is used to process the seasonal component data, including: Step S1441: Project the seasonal component data into the deep feature space through the embedding layer.

[0058] In this embodiment, a high-order feature mapping is performed on the seasonal component data through an embedding layer. Specifically, a fully connected neural network layer can be used to linearly transform the input seasonal component data to a deep feature space of a preset dimension. This operation can enhance the representational ability of the original data, transforming low-dimensional time-series information into high-dimensional feature vectors containing rich patterns. The weight matrix of the embedding layer can be randomly initialized and optimized by the subsequent training process, and the output dimension can be adaptively adjusted according to the length of the input sequence.

[0059] Step S1442: Perform a short-time Fourier transform on the projected features to identify the target frequency and its corresponding period length.

[0060] In this embodiment, a Short-Time Fourier Transform (STFT) is performed on the projected depth features to identify the dominant frequency components and related periodic characteristics. A Hanning window function can be used to segment and window the time series data. The frequency domain distribution of each window is calculated using a Fast Fourier Transform, and frequencies whose energy significantly exceeds a preset threshold are selected as target frequencies, with their corresponding period lengths recorded. This process can reveal the multi-scale periodicity hidden in the time series data, providing a physical basis for subsequent processing.

[0061] Step S1443: For each identified target frequency and its period length, perform the following processing: reshape the one-dimensional sequence into a two-dimensional tensor according to the period length, where the row dimension represents the number of periods and the column dimension represents the time point within the period.

[0062] In this embodiment, for each target frequency and its period length, the one-dimensional feature sequence is reconstructed into a two-dimensional tensor matrix. The specific data processing procedure is as follows: the sequence is divided into multiple consecutive period segments along the time axis, with the number of period segments as the row dimension and the number of data points within a single period as the column dimension. Missing data is filled in using linear interpolation to ensure the integrity of the two-dimensional matrix structure. This step makes the periodic structure implicit in the time dimension explicit, forming a well-organized data carrier capable of two-dimensional convolution processing.

[0063] Step S1444: Input the two-dimensional tensor into a hierarchical convolutional feature extraction subunit that includes multiple convolutions, and capture short-term dependencies within a period and long-term change patterns across periods through hierarchical convolution operations.

[0064] In this embodiment, feature extraction is performed on the reshaped two-dimensional tensor input hierarchical convolutional structure. This structure can include multiple levels of convolution, each consisting of two sets of parallel convolutional layers: the first set uses small-scale convolutional kernels (e.g., 3×3) sliding along the column direction to capture local temporal patterns within a period; the second set uses long strip convolutional kernels (e.g., 1×5) sliding along the row direction to model long-term evolutionary trends across periods. The outputs of each convolutional layer are processed by a nonlinear activation function and then concatenated with the original input residuals to ultimately form a feature map that integrates local details and global evolutionary patterns.

[0065] Step S1445: Reshape the feature tensor output by the hierarchical convolution feature extraction subunit back into one-dimensional form.

[0066] In this implementation, the two-dimensional feature map output by the hierarchical convolution is restored to the original temporal structure. A reverse operation is performed along the row-column dimension: the rows of the matrix are concatenated end-to-end in chronological order to form a continuous one-dimensional feature sequence. The feature lengths are aligned using deconvolution to ensure that the length of the restored sequence is exactly the same as the original input. This process preserves the spatial correlation characteristics extracted by convolution while restoring the continuity of the temporal dimension.

[0067] Step S1446: Using an amplitude-based adaptive aggregation method, the one-dimensional representations corresponding to each significant frequency are weighted and fused to generate a time-series feature representation, wherein the weighting coefficients are dynamically determined by the amplitude values ​​of the corresponding significant frequencies.

[0068] In this embodiment, an amplitude-based adaptive aggregation method can be used to weighted fuse one-dimensional feature representations from different frequencies. The weighting coefficients can be dynamically determined by the amplitude values ​​of the corresponding frequencies. Amplitude represents the importance of each frequency component in the signal; therefore, frequency components with larger amplitudes are assigned higher weights. In this way, the model can automatically adjust the contribution of each frequency component to the temporal feature representation, thereby generating a more accurate and representative temporal feature representation. This weighted fusion method ensures that the model can dynamically adjust the weights according to the periodic features in the data, thereby improving the expressive power of temporal features and enhancing the accuracy and robustness in tasks such as anomaly detection.

[0069] This implementation accurately captures the periodic features in the seasonal component through deep feature space projection, short-time Fourier transform, and reshaping of periodic data. The hierarchical convolutional feature extraction subunit effectively captures short-term dependencies within the period and long-term variation patterns across periods, thereby enhancing the expressive power of time-series data. Finally, the amplitude-based adaptive aggregation method dynamically adjusts the weighting coefficients according to the amplitude of each significant frequency, ensuring accurate fusion and expression of time-series features and improving the model's ability to identify complex time-series patterns and detect anomalies.

[0070] like Figure 3 and Figure 6 As shown, in some embodiments, the feature extraction module further includes a correlation feature extraction unit, which is used to process the temporal feature representation, including: Step S1447: Project the temporal feature representation onto the deep feature space through the embedding layer.

[0071] In this embodiment, the temporal feature representation output by the temporal feature extraction unit is input into a new embedding layer to perform a higher-order feature space transformation. The dimensions of the input features can be mapped to a preset depth space through a fully connected neural network layer. This process is used to eliminate the dimensional differences between different variables and enhance the separability of implicit association patterns.

[0072] Step S1448: Use short-time Fourier transform to analyze the projected features and find the target frequency and its corresponding period length.

[0073] In this embodiment, a second-order spectral analysis is performed on the features after depth projection. A windowed short-time Fourier transform (with adaptively adjustable window length) is used to analyze the frequency domain characteristics of the feature sequence to identify target frequency components whose energy significantly exceeds the dynamic threshold, and to accurately record their frequency values ​​and corresponding period lengths.

[0074] Step S1449: For each target frequency and its corresponding period length, the following processing is performed: First, the one-dimensional sequence is converted into a two-dimensional tensor through a reshaping operation. The dimension of this two-dimensional tensor is determined by the period length and the frequency dimension. Then, graph convolutional feature extraction is performed on the two-dimensional tensor to obtain node features. Finally, multilayer perceptron processing is applied to the node features. Specifically, performing graph convolutional feature extraction on the two-dimensional tensor to obtain node features may further include: generating node feature vectors through linear projection, calculating an adaptive adjacency matrix based on the node feature vectors, modeling dynamic dependencies between variables, applying a MixHop graph convolutional network to process the two-dimensional tensor, and updating node features through multi-order neighbor feature aggregation.

[0075] In this embodiment, for each target frequency period, the one-dimensional feature sequence is reshaped into a two-dimensional tensor. The period length is used as the row dimension (representing the number of period segments), and the original variable dimension is used as the column dimension (representing the number of variables in the system), forming a two-dimensional matrix structure where rows correspond to time succession and columns correspond to the variable space.

[0076] In this implementation, node feature vectors can be generated through a linear projection layer, followed by the construction of an adaptive adjacency matrix. For example, two sets of trainable parameter matrices can be used to calculate the send-receive relationship between nodes, and then dynamic adjacency weights can be generated through matrix multiplication and exponential normalization operations.

[0077] In this implementation, the MixHop graph convolution mechanism can be applied. For example, a MixHop graph convolutional network can be used to process the data on a two-dimensional tensor, leveraging the network's ability to aggregate multi-level neighbor features to further update and optimize node features. The MixHop method can handle multi-level neighborhood information, effectively improving the expressive power of feature extraction.

[0078] In this embodiment, the node features output by graph convolution can be input into a multilayer perceptron (MLP) containing multiple fully connected layers. The GeLU activation function can be used to enhance the nonlinear expressive power, and finally, a refined node feature representation is output.

[0079] Step S14410: Reshape the processed node features back into a one-dimensional form.

[0080] In this implementation, after graph convolutional feature extraction and multilayer perceptron processing, the model reshapes the obtained node features back into a one-dimensional form. The purpose of this process is to compress the multidimensional features extracted from the graph convolutional network and perceptron back into a one-dimensional representation, enabling subsequent processing and fusion operations to be performed more efficiently. Through this reshaping operation, the model transforms complex high-dimensional features into a low-dimensional vector that is easier to process, facilitating further feature fusion and anomaly detection tasks.

[0081] In this implementation, the matrix can be expanded sequentially along the row dimension (time axis) while preserving the column dimension (variable dimension) structure, ensuring that the length of the restored feature sequence is completely consistent with the original input. The row and column dimensions are transformed through a transpose operation, allowing the feature representation to be re-adapted to subsequent processing flows.

[0082] Step S14411: Using an amplitude-based adaptive aggregation method, the one-dimensional representations corresponding to each frequency are weighted and merged to generate a correlation feature representation, where the weights are determined by the amplitude values ​​of the corresponding significant frequencies.

[0083] In this embodiment, an amplitude-based adaptive aggregation method can be used to weight and fuse one-dimensional feature representations from different frequencies. The weighting coefficients can be dynamically determined based on the amplitude values ​​of the corresponding frequencies. The amplitude value reflects the significance of each frequency component in the signal; therefore, frequency components with larger amplitude values ​​will be assigned higher weights, allowing the model to pay more attention to the contribution of key frequencies to the data features. Through this weighted merging, a correlation feature representation is generated, which can effectively characterize the interdependencies between multivariate variables in time-series data, and is ultimately used for anomaly detection and other analytical tasks.

[0084] This implementation effectively uncovers dynamic relationships among multivariate variables by combining deep feature space projection of temporal features, short-time Fourier transform, and graph convolution feature extraction. First, an embedding layer projects the temporal feature representation onto a deep feature space, enhancing the data's expressive power. Then, the short-time Fourier transform helps identify target frequencies and their periodic features, providing frequency information for subsequent processing. Through reshaping operations and graph convolution feature extraction, the model effectively captures local features within a period and cross-period correlations, thereby uncovering complex interaction patterns in the time-series data. Finally, an amplitude-based adaptive aggregation method is used to weightedly fuse features from different frequencies, ensuring that the dynamic importance of each frequency component is fully reflected, generating accurate relevance feature representations. This process significantly improves the model's ability to identify complex dependencies among multiple variables, thereby enhancing the accuracy and robustness of tasks such as anomaly detection.

[0085] In some implementations, the feature extraction module further includes a feature fusion unit, which is used to fuse the correlation feature representation with the trend component data to obtain the global feature.

[0086] This implementation effectively integrates different dimensions of information in time series data by fusing correlation feature representations with trend component data. Correlation feature representations capture the dynamic dependencies between multivariate variables, while trend component data reveals long-term trends in the data. Through a feature fusion unit, these two different types of information are combined to generate a global feature that simultaneously preserves the long-term evolution trend and complex intervariate correlations of the data. This fusion method enables the model to more comprehensively understand and model the multidimensional features of time series data, enhances the ability to identify anomalous patterns, and improves the model's performance and robustness in practical applications.

[0087] In some implementations, the step of executing a preset mask generation strategy for the global features to obtain at least one type of mask data includes: Step S1462: Analyze the global features based on the local outlier factor algorithm, identify outliers and mark them as forced mask regions.

[0088] In this embodiment, the Local Outlier Factor (LOF) algorithm can be used to perform density anomaly analysis on global features. The specific data processing procedure can be as follows: For the feature vector at each time point, calculate its k-nearest neighbor distance (the neighborhood size k is dynamically adjusted according to the time series length); by comparing the local reachability density with the neighborhood average density, obtain the LOF value representing the degree of outlier; set a dynamic judgment threshold θ, and mark time points with LOF values ​​exceeding θ as potential anomalies; expand a preset window range (the window size is inversely proportional to the data sampling frequency) outwards from these time points to both sides of the time axis, forming continuous suspicious anomaly regions; finally, set this region as a forced masking region (all data points within this region are marked as needing to be masked).

[0089] Step S1464: Apply a preset raster masking strategy along the time dimension to generate at least three complementary mask instances by masking data at fixed intervals; wherein the masking windows of each mask instance are staggered along the time axis, and the window that is masked in any raster masking strategy remains unmasked in other raster masking strategies.

[0090] In this embodiment, a preset raster masking strategy can be applied along the time dimension to mask data at fixed intervals, generating at least three complementary mask instances. Specifically, by designing three different masking strategies, three mask instances can be constructed, ensuring that in any strategy, the masked window, excluding outliers, is an unmasked window under the other two strategies. The construction of the mask window can be based on the idea of ​​raster masking, that is, masking data at certain intervals along the time dimension of the time series. In this way, masked windows and unmasked windows are alternately distributed, forming multiple mask instances. For example, there can be three different raster masking strategies, each masking data in different time periods, forming complementary mask instances. This mask instance ensures the diversity of data processing under different strategies by masking and retaining data in different time windows.

[0091] In a specific implementation plan, a triple grating masking strategy can be implemented along the time dimension: First mask instance generation: Divide the time axis into equal-length windows with a fixed interval P (P=0.3×time series length) and perform full window masking on windows with odd numbers (windows 1, 3, 5...).

[0092] Second mask instance generation: Under the same segmented window, mask the windows whose numbers are multiples of 3 plus 1 (windows 1, 4, 7...).

[0093] Third mask instance generation: Mask the remaining uncovered windows (i.e. windows that are not odd in number and not numbered 3k+1). This mechanism allows the masking windows of the three mask instances to be completely staggered on the timeline, and ensures that at any given time point, the masking windows are not masked in at least two mask instances.

[0094] Step S1466: Combine the forced mask region with the shielding region of the raster mask instance to generate the final mask data.

[0095] In this embodiment, firstly, the shielding regions of the triple grating mask instance are logically ORed to form a basic grating mask region. Then, the basic grating mask region and the forced mask region obtained in step S1462 are logically unioned. Next, a morphological closing operation is performed on the union region to eliminate minute gaps and generate a continuous shielding region.

[0096] This implementation effectively improves the robustness of the anomaly detection model to missing or anomalous data by combining the Local Outlier Factor (LOF) algorithm and a raster masking strategy. First, the LEF algorithm helps identify outliers in global features and marks them as mandatory masking regions, ensuring the model can specifically handle anomalous data. Second, the raster masking strategy generates multiple complementary mask instances, distributing the masking windows alternately across the time dimension, thus avoiding repeated masking of the same data region and enhancing data diversity. Finally, the mandatory masking regions are combined with the masking regions of the raster mask instances to generate the final masked data, further improving the model's adaptability to different types of missing data. This strategy not only improves the accuracy of anomaly detection but also enhances the model's ability to handle complex time-series data, making it more robust and accurate in practical applications.

[0097] like Figure 4 As shown, in some embodiments, the anomaly detection backbone module is a diffusion model based on a stacked Transformer encoder; the preset loss function includes a diffusion loss term, a temporal feature loss term, and a correlation feature loss term; wherein, the diffusion loss term is used to calculate the L2 norm distance between the reconstructed multivariate time series data output by the anomaly detection backbone module and the unmasked region in the global features; the temporal feature loss term is used to constrain the reconstruction error between the temporal feature representation output by the feature extraction module and the seasonal component data; the correlation feature loss term is used to constrain the similarity between the adaptive adjacency matrix generated by the feature extraction module and the correlation matrix of the true variables.

[0098] In this embodiment, the diffusion model based on stacked Transformer encoders can employ a deep residual network architecture. The core of the network consists of four identical cascaded residual layers, which are ultimately fused and reconstructed through a multilayer perceptron (MLP). This structure preserves shallow feature information through residual connections while gradually extracting deep semantic features through hierarchical abstraction. Residual connections enable information to propagate across layers, effectively avoiding the gradient vanishing problem that may occur during training and ensuring the effectiveness of the deep network.

[0099] In a specific implementation scheme, the internal structure of the residual layer can be as follows: The conditional information injection path processes two control signals and fuses them with global features. These control signals can include a diffusion step encoding signal and a mask position signal. The diffusion step encoding signal maps the diffusion step sequence number (representing the noise level) to a continuous vector through an embedding layer, reflecting the current noise level. This signal provides the model with environmental awareness information about the noise, helping it better adapt to different noise levels during reconstruction. The mask position signal encodes the features of the mask position, helping the model focus on missing regions in the time series. The mask position signal is also mapped to a feature vector through an embedding layer, providing environmental state information for subsequent feature transformations. These two signals are fused through a fully connected layer and a Swish activation function to generate a dynamic conditional vector. This conditional vector provides environmental awareness for subsequent feature transformations, thereby enhancing the model's adaptability to different noise distributions.

[0100] The global feature processing path involves the input global features undergoing the following processing stages: Feature initialization: First, the input global features are aligned and initialized in terms of dimensions using a 1×1 convolutional layer. This step ensures that the dimensions of the input features match the inputs of subsequent layers in the network.

[0101] Nonlinear activation: Next, the features are activated by the ReLU function, which gives the features nonlinear expressive power.

[0102] Spatiotemporal Feature Extraction: After initialization and nonlinear activation, the features are fed into a four-level stacked Transformer encoding layer. Each layer's encoder uses a multi-head self-attention mechanism to capture long-range dependencies between variables. The self-attention mechanism enables features at each time step to interact with features at other time steps, thereby capturing global dependencies.

[0103] After each encoder layer, the feedforward neural network further performs dimensional scaling and nonlinear transformations on the features.

[0104] Dual-head output mechanism: Residual Header: This part processes the Transformer encoder output through 1×1 convolutions, generates residual feature vectors, and passes them to the next residual layer.

[0105] Output Header: This section first applies the ReLU activation function to the output of the Transformer encoder, and then generates the contribution of the current layer to the data reconstruction through 1×1 convolution. The features formed by the output heads of each layer provide different levels of contribution to the final reconstructed sequence, ultimately constituting the refined features of the reconstruction.

[0106] In one specific implementation, the reconstructed features from the output heads of the four layers can be fused together using a weighted summation. The weights of each layer are dynamically optimized based on the training process to better fuse features from different layers. The fused features can then be input into a multilayer perceptron (MLP) and undergo a nonlinear transformation to generate the final reconstructed sequence. The depth of the MLP ensures that the model can handle complex patterns and generate high-quality reconstruction results, further improving anomaly detection capabilities.

[0107] In this embodiment, the diffusion loss term is defined as the L2 norm distance between the reconstructed data and the original global features in the unmasked region. The specific data processing procedure is as follows: First, the reconstructed multivariate time series data output by the anomaly detection backbone module is extracted and compared point by point with the original global features of the input model. Then, based on the mask data identifier, only the unmasked region (i.e., data points not covered by forced masking or raster mask) is selected to calculate the error. Finally, the squares of the reconstruction errors of all unmasked points are summed and averaged.

[0108] In this embodiment, the temporal feature loss term can be used to constrain the reconstruction error between the temporal feature representation output by the feature extraction module and the seasonal component data. The seasonal component data contains periodic variations in the time series, while the temporal feature representation is the key information extracted from the input data by the feature extraction module. By comparing the reconstruction errors of the two, the temporal feature loss term ensures that the model can not only recover the overall trend of the data during training but also accurately capture periodic fluctuations, further improving the model's ability to model temporal variations.

[0109] In this implementation, the purpose of the relevant feature loss term is to constrain the similarity between the adaptive adjacency matrix generated by the feature extraction module and the true variable correlation matrix. The adaptive adjacency matrix, generated by the feature extraction module, describes the interdependencies between variables in a multivariate time series. The true variable correlation matrix is ​​calculated based on the actual statistical correlation of the data. By comparing the similarity between the adaptive adjacency matrix and the true correlation matrix, the relevant feature loss term ensures that the model can accurately capture the dynamic relationships between variables, improving the model's ability to identify potential dependencies in the data, thereby enhancing the accuracy of anomaly detection.

[0110] This implementation significantly improves the accuracy and robustness of the anomaly detection model by combining a diffusion model based on a stacked Transformer encoder with a precisely designed loss function. The diffusion loss term ensures that the model can accurately reconstruct multivariate time series data and measures the reconstruction error using L2 norm distance, avoiding interference from anomalous data on the model output. The temporal feature loss term effectively captures the periodic and trend changes in the data by constraining the reconstruction error between the temporal feature representation and the seasonal component data, further enhancing the model's ability to learn temporal patterns. The correlation feature loss term ensures that the model can accurately model the dynamic dependencies between variables by constraining the similarity between the generated adaptive adjacency matrix and the correlation matrix of the true variables. In summary, the design of these loss terms not only enhances the model's ability to identify anomalous data but also improves its ability to model complex time series data and multivariate relationships, ultimately achieving more efficient and accurate anomaly detection.

[0111] In some embodiments, the method further includes: Step S16: Use the anomaly detection model as the teacher model.

[0112] Step S18: Construct a student model; wherein the model structure of the student model is the same as that of the anomaly detection model, and the number of sampling steps and parameters of the student model is less than that of the teacher model.

[0113] Step S110: Iteratively execute distillation training; wherein, during the distillation training process, the sampling steps of the student model are gradually reduced, and the output distribution of the teacher model is used as a supervision signal to optimize the student model.

[0114] Step S112: After training, use the student model as the lightweight anomaly detection model.

[0115] This implementation effectively transforms a complex teacher model into a lightweight student model through distillation training, significantly reducing computational overhead while maintaining anomaly detection performance. The teacher model, a high-performance anomaly detection model, boasts a high number of sampling steps and parameters, enabling it to extract complex features from the data. The student model, through distillation training, gradually reduces the number of sampling steps and parameters, progressively learning the output distribution of the teacher model. The student model achieves similar detection accuracy to the teacher model with lower computational resources, making it suitable for resource-constrained environments. Through distillation training, the student model not only maintains anomaly detection accuracy but also achieves model lightweighting, enhancing its practical application value, particularly in edge devices or embedded systems, where it provides higher computational efficiency and inference speed.

[0116] like Figure 7As shown, this embodiment provides an anomaly detection method, and the subject executing the method can be a resource-constrained edge computing device.

[0117] The method may include: Receive multivariate time series data to be detected; The multivariate time series data is input into the anomaly detection model provided in the above embodiments, or into the lightweight anomaly detection model provided in the above embodiments, to obtain the reconstructed multivariate time series data; Error data is calculated based on the multivariate time series data to be detected and the reconstructed multivariate time series data.

[0118] In this embodiment, the error data can be point-level error (absolute error).

[0119] Based on the error data, outliers in the multivariate time series data to be detected are identified.

[0120] In this embodiment, an error threshold can be set, and the error data can be compared with the threshold. If the error exceeds the preset threshold, the point is considered an outlier.

[0121] The anomaly detection method provided in this embodiment can efficiently identify outliers in data by combining the reconstruction and error calculation of multivariate time series data. The method first inputs the multivariate time series data to be detected into a trained anomaly detection model or a lightweight model to obtain reconstructed multivariate time series data. Then, the error between the data to be detected and the reconstructed data is calculated; data points with larger errors are identified as anomalies. This method can improve computational efficiency while maintaining high detection accuracy, making it suitable for application in resource-constrained environments, and particularly applicable to real-time monitoring and processing of large-scale data streams. Through this process, abnormal data can be identified quickly and accurately, thereby providing support for subsequent anomaly handling and decision-making.

[0122] In one specific implementation scheme, a lightweight time series anomaly detection method for cloud computing environments is provided. This implementation scheme relates to the field of information technology and mainly proposes a lightweight time series anomaly detection method based on feature extraction and a diffusion model for time series anomaly detection tasks in cloud environments. By improving the diffusion model, the impact of node performance imbalance and network volatility in cloud environments is reduced.

[0123] The cloud environment primarily refers to a cross-domain, interconnected computing environment formed through open collaboration and resource sharing among different cloud service providers under the background of cloud computing technology. With the rapid development of cloud computing technology, the cost-effectiveness of sensor, network, storage, and processing technologies is constantly improving, making time series anomaly detection increasingly important in scientific research, industrial production, environmental monitoring, and many other fields. However, current time series anomaly detection in the cloud environment still faces the following challenges: Difficulty in multi-timescale feature extraction: The interactions between time-series data entities shape the complex temporality and correlation of time series, leading to different manifestations and changes in their periodicity and correlation across multiple time scales, increasing the difficulty of feature extraction. Diversity and unknown nature of anomaly types: Time series anomalies exhibit diversity in type, form, length, and density, and often involve unknown anomaly types, placing higher demands on the generalization performance and robustness of anomaly detection algorithms. Resource constraints and real-time requirements: In the cloud computing environment, anomaly detection systems need to operate under strict resource constraints and meet high real-time requirements to achieve timely detection and rapid response.

[0124] In existing technologies, traditional anomaly detection methods rely on manually defined features or statistical indicators, making it difficult to capture complex patterns and hidden relationships. In recent years, deep learning methods have significantly reduced the reliance on manual feature engineering, with diffusion models demonstrating superior performance in time-series anomaly detection tasks due to their powerful generative capabilities. However, diffusion models typically require multiple iterations to generate high-quality samples, resulting in high computational complexity and making them unsuitable for resource-constrained scenarios and applications with high real-time requirements.

[0125] Therefore, the core task of time series anomaly detection research is to reduce the computational complexity of diffusion models and improve inference speed while ensuring high detection performance.

[0126] This implementation plan aims to propose a lightweight time series anomaly detection method based on a diffusion model for cloud computing environments, improving the performance and efficiency of time series anomaly detection tasks and adapting to resource-constrained scenarios and real-time requirements. Specifically, it includes the following three points: To address the challenge of capturing periodicity and correlation across multiple time scales in real-world multivariate time-series data and improve the accuracy of anomaly detection, this implementation proposes a multivariate feature extraction method based on a dual-module architecture. By mapping time series data to a two-dimensional space and combining it with a convolutional network, intra-periodic and inter-periodic variations across multiple time scales are discovered. Subsequently, a graph network is used in the two-dimensional space to learn the interdependencies between variables, revealing correlations across multiple scales.

[0127] To reduce the computational complexity of the diffusion model and thus improve inference speed, this implementation scheme designs a time series anomaly detection method based on the diffusion model. It constructs a partial mask sequence by using a combined masking strategy based on local outlier factor technology, introduces stacked Transformer coding layers as the backbone network, and uses the multi-level denoised output generated during inference as the anomaly detection signal.

[0128] To meet the resource constraints and real-time requirements of the cloud environment, the limited resources of edge devices are utilized in a fine-grained manner. A lightweight model structure is set up by optimizing the structure, and the inference speed is greatly improved by progressive knowledge distillation technology.

[0129] To address the shortcomings of existing technologies and improve the real-time performance and accuracy of time series anomaly detection, this implementation plan proposes a lightweight time series anomaly detection method based on a diffusion model for cloud computing environments. The specific steps of the implementation plan are as follows: Step 1: Extract temporal and correlation features from the time series data.

[0130] For multidimensional time series, data cleaning, standardization, and sliding window sampling are first performed, followed by decomposition into trend and seasonal components. For the seasonal component, temporal features are extracted using a residual-stacked Timestrans Block. Within the Timestrans Block, Fourier transform captures local periodic features, mapping the one-dimensional time series data to a two-dimensional space. A two-dimensional convolutional network is used to model short-term and long-term dependencies, ultimately generating a temporal feature representation. Next, multivariate correlation features are extracted from the temporal features using a residual-stacked Corretrans Block. Within the Corretrans Block, linear projection and adaptive adjacency matrices are used to dynamically model variable dependencies, and a graph convolutional network captures multi-order neighbor features. These features are then weighted and aggregated using significant frequency amplitude values ​​discovered by Fourier transform to generate a correlation feature representation, which is fused with the trend component to obtain a global feature representation, which is then saved as input for subsequent modules.

[0131] Step 2: Jointly train the feature extraction module and the Diffusion anomaly detection module Using a diffusion model as the core component, combined with global feature representation and a masking strategy, a model for anomaly detection is trained, including a pre-feature extraction module. First, the global feature representation is preprocessed, selecting parameters such as context window size and stride. Then, a raster masking strategy and outlier factor algorithm are designed to generate a mask sequence simulating unobserved data points. Specifically, this implementation first detects outliers in the time series based on the Local Outlier Factor (LOF) algorithm, combined with a set threshold. Determine the location of the observation point. For points exceeding a set threshold... The points are marked as the data to be observed, that is, in the mask sequence. The corresponding value at the specified position is set to 0. Subsequently, to further enhance the diversity and information preservation capability of the masked sequence, this implementation introduces a raster masking strategy. This strategy masks data at fixed intervals along the time dimension, dividing the original time series into several windows, with masked and unmasked windows interleaved. Three different masking strategies are designed. This implementation scheme constructs three mask instances. Furthermore, it is ensured that, under any given strategy, the masked window (excluding outliers) is an unmasked window under the other two strategies. Subsequently, a diffusion model is used to model the multivariate time series (MTS), learning to impute missing values ​​caused by data masking through training. During training, appropriate hyperparameters such as learning rate, number of iterations, and batch size are selected, and the prediction error set for each denoising step in the training nodes is saved. After training the diffusion model, a multi-level inference stage is entered, combining a dynamic threshold mechanism and a voting strategy to output the final anomaly detection result based on the imputation error.

[0132] Step 3: Use progressive distillation to obtain a set of lightweight anomaly detection models. To improve model inference efficiency and reduce computational costs, this implementation scheme employs an iterative training method based on knowledge distillation. Specifically, a progressive distillation strategy is used to gradually transform the diffusion model (teacher model) trained in step four into a student model with fewer sampling steps. During training, the sampling steps of the student model are gradually reduced, starting with the complete sampling steps of the teacher model. At each stage, the output distribution of the teacher model is used as a supervisory signal to optimize the student model. Through multiple iterations, the student model gradually learns the core feature representation capabilities of the teacher model, ultimately generating a set of lightweight models. These models significantly reduce inference time while maintaining high detection performance.

[0133] This implementation plan proposes a lightweight time series anomaly detection method for cloud environments, whose advantages are mainly reflected in the following aspects: This implementation scheme, through the design of a dual-model architecture feature extraction method, enables it to accurately capture multi-level dynamic patterns in time series, significantly improving the comprehensiveness and accuracy of feature extraction.

[0134] This implementation scheme utilizes an unconditional diffusion model to generate high-quality interpolation results, learning normal patterns without relying on labeled data. It also designs a multi-level inference mechanism combined with a dynamic error threshold adjustment strategy, enabling flexible adaptation to different types of anomalies and effective handling of unknown anomalies, thus effectively addressing the diversity and uncertainty of anomaly types.

[0135] This implementation plan proposes an iterative training method based on knowledge distillation, which gradually distills the complex teacher diffusion model into a lightweight student model, thereby reducing computational complexity and improving inference speed.

[0136] In this implementation scheme, the masking strategy may be: Use the Local Outlier Factor (LOF) algorithm to analyze time series data and identify potential outliers. Constructing masked windows: This involves applying the concept of raster masks to the time series data, masking the data at regular intervals along the time dimension. Masked and unmasked windows are interleaved, forming multiple mask instances (e.g., three different masking strategies). In this way, the original time series is divided into multiple windows, with some windows masked and others remaining unmasked.

[0137] The above-mentioned anomalies and time series data points in the mask window are masked.

[0138] like Figure 8 As shown in this implementation scheme, the training process can be: The feature representation data obtained from feature extraction is divided into two parts: unmasked regions and masked regions. Noise is added to the masked regions, and the unmasked regions are replaced with noise and then merged into blocks to form a complete training input sequence. The idea of ​​the unconditional diffusion model is used to denoise and reconstruct the masked regions in the time series. The denoising backbone network is the TimesDiff backbone network. The training loss function consists of three parts: diffusion loss (…). : Measures the model's effectiveness in denoising the masked region, aiming to minimize the difference between predicted noise and actual noise. Temporal loss ( ): Ensures the model can effectively capture the temporal characteristics of the time series. Correlation loss ( This ensures the model can detect spatial correlations in multidimensional time series. By jointly optimizing the parameters of all modules, it avoids getting trapped in local optima and improves overall performance.

[0139] In this implementation scheme, the reasoning process can be: In each denoising step of the diffusion model, TimesDiff uses the denoising model to generate a prediction result, compares it with the true time series, and calculates the prediction error. Each denoising step generates an intermediate output, which has the same dimension as the original time series, and also corresponds to a prediction error.

[0140] The prediction error of the final denoising step is used as a benchmark to measure the quality of imputation at each step. The outlier prediction results from all denoising steps are summarized, and the number of times each time step is marked as an outlier in different denoising steps (votes) is counted. If the number of votes at a time step exceeds a set threshold, it is marked as a final outlier.

[0141] According to an embodiment of the present invention, an electronic device is provided. The electronic device in this embodiment may include one or more of the following components: a processor, a network interface, memory, non-volatile memory, and one or more application programs, wherein the one or more application programs may be stored in the non-volatile memory and configured to be executed by one or more processors, and the one or more programs are configured to perform the methods described in the foregoing method embodiments.

[0142] According to embodiments of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a computer, causes the computer to perform the method described in any of the above embodiments.

[0143] According to embodiments of the present invention, a computer program product comprising instructions is also provided, which, when executed by a computer, cause the computer to perform a method in any of the above embodiments.

[0144] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0145] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments, and will not be repeated here.

[0146] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0147] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0148] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A training method for an anomaly detection model, characterized in that, include: Obtain raw multivariate time series data; The anomaly detection model is trained based on the original multivariate time series data and a preset mask generation strategy to obtain the anomaly detection model; wherein, the time series anomaly detection model to be trained includes a feature extraction module and an anomaly detection backbone module. The feature extraction module is used to extract corresponding global features based on the original multivariate time series data, and the preset mask generation strategy is used to generate mask data based on the global features; the anomaly detection backbone module performs multi-layer denoising based on the global features and the mask data, and outputs the reconstructed multivariate time series data.

2. The method according to claim 1, characterized in that, The step of training the anomaly detection model based on the original multivariate time series data and a preset mask generation strategy to obtain the anomaly detection model includes: Based on the original multivariate time series data, a preset preprocessing is performed to obtain trend component data and seasonal component data; The trend component data and seasonal component data are respectively input into the feature extraction module for feature extraction to obtain global features; For the global features, a preset mask generation strategy is executed to obtain at least one type of mask data; Using the mask data of at least one of the above and the global features as input, the data is fed into the anomaly detection backbone module. Under the control of a preset loss function, joint training is performed on the feature extraction module and the anomaly detection backbone module to obtain an anomaly detection model.

3. The method according to claim 2, characterized in that, The feature extraction module includes a time-series feature extraction unit, which is used to process the seasonal component data, including: The seasonal component data is projected into a deep feature space through an embedding layer; Perform a short-time Fourier transform on the projected features to identify the target frequency and its corresponding period length; For each identified target frequency and its period length, the following processing is performed: the one-dimensional sequence is reshaped into a two-dimensional tensor according to the period length, where the row dimension represents the number of periods and the column dimension represents the time point within the period; The two-dimensional tensor is input into a hierarchical convolutional feature extraction subunit that includes multiple convolutions, and the short-term dependencies within the period and the long-term change patterns across periods are captured through hierarchical convolution operations. Reshape the feature tensor output by the hierarchical convolutional feature extraction subunit back into a one-dimensional form; An amplitude-based adaptive aggregation method is adopted to weight and fuse the one-dimensional representations corresponding to each significant frequency to generate a time-series feature representation, wherein the weighting coefficients are dynamically determined by the amplitude values ​​of the corresponding significant frequencies.

4. The method according to claim 3, characterized in that, The feature extraction module further includes a correlation feature extraction unit, which is used to process the temporal feature representation, including: The temporal feature representation is projected onto the deep feature space through an embedding layer; The projected features were analyzed using short-time Fourier transform to identify the target frequency and its corresponding period length. For each target frequency and its corresponding period length, the following processing is performed: First, the one-dimensional sequence is converted into a two-dimensional tensor through a reshaping operation. The dimension of the two-dimensional tensor is determined by the period length and the frequency dimension. Then, graph convolution feature extraction is performed on the two-dimensional tensor to obtain node features. Finally, multilayer perceptron processing is performed on the node features. Reshape the processed node features back into a one-dimensional form; An amplitude-based adaptive aggregation method is used to weight and merge the one-dimensional representations corresponding to each frequency to generate a correlation feature representation, where the weights are determined by the amplitude values ​​of the corresponding significant frequencies.

5. The method according to claim 4, characterized in that, The feature extraction module further includes a feature fusion unit, which is used to fuse the correlation feature representation with the trend component data to obtain the global feature.

6. The method according to claim 2, characterized in that, The step of executing a preset mask generation strategy for the global features to obtain at least one type of mask data includes: Based on the local outlier factor algorithm, the global features are analyzed to identify outliers and mark them as forced mask regions. A preset raster masking strategy is applied along the time dimension to generate at least three complementary mask instances by masking data at fixed intervals; wherein the masking windows of each mask instance are staggered along the time axis, and the window that is masked in any raster masking strategy remains unmasked in other raster masking strategies. The forced mask region is combined with the shielding region of the raster mask instance to generate the final mask data.

7. The method according to claim 2, characterized in that, The anomaly detection backbone module is a diffusion model based on a stacked Transformer encoder; the preset loss function includes a diffusion loss term, a temporal feature loss term, and a correlation feature loss term; wherein, the diffusion loss term is used to calculate the L2 norm distance between the reconstructed multivariate time series data output by the anomaly detection backbone module and the unmasked region in the global features; the temporal feature loss term is used to constrain the reconstruction error between the temporal feature representation output by the feature extraction module and the seasonal component data; the correlation feature loss term is used to constrain the similarity between the adaptive adjacency matrix generated by the feature extraction module and the correlation matrix of the true variables.

8. The method according to claim 1, characterized in that, The method further includes: The anomaly detection model is used as the teacher model; Construct a student model; wherein the model structure of the student model is the same as that of the anomaly detection model, and the number of sampling steps and parameters of the student model is less than that of the teacher model; Iterative distillation training is performed; during the distillation training process, the number of sampling steps of the student model is gradually reduced, and the output distribution of the teacher model is used as a supervision signal to optimize the student model. After training, the student model was used as a lightweight anomaly detection model.

9. An anomaly detection method, characterized in that, include: Receive the multivariate time series data to be detected; The multivariate time series data is input into any one of the anomaly detection models described in claims 1-7, or into the lightweight anomaly detection model described in claim 8, to obtain the reconstructed multivariate time series data. Based on the multivariate time series data to be detected and the reconstructed multivariate time series data, error data is calculated; Based on the error data, outliers in the multivariate time series data to be detected are identified.

10. An electronic device, characterized in that, include: A memory, and one or more processors communicatively connected to the memory; The memory stores instructions that can be executed by the one or more processors to cause the one or more processors to implement the method as described in any one of claims 1 to 8.

Images