Unsupervised time series anomaly detection method and system based on frequency domain enhancement

Abstract

The application discloses a kind of based on frequency domain enhancement's unsupervised time series anomaly detection method and system, the application includes the multiple data of equipment and is constituted multidimensional time series according to the dimension standardization processing of data;After multidimensional time series is converted to frequency domain, low-frequency component is extracted, low-frequency component is converted back to time domain and is rectified to obtain the multidimensional time series after frequency domain rectification;Depth neural network for extracting features is constructed and depth neural network is trained using the multidimensional time series after frequency domain rectification;Unlabeled multidimensional equipment data is detected for anomaly using trained depth neural network.The application aims at the internal dependence relationship of time series to be modeled to fully learn and represent the "normality" of time series to realize accurate anomaly detection, alleviate the deviation of detection result caused by noise accumulation in the modeling process of dependence relationship of prior art.

Description

Technical Field

[0001] The present invention relates to a time series anomaly detection technology in the field of equipment fault detection, and in particular to an unsupervised time series anomaly detection method and system based on frequency domain enhancement. Background Art

[0002] Time series anomaly detection is ubiquitous in many real-world applications, identifying potential failures, risks, and incidents in a wide range of systems, including industrial infrastructure, spacecraft, cloud servers, and even various human health indicators. Anomalies in these systems arise from a variety of factors, including environmental factors (such as temperature and humidity fluctuations), equipment aging, operational errors, system load variations, and external interference (such as power fluctuations or cyberattacks). These factors can all cause anomalies or noise in time series data. The ability to detect anomalies in time series data is fundamental to ensuring the reliability, efficiency, and security of these complex systems.

[0003] Fundamentally, time series anomaly detection depends on accurately learning the "normality" of data. However, this task remains challenging, primarily due to the inherent complexity of time series data. This complexity is characterized by the interplay of influences across dimensions and time, constituting an intricate web of dependencies within time series data. These relationships can be categorized into three distinct but interconnected types of dependencies. (i) Temporal dependencies arise from the sequential nature of data collection and reflect how past values ​​influence future observations. (ii) Dimensional dependencies reflect the mutual influence and constraints between different variables or features in a time series. (iii) Asynchronous inter-dimensional dependencies describe the delayed response or lag effect that one variable in a system may have on other variables. Understanding and modeling these multifaceted dependencies is crucial, as they collectively define the underlying structure and behavior of time series data.

[0004] Current time series anomaly detection methods utilize various neural network architectures to model time series features in a feedforward process and represent them in an embedding space. Early studies utilized recurrent neural networks to model contextual sequence information. Subsequent studies employed convolutional neural networks, such as Conv1d and temporal convolutional networks (TCNs), to more effectively capture local temporal dependencies. Recent studies have adopted the Transformer architecture, enabling the modeling of long-term dependencies. Others have constructed graph structures (each node represents a variable) and explicitly modeled inter-variable dependencies through the information passing mechanism of graph neural networks. These methods have demonstrated promising results, successfully modeling long-term temporal and inter-variable dependencies. However, in complex time series, anomalies can manifest as local extreme points, cyclical deviations, or synchronous / asynchronous interactions between variables. Consequently, these methods suffer from a significant limitation: they often overlook the equally important asynchronous inter-variable dependencies. This shortcoming can lead to high false negative rates, particularly when detecting anomalous patterns caused by complex interactions between variables and time offsets. Summary of the Invention

[0005] The technical problem to be solved by the present invention is as follows: In response to the above-mentioned problems of the prior art, an unsupervised time series anomaly detection method and system based on frequency domain enhancement are provided. The present invention aims to model the intrinsic dependency of the time series to fully learn and represent the "normality" of the time series, thereby achieving accurate anomaly detection and alleviating the problem of deviation in detection results caused by noise accumulation in the dependency modeling process of the prior art.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] An unsupervised time series anomaly detection method based on frequency domain enhancement includes the following steps:

[0008] S1, standardize the multidimensional time series composed of various data of the equipment according to the dimensions of the data;

[0009] S2, after slicing the multidimensional time series, convert it to the frequency domain and extract the low-frequency components, convert the low-frequency components back to the time domain and perform rectification to obtain multidimensional time slices after frequency domain rectification ;

[0010] S3, constructs a deep neural network for feature extraction and uses multi-dimensional time slices after frequency domain rectification training deep neural networks;

[0011] S4, uses the trained deep neural network to perform anomaly detection on unlabeled multi-dimensional device data.

[0012] Optionally, in step S2, converting the multidimensional time series into the frequency domain and extracting the low-frequency components, converting the low-frequency components back to the time domain and rectifying them to obtain the multidimensional time series after frequency domain rectification includes:

[0013] S2.1, first time-slice the multidimensional time series, then apply the real fast Fourier transform (rFFT) to each time slice according to the dimension of the data to convert it into the frequency domain, obtaining a complex array in the frequency domain, which contains the amplitude and phase information of each positive frequency component;

[0014] S2.2, based on the set frequency threshold, extracting the positive frequency components below the frequency threshold;

[0015] S2.3, reconstructing the positive frequency components obtained after extracting the positive frequency components below the frequency threshold by filling the missing positive frequency components above the frequency threshold with zeros to the original length, thereby obtaining a complex number array reconstructed to the original length;

[0016] S2.4, applying the inverse real fast Fourier transform irFFT to the complex array reconstructed to the original length;

[0017] S2.5, the time domain data obtained by the inverse real fast Fourier transform irFFT is rectified using the activation function to obtain the time slice after frequency domain rectification , thus obtaining the time slices after frequency domain rectification in each dimension Multi-dimensional time slices.

[0018] Optionally, the deep neural network constructed in step S3 includes a feature extraction module, which is used to extract features that can effectively distinguish normal behavior from abnormal behavior from the multidimensional time series after frequency domain rectification. The feature extraction module includes a fully dependent capture attention layer, a feedforward network layer, a residual connection and a layer normalization layer connected in sequence. The outputs of the fully dependent capture attention layer and the feedforward network layer are added through the residual connection and then passed through the layer normalization layer to obtain features that can effectively distinguish normal behavior from abnormal behavior. The processing of the input time series by the fully dependent capture attention layer includes: adding each time slice obtained in step S2 to the input time slice. As a matrix, the attention between each element in the matrix and its cross-corresponding element is captured by cross attention, including: slicing the time Use three different transformation matrices according to the following formula , and To generate the query matrix , key matrix Sum Matrix :

[0019] ,

[0020] ,

[0021] ,

[0022] For the query matrix Each element in , calculate the attention score of row i according to the following formula and the jth column attention score :

[0023] ,

[0024] ,

[0025] In the above formula, and Key matrices The i-th and j-th rows of Represents a transpose operation;

[0026] In order to unify the row and column attention scores into the probability distribution, the attention score of the i-th row is and the jth column attention score After cascading, perform the softmax operation:

[0027] ,

[0028] ,

[0029] In the above formula, Indicates that the attention score of row i is and the jth column attention score cascade, is the attention score after cascading, is the attention score unified into the probability distribution;

[0030] Attention scores that are unified into probability distributions Separate into row vectors and column vector , then the value matrix is ​​calculated according to the following formula Weight the corresponding rows and columns in :

[0031] ,

[0032] ,

[0033] ,

[0034] In the above formula, and The value matrix The i-th row and j-th column in and are the i-th row and j-th column dependencies of the element, The row and column dependencies of the elements are obtained to obtain the row and column dependencies of all elements The row and column dependency matrix formed by the combination ; Then all the time slices obtained are The row and column dependency matrix Considered as a matrix, the row-column dependency matrix between each element in the matrix and its cross-corresponding element is captured again by cross attention , because the row and column dependent matrix The cross-corresponding element of each element in already contains the dependency relationship with its respective cross-corresponding element, so the row-column dependency matrix The matrix aggregates the time slices The dependency relationship between each element in the model and all global elements is calculated, thereby obtaining features that can effectively distinguish normal and abnormal behaviors.

[0035] Optionally, in step S3, training the deep neural network using the multidimensional time series after frequency domain rectification includes:

[0036] S3.1, initialize the current training round and the network parameters of the deep neural network;

[0037] S3.2, determine whether the current training rounds have reached the maximum number of rounds. If so, save the trained network parameters of the deep neural network and jump to step S4; otherwise, jump to step S3.3;

[0038] S3.3, determine whether the current training round has reached the maximum batch size. If so, jump to step S3.2; otherwise, jump to step S3.4;

[0039] S3.4, input the sample data processed according to steps S1 and S2 into the deep neural network, and calculate the loss using a preset loss function based on the prediction results obtained by the deep neural network;

[0040] S3.5, update the network parameters of the deep neural network according to the gradient optimization of the loss;

[0041] S3.6, add 1 to the current training round and jump to step S3.3.

[0042] Optionally, when a preset loss function is used to calculate the loss in step S3.4, the function expression of the loss function used is:

[0043] ,

[0044] In the above formula, is the loss function, is the number of slices, For deep neural networks Based on the kth time slice and network parameters The result obtained is, is the center point of the hypersphere in the feature space of the multidimensional time series.

[0045] Optionally, in step S4, the function expression for performing anomaly detection on unlabeled multi-dimensional device data using the trained deep neural network is:

[0046] ,

[0047] In the above formula, is the anomaly score of multi-dimensional device data, is the time slice of multi-dimensional device data, For deep neural networks Time slicing based on multi-dimensional device data and trained network parameters The result obtained is, is the center point of the hypersphere in the feature space of the multidimensional time series.

[0048] Optionally, the various data of the device in step S1 include part or all of the device's temperature, humidity, voltage, current, aging index data, human health indicators, operation behavior data, load data and network traffic data.

[0049] In addition, the present invention also provides an unsupervised time series anomaly detection system based on frequency domain enhancement, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the unsupervised time series anomaly detection method based on frequency domain enhancement.

[0050] In addition, the present invention also provides a computer-readable storage medium, which stores a computer program or instruction, and the computer program or instruction is programmed or configured to execute the unsupervised time series anomaly detection method based on frequency domain enhancement through a processor.

[0051] In addition, the present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the unsupervised time series anomaly detection method based on frequency domain enhancement through a processor.

[0052] Compared with existing technologies, the present invention offers the following key advantages: 1. To model the intrinsic dependencies of time series, fully learn and represent the "normality" of time series and thus achieve accurate anomaly detection, the present invention proposes a frequency domain rectifier that effectively eliminates noise while further facilitating the capture of frequency domain features to capture multiple dependencies. This effectively alleviates the problem of existing technologies in the dependency modeling process, where the multiple participation of various elements in the attention score calculation leads to significant noise accumulation, which in turn causes bias in detection results. This allows for more efficient downstream correlation learning, addressing the inability of existing unsupervised anomaly detection methods to capture asynchronous inter-dimensional dependencies and the impact of noise on detection performance. 2. The frequency domain rectification technique in the present invention significantly improves the signal-to-noise ratio by filtering out high-frequency noise components in the frequency domain. This approach not only preserves key low-frequency trends and patterns in the signal, but also reduces false positives caused by noise, allowing the model to focus more on capturing fundamental trends and long-term dependencies in time series data. Furthermore, the comprehensive dependency capture attention layer captures the intrinsic dependencies of time series data in the time domain. This fusion of frequency and time domain information gives the model better generalization capabilities, independent of specific anomaly patterns or data distributions, making it adaptable to a variety of different application scenarios and data types. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Schematic diagram of the basic process of the method of the embodiment of the present invention.

[0054] Figure 2 Schematic diagram of the basic principle of the method of the embodiment of the present invention.

[0055] Figure 3 Schematic diagram of the process of training a deep neural network in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The specific technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0057] like Figure 1 As shown, the unsupervised time series anomaly detection method based on frequency domain enhancement in this embodiment includes the following steps:

[0058] S1, standardize the multidimensional time series composed of various data of the equipment according to the dimensions of the data;

[0059] S2, after slicing the multidimensional time series, convert it to the frequency domain and extract the low-frequency components, convert the low-frequency components back to the time domain and perform rectification to obtain multidimensional time slices after frequency domain rectification ;

[0060] S3, constructs a deep neural network for feature extraction and uses multi-dimensional time slices after frequency domain rectification training deep neural networks;

[0061] S4, uses the trained deep neural network to perform anomaly detection on unlabeled multi-dimensional device data.

[0062] The various device data in step S1 of this embodiment includes some or all of the device's temperature, humidity, voltage, current, aging indicator data, human health indicators, operational behavior data, load data, and network traffic data. The standardization in step S1 refers to normalization, and other conventional preprocessing methods may be further employed as needed, including outlier filtering and missing value interpolation.

[0063] In step S2 of this embodiment, the multidimensional time series is converted to the frequency domain and the low-frequency components are extracted. The low-frequency components are converted back to the time domain and rectified to obtain the multidimensional time series after frequency domain rectification.

[0064] S2.1, first time-slice the multidimensional time series, then apply the real fast Fourier transform (rFFT) to each time slice according to the dimension of the data to convert it to the frequency domain, obtaining a complex array in the frequency domain, which contains the amplitude and phase information of each positive frequency component; apply the real fast Fourier transform (rFFT) to convert the acquired pre-processed data to the frequency domain. In the frequency domain, the signal is represented as a combination of its constituent frequencies, and each frequency has corresponding amplitude and phase information. For real input signals, the result of the rFFT is a complex array that contains the amplitude and phase information of the positive frequency components. Converting data to the frequency domain can reveal patterns and relationships that are not obvious in the time domain, thereby better obtaining useful information;

[0065] S2.2, based on the set frequency threshold, extracting the positive frequency components below the frequency threshold;

[0066] S2.3, after extracting the positive frequency components below the frequency threshold, the missing positive frequency components above the frequency threshold are padded with zeros to reconstruct the positive frequency components to the original length, thereby obtaining a complex number array reconstructed to the original length. In the frequency domain, low-frequency components usually contain the main trend and periodic information of the signal, while high-frequency components usually contain more random fluctuations and noise. The high-frequency components are filtered out, and only the low-frequency components are retained. This process sets the frequency components above a certain threshold in the frequency domain representation to zero. This threshold is dynamically determined based on the characteristics of the data set. This allows the model to focus on capturing low-frequency components that capture basic trends and long-term correlations. This process can more accurately model the underlying pattern of the signal while effectively filtering out noise. After filtering out the high-frequency components, we need to reconstruct the data by padding with zeros. Zero padding ensures that the length of the original data can be restored after the inverse transform, maintaining the integrity of the data. Zero padding in the frequency domain can reduce oscillations and discontinuities in the inverse transformed signal, resulting in a smoother output, and can prevent aliasing during the inverse transform, ensuring that the reconstructed signal does not introduce new frequency components.

[0067] S2.4, applying the inverse real fast Fourier transform irFFT to the complex array reconstructed to the original length;

[0068] S2.5, the time domain data obtained by the inverse real fast Fourier transform irFFT is rectified using the activation function to obtain the time slice after frequency domain rectification , thus obtaining the time slices after frequency domain rectification in each dimension Multi-dimensional time slices.

[0069] The deep neural network constructed in step S3 of this embodiment includes a feature extraction module, which is used to extract features that can effectively distinguish normal behavior from abnormal behavior from the multidimensional time series after frequency domain rectification. The feature extraction module includes a fully dependent capture attention layer, a feedforward network layer, a residual connection, and a layer normalization layer connected in sequence. The outputs of the fully dependent capture attention layer and the feedforward network layer are added through a residual connection and then passed through a layer normalization layer to obtain features that can effectively distinguish normal behavior from abnormal behavior. The processing of the input time series by the fully dependent capture attention layer includes:

[0070] 1) Each time slice obtained in step S2 Viewed as a matrix, the attention between each element in the matrix and its cross-corresponding element is captured by cross attention, including:

[0071] Slice time Use three different transformation matrices according to the following formula , and To generate the query matrix , key matrix Sum Matrix :

[0072] ,

[0073] ,

[0074] ,

[0075] For the query matrix Each element in , calculate the attention score of row i according to the following formula and the jth column attention score :

[0076] ,

[0077] ,

[0078] In the above formula, and Key matrices The i-th and j-th rows of Represents a transpose operation;

[0079] In order to unify the row and column attention scores into the probability distribution, the attention score of the i-th row is and the jth column attention score After cascading, perform the softmax operation:

[0080] ,

[0081] ,

[0082] In the above formula, Indicates that the attention score of row i is and the jth column attention score cascade, is the attention score after cascading, is the attention score unified into the probability distribution;

[0083] Attention scores that are unified into probability distributions Separate into row vectors and column vector , then the value matrix is ​​calculated according to the following formula Weight the corresponding rows and columns in :

[0084] ,

[0085] ,

[0086] ,

[0087] In the above formula, and The value matrix The i-th row and j-th column in and are the i-th row and j-th column dependencies of the element, The row and column dependencies of the elements are obtained to obtain the row and column dependencies of all elements The row and column dependency matrix formed by the combination ;

[0088] 2) Slice all the time The row and column dependency matrix Considered as a matrix, the row-column dependency matrix between each element in the matrix and its cross-corresponding element is captured again by cross attention , because the row and column dependent matrix The cross-corresponding element of each element in already contains the dependency relationship with its respective cross-corresponding element, so the row-column dependency matrix The matrix aggregates the time slices The dependency relationship between each element in the dataset and all global elements is analyzed, thereby obtaining features that can effectively distinguish normal from abnormal behavior. The comprehensive dependency capture attention layer is designed to deeply understand the comprehensive dependencies in time series data. Such dependencies include, but are not limited to, temporal dependencies, inter-variable dependencies, and asynchronous inter-variable dependencies. To effectively model these dependencies, this module adopts the following strategies: Cross-Attention Computation: Cross-Attention Capture treats each slice of the time series data as a matrix and, through a self-attention mechanism, computes the attention between each element in the matrix and its cross-corresponding elements (i.e., the values ​​of the same variable at different time points and the values ​​of different variables at the same time point). Double-Layer Attention Aggregation: To capture more complex dependencies, cross-attention capture is used again through a second attention computation to aggregate information. The first computation focuses on capturing row and column dependencies of elements, while the second computation further aggregates these dependencies, achieving a comprehensive modeling of the time series data. The comprehensive dependency capture attention layer can comprehensively capture the complex dependencies in time series data, providing richer contextual information for time series anomaly detection, thereby improving detection accuracy and robustness. Furthermore, through sophisticated design, this layer effectively reduces computational complexity and can process high-dimensional time series data while maintaining computational efficiency, making it suitable for application scenarios of various scales. Furthermore, this embodiment also includes constructing a feedforward network layer for further nonlinear transformation of the output of the multi-head attention layer, enhancing the model's ability to express features and capturing deeper feature relationships. To avoid gradient problems in deep network training, the residual connection and layer normalization layers add the output of each sublayer (attention layer and feedforward network layer) to the input to form a residual connection. Furthermore, layer normalization is applied to the output of the residual connection to stabilize the training process.

[0089] In this embodiment, the maximum number of epochs and the maximum number of batches per epoch are first set. At the beginning of each iteration, mini-batch data is selected from the training set and standardized. Next, these data are passed through the frequency domain rectification module for low-frequency feature extraction, in which the data is converted to the frequency domain through fast Fourier transform, low-frequency features are extracted, high-frequency noise is filtered out, and zero-filling is performed. The data is then reconstructed back to the time domain through inverse Fourier transform. The reconstructed data then enters the feature extraction module, in which the comprehensive dependency capture attention layer performs comprehensive dependency modeling on the data, and captures the global complex dependency pattern through double-layer attention aggregation. The feedforward network layer performs further nonlinear transformation on these features, while the residual connection and layer normalization layer stabilize the training process. Finally, the model calculates the loss through the classification loss function, which measures the distance of the normal sample from the center of the hypersphere in the feature space. The model parameters are updated through backpropagation and gradient descent. This process continues until the set maximum number of epochs is reached, and finally a model that can accurately detect anomalies in time series data is trained. Specifically, Figure 3 As shown, in step S3 of this embodiment, training a deep neural network using the multidimensional time series after frequency domain rectification includes:

[0090] S3.1, initialize the current training epoch, the maximum batch size for each training epoch, and the network parameters of the deep neural network;

[0091] S3.2, determine whether the current training rounds have reached the maximum number of rounds. If so, save the trained network parameters of the deep neural network and jump to step S4; otherwise, jump to step S3.3;

[0092] S3.3, determine whether the current training round has reached the maximum batch size. If so, jump to step S3.2; otherwise, jump to step S3.4;

[0093] S3.4, input the sample data processed according to steps S1 and S2 into the deep neural network, and calculate the loss using a preset loss function based on the prediction results obtained by the deep neural network;

[0094] S3.5, update the network parameters of the deep neural network according to the gradient optimization of the loss;

[0095] S3.6, add 1 to the current training round and jump to step S3.3.

[0096] The anomaly detection method adopted by the present invention is based on a deep classification classifier, which aims to identify and distinguish normal patterns from abnormal patterns in time series data. The method defines a hypersphere by learning the intrinsic distribution of normal data. The hypersphere can tightly surround normal data points and exclude abnormal data points. During training, first, a hypersphere center point c is defined in the feature space, which represents the center position of the normal time series data. Then, the deep neural network constructed in step S3 is used as the nonlinear mapping function ϕ to map the input data to a low-dimensional feature space. A classification loss function is defined to measure the square distance between the normal sample and the center of the hypersphere in the feature space. The goal of the loss function is to minimize the distance between the normal sample and the center point, so that the model can identify and exclude abnormal data. Specifically, when the preset loss function is used to calculate the loss in step S3.4 of this embodiment, the function expression of the loss function used is:

[0097] ,

[0098] In the above formula, is the loss function, is the number of slices, For deep neural networks Based on the kth time slice and network parameters The result obtained is, The center of the hypersphere in the feature space of the multidimensional time series is represented by the training data. The model parameters are updated using optimization algorithms such as backpropagation and gradient descent to minimize the classification loss function. This step is repeated until the maximum number of batches per epoch is reached. The number of epochs is updated until the pre-set maximum number of epochs is reached. The trained anomaly scoring model is then used as the anomaly detection model to be tested.

[0099] In step S4, the function expression for performing anomaly detection on unlabeled multi-dimensional device data using the trained deep neural network is:

[0100] ,

[0101] In the above formula, is the anomaly score of multi-dimensional device data, is the time slice of multi-dimensional device data, For deep neural networks Time slicing based on multi-dimensional device data and trained network parameters The result obtained is, is the center point of the hypersphere in the feature space of the multidimensional time series. If the anomaly score of a new sample exceeds the preset threshold, it is classified as an anomaly.

[0102] As an optional implementation, Figure 2 As shown, the unsupervised time series anomaly detection method based on frequency domain enhancement in this embodiment includes: a data preprocessing module, which is used to standardize the multidimensional time series composed of various data of the device according to the dimension of the data; a frequency domain rectification module, which is used to convert the multidimensional time series into the frequency domain and extract the low-frequency components, convert the low-frequency components back to the time domain and rectify them to obtain the multidimensional time series after frequency domain rectification; a feature extraction module, which is used to construct a deep neural network for feature extraction and train the deep neural network using the multidimensional time series after frequency domain rectification; and anomaly detection module, which is used to use the trained deep neural network to perform anomaly detection on unlabeled multidimensional device data. The main functions of each module are described as follows: (1) The data preprocessing module is responsible for preprocessing the original time series data, including setting iteration parameters and dimensional standardization of mini-batch data to ensure the consistency of the data in different dimensions and provide high-quality input for subsequent processing. (2) The frequency domain rectification module converts the time series data into the frequency domain by applying fast Fourier transform, focusing on extracting the low-frequency trend and periodic information of the signal while filtering out high-frequency noise. Through this process, we can significantly improve the signal-to-noise ratio of the signal and retain key pattern information, which is beneficial for subsequent feature extraction and anomaly detection. Zero padding and inverse Fourier transform further ensure the integrity and smoothness of the data in the time domain, avoiding aliasing during the inverse transformation process. (3) Based on frequency domain rectification, the feature extraction module uses the comprehensive dependency capture attention layer to deeply explore the comprehensive dependency relationship in the time series data, including time dependency, inter-variable dependency, and asynchronous inter-variable dependency. Through the two-layer attention aggregation strategy, the module can fully capture the complex interaction pattern of the data. Combined with the feedforward network layer and residual connection, the module further extracts key features that are helpful for anomaly detection. At the same time, layer normalization is used to stabilize the training process and enhance the generalization ability of the model. This series of feature extraction processes ensures that the model can extract features that effectively distinguish normal and abnormal behaviors from the processed data. (4) The anomaly detection module is based on a deep classification classifier. This module maps the data to the feature space through a nonlinear mapping function and defines a classification loss function to distinguish normal and abnormal data, ultimately achieving accurate anomaly detection of time series data.

[0103] In summary, in order to model the intrinsic dependencies of time series, fully learn and represent the "normality" of time series and thus achieve accurate anomaly detection, this embodiment proposes a frequency domain rectifier to effectively eliminate noise while further promoting the capture of frequency domain features to capture multiple dependencies. This can effectively alleviate the problem of significant noise accumulation caused by the multiple participation of various elements in the attention score calculation during the dependency modeling process in existing technologies, thereby leading to biased detection results. It also promotes more effective downstream correlation learning and solves the problems of existing unsupervised anomaly detection methods in being unable to capture asynchronous inter-dimensional dependencies and the impact of noise on detection performance. The frequency domain rectification technology in this embodiment significantly improves the signal-to-noise ratio of the signal by filtering out high-frequency noise components in the frequency domain. This method not only retains key low-frequency trends and patterns in the signal, but also reduces false positives caused by noise, allowing the model to focus more on capturing the basic trends and long-term dependencies in time series data. In addition, the comprehensive dependency capture attention layer captures the intrinsic dependencies of time series data in the time domain. This fusion of frequency and time domain information gives the model better generalization capabilities, independent of specific anomaly patterns or data distributions, making it adaptable to a variety of different application scenarios and data types.

[0104] In addition, this embodiment also provides an unsupervised time series anomaly detection system based on frequency domain enhancement, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the unsupervised time series anomaly detection method based on frequency domain enhancement. In addition, this embodiment also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program or instructions, wherein the computer program or instructions are programmed or configured to execute the unsupervised time series anomaly detection method based on frequency domain enhancement through a processor. In addition, this embodiment also provides a computer program product, comprising a computer program or instructions, wherein the computer program or instructions are programmed or configured to execute the unsupervised time series anomaly detection method based on frequency domain enhancement through a processor.

[0105] Those skilled in the art should understand that the technical solutions provided by the embodiments of the present application may be in the form of methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0106] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An unsupervised time series anomaly detection method based on frequency domain enhancement, characterized in that: The steps include: S1, standardize the multidimensional time series composed of various data of the equipment according to the dimensions of the data; S2, after slicing the multidimensional time series, convert it to the frequency domain and extract the low-frequency components, convert the low-frequency components back to the time domain and perform rectification to obtain multidimensional time slices after frequency domain rectification ; S3, constructs a deep neural network for feature extraction and uses multi-dimensional time slices after frequency domain rectification training deep neural networks; S4, uses the trained deep neural network to perform anomaly detection on unlabeled multi-dimensional device data; The deep neural network constructed in step S3 includes a feature extraction module, which is used to extract features that can effectively distinguish normal behavior from abnormal behavior from the multidimensional time series after frequency domain rectification. The feature extraction module includes a fully dependent capture attention layer, a feedforward network layer, a residual connection and a layer normalization layer connected in sequence. The outputs of the fully dependent capture attention layer and the feedforward network layer are added through the residual connection and then passed through the layer normalization layer to obtain features that can effectively distinguish normal behavior from abnormal behavior. The processing of the input time series by the fully dependent capture attention layer includes: first, each time slice obtained in step S2 is added. As a matrix, the attention between each element in the matrix and its cross-corresponding element is captured by cross attention, including: slicing the time Use three different transformation matrices according to the following formula , and To generate the query matrix , key matrix Sum Matrix : ， ， ， For the query matrix Each element in , calculate the attention score of row i according to the following formula and the jth column attention score : ， ， In the above formula, and Key matrices The i-th and j-th rows of Represents a transpose operation; In order to unify the row and column attention scores into the probability distribution, the attention score of the i-th row is and the jth column attention score After cascading, perform the softmax operation: ， ， In the above formula, Indicates that the attention score of row i is and the jth column attention score cascade, is the attention score after cascading, is the attention score unified into the probability distribution; Attention scores that are unified into probability distributions Separate into row vectors and column vector , then the value matrix is ​​calculated according to the following formula Weight the corresponding rows and columns in : ， ， ， In the above formula, and The value matrix The i-th row and j-th column in and are the i-th row and j-th column dependencies of the element, The row and column dependencies of the elements are obtained to obtain the row and column dependencies of all elements The row and column dependency matrix formed by the combination ; Then all the time slices obtained are The row and column dependency matrix Considered as a matrix, the row-column dependency matrix between each element in the matrix and its cross-corresponding element is captured again by cross attention , row-column dependency matrix The cross-corresponding element of each element in already contains the dependency relationship with its respective cross-corresponding element, and the row-column dependency matrix The matrix aggregates the time slices The dependency relationship between each element in the model and all global elements is calculated, thereby obtaining features that can effectively distinguish normal and abnormal behaviors.

2. The unsupervised time series anomaly detection method based on frequency domain enhancement according to claim 1 is characterized in that In step S2, the multidimensional time series is converted to the frequency domain and the low-frequency components are extracted. The low-frequency components are converted back to the time domain and rectified to obtain the multidimensional time series after frequency domain rectification. S2.1, first time-slice the multidimensional time series, then apply the real fast Fourier transform (rFFT) to each time slice according to the dimension of the data to convert it into the frequency domain, obtaining a complex array in the frequency domain, which contains the amplitude and phase information of each positive frequency component; S2.2, based on the set frequency threshold, extracting the positive frequency components below the frequency threshold; S2.3, reconstructing the positive frequency components obtained after extracting the positive frequency components below the frequency threshold by filling the missing positive frequency components above the frequency threshold with zeros to the original length, thereby obtaining a complex number array reconstructed to the original length; S2.4, applying the inverse real fast Fourier transform irFFT to the complex array reconstructed to the original length; S2.5, the time domain data obtained by the inverse real fast Fourier transform irFFT is rectified using the activation function to obtain the time slice after frequency domain rectification , thus obtaining the time slices after frequency domain rectification in each dimension Multi-dimensional time slices.

3. The unsupervised time series anomaly detection method based on frequency domain enhancement according to claim 1 is characterized in that In step S3, the deep neural network is trained using the multidimensional time series after frequency domain rectification, including: S3.1, initialize the current training round and the network parameters of the deep neural network; S3.2, determine whether the current training rounds have reached the maximum number of rounds. If so, save the trained network parameters of the deep neural network and jump to step S4; otherwise, jump to step S3.3; S3.3, determine whether the current training round has reached the maximum batch size. If so, jump to step S3.2; otherwise, jump to step S3.4; S3.4, input the sample data processed according to steps S1 and S2 into the deep neural network, and calculate the loss using a preset loss function based on the prediction results obtained by the deep neural network; S3.5, update the network parameters of the deep neural network according to the gradient optimization of the loss; S3.6, add 1 to the current training round and jump to step S3.

3.

4. The unsupervised time series anomaly detection method based on frequency domain enhancement according to claim 3 is characterized in that: When the preset loss function is used for loss calculation in step S3.4, the function expression of the loss function used is: ， In the above formula, is the loss function, is the number of slices, For deep neural networks Based on the kth time slice and network parameters The result obtained is, is the center point of the hypersphere in the feature space of the multidimensional time series.

5. The unsupervised time series anomaly detection method based on frequency domain enhancement according to claim 4 is characterized in that: In step S4, the function expression for performing anomaly detection on unlabeled multi-dimensional device data using the trained deep neural network is: ， In the above formula, is the anomaly score of multi-dimensional device data, is the time slice of multi-dimensional device data, For deep neural networks Time slicing based on multi-dimensional device data and trained network parameters The result obtained is, is the center point of the hypersphere in the feature space of the multidimensional time series.

6. The unsupervised time series anomaly detection method based on frequency domain enhancement according to claim 1 is characterized in that The various data of the device in step S1 include part or all of the device's temperature, humidity, voltage, current, aging index data, human health index, operation behavior data, load data and network traffic data.

7. An unsupervised time series anomaly detection system based on frequency domain enhancement, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the unsupervised time series anomaly detection method based on frequency domain enhancement according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program or instruction stored therein, characterized in that: The computer program or instruction is programmed or configured to execute the unsupervised time series anomaly detection method based on frequency domain enhancement according to any one of claims 1 to 6 through a processor.

9. A computer program product comprising a computer program or instructions, characterized in that The computer program or instruction is programmed or configured to execute the unsupervised time series anomaly detection method based on frequency domain enhancement according to any one of claims 1 to 6 through a processor.

Images