A method for industrial Internet time series anomaly detection based on multi-scale and multi-features

Abstract

The present invention relates to a method for detecting industrial Internet time series anomaly based on multi-scale and multi-features, and belongs to the technical field of time series anomaly in the field of industrial Internet. The present invention establishes an industrial Internet time series anomaly detection model based on a time series period conversion network, a multi-feature extraction network, a data feature aggregation network and a frequency feature aggregation network; a variety of periodic information existing in a time series is captured through a periodic extraction module, and then the features between time series cycles, the features between channels of different dimensions and the features of time series within a cycle are extracted through a multi-feature extraction network respectively; then, a variety of features are aggregated based on a data feature aggregation network; finally, aggregation is performed based on a frequency feature aggregation network, and frequency aggregation features of a time series data sequence in a time series period are output, and the frequency aggregation features are used to detect anomalies of the time series data.

Description

Technical Field

[0001] The present invention relates to the field of time series anomaly technology in the field of industrial Internet, and specifically to a method for detecting time series anomaly in industrial Internet based on multi-scale and multi-feature. Background Art

[0002] Industrial Internet is to connect industrial equipment, production lines, edge servers, cloud, products, etc. through the network, and optimize the entire production process through big data analysis, artificial intelligence and other technologies. The main goal of Industrial Internet is to improve production efficiency, enhance product quality and reduce various faults in the industrial production process. It is a key technology to promote the digitalization, networking and intelligent transformation of the manufacturing industry.

[0003] Servers are one of the core infrastructures of the Industrial Internet, providing the necessary computing, storage, and network resources for the Industrial Internet. In the Industrial Internet, servers are responsible for collecting, processing, and analyzing massive amounts of data from various industrial devices and sensors. They run critical industrial software and applications, perform complex data analysis and artificial intelligence algorithms, and provide support for decision-making. At the same time, servers also assume the functions of industrial control systems, monitoring the production process in real time and remotely operating it. With the development of the Industrial Internet, the demand for high-performance and high-reliability servers is also increasing, which has promoted the development of dedicated industrial servers.

[0004] With the help of predictive maintenance and intelligent diagnosis technology, the Industrial Internet can detect potential equipment failures in advance, prevent downtime and accidents on the production line, reduce maintenance costs and production losses, and ensure the continuity and stability of the production process. For example, the Chinese patent applications with patent publication numbers CN116522265A, CN118410436A, and CN117574298A.

[0005] In industrial environments, the continuous and stable operation of servers is crucial to ensuring production continuity and data security. Therefore, the application of anomaly detection technology can effectively reduce downtime losses, improve overall production efficiency, and provide strong guarantees for the stable operation of the industrial Internet. Anomaly detection technology can monitor the performance indicators and operating status of the server in real time, and promptly detect potential problems and abnormal behaviors. By analyzing the server's CPU usage, memory usage, network traffic, disk I / O and other data, the anomaly detection system can identify abnormal patterns that do not match the normal pattern, thereby warning of possible failures. This preventive maintenance method can greatly reduce the risk of unexpected server downtime and improve the reliability and stability of the system.

[0006] In the existing technology, time series anomaly detection for the industrial Internet is mainly based on traditional machine learning methods. Algorithms based on traditional machine learning include SVM, decision tree, NB and other methods. However, such methods usually rely on feature engineering and relatively simple model structures, and have limited ability to express data. Summary of the invention

[0007] In view of the above problems, the present invention proposes a method for industrial Internet time series anomaly detection based on multi-scale and multi-features. The present invention establishes an industrial Internet time series anomaly detection model based on a time series period conversion network, a multi-feature extraction network, a data feature aggregation network and a frequency feature aggregation network; through a periodicity extraction module, a variety of periodic information existing in the time series is captured, and secondly, the features between the time series cycles, the features between channels of different dimensions and the features of the time series within the cycle are extracted through a multi-feature extraction network respectively; again, a variety of features are aggregated based on a data feature aggregation network; finally, aggregation is performed based on a frequency feature aggregation network, and the frequency aggregation features of the time series data sequence in the time series cycle are output, and the frequency aggregation features are used to detect anomalies in the time series data.

[0008] The present invention provides a method for detecting industrial Internet time series anomalies based on multi-scale and multi-feature extraction, comprising:

[0009] Step S1, building an industrial Internet time series anomaly detection framework; embedding a time series period conversion network, a multi-feature extraction network, a data feature aggregation network and a frequency feature aggregation network in the industrial Internet time series anomaly detection framework to obtain an original industrial Internet time series anomaly detection model;

[0010] Step S2: h =1, when h =1, it is the first time series data sequence of the Industrial Internet;

[0011] Step S3: Get the Industrial Internet h A time series data sequence and input into the time series period conversion network D h , perform wavelet transform to obtain transformed data, and then perform h The time series data sequence is divided into periods to obtain multiple time series periods;

[0012] Step S4: k =1, k =1 means h The first time series cycle of a time series data sequence;

[0013] Step S5: Extracting network based on multiple features A h,k Get the first hThe time series data sequence is in the time series period k The multiple features of

[0014] Step S6: h The time series data sequence is in the time series period k Feature aggregation network for multi-feature input data B h,k , output h The time series data sequence is in the time series period k The aggregated features of

[0015] Step S7, determining the loss function and evaluation index;

[0016] Step S8: h The time series data sequence is in the time series period k The aggregated features are input into the frequency feature aggregation network C h,k Aggregate and output the h The time series data sequence is in the time series period k The frequency aggregation features are used to obtain the updated industrial Internet time series anomaly detection model A h,k+1 ;

[0017] Step S9: Determination k Is it greater than or equal to K , K Indicates the total number of time series cycles. If yes, the Industrial Internet time series anomaly detection model will be updated A h,k+1 As the h Industrial Internet Time Series Anomaly Detection Model Based on Time Series Data Sequences E h , go to the next step; if not, k = k +1, return to step S5;

[0018] Step S10: Determination h Is it greater than or equal to H , H is the total number of time series data. If so, h Industrial Internet Time Series Anomaly Detection Model Based on Time Series Data Sequences E h As the final industrial Internet time series anomaly detection model E If not, h = h +1, return to step S3;

[0019] Step S11: Using the final industrial Internet time series anomaly detection model EPerform anomaly detection on time series data of the Industrial Internet.

[0020] Preferably, the specific steps of obtaining multiple timing cycles in step S3 include:

[0021] Step S31: Setting the timing cycle conversion network D h ; Get the Industrial Internet h The time series data sequence is input into the time series period conversion network, and the wavelet transform is performed to extract the amplitude at each moment and its corresponding frequency;

[0022] Step S32, preset a frequency spacing; detect the amplitudes at multiple moments according to the frequency spacing, filter out frequencies corresponding to larger amplitudes, and obtain multiple filtered frequencies;

[0023] Step S33: based on the screened multiple frequencies, divide the time series data sequence into periods to obtain multiple time series periods.

[0024] Furthermore, the expressions of the amplitude and frequency in step S31 are respectively:

[0025]

[0026]

[0027] in, For the h The amplitude of the time series data, (•) is the amplitude extraction function, is the wavelet transform function, (•) is the mean function for the dimension, For the h A time series data sequence, For the h The frequency of the time series data, (•) is the frequency extraction function.

[0028] Preferably, the multi-feature extraction network in step S1 includes a channel feature extraction network, a period feature extraction network and a timing feature extraction network.

[0029] Preferably, the channel feature extraction network in step S1 includes a linear layer network, an activation layer network, a fitting layer network and a normalization layer network.

[0030] Preferably, the data feature aggregation network in step S1 includes a shape change layer, a sorting layer and a linear layer.

[0031] Preferably, step S5 h The time series data sequence is in the time series period kThe multiple features include h The time series data sequence is in the time series period k The channel characteristics of h The time series data sequence is in the time series period k The periodic characteristics and h The time series data sequence is in the time series period k The timing characteristics of .

[0032] Further, step S6 outputs the h The time series data sequence is in the time series period k The specific steps of the aggregated features include:

[0033] In the data feature aggregation network B h,k In h The time series data sequence is in the time series period k Feature aggregation network for multi-feature input data B h,k , make changes and get h The time series data sequence is in the time series period k Changed cycle characteristics;

[0034] The first h The time series data sequence is in the time series period k The channel characteristics of h The time series data sequence is in the time series period k The changed cycle characteristics and h The time series data sequence is in the time series period k The time series features are aggregated to obtain the h The time series data sequence is in the time series period k The aggregated features are output.

[0035] Compared with the prior art, the present invention has at least the following beneficial effects:

[0036] (1) The method of the present invention extracts a variety of periodic information through multi-period analysis, which enhances the model's understanding and analysis of periodicity at different time scales;

[0037] (2) The present invention extracts features from several different aspects, making the feature extraction more complete and enhancing the modeling capability of complex data. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings are only for the purpose of illustrating particular embodiments and are not to be construed as limiting the invention.

[0039] Figure 1 It is a schematic diagram of the structure of the industrial Internet time series anomaly detection model in an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. In addition, the present invention can also be implemented in other ways different from those described herein, and therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0041] A specific embodiment of the present invention, as Figure 1 , discloses a method for detecting industrial Internet time series anomalies based on multi-scale and multi-feature extraction. In order to illustrate the effectiveness of the method proposed in the present invention, the above technical solution of the present invention is described in detail through a specific embodiment below. The specific implementation steps are as follows:

[0042] Step S1, building an industrial Internet time series anomaly detection framework; embedding a time series period conversion network, a multi-feature extraction network, a data feature aggregation network and a frequency feature aggregation network in the industrial Internet time series anomaly detection framework to obtain an original industrial Internet time series anomaly detection model;

[0043] Step S2: h =1, when h =1, it is the first time series data sequence of the Industrial Internet;

[0044] It can be understood that the time series data of the industrial Internet includes server data; the server data includes CPU occupancy rate and disk occupancy rate;

[0045] Step S3: Get the Industrial Internet h A time series data sequence and input into the time series period conversion network D h , performing wavelet transform to obtain transformed data, and periodizing the time series data sequence based on the transformed data to obtain multiple time series periods;

[0046] Step S4: k =1, k =1 means h The first time series cycle of a time series data sequence;

[0047] Step S5: Extracting network based on multiple features A h,k Get the first h The time series data sequence is in the time series period k The multiple features of

[0048] Step S6:h The time series data sequence is in the time series period k Feature aggregation network for multi-feature input data B h,k , output h The time series data sequence is in the time series period k The aggregated features of

[0049] Step S7, determining the loss function and evaluation index;

[0050] Step S8: h The time series data sequence is in the time series period k The aggregated features are input into the frequency feature aggregation network C h,k Aggregate and output the h The time series data sequence is in the time series period k The frequency aggregation features are used to obtain the updated industrial Internet time series anomaly detection model A h,k+1 ;

[0051] Step S9: Determination k Is it greater than or equal to K , K Indicates the total number of time series cycles. If yes, the Industrial Internet time series anomaly detection model will be updated A h,k+1 As the h Industrial Internet Time Series Anomaly Detection Model Based on Time Series Data Sequences E h , go to the next step; if not, k = k +1, return to step S5;

[0052] Step S10: Determination h Is it greater than or equal to H , H is the total number of time series data. If so, h Industrial Internet Time Series Anomaly Detection Model Based on Time Series Data Sequences E h As the final industrial Internet time series anomaly detection model E If not, h = h +1, return to step S3;

[0053] Step S11: Using the final industrial Internet time series anomaly detection model E Perform anomaly detection on time series data of the Industrial Internet.

[0054] Preferably, the multi-feature extraction network in step S1 includes a channel feature extraction network, a period feature extraction network and a time series feature extraction network;

[0055] Furthermore, the channel feature extraction network includes a linear layer network, an activation layer network, a fitting layer network and a normalization layer network;

[0056] The periodic feature extraction network includes a first shape change layer, an attention layer, a fitting layer, a normalization layer, a first convolutional layer, an activation layer, a second fitting layer, a second convolutional layer, a third fitting layer and a second shape change layer;

[0057] The temporal feature extraction network includes a first dimension conversion layer, a TCN network layer and a second dimension conversion layer.

[0058] Preferably, the data feature aggregation network in step S1 includes: a shape change layer, a sorting layer and a linear layer.

[0059] Preferably, the specific steps of obtaining multiple timing cycles in step S3 include:

[0060] Step S31: Setting the timing cycle conversion network D h ; Get the Industrial Internet h The time series data sequence is input into the time series period conversion network, and the wavelet transform is performed to extract the amplitude at each moment and its corresponding frequency;

[0061] Step S32, preset a frequency spacing; detect the amplitudes at multiple moments according to the frequency spacing, filter out frequencies corresponding to larger amplitudes, and obtain multiple filtered frequencies;

[0062] Step S33: based on the screened multiple frequencies, divide the time series data sequence into periods to obtain multiple time series periods.

[0063] Furthermore, the expressions of the amplitude and frequency in step S31 are respectively:

[0064]

[0065]

[0066] in, (•) is the amplitude extraction function, (•) is the frequency extraction function, (•) is the mean function for the dimension, which transforms the multidimensional variable into a one-dimensional variable. For the h The amplitude of the time series data, For the h The frequency of the time series data, is the wavelet transform function, For the h A time series data sequence.

[0067] Preferably, the expression of the multiple timing cycles in step S3 is:

[0068]

[0069] in, Indicates h Time series data K Timing cycles; For the k The filtered frequency corresponding to the timing cycle is For the h The time series data k A timing cycle.

[0070] Preferably, step S5 h The time series data sequence is in the time series period k The multiple features include h The time series data sequence is in the time series period k The channel characteristics of h The time series data sequence is in the time series period k The periodic characteristics and h The time series data sequence is in the time series period k The timing characteristics of

[0071] Furthermore, the h The time series data sequence is in the time series period k The expression of the channel characteristics is:

[0072]

[0073]

[0074]

[0075] in, For the h The time series data series is k The initial channel features extracted from the time series period, is the time series data sequence of the Industrial Internet, Indicates h A time series data sequence, T represents transpose, For the h The time series data series is k The activation layer output of the channel feature extraction network is a time series cycle. For the hThe time series data series is k The timing cycles are output at the fitting layer of the channel feature extraction network. is the activation function, is the linear layer function, It is a random inactivation function. During the training process, the output of some neurons is set to zero with a certain probability to prevent the neural network from overfitting. The layer normalization function performs normalization on the input data of each layer so that the output of each neuron has zero mean and unit variance, thereby alleviating the gradient vanishing or gradient exploding problem in neural network training.

[0076] Furthermore, the h The time series data sequence is in the time series period k The expression of the periodic characteristic is:

[0077]

[0078]

[0079]

[0080]

[0081]

[0082] in, is the shape change function, For the h The time series data series is k The original periodic characteristics of the time series period; is the attention function, is the first convolution function, is the second convolution function, For the h The time series data series is k The output of the attention layer of the period feature extraction module for the time series cycle, For the h The time series data series is k The output of the attention layer of the period feature extraction module is k The result of normalizing the original periodic features of the time series period is: For the h The time series data series is k The output of the fitting layer of the period feature extraction module for the time series period, For the h The time series data series is k The periodic characteristics of a timing cycle.

[0083] Furthermore, the h The time series data sequence is in the time series period k The expression of the time series characteristics is:

[0084]

[0085]

[0086] in, For the h The time series data series is k The initial timing characteristics of the timing cycle represent The tensor after dimension transformation, For the h The time series data series is k The timing characteristics of the timing cycle, is the dimension conversion function, is the TCN network structure function.

[0087] Preferably, step S6 outputs the h The time series data sequence is in the time series period k The specific steps of the aggregated features include:

[0088] In the data feature aggregation network B h,k In h The time series data sequence is in the time series period k Feature aggregation network for multi-feature input data B h,k , make changes and get h The time series data sequence is in the time series period k Changed cycle characteristics;

[0089] The first h The time series data sequence is in the time series period k The channel characteristics of h The time series data sequence is in the time series period k The changed cycle characteristics and h The time series data sequence is in the time series period k The time series features are aggregated to obtain the h The time series data sequence is in the time series period k Aggregate the features and output them;

[0090] Preferably, step S6 h The time series data sequence is in the time series period k The expression of the aggregated features is:

[0091]

[0092]

[0093]

[0094] in, For the h The time series data series is k Aggregate characteristics of the timing cycles of the frequencies, For the h The time series data series is k The periodic characteristics after the timing cycle is changed, It is a sorting function that arranges the data in a time series period in order along a certain dimension. For the h The time series data series is k Aggregated characteristics of the timing cycles of the frequencies.

[0095] Preferably, the loss function in step S7 is expressed as:

[0096]

[0097] in, Represents the k The true value of the frequency aggregation feature of the time series period, Represents the k The predicted value of the frequency aggregation feature of the time series period, represents the loss function.

[0098] Preferably, the evaluation indicators in step S7 include precision, recall, F1-score and accuracy, and the expressions are:

[0099]

[0100]

[0101]

[0102] in, is the precision rate, which indicates the proportion of samples predicted by the model to be positive that are actually positive; Indicates the number of samples that are actually positive and predicted to be positive. Indicates the number of samples that are actually negative and predicted to be positive. Indicates the number of samples that are actually positive and predicted to be negative. for Precision and Recall The harmonic mean of It indicates the proportion of samples that are actually positive that the model correctly identifies as positive.

[0103] Preferably, step S8 h The time series data sequence is in the time series period k The frequency aggregation feature of is expressed as:

[0104]

[0105]

[0106] in, For the h The corresponding original frequency aggregation features of the time series data series, For the h The frequency aggregation features corresponding to the time series data series.

[0107] The present invention further includes step S12, performing simulation using the industrial Internet time series anomaly detection model to obtain a simulation result;

[0108] Specifically, the dataset used is the SMD dataset (Server Machine Dataset), which collects data from 28 servers running continuously for 5 weeks. The time interval between two adjacent groups is 1 minute, and each machine collects information of 38 dimensions (variables). The simulation results are shown in Table 1.

[0109] The proposed method is compared with TimesNet, Dlinear, and PatchTST methods; TimesNet uses a two-dimensional Inception network structure to reshape the one-dimensional time series data into a two-dimensional form after calculating the cycle through Fourier transform, and then extracts features through multi-layer two-dimensional convolution. The experimental results show that the F1-score of TimesNet reaches 0.8581;

[0110] The Dlinear model first decomposes the time series data into trend items and residual items, and then processes these two parts through linear layers and finally outputs the results. The experimental results show that the F1-score of the model is 0.7710;

[0111] PatchTST is a Transformer-based model whose core assumption is that different dimensions of multidimensional time series data are independent of each other. The model divides the data into multiple patches through a sliding window. In anomaly detection training, the output of the model fits the input as closely as possible. Experimental results show that the F1-score of this method reaches 0.8444;

[0112] Table 1 Comparison of different methods

[0113]

[0114] From the above comparative experimental results, it can be seen that the F1-score of the TimesNet method is 0.8581 and the Precision is 0.8866. The F1-score of Dlinear is 0.7710, and the F1-score of PatchTST is 0.8444. Although the precision index of the method proposed in the present invention is 0.8834, which is slightly lower than the result of TimesNet, the most important index F1-score can reach 0.8611, achieving better performance. This proves the effectiveness of the method proposed in the present invention.

[0115] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for detecting anomalies in industrial Internet time series based on multi-scale and multi-features, characterized in that: include: Step S1, building an industrial Internet time series anomaly detection framework; embedding a time series period conversion network, a multi-feature extraction network, a data feature aggregation network and a frequency feature aggregation network in the industrial Internet time series anomaly detection framework to obtain an original industrial Internet time series anomaly detection model; Step S2: h =1, when h =1, it is the first time series data sequence of the Industrial Internet; Step S3: Get the Industrial Internet h A time series data sequence and input into the time series period conversion network D h , perform wavelet transform to obtain transformed data, and then perform h The time series data sequence is divided into periods to obtain multiple time series periods; Step S4: k =1, k =1 means h The first time series cycle of a time series data sequence; Step S5: Extracting network based on multiple features A h,k Get the first h The time series data sequence is in the time series period k The multiple features of Step S6: h The time series data sequence is in the time series period k Feature aggregation network for multi-feature input data B h,k , output h The time series data sequence is in the time series period k The aggregated features of Step S7, determining the loss function and evaluation index; Step S8: h The time series data sequence is in the time series period k The aggregated features are input into the frequency feature aggregation network C h,k Aggregate and output the h The time series data sequence is in the time series period k The frequency aggregation features are used to obtain the updated industrial Internet time series anomaly detection model A h,k+1 ; Step S9: Determination k Is it greater than or equal to K , K Indicates the total number of time series cycles. If yes, the Industrial Internet time series anomaly detection model will be updated A h,k+1 As the h Industrial Internet Time Series Anomaly Detection Model Based on Time Series Data Sequences E h , go to the next step; if not, k = k +1, return to step S5; Step S10: Determination h Is it greater than or equal to H , H is the total number of time series data. If so, h Industrial Internet Time Series Anomaly Detection Model Based on Time Series Data Sequences E h As the final industrial Internet time series anomaly detection model E If not, h = h +1, return to step S3; Step S11: Using the final industrial Internet time series anomaly detection model E Perform anomaly detection on time series data of the Industrial Internet.

2. The method for detecting anomalies in industrial Internet time series based on multi-scale and multi-features according to claim 1 is characterized in that: The specific steps of obtaining multiple timing cycles in step S3 include: Step S31: Setting the timing cycle conversion network D h ; Get the Industrial Internet h The time series data sequence is input into the time series period conversion network, and the wavelet transform is performed to extract the amplitude at each moment and its corresponding frequency; Step S32, preset a frequency spacing; detect the amplitudes at multiple moments according to the frequency spacing, filter out frequencies corresponding to larger amplitudes, and obtain multiple filtered frequencies; Step S33: based on the screened multiple frequencies, divide the time series data sequence into periods to obtain multiple time series periods.

3. The method for detecting anomalies in industrial Internet time series based on multi-scale and multi-features according to claim 2 is characterized in that: The expressions of the amplitude and frequency in step S31 are: in, For the h The amplitude of the time series data, (•) is the amplitude extraction function, is the wavelet transform function, (•) is the mean function for the dimension, For the h A time series data sequence, For the h The frequency of the time series data, (•) is the frequency extraction function.

4. The method for detecting anomalies in industrial Internet time series based on multi-scale and multi-features according to claim 1 is characterized in that: The multi-feature extraction network in step S1 includes a channel feature extraction network, a period feature extraction network and a timing feature extraction network.

5. The method for detecting anomalies in industrial Internet time series based on multi-scale and multi-features according to claim 4 is characterized in that: The channel feature extraction network in step S1 includes a linear layer network, an activation layer network, a fitting layer network and a normalization layer network.

6. The method for detecting anomalies in industrial Internet time series based on multi-scale and multi-features according to claim 1 is characterized in that: The data feature aggregation network in step S1 includes a shape change layer, a sorting layer and a linear layer.

7. The method for detecting anomalies in industrial Internet time series based on multi-scale and multi-features according to claim 1 is characterized in that: Step S5 h The time series data sequence is in the time series period k The multiple features include h The time series data sequence is in the time series period k The channel characteristics of h The time series data sequence is in the time series period k The periodic characteristics and h The time series data sequence is in the time series period k The timing characteristics of .

8. The method for detecting anomalies in industrial Internet time series based on multi-scale and multi-features according to claim 7 is characterized in that: Step S6 outputs the h The time series data sequence is in the time series period k The specific steps of the aggregated features include: The first h The time series data sequence is in the time series period k Feature aggregation network for multi-feature input data B h,k , obtain the h The time series data sequence is in the time series period k Changed cycle characteristics; The first h The time series data sequence is in the time series period k The channel characteristics of h The time series data sequence is in the time series period k The changed cycle characteristics and h The time series data sequence is in the time series period k The time series features are aggregated to obtain the h The time series data sequence is in the time series period k The aggregated features are output.

Images