Communication time series data anomaly detection method based on Transform architecture

Abstract

The invention discloses a communication time series data anomaly detection method based on a Transform architecture, and the method comprises the following steps: S1), carrying out the modeling prediction of original time series data and first-order difference time series data, and taking the prediction result of the first-order difference as a correction factor of the prediction result of the original time series data; s2) modifying a position code addition strategy in an original Transform architecture, introducing an exponential decay strategy of a 1 + half-life algorithm into relative time coding, and multiplying absolute time coding and relative time coding to fuse time coding features; and S3) the DTW is used to calculate the distance between the time sequence data predicted by Transform end-to-end prediction and the actual time sequence data. According to the method, deep learning and a traditional data mining model can be combined, distance calculation is fused, and dynamic adjustment can be performed according to actual conditions, so that an accurate anomaly detection effect is achieved.

Description

Technical Field

[0001] The present invention relates to a communication time series data anomaly detection method, and in particular to a communication time series data anomaly detection method based on a Transformer architecture. Background Art

[0002] In the field of aviation communications, a large amount of communication time series data is generated, and anomaly detection of time series data is an important technical means to ensure aviation safety. By detecting anomalies in time series data, anomalies can be discovered in real time and early warnings can be issued. Traditional anomaly detection technology mainly identifies abnormal situations based on rules and thresholds. For example, when the speed, altitude or heading of an aircraft exceeds the preset safety parameters, the system will issue an alarm. This traditional detection technology cannot use the large amount of data and complex environments that are rapidly accumulated now.

[0003] With the development of technology, people have begun to use machine learning methods to detect anomalies in time series data. For example, Chinese patent document No. CN117932519A discloses a new energy vehicle charging station monitoring system. The safety warning module combines support vector machines, decision trees and isolation forest algorithms to improve system safety. In terms of pricing optimization, the ARIMA model and Q learning algorithm are used to adjust charging prices according to grid load and user demand to improve fairness and market response. However, the calculation based on the mining algorithm cannot process long sequence data, and the sequence length is limited.

[0004] With the development of deep learning technology, people have begun to try to use neural network methods for anomaly detection. For example, Chinese patent document No. CN117909178A discloses a method in an integrated online log anomaly detection model based on LSTM. The deep learning model LSTM is used as a weak learner model, and the weak learner is trained by transfer learning to better learn the data features of the log. Then the weak learner is integrated into a strong learner to perform real-time anomaly detection on the incoming logs. This model has a better adaptation to long sequence data. However, the LSTM architecture does not take into account the time trend, such as the time trend, data characteristics, and periodic characteristics in the ARIMA model, which cannot be added to the LSTM architecture. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a communication time series data anomaly detection method based on the Transformer architecture, which can combine deep learning and traditional data mining models, integrate distance calculation, and can be dynamically adjusted according to actual conditions to achieve accurate anomaly detection effect.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is to provide a communication time series data anomaly detection method based on the Transformer architecture, comprising the following steps: S1) modeling and predicting the original time series data and the first-order difference time series data respectively, and using the prediction result of the first-order difference as the correction factor of the prediction result of the original time series data; S2) modifying the strategy of adding position codes in the original Transformer architecture, introducing the exponential decay strategy of the 1+half-life algorithm in the relative time coding, and multiplying the absolute time coding and the relative time coding to fuse the time coding features; S3) using DTW to calculate the distance between the time series data predicted by the Transformer end-to-end and the actual time series data.

[0007] Furthermore, in step S1, the time series data and the first-order interpolation are respectively input into the Transformer framework as sequence data to remove the trend features in the time series data and extract the essential features of the time series data; the first-order difference of the time series data is:

[0008] ΔY t =Y t -Y t-1 ;

[0009] ΔY t Represents the first-order difference value at time t.

[0010] Furthermore, the step S2 modifies the original position code in the Transformer into a time code, and the time code is divided into an absolute time code and a relative time code;

[0011] For each time step t, the absolute time code is calculated as follows:

[0012] TE(t,2i)=sin(t / 10000 2i / d )

[0013] TE(t,2i+1)=cos(t / 10000 2i / d )

[0014] Where d is the dimension of the model and i is the index of the dimension;

[0015] For each time step t, the relative time encoding is calculated as follows:

[0016]

[0017] where λ is the decay constant and i is the index of the dimension.

[0018] Further, k is the time point of the test half-life.

[0019] Furthermore, the value of k is 250-350.

[0020] Furthermore, the value of k is 300.

[0021] Further, assuming that there are two time series A and B, with lengths of m and n respectively, step S3 includes: S31) initializing the distance matrix: creating a matrix D of size (m+1)×(n+1), where D[i][j] represents the minimum cumulative distance between the first i elements of sequence A and the first j elements of sequence B; initializing D[0][0]=0, and the other D[i][0] and D[0][j] are infinite; S32) calculating the cumulative distance of each element: for i from 1 to m, j from 1 to n, fill the matrix in the following manner: D[i][j]=cos t(A[i],B[j])+min(D[i-1][j],D[i][j-1],D[i-1][j-1]) where cost(A[i],B[j]) is the Euclidean distance between A[i] and B[j]; S33) Find the best path: the cumulative distance D[m][n] is the minimum cumulative distance from A[1] to A[m] and from B[1] to B[n].

[0022] Compared with the prior art, the present invention has the following beneficial effects: the communication time series data anomaly detection method based on the Transformer architecture provided by the present invention adopts the Transformer architecture to make end-to-end predictions to predict the next signal value, and then calculates the distance based on DTW, combining the difference between the predicted value and the true value. This method can not only be based on the distance calculation embedded in the high-dimensional space, but also combines the low-dimensional distance characteristics of the real space; it integrates the distance calculation, sets the dynamic threshold, and dynamically adjusts to achieve accurate anomaly detection effects. In addition, the present invention introduces the relevant technology of time coding in the Transformer architecture, thereby taking into account the periodicity of information and the attenuation of information over long distances. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a flow chart of communication time series data anomaly detection based on Transformer architecture of the present invention;

[0024] Figure 2 It is a schematic diagram of the change of mean and variance of time series data in the field of aviation communication of the present invention over time;

[0025] Figure 3 It is a schematic diagram of the original time series data and the first-order difference time series data of the present invention;

[0026] Figure 4 It is a structural diagram of the time coding of the present invention;

[0027] Figure 5The figure is a schematic diagram of the Transformer prediction result plus the DTW distance calculation of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0029] The communication time series data anomaly detection method based on the Transformer architecture provided by the present invention first analyzes the time series data, and inputs the time series data and the first-order interpolation as sequence data into the Transformer framework respectively. After end-to-end learning using the Transformer architecture, the communication value of the next time node is predicted, and then the DTW (Dynamic Time Warping) calculation method is used to calculate the distance error of the distance sequence, and the anomaly is judged according to the error value.

[0030] See also Figure 1 The communication time series data anomaly detection method based on the Transformer architecture provided by the present invention introduces the first-order difference of the time series data, that is, when processing the original time series data, the first-order difference data is also input into the Transformer architecture as the original data, and the trend characteristics in the time series data are removed, so that the essential characteristics of the time series data can be extracted. The first-order difference of the time series data is:

[0031] ΔY t =Y t -Y t-1

[0032] Here ΔY t Represents the first-order difference value at time t

[0033] Time series data Y in the field of aviation communications t , contains many characteristics of time trend factors (such as seasons, etc.), which can make the mean and variance of the time series change over time, thus leading to non-stationarity. Figure 2 As shown, it can be seen that through differentiation, linear trends can be effectively predicted and the impact of seasonality and other time factors on time series data can be partially reduced.

[0034] The present invention introduces time coding into the Transformer framework, that is, the original position coding in the Transformer is modified into time coding. The time coding is divided into two: absolute time and relative time.

[0035] Absolute time encoding: For each time step t, calculate:

[0036] TE(t,2i)=sin(t / 10000 2i / d )

[0037] TE(t,2i+1)=cos(t / 10000 2i / d )

[0038] Where d is the dimension of the model and i is the index of the dimension

[0039] Relative time encoding: For each time step t, calculate:

[0040] A(t,i)=1+exp(-λ·|ti|)

[0041] Where λ is the decay constant and i is the dimension index

[0042] In the exponential decay process, the decay constant λ is a constant that represents the decay rate of the process. There is a direct mathematical relationship between the decay constant and the half-life. In the aviation communication time series data, if we want to eliminate the influence of the signal that has been too long on the current time signal, we need to set a half-life. Combined with the Transformer architecture, the reasonable half-life for testing is 250-350 time points, preferably 300 time points, then The half-life constant is tested here based on real data from the communications field.

[0043] In relative time coding, the idea of ​​half-life is adopted, that is, when considering the impact of long-distance information on the current moment, not only the periodic impact but also the distance impact should be considered. For information exceeding a certain distance length, not too much information is needed to interfere with the current information.

[0044] The introduction of time coding means that the ability to record the information at the moment in time distribution must be considered, and the ability to interfere with the information at the moment over a long period of time must also be considered. Based on this, the present invention adds two time codings to the architecture, absolute time coding and relative time coding, to obtain two different influencing parameters in training.

[0045] The present invention uses the transformer architecture of the self-attention mechanism to perform end-to-end prediction of time series data. Predict the time series data within a certain length range, and then calculate the error with the real data. The error calculation method adopts the DTW (dynamic time warping) method to perform statistical analysis on the obtained distance and obtain the threshold for judging the error standard. If the error threshold is exceeded, it is judged as abnormal. This method can calculate and predict the time series data in real time and judge the abnormality based on the error.

[0046] The DTW model used in the present invention is an algorithm for calculating the minimum cumulative distance between two sequences. Assume that there are two time series A and B, with lengths of m and n respectively. The following are the basic steps for calculating the DTW distance:

[0047] Initialize the distance matrix: Create a matrix D of size (m+1)×(n+1), where D[i][j] represents the minimum cumulative distance between the first i elements of sequence A and the first j elements of sequence B. Initialize D[0][0]=0 and other D[i][0] and D[0][j] to infinity (or a sufficiently large number), representing boundary conditions.

[0048] Calculate the cumulative distance for each element: for i from 1 to m, j from 1 to n, fill the matrix as follows:

[0049] D[i][j]=cos t(A[i],B[j])+min(D[i-1][j],D[i][j-1],D[i-1][j-1])

[0050] where cost(A[i],B[j]) is usually the Euclidean distance between A[i] and B[j] or another suitable distance metric.

[0051] Find the best path: The cumulative distance D[m][n] is the minimum cumulative distance from A[1] to A[m] and from B[1] to B[n].

[0052] This method also needs to be trained with real data to obtain real error data, and then analyze and set the threshold based on historical error data. DTW can compare the time misalignment of two time series data distributions, which is very suitable for this application.

[0053] The original time series data contains three characteristics: trend characteristics, time characteristics and cycle characteristics.

[0054] from Figure 3 It can be seen that after the first-order difference, the trend characteristics are removed, the data fluctuation distribution is more natural, and it will be more in line with the characteristics of the normal distribution. When the amount of data is large enough, it can be regarded as a normal distribution. The present invention models and predicts the original time series data and the first-order difference time series data respectively, and the prediction result of the first-order difference is used as a correction factor for the prediction result of the original time series data.

[0055] Here, the present invention modifies the strategy of adding position codes in the original Transformer architecture. Because in relative time coding, the present invention introduces an exponential decay strategy of the time interval half-life. Therefore, the present invention multiplies the absolute time code and the relative time code to fuse the time coding features, such as Figure 4 shown.

[0056] Finally, DTW is used to calculate the distance between the time series data predicted by Transformer end-to-end and the actual time series data. The Transformer prediction result of the present invention plus the DTW distance calculation is as follows Figure 5As shown. In time series forecasting, especially long-term forecasting, the forecast output may be slightly misaligned with the actual value in time. DTW can effectively measure the similarity between two time series by flexibly matching time points, even if they are offset on the time axis. In time series forecasting, especially long-term forecasting, the forecast output may be slightly misaligned with the actual value in time. DTW can effectively measure the similarity between two time series by flexibly matching time points, even if they are offset on the time axis. Using traditional error metrics such as mean square error or absolute error may not fully reflect the quality of the forecast, especially when the time series contains complex patterns and nonlinear relationships. DTW provides a way to evaluate the quality of the forecast by the overall series shape.

[0057] The beneficial effects of the present invention are:

[0058] 1. The present invention uses the Transformer architecture to detect anomalies in aviation communication time series data. The architecture can effectively capture long-range dependencies in time series data through the sub-attention mechanism. This is particularly important for aviation communication data, because aircraft data and signals often involve complex temporal dynamics that may span a long period of time;

[0059] 2. The present invention introduces the first-order difference modeling data of time series data, which removes the trend characteristics of the time period and can also eliminate the fluctuations of time series data caused by factors such as seasons, so as to better obtain the natural fluctuation characteristics of the data;

[0060] 3. The present invention modifies the position coding in the Transformer architecture and adopts the method of fusing absolute time coding and relative time coding to obtain the overall time coding. This method can better train two parameters, one is the ability to record long-term information and the other is the ability to interfere with long-term information.

[0061] 4. The present invention introduces the DTW dynamic time warping calculation method in error calculation. While utilizing the dense space distance, it also takes into account the error between real data, compensates for the real data deviation problem caused by the loss of information in mapping the dense space, and is more robust to time series data with time offset.

[0062] Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some modifications and improvements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be based on the definition of the claims.

Claims

1. A communication time series data anomaly detection method based on Transformer architecture, characterized in that: The steps include: S1) Modeling and predicting the original time series data and the first-order difference time series data respectively, and the prediction result of the first-order difference is used as a correction factor for the prediction result of the original time series data; S2) Modify the strategy of adding position codes in the original Transformer architecture, introduce the exponential decay strategy of 1+half-life algorithm in relative time coding, and multiply the absolute time coding and relative time coding to fuse the time coding features; S3) Use DTW to calculate the distance between the time series data predicted by Transformer end-to-end and the actual time series data.

2. The communication time series data anomaly detection method based on Transformer architecture according to claim 1 is characterized in that: In step S1, the time series data and the first-order interpolation are input into the Transformer framework as sequence data to remove the trend features in the time series data and extract the essential features of the time series data; the first-order difference of the time series data is: ΔY t =Y t -Y t-1 ; ΔY t Represents the first-order difference value at time t.

3. The communication time series data anomaly detection method based on Transformer architecture as claimed in claim 1, characterized in that: The step S2 modifies the original position code in the Transformer into a time code, where the time code is divided into an absolute time code and a relative time code; For each time step t, the absolute time code is calculated as follows: TE(t,2i)=sin(t / 10000 2i / d) TE(t,2i+1)=cos(t / 10000 2i / d ) Where d is the dimension of the model and i is the index of the dimension; For each time step t, the relative time encoding is calculated as follows: where λ is the decay constant and i is the index of the dimension.

4. The communication time series data anomaly detection method based on Transformer architecture as claimed in claim 3 is characterized in that: k is the time point of the test half-life.

5. The communication time series data anomaly detection method based on Transformer architecture as claimed in claim 4, characterized in that: The value of k is 250-350.

6. The communication time series data anomaly detection method based on Transformer architecture according to claim 5 is characterized in that: The value of k is 300.

7. The communication time series data anomaly detection method based on Transformer architecture according to claim 1 is characterized in that: Assuming there are two time series A and B, with lengths of m and n respectively, step S3 includes: S31) Initialize the distance matrix: create a matrix D of size (m+1)×(n+1), where D[i][j] represents the minimum cumulative distance between the first i elements of sequence A and the first j elements of sequence B; initialize D[0][0]=0, and the other D[i][0] and D[0][j] are infinite; S32) Calculate the cumulative distance of each element: For i from 1 to m, j from 1 to n, fill the matrix in the following way: D[i][j]=cost(A[i],B[j])+min(D[i-1][j],D[i][j-1],D[i-1][j-1]) where cos t(A[i],B[j]) is the Euclidean distance between A[i] and B[j]; S33) Find the best path: The cumulative distance D[m][n] is the minimum cumulative distance from A[1] to A[m] and from B[1] to B[n].

Images