Network traffic time sequence prediction method based on STFT and neural network

Abstract

According to the network traffic time sequence prediction method disclosed by the invention, firstly, a convolutional layer is used for carrying out information fusion of adjacent timestamps on a time domain, and then frequency mixing operation is carried out on a short-time Fourier transform spectrum, namely, different frequency point information on the same timestamp is fused by using a neural network, and mutual influence between frequency points is captured. Performing frame mixing operation on the short-time Fourier transform spectrum, namely generating a frequency point value of a future frame by using a change trend of each frequency point in a neural network synthesis time sequence; and finally, carrying out inverse short-time Fourier transform to transform back to a time domain to obtain a final prediction result.

Description

Technical Field

[0001] The invention belongs to the technical field of time series prediction, and in particular relates to a network traffic time series prediction method based on short-time Fourier transform (STFT) and neural network. Background Art

[0002] In modern networks, as the control and forwarding capabilities of network devices are tightly coupled, the packet processing capabilities of a single device have gradually become a bottleneck for improving the overall efficiency of the network. To solve this problem, Software-Defined Network (SDN) came into being. SDN deeply decouples the management layer from the data forwarding layer of the network, allowing the controller to centrally grasp the global network topology information and traffic information, thereby effectively improving the management and control efficiency of the network. In this context, the importance of network traffic prediction has become increasingly prominent. Accurate traffic prediction can better understand and analyze network data, discover hidden rules and patterns, and improve the level of intelligent management of the network. At the same time, traffic prediction can also help SDN controllers detect abnormal traffic early, determine whether traffic changes are normal, and then make targeted network adjustments and fault prevention. In addition, traffic prediction also provides strong support for the policy formulation and resource allocation of SDN controllers, enabling them to optimize network resource configuration on a global scale and improve the overall performance of the network. By predicting possible traffic peaks in advance, SDN controllers can pre-adjust network configuration to avoid potential network congestion and improve the quality of service (QoS) of the network. Accurate traffic forecasting can also reduce the need for network over-configuration, optimize the utilization efficiency of network resources, and thus reduce operating costs.

[0003] At the same time, network traffic data is essentially a kind of time series data. Time series data consists of a series of data points arranged in discrete time order, which is very common in real-world applications, covering many fields such as financial risk assessment, energy sustainability, weather forecasting, etc. Time series data is unique due to its complexity and dynamics, which include the interweaving of nonlinear patterns and trends that change over time. Therefore, analyzing time series data is of great significance in real-world scenarios and has long been a topic of extensive research.

[0004] Traditional time series analysis methods such as autoregressive integrated moving average (ARIMA), exponential smoothing, and spectral analysis have played an important role in time series data analysis. These models are based on statistical methods and have played an important role in identifying patterns, trends, and seasonal changes in data. However, these traditional models have significant limitations. They perform poorly in capturing complex nonlinear relationships and long-term dependencies, which are very common in real-world time series data. The core problem is that traditional models usually assume linearity and stationarity, which greatly limits their ability to adapt to dynamic and changing data streams, thereby limiting their practical application in scenarios where data behavior changes over time. In recent years, the field of time series analysis has made significant progress, and technical methods have shifted from traditional statistical methods to advanced deep learning models. Among deep learning models, the development of Transformer-based models is particularly prominent (see Zhou H, Zhang S, Peng J, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting [C] / / Proceedings of the AAAI conference on artificial intelligence. 2021, 35 (12): 11106-11115.). However, when we apply the Transformer model to time series prediction, especially in the field of long-term time series prediction (LTSF), some problems gradually emerge. Although Transformer has demonstrated excellent capabilities in processing natural language processing and computer vision tasks with its multi-head self-attention mechanism, time series prediction essentially pays more attention to the temporal order and dynamic changes between data points. The self-attention mechanism of Transformer is permutation invariant, which means that it cannot perfectly capture the temporal information in the time series to a certain extent, even if it is compensated by introducing techniques such as position encoding.

[0005] In 2023, DLinear, a simple and effective pure linear model, was launched. It beat the Transformer model in multiple data sets and had fewer parameters (see Zeng A, Chen M, Zhang L, et al. Are transformers effective for time series forecasting? [C] / / Proceedings of the AAAI conference on artificial intelligence. 2023, 37(9): 11121-11128). DLinear decouples time series into seasonal information and trend information in the time domain, and achieves good results on data sets with good periodicity. However, this pure time domain model does not have a global perspective and cannot capture global laws. In 2024, the frequency-domain based linear model FITS came out. It takes advantage of the sparsity and compactness of information in the frequency domain, performs Fourier transform on the signal, applies complex-valued linear layers for interpolation in the frequency domain, and then converts it back to the time domain (see Xu Z, Zeng A, Xu Q. FITS: Modeling time series with $10k$ parameters [J]. arXiv preprint arXiv: 2307.03756, 2023). Summary of the invention

[0006] Most Transformer-based models, such as Pyraformer, Informer, Autoformer, etc., introduce a large number of parameters that need to be learned due to the stacking of self-attention layers, but their performance is very general. The present invention proposes a new lightweight time series prediction method to reduce the number of parameters.

[0007] The existing FITS model performs time series prediction from the perspective of the frequency domain, but the Fourier transform obtains global information, that is, a single point in the frequency domain does not have time information. In the real world, the frequency of a time series usually changes, and this change is reflected in two aspects: first, the amplitude of a certain frequency changes over time; second, the main frequency component of a time signal changes over time. Simply applying the Fourier transform globally cannot extract information about how the frequency changes over time. The prediction method proposed in the present invention is based on short-time Fourier transform (or wavelet transform) and neural network, which can well capture the changing trend of frequency and has good robustness.

[0008] According to one aspect of the present invention, a network traffic time series prediction method is provided, the method comprising:

[0009] Step 1: Input the original time series x to be predicted into a one-dimensional convolution layer and add it to the original time series x. Assuming the convolution kernel length is K, the output x conv for:

[0010] x conv =Conv1d(x,K)+x;

[0011] Step 2: x conv Perform short-time Fourier transform STFT to capture the changing law of each frequency point with respect to the time frame. Assume that the number of FFT transformation points per frame of STFT is n, the jump step length is h, and the window length is l, then the number of time frames is in Indicates rounding up, the frequency point number is Then output for:

[0012] X STFT =STFT(x conv , n, h, l);

[0013] Step 3: Output X STFT The time-frequency mixing layer is used to fully mix the information of different frequency points and calculate the amplitude and phase of each frequency point in the future. The time-frequency mixing layer includes the frequency mixing layer and the frame mixing layer. The frequency mixing layer uses a simple fully connected linear layer. Mix different frequency information under the same timestamp, and apply frequency mixing operation to all timestamps at the same time. The output for:

[0014] X fmix =f mix (X STFT ,dim=0)

[0015] Among them, dim = 0 means that the operation is performed along the frequency dimension. Let the learnable weight matrix of the frequency mixing layer be The frequency mixing operation is then:

[0016] X fmix =W fmix X STFT

[0017] Pass the output through the frame blending layer A different linear mapping is applied to each frequency point. For the i-th frequency point, its linear mapping is expressed as Then output The value at the i-th frequency point for:

[0018]

[0019] in, represents the value of the frequency mixing layer output at the i-th frequency point, let P i The corresponding learnable weight matrix is The frame mixing layer operation is:

[0020]

[0021] The outputs at all frequency points are stacked and reconstructed to obtain

[0022]

[0023] Among them, Stack means stacking;

[0024] Step 4: Perform inverse short-time Fourier transform (ISTFT) on the output: Get the predicted output

[0025]

[0026] Step 1 can omit the operation of adding to the original time series x, and the output x conv for:

[0027] x conv =Conv1d(x,K).

[0028] In step 3, after STFT and before the time-frequency mixing layer, the cutoff frequency COF is set, and the frequency points exceeding COF are set to zero and deleted to achieve the purpose of filtering, which is expressed as:

[0029]

[0030] in, The final result Before the inverse short-time Fourier transform, it is necessary to re-fill with zeros to obtain

[0031] According to another aspect of the present invention, there is provided a computer program product encoded on a computer storage medium, comprising instructions which, when executed by one or more computers, will cause the one or more computers to perform the above method. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A network traffic time series prediction method based on short-time Fourier transform and neural network according to an embodiment of the present invention is shown.

[0033] Figure 2 A graph showing network traffic prediction results obtained based on a network traffic time series prediction method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0034] Below, with reference to the accompanying drawings, embodiments of the present disclosure are described. In the accompanying drawings, the same reference numerals are used for identical or corresponding structural elements. In addition, in the description of the following embodiments, repeated descriptions of identical or corresponding structural elements may be omitted. However, even if the description of structural elements is omitted, it does not mean that those structural elements are not included in a certain embodiment.

[0035] Although the process steps, method steps, and algorithms are described in order in the flowcharts illustrated in this specification, these processes, methods, and algorithms can be configured to operate in any suitable order. In other words, the process steps, method steps, and algorithm steps described in various embodiments of the present disclosure do not need to be performed in the order described in the present disclosure.

[0036] In addition, even if some steps are described as being performed non-simultaneously, these steps may be performed simultaneously in other embodiments. In addition, the examples of the processes described by the drawings do not mean that the illustrated processes exclude other changes and modifications thereto, and do not mean that the illustrated processes or any one of their steps are essential to one or more of the various embodiments of the present disclosure, and do not mean that the illustrated processes are preferred.

[0037] Figure 1 A network traffic time series prediction method based on short-time Fourier transform and neural network according to an embodiment of the present invention is shown.

[0038] In context, represents the field of real numbers, represents the n-dimensional real vector space, Represents a real matrix space of dimension m×n. represents the complex domain, and Similarly, the corresponding complex vector space and complex matrix space are represented.

[0039] For a time series with C channels, assuming that the length of the time series is L and the time length to be predicted is H, the task can be described as in, is the predicted value of H time stamps in the future from time t+1, is the predicted value of t and its previous L timestamps, That is, one arrive 's mapping (→ represents mapping, the same applies below).

[0040] In particular, we adopt a channel-independent perspective to model the time series prediction problem, that is, using the same mapping to solve the prediction problem of each channel separately. Specifically, our goal is to find a mapping Where (i) represents the i-th channel. For the convenience of description, we will record the time series of each channel separately as The predicted sequence output is recorded as

[0041] The network traffic time series prediction method is as follows:

[0042] The first step is the information fusion convolution layer. The original time series x to be predicted is input into a one-dimensional convolution layer and added to the original time series x. This method smoothes the input signal on the one hand, and allows the value of each timestamp to fully integrate the information of the surrounding timestamps on the other hand. Assuming the length of the convolution kernel is K, its output x conv can be expressed as:

[0043] x conv =Conv1d(x, K)+x

[0044] The parameters of the convolution kernel can be obtained by training.

[0045] Here the residual connection is optional, that is, the summation with the input can be removed, expressed as:

[0046] x conv =Conv1d(x, K)

[0047] The second step is to conv Perform short-time Fourier transform (STFT) to capture the changing pattern of each frequency point with respect to the time frame. Assume that the number of FFT transformation points per frame of STFT is n, the hop length is h, and the window length is l, then the number of time frames is in Indicates rounding up. The frequency points are (N is usually an even number), then the output It can be expressed as:

[0048] X STFT =STFT(x conv , n, h, l)

[0049] The third step is to pass the output through the time-frequency mixing layer to fully mix the information of different frequency points and calculate the amplitude and phase of each frequency point in the future (that is, the STFT spectrum of the output sequence). Specifically, the time-frequency mixing layer is divided into two parts: the frequency mixing layer and the frame mixing layer. The frequency mixing layer uses a simple fully connected linear layer Mix different frequency information at the same timestamp. We apply the frequency mixing operation to all timestamps at the same time, and its output is can be expressed as:

[0050] X fmix =f mix (X STFT ,dim=0)

[0051] Among them, dim = 0 means that the operation is performed along the frequency dimension. Furthermore, let the learnable weight matrix of the frequency mixing layer be Then the frequency mixing operation can be expressed as:

[0052] X fmix =W fmix X STFT

[0053] Next, the output is passed through the frame blending layer Specifically, a different linear mapping is applied to each frequency point. For the i-th frequency point, its linear mapping is expressed as Then output The value at the i-th frequency point It can be expressed as:

[0054]

[0055] in, represents the value of the frequency mixing layer output at the i-th frequency point. Furthermore, let P i The corresponding learnable weight matrix is Then the frame mixing layer operation can be expressed as:

[0056]

[0057] The outputs at all frequency points are stacked and reconstructed to obtain

[0058]

[0059] Among them, Stack means stacking.

[0060] Optionally, after STFT and before the time-frequency mixing layer, a cutoff frequency COF can be set to achieve the purpose of filtering by setting the frequency points exceeding eOF to zero and deleting them, which can be expressed as:

[0061]

[0062] Here Correspondingly, the final Before the inverse short-time Fourier transform, it is necessary to re-fill with zeros to obtain Note that the shapes of the various learnable weight matrices in the middle also need to be adjusted accordingly. For example, The shape of is adjusted to COF×COF, then the subsequent stacking operation will become

[0063] In addition, the structure of the frequency mixing and frame mixing layers can be changed. For example, here is a single linear layer without bias. In actual use, bias items can be added according to needs or replaced with multi-layer fully connected layers with activation layers or other neural network structures.

[0064] The fourth step is to perform an inverse short-time Fourier transform on the output Get the predicted output

[0065]

[0066] Figure 2 The results of network traffic prediction using the method provided by the embodiment of the present invention are shown. Our model can accurately capture the periodic changes and peak and valley patterns of network traffic using a small number of parameters.

[0067] The network traffic time series prediction method disclosed in the present invention uses a convolutional layer to fuse information of adjacent time stamps in the time domain. A frequency mixing operation is performed on the short-time Fourier transform spectrum. A neural network is used to fuse information of different frequency points on the same time stamp to capture the mutual influence between the frequency points. In addition, a frame mixing operation is performed on the short-time Fourier transform spectrum. A neural network is used to generate the frequency point value of the future frame by synthesizing the change trend of each frequency point in the time series.

[0068] The network traffic time series prediction method disclosed in the present invention further reduces the number of parameters by performing filtering operations on the STFT spectrum.

[0069] Compared with DLinear and many Transformer models, the network traffic time series prediction method disclosed in this paper greatly reduces the number of parameters. The simulated experimental results are as follows:

[0070] Model Parameter quantity Informer 12.53M <![CDATA[ Autoformer ]]> 12.22M <![CDATA[ FEDformer ]]> 17.98M <![CDATA[ FiLM ]]> 12.22M <![CDATA[ PatchTST ]]> 6.31M <![CDATA[ DLinear ]]> 485.3K This method 50-100K

[0071] In addition to network traffic time series, this method can also be applied to other time series data. This method can capture the dynamic change trend of frequency over time, and surpasses DLinear, FITS and a number of Transformer-type models in prediction effect. We use MSE (Mean Squared Error Loss) to measure the performance of the model. And use the two data sets ETTh1 and ETTh2 (which are public data sets used to predict the status of power network nodes, the data set can be obtained from https: / / github.com / zhouhaoyi / ETDataset) to evaluate the effect of the model. The results show that this method surpasses the existing baseline model in all indicators and has strong prediction ability.

[0072]

[0073] The embodiments of the present disclosure can be written as computer programs and can be implemented in general-use digital computers that execute the programs using a computer readable recording medium.

[0074] In this state, the medium can continuously store programs that can be executed by the computer, or can temporarily store programs for execution or downloading. In addition, the medium can be a variety of recording devices or storage devices in combination with single or multiple hardware, not limited to media that are directly connected to the computer system, and can exist in a distributed manner on the network. Examples of media include magnetic storage media (such as floppy disks or hard disks) configured to store program instructions, optical recording media (such as CD-ROMs or DVDs), magneto-optical media (such as floppy disks), and ROM, RAM, flash memory, etc. In addition, examples of other media may include: application stores for distributing applications, sites for providing or distributing various other software, and recording media or storage media managed at a server.

[0075] Although the present disclosure has been specifically shown and described with reference to preferred embodiments using specific terms, the embodiments and terms should be considered in a descriptive sense only and not for the purpose of limitation. Therefore, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present disclosure as defined by the appended claims.

Claims

1. A network traffic time series prediction method, the method comprising: Step 1: Input the original time series x to be predicted into a one-dimensional convolution layer and add it to the original time series x. Assuming the convolution kernel length is K, the output x conv for: x conv =Conv1d(x,K)+x; Step 2: x conv Perform short-time Fourier transform STFT to capture the changing law of each frequency point with respect to the time frame. Assume that the number of FFT transformation points per frame of STFT is n, the jump step length is h, and the window length is l, then the number of time frames is in Indicates rounding up, the frequency point number is Then output for: X STFT =STFT(x conv ,n,h,l); Step 3: Output X STFT The time-frequency mixing layer is used to fully mix the information of different frequency points and calculate the amplitude and phase of each frequency point in the future. The time-frequency mixing layer includes the frequency mixing layer and the frame mixing layer. The frequency mixing layer uses a simple fully connected linear layer. Mix different frequency information under the same timestamp, and apply frequency mixing operation to all timestamps at the same time. The output for: Among them, dim = 0 means that the operation is performed along the frequency dimension. Let the learnable weight matrix of the frequency mixing layer be The frequency mixing operation is then: Pass the output through the frame mixing layer P: A different linear mapping is applied to each frequency point. For the i-th frequency point, its linear mapping is expressed as P i : Then output The value at the i-th frequency point for: in, represents the value of the frequency mixing layer output at the i-th frequency point, let P i The corresponding learnable weight matrix is The frame mixing layer operation is: The outputs at all frequency points are stacked and reconstructed to obtain Among them, Stack means stacking; Step 4: Perform inverse short-time Fourier transform (ISTFT) on the output: Get the predicted output 2. The method according to claim 1, characterized in that Step 1 omits the addition operation to the original time series x, and the output is x conv for: x conv =Conv1d(x,K)。 3. The method according to claim 1, characterized in that In step 3, after STFT and before time-frequency hybrid MLP, the cutoff frequency COF is set, and the frequency points exceeding COF are set to zero and deleted to achieve the purpose of filtering, which is expressed as: in, The final result Before the inverse short-time Fourier transform, it is necessary to re-fill with zeros to obtain 4. A non-transitory computer-readable medium having instructions recorded thereon, which, when executed by one or more computers, will cause the one or more computers to execute the method according to any one of claims 1 to 3.

Images