A network traffic time series prediction method based on STFT and neural network

Abstract

The disclosed network traffic time series prediction method first uses a convolutional layer to fuse information from adjacent timestamps in the time domain. It then performs a frequency mixing operation on the short-time Fourier transform spectrum, using a neural network to fuse information from different frequencies at the same timestamp and capture the mutual influence between the frequencies. Next, a frame mixing operation is performed on the short-time Fourier transform spectrum, using a neural network to synthesize the changing trends of each frequency in the time series to generate the frequency values ​​of future frames. Finally, an inverse short-time Fourier transform is performed to transform the information back to the time domain to obtain the final prediction result.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series prediction, and in particular relates to a network traffic time series prediction method based on short-time Fourier transform (STFT) and neural network. Background Art

[0002] In modern networks, with the tight coupling of control and forwarding capabilities of network devices, the packet processing capacity of individual devices has gradually become a bottleneck for improving overall network efficiency. To address this issue, Software-Defined Networking (SDN) has emerged. SDN deeply decouples the network management layer from the data forwarding layer, enabling controllers to centrally monitor global network topology and traffic information, effectively improving network management and control efficiency. In this context, network traffic forecasting has become increasingly important. Accurate traffic forecasting can better understand and analyze network data, uncover hidden patterns and patterns, and enhance intelligent network management. Traffic forecasting also helps SDN controllers detect abnormal traffic early, determine whether traffic changes are normal, and implement targeted network adjustments and fault prevention. Furthermore, traffic forecasting provides strong support for SDN controllers' policy formulation and resource allocation, enabling them to optimize network resource allocation globally and improve overall network performance. By predicting potential traffic peaks in advance, SDN controllers can proactively adjust network configurations to avoid potential network congestion and improve network Quality of Service (QoS). Accurate traffic forecasting can also reduce the need for network over-configuration, optimize the utilization efficiency of network resources, and thus reduce operating costs.

[0003] At the same time, network traffic data is essentially a type of time series data. Time series data consists of a series of data points arranged in discrete time sequences. These data are very common in real-world applications, encompassing a wide range of fields, including financial risk assessment, energy sustainability, and weather forecasting. Time series data is unique due to its complexity and dynamic nature, including the interweaving of nonlinear patterns and trends that change over time. Therefore, analyzing time series data is of great significance in real-world scenarios and has long been a subject of extensive research.

[0004] Traditional time series analysis methods, such as autoregressive integrated moving average (ARIMA), exponential smoothing, and spectral analysis, play an important role in time series data analysis. These models, based on statistical methods, play an important role in identifying patterns, trends, and seasonal changes in data. However, these traditional models have significant limitations. They perform poorly in capturing complex nonlinear relationships and long-term dependencies, which are very common in real-world time series data. The core problem is that traditional models often assume linearity and stationarity, which greatly limits their ability to adapt to dynamic and changing data streams, thereby restricting their practical application in scenarios where data behavior changes over time. In recent years, the field of time series analysis has made significant progress, and technical methods have shifted from traditional statistical methods to advanced deep learning models. Among deep learning models, the development of Transformer-based models is particularly prominent (see Zhou H, Zhang S, Peng J, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting [C] / / Proceedings of the AAAI conference on artificial intelligence. 2021, 35 (12): 11106-11115.). However, when we apply the Transformer model to time series forecasting, especially in the field of long-term time series forecasting (LTSF), some problems gradually emerge. Although Transformer has demonstrated excellent capabilities in processing natural language processing and computer vision tasks with its multi-head self-attention mechanism, time series forecasting is essentially more concerned with the temporal order and dynamic changes between data points. The self-attention mechanism of Transformer is permutation invariant, which means that it cannot perfectly capture the temporal information in the time series to a certain extent, even if it is compensated by introducing techniques such as position encoding.

[0005] In 2023, DLinear, a simple and effective pure linear model, was introduced. It outperformed Transformer-like models on multiple datasets while having fewer parameters (see Zeng A, Chen M, Zhang L, et al. Are transformers effective for time series forecasting? [C] / / Proceedings of the AAAI conference on artificial intelligence. 2023, 37(9): 11121-11128). DLinear decouples time series into seasonal and trend information in the time domain, achieving excellent results on datasets with good periodicity. However, this pure time domain model lacks a global perspective and cannot capture global patterns. In 2024, the frequency domain-based linear model FITS came out. It takes advantage of the sparsity and compactness of information in the frequency domain, performs Fourier transform on the signal, applies complex-valued linear layers for interpolation in the frequency domain, and then converts it back to the time domain (see Xu Z, Zeng A, Xu Q. FITS: Modeling time series with $10k$ parameters [J]. arXiv preprint arXiv: 2307.03756, 2023). Summary of the Invention

[0006] Most Transformer-based models, such as Pyraformer, Informer, and Autoformer, introduce a large number of parameters that need to be learned due to the stacking of self-attention layers, but their performance is very mediocre. This paper proposes a new lightweight time series prediction method that reduces the number of parameters.

[0007] The existing FITS model performs time series prediction from the perspective of the frequency domain, but the Fourier transform obtains global information, that is, a single point in the frequency domain does not have time information. In the real world, the frequency of a time series usually changes, and this change is reflected in two aspects: first, the amplitude of a certain frequency changes over time; second, the main frequency component of a time signal changes over time. Simply applying the Fourier transform globally cannot extract information about how the frequency changes over time. The prediction method proposed in the present invention is based on short-time Fourier transform (or wavelet transform) and neural network, which can well capture the changing trend of frequency and has good robustness.

[0008] According to one aspect of the present invention, a network traffic time series prediction method is provided, the method comprising:

[0009] Step 1: Input the original time series x to be predicted into a one-dimensional convolution layer and add it to the original time series x. Assuming the convolution kernel length is K, the output x conv for:

[0010] x conv =Conv1d(x,K)+x;

[0011] Step 2: For x conv Perform short-time Fourier transform (STFT) to capture the changing pattern of each frequency point with respect to the time frame. Assume that the number of FFT transformation points per frame of STFT is n, the jump length is h, and the window length is l, then the number of time frames is in Indicates rounding up, the frequency point number is Then output for:

[0012] X STFT =STFT(x conv , n, h, l);

[0013] Step 3: Output X STFT The time-frequency mixing layer is used to fully mix the information of different frequency points and calculate the amplitude and phase of each frequency point in the future. The time-frequency mixing layer includes the frequency mixing layer and the frame mixing layer. The frequency mixing layer uses a simple fully connected linear layer. Mix different frequency information at the same timestamp, and apply frequency mixing operation to all timestamps at the same time. The output for:

[0014] X fmix =f mix (X STFT ,dim=0)

[0015] Among them, dim=0 means that the operation is performed along the frequency dimension. Let the learnable weight matrix of the frequency mixing layer be The frequency mixing operation is then:

[0016] X fmix =W fmix X STFT

[0017] Pass the output through the frame blending layer A different linear mapping is applied to each frequency point. For the i-th frequency point, its linear mapping is expressed as Then output The value at the i-th frequency point for:

[0018]

[0019] in, Represents the value of the frequency mixing layer output at the i-th frequency point, let P i The corresponding learnable weight matrix is The frame mixing layer operation is:

[0020]

[0021] The outputs at all frequency points are stacked and reconstructed to obtain

[0022]

[0023] Among them, Stack means stacking;

[0024] Step 4: Perform inverse short-time Fourier transform (ISTFT) on the output: Get the predicted output

[0025]

[0026] Step 1 can omit the addition operation with the original time series x, and the output is x conv for:

[0027] x conv =Conv1d(x,K).

[0028] In step 3, after the STFT and before the time-frequency mixing layer, the cutoff frequency COF is set. The frequency points exceeding the COF are set to zero and deleted to achieve the purpose of filtering, which can be expressed as:

[0029]

[0030] in, The final result Before the inverse short-time Fourier transform, it is necessary to re-fill with zeros to obtain

[0031] According to another aspect of the present invention, there is provided a computer program product encoded on a computer storage medium, comprising instructions which, when executed by one or more computers, cause the one or more computers to perform the above method. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A network traffic time series prediction method based on short-time Fourier transform and neural network according to an embodiment of the present invention is shown.

[0033] Figure 2 A graph showing network traffic prediction results obtained based on the network traffic time series prediction method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0034] Below, with reference to the accompanying drawings, embodiments of the present disclosure are described. In the accompanying drawings, the same reference numerals are used for identical or corresponding structural elements. In addition, in the description of the following embodiments, repeated descriptions of identical or corresponding structural elements may be omitted. However, even if the description of a structural element is omitted, it does not mean that those structural elements are not included in a certain embodiment.

[0035] Although the process steps, method steps, and algorithms are described in sequence in the flowcharts shown in this specification, these processes, methods, and algorithms can be configured to operate in any suitable order. In other words, the process steps, method steps, and algorithm steps described in various embodiments of the present disclosure do not need to be performed in the order described in the present disclosure.

[0036] Furthermore, even if some steps are described as being performed non-simultaneously, these steps may be performed simultaneously in other embodiments. Furthermore, the examples of the processes described with the accompanying drawings do not mean that the illustrated processes exclude other variations and modifications thereto, nor do they mean that the illustrated processes or any of their steps are essential to one or more of the various embodiments of the present disclosure, nor do they mean that the illustrated processes are preferred.

[0037] Figure 1 A network traffic time series prediction method based on short-time Fourier transform and neural network according to an embodiment of the present invention is shown.

[0038] In context, represents the field of real numbers, represents an n-dimensional real vector space, Represents a real matrix space of dimension m×n. represents the complex field, and Similarly, the corresponding complex vector space and complex matrix space are represented.

[0039] For a time series with C channels, assuming that the length of the time series is L and the length of time to be predicted is H, the task can be described as in, is the predicted value of H time stamps in the future from time t+1, is the predicted value of t and its previous L timestamps, That is, one by arrive 's mapping (→ represents mapping, the same applies below).

[0040] In particular, we adopt a channel-independent perspective to model the time series prediction problem, that is, using the same mapping to solve the prediction problem of each channel separately. Specifically, our goal is to find a mapping Where (i) represents the i-th channel. For the sake of convenience, we will record the time series of each channel separately as The predicted sequence output is recorded as

[0041] The network traffic time series prediction method is as follows:

[0042] The first step is the information fusion convolution layer. The original time series x to be predicted is input into a one-dimensional convolution layer and added to the original time series x. This method smoothes the input signal on the one hand, and on the other hand allows the value of each timestamp to fully integrate the information of the surrounding timestamps. Assuming the convolution kernel length is K, its output x conv can be expressed as:

[0043] x conv =Conv1d(x, K)+x

[0044] The parameters of the convolution kernel can be obtained by training.

[0045] The residual connection here is optional, that is, the summation with the input can be removed, expressed as:

[0046] x conv =Conv1d(x, K)

[0047] The second step is to conv Perform short-time Fourier transform (STFT) to capture the changing pattern of each frequency point with respect to the time frame. Assume that the number of FFT transformation points per frame of STFT is n, the hop length is h, and the window length is l, then the number of time frames is in Indicates rounding up. The frequency points are (N is usually an even number), then the output It can be expressed as:

[0048] X STFT =STFT(x conv , n, h, l)

[0049] The third step is to pass the output through the time-frequency mixing layer to fully mix the information of different frequency points and calculate the amplitude and phase of each frequency point in the future (i.e., the STFT spectrum of the output sequence). Specifically, the time-frequency mixing layer is divided into two parts: the frequency mixing layer and the frame mixing layer. The frequency mixing layer uses a simple fully connected linear layer. Mix different frequency information at the same timestamp. We apply the frequency mixing operation to all timestamps at the same time, and the output is can be expressed as:

[0050] X fmix =f mix (X STFT ,dim=0)

[0051] Among them, dim=0 means that the operation is performed along the frequency dimension. Furthermore, let the learnable weight matrix of the frequency mixing layer be The frequency mixing operation can then be expressed as:

[0052] X fmix =W fmix X STFT

[0053] Next, the output is passed through the frame blending layer Specifically, a different linear mapping is applied to each frequency point. For the i-th frequency point, its linear mapping is expressed as Then output The value at the i-th frequency point It can be expressed as:

[0054]

[0055] in, Represents the value of the frequency mixing layer output at the i-th frequency point. Furthermore, let P i The corresponding learnable weight matrix is The frame mixing layer operation can be expressed as:

[0056]

[0057] The outputs at all frequency points are stacked and reconstructed to obtain

[0058]

[0059] Stack means stacking.

[0060] Optionally, after the STFT and before the time-frequency mixing layer, a cutoff frequency COF can be set to achieve the purpose of filtering by setting the frequencies exceeding the eOF to zero and deleting them. It can be expressed as:

[0061]

[0062] Here Correspondingly, the final Before the inverse short-time Fourier transform, it is necessary to re-fill with zeros to obtain Note that the shapes of the various learnable weight matrices in the middle also need to be adjusted accordingly. For example, The shape of is adjusted to COF×COF, then the subsequent stacking operation will become

[0063] In addition, the structure of the frequency mixing and frame mixing layers can be changed. For example, here it is a single linear layer without bias. In actual use, bias items can be added according to needs or replaced with multi-layer fully connected layers with activation layers or other neural network structures.

[0064] The fourth step is to perform inverse short-time Fourier transform on the output Get the predicted output

[0065]

[0066] Figure 2 The results of network traffic prediction using the method provided by the embodiment of the present invention are shown. Our model can accurately capture the periodic changes and peak and valley patterns of network traffic using a small number of parameters.

[0067] The network traffic time series prediction method disclosed in this disclosure uses a convolutional layer to fuse information from adjacent timestamps in the time domain. A frequency blending operation is performed on the short-time Fourier transform spectrum. A neural network is used to fuse information from different frequencies at the same timestamp to capture the mutual influence between the frequencies. Furthermore, a frame blending operation is performed on the short-time Fourier transform spectrum. The neural network uses this to synthesize the changing trends of each frequency point in the time series and generate the frequency value of the future frame.

[0068] The network traffic time series prediction method disclosed in the present invention further reduces the number of parameters by performing a filtering operation on the STFT spectrum.

[0069] Compared with DLinear and many Transformer-type models, the network traffic time series prediction method disclosed in this paper greatly reduces the number of parameters. The experimental results after simulation are as follows:

[0070] Model Parameter quantity Informer 12.53M <![CDATA[ Autoformer ]]> 12.22M <![CDATA[ FEDformer ]]> 17.98M <![CDATA[ FiLM ]]> 12.22M <![CDATA[ PatchTST ]]> 6.31M <![CDATA[ DLinear ]]> 485.3K This method 50-100K

[0071] In addition to network traffic time series, this method can also be applied to other time series data. This method can capture the dynamic change trend of frequency over time, and surpasses DLinear, FITS and a number of Transformer-type models in terms of prediction effect. We use MSE (Mean Squared Error Loss) to measure the performance of the model. We also use the ETTh1 and ETTh2 datasets (which are public datasets used to predict the status of power network nodes, which can be obtained from https: / / github.com / zhouhaoyi / ETDataset) to evaluate the effect of the model. The results show that this method surpasses the existing baseline model in all indicators and has strong prediction capabilities.

[0072]

[0073] The embodiments of the present disclosure can be written as computer programs and can be implemented in general-use digital computers that execute the programs using a computer readable recording medium.

[0074] In this state, the medium can continuously store programs that can be executed by the computer, or can temporarily store programs for execution or downloading. In addition, the medium can be various recording devices or storage devices in combination with single or multiple hardware, not limited to media that are directly connected to the computer system, and can exist in a distributed manner on the network. Examples of media include magnetic storage media (such as floppy disks or hard disks) configured to store program instructions, optical recording media (such as CD-ROMs or DVDs), magneto-optical media (such as floppy disks), and ROM, RAM, flash memory, etc. In addition, examples of other media can include: application stores for distributing application programs, sites for providing or distributing various other software, and recording media or storage media managed at a server.

[0075] Although the present disclosure has been particularly shown and described with reference to preferred embodiments using specific terminology, the embodiments and terminology should be considered in a descriptive sense only and not for purposes of limitation. Therefore, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present disclosure as defined by the appended claims.

Claims

1. A network traffic time series prediction method, the method comprising: Step 1: Input the original time series x to be predicted into a one-dimensional convolution layer and add it to the original time series x. Assuming the convolution kernel length is K, the output x conv for: x conv =Conv1d(x,K)+x; Step 2: For x conv Perform short-time Fourier transform (STFT) to capture the changing pattern of each frequency point with respect to the time frame. Assume that the number of FFT transformation points per frame of STFT is n, the jump length is h, and the window length is l, then the number of time frames is in Indicates rounding up, the frequency point number is Then output for: X STFT =STFT(x conv ,n,h,l); Step 3: Output X STFT The time-frequency mixing layer is used to fully mix the information of different frequency points and calculate the amplitude and phase of each frequency point in the future. The time-frequency mixing layer includes the frequency mixing layer and the frame mixing layer. The frequency mixing layer uses a simple fully connected linear layer. Mix different frequency information at the same timestamp, and apply frequency mixing operation to all timestamps at the same time. The output for: Among them, dim=0 means that the operation is performed along the frequency dimension. Let the learnable weight matrix of the frequency mixing layer be The frequency mixing operation is then: Pass the output through the frame mixing layer P: A different linear mapping is applied to each frequency point. For the i-th frequency point, its linear mapping is expressed as P i : Then output The value at the i-th frequency point for: in, Represents the value of the frequency mixing layer output at the i-th frequency point, let P i The corresponding learnable weight matrix is The frame mixing layer operation is: The outputs at all frequency points are stacked and reconstructed to obtain Among them, Stack means stacking; Step 4: Perform inverse short-time Fourier transform (ISTFT) on the output: Get the predicted output 2. The method according to claim 1, characterized in that Step 1 omits the addition operation with the original time series x, and the output is x conv for: x conv =Conv1d(x,K)。 3. The method according to claim 1, characterized in that In step 3, after STFT and before time-frequency mixing MLP, the cutoff frequency COF is set. The frequency points exceeding COF are set to zero and deleted to achieve the purpose of filtering, which can be expressed as: in, The final result Before the inverse short-time Fourier transform, it is necessary to re-fill with zeros to obtain 4. A non-transitory computer-readable medium having instructions recorded thereon, which, when executed by one or more computers, cause the one or more computers to perform the method according to any one of claims 1 to 3.

Images