Network traffic prediction method based on local downsampling and sparse frequency grouping

Abstract

The invention discloses a network flow prediction method based on local downsampling and sparse frequency grouping. The method comprises the following steps: sequentially segmenting an input sequence into P regions; performing down-sampling operation in the region; carrying out real sequence fast Fourier transform on a downsampling result; executing a frequency truncation operation on the result; executing sparse frequency grouping and sparse frequency mixing operation; executing cross-regional frequency point prediction; zero padding and inverse real sequence fast Fourier transform irFFT are executed; and performing a sequence rearrangement operation to obtain an output sequence.

Description

Technical Field

[0001] The invention belongs to the technical field of time series prediction, and in particular relates to a network traffic prediction method based on local downsampling and sparse frequency grouping. Background Art

[0002] Network traffic forecasting plays an important role in modern communication networks, especially in the context of growing network load and increasing data transmission demand. Whether it is to avoid network congestion, reasonably allocate bandwidth, or provide support for traffic anomaly detection and security protection, predicting future network traffic changes is crucial. Network administrators and operators rely on this forecasting capability to ensure efficient use of network resources while maintaining network service quality and security. To achieve these goals, network traffic forecasting often relies on a powerful data analysis tool - time series forecasting technology.

[0003] Time series prediction technology is the theoretical basis of network traffic prediction, because network traffic has significant time dependence. Network data shows complex fluctuations, trends and seasonal changes in different time periods. Therefore, using time series prediction technology to capture and analyze these change patterns can effectively improve the accuracy of network traffic prediction.

[0004] Time series prediction is not only used in network traffic prediction, it is widely used in many fields, such as financial markets, weather forecasts, energy demand forecasts, etc. Time series data has a key feature, that is, the order of data points is very important, and there is often correlation between adjacent data. By deeply exploring the potential patterns in the time series, the prediction model can capture trends and cyclical changes, thereby predicting future changes. This is very consistent with the needs of network traffic prediction, because network traffic is also affected by various cyclical, seasonal and sudden events.

[0005] In network traffic forecasting, classic time series forecasting models such as ARIMA and SARIMA have been widely used. They can provide accurate traffic forecasts for network management by modeling trends and fluctuations in historical data. However, as the scale of the network continues to expand, traffic patterns become more and more complex, and traditional time series models have shown certain limitations when processing large-scale, nonlinear, and multi-dimensional network data.

[0006] To solve this problem, machine learning and deep learning technologies have gradually become a new trend in time series prediction in recent years. Recurrent neural networks (RNNs) and their variants, such as long short-term memory networks (LSTMs) and gated recurrent units (GRUs), have been introduced into network traffic prediction due to their advantages in processing long-term dependent data. Through these models, the prediction system can capture complex patterns in network traffic and predict future traffic changes. The more advanced Transformer model, which first emerged in the field of natural language processing, is now also used in time series prediction. Its ability to process data in parallel makes it particularly effective in processing large-scale network traffic. In addition, the self-attention mechanism in the Transformer can effectively extract the relationship between different parts of a long sequence, which is particularly suitable for long sequence prediction. Summary of the invention

[0007] On the one hand, due to the inherent characteristics of the self-attention mechanism of Transformer-type models, it is easy to blur the sequential information of time series sampling points; on the other hand, a number of nonlinear models headed by Transformer-type models have extremely high computational and storage costs due to their huge neural network architecture; thirdly, these models are often prone to overfitting due to their complex architecture, and the trained models are not easy to migrate to other data distributions; fourthly, these models make extensive use of neural networks but lack corresponding interpretability, which makes the models completely black boxes.

[0008] To this end, we propose a network traffic prediction method based on local downsampling and sparse frequency grouping for the prediction of network traffic and other time series.

[0009] The above method has the following advantages:

[0010] The number of parameters is extremely low, the amount of computation is extremely low, and the inference and training speed is extremely fast. Compared with Transformer and convolution models, the method we proposed can reduce the number of parameters by up to 7 orders of magnitude and the storage overhead by up to 2 orders of magnitude. At the same time, the prediction performance of this method is the best.

[0011] It should have a solid theoretical foundation and be explainable to a certain extent.

[0012] This method has certain universality and transferability. After being trained on one dataset, our method can be seamlessly transferred to another dataset and achieve good results. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A network traffic prediction method based on local downsampling and sparse frequency grouping according to an embodiment of the present invention is shown.

[0014] Figure 2 A schematic diagram of the weight matrix after learning is shown. DETAILED DESCRIPTION

[0015] Below, with reference to the accompanying drawings, embodiments of the present disclosure are described. In the accompanying drawings, the same reference numerals are used for identical or corresponding structural elements. In addition, in the description of the following embodiments, repeated descriptions of identical or corresponding structural elements may be omitted. However, even if the description of structural elements is omitted, it does not mean that those structural elements are not included in a certain embodiment.

[0016] Although the process steps, method steps, and algorithms are described in order in the flowcharts illustrated in this specification, these processes, methods, and algorithms can be configured to operate in any suitable order. In other words, the process steps, method steps, and algorithm steps described in various embodiments of the present disclosure do not need to be performed in the order described in the present disclosure.

[0017] In addition, even if some steps are described as being performed non-simultaneously, these steps may be performed simultaneously in other embodiments. In addition, the examples of the processes described by the drawings do not mean that the illustrated processes exclude other changes and modifications thereto, and do not mean that the illustrated processes or any one of their steps are essential to one or more of the various embodiments of the present disclosure, and do not mean that the illustrated processes are preferred.

[0018] Figure 1 A network traffic prediction method based on local downsampling and sparse frequency grouping according to an embodiment of the present invention is shown.

[0019] The network traffic prediction task can be abstracted as a time series prediction problem. Specifically, given an input sequence of length L (in, represents an L-dimensional real number space, and the subsequent notation is the same). Our goal is to obtain a prediction sequence of output with a length of H. To this end, the method described in the embodiment of the present invention includes 8 steps:

[0020] Step S1, divide the input sequence into regions, and the number of regions is determined by the hyperparameter P. Specifically, for an input sequence x of length L, it is sequentially divided into P regions (Patch), and the length of each region is l = L / P. In this way, the original input sequence After region division, it becomes a matrix form (in, represents a P×l real number matrix space, and the subsequent notation is the same), where the i-th row represents the l sequence values ​​corresponding to the i-th region. The process can be expressed as:

[0021] X=Patch(x)

[0022] Patch represents the operation of area division.

[0023] Step S2, perform intra-regional downsampling operation on the sequence matrix form X obtained in step S1 after the region division. According to the Nyquist sampling theorem, as long as the sampling frequency of a time series is greater than twice its maximum frequency, the original signal can be restored without distortion. Therefore, we set the downsampling factor M based on experience, and the downsampling operation downsamples each row of X, and M subsequences are obtained for each row, and the length of each subsequence is c=l / M. The process can be mathematically expressed as:

[0024]

[0025] in, is the result after downsampling, where Represents a The third-order real tensor space of , the following notations are similar. Downsample(·) represents the downsampling operation, which is actually equivalent to a mapping:

[0026] For intra-region downsampling, when M=1, it is equivalent to no operation.

[0027] The downsampling operation in step S2 converts the complete input sequence into several subsequences so as to share parameters among these subsequences.

[0028] Step S3, for the down-sampling result x, along the subsequence dimension, rFFT (real Fast Fourier Transform) is performed on each subsequence in each region. The sequence value becomes after rFFT sequence values, where floor(·) represents rounding down. This step can be expressed as:

[0029]

[0030] in, is the result after rFFT, Represents a complex set.

[0031] The subsequence rFFT in step S3 performs rFFT operations on the subsequences in parallel (ie, simultaneously) to obtain the frequency spectrum of the subsequences. In step S3, rFFT can be replaced by FFT operation.

[0032] Step S4, frequency truncate the result obtained in S3, i.e., retain the first f c The conversion frequency points can be expressed as:

[0033]

[0034] in, is the result after frequency truncation, and Cut(·) represents the frequency truncation operation. It can be seen that the frequency truncation operation is the result of parallel operation on PM subsequences, and only the first f of each subsequence is retained. c frequency points.

[0035] The frequency truncation operation can discard redundant frequency value information (because high frequencies are mainly noise). You can also skip the frequency truncation operation, in which case f c =f.

[0036] Step S5 is a sparse frequency mixing operation based on sparse frequency grouping. This step fuses the different frequency values ​​of each sequence in parallel through a simple linear layer to facilitate the next prediction operation. The motivation for sparse frequency grouping is very simple. As shown in the figure below, when the sparse frequency grouping strategy is not used, the weight matrix learned by the model presents the pattern in the figure, that is, the same frequency point is most affected by itself and the surrounding frequency points, and is less affected by distant frequency points. Based on this, we propose a sparse frequency grouping method as shown in the figure. First, the frequencies are grouped and different linear layers are applied to the groups. This approach can transform the original dense weight matrix into a sparse weight matrix composed of sparse diagonal blocks, and forward and backward propagation can be performed in parallel in actual training. Specifically, we follow the The frequency dimension in the sequence is grouped, and the f c The frequency points will be divided into K groups, forming ( Represents a shape The fourth-order complex tensor space of ), where the i-th group contains PM with The subsequence of frequency points is recorded as:

[0037]

[0038] Among them, [:,i,:,:] represents the index of the corresponding dimension, and : means that the dimension remains unchanged.

[0039] In the frequency sparse grouping operation, when K=1, it is equivalent to no grouping.

[0040] After sparse frequency grouping, we perform a sparse frequency mixing operation, which involves K different linear layers that perform linear mapping operations on K groups of frequency points. The Kth linear layer is denoted as (SFM stands for Sparse Frequency Mixing, i.e. sparse frequency mixing), then the result obtained after sparse frequency mixing of the i-th group is It can be expressed as:

[0041]

[0042] in, A linear mapping is performed along the second dimension.

[0043] Finally, the results of all groups of sparse frequency mixing are stacked and reshaped to get the output result of this step.

[0044]

[0045] Figure 2 A schematic diagram of the weight matrix after learning is shown.

[0046] Step S6, cross-region frequency prediction. Given a prediction sequence length H and a length of each region l, the number of regions that the prediction sequence can be divided into is Q = H / l. The goal of S6 is to predict the frequency values ​​of the corresponding positions of the following Q regions from the known frequency values ​​of the P regions. Specifically, the method described in the embodiment of the present invention uses a common linear layer L PP (PP stands for Patch Predictor, i.e., cross-region predictor): f c The frequency values ​​of ×M frequency points in P regions are linearly transformed in parallel to obtain the frequency values ​​in the subsequent Q regions. The output of this step It can be expressed as:

[0047]

[0048] Step S7, zero padding and inverse real sequence fast Fourier transform (irFFT). First, the output of S6 is transformed along the second dimension (f c ) to zero pad points to adapt to the sequence length, and then perform irFFT to transform the second dimension to the length of c points, and obtain This process can be expressed mathematically as:

[0049]

[0050] Wherein, Pad(·) is a zero-padding operation.

[0051] Step S8, sequence rearrangement operation. This step takes the result obtained in the previous step Perform rearrangement operations to obtain the final prediction results This step can be expressed as:

[0052]

[0053] Here, Rearr(·) represents a rearrangement operation.

[0054] In steps S1 to S8, all linear mappings can be replaced by deep neural networks (such as multi-layer perceptrons, deep convolutional networks, etc.).

[0055] As shown in Table 1, this method significantly reduces the number of parameters, up to six orders of magnitude, and the inference time and training resources are also significantly improved compared to other methods.

[0056]

[0057] Table 1 Comparison of parameter quantity, video memory usage and batch inference time

[0058] We use MSE (mean square error) to evaluate the quality of prediction, and test the results with other algorithms on four data sets (power network data sets ETTh1, ETTh2, Electricity and traffic network data set Traffic). The results show that the method described in the embodiment of the present invention achieves the best performance in multiple indicators, as shown in Table 2 below:

[0059]

[0060] Table 2 Comparison of the effects of this method and other methods

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. This method greatly reduces the size of the network by downsampling and weight matrix sparsification, while accelerating reasoning and reducing training costs.

[0063] 2. This method achieved good prediction results.

[0064] 3. The downsampling part of this method is based on the Nyquist sampling theorem, and the frequency sparse grouping part is based on the observation of the correlation between frequency points, which has a certain degree of interpretability.

[0065] The embodiments of the present disclosure can be written as computer programs and can be implemented in general-use digital computers that execute the programs using a computer readable recording medium.

[0066] In this state, the medium can continuously store programs that can be executed by the computer, or can temporarily store programs for execution or downloading. In addition, the medium can be a variety of recording devices or storage devices in combination with single or multiple hardware, not limited to media that are directly connected to the computer system, and can exist in a distributed manner on the network. Examples of media include magnetic storage media (such as floppy disks or hard disks) configured to store program instructions, optical recording media (such as CD-ROMs or DVDs), magneto-optical media (such as floppy disks), and ROM, RAM, flash memory, etc. In addition, examples of other media may include: application stores for distributing applications, sites for providing or distributing various other software, and recording media or storage media managed at a server.

[0067] Although the present disclosure has been specifically shown and described with reference to preferred embodiments using specific terms, the embodiments and terms should be considered in a descriptive sense only and not for the purpose of limitation. Therefore, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present disclosure as defined by the appended claims.

Claims

1. A network traffic prediction method based on local downsampling and sparse frequency grouping, the method comprising: S1: Sequentially divide the input sequence x of length L into P regions, and the length of each region is l = L / P, expressed as X = Patch(x), where Patch represents the region division operation; S2: Perform regional downsampling operation on X, set the downsampling factor M, and downsample each row of X. M subsequences are obtained for each row. The length of each subsequence is c = l / M, which is expressed as Downsample(·) represents the downsampling operation; S3: Downsampling results Along the subsequence dimension, a real sequence fast Fourier transform rFFT is performed on each subsequence in each region, which is expressed as S4: Perform frequency truncation operation on the result obtained in S3, expressed as Indicates frequency truncation operation; S5: Perform sparse frequency grouping along The frequency dimension in the sequence is grouped, and the f c The frequency points will be divided into K groups, and the i-th group contains PM with The subsequence of frequency points is expressed as Among them, [:,i,:,:] represents the index of the corresponding dimension, : represents that the dimension remains unchanged. After the sparse frequency grouping, the sparse frequency mixing operation is performed. K different linear layers complete the linear mapping operation for K groups of frequency points respectively. The Kth linear layer is SFM is sparse frequency mixing. The result obtained after sparse frequency mixing of the i-th group is expressed as Stack and reshape the results of all groups of sparse frequency mixing to get the output result and S6: Perform cross-region frequency prediction. Given the prediction sequence length H and the length of each region l, the number of regions divided into the prediction sequence is Q = H / l, using a common linear layer L PP , for f c The frequency values ​​of ×M frequency points in P regions are linearly transformed in parallel to obtain the frequency values ​​in the subsequent Q regions. The output is expressed as S7: Perform zero padding and inverse real sequence fast Fourier transform irFFT, and transform the output of S6 along the second dimension f c Zero padding to points to fit the sequence length, and then irFFT is performed to transform the second dimension back to the length of c points, expressed as Wherein, Pad(·) is a zero-padding operation; and S8: Execute sequence rearrangement operation and convert the result obtained in S7 Perform the rearrangement operation to obtain the final prediction result, which is expressed as Here, Rearr(·) represents a rearrangement operation.

2. The method according to claim 1, characterized in that The real sequence fast Fourier transform rFFT of S3 is replaced by fast Fourier transform FFT.

3. The method according to claim 1, characterized in that Linear mappings are replaced by deep neural networks.

4. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps in any one of claims 1 to 3 when executed.

5. A computer program product, comprising a computer program, which implements the steps of any one of claims 1 to 3 when executed.

6. A network traffic prediction method based on local downsampling and sparse frequency grouping, the method comprising: S1: Sequentially divide the input sequence x of length L into P regions, and the length of each region is l = L / P, expressed as X = Patch(x), where Patch represents the region division operation; S2: Perform regional downsampling operation on X, set the downsampling factor M, and downsample each row of X. M subsequences are obtained for each row. The length of each subsequence is c = l / M, which is expressed as Downsample(·) represents the downsampling operation; S3: Downsampling results Along the subsequence dimension, a real sequence fast Fourier transform rFFT is performed on each subsequence in each region, which is expressed as S4: Perform sparse frequency grouping along The frequency dimension in the sequence is grouped, and the f c The frequency points will be divided into K groups, and the i-th group contains PM with The subsequence of frequency points is expressed as Among them, [:,i,:,:] represents the index of the corresponding dimension, : represents that the dimension remains unchanged. After the sparse frequency grouping, the sparse frequency mixing operation is performed. K different linear layers complete the linear mapping operation for K groups of frequency points respectively. The Kth linear layer is SFM is sparse frequency mixing. The result obtained after sparse frequency mixing of the i-th group is expressed as Stack and reshape the results of all groups of sparse frequency mixing to get the output result and S5: Perform cross-region frequency prediction. Given the prediction sequence length H and the length of each region l, the number of regions divided into the prediction sequence is Q = H / l, using a common linear layer L PP , for f c The frequency values ​​of ×M frequency points in P regions are linearly transformed in parallel to obtain the frequency values ​​in the subsequent Q regions. The output is expressed as S6: Perform zero padding and inverse real sequence fast Fourier transform irFFT, and transform the output of S6 along the second dimension f c Zero padding to points to fit the sequence length, and then irFFT is performed to transform the second dimension back to the length of c points, expressed as Wherein, Pad(·) is a zero-padding operation; and S7: Execute sequence rearrangement operation and convert the result obtained in S7 into Perform the rearrangement operation to obtain the final prediction result, which is expressed as Here, Rearr(·) represents a rearrangement operation.

7. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any one of the above claims 6 when executed.

8. A computer program product, comprising a computer program, which, when executed, implements the steps of any one of the preceding claims 6.

Images