A network traffic prediction method based on local downsampling and sparse frequency grouping

Abstract

The present invention discloses a network traffic prediction method based on local downsampling and sparse frequency grouping, the method comprising: sequentially dividing an input sequence into P regions; performing downsampling operations within the regions; performing real sequence fast Fourier transform on the downsampling results; performing frequency truncation operations on the results; performing sparse frequency grouping and sparse frequency mixing operations; performing cross-region frequency point prediction; performing zero padding and inverse real sequence fast Fourier transform irFFT; and performing sequence rearrangement operations to obtain an output sequence.

Description

Technical Field

[0001] The present invention belongs to the technical field of time series prediction, and in particular relates to a network traffic prediction method based on local downsampling and sparse frequency grouping. Background Art

[0002] Network traffic forecasting plays a crucial role in modern communications networks, particularly amidst growing network loads and increasing data transmission demands. Predicting future network traffic trends is crucial for avoiding network congestion, properly allocating bandwidth, and supporting traffic anomaly detection and security protection. Network administrators and operators rely on this predictive capability to ensure efficient utilization of network resources while maintaining service quality and security. To achieve these goals, network traffic forecasting often relies on a powerful data analysis tool: time series forecasting.

[0003] Time series forecasting technology is the theoretical foundation of network traffic forecasting because network traffic has significant time dependence. Network data exhibits complex fluctuations, trends, and seasonal variations over different time periods. Therefore, using time series forecasting technology to capture and analyze these changing patterns can effectively improve the accuracy of network traffic forecasting.

[0004] Time series forecasting is not limited to network traffic prediction; it is widely used in many fields, such as financial markets, weather forecasting, and energy demand prediction. A key characteristic of time series data is that the order of data points is crucial, and adjacent data often exhibits correlations. By deeply exploring the underlying patterns in time series, forecasting models can capture trends and cyclical changes, thereby predicting future changes. This is highly consistent with the needs of network traffic forecasting, as network traffic is also subject to various cyclical, seasonal, and unexpected events.

[0005] Classic time series prediction models such as ARIMA and SARIMA have been widely used in network traffic forecasting. They can provide accurate traffic forecasts for network management by modeling trends and fluctuations in historical data. However, as networks continue to expand, traffic patterns become increasingly complex, and traditional time series models have shown limitations when processing large-scale, nonlinear, and multi-dimensional network data.

[0006] To address this issue, machine learning and deep learning technologies have emerged as a new trend in time series forecasting in recent years. Recurrent neural networks (RNNs) and their variants, such as long short-term memory (LSTM) and gated recurrent units (GRU), have been introduced to network traffic forecasting due to their superior ability to process long-term dependent data. These models enable forecasting systems to capture complex patterns in network traffic and predict future traffic changes. The more advanced Transformer model, first introduced in the field of natural language processing, is now also being used in time series forecasting. Its ability to process data in parallel makes it particularly effective when handling large-scale network traffic. Furthermore, the self-attention mechanism within the Transformer effectively extracts relationships between different parts of long sequences, making it particularly suitable for long-sequence forecasting. Summary of the Invention

[0007] On the one hand, due to the inherent characteristics of the self-attention mechanism of Transformer-type models, it is easy to blur the sequential information of time series sampling points; on the other hand, a number of nonlinear models headed by Transformer-type models lead to extremely high computational and storage overheads due to their huge neural network architecture; thirdly, these models are often prone to overfitting due to their complex architecture, and trained models are not easy to migrate to other data distributions; fourthly, these models make extensive use of neural networks but lack corresponding interpretability, which makes the models completely black boxes.

[0008] To this end, we propose a network traffic prediction method based on local downsampling and sparse frequency grouping for the prediction of network traffic and other time series.

[0009] The above method has the following advantages:

[0010] Our proposed method has an extremely low number of parameters, minimal computational overhead, and extremely fast inference and training speeds. Compared to Transformer and convolutional models, it can reduce the number of parameters by up to seven orders of magnitude and storage overhead by up to two orders of magnitude. Furthermore, it achieves optimal prediction performance.

[0011] It should have a solid theoretical basis and be explainable to a certain extent.

[0012] This method has certain universality and transferability. After training on one dataset, our method can be seamlessly transferred to another dataset and achieve good results. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A network traffic prediction method based on local downsampling and sparse frequency grouping according to an embodiment of the present invention is illustrated.

[0014] Figure 2 A schematic diagram of the weight matrix after learning is shown. DETAILED DESCRIPTION

[0015] Below, with reference to the accompanying drawings, embodiments of the present disclosure are described. In the accompanying drawings, the same reference numerals are used for identical or corresponding structural elements. In addition, in the description of the following embodiments, repeated descriptions of identical or corresponding structural elements may be omitted. However, even if the description of a structural element is omitted, it does not mean that those structural elements are not included in a certain embodiment.

[0016] Although the process steps, method steps, and algorithms are described in sequence in the flowcharts shown in this specification, these processes, methods, and algorithms can be configured to operate in any suitable order. In other words, the process steps, method steps, and algorithm steps described in various embodiments of the present disclosure do not need to be performed in the order described in the present disclosure.

[0017] Furthermore, even if some steps are described as being performed non-simultaneously, these steps may be performed simultaneously in other embodiments. Furthermore, the examples of the processes described with the accompanying drawings do not mean that the illustrated processes exclude other variations and modifications thereto, nor do they mean that the illustrated processes or any of their steps are essential to one or more of the various embodiments of the present disclosure, nor do they mean that the illustrated processes are preferred.

[0018] Figure 1 A network traffic prediction method based on local downsampling and sparse frequency grouping according to an embodiment of the present invention is illustrated.

[0019] The network traffic prediction task can be abstracted as a time series prediction problem. Specifically, given an input sequence of length L (in, represents an L-dimensional real number space, and the subsequent notation is the same). Our goal is to obtain a prediction sequence of output with a length of H To this end, the method described in the embodiment of the present invention includes 8 steps:

[0020] Step S1: divide the input sequence into regions. The number of regions is determined by the hyperparameter P. Specifically, for an input sequence x of length L, it is divided into P regions (Patch), and the length of each region is l = L / P. After region division, it becomes a matrix form (in, represents a P×l real matrix space, and the subsequent notations are the same), where the i-th row represents the l sequence values ​​corresponding to the i-th region. The process can be expressed as:

[0021] X=Patch(x)

[0022] Among them, Patch represents the operation of area division.

[0023] Step S2: Perform intra-region downsampling on the sequence matrix X obtained in step S1 after region division. According to the Nyquist sampling theorem, as long as the sampling frequency of a time series is greater than twice its maximum frequency, the original signal can be restored without distortion. Therefore, we set the downsampling factor M based on experience. The downsampling operation downsamples each row of X, and M subsequences are obtained for each row, with the length of each subsequence being c = l / M. This process can be expressed mathematically as:

[0024]

[0025] in, is the result after downsampling, where Represents a The third-order real tensor space of , the following notations are similar. Downsample(·) represents the downsampling operation, which is actually equivalent to a mapping:

[0026] For intra-region downsampling, when M=1, it is equivalent to no operation.

[0027] The downsampling operation in step S2 converts the complete input sequence into several subsequences so that parameters can be shared among these subsequences.

[0028] Step S3: For the downsampling result x, along the subsequence dimension, perform rFFT (real Fast Fourier Transform) on each subsequence in each region. The sequence value becomes sequence values, where floor(·) represents rounding down. This step can be expressed as:

[0029]

[0030] in, is the result after rFFT, Represents a set of complex numbers.

[0031] The subsequence rFFT in step S3 performs rFFT operations on the subsequences in parallel (ie, simultaneously) to obtain the frequency spectrum of the subsequences. In step S3, rFFT can be replaced by FFT operation.

[0032] Step S4, frequency truncate the result obtained in S3, i.e. retain the first f c The conversion frequency points can be expressed as:

[0033]

[0034] in, is the result after frequency truncation, and Cut(·) represents the frequency truncation operation. It can be seen that the frequency truncation operation is the result of the parallel operation of PM subsequences, and only the first f of each subsequence is retained. c frequency points.

[0035] The frequency truncation operation can discard redundant frequency value information (because high frequencies are mainly noise). The frequency truncation operation can also be skipped, in which case f c =f.

[0036] Step S5 is a sparse frequency mixing operation based on sparse frequency grouping. This step fuses the different frequency values ​​of each sequence in parallel through a simple linear layer to facilitate the next prediction operation. The motivation for sparse frequency grouping is very simple. As shown in the figure below, when the sparse frequency grouping strategy is not used, the weight matrix learned by the model presents the pattern in the figure, that is, the same frequency point is most affected by itself and the surrounding frequency points, and is less affected by distant frequency points. Based on this, we propose a sparse frequency grouping method as shown in the figure. First, the frequencies are grouped and different linear layers are applied to the groups. This approach can transform the original dense weight matrix into a sparse weight matrix composed of sparse diagonal blocks, and forward and backward propagation can be performed in parallel in actual training. Specifically, we follow the The frequency dimension in the sequence is grouped, and the f c The frequency points will be divided into K groups, forming ( Represents a shape The fourth-order complex tensor space of , where the i-th group contains PM with The subsequence of frequency points is recorded as:

[0037]

[0038] Among them, [:,i,:,:] represents the index of the corresponding dimension, and : means that the dimension remains unchanged.

[0039] In the frequency sparse grouping operation, when K=1, it is equivalent to no grouping.

[0040] After sparse frequency grouping, we perform a sparse frequency mixing operation, which involves K different linear layers performing linear mapping operations on K groups of frequency points. The Kth linear layer is denoted as (SFM stands for Sparse Frequency Mixing, i.e. sparse frequency mixing), then the result obtained after sparse frequency mixing of group i is It can be expressed as:

[0041]

[0042] in, A linear mapping is performed along the second dimension.

[0043] Finally, the results of all groups of sparse frequency mixing are stacked and reshaped to obtain the output result of this step.

[0044]

[0045] Figure 2 A schematic diagram of the weight matrix after learning is shown.

[0046] Step S6, cross-region frequency prediction. Given a prediction sequence length H and a length of each region l, the number of regions that the prediction sequence can be divided into is Q = H / l. The goal of S6 is to predict the frequency values ​​of the corresponding positions of the next Q regions based on the known frequency values ​​of the P regions. Specifically, the method described in the embodiment of the present invention uses a common linear layer L PP (PP stands for Patch Predictor, i.e., cross-region predictor): f c The frequency values ​​of ×M frequency points in P regions are linearly transformed in parallel to obtain the frequency values ​​in the subsequent Q regions. The output of this step It can be expressed as:

[0047]

[0048] Step S7, zero padding and inverse real sequence fast Fourier transform (irFFT). First, the output of S6 is transformed along the second dimension (f c ) fill with zero to points to adapt to the sequence length, and then perform irFFT to transform the second dimension to the length of c points, and obtain This process can be expressed mathematically as:

[0049]

[0050] Among them, Pad(·) is a zero-padding operation.

[0051] Step S8, sequence rearrangement operation. This step will get the result of the previous step Perform rearrangement operations to obtain the final prediction results This step can be expressed as:

[0052]

[0053] Here, Rearr(·) represents a rearrangement operation.

[0054] In steps S1 to S8, all linear mappings can be replaced by deep neural networks (such as multi-layer perceptrons, deep convolutional networks, etc.).

[0055] As shown in Table 1, this method significantly reduces the number of parameters, up to six orders of magnitude, and significantly improves the inference time and training resources compared to other methods.

[0056]

[0057] Table 1 Comparison of parameter size, memory usage, and batch inference time

[0058] We use MSE (mean square error) to evaluate the quality of predictions and compare the results with other algorithms on four datasets (power network datasets ETTh1, ETTh2, Electricity, and traffic network dataset Traffic). The results show that the method described in the embodiment of the present invention achieves the best performance in multiple indicators, as shown in Table 2 below:

[0059]

[0060] Table 2 Comparison of the effects of this method and other methods

[0061] Compared with the prior art, the present disclosure has the following beneficial effects:

[0062] 1. This method greatly reduces the size of the network by downsampling and weight matrix sparsification, while accelerating reasoning and reducing training overhead.

[0063] 2. This method achieved good prediction results.

[0064] 3. The downsampling part of this method is based on the Nyquist sampling theorem, and the frequency sparse grouping part is based on the observation of the correlation between frequency points, which has certain interpretability.

[0065] The embodiments of the present disclosure can be written as computer programs and can be implemented in general-use digital computers that execute the programs using a computer readable recording medium.

[0066] In this state, the medium can continuously store programs that can be executed by the computer, or can temporarily store programs for execution or downloading. In addition, the medium can be various recording devices or storage devices in combination with single or multiple hardware, not limited to media that are directly connected to the computer system, and can exist in a distributed manner on the network. Examples of media include magnetic storage media (such as floppy disks or hard disks) configured to store program instructions, optical recording media (such as CD-ROMs or DVDs), magneto-optical media (such as floppy disks), and ROM, RAM, flash memory, etc. In addition, examples of other media can include: application stores for distributing application programs, sites for providing or distributing various other software, and recording media or storage media managed at a server.

[0067] Although the present disclosure has been particularly shown and described with reference to preferred embodiments using specific terminology, the embodiments and terminology should be considered in a descriptive sense only and not for purposes of limitation. Therefore, it will be understood by those skilled in the art that various changes in form and details may be made without departing from the spirit and scope of the present disclosure as defined by the appended claims.

Claims

1. A network traffic prediction method based on local downsampling and sparse frequency grouping, the method comprising: S1: Sequentially divide the input sequence x of length L into P regions, where the length of each region is l = L / P, expressed as X = Patch(x), where Patch represents the region partitioning operation; S2: Perform a regional downsampling operation on X, set the downsampling factor M, and downsample each row of X. For each row, M subsequences are obtained. The length of each subsequence is c = l / M, which is expressed as Downsample(·) represents the downsampling operation; S3: Downsampling results Along the subsequence dimension, a real sequence fast Fourier transform rFFT is performed on each subsequence in each region, which is expressed as S4: Perform frequency truncation on the result obtained in S3, expressed as Indicates frequency truncation operation; S5: Perform sparse frequency grouping along The frequency dimension in the sequence is grouped, and the f c The frequency points will be divided into K groups, and the i-th group contains PM with A subsequence of frequency points, represented as Among them, [:,i,:,:] represents the index of the corresponding dimension, : represents that the dimension remains unchanged. After the sparse frequency grouping, the sparse frequency mixing operation is performed. K different linear layers complete the linear mapping operation for K groups of frequency points respectively. The Kth linear layer is SFM is sparse frequency mixing. The result obtained after sparse frequency mixing of group i is expressed as The results of all groups of sparse frequency mixing are stacked and reshaped to obtain the output result and S6: Perform cross-region frequency prediction. Given the prediction sequence length H and the length of each region l, the number of regions divided into the prediction sequence is Q = H / l, using a common linear layer L PP , for f c The frequency values ​​of ×M frequency points in P regions are linearly transformed in parallel to obtain the frequency values ​​in the subsequent Q regions. The output is expressed as S7: Perform zero padding and inverse real sequence fast Fourier transform irFFT, and transform the output of S6 along the second dimension f c Zero padding to points to adapt to the sequence length, and then perform irFFT to transform the second dimension back to the length of c points, which is expressed as Wherein, Pad(·) is a zero-padding operation; and S8: Execute sequence rearrangement operation and convert the result obtained in S7 into Perform the rearrangement operation to obtain the final prediction result, which is expressed as Here, Rearr(·) represents a rearrangement operation.

2. The method according to claim 1, characterized in that The real sequence fast Fourier transform rFFT of S3 is replaced by fast Fourier transform FFT.

3. The method according to claim 1, characterized in that Linear mappings are replaced by deep neural networks.

4. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any one of claims 1 to 3 when executed.

5. A computer program product comprising a computer program, wherein the computer program is configured to implement the steps of any one of claims 1 to 3 when the computer program is executed.

6. A network traffic prediction method based on local downsampling and sparse frequency grouping, the method comprising: S1: Sequentially divide the input sequence x of length L into P regions, where the length of each region is l = L / P, expressed as X = Patch(x), where Patch represents the region partitioning operation; S2: Perform a regional downsampling operation on X, set the downsampling factor M, and downsample each row of X. For each row, M subsequences are obtained. The length of each subsequence is c = l / M, which is expressed as Downsample(·) represents the downsampling operation; S3: Downsampling results Along the subsequence dimension, a real sequence fast Fourier transform rFFT is performed on each subsequence in each region, which is expressed as S4: Perform sparse frequency grouping along The frequency dimension in the sequence is grouped, and the f c The frequency points will be divided into K groups, and the i-th group contains PM with A subsequence of frequency points, represented as Among them, [:,i,:,:] represents the index of the corresponding dimension, : represents that the dimension remains unchanged. After the sparse frequency grouping, the sparse frequency mixing operation is performed. K different linear layers complete the linear mapping operation for K groups of frequency points respectively. The Kth linear layer is SFM is sparse frequency mixing. The result obtained after sparse frequency mixing of group i is expressed as The results of all groups of sparse frequency mixing are stacked and reshaped to obtain the output result and S5: Perform cross-region frequency prediction. Given the prediction sequence length H and the length of each region l, the number of regions divided into the prediction sequence is Q = H / l, using a common linear layer L PP , for f c The frequency values ​​of ×M frequency points in P regions are linearly transformed in parallel to obtain the frequency values ​​in the subsequent Q regions. The output is expressed as S6: Perform zero padding and inverse real sequence fast Fourier transform irFFT, and transform the output of S6 along the second dimension f c Zero padding to points to adapt to the sequence length, and then perform irFFT to transform the second dimension back to the length of c points, which is expressed as Wherein, Pad(·) is a zero-padding operation; and S7; perform sequence rearrangement operation and convert the result obtained in S7 into Perform the rearrangement operation to obtain the final prediction result, which is expressed as Here, Rearr(·) represents a rearrangement operation.

7. A computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of any one of the above claims 6 when executed.

8. A computer program product comprising a computer program, which, when executed, implements the steps of any one of the preceding claims 6.

Images