A method for ultra-long-term network traffic prediction based on deep learning

Abstract

The present invention discloses a method for predicting ultra-long-term network traffic based on deep learning. The method inputs bandwidth data enhanced with periodic information into an encoder-decoder architecture of a neural network and outputs a prediction of future bandwidth. The encoder inputs bandwidth data enhanced with historical periodic information and outputs periodic high-dimensional features extracted from the bandwidth data enhanced with historical periodic information. The decoder inputs bandwidth data enhanced with historical and future time slot periodic information and fuses the periodic high-dimensional features extracted from the bandwidth data enhanced with historical periodic information, outputting a prediction of future bandwidth. The ultra-long-term network traffic prediction method based on deep learning proposed in the present invention can model strongly correlated data (year, day) and weakly correlated data (week, month), respectively, achieving accurate ultra-long-term prediction of traffic data for an entire month or even longer, significantly improving the accuracy of ultra-long-term traffic prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of network traffic prediction, and in particular to a method for ultra-long-term network traffic prediction based on deep learning. Background Art

[0002] For Internet applications, server bandwidth fees are usually charged on a monthly basis. Therefore, predicting network traffic for the next month is important for cost management.

[0003] In the field of network traffic prediction, current traffic prediction methods are mainly divided into two categories based on the prediction scope:

[0004] 1. Short-term traffic prediction: This type of method focuses on predicting the traffic in the next time slot and is mainly used for real-time network capacity estimation of cellular networks.

[0005] 2. Long-term traffic prediction: This method is aimed at traffic prediction within the hourly time range.

[0006] Existing long-term traffic forecasting methods (C. Zhang and P. Patras, “Long-term mobile traffic forecasting using deep spatio-temporal neural networks,” in Proceedings of the Nineteenth ACM International Symposium on Mobile Ad Hoc Networking and Computing, MobiHoc 2018, Los Angeles, CA, USA, June 26-29, 2018, pp. 231–240) can only accurately predict traffic for a few dozen time slots (corresponding to a few dozen hours). Server bandwidth charges are typically based on the average or peak bandwidth usage for the current month, creating a need for ultra-long-term traffic forecasting for internet applications—predicting traffic for up to 720 time slots (720 hours) or even longer over the next month. Using ultra-long-term traffic forecasting, internet applications can dynamically adjust their traffic based on their monthly budget, thereby controlling bandwidth costs to within budget while fully utilizing the budget.

[0007] Previous long-term traffic forecasting efforts primarily focused on relatively long-term predictions (hourly), using statistical models (such as Prophet) and recurrent neural networks. However, as the forecast period increases, the results become increasingly distorted, falling far short of meeting the requirements for full-month traffic forecasts and making it difficult to accurately predict very long-term traffic. Analysis reveals that traffic has a multi-level periodicity spanning daily, weekly, monthly, and annual periods, making it difficult to effectively model long-term temporal dependencies using short-term historical data. The latest Transformer-based long-term time series forecasting methods Informer (H.Zhou, S.Zhang, J.Peng, S.Zhang, J.Li, H.Xiong, and W.Zhang, “Informer:Beyond efficient transformer forlong sequence time-series forecasting,” in Thirty-Fifth AAAI Conference onArtificial Intelligence, AAAI 2021, Virtual Event, February 2-9, 2021, 2021, pp.11106–11 115.) and Autoformer (H.Wu, J.Xu, J.Wang, and M.Long, “Autoformer:Decomposition transformers with Auto-Correlation for long-term seriesforecasting,” in 35th Conference on Neural Information Processing Systems(NeurIPS 2021), 2021.) only use 96 historical time slots as input, and it is still difficult to model important annual-level periodicity.

[0008] Therefore, how to improve the accuracy of ultra-long-term prediction of network traffic is an urgent problem to be solved. Summary of the Invention

[0009] Aiming at the problem of ultra-long-term traffic prediction for Internet applications, the present invention proposes an ultra-long-term network traffic prediction method based on deep learning. The method uses short-term historical data and historical data from the previous year to fully model the multi-level periodicity in traffic, thereby achieving the purpose of accurately predicting ultra-long-term traffic.

[0010] In order to achieve the above object, the present invention provides the following technical solutions:

[0011] The present invention provides a method for ultra-long-term network traffic prediction based on deep learning, comprising the following steps:

[0012] S1. Build bandwidth data enhanced with cycle information;

[0013] S2, input the bandwidth data enhanced with period information into the encoder-decoder architecture of the neural network, and output the predicted result of future bandwidth;

[0014] The encoder takes as input the bandwidth data enhanced with historical periodic information, and outputs the periodic high-dimensional features extracted from the bandwidth data enhanced with historical periodic information. The decoder takes as input the bandwidth data enhanced with historical and future time slot periodic information, fuses the periodic high-dimensional features extracted from the bandwidth data enhanced with historical periodic information, and outputs the predicted result of future bandwidth.

[0015] The encoder includes 1) a bandwidth sequence embedding module, and 2) M sequentially connected encoding modules, where each encoding module includes: a) an autocorrelation submodule, b) a first sequence decomposition submodule, c) a feedforward submodule, and d) a second sequence decomposition submodule;

[0016] The decoder includes 1) a bandwidth sequence embedding module, and 2) N sequentially connected decoding modules, where each decoding module includes: a) a first autocorrelation submodule, b) a first sequence decomposition submodule, c) a second autocorrelation submodule, d) a second sequence decomposition submodule, e) a feedforward submodule, and f) a third sequence decomposition submodule.

[0017] Furthermore, in step S1, for any time slot τ∈[tH,…,t+T-1], bandwidth data enhanced with period information is constructed according to the following steps:

[0018] S11, set the bandwidth d of a time slot τ and the bandwidth d of the time slot at the same position as this time slot in the previous year τ-Y Add bandwidth data;

[0019] S12, using a four-dimensional time code τ Enhances periodic information at all levels, four-dimensional time coding τ Includes the time slot's position on the day The position of the time slot date in the week The position of the time slot date in this month The position of the time slot date in the current year Get the bandwidth data enhanced by the period information of time slot τ

[0020] Furthermore, in step S2, each encoding module is defined as follows:

[0021]

[0022]

[0023] Among them, SeriesDecomp represents the sequence decomposition submodule, Auto-Correlation represents the autocorrelation submodule, and FeedForward represents the fully connected submodule. Indicates the input of the Lth encoder, Indicates the periodic output of the Lth encoder after the autocorrelation submodule and the first sequence decomposition submodule, Indicates the periodic output of the L-th encoder after the fully connected submodule and the second sequence decomposition submodule.

[0024] Furthermore, in step S2, each decoding module is defined as follows:

[0025]

[0026]

[0027]

[0028]

[0029] Among them, SeriesDecomp represents the sequence decomposition submodule, Auto-Correlation represents the autocorrelation submodule, FeedForward represents the fully connected submodule, and W l,1 ,W l,2 ,W l,3 is a learnable parameter; Indicates the input of the Lth decoder, Indicates the periodic output of the L-th decoder after the i-th autocorrelation submodule or fully connected submodule and the i-th sequence decomposition submodule, Indicates the trend output of the L-th decoder after the i-th autocorrelation submodule or fully connected submodule and the i-th sequence decomposition submodule; Indicates the trend output after aggregation.

[0030] Furthermore, in step S2, the bandwidth sequence embedding submodule uses a one-dimensional convolution module, and the input of each convolution block is F c in , F c in The bandwidth data of the period information enhancement from tH to t-1 is output as: F c out =W c *F c in +b c , where Wc and b c are both learnable parameters, and * represents the convolution operation.

[0031] Further, in step S2, the sequence decomposition sub-module inputs the time-series data and outputs the periodic part X s and the trend part X t extracted from the time-series data respectively. Padding is adopted to ensure the invariant sequence length, and the moving average calculation is performed through the average pooling layer to extract the trend of the time-series data.

[0032] Further, for the time-series data X∈R of length L L×d , the trend part X t output by the sequence decomposition sub-module is expressed as: X t = AvgPool(Padding(X)), and the periodic part X s is expressed as: X s = X - X t , where AvgPool represents the average pooling layer and Padding represents padding.

[0033] Further, in step S2, the definition of the autocorrelation sub-module is:

[0034]

[0035] where performs a time lag of t steps on the sequence , that is i , and returns the sequence . respectively represent the query sequence and the key sequence; represents the cross-correlation coefficient between [[ID=5且2]];

[0036] is the autocorrelation coefficient, expressed as:

[0037]

[0038] L is the time-series length.

[0039] Further, a neural network is adopted to predict at most the next T one-hour time slots , expressed as: For f t , t ∈ [t, t + T - 1], its unknown member d t is filled with 0. For the case where the actual required prediction length T′ < T, the first T are directly truncated ′The predicted value is used to obtain the predicted result.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The ultra-long-term network traffic prediction method based on deep learning proposed in this invention introduces an embedding layer that integrates historical data and time coding. It can model strongly correlated data (year, day) and weakly correlated data (week, month) separately, and realize accurate ultra-long-term prediction of traffic data for an entire month or even longer period of time, greatly improving the accuracy of ultra-long-term traffic prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0043] Figure 1 A model architecture diagram of the deep learning-based ultra-long-term network traffic prediction method provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] To better understand the present technical solution, the following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solution in the embodiments of the present invention. Obviously, the examples described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field based on this application fall within the scope of protection of the present invention.

[0045] The ultra-long-term network traffic prediction method based on deep learning proposed in this paper has a model architecture as follows: Figure 1 As shown, the following steps are included:

[0046] S1. Build bandwidth data enhanced with cycle information.

[0047] In the traffic prediction problem, using d t represents the bandwidth value of time slot t. Before time slot t, we need to predict t,…,T, a total of T-t+1 time slots. In order to utilize the autocorrelation of the bandwidth sequence, we use the bandwidth of the past H time slots to predict the bandwidth of the future T-t+1 time slots:

[0048]

[0049] in Represents the bandwidth prediction result of time slot t.

[0050] Specifically, in the present invention, for any time slot τ ∈ [t - H, …, t + T - 1], the bandwidth data enhanced with periodic information is constructed according to the following steps:

[0051] S11. To capture the daily and grade-level periodicities, the bandwidth d of a certain time slot τ and the bandwidth d of the time slot at the same position in the previous year as this time slot τ-Y are added to the bandwidth data;

[0052] S12. To enhance the periodicity information of all levels, a four-dimensional time encoding e τ is used to enhance the periodicity information of all levels. The four-dimensional time encoding e τ includes the position of the time slot in the day the position of the time slot date in the week the position of the time slot date in the month the position of the time slot date in the year to obtain the bandwidth data of time slot τ enhanced with periodic information

[0053] To predict at most the next T one-hour time slots, the present invention uses a neural network

[0054]

[0055] For f t , t ∈ [t, t + T - 1], its unknown member d t is filled with 0. For the case where the actual required prediction length T′ < T, the prediction result is directly obtained by truncating the first T′ predicted values.

[0056] S2. Input the bandwidth data enhanced with periodic information into the encoder-decoder architecture of the neural network, and output the prediction result of the future bandwidth;

[0057] Among them, the input of the encoder is the historical bandwidth data enhanced with periodic information [f t-H , …, f t-1 , and the output is the periodic high-dimensional features extracted from the historical bandwidth data enhanced with periodic information The input of the decoder is the historical and future time slot bandwidth data enhanced with periodic information [f t-H / 2 , …, f t-1 , f t , …, f t+T-1 , and it fuses the periodic high-dimensional features extracted from the historical bandwidth data enhanced with periodic information The output is the prediction result of the future bandwidth

[0058] The encoder includes 1) a bandwidth sequence embedding module, and 2) M sequentially connected encoding modules, where each encoding module includes: a) an autocorrelation submodule, b) a first sequence decomposition submodule, c) a feedforward submodule, and d) a second sequence decomposition submodule;

[0059] Each encoding module is defined as follows:

[0060]

[0061]

[0062] Among them, SeriesDecomp represents the sequence decomposition submodule, Auto-Correlation represents the autocorrelation submodule, and FeedForward represents the fully connected submodule. Indicates the input of the Lth encoder, Indicates the periodic output of the Lth encoder after the autocorrelation submodule and the first sequence decomposition submodule, This indicates the periodic output of the Lth encoder after the fully connected submodule and the second sequence decomposition submodule. The encoder can extract high-dimensional feature representations that indicate historical bandwidth data from the input, which can help the decoder capture the periodic dependencies of long time series.

[0063] The decoder includes 1) a bandwidth sequence embedding module, and 2) N sequentially connected decoding modules, where each decoding module includes: a) a first autocorrelation submodule, b) a first sequence decomposition submodule, c) a second autocorrelation submodule, d) a second sequence decomposition submodule, e) a feedforward submodule, and f) a third sequence decomposition submodule.

[0064] Each decoding module is defined as follows:

[0065]

[0066]

[0067]

[0068]

[0069] Among them, SeriesDecomp represents the sequence decomposition submodule, Auto-Correlation represents the autocorrelation submodule, FeedForward represents the fully connected submodule, and W l,1 ,W l,2 ,W l,3 is a learnable parameter; Indicates the input of the Lth decoder, Indicates the periodic output of the L-th decoder after the i-th autocorrelation submodule or fully connected submodule and the i-th sequence decomposition submodule, Indicates the trend output of the L-th decoder after the i-th autocorrelation submodule or fully connected submodule and the i-th sequence decomposition submodule; The decoder can leverage the high-dimensional periodic features of historical bandwidth data extracted by the encoder to better capture the cyclical dependencies of long-term time series. It can also simultaneously predict both periodicity and trend, and by aggregating both, it can improve prediction accuracy.

[0070] The bandwidth sequence embedding submodule uses a one-dimensional convolution module, and the input of each convolution block is The bandwidth data is enhanced with periodic information from tH to t-1, and the output is: Where W c and b c These are all learnable parameters, and * denotes a convolution operation. One-dimensional convolution is used to fuse the highly autocorrelated bandwidth data from the same period last year with the current bandwidth data and convert them into high-dimensional features.

[0071] The sequence decomposition submodule takes time series data as input and outputs the periodic parts X extracted from the time series data. s and Trend Section X t ,The sequence decomposition module can decompose time series data into two ,parts: trend and periodicity, which respectively reflect the changing trend and ,periodicity of time series data.

[0072] For time series data X∈R with length L L×d , the trend part X output by the sequence decomposition submodule t Expressed as: X t =AvgPool(Padding(X)), period part X s Expressed as: X s =XX t , AvgPool represents the average pooling layer, and Padding represents padding. This module uses padding to ensure that the sequence length remains unchanged and performs moving average calculations through the average pooling layer to extract trends in time series data.

[0073] In order to enhance the information utilization between data at different time series positions, the present invention uses the autocorrelation submodule to aggregate the information of the time-lagged series. To calculate the correlation between two series, the present invention uses the autocorrelation coefficient:

[0074]

[0075] Where L is the time series length.

[0076] The autocorrelation submodule is defined as:

[0077]

[0078] in Pair Sequence Conducted t i The time lag of the step is Return sequence represent query sequence and key sequence respectively; express and The mutual correlation coefficient.

[0079] The autocorrelation submodule uses the autocorrelation coefficient corresponding to the time lag step to weight the sequences with different time lag steps, and obtains a new sequence that integrates the position information of each time series.

[0080] The present invention proposes a method for ultra-long-term network traffic prediction based on deep learning, which introduces an embedding layer that integrates historical data and time coding. It can model strongly correlated data (year, day) and weakly correlated data (week, month) separately, and achieve accurate ultra-long-term prediction of traffic data for a whole month or even longer. The present invention uses 2 years of network traffic data for experiments, and sets 70%, 10% and 20% of the data in chronological order as training sets, validation sets and test sets. For the prediction model involved in the experiment, we set the number of historical data time slots H to 96 and the number of predicted time slots T to 720. The experimental results show that the mean absolute error (MAE) of the ultra-long-term network traffic prediction method based on deep learning of the present invention is: 0.2450, the MAE of Prophet is: 0.4111, the MAE of Autoformer is: 0.5411, and the MAE of Informer is: 0.4312. Experiments have shown that the ultra-long-term network traffic prediction method based on deep learning proposed in this invention reduces the mean absolute error (MAE) by 40.4%, 54.11%, and 43.11% compared with Prophet, Autoformer, and Informer, significantly improving the accuracy of ultra-long-term traffic prediction.

[0081] The foregoing description is merely a detailed description of the preferred embodiments and principles of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, any modifications, equivalent substitutions, improvements, etc. made based on the ideas provided by the present invention and within the spirit and principles of the present invention should be considered to be within the scope of protection of the present invention.

Claims

1. A method for ultra-long-term network traffic prediction based on deep learning, characterized in that: The steps include: S1. Construct bandwidth data enhanced with period information. For any time slot τ∈[tH,…,t+T-1], where t is the current time slot, H is the length of historical bandwidth data input to the model, and T is the length of future bandwidth data predicted by the model, construct bandwidth data enhanced with period information as follows: S11, set the bandwidth d of a time slot τ and the bandwidth d of the time slot at the same position as this time slot in the previous year τ-Y Add bandwidth data; S12, using a four-dimensional time code τ Enhances periodic information at all levels, four-dimensional time coding τ Includes the time slot's position on the day The position of the time slot date in the week The position of the time slot date in this month The position of the time slot date in the current year Get the bandwidth data enhanced by the period information of time slot τ S2, input the bandwidth data enhanced with period information into the encoder-decoder architecture of the neural network, and output the predicted result of future bandwidth; The encoder takes as input the bandwidth data enhanced with historical periodic information, and outputs the periodic high-dimensional features extracted from the bandwidth data enhanced with historical periodic information. The decoder takes as input the bandwidth data enhanced with historical and future time slot periodic information, fuses the periodic high-dimensional features extracted from the bandwidth data enhanced with historical periodic information, and outputs the predicted result of future bandwidth. The encoder includes 1) a bandwidth sequence embedding module, and 2) M sequentially connected encoding modules, where each encoding module includes: a) an autocorrelation submodule, b) a first sequence decomposition submodule, c) a feedforward submodule, and d) a second sequence decomposition submodule; The decoder includes 1) a bandwidth sequence embedding module, and 2) N sequentially connected decoding modules, where each decoding module includes: a) a first autocorrelation submodule, b) a first sequence decomposition submodule, c) a second autocorrelation submodule, d) a second sequence decomposition submodule, e) a feedforward submodule, and f) a third sequence decomposition submodule.

2. The ultra-long-term network traffic prediction method based on deep learning according to claim 1 is characterized in that: In step S2, each encoding module is defined as follows: Among them, SeriesDecomp represents the sequence decomposition submodule, Auto-Correlation represents the autocorrelation submodule, and FeedForward represents the fully connected submodule. Indicates the input of the Lth encoder, Indicates the periodic output of the Lth encoder after the autocorrelation submodule and the first sequence decomposition submodule, Indicates the periodic output of the L-th encoder after the fully connected submodule and the second sequence decomposition submodule.

3. The ultra-long-term network traffic prediction method based on deep learning according to claim 1 is characterized in that: In step S2, each decoding module is defined as follows: Among them, SeriesDecomp represents the sequence decomposition submodule, Auto-Correlation represents the autocorrelation submodule, FeedForward represents the fully connected submodule, and W l,1 ,W l,2 ,W l,3 is a learnable parameter; Indicates the input of the lth decoder, Indicates the periodic output of the L-th decoder after the i-th autocorrelation submodule or fully connected submodule and the i-th sequence decomposition submodule, Indicates the trend output of the i-th autocorrelation submodule or fully connected submodule and the i-th sequence decomposition submodule in the l-th decoder; Indicates the trend output after aggregation.

4. The ultra-long-term network traffic prediction method based on deep learning according to claim 1 is characterized in that: In step S2, the bandwidth sequence embedding submodule uses a one-dimensional convolution module, and the input of each convolution block is The bandwidth data is enhanced with periodic information from th to T-1, and the output is: Where W c and b c They are all learnable parameters, and * represents convolution operation.

5. The method for ultra-long-term network traffic prediction based on deep learning according to claim 1, characterized in that: In step S2, the sequence decomposition submodule inputs the time series data and outputs the period parts X extracted from the time series data. s and Trend Section X t , padding is used to ensure that the sequence length remains unchanged, and moving average calculation is performed through the average pooling layer to extract the trend of time series data.

6. The method for ultra-long-term network traffic prediction based on deep learning according to claim 5, characterized in that: For time series data X∈R with length L L×d , the trend part X output by the sequence decomposition submodule t Expressed as: X t =AvgPool(Padding(X)), period part X s Expressed as: X s =XX t , AvgPool represents the average pooling layer, and Padding represents padding.

7. The ultra-long-term network traffic prediction method based on deep learning according to claim 1 is characterized in that: In step S2, the autocorrelation submodule is defined as: in Pair Sequence Conducted t i The time lag of the step is Return sequence Represent the query sequence and key sequence respectively; express and The mutual correlation coefficient of is the autocorrelation coefficient, expressed as: L is the time series length.

8. The method for ultra-long-term network traffic prediction based on deep learning according to claim 1, characterized in that: Using a neural network Predict the next at most T one-hour time slots Expressed as: For f t , t ∈ [t, t + T - 1], its unknown member d t Is filled with 0. For the case where the actual required prediction length T′ < T, the prediction result is directly obtained by truncating the first T′ prediction values.

Images