Network traffic prediction method based on deep learning technology
This network traffic prediction method, which combines multi-scale convolution and Transformer, overcomes the limitations of data feature extraction in existing technologies and achieves higher prediction accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU FUTURE NETWORKS INNOVATION
- Filing Date
- 2025-07-30
- Publication Date
- 2026-05-01
AI Technical Summary
Existing network traffic prediction methods have limitations in extracting data features, making it difficult to effectively handle high-dimensional and nonlinear traffic data, resulting in limited prediction accuracy. Furthermore, they are difficult to model over long periods and have limited generalization ability.
We employ a combination of multi-scale convolution and Transformer, using generative adversarial networks for multi-scale feature extraction and global temporal dependency modeling. By optimizing loss functions based on mean squared error, mean absolute error, and difference assessment, we enhance the model's feature extraction and prediction capabilities.
It improves the accuracy of network traffic prediction, overcomes the limitations of existing methods, and enhances the model's ability to model traffic change trends and its robustness.
Smart Images
Figure CN121967248A_ABST
Abstract
Description
A network traffic prediction method based on deep learning technology Technical Field
[0001] This invention relates to the field of network communication technology, and in particular to a network traffic prediction method based on deep learning technology. Background Technology
[0002] Network traffic forecasting is a core task in the optimization and management of modern communication networks, and it is of great significance for network planning, resource allocation, and anomaly detection. Traditional network traffic forecasting methods mainly rely on statistical models and machine learning algorithms. However, these methods are usually based on linear assumptions or specific probability distributions, making it difficult to effectively handle high-dimensional and nonlinear traffic data, thus limiting the prediction accuracy of the models when faced with complex traffic patterns.
[0003] In recent years, deep learning technology has made significant progress in time series prediction tasks. Deep learning methods can automatically extract high-level features from traffic data through nonlinear transformations, and to some extent alleviate the limitations of traditional methods. However, existing methods still have certain limitations: 1. Insufficient local feature capture: Traditional CNN models typically use fixed-size convolutional kernels, which are insufficient for feature extraction at different scales. 2. Difficulty in modeling long-term dependencies: RNN-type models are prone to gradient vanishing or exploding problems when dealing with long-sequence traffic prediction, making it difficult to effectively capture long-term dependencies in traffic data. 3. Limited model generalization ability: Deep learning models are highly sensitive to the distribution of training data. When the spatiotemporal characteristics of traffic data fluctuate drastically, the model's prediction performance may drop sharply, causing the prediction results to deviate from the actual requirements.
[0004] Therefore, there is an urgent need for a new network traffic prediction method to address the limitations of existing network traffic prediction methods in extracting data features and to improve the accuracy of network traffic prediction. Summary of the Invention
[0005] The purpose of this invention is to provide a network traffic prediction method based on deep learning technology, which combines multi-scale convolution and Transformer to overcome the limitations of existing network traffic prediction methods in extracting data features and improve the accuracy of network traffic prediction.
[0006] To achieve the objectives of this invention, the technical solution is as follows:
[0007] A network traffic prediction method based on deep learning technology includes the following steps: S1, acquiring network traffic data;
[0008] S2. Preprocess the acquired network traffic data and then standardize it to generate standardized network traffic data.
[0009] S3. Input standardized network traffic data into the generator of the generative adversarial network, extract local features at different time scales through convolutional kernels of different sizes, and dynamically weight and fuse the multi-scale features to generate a multi-scale feature map.
[0010] S4. Input the multi-scale feature maps into the Transformer encoder and model the global temporal dependency through a multi-head self-attention mechanism;
[0011] S5. Input the features generated by the Transformer encoder into the Transformer decoder, and the Transformer decoder generates traffic prediction values for several future time steps, thus completing the generator's generation of prediction values.
[0012] S6. Input the traffic prediction value output by the generator and the real traffic data into the discriminator of the generative adversarial network. Extract the local and global patterns of the traffic data through the multilayer perceptron and output the difference between the generated data and the real traffic.
[0013] Furthermore, the method also includes step S7, optimizing the loss function constructed from the traffic prediction values.
[0014] Furthermore, the network traffic data obtained in step S1 includes traffic value and time.
[0015] Furthermore, the preprocessing of the acquired network traffic data described in step S2 includes removing invalid data, removing abnormal traffic, filling in missing values, and standardizing the data.
[0016] Furthermore, the removal of abnormal traffic refers to identifying and removing abnormal traffic data from the dataset using the 3σ principle; missing value imputation is achieved by linearly fitting the continuous missing values to the adjacent points of the time series using linear interpolation; the data standardization is achieved by unifying the data scale using the Z-score standardization method.
[0017] Furthermore, step S2, data preprocessing, also includes dividing the network traffic data into time windows.
[0018] Furthermore, in step S3, the standardized network traffic data is input into the generator of the generative adversarial network, and local features at different time scales are extracted by convolutional kernels of different sizes. The multi-scale features are dynamically weighted and fused to generate a multi-scale feature map. Specifically, this includes: S31, using convolutional kernels of 3×1, 5×1, and 7×1 sizes to perform convolution operations on the time series data to extract features at different time scales, and then connecting a ReLU activation function after each convolution operation.
[0019] S32. By performing a feature concatenation operation, features of different scales are concatenated along the channel dimension to form a fused feature vector containing multi-granularity information:
[0020]
[0021] S33. Using the channel attention mechanism, global average pooling is performed on the feature map in the time dimension to obtain a feature matrix containing time information: Where S is the average pooling result and T represents time;
[0022] S34. Add a fully connected layer to learn attention weights, and enhance non-linear expressiveness through the ReLU activation function: ,
[0023] Where W1 and W2 are the weight matrices of the convolution kernel, and the Softmax function is used for normalization;
[0024] S35. Weighted integration of cascaded features is performed to highlight key scale features. Simultaneously, residual connections are used to enhance gradient propagation.
[0025] F fused F represents the feature vector matrix after attention weight fusion, α represents the corresponding attention weight, ⊙ represents channel-wise multiplication, and F concat This represents a fused feature vector matrix containing multi-granularity information.
[0026] Furthermore, step S4, which involves inputting the multi-scale feature map into the Transformer encoder and modeling the global temporal dependency through a multi-head self-attention mechanism, includes the following steps: S41, adding positional encoding to the fused feature vector to preserve the sequential information of the time series;
[0027] S42. The temporal dependencies between features are calculated using a multi-head self-attention mechanism. This involves performing a linear transformation on Q, K, and V using each attention head to learn the association weights between features at different scales. The calculation formula for the self-attention mechanism is as follows:
[0028] Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, and dk represents the dimension of the key vector;
[0029] S43. After nonlinear mapping and residual connections in the feedforward neural network, the final output is a high-dimensional representation containing spatiotemporal global features. The feedforward network formula is as follows: Where W3 and W4 are the weight matrices of the convolution kernel, and b1 and b2 are the biases.
[0030] Furthermore, in step S5, the step of inputting the features generated by the Transformer encoder into the Transformer decoder, and having the Transformer decoder generate traffic prediction values for several future time steps, thus completing the generator's generation of prediction values, includes the following steps:
[0031] S51. Map the channel dimension to a word vector space and treat the time series length as the sequence length, embedding the temporal relationship of the time series through positional encoding; S52. Calculate the internal dependencies of the generated prediction sequence through a masked multi-head self-attention mechanism; S53. Interact the Q value of the decoder with the K and V values output by the encoder through an encoder-decoder attention mechanism; S54. Output the predicted flow value at each time step through the nonlinear mapping and residual connection of the feedforward neural network.
[0032] Furthermore, step S6, which involves inputting the traffic prediction value output by the generator and the real traffic data into the discriminator of the generative adversarial network, and extracting local and global patterns of the traffic data using a multilayer perceptron, outputting the difference between the generated data and the real traffic, specifically includes: using a multilayer perceptron to extract local and global patterns of the traffic data through multilayer nonlinear transformation. The discriminator outputs a probability value D(x) ∈ [0,1] representing the confidence level of the input data. The distribution difference between the generated data and the real data is quantified using cross-entropy loss, and the difference evaluation formula is as follows:
[0033] ;in, D(y) represents the discriminator loss function. real D(y) represents the probability that the discriminator classifies the real data. gen () represents the probability that the discriminator will classify the generated data. This is achieved by maximizing... Optimize the ability to identify real samples, while minimizing Optimize the ability to reject generated samples.
[0034] Furthermore, in step S7, the optimization of the loss function for constructing the traffic forecast value includes: combining the mean squared error (MSE), the mean absolute error (MAE), and L... GAN A dynamic weighted composite loss function is constructed, and the formula for the loss function is as follows: Where L represents the composite loss function, and λ1, λ2, and λ3 are adaptively optimized through backpropagation. This is the preprocessed network traffic value. These are predicted values.
[0035] Compared with existing technologies, this invention has significant advantages: It standardizes the raw traffic data and divides it into time windows to provide input data for subsequent feature extraction and modeling. Employing a generative adversarial network (GAN), the generator first performs multi-scale feature extraction, using convolutional kernels of different sizes to capture details and features of traffic data at different scales, enhancing the model's ability to model traffic change trends. Secondly, an attention mechanism is used to weightedly fuse features at different scales, enhancing the model's sensitivity to key features and improving its feature extraction capabilities, thus achieving multi-scale feature fusion. Finally, a Transformer decoder is used to generate traffic predictions for several future time steps, completing the generator's prediction process. The loss function uses mean squared error (MSE), mean absolute error (MAE), and a difference assessment formula to minimize the difference between the predicted and true values. MSE measures the squared difference between the predicted and true values, emphasizing the penalty for large errors, while MAE is used to improve the model's robustness. By continuously optimizing the discriminator's recognition ability, the data generated by the generator becomes increasingly closer to the real data. Ultimately, this overcomes the limitations of existing network traffic prediction methods in extracting data features and improves the accuracy of network traffic prediction. Attached Figure Description
[0036] Figure 1 is a flowchart of the network traffic prediction method based on deep learning technology of the present invention.
[0037] Figure 2 is a diagram of the generative adversarial network structure of the present invention.
[0038] Figure 3 is a schematic diagram of the overall structure of the multi-scale convolution module of the present invention.
[0039] Figure 4 is a structural diagram of the Transformer of the present invention. Detailed Implementation
[0040] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.
[0041] A network traffic prediction method based on deep learning technology includes the following steps: S1, acquiring network traffic data;
[0042] S2. Preprocess the acquired network traffic data and then standardize it to generate standardized network traffic data.
[0043] S3. Input standardized network traffic data into the generator of the generative adversarial network, extract local features at different time scales through convolutional kernels of different sizes, and dynamically weight and fuse the multi-scale features to generate a multi-scale feature map.
[0044] S4. Input the multi-scale feature maps into the Transformer encoder and model the global temporal dependency through a multi-head self-attention mechanism;
[0045] S5. Input the features generated by the Transformer encoder into the Transformer decoder, and the Transformer decoder generates traffic prediction values for several future time steps, thus completing the generator's generation of prediction values.
[0046] S6. Input the traffic prediction value output by the generator and the real traffic data into the discriminator of the generative adversarial network, extract the local and global patterns of the traffic data through the multilayer perceptron, and output the difference between the generated data and the real traffic.
[0047] S7. Optimize the loss function constructed from the traffic forecast values.
[0048] Furthermore, the network traffic data acquired in step S1 includes traffic values and time. Since the acquired network traffic data is raw data, it may contain missing values, outliers, or noisy data, requiring preprocessing. For example, traffic data for certain time periods may be missing due to equipment failure or network interruption, or there may be sudden spikes in abnormal traffic. To address these issues, the preprocessing of the acquired network traffic data described in step S2 includes removing invalid data, removing abnormal traffic, imputing missing values, and data standardization.
[0049] The removal of abnormal traffic employs the 3σ principle, based on the normal distribution characteristics, setting a threshold range [μ-3σ, μ+3σ], where μ is the mean of the traffic data and σ is the standard deviation. The formula for identifying and removing abnormal traffic from the dataset using the 3σ principle is as follows:
[0050] If |y-μ|>3σ, then y is removed, where y is the original flow rate value, μ is the mean of the flow rate data, and σ is the standard deviation.
[0051] Missing value imputation refers to the process of filling in missing values for continuous data by linearly fitting adjacent points in the time series using linear interpolation. For example, if traffic data for a certain time point is missing, imputed values can be generated by linearly fitting data from adjacent time points using the following formula:
[0052] , where φ(x) represents the missing flow value result, x0 and y0 represent the time and flow value of the previous moment, x1 and y1 represent the time and flow value of the next moment, and x represents the time of the current moment.
[0053] Data standardization refers to scaling all raw traffic data using the Z-score standardization formula. Z-score standardization is suitable for traffic data that follows a normal distribution and can more effectively preserve the statistical characteristics of the data. The formula is as follows:
[0054] , where y z Let y represent the standardized flow rate, y be the original flow rate, and μ and σ be the mean and standard deviation of the training set, respectively. After standardization, the data distribution approximates a standard normal distribution, i.e., the mean is 0 and the standard deviation is 1, which helps to accelerate model convergence.
[0055] Furthermore, in this invention, step S2, data preprocessing, also includes dividing the network traffic data into time windows. Specifically, the length and step size of the sliding window are set, and the network traffic data is divided into fixed-length time windows, such as 1 minute or 1 hour, which become the new basic unit.
[0056] The preprocessing steps described above are: removing invalid data, removing abnormal traffic, filling in missing values, data standardization, and dividing the data into time windows.
[0057] As a specific embodiment, step S3, which involves inputting standardized network traffic data into a multi-scale convolution module, extracting local features at different time scales using convolution kernels of different sizes, and dynamically weighting and fusing the multi-scale features to generate a multi-scale feature map, specifically includes:
[0058] S31. Parallel convolutional kernels of sizes 3×1, 5×1, and 7×1 are used to extract local features at different time scales, outputting local temporal feature vectors. Specifically, the 3×1 convolutional kernel is used to extract short-term fluctuation features. A 5×1 convolution kernel is used to capture change patterns at medium timescales. 7×1 convolutional kernels are used to identify long-term trend features. The output of each convolutional layer undergoes a non-linear transformation using the ReLU activation function to enhance the model's expressive power. During convolution calculation, padding is selected based on the kernel size to ensure that the short-term, medium-term, and long-term trend features extracted by the kernel are aligned in the time dimension. The formula for calculating the convolution kernel is as follows:
[0059]
[0060] S32. By performing a feature concatenation operation, features of different scales are concatenated along the channel dimension to form a fused feature vector containing multi-granularity information:
[0061]
[0062] S33. Using the channel attention mechanism, global average pooling is performed on the feature map in the time dimension to obtain a feature matrix containing time information: , where S is the average pooling result and T represents time;
[0063] S34. Add a fully connected layer to learn attention weights, and enhance non-linear expressiveness through the ReLU activation function: , where α represents the corresponding attention weight, W1 and W2 represent the weight matrices of the convolution kernel, and the Softmax function is used for normalization;
[0064] S35. Weighted integration of cascaded features is performed to highlight key scale features. Simultaneously, residual connections are used to enhance gradient propagation.
[0065] F fused F represents the feature vector matrix after attention weight fusion, α represents the corresponding attention weight, ⊙ represents channel-wise multiplication, and F concat This represents a fused feature vector matrix containing multi-granularity information.
[0066] Furthermore, step S4, which involves inputting the multi-scale feature map into the Transformer encoder and modeling the global temporal dependency through a multi-head self-attention mechanism, includes the following steps: S41, adding positional encoding to the fused feature vector to preserve the sequential information of the time series;
[0067] S42. The temporal dependencies between features are calculated using a multi-head self-attention mechanism. This involves performing a linear transformation on Q, K, and V using each attention head to learn the association weights between features at different scales. The calculation formula for the self-attention mechanism is as follows:
[0068] Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, and dk represents the dimension of the key vector;
[0069] S43. After nonlinear mapping and residual connections in the feedforward neural network, the final output is a high-dimensional representation containing spatiotemporal global features. The feedforward network formula is as follows: ,
[0070] In this model, W3 and W4 are the weight matrices of the convolutional kernels, and b1 and b2 are the biases. The Transformer encoder enhances the model's ability to capture long-term dependency patterns and also achieves collaborative modeling of local details and global trends through parallel computation of multi-head attention. The feedforward network further extracts the nonlinear relationships of features. The feedforward network consists of two fully connected layers with a ReLU activation function in between.
[0071] Furthermore, in step S5, the step of inputting the features generated by the Transformer encoder into the Transformer decoder, and having the Transformer decoder generate traffic prediction values for several future time steps, includes the following steps:
[0072] S51. Map the channel dimension to a word vector space and treat the time series length as the sequence length, embedding the temporal relationship of the time series through position encoding;
[0073] S52. The internal dependencies of the generated prediction sequence are calculated through the masked multi-head self-attention mechanism to ensure that the decoder relies only on historical information when generating the prediction value at the current time step.
[0074] S53. Through the encoder-decoder attention mechanism, the Q value of the decoder interacts with the K and V values of the encoder output, enabling the decoder to pay more attention to the global spatiotemporal features of the encoder output, thereby establishing the sequential correlation in the temporal space.
[0075] S54. Through nonlinear mapping and residual connection of the feedforward neural network, the predicted flow value at each time step is output.
[0076] Furthermore, step S6, which involves inputting the traffic prediction value output by the generator and the real traffic data into the discriminator of the generative adversarial network, and extracting local and global patterns of the traffic data using a multilayer perceptron, outputting the difference between the generated data and the real traffic, specifically includes: using a multilayer perceptron to extract local and global patterns of the traffic data through multilayer nonlinear transformation. The discriminator outputs a probability value D(x) ∈ [0,1] representing the confidence level of the input data. The distribution difference between the generated data and the real data is quantified using cross-entropy loss, and the difference evaluation formula is as follows:
[0077] ; where D(y) real D(y) represents the probability that the discriminator classifies the real data. gen () represents the probability that the discriminator will classify the generated data. This is achieved by maximizing... Optimize the ability to identify real samples, while minimizing Optimize the ability to reject generated samples.
[0078] Furthermore, in step S7, the optimization of the loss function for constructing the traffic forecast value includes: combining the mean squared error (MSE), the mean absolute error (MAE), and L... GAN A dynamic weighted composite loss function is constructed, and the formula for the loss function is as follows: Where L represents the composite loss function, and λ1, λ2, and λ3 are adaptively optimized through backpropagation. This is the preprocessed network traffic value. These are predicted values.
Claims
1. A network traffic prediction method based on deep learning technology, characterized in that: The method includes the following steps: S1, acquiring network traffic data; S2, preprocessing the acquired network traffic data and then standardizing it to generate standardized network traffic data; S3, inputting the standardized network traffic data into the generator of a generative adversarial network (GAN), extracting local features at different time scales through convolutional kernels of different sizes, dynamically weighting and fusing the multi-scale features to generate a multi-scale feature map; S4, inputting the multi-scale feature map into a Transformer encoder, modeling global temporal dependencies through a multi-head self-attention mechanism; S5, inputting the features generated by the Transformer encoder into a Transformer decoder, which generates traffic prediction values for several future time steps, completing the generator's generation of prediction values; S6, inputting the traffic prediction values output by the generator and the real traffic data into the discriminator of the GAN, extracting local and global patterns of the traffic data through a multilayer perceptron, and outputting the difference between the generated data and the real traffic.
2. The network traffic prediction method based on deep learning technology according to claim 1, characterized in that: The method further includes step S7, optimizing the loss function constructed from the traffic prediction values.
3. The network traffic prediction method based on deep learning technology according to claim 2, characterized in that: The network traffic data obtained in step S1 includes traffic value and time.
4. The network traffic prediction method based on deep learning technology according to claim 3, characterized in that: The preprocessing of the acquired network traffic data described in step S2 includes removing invalid data, removing abnormal traffic, filling in missing values, and standardizing data.
5. The network traffic prediction method based on deep learning technology according to claim 4, characterized in that: Abnormal traffic removal refers to identifying and removing abnormal traffic data from the dataset using the 3σ principle; missing value imputation is achieved by linearly fitting the continuous missing values to adjacent points in the time series using linear interpolation; and data standardization is achieved by unifying the data scale using the Z-score standardization method.
6. The network traffic prediction method based on deep learning technology according to claim 4, characterized in that: Step S2, data preprocessing, also includes dividing the network traffic data into time windows.
7. The network traffic prediction method based on deep learning technology according to claim 4, characterized in that: Step S3 involves inputting standardized network traffic data into the generator of a generative adversarial network (GAN). Local features at different time scales are extracted using convolutional kernels of different sizes in the multi-scale convolution module. These multi-scale features are then dynamically weighted and fused to generate a multi-scale feature map. Specifically, this includes: S31, performing convolution operations on the data using 3×1, 5×1, and 7×1 kernels respectively to extract features at different time scales, with a ReLU activation function applied after each convolution operation; and S32, concatenating features at different scales along the channel dimension through a feature concatenation operation to form a fused feature vector containing multi-granularity information. S33. Using channel attention, global average pooling is performed on the feature map along the time dimension to obtain a feature matrix containing time information. The calculation formula is: Where S is the average pooling result and T represents time; S34, add a fully connected layer to learn attention weights, and enhance the nonlinear expressive power through the ReLU activation function: Where W1 and W2 are the weight matrices of the convolution kernel, and the Softmax function is used for normalization; S35, the concatenated features are weighted and integrated to highlight key scale features, and residual connections are used to enhance gradient propagation: F fused F represents the feature vector matrix after attention weight fusion, α represents the corresponding attention weight, ⊙ represents channel-wise multiplication, and F concat This represents a fused feature vector matrix containing multi-granularity information.
8. The network traffic prediction method based on deep learning technology according to claim 4, characterized in that: Step S4, which involves inputting multi-scale feature maps into the Transformer encoder and modeling global temporal dependencies using a multi-head self-attention mechanism, includes the following steps: S41, adding positional encoding to the fused feature vector to preserve the sequential information of the time series; S42, calculating the temporal dependencies between features using a multi-head self-attention mechanism, i.e., performing linear transformations on Q, K, and V through each attention head to learn the association weights between features at different scales, where the calculation formula for the self-attention mechanism is: Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, and d k S43 represents the dimension of the key vector; after nonlinear mapping and residual connections in the feedforward neural network, the final output is a high-dimensional representation containing spatiotemporal global features. The feedforward network formula is: Where W3 and W4 are the weight matrices of the convolution kernel, and b1 and b2 are the biases.
9. The network traffic prediction method based on deep learning technology according to claim 4, characterized in that: In step S5, the step of inputting the features generated by the Transformer encoder into the Transformer decoder, and having the Transformer decoder generate traffic prediction values for several future time steps, includes the following steps: S51, mapping the channel dimension to a word vector space, and treating the time series length as the sequence length, embedding the temporal relationship of the time series through positional encoding; S52, calculating the internal dependencies of the generated prediction sequence through a masked multi-head self-attention mechanism; S53, interacting the Q value of the decoder with the K and V values output by the encoder through an encoder-decoder attention mechanism; S54, outputting the predicted traffic value for each time step through nonlinear mapping and residual connections of a feedforward neural network.
10. The network traffic prediction method based on deep learning technology according to claim 4, characterized in that: In step S6, the difference between the generated data and the actual traffic flow includes: using a multilayer perceptron to extract local and global patterns from the traffic flow data through multilayer nonlinear transformation; the discriminator outputs a probability value D(x) ∈ [0,1] to represent the confidence level of the input data. The distribution difference between the generated data and the actual data is quantified using cross-entropy loss, and the difference evaluation formula is as follows: ;where D(y) real D(y) represents the probability that the discriminator classifies the real data. gen () represents the discrimination probability of the discriminator on the generated data; in step S7, the optimization of the loss function for constructing the traffic prediction value includes: combining the mean squared error (MSE), mean absolute error (MAE), and L GAN A dynamic weighted composite loss function is constructed, and the formula for the loss function is as follows: Where L represents the composite loss function, and λ1, λ2, and λ3 are adaptively optimized through backpropagation. This is the preprocessed network traffic value. These are predicted values.