A network traffic prediction method, system, medium, device and terminal

Abstract

The application belongs to the technical field of network traffic prediction, and discloses a network traffic prediction method, system, medium, equipment and terminal, historical network traffic data and related feature data are acquired, and the data is preprocessed to obtain an eigenfunction and a residual subsequence; sample entropy of the decomposed subsequence is calculated, the eigenfunction is modeled by using an LSTM model and an ARIMA model respectively, and data predicted by the ARIMA model and the LSTM model is reconstructed; a Transformer network traffic prediction model based on expanded causal convolution is constructed and trained, new data is trained and predicted by using the trained hybrid model, and a prediction value of network traffic is obtained. The application combines a statistical model and a machine learning-based model, is based on expanded causal convolution, improves the prediction accuracy of network traffic, strengthens the time correlation between network traffic data, and enhances feature capture in a long period.

Description

Technical Field

[0001] This invention belongs to the field of network traffic prediction technology, and particularly relates to a network traffic prediction method, system, medium, device and terminal. Background Technology

[0002] Currently, with the development of the internet, we are entering the era of the Internet of Things (IoT). IoT is an emerging field that promises ubiquitous internet connectivity, turning ordinary objects into connected devices. The IoT paradigm is changing the way people interact with their surroundings, paving the way for the creation of a widely connected infrastructure to support innovative services and promising greater flexibility and efficiency. However, the continuous advancement of IoT technology has also brought some pressing technical challenges. For example, the increasing scale of connected devices makes the network increasingly congested. While this can be addressed by increasing the bandwidth of communication lines, such as using fiber optic communication, the cost will be enormous as the scale of IoT expands. A better approach is to design network traffic scheduling algorithms. If network traffic can be predicted in advance, the server can guide terminal devices to use different routes more evenly. Therefore, network traffic prediction is crucial.

[0003] The Internet of Things (IoT) has permeated every aspect of people's lives. The development of wireless network communication technology is closely related to people's daily routines, and network traffic data reflects local lifestyles to a certain extent, containing massive amounts of information. Network traffic prediction is of positive significance for large-scale network capacity planning, network equipment design, network resource management, and user behavior regulation. Traffic prediction technology refers to predicting future traffic data based on detected historical data and by establishing appropriate traffic models. The prediction results can reveal the general trend of future traffic. If a threshold is given, an early warning is issued when the predicted result exceeds the threshold, allowing network administrators to check the network status in advance, detect potential attacks and intrusions, and achieve network intrusion detection. By predicting network traffic, we can understand the traffic situation and trends between networks, thereby optimizing the network more effectively and designing better routing and load balancing. Network traffic prediction can also determine network congestion control, reducing information loss and latency caused by network congestion, fully utilizing network resources, and improving service quality. Furthermore, network traffic prediction makes it possible to provide early warnings of malicious network attacks.

[0004] However, due to the massive number of terminal devices connected to the Internet of Things (IoT), network traffic data in the IoT exhibits more complex characteristics and contains more intricate information. Network traffic is characterized by abrupt changes, weak coupling, and nonlinearity. Therefore, neither a single linear model nor a single nonlinear model can adequately describe the characteristics and patterns of network traffic variation. Accurately analyzing the linear and nonlinear factors in network traffic is crucial for improving the accuracy of network traffic prediction. Since a single model has limited ability to reveal linear and nonlinear relationships in network traffic, combining these two models can extract the linear and nonlinear relationships in network traffic prediction, thereby improving the accuracy of data prediction. Improving the accuracy of network traffic data prediction has significant implications for network operation management, resource allocation, path planning, network congestion control, and prevention of network attacks.

[0005] Network traffic prediction has become a crucial issue in the computer networking field in recent years. Autoregressive Differential Moving Average (ARIMA) and Long Short-Term Memory (LSTM) neural networks have consistently received widespread attention from researchers in this area. Empirical Mode Decomposition (EMD), since its inception in 1998, has been extensively applied in various research areas. In 2021, Miao et al. proposed a method that uses EMD to model network traffic and an improved Autoregressive Moving Average (ARMA) model for prediction. They proposed an optimization function to reduce the prediction error of the network traffic model, and the results demonstrated the feasibility of the proposed scheme. SimaSiami-Namini et al. compared the performance and efficiency of ARIMA and LSTM models. Empirical studies show that deep learning-based algorithms (such as LSTM) outperform traditional algorithms based on ARIMA models. More specifically, compared to ARIMA, LSTM achieves an average error rate reduction of 84-87%, indicating that LSTM is superior to ARIMA. However, using only LSTM models for network traffic prediction wastes the existing linear structure of the network traffic data.

[0006] The goal of traffic forecasting is to predict future traffic flow by deriving spatiotemporal patterns from historical data. Before the advent of deep learning, traffic forecasting heavily relied on time series analysis methods. Traditional statistical methods, such as Autoregressive Integrated Moving Average (ARIMA), Kalman filtering, and Vector Autoregression (VAR) models, were widely used in traffic forecasting. While these models are simple and easy to use, their performance was limited by their inability to handle nonlinear data. The abrupt changes, weak coupling, and nonlinearity of network traffic data were once intractable problems. Recently, deep learning-based methods have shown significant advantages in network traffic forecasting. However, existing methods still suffer from incomplete measurements of long-term and short-term time dependencies. For example, most deep learning models fail to predict over excessively long periods. Furthermore, an increasing number of devices accessing the network are mobile, and changes in network topology give traffic data distinct spatial characteristics, which existing methods do not consider. Therefore, there is an urgent need to design a new network traffic forecasting method.

[0007] Based on the above analysis, the problems and shortcomings of the existing technology are as follows:

[0008] (1) Due to the huge number of terminal devices connected to the Internet of Things, neither a single linear model nor a nonlinear model can well describe the characteristics and changing patterns of network traffic.

[0009] (2) In the prior art, only the LSTM model based on deep learning algorithm is used to predict network traffic data, and the existing linear structure in the network traffic data will be wasted.

[0010] (3) The limitations of traditional statistical methods in handling nonlinear data restrict the performance of the model.

[0011] (4) Existing forecasting methods still have the problem of incomplete measurement of long-term and short-term time dependence.

[0012] (5) Changes in network topology cause traffic data to have obvious spatial characteristics, and existing methods do not take into account the impact of spatial characteristics. Summary of the Invention

[0013] To address the problems existing in the prior art, this invention provides a network traffic prediction method, system, medium, device, and terminal, and particularly relates to a Transformer network traffic prediction method, system, medium, device, and terminal based on dilated causal convolution and ARIMA-LSTM.

[0014] This invention is implemented as follows: a network traffic prediction method, comprising: acquiring historical network traffic data and related feature data and preprocessing the data to obtain intrinsic mode functions and residual subsequences; calculating the sample entropy of the decomposed subsequences; modeling the intrinsic mode functions using LSTM and ARIMA models respectively; reconstructing the data predicted by the ARIMA and LSTM models; constructing and training a Transformer network traffic prediction model based on dilated causal convolution; and using the trained hybrid model to train and predict new data to obtain predicted values ​​of network traffic.

[0015] Furthermore, the network traffic prediction method includes the following steps:

[0016] Step 1: For a target network with multiple terminal nodes, acquire historical network traffic data and related feature data, and construct a network traffic feature matrix as model input;

[0017] Step 2: Preprocess the acquired data using empirical mode decomposition to obtain multiple intrinsic mode functions and a residual subsequence, and calculate the sample entropy of each subsequence after decomposition.

[0018] Step 3: Use the LSTM model to model the intrinsic mode functions with high sample entropy, and use the ARIMA model to model the intrinsic mode functions with low sample entropy and the residual subsequences.

[0019] Step 4: Calculate the correlation coefficient between the time series of the target node and other nodes to represent the degree of spatial influence between nodes, and construct a Transformer network traffic prediction model based on dilated causal convolution.

[0020] Step 5: Train the Transformer network traffic prediction model based on dilated causal convolution, and reconstruct the data predicted by the ARIMA model and LSTM model.

[0021] Step 6: Use the trained hybrid model to predict the new data and obtain the predicted value of network traffic.

[0022] Furthermore, in step one, for a target network with multiple terminal nodes, historical network traffic data and related feature data are acquired, and a network traffic feature matrix is ​​constructed as model input, including:

[0023] (1) For a target network with multiple terminal nodes, obtain historical network traffic data, including source IP address, destination IP address, transport layer protocol, source port and port; obtain relevant characteristic data, including required bandwidth, actual bandwidth and number of connected terminals;

[0024] (2) Construct a feature matrix from the acquired data. The feature matrix is ​​represented as follows:

[0025]

[0026] Where N represents the number of nodes in the target network, T represents the number of time steps, and M represents the network traffic-related features;

[0027] (3) Divide the processed data into a training set, a validation set and a test set in a ratio of 7:1:2. The training set is used to build the model framework, the validation set is used to obtain the optimal parameters, and the test set is used to evaluate the accuracy of the traffic prediction model.

[0028] Furthermore, step two involves preprocessing the acquired data using empirical pattern decomposition and calculating the sample entropy of each decomposed subsequence, including:

[0029] Empirical mode decomposition (EMD) breaks down collected network traffic data into local feature signals at different time scales containing the original signal. These local feature signals are intrinsic mode functions (IMFs). The processed data includes many IMFs and a residual subsequence. The specific processing steps are as follows:

[0030] (1) Based on the upper and lower extreme points of the original signal X(t), the upper and lower envelopes are fitted by cubic spline curves respectively;

[0031] (2) Find the mean of the upper and lower envelopes and draw the mean envelope e. mean ;

[0032] (3) Subtract the mean envelope e from the original signal X(t) mean The intermediate signal h is obtained. n ;

[0033] (4) Determine whether the intermediate signal meets the two conditions of IMF. If it does, the signal is an IMF component. If it does not, based on the signal, repeat the analysis of steps (1) to (4) and obtain the IMF component by several iterations.

[0034] (5) After obtaining the first IMF, subtract IMF1 from the original signal to obtain the new original signal; then, through the analysis in steps (1) to (4), obtain IMF2; and so on, when r n When the function becomes a monotonic function or falls below a predefined threshold, the screening process terminates, and the EMD decomposition is completed.

[0035] Sample entropy measures the complexity of a time series by quantifying the probability of new patterns emerging in the signal. The higher the probability of a new pattern emerging, the greater the complexity of the sequence, as defined below:

[0036]

[0037] Among them, B m (r) is the probability that two sequences match at m points under a similarity tolerance of r, A m (r) is the probability that two sequences match at m+1 points.

[0038] Furthermore, step three involves using an LSTM model to model the intrinsic mode functions with high sample entropy, and using an ARIMA model to model the intrinsic mode functions with low sample entropy and the residual subsequences. This includes:

[0039] (1) The entire neural network is divided into three layers: the input layer, the recurrent layer, and the fully connected layer. The fully connected layer also serves as the output layer. The training of the neural network consists of multiple iterations, and each iteration includes two steps: forward propagation and backward propagation.

[0040] (2) During the forward propagation of the loop layer, the input of the current time step and the output of the previous time step are used to perform calculations to obtain the output of the current time step, which is used to model the relationship between the input and the output, as well as the historical correlation between the outputs.

[0041] The forward propagation process of the recurrent layer is as follows:

[0042] f (t) =σ(w f x (t) +u f h (t-1) +b f );

[0043] i (t) =σ(w i x (t) +u i h (t-1) +b i );

[0044]

[0045]

[0046] o (t) =σ(w o x (t) +u o h (t-1) +b o );

[0047] h (t) =o (t) *C (t) ;

[0048] Here, x is the input of the recurrent layer, h and C are two hidden states, and h is also the output of the recurrent layer; the subscript t represents the value at a certain time step, σ is the Sigmoid function; w, u and b are parameters, there are four sets in total, w is the relationship between input and output, u is the historical correlation of output, and b is the offset; all parameters are initialized to random values, and the hidden states are initialized to zero.

[0049] (3) During the backpropagation process of the recurrent layer, the gradient of each parameter and the hidden state is calculated in turn; after calculating the hidden state of the last time step, the parameter of the last time step is calculated; then the hidden state of the second to last time step is calculated, and so on; after obtaining the gradient of each parameter, the gradient is subtracted by a set multiple to complete the backpropagation.

[0050] (4) The fully connected layer performs a linear affine operation to compress the output dimension of the loop layer so that the output dimension of the loop layer is consistent with the label.

[0051] ARIMA is a model that transforms a non-stationary time series into a stationary time series and then regresses the dependent variable only on the lagged value q and the present value and lagged value of the random error term.

[0052] Where AR is autoregressive, p is the autoregressive term; MA is moving average, q is the number of moving average terms; and d is the number of differences performed when the time series becomes stationary.

[0053] Furthermore, step four involves calculating the correlation coefficients between the time series of the target node and other nodes, representing the degree of spatial influence between the nodes, including:

[0054] The correlation coefficient between the target node and other node time series is calculated using the Pearson correlation coefficient, which represents the degree of spatial influence between the nodes. The calculation formula is as follows:

[0055]

[0056] Among them, Y * Y represents the network traffic of the target node at a certain time slice. i Let σ represent the network traffic of the i-th node in the same time slice, Cov(·) represent the covariance function, and σ represent the network traffic of the i-th node in the same time slice. * and σ i Indicates variance.

[0057] Furthermore, the training of the Transformer network traffic prediction model based on dilated causal convolution in step five includes:

[0058] (1) Expand the feature dimension of the input three-dimensional feature vector using a 1×1 convolution;

[0059] (2) Use 1×1 convolution again to reduce the dimension to 1, integrate the input data into a two-dimensional feature vector, and use it as the input of the extended causal convolutional network;

[0060] (3) Construct an extended causal convolutional network; whereby the causal convolutional network abstracts the sequence problem as follows:

[0061]

[0062] Among them, X t Let T be a one-dimensional vector containing n features, and T be the time slice.

[0063] The definition of a dilated convolutional network is:

[0064]

[0065] Where s is the sequence element, d is the expansion coefficient, and sd·i describes the past direction, called * d To expand the convolution operator to distinguish it from normal convolution operations;

[0066] (4) Build the Transformer model, as follows:

[0067] 1) The encoder consists of n sub-layers with the same structure. Each sub-layer includes a multi-head self-attention mechanism and a feedforward neural network. The output of each part is connected by residuals.

[0068] 2) The decoder consists of n sub-layers with the same structure. Each sub-layer includes a multi-head self-attention mechanism, a masked multi-head self-attention mechanism, and a feedforward neural network. The output of each part is connected by residuals.

[0069] 3) The model is output after passing through a linear layer and a softmax function.

[0070] The various parts of the Transformer model are defined as follows:

[0071] The self-attention mechanism is defined as:

[0072]

[0073] Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, and d k Let Q and K be the number of columns.

[0074] Feedforward neural networks are defined as follows:

[0075] max(0,x);

[0076] max(0,XW1+b1)W2+b2;

[0077] Where X is the input to the feedforward neural network;

[0078] Residual connectivity is defined as:

[0079] LayerNorm(X+MultiHeadAttention(x));

[0080] LayerNorm(X+FeedForward(X));

[0081] Where X is the input of the multi-head self-attention mechanism or the feedforward neural network, MultiHeadAttention(x) is the output of the multi-head self-attention mechanism, and FeedForward(x) is the output of the feedforward neural network.

[0082] The model is built using the training set, and the parameters are tuned using the validation set, thus optimizing the model.

[0083] Furthermore, step six involves using the trained model to train the new data and obtain predicted network traffic values, including:

[0084] (1) Use the test set to evaluate the prediction accuracy of the Transformer network traffic prediction model based on dilated causal convolution;

[0085] (2) Collect new network traffic data to predict network traffic data.

[0086] Another object of the present invention is to provide a network traffic prediction system applying the aforementioned network traffic prediction method, the network traffic prediction system comprising:

[0087] The data acquisition module is used to acquire historical network traffic data and related feature data for a target network with multiple terminal nodes, and construct a network traffic feature matrix as model input.

[0088] The data preprocessing module is used to preprocess the acquired data using empirical mode decomposition to obtain multiple intrinsic mode functions and a residual subsequence, and to calculate the sample entropy of each subsequence.

[0089] The intrinsic mode function modeling module is used to model intrinsic mode functions with high sample entropy using the LSTM model, and to model intrinsic mode functions with low sample entropy and residual subsequences using the ARIMA model.

[0090] The prediction model building module is used to calculate the correlation coefficient between the time series of the target node and other nodes, and to build a Transformer network traffic prediction model based on dilated causal convolution.

[0091] The model training and reconstruction module is used to train the Transformer network traffic prediction model based on dilated causal convolution and to reconstruct the data predicted by the ARIMA model and LSTM model.

[0092] The network traffic prediction module is used to train a pre-trained hybrid model on new data to predict network traffic.

[0093] Another object of the present invention is to provide a computer device including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the network traffic prediction method described above.

[0094] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the network traffic prediction method.

[0095] Another objective of this invention is to provide an information data processing terminal for implementing the aforementioned network traffic prediction system.

[0096] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0097] First, addressing the technical problems existing in the prior art and the difficulty of solving them, this paper closely analyzes, in conjunction with the technical solution to be protected by this invention and the results and data obtained during the research and development process, how the technical solution of this invention solves the technical problems, and the inventive technical effects brought about by solving these problems. The specific description is as follows:

[0098] This invention provides a Transformer network traffic prediction method based on dilated causal convolution, which combines nonlinear and linear models. By integrating statistical and machine learning models, it improves the accuracy of network traffic prediction. Based on dilated causal convolution, this invention strengthens the representation of the temporal correlation between network traffic data, enhances the capture of long-term features, and considers the spatial influence between nodes in the network, thereby improving the accuracy of network traffic prediction and making it applicable to network resource allocation.

[0099] This invention uses empirical mode decomposition to process the collected data, decomposing the data into many local feature signals at different time scales containing the original signal. Each subsequence obtained by decomposition is ensured to be stationary data, enabling the ARIMA model to capture the linear structure in the data. At the same time, the LSTM model is used to process the nonlinear structure in the data, thereby improving the prediction accuracy of network traffic to a certain extent.

[0100] In addition, this invention employs dilated causal convolution to process network traffic data with longer periodic dependencies, capturing spatiotemporal feature information from historical traffic data over a longer period. Simultaneously, it uses the Transformer model, whose self-attention mechanism further processes long-period feature information, improving prediction accuracy and overcoming the problems of low accuracy in traditional machine learning methods and failure of deep learning methods to predict long-term traffic, thus effectively improving the accuracy of long-term network traffic prediction.

[0101] Second, considering the technical solution as a whole or from a product perspective, the technical effects and advantages of the technical solution to be protected by this invention are specifically described as follows:

[0102] This invention combines linear models and neural networks to improve the accuracy of network traffic prediction and also solves the problems of poor traffic prediction accuracy, low resource utilization, and low network throughput in 5G networks.

[0103] Third, as supplementary evidence of the inventive step of the claims of this invention, it is also reflected in the following important aspects:

[0104] (1) The expected benefits and commercial value of the technical solution of this invention after transformation are as follows:

[0105] Network traffic prediction is of great significance for large-scale network capacity planning, network equipment design, network resource management, and user behavior regulation. The prediction results can reveal the general trend of future traffic; accurate predictions can help administrators detect potential attacks and intrusions; accurate predictions can better optimize routing design and load balancing; and accurate predictions can fully utilize network resources and improve service quality. Improving the accuracy of network traffic prediction is crucial for the evolution from 5G to 6G networks.

[0106] (2) The technical solution of this invention fills a technical gap in the industry both domestically and internationally:

[0107] This invention is the first to use Transformer to predict network traffic data. Attached Figure Description

[0108] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0109] Figure 1 This is a flowchart of the network traffic prediction method provided in an embodiment of the present invention;

[0110] Figure 2 This is the input data feature map provided in the embodiments of the present invention;

[0111] Figure 3 This is a diagram illustrating the decomposition process of empirical pattern decomposition provided in an embodiment of the present invention;

[0112] Figure 4 This is a structural diagram of the LSTM model provided in the embodiments of the present invention;

[0113] Figure 5 This is an internal structure diagram of the LSTM model loop layer provided in the embodiment of the present invention;

[0114] Figure 6 This is a flowchart of the ARIMA model provided in the embodiments of the present invention;

[0115] Figure 7 This is a diagram illustrating the spatiotemporal information integration process provided in an embodiment of the present invention;

[0116] Figure 8 This is a diagram of the dilated causal convolution structure provided in an embodiment of the present invention;

[0117] Figure 9 This is a Transformer structure diagram provided in an embodiment of the present invention. Detailed Implementation

[0118] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0119] To address the problems existing in the prior art, the present invention provides a network traffic prediction method, system, medium, device, and terminal. The present invention will be described in detail below with reference to the accompanying drawings.

[0120] To enable those skilled in the art to fully understand how the present invention is specifically implemented, this section provides an explanatory description of the embodiments that expand upon the technical solutions of the claims.

[0121] like Figure 1 As shown, the network traffic prediction method provided in this embodiment of the invention includes the following steps:

[0122] S101, acquire historical network traffic data and related feature data, preprocess the data to obtain intrinsic modulus functions and residual subsequences, and calculate the sample entropy of each subsequence after decomposition;

[0123] S102, the intrinsic mode function is modeled using LSTM and ARIMA models respectively, and the data predicted by ARIMA and LSTM models is reconstructed.

[0124] S103. Construct and train a Transformer network traffic prediction model based on dilated causal convolution. Use the trained hybrid model to train and predict new data to obtain the predicted value of network traffic.

[0125] As a preferred embodiment, the network traffic prediction method provided by this invention includes the following steps:

[0126] Step S1: For a target network with multiple terminal nodes, acquire historical network traffic data and related feature data, and construct a network traffic feature matrix as model input;

[0127] Step S2: The acquired data is preprocessed using empirical mode decomposition to obtain multiple intrinsic mode functions and a residual subsequence, and the sample entropy of each subsequence after decomposition is calculated respectively.

[0128] Step S3: Use the LSTM model to model the intrinsic mode functions with high sample entropy, and use the ARIMA model to model the intrinsic mode functions with low sample entropy and the residual subsequences.

[0129] Step S4: Calculate the correlation coefficient between the time series of the target node and other nodes to represent the degree of spatial influence between nodes, and construct a Transformer network traffic prediction model based on dilated causal convolution.

[0130] Step S5: Train the Transformer network traffic prediction model based on dilated causal convolution, and reconstruct the data predicted by the ARIMA model and LSTM model.

[0131] Step S6: Use the trained hybrid model to predict the new data and obtain the predicted value of network traffic.

[0132] In step S1 of this embodiment of the invention, for a target network with multiple terminal nodes, historical network traffic data and related feature data are obtained, and a network traffic feature matrix is ​​constructed as model input, as follows:

[0133] (1) For a target network with multiple terminal nodes, obtain historical network traffic data, including: source IP address, destination IP address, transport layer protocol, source port and port; obtain relevant feature data, including: required bandwidth, actual bandwidth and number of connected terminals.

[0134] (2) Figure 2As shown, the acquired data is constructed into a feature matrix, represented as follows:

[0135]

[0136] Where N represents the number of nodes in the target network, T represents the number of time steps, and M represents the network traffic-related features.

[0137] (3) Divide the processed data into a training set, a validation set and a test set in a ratio of 7:1:2. The training set is used to build the model framework, the validation set is used to obtain the optimal parameters, and the test set is used to evaluate the accuracy of the traffic prediction model.

[0138] like Figure 3 As shown, step 2 of this embodiment of the invention, which involves preprocessing the collected data using empirical pattern decomposition, is as follows:

[0139] Empirical mode decomposition (EMD) breaks down collected network traffic data into local feature signals at different time scales containing the original signal. These signals are intrinsic mode functions (EMFs). The processed data includes many EMFs and a residual subsequence. The specific processing steps are as follows:

[0140] (1) Based on the upper and lower extreme points of the original signal X(t), the upper and lower envelopes are fitted by cubic spline curves respectively.

[0141] (2) Find the mean of the upper and lower envelopes and draw the mean envelope e. mean .

[0142] (3) Subtract the mean envelope e from the original signal X(t) mean The intermediate signal h is obtained. n .

[0143] (4) Determine whether the intermediate signal meets the two conditions of IMF. If it does, the signal is an IMF component. If it does not, perform the analysis in (1) to (4) again based on the signal. Obtaining IMF components usually requires several iterations.

[0144] (5) After obtaining the first IMF, subtract IMF1 from the original signal to obtain the new original signal. Then, through the analysis in (1) to (4), IMF2 can be obtained, and so on. When r n The screening process terminates when the function becomes monotonic or falls below a predefined threshold. EMD decomposition is then complete.

[0145] The calculation of the sample entropy of each subsequence after decomposition in step 2 of the present invention is as follows;

[0146] Sample entropy measures the complexity of a time series by quantifying the probability of new patterns emerging in the signal. The higher the probability of new patterns emerging, the greater the complexity of the sequence. Its definition is as follows:

[0147]

[0148] Among them, B m (r) is the probability that two sequences match at m points under a similarity tolerance of r, A m (r) is the probability that two sequences match at m+1 points.

[0149] Step 3 of this embodiment of the invention, which uses an LSTM model to model the intrinsic mode function with high sample entropy, is as follows:

[0150] (1) The entire neural network is divided into three layers: input layer, recurrent layer, and fully connected layer. The fully connected layer also serves as the output layer. The training of the neural network consists of multiple iterations, and each iteration involves two steps: forward propagation and backward propagation.

[0151] (2) Figure 4 As shown, during the forward propagation process of the loop layer, the input of the current time step and the output of the previous time step are used to perform calculations to obtain the output of the current time step, so as to model the relationship between input and output, as well as the historical correlation between outputs.

[0152] like Figure 5 As shown, the forward propagation process of the loop layer is as follows:

[0153] f (t) =σ(w f x (t) +u f h (t-1) +b f );

[0154] i (t) =σ(w i x (t) +u i h (t-1) +b i );

[0155]

[0156]

[0157] o (t) =σ(w o x (t) +u o h (t-1) +b o );

[0158] h (t) =o (t) *C (t) ;

[0159] Here, x is the input of the recurrent layer, h and C are two hidden states, and h is also the output of the recurrent layer; the subscript t represents the value at a certain time step, σ is the Sigmoid function; w, u and b are parameters, there are four sets in total, w is the relationship between the input and output, u is the historical correlation of the output, and b is the offset; all parameters are initialized to random values, and the hidden states are initialized to zero.

[0160] (3) During the backpropagation process of the recurrent layer, the gradient of each parameter and the hidden state is calculated in sequence. First, the hidden state of the last time step is calculated, then the parameter of the last time step is calculated, then the hidden state of the second to last time step is calculated, and so on. After obtaining the gradient of each parameter, the backpropagation is completed by subtracting the set multiple of the gradient of each parameter.

[0161] (4) The fully connected layer performs ordinary linear affine operations to compress the output dimension of the loop layer to make it consistent with the label.

[0162] like Figure 6 As shown, step 3 of this embodiment of the invention uses the ARIMA model to model the intrinsic mode function with low sample entropy and the residual subsequence, specifically as follows:

[0163] ARIMA is a model that transforms a non-stationary time series into a stationary time series and then regresses the dependent variable only on its lagged values ​​(q) and the present and lagged values ​​of the random error term.

[0164] AR stands for autoregression, and p is the autoregressive term.

[0165] MA is the moving average, and q is the number of terms in the moving average.

[0166] d represents the number of differences performed when the time series becomes stationary.

[0167] The step S4 provided in this embodiment of the invention calculates the correlation coefficient between the time series of the target node and other nodes to represent the degree of spatial influence between the nodes, as follows:

[0168] The correlation coefficient between the target node and other node time series is calculated using the Pearson correlation coefficient, which represents the degree of spatial influence between the nodes. The formula is as follows:

[0169]

[0170] Among them, Y *Y represents the network traffic of the target node at a certain time slice. i Let σ represent the network traffic of the i-th node in the same time slice, Cov(·) represent the covariance function, and σ represent the network traffic of the i-th node in the same time slice. * and σ i Indicates variance.

[0171] The training of the network traffic prediction model based on dilated causal convolution in step S5 of this embodiment of the invention is as follows:

[0172] (1) As Figure 7 As shown, the input 3D feature vector is expanded using a 1×1 convolution to improve the feature extraction capability for multiple nodes.

[0173] (2) Use 1×1 convolution again to reduce the dimension to 1, so that the input data can be integrated into a two-dimensional feature vector as the input of the extended causal convolutional network.

[0174] (3) Figure 8 As shown, an extended causal convolutional network is constructed, where the causal convolutional network abstracts the sequence problem as follows:

[0175]

[0176] Among them, X t Let T be a one-dimensional vector containing n features, and T be the time slice.

[0177] The definition of a dilated convolutional network is:

[0178]

[0179] Where s is the sequence element, d is the expansion coefficient, and sd·i describes the past direction, called * d To expand the convolution operator to distinguish it from normal convolution operations.

[0180] (4) Figure 9 As shown, the Transformer model is built as follows:

[0181] 1) The encoder consists of n sub-layers with the same structure. Each sub-layer includes a multi-head self-attention mechanism and a feedforward neural network. The output of each part adopts residual connection.

[0182] 2) The decoder consists of n sub-layers with the same structure. Each sub-layer includes a multi-head self-attention mechanism, a masked multi-head self-attention mechanism, and a feedforward neural network. The output of each part is connected by residual connections.

[0183] 3) The model is output after passing through a linear layer and a softmax function.

[0184] The various parts of the Transformer model are defined as follows:

[0185] The self-attention mechanism is defined as:

[0186]

[0187] Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, and d k Let Q be the number of columns in matrices Q and K.

[0188] Feedforward neural networks are defined as follows:

[0189]

[0190] Where X is the input to the feedforward neural network.

[0191] Residual connectivity is defined as:

[0192] LayerNorm(X+MultiHeadAttention(x));

[0193] LayerNorm(X+FeedForward(X));

[0194] Where X is the input of the multi-head self-attention mechanism or the feedforward neural network, MultiHeadAttention(x) is the output of the multi-head self-attention mechanism, and FeedForward(x) is the output of the feedforward neural network.

[0195] The model is built using the training set and the parameters are tuned using the validation set, thus optimizing the model.

[0196] The step S6 provided in this embodiment of the invention, which involves using the trained model to train new data to obtain predicted network traffic values, is as follows:

[0197] (1) Use the test set to evaluate the prediction accuracy of the Transformer network traffic prediction model based on dilated causal convolution.

[0198] (2) Collect new network traffic data to predict network traffic data.

[0199] The network traffic prediction system provided in this embodiment of the invention includes:

[0200] The data acquisition module is used to acquire historical network traffic data and related feature data for a target network with multiple terminal nodes, and construct a network traffic feature matrix as model input.

[0201] The data preprocessing module is used to preprocess the acquired data using empirical mode decomposition to obtain multiple intrinsic mode functions and a residual subsequence, and to calculate the sample entropy of each subsequence.

[0202] The intrinsic mode function modeling module is used to model intrinsic mode functions with high sample entropy using the LSTM model, and to model intrinsic mode functions with low sample entropy and residual subsequences using the ARIMA model.

[0203] The prediction model building module is used to calculate the correlation coefficient between the time series of the target node and other nodes, and to build a Transformer network traffic prediction model based on dilated causal convolution.

[0204] The model training and reconstruction module is used to train the Transformer network traffic prediction model based on dilated causal convolution and to reconstruct the data predicted by the ARIMA model and LSTM model.

[0205] The network traffic prediction module is used to train a pre-trained hybrid model on new data to predict network traffic.

[0206] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0207] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A network traffic prediction method, characterized in that, The network traffic prediction method includes: acquiring historical network traffic data and related feature data, preprocessing the data to obtain the intrinsic mode function and residual subsequences; calculating the sample entropy of the decomposed subsequences, modeling the intrinsic mode function using LSTM and ARIMA models respectively, and reconstructing the data predicted by ARIMA and LSTM models; constructing and training a Transformer network traffic prediction model based on dilated causal convolution, and using the trained hybrid model to train and predict new data to obtain the predicted value of network traffic. Network traffic prediction methods include the following steps: Step 1: For a target network with multiple terminal nodes, acquire historical network traffic data and related feature data, and construct a network traffic feature matrix as model input; Step 2: Preprocess the acquired data using empirical mode decomposition to obtain multiple intrinsic mode functions and a residual subsequence, and calculate the sample entropy of each subsequence after decomposition. Step 3: Use the LSTM model to model the intrinsic mode functions with high sample entropy, and use the ARIMA model to model the intrinsic mode functions with low sample entropy and the residual subsequences. Step 4: Calculate the correlation coefficient between the time series of the target node and other nodes to represent the degree of spatial influence between nodes, and construct a Transformer network traffic prediction model based on dilated causal convolution. Step 5: Train the Transformer network traffic prediction model based on dilated causal convolution, and reconstruct the data predicted by the ARIMA model and LSTM model. Step 6: Use the trained hybrid model to predict the new data and obtain the predicted value of network traffic.

2. The network traffic prediction method as described in claim 1, characterized in that, Step one, for a target network with multiple terminal nodes, involves acquiring historical network traffic data and related feature data, and constructing a network traffic feature matrix as model input, including: (1) For a target network with multiple terminal nodes, obtain historical network traffic data, including source IP address, destination IP address, transport layer protocol, source port and destination port; obtain relevant feature data, including required bandwidth, actual bandwidth and number of connected terminals; (2) Construct a feature matrix from the acquired data. The feature matrix is ​​represented as follows: ； Where N represents the number of nodes in the target network, T represents the number of time steps, and M represents the network traffic-related features; (3) Divide the processed data into a training set, a validation set and a test set in a ratio of 7:1:

2. The training set is used to build the model framework, the validation set is used to adjust the parameters, and the test set is used to evaluate the accuracy of the traffic prediction model.

3. The network traffic prediction method as described in claim 1, characterized in that, Step two involves preprocessing the acquired data using empirical pattern decomposition, including: Empirical mode decomposition (EMD) breaks down collected network traffic data into local feature signals at different time scales containing the original signal. These local feature signals are intrinsic mode functions (IMFs). The processed data includes many IMFs and a residual subsequence. The specific processing steps are as follows: (1) Based on the upper and lower extreme points of the original signal X(t), fit the upper and lower envelopes using cubic spline curves respectively; (2) Find the mean of the upper and lower envelopes and draw the mean envelope e. mean ; (3) Subtract the mean envelope e from the original signal X(t) mean The intermediate signal h is obtained. n ; (4) Determine whether the intermediate signal meets the two conditions of IMF. If it does, the signal is an IMF component. If it does not, based on the signal, repeat the analysis of steps (1) to (4) and obtain the IMF component by several iterations. (5) After obtaining the first IMF, subtract IMF1 from the original signal to obtain the new original signal; then, through the analysis in steps (1) to (4), obtain IMF2; and so on, when r n When the function becomes monotonic or falls below a predefined threshold, the screening process terminates, and the EMD decomposition is completed. The sample entropy of each decomposed subsequence is calculated separately, including: Sample entropy measures the complexity of a time series by quantifying the probability of new patterns emerging in the signal. The higher the probability of a new pattern emerging, the greater the complexity of the sequence, as defined below: ； in, It is the probability that two sequences match at m points under a similarity tolerance of r. It is the probability that two sequences match m+1 points.

4. The network traffic prediction method as described in claim 1, characterized in that, Step three involves using an LSTM model to model the intrinsic mode functions with high sample entropy, and using an ARIMA model to model the intrinsic mode functions with low sample entropy and the residual subsequences. (1) The entire neural network is divided into three layers: the input layer, the recurrent layer, and the fully connected layer. The fully connected layer also serves as the output layer. The training of the neural network consists of multiple iterations, and each iteration includes two steps: forward propagation and backward propagation. (2) During the forward propagation of the loop layer, the input of the current time step and the output of the previous time step are used to perform calculations to obtain the output of the current time step, which is used to model the relationship between the input and the output, as well as the historical correlation between the outputs; The forward propagation process of the recurrent layer is as follows: ； ； ； ； ； ； Here, x is the input of the recurrent layer, h and C are two hidden states, and h is also the output of the recurrent layer; the subscript t represents the value at a certain time step, σ is the Sigmoid function; w, u and b are parameters, there are four sets in total, w is the relationship between input and output, u is the historical correlation of output, and b is the offset; all parameters are initialized to random values, and the hidden states are initialized to zero. (3) During the backpropagation process of the recurrent layer, the gradient of each parameter and the hidden state is calculated in turn; after calculating the hidden state of the last time step, the parameter of the last time step is calculated; then the hidden state of the second to last time step is calculated, and so on; after obtaining the gradient of each parameter, the gradient is subtracted by a set multiple to complete the backpropagation. (4) The fully connected layer performs a linear affine operation to compress the output dimension of the loop layer so that the output dimension of the loop layer is consistent with the label; ARIMA is a model that transforms a non-stationary time series into a stationary time series and then regresses the dependent variable only on the lagged value q and the present value and lagged value of the random error term. Where AR is autoregressive, p is the autoregressive term; MA is moving average, q is the number of moving average terms; and d is the number of differencing operations performed when the time series becomes stationary. Step four involves calculating the correlation coefficients between the time series of the target node and other nodes, representing the degree of spatial influence between the nodes, including: The correlation coefficient between the target node and other node time series is calculated using the Pearson correlation coefficient, which represents the degree of spatial influence between the nodes. The calculation formula is as follows: ； Among them, Y ∗ Y represents the network traffic of the target node at a certain time slice. i Let σ represent the network traffic of the i-th node in the same time slice, Cov(·) represent the covariance function, and σ represent the network traffic of the i-th node in the same time slice. ∗ and σ i Indicates variance.

5. The network traffic prediction method as described in claim 1, characterized in that, Step five involves training a Transformer network traffic prediction model based on dilated causal convolutions, which includes: (1) Expand the feature dimension of the input three-dimensional feature vector using a 1×1 convolution; (2) Use 1×1 convolution again to reduce the dimension to 1, integrate the input data into a two-dimensional feature vector, and use it as the input of the extended causal convolutional network; (3) Construct an extended causal convolutional network; wherein, the causal convolutional network abstracts the sequence problem as follows: ； Where, x t Let T be a one-dimensional vector containing n features, and T be the time slice. The definition of a dilated convolutional network is: ； Where s is the sequence element, d is the expansion coefficient, and sd·i describes the past direction, called * d To expand the convolution operator to distinguish it from normal convolution operations; (4) Build the Transformer model, as follows: 1) The encoder consists of n sub-layers with the same structure. Each sub-layer includes a multi-head self-attention mechanism and a feedforward neural network. The output of each part is connected by residuals. 2) The decoder consists of n sub-layers with the same structure. Each sub-layer includes a multi-head self-attention mechanism, a masked multi-head self-attention mechanism, and a feedforward neural network. The output of each part is connected by residuals. 3) The model passes through a linear layer and outputs a softmax function; The various parts of the Transformer model are defined as follows: The self-attention mechanism is defined as: ； Where Q, K, and V represent the query matrix, key matrix, and value matrix, respectively, and d k Let Q and K be the number of columns. Feedforward neural networks are defined as follows: ； ； Where X is the input to the feedforward neural network; Residual connectivity is defined as: ； ； Where X is the input of the multi-head self-attention mechanism or the feedforward neural network, MultiHeadAttention(x) is the output of the multi-head self-attention mechanism, and FeedForward(x) is the output of the feedforward neural network; Use the training set to build the model, use the validation set to tune the parameters, and then tune the model to its optimal state. Step six involves training the pre-trained model on new data to obtain network traffic predictions. This includes: evaluating the prediction accuracy of the Transformer network traffic prediction model based on dilated causal convolution using a test set; and collecting new network traffic data for network traffic prediction.

6. A network traffic prediction system applying the network traffic prediction method as described in any one of claims 1 to 5, characterized in that, Network traffic prediction systems include: The data acquisition module is used to acquire historical network traffic data and related feature data for a target network with multiple terminal nodes, and construct a network traffic feature matrix as model input. The data preprocessing module is used to preprocess the acquired data using empirical mode decomposition to obtain multiple intrinsic mode functions and a residual subsequence, and to calculate the sample entropy of each subsequence. The intrinsic mode function modeling module is used to model intrinsic mode functions with high sample entropy using the LSTM model, and to model intrinsic mode functions with low sample entropy and residual subsequences using the ARIMA model. The prediction model building module is used to calculate the correlation coefficient between the time series of the target node and other nodes, and to build a Transformer network traffic prediction model based on dilated causal convolution. The model training and reconstruction module is used to train the Transformer network traffic prediction model based on dilated causal convolution and to reconstruct the data predicted by the ARIMA model and LSTM model. The network traffic prediction module is used to train a pre-trained hybrid model on new data to predict network traffic.

7. A computer device, characterized in that, The computer device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the network traffic prediction method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the network traffic prediction method as described in any one of claims 1 to 5.

9. An information data processing terminal, characterized in that, The information data processing terminal is used to implement the network traffic prediction system as described in claim 6.

Images