Network traffic matrix prediction method based on multi-scale convolution and attention collaborative enhancement LSTM

Abstract

The invention provides a network traffic matrix prediction method based on multi-scale convolution and attention collaborative enhancement LSTM (Long Short Term Memory), and solves the problems that an existing traffic matrix prediction model is insufficient in multi-scale feature capture, weak in space-time correlation modeling and poor in generalization. A dynamic multi-scale convolution module (adopting a 3 * 3 / 5 * 5 / 7 * 7 two-dimensional convolution kernel) is used to extract local fine-grained association and global coarse-grained trend of a two-dimensional traffic matrix, deep fusion of LSTM time sequence features and two-dimensional multi-scale spatial features is realized in combination with a cross attention mechanism, and time-space cross-dimension association is enhanced. And meanwhile, the problem of gradient disappearance of the deep network is relieved by relying on residual connection, and topology independence is realized based on a flow-by-flow method so as to adapt to traffic matrixes of different dimensions. The method specifically comprises the following steps: performing stream-by-stream division and normalization preprocessing on an original two-dimensional traffic matrix, and after processing by modules such as dynamic multi-scale convolution, cross attention enhancement LSTM, residual block and gating screening and the like, outputting a single OD stream predicted value and reconstructing a complete traffic matrix; and integrating objective functions of the encoder and the decoder, taking the MSE as a loss function, and adopting an Adam optimizer to iteratively update model parameters until the model converges. Experiments show that the MAE and the MSE of the method on real data sets such as Ailene, GEANT and the like are better than those of mainstream models such as SVR, LSTM, MTGNN and the like, the reconstruction precision of a high-dimensional traffic matrix is particularly remarkable, and a reliable basis can be provided for network congestion control and resource optimization.

Description

Technical Field

[0001] This invention belongs to the field of complex networks and relates to a network traffic matrix prediction method based on multi-scale convolution and attention-based long short-term memory (LSTM) networks. It is applicable to traffic distribution prediction in complex network scenarios such as communication networks and data center networks, and can support network congestion control, resource optimization and allocation, and efficient network management. Background Technology

[0002] A network traffic matrix can comprehensively describe the traffic distribution between different source-destination node pairs (OD pairs) in a network. Accurate traffic matrix prediction is one of the core technologies for achieving intelligent network management—predicting future traffic distribution through historical traffic data helps operators identify traffic hotspots in advance, optimize link resource allocation, and improve service quality. However, network traffic matrix prediction faces three major challenges:

[0003] 1. Difficulty in capturing multi-scale features: Network traffic exhibits both short-term fine-grained fluctuations (such as burst traffic) and long-term overall trends (such as periodic traffic changes). Traditional models (such as ARMA and ARIMA) can only handle linear stationary data and cannot effectively cover multi-scale dynamic features.

[0004] 2. Insufficient spatiotemporal correlation modeling: The flow matrix contains sequential dependencies in the time dimension and OD pair correlation characteristics in the spatial dimension. Existing machine learning methods (such as SVR and KNN) are difficult to capture high-order abstract spatiotemporal features. Although hybrid models (such as CNN-LSTM) combine convolution and temporal modeling capabilities, their characterization of cross-dimensional correlations is still weak.

[0005] 3. Imbalance between model complexity and generalization: Models such as Transformer capture global dependencies through self-attention mechanisms, but their computational complexity is extremely high when processing high-dimensional data, making it difficult to meet the needs of real-time prediction. At the same time, most models rely on network topology information, but in real-world scenarios, the topology is often missing or dynamically changing, resulting in poor generalization ability of the model in heterogeneous networks.

[0006] In existing technologies, the FBF-LSTM model based on the "flow-by-flow" method achieves topology independence and reduces the dependence on topological information by predicting the flow reconstruction matrix of each OD separately. However, it still has limitations: relying solely on the temporal modeling capability of LSTM, it is insufficient in capturing local feature fluctuations and sudden flow; it does not consider the fusion of multi-scale temporal features, and cannot fully characterize the multi-scale evolution of flow; and it lacks the ability to explore potential correlations between different OD flows, thus limiting its feature representation capability.

[0007] To address this, this invention proposes an LSTM network (F-DMSRAL) that integrates dynamic multi-scale convolution, cross-attention mechanism, and residual learning. By coordinating multiple modules to enhance feature capture capabilities, it ensures prediction accuracy while balancing model generalization and computational efficiency, thus solving the problems of missing multi-scale features, weak spatiotemporal correlation, and poor generalization in existing methods. Summary of the Invention

[0008] The core objective of this invention is to provide a high-precision, high-generalization network traffic matrix prediction method. This method can predict traffic in the short and long term (1-3 time steps ahead). It extracts multi-scale traffic features through dynamic multi-scale convolution, strengthens spatiotemporal correlation through cross attention, alleviates gradient vanishing through residual connections, and achieves topology-independent traffic prediction by combining the flow-by-flow method.

[0009] The specific technical solution consists of the following three parts:

[0010] (1) Preprocessing of two-dimensional network traffic matrix

[0011] To adapt the model input, preprocessing is performed to address the dimensionality and data distribution of the original two-dimensional network traffic matrix.

[0012] ①Structured representation of two-dimensional matrices

[0013] Suppose there are N nodes in the network, then the two-dimensional traffic matrix can be represented as follows: Where X t (i,j) represents the OD pair flow value from node i to node j at time t; the flow prediction process for the next 1-3 time steps can be represented as:

[0014]

[0015] in, The prediction target of this invention is represented as a flow matrix for the next 1-3 time points; M represents the entire prediction model, {X} t-L+1 ,X t-L+2 ,…,X t} represents the historical traffic sequence, and L is the length of the historical sequence.

[0016] ② Flow-based classification and normalization

[0017] First, perform flow-by-flow partitioning, splitting the two-dimensional network traffic matrix into N*N OD pairs {x1,x2,…,x M (Each sequence corresponds to the historical flow of an OD pair), and subsequent two-dimensional matrix prediction is achieved through "single OD flow prediction - matrix reconstruction"; then, each OD flow sequence is normalized to eliminate the influence of dimensions, specifically by dividing the flow value by the maximum value in the same flow. The normalization formula is:

[0018]

[0019] in, It is the OD pair flow value from node i to node j at time t. This is to find the maximum value in the OD flow with source node i.

[0020] (2) Construction of dynamic multi-scale convolution module

[0021] To address the spatial multi-scale characteristics of the two-dimensional flow matrix (such as the fine-grained association of local OD to clusters and the coarse-grained trend of the global matrix), this invention designs a dynamic weight allocation structure using 3×3, 5×5, and 7×7 two-dimensional convolutional kernels.

[0022] Convolutional kernels are the core tool for automatic feature extraction in neural networks. Essentially, they are adjustable weight matrices used to extract features from different levels of data. The convolution process can be represented as:

[0023]

[0024] Wherein, the convolution result Y[i] is the feature map at the i-th time step. Let X[i] represent the input sequence from X[i] to X[i+L-1] with a length of L, where X[i+k] is the value of X at the (i+k)th time step, W[k] represents the corresponding weight value, and b is used as a bias term, which is set appropriately to better simulate the true value.

[0025] The preprocessed OD flow sequence is processed in parallel using three different 2D convolutional kernels. The 3×3 2D convolutional kernel is used to capture local fine-grained features (such as the short-term fluctuation correlation of OD flow at adjacent time points); the 5×5 2D convolutional kernel is used to characterize medium-term features (such as the fluctuation trend correlation of OD flow at multiple consecutive time points); and the 7×7 2D convolutional kernel is used to extract global coarse-grained features (such as the overall change trend of OD flow over a long period).

[0026] These three convolution kernels are set to k i = {k3, k5, k7}, where i is the size of the convolutional kernel. Different kernel sizes can capture different receptive fields and extract features of different granularities. For each convolutional kernel, the feature map can be calculated as follows:

[0027] Y i =ReLU(X*W i (4)

[0028] Among them, Y i Represents the corresponding convolution kernel k i Feature maps, where X is the input data, W irepresents the convolution kernel parameters, * indicates a convolution operation with padding, and the ReLU activation function introduces non-linear features and suppresses negative responses; its expression is:

[0029] ReLU(x) = max(0,x) (5)

[0030] The function max(0,x) retrieves the maximum value of 0 and x. When the input x is greater than 0, the output is x itself; when the input x is less than or equal to 0, the output is 0.

[0031] α=Softmax(W2·ReLU(W1·GAP(X))) (6)

[0032] Where α is the dynamic weight matrix, indicating the importance weight of each convolutional kernel; GAP (Global Average Pooling) compresses the sequence dimension and extracts contextual features; W1 and W2 map the global features onto various convolutional kernels to obtain weight scores; the Softmax function then performs a normalization operation, transforming the output value range to between 0 and 1, and making the sum of all elements equal to 1. The Softmax function can be expressed as:

[0033]

[0034] Finally, multi-scale features are aggregated using a weight matrix, enabling the model to select the optimal feature combination. Multi-scale feature fusion is represented as:

[0035]

[0036] Where O represents the result of multi-scale fusion, e represents element-wise multiplication, α[:,i] represents the weight value of the i-th convolutional kernel branch, and F is the feature map output by the i-th convolutional kernel branch.

[0037] (3) Construction of Cross-Attention Enhanced LSTM Module

[0038] Attention mechanisms are neural network structures that mimic the selective nature of human attention, allowing models to focus on relatively crucial parts when processing data. General attention mechanisms are primarily used to address information interaction within a single sequence. This invention, to strengthen the correlation between "spatial multi-scale features and temporal series features" in the network traffic matrix (the correlation between the temporal series dependencies captured by LSTM and the spatial multi-scale features extracted by two-dimensional convolution), designs a cross-attention module capable of handling cross-sequence information interaction, achieving deep fusion of LSTM output and two-dimensional multi-scale features:

[0039] Adding temporal and positional information to the input sequence compensates for the lack of inherent positional awareness in the attention mechanism itself; this is achieved by incorporating positional encoding. The process is as follows:

[0040] Q′=Q+P (9)

[0041] K′V′=KV+P (10)

[0042] Where P is the learnable positional encoding parameter, Q is the query sequence, and KV is the key-value sequence. Q′ and K′V′ are sequences with added positional encoding. Subsequently, multiple sets of attention weights are computed in parallel to capture the feature interactions between different sequences, uncovering deeper implicit relationships between them. The attention computation process can be represented as:

[0043]

[0044] S=Q′K′ T (12)

[0045]

[0046] A = Softmax(S scaled (14)

[0047] F ca =AV′ (15)

[0048] First, calculate the similarity S between query Q′ and key K′, and divide by... S is obtained by scaling. scaled This is to prevent the gradient from vanishing due to an excessively large dot product. Then, a softmax normalization operation is performed to obtain the attention weight matrix A, which represents the dynamic attention level of query Q′ to key K′. Finally, the value sequence V′ is weighted and summed using the attention weight matrix A to obtain the module output F. ca The entire process set is represented by formula (11).

[0049] (4) Construction of residual block and two-dimensional matrix prediction module

[0050] ① Residual connection design

[0051] To alleviate the gradient vanishing problem in deep networks, this invention uses the temporal features output by LSTM as residual terms and fuses them with cross-attention enhanced features through element-wise addition, thus preserving the original temporal information.

[0052] F res =F LSTM +F ca (16)

[0053] Among them, F res These are the features after residual fusion.

[0054] ② Gating Mechanism and Dimension Mapping

[0055] Gating mechanism screening: Introducing a channel-level gating unit for F res Feature filtering is performed to enhance key spatiotemporal features and suppress redundant information.

[0056] G=σ(W g ·F res +b g (17)

[0057] F g =Ge F res (18)

[0058] Among them, W g and b g Here, is the gating parameter, e represents element-wise multiplication, and F... g These are the features after gating and filtering.

[0059] Dimension mapping: F is mapped through a fully connected layer. g The feature dimension is mapped from L (historical sequence length) to 1 (the flow value dimension at a single future moment):

[0060] F fc =W fc2 ·F g +b fc2 (19)

[0061] Among them, W fc2 and b fc2 For fully connected layer parameters, F fc These are the predicted features for a single OD flow after dimension mapping.

[0062] ③ Reconstruction of two-dimensional flow matrix

[0063] The predicted value of a single OD flow is output through a linear layer.

[0064]

[0065] Among them, W l and b l For linear layer parameters, the predicted values ​​of all N*N OD flows are rearranged according to the original OD pair positions to reconstruct the complete network traffic matrix.

[0066] (5) Model Training and Optimization

[0067] Initialize model parameters; load preprocessed OD flow sequence data and input them into the model in batches; sequentially pass the data through a 2D dynamic multi-scale convolutional module, LSTM layer, cross-attention module, residual block, gated unit, and dimension mapping layer to output single OD flow prediction values; calculate the MSE loss and update all model parameters through backpropagation; evaluate model performance on the validation set after each training round, and stop training if the loss does not decrease for 10 consecutive validation rounds; after training, reconstruct the 2D flow matrix for all OD flow prediction values ​​to complete the prediction task. Attached Figure Description

[0068] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following figures are provided for illustration:

[0069] Figure 1 This is a flowchart of the network traffic matrix prediction method proposed in this invention (F-DMSRAL model structure diagram).

[0070] Figure 2 The results of MSE experiments on the Abilene dataset under different rounds and batch sizes are shown.

[0071] Figure 3 The results of MAE experiments on the Abilene dataset under different rounds and batch sizes are shown.

[0072] Figure 4 The results of MSE experiments on the GEANT dataset under different rounds and batch sizes are shown.

[0073] Figure 5 The results of MAE experiments on the GEANT dataset under different rounds and batch sizes are shown.

[0074] Figure 6 This is a comparison chart of the actual and predicted flow rates on the 118th OD flow in the Abilene dataset, based on the present invention.

[0075] Figure 7 This is a comparison chart of the actual and predicted flow rates on the 469th OD flow in the GEANT dataset, based on the present invention.

[0076] Figure 8 The results of the MSE experiment are presented for evaluation across datasets.

[0077] Figure 9 The results of the MAE experiment are presented for evaluation across datasets.

[0078] Figure 10 The figure shows the MSE experimental results of the ablation experiment on the Abilene dataset.

[0079] Figure 11 The image shows the MAE experimental results of the ablation experiment on the Abilene dataset.

[0080] Figure 12 The figure shows the MSE experimental results of the ablation experiment on the GEANT dataset.

[0081] Figure 13 The image shows the MAE experimental results of the ablation experiment on the GEANT dataset. Detailed Implementation

[0082] To make the technical solution, advantages and objectives of the present invention clearer, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0083] To evaluate the advantages of this invention compared to other network traffic matrix prediction methods, publicly available datasets were obtained from the internet, and this invention was compared with classic network traffic matrix prediction methods:

[0084] (1) Public datasets

[0085] The following section explains the real-world network dataset selected for network traffic matrix prediction.

[0086] Table 1 Information on Real Network Datasets

[0087] Dataset Number of nodes Number of links OD Stream Quantity Sampling interval Data quantity Abilene 12 15 144 5min 48096 GEANT 23 38 529 15min 10772

[0088] Abilene and GEANT provide traffic data for the U.S. Research and Education Network and the European Research and Education Network.

[0089] (2) Experimental setup

[0090] The experiment was conducted on a computer equipped with an Intel Xeon W-2133 CPU, an NVIDIA-GeForce RTX 2080 Ti graphics card, and 93Gi memory.

[0091] All models used the Adam optimizer with a loss function and a learning rate of 0.0001, and the input historical sequence length was 12. To avoid overfitting, a dropout rate of 0.5 was used uniformly. Baseline models were tested for 200 epochs with a batch size of 64. LSTM and GRU were used as lightweight models, with their epochs set to 50, to obtain their best prediction results for model comparison. To achieve optimal results for the model of this invention, experimental graphs of epoch size and batch size were provided, as shown below. Figure 2 , Figure 3 , Figure 4 , Figure 5 As shown, the batch size of both the Abilene and GEANT datasets is 5000, and the round sizes are 190 and 170, respectively.

[0092] The entire dataset is divided into training, validation, and test sets according to a ratio of 70%, 10%, and 20%, respectively.

[0093] (2) Evaluation indicators

[0094] This invention selects two commonly used evaluation metrics in network traffic matrix prediction methods: MAE (Mean Absolute Error) and MSE (Mean Squared Error). The calculation formulas for these metrics are as follows:

[0095]

[0096] Among them, y i These are real traffic data. These are the model predictions. m is the number of predicted flows. These metrics mean that the lower the MSE and MAE values, the better the model performance.

[0097] (3) Benchmark Algorithm

[0098] SVR: A network traffic matrix prediction method based on the principle of support vector machine. It is a machine learning algorithm that constructs the optimal regression hyperplane in a high-dimensional feature space to achieve accurate prediction of continuous variables.

[0099] LSTM: A special variant of recurrent neural network (RNN) that solves the gradient vanishing / exploding problem when traditional RNNs process long sequences by using three gating structures: input gate, forget gate, and output gate, as well as independent cell states. It can effectively capture long-term dependencies in time series data.

[0100] GRU is a simplified and improved version of LSTM, containing only two gating mechanisms: update gate and reset gate. While retaining the ability to capture long sequence dependencies, it reduces the number of parameters and computation steps, thereby reducing model complexity and improving training efficiency.

[0101] DCRNN: A deep learning model that combines diffusing convolution to capture spatial dependencies of graph structures with gated recurrent units to learn temporal dependencies, for prediction of graph-structured temporal data such as network traffic.

[0102] LSTNet: A deep learning model for multivariate time series prediction that combines convolutional layers to extract short-term local dependencies between variables, recurrent layers (including recurrent skip structures) to capture long-term temporal dependencies, and incorporates autoregressive components to solve the scale insensitivity problem of neural networks.

[0103] MTGNN: An end-to-end deep learning model for multivariate time series prediction that adaptively extracts unidirectional hidden dependencies between variables through graph learning layers, captures spatial dependencies by combining hybrid jump propagation layers and extended inception layers to capture temporal dependencies.

[0104] (4) Experimental results on real network datasets

[0105] Table 2 Comparison of NMI of each algorithm on real datasets

[0106]

[0107]

[0108] To more intuitively demonstrate the effectiveness of this invention in prediction accuracy, the prediction results are visualized by randomly selecting OD flows from two datasets to generate a comparison chart of actual and predicted traffic. For example... Figure 6 and Figure 7 As shown, the present invention has excellent prediction performance on all datasets.

[0109] (5) Evaluation of experimental results across data

[0110] This invention underwent cross-dataset model evaluation to demonstrate that it can be applied to other networks with different topologies.

[0111] To better evaluate the experiments, two more datasets have been added: Germany and Nobel, in addition to Abilene and GEANT. Germany

[0112] There are 50 nodes, while Nobel has 17 nodes. Both have a 5-minute time sampling interval, 288 traffic matrices, and a data sampling duration of one day.

[0113] The cross-dataset evaluation is shown in the figure. In the figure, the horizontal axis represents the dataset used for evaluation; to maintain consistency with the previous dataset split, the test dataset accounts for 20%. The vertical axis represents the datasets used for cross-dataset evaluation. The datasets on the horizontal axis are used as the base value, and the data on the diagonal are set to 100%. The cross-dataset evaluation result is divided by the base value, and the ratio represents the applicability of the evaluation model.

[0114] from Figure 8 , Figure 9 It can be seen that the present invention has strong generalization ability, and even without the topology structure, it can achieve good performance evaluation across datasets.

[0115] (5) Ablation test

[0116] To verify whether the innovative module of this invention is optimal, an ablation experiment was conducted, such as... Figure 10 , Figure 11 , Figure 12 , Figure 13As shown, D represents the dynamic multi-scale convolution module, C represents the cross-attention module, R represents the residual module, and the others represent models that combine two of these modules. It can be seen that this invention achieves optimal performance across all datasets.

[0117] All the experiments above demonstrate that this invention, for two-dimensional network traffic matrix prediction, comprehensively captures both fine-grained local correlations and coarse-grained global trends of the matrix through parallel processing of 3×3 / 5×5 / 7×7 two-dimensional convolutional kernels, solving the problem of missing two-dimensional features in traditional models. It leverages a cross-attention mechanism to achieve deep fusion of LSTM temporal features and two-dimensional multi-scale spatial features, strengthening the cross-dimensional correlation between "time and space" to enhance feature representation capabilities. It achieves topology independence through a flow-by-flow method, adapting to different network traffic matrix datasets, and optimizes computational complexity through residual connections and dynamic convolution to meet real-time requirements. On real datasets such as Abilene and GEANT, its network traffic matrix prediction MAE and MSE are superior to mainstream models such as SVR, LSTM, and MTGNN, especially showing significant improvement in accuracy for high-dimensional matrix reconstruction such as GEANT 23×23. This provides a more reliable two-dimensional traffic distribution basis for network resource scheduling and congestion control. Overall, it outperforms existing technologies in feature capture, correlation modeling, generalization efficiency, and prediction accuracy.

[0118] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any transformations or substitutions that can be conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of the present invention.

Claims

1. A network traffic matrix prediction method based on multi-scale convolution and attention collaborative enhancement LSTM, characterized in that, The multi-scale features of the network traffic matrix are extracted through a dynamic multi-scale convolution module, the association between the LSTM time sequence features and the spatial multi-scale features is strengthened by combining a cross-attention mechanism, the gradient disappearance problem of the deep network is relieved by relying on a residual connection, and the traffic prediction is realized without topology based on the per-flow method, and finally the traffic matrix of 1-3 time steps in the future is output; The specific steps include: performing per-flow division and normalization preprocessing on the original traffic matrix, processing through a dynamic multi-scale convolution, a cross-attention enhanced LSTM, a residual block and a gating filtering module, outputting a single OD flow prediction value and reconstructing a complete traffic matrix, and iteratively training the model until convergence through an MSE loss function and an Adam optimizer.

2. The network traffic matrix preconditioner of claim 1, wherein, The specific steps include: Let N be the number of nodes in the network, the two-dimensional flow matrix can be expressed as where X t (i,j) represents the OD pair flow value from node i to node j at time t; the flow prediction process for the next 1-3 time points can be expressed as: wherein, is the prediction target of the present application, representing the flow matrix of the future 1-3 time points; M represents the entire prediction model, {X t-L+1 ,X t-L+2 ,…,X t} is the historical flow sequence, and L is the length of the historical sequence. Then, perform flow-by-flow partitioning, splitting the network traffic matrix into N*N OD pairs {x1,x2,…,x M (Each sequence corresponds to the historical flow of an OD pair), and then two-dimensional matrix prediction is achieved through "single OD flow prediction - matrix reconstruction"; next, each OD flow sequence is normalized to eliminate the influence of dimensions, specifically by dividing the flow value by the maximum value in the same flow, and the normalization formula is: wherein, is the OD pair flow value from node i to node j at time t, is to find the maximum value of OD flow from source node i.

3. The dynamic multi-scale convolution module of claim 1, wherein, Parallel processing is performed on the 3*3, 5*5 and 7*7 two-dimensional convolution kernels, and multi-scale feature fusion is realized through dynamic weight distribution, which specifically includes: For the spatial multi-scale features of the two-dimensional traffic matrix (such as the fine-grained association of local OD pairs to clusters and the coarse-grained trend of the global matrix), the application designs a dynamic weight distribution structure of 3*3, 5*5 and 7*7 two-dimensional convolution kernels. The convolution kernel is a core tool for automatic feature extraction in a neural network, which is essentially a tunable weight matrix used to mine features at different levels from data, and its convolution process can be represented as: wherein the convolution result Y[i] is a feature map at the i-th time step, represents an input sequence of length L from X[i] to X[i+L-1], X[i+k] is the value of X at the i+k-th time step, W[k] represents the corresponding weight value, and b is a bias term, which is set reasonably to better simulate the true value. Parallel processing is performed on the 3*3, 5*5 and 7*7 two-dimensional convolution kernels, and multi-scale feature fusion is realized through dynamic weight distribution, which specifically includes: The 7*7 two-dimensional convolution kernel is used to extract global coarse-grained features (such as the overall change trend of the OD flow in a long period). The three convolution kernels are set as k i = {k3, k5, k7}, where i is the size of the convolution kernel, and different sizes of the convolution kernel can obtain different receptive fields and extract features of different granularities. For each convolution kernel, the calculation of the feature map can be represented as: Y i = ReLU(X * W i ) (4) where Y i represents the feature map corresponding to the convolution kernel k i , X is the input data, W i is the convolution kernel parameter, * is the convolution operation with padding, and the activation function ReLU introduces nonlinear features and suppresses negative responses, and its expression is: ReLU(x)=max(0,x) (5) Where max(0,x) is to obtain the maximum of 0 and x, when the input x is greater than 0, the output is x itself; when the input x is less than or equal to 0, the output is 0. α=Softmax(W2·ReLU(W1·GAP(X))) (6) Where alpha is a dynamic weight matrix indicating the importance weight of each convolution kernel; GAP (Global Average Pooling) compresses the sequence dimension and extracts context features; W1 and W2 map the global features to various convolution kernels to obtain weight scores; the Softmax function then performs a normalization operation to convert the range of output values to 0 to 1, and the sum of all elements is equal to 1. The Softmax function can be represented as: Finally, the multi-scale features are aggregated through the weight matrix, so that the model can select the optimal feature combination. The multi-scale feature fusion is represented as: Where O represents the result of multi-scale fusion, e represents the element-wise product, alpha[:,i] represents the weight value of the i-th convolution kernel branch, and F is the feature map output by the i-th convolution kernel branch.

4. The cross-attention augmented LSTM module construction of claim 1, wherein, Feature fusion is realized through position encoding and cross-sequence attention calculation, which specifically includes: Attention mechanism is a kind of neural network structure that simulates the selective characteristics of human attention, which can make the model focus on the relatively key part when processing data. Generally, attention mechanism is mainly used to solve the problem of information interaction within a single sequence. In order to strengthen the correlation between the "spatial multi-scale feature-time sequence feature" of the network flow matrix (the correlation between the time sequence dependence captured by LSTM and the spatial multi-scale feature extracted by two-dimensional convolution), a cross-attention module capable of processing cross-sequence information interaction is designed to realize the deep fusion of the LSTM output and the two-dimensional multi-scale feature. Time position information is added to the input sequence to make up for the lack of inherent position perception of the attention mechanism itself, which is realized by incorporating position encoding. The process is as follows: Q' = Q + P (9) K'V' = KV + P (10) Where, P is the learnable position encoding parameter, Q is the query sequence, KV is the key-value sequence. Q' and K'V' are sequences after adding position encoding. Then, the feature interaction of different sequences is captured by calculating multiple sets of attention weights in parallel to mine deeper implicit relationships between sequences. The attention calculation process can be represented as: S = Q'K' T (12) A = Softmax(S scaled ) (14) F ca = AV' (15) First, compute the similarity S between query Q' and key K', divide by Scaling to get S scaled This is to prevent the gradient vanishing caused by the dot product result being too large. Then, the Softmax normalization operation is performed to get the attention weight matrix A, which represents the dynamic attention degree of the query Q' to the key K'. Finally, the value sequence V' is weighted and summed using the attention weight matrix A to get the module output F ca The whole process is represented as formula (11).

5. The residual block and gated sieve module construction of claim 1, wherein, Including residual connection, gating mechanism and dimension mapping, specifically including: In order to alleviate the gradient vanishing problem in deep network, the time sequence feature output by LSTM is taken as a residual term, which is element-wise added to the cross-attention enhanced feature to preserve the original time sequence information: F res = F LSTM + F ca (16) where F res is the residual fused feature. Gating mechanism screening: Introducing channel-level gating units to F res Feature screening, strengthening key spatio-temporal features and suppressing redundant information: G = σ(W g • F res + b g ) (17) F g = Ge F res (18) where W g and b g are gating parameters, e is an element-wise multiplication, and F g is the gated filtered feature. Dimension mapping: F g 's feature dimension from L (length of historical sequence) to 1 (dimension of traffic value at single future time instant) through a fully connected layer: F fc = W fc2 • F g + b fc2 (19) where W fc2 and b fc2 are the fully connected layer parameters, F fc is the dimension-mapped single-OD flow prediction feature. Network traffic matrix reconstruction outputs a prediction of individual OD flows through a linear layer where W l and b l are linear layer parameters, the predicted values for all N*N OD flows are rearranged by the original OD pair positions to reconstruct the complete network flow matrix. 6.The model training and convergence determination according to claim 1, wherein, MSE is used as the loss function: Calculate the MSE loss, update all model parameters through back propagation; evaluate the model performance on the validation set after each training round, if the validation loss does not decrease for 10 consecutive rounds, stop training; after training, reconstruct the two-dimensional flow matrix for all OD flow prediction values to complete the prediction task.

Images