A network traffic prediction method based on space-time informer

Abstract

A network traffic prediction method based on space-time Informer extracts the spatial features and time features of traffic data through a spatial feature module and a time feature module. The spatial feature module uses a two-dimensional convolutional neural network to extract local spatial features, and after combining patch blocking and spatial position coding injection, it uses a sparse self-attention mechanism to capture global spatial dependencies. The time feature module uses multi-scale convolution to capture local time features at different time scales, extracts global time features through a cascading structure, and performs adaptive time downsampling on the spatial features through maximum pooling to align the dimensions of the spatial features and the time features. Then, the space-time features are fused into a unified encoder output, and finally, a generative decoder is used to output multi-step prediction results in parallel. The present application can simultaneously capture the local and global dependencies of traffic data in the spatial dimension and the multi-scale dynamic changes in the time dimension, thereby improving the prediction accuracy while maintaining low computational complexity and inference delay.

Description

Technical Field

[0001] This invention relates to the field of network traffic prediction, and in particular to a cellular network traffic prediction method based on the Informer architecture that integrates spatiotemporal feature modeling capabilities, specifically a network traffic prediction method based on spatiotemporal Informer. Background Technology

[0002] In today's digital age, spatiotemporal sequence data is widely used in numerous fields, such as network traffic monitoring, weather forecasting, and intelligent traffic management. Accurate prediction of spatiotemporal sequences not only helps in advance resource allocation planning and system scheduling optimization but also provides crucial information for decision-making. However, cellular network traffic data is highly complex. On the one hand, in the temporal dimension, traffic data is affected by factors such as time periods, holidays, and seasons, exhibiting multi-scale dynamic changes, including short-term fluctuations at the minute level, periodic changes at the hour level, and trend evolution over longer periods. The non-stationary temporal characteristics require models to simultaneously handle multi-scale dependencies ranging from minutes to weeks; traditional recurrent neural networks are ill-suited for such long-span modeling due to the gradient vanishing problem. On the other hand, in the spatial dimension, different base station coverage areas exhibit significantly different traffic distribution characteristics due to differences in population density, user behavior habits, and geographical location. Traffic in local areas may deviate from the global trend due to local events, and global and local spatiotemporal characteristics are intertwined and mutually reinforcing. Even more complex is the fact that as the predicted sequence length increases, standard attention mechanisms... Computational complexity poses a significant challenge to training efficiency and hardware resources.

[0003] Currently, methods in the field of network traffic prediction can be mainly divided into the following categories: The first category is traditional statistical models, such as autoregressive models like ARIMA and SARIMA. These methods can only handle single time series and cannot model spatial dependencies, and their ability to fit nonlinear patterns is limited. The second category is methods based on recurrent neural networks, such as LSTM and ConvLSTM. Although they can capture temporal dependencies to some extent, they are limited by the vanishing gradient problem, making it difficult to effectively model long-term dependencies, and their inference efficiency decreases significantly with increasing sequence length. The third category is methods based on convolutional neural networks, such as ST-ResNet. These methods can extract spatial features, but they are insufficient in capturing long-distance temporal dependencies.

[0004] In recent years, Transformer-based models have made significant progress in sequence prediction due to their powerful global modeling capabilities. However, the self-attention mechanism of the standard Transformer has limitations. The computational complexity of the model increases dramatically with sequence length, making it unsuitable for practical applications of long sequence prediction. While the Informer model reduces complexity to [a higher level] through the ProbSparse sparse attention mechanism... However, its original design mainly focuses on single time series prediction and lacks the ability to model spatial features, thus failing to fully extract spatiotemporal correlation information from traffic data. Therefore, how to effectively capture the local and global dependencies of traffic data in the spatial dimension and the multi-scale dynamic changes in the temporal dimension while maintaining low computational complexity is a technical problem that urgently needs to be solved in the field of network traffic prediction. Summary of the Invention

[0005] To address the problems of existing network traffic prediction methods, such as their inability to effectively model spatiotemporal multi-scale features, insufficient prediction accuracy for long sequences, and excessive computational complexity, the present invention aims to provide a network traffic prediction method based on spatiotemporal Informer, which can simultaneously capture the local and global dependencies of traffic data in the spatial dimension and the multi-scale dynamic changes in the temporal dimension, thereby improving prediction accuracy while maintaining low computational complexity.

[0006] The technical solution to achieve the objective of this invention is: a network traffic prediction method based on spatiotemporal Informer, comprising the following steps:

[0007] S1. Model cellular network traffic data as a four-dimensional spatiotemporal grid tensor to construct a data model for traffic prediction;

[0008] S2. Construct a spatial feature module, use a two-dimensional convolutional neural network to extract local spatial features, combine Patch segmentation and two-dimensional spatial location encoding injection, and use the ProbSparse sparse self-attention mechanism to capture global spatial dependencies.

[0009] S3. Construct a temporal feature module that works in parallel with the spatial feature module. Use multi-scale dilated causal convolution to capture local temporal features at different time scales. After injecting time location encoding and timestamp encoding, extract global temporal features through a cascaded structure of multi-layer ProbSparse sparse self-attention and attention distillation downsampling.

[0010] S4. Spatial features are adaptively downsampled and aligned in the temporal dimension through max pooling. The spatiotemporal features are then fused into a unified encoder output through channel splicing and linear projection. Finally, the generative decoder outputs multi-step prediction results in parallel.

[0011] Furthermore, the traffic data modeling method in step S1 is as follows: The cellular network coverage area is divided into... The grid, traffic data contains For each business type, historical traffic data is organized as a four-dimensional tensor. ,in For historical time steps, and These represent the number of grid divisions in the row and column directions, respectively. Number of business types; Time of the first Traffic space matrix of various business types middle, Represents grid coordinates Traffic value at the location; based on past data Historical traffic information at each time step to predict future traffic. The flow values ​​X of all grids at each time step are used to formulate the prediction problem as follows:

[0012]

[0013] in The predictive model to be learned is responsible for extracting spatiotemporal features from historical traffic sequences to infer future traffic evolution.

[0014] Furthermore, the spatial feature module in step S2, for each time step The spatial flow matrix is ​​processed independently, specifically including the following steps:

[0015] S21. Local Spatial Feature Extraction: Let... The input flow space matrix at time t is A two-dimensional convolutional neural network is used to extract spatial features from the input, with a kernel size of [missing value]. The spatial resolution is preserved using the padding='same' strategy, and the convolution operation is calculated using the following formula b:

[0016]

[0017] in For the first Each output channel is in the input channel Core location The weight of the position, For the first The bias term for each channel, The output feature map is the spatial location index of the output feature map. , Where W is the number of feature channels in the convolution output, W is the weight, and b is the bias term.

[0018] S22. Patch segmentation and feature mapping: Patch the feature map... Divide the area into uniform, non-overlapping local blocks, with each patch being [size missing]. Total number of patches Each patch is flattened and mapped to a unified low-dimensional feature space via linear projection, calculated by the following formula c:

[0019]

[0020] in For Patch index, For the first The feature blocks corresponding to each patch It is a learnable linear mapping matrix. For bias terms, For the embedding dimension of the patch, combine all patches to obtain the feature vector set. ;

[0021] S23. Two-dimensional spatial location code injection: Before spatial interaction, two-dimensional spatial location codes are explicitly injected to enable the model to perceive the geographic topological relationships of each patch on the two-dimensional plane, calculated by the following formula d:

[0022]

[0023] in Based on the two-dimensional coordinates of each patch in the original grid The generated learnable position encoding matrix, This is the set of feature vectors encoded by the two-dimensional spatial location.

[0024] S24. Global spatial interaction based on ProbSparse sparse self-attention: using three sets of independent learnable weight matrices to... Projected into query, key, and value matrices respectively, the results are calculated using the following equation:

[0025]

[0026] in The projection weight matrix is... The key vector dimension; the ProbSparse mechanism uses sparse metrics. Filter the Top- with the most information The number of active queries is calculated using the following formula:

[0027]

[0028] Perform the full dot product attention calculation only on the top u active queries selected, and fill the outputs of the remaining queries with the mean of the value matrix, reducing the computational complexity to [missing value]. Sparse attention output is denoted as Introducing residual connectivity and layer normalization, the following formula g is used for calculation:

[0029]

[0030] History After executing the above spatial feature extraction process at each time step, The spatial features at each moment are pieced together along the time dimension and calculated using the following formula h:

[0031]

[0032] Furthermore, the time feature module in step S3 works in parallel with the spatial feature module, independently receiving the raw traffic data and processing it. Temporal features are extracted in parallel using spatial patches, with each patch sharing the same set of model parameters. The specific steps include:

[0033] S31. Multi-scale dilated causal convolution extracts local temporal features:

[0034] Let the feature tensor of the input sequence be... ,in For the number of spatial patches, To input the time step, The input feature dimension is defined; dilated causal convolutions are applied at short-term, medium-term, and long-term scales, with corresponding dilation rates of [missing values]. , and ,in Short-term scale uses inflation rate The kernel size is The causal convolution is calculated by the following formula:

[0035]

[0036] in For Patch indexing, output features Medium-term scale uses expansion rate kernel size It is calculated by the following formula:

[0037]

[0038] Output Long-term scale use of expansion rate kernel size It is calculated by the following formula (k):

[0039]

[0040] Output Features from the three scales are concatenated along the channel dimension and fused through a linear projection layer to a unified hidden layer dimension. It can be calculated using the following formula:

[0041]

[0042] S32. Time and Location Encoding Injection:

[0043] Temporal characteristics Explicitly injected location information is calculated using a combination of local absolute location encoding and global timestamp encoding, as shown in the following formula:

[0044]

[0045] in This is an absolute positional encoding based on sequence position. For timestamp-based semantic encoding, calendar attributes such as hours and days of the week are embedded into dense vectors;

[0046] S33. Encoder layer stacking:

[0047] Introduction The stacking structure of layer encoder blocks, with the initial input being... In the The layer executes two steps sequentially; the first step is the ProbSparse self-attention interaction, calculated by the following formula n:

[0048]

[0049] The second step is attention distillation downsampling. After extracting features from adjacent time steps using one-dimensional convolution, time-dimensional downsampling is performed using max pooling with a stride of 2, calculated by the following formula:

[0050]

[0051] Each distillation operation halves the sequence length while preserving the most significant characteristic responses; after... The final global time feature is output after layer iteration, calculated by the following formula p:

[0052]

[0053] Furthermore, the feature fusion and prediction method in step S4 includes three stages: temporal adaptive downsampling alignment, spatiotemporal feature stitching and projection fusion, and generative decoder.

[0054] S41, Adaptive downsampling alignment in the time dimension: Due to the introduction of the time feature module... Layer attention distillation, spatial features With time characteristics There are scale differences in the time dimension, affecting spatial characteristics. Using a step size along the time dimension The size of the nucleus is The one-dimensional max pooling operation is used for downsampling, and the result is calculated using the following formula:

[0055]

[0056] Max pooling is performed on each length of The maximum value is taken within the time window to retain the moment with the strongest spatial characteristic response, while also filtering out high-frequency temporal noise;

[0057] S42. Spatiotemporal Feature Stitching and Projection Fusion: This involves stitching and projecting aligned spatial features... With time characteristics Concatenation is performed along the feature dimension, calculated using the following formula for r:

[0058]

[0059] Through a learnable weight matrix and bias Map the concatenated features to a unified hidden layer dimension of the model. Calculated by the following formula:

[0060]

[0061] In this stitching and projection fusion strategy, the weight matrix... It can adaptively adjust the contribution ratio of each dimension of spatial and temporal features. This is the final output of the encoder;

[0062] S43, Generative Decoder: The decoding process consists of four stages:

[0063] Phase 1: Constructing the decoder input sequence , consisting of a length of The guiding sequence and length are It is composed of a sequence of zero-padding placeholders, and simultaneously injected with the absolute position code and timestamp code of the target time period. Each spatial patch is considered as a batch dimension;

[0064] In the second stage, the input is fed into the Informer decoder, where Masked multi-head sparse self-attention and multi-head cross-attention operations are performed sequentially, with the encoder output... As keys and values Each spatial patch shares decoder weights and is decoded in parallel over time, outputting decoded features calculated by the following formula:

[0065]

[0066] In the third stage, fully connected layers are used for prediction projection, which reduces the hidden layer dimensions. The total number of pixels projected back to the original grid and the number of service channels are calculated using the following formula:

[0067]

[0068] in The projection weight matrix is... For Patch size, Number of business types;

[0069] In the fourth stage, the spatial structure is restored through dimension transpose and tensor reshaping operations, calculated using the following formula v:

[0070]

[0071] The final output global mesh will be available in the future. Traffic forecast values ​​at each time step.

[0072] The present invention has the following beneficial effects:

[0073] By combining two-dimensional convolution with patch segmentation, two-dimensional spatial location encoding, and ProbSparse sparse self-attention mechanism in the spatial feature module, it is possible to simultaneously capture the local dependencies and global correlation patterns of traffic data in the spatial dimension, overcoming the limitation of traditional methods that can only handle a single spatial scale.

[0074] By combining multi-scale dilated causal convolution with temporal position encoding, multi-layer ProbSparse sparse self-attention, and attention distillation downsampling cascade structure in the temporal feature module, it is possible to effectively capture temporal features at three levels: short-term fluctuations, medium-term cycles, and long-term trends without increasing the number of parameters. This solves the problem that recurrent neural networks are difficult to model long-term dependencies. At the same time, by compressing the sequence length exponentially through layer-by-layer distillation, computational overhead is significantly reduced.

[0075] By employing the ProbSparse sparse attention mechanism, the computational complexity is reduced from... Reduce to By combining a generative decoder, the cumulative delay of autoregressive stepwise decoding is avoided, which keeps the inference time of the model stable at a low level in long sequence prediction tasks.

[0076] By using max pooling to achieve adaptive alignment of spatial and temporal features in the temporal dimension, and then performing deep fusion through channel splicing and linear projection, the complementarity of spatiotemporal features in traffic data is fully explored, and the prediction accuracy is better than existing traditional statistical models, neural network models and basic Transformer models.

[0077] The generative decoder uses Masked sparse self-attention and cross-attention mechanisms to perform multi-step temporal predictions in parallel on all spatial patches. After projection and spatial reconstruction, it outputs the flow prediction results of the global grid, avoiding the cumulative error of step-by-step decoding. Attached Figure Description

[0078] Figure 1 This is a schematic diagram of the overall structure of the ST-Informer model of the present invention;

[0079] Figure 2 is a flowchart of the method of the present invention. Detailed Implementation

[0080] The present invention will be further described and illustrated below with reference to the accompanying drawings and embodiments:

[0081] A network traffic prediction method based on spatiotemporal Informer is proposed. This model, built upon the Seq2Seq framework of Informer, comprises four core components: a spatial feature module, a temporal feature module, a spatiotemporal feature fusion module, and a generative decoder. These components work collaboratively to fully exploit the spatiotemporal correlations within traffic data, and include the following steps:

[0082] S1, Traffic Data Modeling

[0083] Divide the cellular network coverage area into The grid, traffic data contains Various service types, such as calls, text messages, and internet data traffic. Historical traffic data is organized as a four-dimensional tensor. ,in For historical time steps, and These represent the number of grid divisions in the row and column directions, respectively. This represents the number of business types. Time of the first Traffic space matrix of various business types middle, Represents grid coordinates The traffic value at the location. Cellular network traffic is affected by both temporal and spatial factors: in the temporal dimension, traffic exhibits multi-scale periodic fluctuations and sudden changes with factors such as time of day, holidays, and seasons; in the spatial dimension, different base station coverage areas exhibit significantly different traffic distribution characteristics due to differences in population density, user behavior habits, and geographical location. Based on past... Historical traffic information at each time step to predict future traffic. The flow values ​​of all grids at each time step are used to formulate the prediction problem as follows:

[0084]

[0085] in The predictive model to be learned is responsible for extracting spatiotemporal features from historical traffic sequences to infer future traffic evolution.

[0086] S2, Constructing Spatial Feature Modules

[0087] The spatial feature module is responsible for extracting the spatial dependencies of traffic data for each time step. Its spatial flow matrix is ​​processed independently.

[0088] First, local spatial feature extraction is performed. Let... The input flow space matrix at time t is A two-dimensional convolutional neural network is used to extract spatial features from the input, with a convolutional kernel size of [size missing]. The 'same' padding strategy is used to preserve spatial resolution, and the convolution operation is calculated using formula b:

[0089]

[0090] in For the first Each output channel is in the input channel Core location The weight of the position, For the first The bias term for each channel, This provides the spatial index for the output feature map. The output feature map... Each element in the dataset incorporates traffic fluctuation characteristics of various business types within its local receptive field. is the number of feature channels in the convolution output, W is the weight, and b is the bias term.

[0091] Subsequently, drawing on the patching strategy from the visual Transformer, the feature maps were... Divide the area into uniform, non-overlapping local blocks, with each patch being [size missing]. Total number of patches Each patch is flattened and mapped to a unified low-dimensional feature space via linear projection, calculated using equation c:

[0092]

[0093] in For Patch index, For the first The feature blocks corresponding to each patch It is a learnable linear mapping matrix. For bias terms, Embed the dimension of the patch. Combine all patches to obtain the feature vector set. Through this step, the number of space tokens increased from the original... Down to This significantly reduces computational cost while preserving spatial information within the patch.

[0094] The self-attention mechanism itself has permutation invariance; if directly... When inputting the attention layer, the network becomes unaware of the geographic topological relationships of each patch on a two-dimensional plane. Therefore, before extracting spatial features, a two-dimensional spatial location code is explicitly injected, calculated using formula d:

[0095]

[0096] in Based on the two-dimensional coordinates of each patch in the original grid The generated learnable location encoding matrix. This step ensures that the model can distinguish patches with different spatial orientations during subsequent attention interactions, preserving the spatial topology information of the traffic data.

[0097] After injecting location information, a ProbSparse sparse self-attention mechanism is applied to capture global spatial features across regions. This is achieved through three sets of independent, learnable weight matrices. Projected into query, key, and value matrices respectively, and calculated using equation e:

[0098]

[0099] in There are three sets of learnable weight matrices. The key vector dimension. The ProbSparse mechanism evaluates the sparsity measure of each query. To filter the Top- with the most information The number of active queries is calculated using formula f:

[0100]

[0101] This metric measures the query The difference between the attention distribution and the uniform distribution: The larger the value, the more focused the query is on a few key keys, and the richer the spatial information it carries. Only the filtered Top- One active query performs the full dot product attention computation, while the outputs of the remaining queries are filled with the mean of the value matrix, thus reducing the computational complexity to [value missing]. The output of sparse attention is denoted as .

[0102] Residual connections and layer normalization are introduced to stabilize the training of deep networks, calculated using formula g:

[0103]

[0104] History After executing the above spatial feature extraction process at each time step, Spatial features at each moment are pieced together along the time dimension to form a complete spacetime tensor. Calculated using formula h:

[0105]

[0106] While effectively compressing the number of tokens in the space, it fully preserves... The spatial resolution of each local grid block lays the foundation for subsequent spatiotemporal feature fusion.

[0107] S3, Constructing the Time Feature Module

[0108] The temporal feature module and the spatial feature module work in parallel, independently receiving raw traffic data and responsible for extracting multi-scale dynamic features along the temporal dimension. This module... Temporal features are extracted in parallel using spatial patches, and each patch shares the same set of model parameters.

[0109] First, multi-scale dilated causal convolution is performed to extract local temporal features. Let the input sequence feature tensor be... ,in For the number of spatial patches, To input the time step, The input feature dimension is the number of business types. Unlike standard convolution, dilated causal convolution introduces a dilation rate parameter. Inserting gaps between convolution kernel elements allows for an exponential expansion of the receptive field without increasing the number of parameters. Simultaneously, causal constraints ensure that the output at the current moment depends only on historical information, conforming to the physical causal laws of time series.

[0110] For the three time scales of short-term fluctuations, medium-term cycles, and long-term trends, the inflation rate is used respectively. One-dimensional causal convolution processes the input sequence. For short-term scales, dilation rate is used. The kernel size is The causal convolution is calculated by equation i:

[0111]

[0112] in For Patch index, For convolution kernel weights, This is a bias term. When... At that time, the convolution kernel samples on consecutive time steps, and the receptive field is This time step is suitable for capturing local fluctuation patterns between adjacent time steps. Output features .

[0113] For medium-scale use of expansion rate (like The corresponding 1-hour interval and the kernel size Calculated by formula j:

[0114]

[0115] Output Effectively experience the wild Each time step. For long-term scales, use the expansion rate. ,like The corresponding 6-hour interval and convolution kernel size Calculated using formula k:

[0116]

[0117] Output By setting an increasing expansion rate The three-scale convolution samples the input sequence at different temporal resolutions, capturing local temporal features at three levels: short-term fluctuations, medium-term cycles, and long-term trends.

[0118] Subsequently, the features at the three scales are concatenated along the channel dimension, and then fused and mapped to a unified hidden layer dimension through a linear projection layer. Calculated using formula l:

[0119]

[0120] Self-attention mechanisms lack the inherent ability to handle sequential order, therefore they must be adapted to temporal features. Explicitly inject location information. This module uses a combination of local absolute location encoding and global timestamp encoding, calculated using the m-form:

[0121]

[0122] in It is an absolute positional encoding based on sequence position, used to identify the relative order of each time step in the input sequence; For timestamp-based semantic encoding, calendar attributes such as hour and day of the week corresponding to each time step are embedded into a dense vector. The superposition of the two encodings enables the model to simultaneously perceive the sequential position and absolute time attributes of the time step during subsequent attention interactions.

[0123] As the length of network traffic sequences increases, single-layer attention mechanisms become insufficient to fully capture macroscopic evolutionary trends over extremely long spans. Therefore, this module introduces... The stacking structure of layer encoder blocks, with the initial input being... In the layer( Within the ), the feature update process includes the following two steps.

[0124] The first step is sparse self-attention interaction. Based on the output of the previous layer... Through independent weight matrices Projection generates query, key, and value matrices, and the ProbSparse sparse attention mechanism is executed to extract global temporal dependencies, calculated using equation n:

[0125]

[0126] Similar to the ProbSparse mechanism in the spatial module, this module also uses sparse metrics to filter active queries, maintaining long-range modeling capabilities while keeping the computational complexity of the time dimension under control. .

[0127] The second step is attention distillation downsampling. A distillation operation is applied after the attention layer. After extracting features from adjacent time steps using one-dimensional convolution, a max-pooling layer with a stride of 2 is used to force downsampling in the time dimension, calculated using equation o:

[0128]

[0129] One-dimensional convolution slides along the time dimension to extract local patterns, the ELU activation function introduces a non-linear transformation, and max pooling with a stride of 2 halves the sequence length. Each distillation operation retains the most significant feature responses within each time window while filtering out redundant high-frequency noise.

[0130] go through After layer iterations, the temporal feature module outputs the final global feature representation, calculated using formula p:

[0131]

[0132] This design reduces the physical length of the sequence exponentially—halving it with each layer. Post-layer time dimension compression to the original length —While significantly reducing subsequent computational overhead and GPU memory usage, it retains the core timing evolution mode.

[0133] S4, Spatiotemporal Feature Fusion and Generative Decoding

[0134] Spatial features containing global topological information were obtained separately. With time characteristics that condense multi-scale evolutionary patterns Then, the two need to be deeply fused to construct a unified encoder output. This is because the time feature module introduces... Layer attention distillation reveals a significant scale difference between the two in the time dimension: It has been preserved in its original form. One time step, and The time dimension has been compressed to .

[0135] First, adaptive downsampling alignment is performed along the time dimension. Considering that network traffic sequences contain bursty peak signals that need to be preserved, the alignment is performed along the time dimension. Using a step size of The size of the nucleus is The one-dimensional max pooling operation is calculated using the q-form:

[0136]

[0137] Max pooling operation is performed on each length of The maximum value is taken within the time window, which means that for each spatial patch, the moment with the strongest spatial feature response is retained within the corresponding time interval. This operation not only ensures that the spatial features are strictly aligned with the temporal features in terms of tensor shape, but also filters out high-frequency temporal noise and retains the most significant spatial linkage signal within each time window.

[0138] Subsequently, feature fusion was performed using a combination of channel stitching and linear projection. and Concatenate along the feature dimension, calculated using the r formula:

[0139]

[0140] Through a learnable weight matrix and bias Map the concatenated features to a unified hidden layer dimension of the model. Calculated using formula s:

[0141]

[0142] Compared to element-wise addition, this fusion strategy of splicing and projection has stronger expressive power: weight matrix The parameters in the model can adaptively adjust the contribution ratio of each dimension of spatial and temporal features during backpropagation, thereby learning the optimal combination of the two types of features. This is the final output of the encoder, which will be used as key information for the decoder in subsequent prediction generation.

[0143] Finally, the generative decoder performs multi-step parallel prediction, and its decoding and reconstruction process is divided into the following four stages:

[0144] Phase 1: Decoder input construction.

[0145] Construct the decoder input sequence It is composed of two parts: the length is The guide sequence, taken from the real historical data at the end of the encoder input, provides contextual start information for the decoder, and has a length of [missing information]. The zero-padding placeholder sequence corresponds to the future time step to be predicted. Simultaneously, the absolute position encoding and timestamp encoding of the target time period are injected into the decoder input. In terms of computation, [the following is omitted as the text is incomplete and requires further context]. Each spatial patch is treated as a batch dimension, so that the sequence length dimension is aligned with the self-attention computation direction inside the decoder.

[0146] The second stage is parallel sequential decoding.

[0147] The above input is fed into the Informer decoder, which performs two attention operations sequentially: First, Masked multi-head sparse self-attention, which uses a masking matrix to ensure that each time step can only focus on the current and previous positions, preventing future information leakage; second, multi-head cross-attention, which converts the encoder output... As keys and values, the decoder's self-attention output serves as a query, retrieving information relevant to the current prediction time from the spatiotemporal fusion features of the encoder. This is because spatial topological information has already been deeply fused into the encoder during the encoding stage. middle, Each spatial patch shares decoder weights in this stage, and independently and infers its future trend in parallel over time, outputting decoding features, calculated by formula t:

[0148]

[0149] Phase 3, predicting projection.

[0150] Using fully connected layers Perform feature dimension mapping, and map the hidden layer dimensions. The total number of pixels and service channels included in the projected original mesh are calculated using formula u:

[0151]

[0152] in The projection weight matrix is... For Patch size, This represents the number of business types. This projection restores the abstract features of each patch to the ones it covers. Each grid in Traffic forecast values ​​for each business type.

[0153] Phase 4: Restoring the spatial structure. Through dimension transpose and tensor reshaping operations, the parallel patch prediction results are restored to the physical spatial topology, calculated using formula v:

[0154]

[0155] Final output That is, the entire domain Grid in the Future The traffic prediction values ​​for each time step are calculated. The entire decoding process avoids the accumulated errors and delays of autoregressive stepwise decoding, and can output the results of all prediction time steps in parallel at once.

[0156] Example 1

[0157] See Figure 1 Perform network traffic prediction using the following steps:

[0158] S1. Traffic data modeling uses a telecommunications dataset from a certain city, which divides the city's urban area into... The grid, i.e. , Traffic data includes three CDR service types: Call, SMS, and Internet traffic. The time granularity is 10 minutes. Historical traffic data is organized as a four-dimensional tensor. In the data preprocessing stage, missing time steps are filled using linear interpolation, and the data is scaled using Min-Max normalization. Interval. The data is divided into training, validation, and test sets in chronological order, with a ratio of 7:1:2. Input sequence length. With 96 time steps (16 hours) set, the prediction problem is formulated as formula a.

[0159] S2. Constructing the Spatial Feature Module: The original traffic data first enters a two-dimensional convolutional layer, with a kernel size of [missing value]. The spatial resolution is maintained by using the 'same' padding strategy, and local spatial features are extracted from the data using equation (b), outputting a feature map. Subsequently, drawing inspiration from the patching strategy in visual Transformers, the feature map was divided into sections of varying sizes. ( Non-overlapping patches, total number of patches The patch is mapped to a linear projection layer. The dimensional feature space is obtained by calculation from equation (c). To enable the model to perceive the spatial topological relationships of each patch, a two-dimensional spatial location code is injected, which is calculated by equation (d). .right The ProbSparse sparse self-attention mechanism is applied. Query, key, and value matrices are generated by projection using equation (e), and active queries are selected by sparse metrics calculated using equation (f). Residual connections and layer normalization are introduced, and spatial features are obtained by calculation using equation g. .right After executing the above process at each time step, the result is obtained by concatenating the h-form along the time dimension. .

[0160] S3. Constructing the Temporal Feature Module: The temporal feature module works in parallel with the spatial feature module, independently receiving raw traffic data. Multi-scale dilated causal convolution is employed, with short-term dilation rates set separately. Medium-term inflation rate Corresponding to 1-hour intervals and long-term expansion rate For a 6-hour interval, the short-term characteristics are calculated using formulas i, j, and k, respectively. Mid-term characteristics and long-term characteristics The outputs from the three scales are concatenated along the channel dimension and then fused and mapped to a unified dimension via a linear projection layer. Calculated by equation l The local absolute position code and global timestamp code are injected, and calculated using formula m. Subsequently through The cascaded structure of layer encoder blocks, each layer sequentially executes the ProbSparse sparse self-attention interaction formula (n) and the attention distillation downsampling formula (o), halving the sequence length at each layer. After layer iteration, the global time features are calculated using formula p. .

[0161] S4. Spatiotemporal Feature Fusion and Generative Decoding: Due to the attention distillation of the temporal feature module, the temporal dimension is compressed. First, the spatial features... Using a step size along the time dimension The size of the nucleus is The one-dimensional max pooling operation is used for adaptive downsampling alignment, and the result is obtained from the q-form. .Will and Concatenated along the feature dimension, obtained by r-formation. The concatenated features are mapped to a learnable weight matrix. The encoder output is obtained from the s-square space. .

[0162] Generative decoders construct decoder inputs that include a guiding sequence and zero-padding placeholders. Each spatial patch is treated as a batch dimension, and internally, Masked multi-head sparse self-attention and multi-head cross-attention operations are executed sequentially. As key-value pair Each patch shares decoder weights and is decoded in parallel along the time axis. The decoding features are obtained from equation t. Predictive projection is performed using a fully connected layer, and the result is calculated using the U-form. Finally, the spatial structure is restored through dimension transposition and tensor reshaping, and the final prediction result is obtained by V-form calculation. The model uses the Adam optimizer with an initial learning rate of 0.0001, employs the ReduceLROnPlateau learning rate scheduling strategy, weight decay of 1e-4, a dropout ratio of 0.2, and an early stopping patience of 15. The model's hidden layer dimensions are... The dimensions of both the feedforward network and the multi-head attention network are set to 128, the number of multi-head attention heads is 4, the encoder and decoder are both single-layer structures, the sparsity factor is set to 5, the activation function is GELU, and the loss function is mean squared error (MSE).

[0163] The above description is merely a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A network traffic prediction method based on spatiotemporal Informer, comprising the following steps: S1. Model cellular network traffic data as a four-dimensional spatiotemporal grid tensor to construct a data model for traffic prediction; S2. Construct a spatial feature module, use a two-dimensional convolutional neural network to extract local spatial features, combine Patch segmentation and two-dimensional spatial location encoding injection, and use the ProbSparse sparse self-attention mechanism to capture global spatial dependencies. S3. Construct a temporal feature module that works in parallel with the spatial feature module. Use multi-scale dilated causal convolution to capture local temporal features at different time scales. After injecting time location encoding and timestamp encoding, extract global temporal features through a cascaded structure of multi-layer ProbSparse sparse self-attention and attention distillation downsampling. S4. Spatial features are adaptively downsampled and aligned in the temporal dimension through max pooling. The spatiotemporal features are then fused into a unified encoder output through channel splicing and linear projection. Finally, the generative decoder outputs multi-step prediction results in parallel.

2. The network traffic prediction method based on spatiotemporal Informer according to claim 1, characterized in that, The traffic data modeling method in step S1 is as follows: The cellular network coverage area is divided into... The grid, traffic data contains For each business type, historical traffic data is organized as a four-dimensional tensor. ,in For historical time steps, and These represent the number of grid divisions in the row and column directions, respectively. Number of business types; Time of the first Traffic space matrix of various business types middle, Represents grid coordinates Traffic value at the location; based on past Historical traffic information at each time step to predict future traffic. The flow values ​​X of all grids at each time step are used to formulate the prediction problem as follows: in The predictive model to be learned is responsible for extracting spatiotemporal features from historical traffic sequences to infer future traffic evolution.

3. The network traffic prediction method based on spatiotemporal Informer according to claim 1, characterized in that, The spatial feature module in step S2, for each time step The spatial flow matrix is ​​processed independently, specifically including the following steps: S21. Local Spatial Feature Extraction: Let... The input flow space matrix at time t is A two-dimensional convolutional neural network is used to extract spatial features from the input, with a kernel size of [missing value]. The spatial resolution is preserved using the padding='same' strategy, and the convolution operation is calculated using the following formula b: in For the first Each output channel is in the input channel Core location The weight of the position, For the first The bias term for each channel, The output feature map is the spatial location index of the output feature map. , Where W is the number of feature channels in the convolution output, W is the weight, and b is the bias term. S22. Patch segmentation and feature mapping: Patch the feature map... Divide the area into uniform, non-overlapping local blocks, with each patch being [size missing]. Total number of patches Each patch is flattened and mapped to a unified low-dimensional feature space via linear projection, calculated by the following formula c: in For Patch index, For the first The feature blocks corresponding to each patch It is a learnable linear mapping matrix. For bias terms, For the embedding dimension of the patch, combine all patches to obtain the feature vector set. ; S23. Two-dimensional spatial location code injection: Before spatial interaction, two-dimensional spatial location codes are explicitly injected to enable the model to perceive the geographic topological relationships of each patch on the two-dimensional plane, calculated by the following formula d: in Based on the two-dimensional coordinates of each patch in the original grid The generated learnable position encoding matrix, This is the set of feature vectors encoded by the two-dimensional spatial location. S24. Global spatial interaction based on ProbSparse sparse self-attention: using three sets of independent learnable weight matrices to... Projected into query, key, and value matrices respectively, the results are calculated using the following equation: in The projection weight matrix is... The dimension is the key vector; the ProbSparse mechanism uses sparse metrics. Filter the Top- with the most information The number of active queries is calculated using the following formula: Perform the full dot product attention calculation only on the top u active queries selected, and fill the outputs of the remaining queries with the mean of the value matrix, reducing the computational complexity to [missing value]. Sparse attention output is denoted as Introducing residual connectivity and layer normalization, the following formula g is used for calculation: History After executing the above spatial feature extraction process at each time step, The spatial features at each moment are pieced together along the time dimension and calculated using the following formula h: 。 4. The network traffic prediction method based on spatiotemporal Informer according to claim 1, characterized in that... Step S3 specifically includes the following steps: S31. Multi-scale dilated causal convolution extracts local temporal features: Let the feature tensor of the input sequence be... ,in For the number of spatial patches, To input the time step, The input feature dimension is defined; dilated causal convolutions are applied at short-term, medium-term, and long-term scales, with corresponding dilation rates of [missing values]. , and ,in Short-term scale uses inflation rate The kernel size is The causal convolution is calculated by the following formula: in For Patch indexing, output features Medium-term scale uses expansion rate kernel size It is calculated by the following formula: Output Long-term scale use of expansion rate kernel size It can be calculated using the following formula k: Output Features from the three scales are concatenated along the channel dimension and fused through a linear projection layer to a unified hidden layer dimension. It can be calculated using the following formula: ； S32. Time and Location Encoding Injection: Temporal characteristics Explicitly injected location information is calculated using a combination of local absolute location encoding and global timestamp encoding, as shown in the following formula: in For absolute position encoding based on sequence position, For timestamp-based semantic encoding, calendar attributes such as hours and days of the week are embedded into dense vectors; S33. Encoder layer stacking: Introducing a stacked structure of layer encoder blocks, with the initial input being... In the The layer executes two steps sequentially; the first step is the ProbSparse self-attention interaction, calculated by the following formula n: The second step is attention distillation downsampling. After extracting features from adjacent time steps using one-dimensional convolution, time-dimensional downsampling is performed using max pooling with a stride of 2, calculated by the following formula: Each distillation operation halves the sequence length while preserving the most significant characteristic responses; after... The final global time feature is output after layer iteration, calculated by the following formula p: 。 5. A network traffic prediction method based on spatiotemporal Informer according to claim 1, characterized in that... Step S4 specifically includes: S41. Adaptive downsampling alignment in the time dimension: for spatial features Using a step size along the time dimension The size of the nucleus is The one-dimensional max pooling operation is used for downsampling, and the result is calculated using the following formula: Max pooling is performed on each length of The maximum value is taken within the time window to retain the moment with the strongest spatial characteristic response, while also filtering out high-frequency temporal noise; S42. Spatiotemporal Feature Stitching and Projection Fusion: This involves stitching and projecting aligned spatial features... With time characteristics Concatenation is performed along the feature dimension, calculated using the following formula for r: Through the weight matrix and bias Map the concatenated features to a unified hidden layer dimension of the model. Calculated by the following formula: In this stitching and projection fusion strategy, the weight matrix It can adaptively adjust the contribution ratio of each dimension of spatial and temporal features. This is the final output of the encoder; S43, Generative Decoder: The decoding process consists of four stages: Phase 1: Constructing the decoder input sequence , consisting of a length of The guiding sequence and length are It is composed of a sequence of zero-padding placeholders, and simultaneously injected with the absolute position code and timestamp code of the target time period. Each spatial patch is considered as a batch dimension; In the second stage, the decoder input sequence is fed into the generative decoder, where masked multi-head sparse self-attention and multi-head cross-attention operations are performed sequentially, with the encoder output... As keys and values Each spatial patch shares decoder weights and is decoded in parallel over time, outputting decoded features calculated by the following formula: ； In the third stage, fully connected layers are used for prediction projection, which reduces the hidden layer dimensions. The total number of pixels projected back to the original grid and the number of service channels are calculated using the following formula: in The projection weight matrix is... For Patch size, Number of business types; In the fourth stage, the spatial structure is restored through dimension transpose and tensor reshaping operations, calculated using the following formula v: The final output global mesh will be available in the future. Traffic forecast values ​​at each time step.

Images