Traffic flow prediction method based on multilevel cross attention and neighborhood perception mechanism

Abstract

The invention discloses a traffic flow prediction method based on a multilevel cross attention and neighborhood perception mechanism, and relates to the technical field of traffic flow prediction, and the method comprises the steps: carrying out the cleaning, standardization and feature extraction of original traffic data, and generating a node feature matrix and an adjacent matrix; encoding the two layers into high-dimensional implicit representation by utilizing BTE and NASAN, and constructing an encoding layer and a feature extraction layer; the method comprises the following steps: randomly segmenting data into a plurality of groups, obtaining a reference model through forward-back propagation training, loading target traffic data of a to-be-predicted area after verification evaluation, learning rate scheduling and early stop inspection reach the standard, and outputting a prediction result through one-time forward reasoning of the reference model. According to the method, the prediction precision can be improved, the reasoning delay is compressed, the interpretability is enhanced, and the real-time deployment requirement is met.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of traffic flow prediction, and in particular to a traffic flow prediction method based on a multi-level cross-attention and neighborhood perception mechanism. BACKGROUND

[0002] In the intelligent transportation, smart city and Internet of Things prediction scenarios, the spatio-temporal regression task (especially traffic flow prediction) is a core basic capability. The existing technologies can be divided into three categories in general: The first category of statistical learning models, represented by ARIMA, Prophet and their variants, rely on the stationarity assumption of time series, can describe short-term linear correlation, but have limited ability to describe sudden events, long-period dependence and nonlinear spatial interaction, and are difficult to integrate road network topology information.

[0003] The second category of traditional machine learning models, such as support vector machine (SVM), random forest (RF), gradient boosting tree (GBDT), etc., introduce external factors such as holidays and weather through manual feature engineering, and perform well on small and medium-sized data. However, they usually "emphasize features and ignore structure", cannot capture high-order spatial correlation and long-range temporal dependence end-to-end, and have limited model capacity, making it difficult to use large-scale unlabeled data.

[0004] The third category of deep learning models, including recurrent classes (LSTM / GRU) and their spatio-temporal variants (ConvLSTM, ST-LSTM), can model long-term temporal dependencies, but have high computational complexity and lack explicit expression of road network topology. Graph neural network classes (GCN, GAT, ST-GCN) explicitly encode spatial structure through adjacency matrices, but face "over-smoothing" and "long-range bottleneck" when stacked multiple layers, making it difficult to balance local details and global patterns. Transformer and its spatio-temporal variants (TemporalTransformer, ST-Transformer) use self-attention mechanisms to capture long-range dependencies, but the computational complexity grows quadratically with sequence length; at the same time, existing methods generally ignore the influence of "neighborhood multi-order topology" and "gradient-sensitive features" on regression accuracy, resulting in a trade-off between accuracy and efficiency in real-time scenarios.

[0005] The existing technologies have the following defects in common: ① Long-term temporal dependence capture is insufficient; ② Spatial correlation modeling is limited to single-order adjacency, and cannot adaptively integrate multi-order neighborhood information; ③ The model structure has high complexity, making it difficult to meet the millisecond-level real-time prediction requirements; ④ Lack of dynamic weighting and interpretability design for prediction error at different time scales. Therefore, there is an urgent need for a new regression prediction method that can simultaneously solve the above defects. SUMMARY

[0006] The technical problem solved by the present application is to overcome the shortcomings of the prior art, and specifically provides a traffic flow prediction method based on multi-level cross attention and neighborhood perception mechanism, which is specifically as follows: 1) In a first aspect, the present application provides a traffic flow prediction method based on multi-level cross attention and neighborhood perception mechanism, and the specific technical solutions are as follows: The original traffic data is preprocessed to obtain a node feature matrix and an adjacency matrix, and the preprocessing includes data cleaning, standardization and feature extraction processing; The node feature matrix and the adjacency matrix are converted into high-dimensional hidden representations by BTE and NASAN, and the encoding layer and the feature extraction layer in the target model are constructed based on the high-dimensional hidden representations; The original traffic data is randomly divided into multiple groups of divided sets, and the target model is trained by combining forward propagation and back propagation to obtain a reference model; The reference model is sequentially verified and evaluated, the learning rate is scheduled, and the early stopping check processing is performed, and when the reference model meets the preset requirements, the target traffic data corresponding to the to-be-predicted area is obtained, the target traffic data is predicted by the reference model, and the prediction result is obtained.

[0007] The traffic flow prediction method based on multi-level cross attention and neighborhood perception mechanism provided by the present application has the following beneficial effects: By integrating data cleaning, standardization and feature extraction preprocessing, the node feature matrix and the adjacency matrix are obtained, and the original low-dimensional data is upgraded to high-dimensional hidden representations rich in space-time correlation by BTE combined with NASAN. The model can capture long-term time dependence and multi-order neighborhood spatial relationship without increasing redundant parameters, significantly reducing the computational complexity. With the help of random division of multiple data sets, forward-backward propagation closed loop training, and verification-scheduling-early stopping three-level quality control, the model can adaptively adjust the learning rate and stopping time during the training stage, avoid overfitting, and improve the generalization performance. When the model meets the preset requirements, only one forward inference is needed to realize millisecond-level high-precision prediction of real-time target traffic data. The overall inference speed is improved by more than 30% compared with existing methods, and the explainability is provided by attention weight and feature importance score, which meets the prediction accuracy, real-time performance and explainability requirements.

[0008] On the basis of the above-mentioned scheme, the present application can also be improved as follows.

[0009] Further, the determination process of the node feature matrix and the adjacency matrix is as follows: The original traffic data is subjected to data cleaning to obtain first traffic data, and the data cleaning includes: detecting abnormal value processing, removing abnormal value processing, and filling missing value processing by spatio-temporal interpolation; The first traffic data is subjected to feature extraction to obtain spatial features and time features, an adjacency matrix is constructed based on the spatial features, and a node feature matrix is constructed based on the time features.

[0010] The above further scheme has the beneficial effects that: By performing three-level cleaning of abnormal value detection-removal-spatio-temporal interpolation first to obtain clean and complete first traffic data, and then extracting spatial features and time features from it in parallel and constructing an adjacency matrix and a node feature matrix respectively, the adjacency matrix can accurately reflect the real road network topology and multi-order neighborhood relationship, and the node feature matrix can carry time periodicity, holiday effect and historical statistical information at the same time, thereby significantly reducing noise interference, improving feature expression ability, and providing high signal-to-noise ratio input for subsequent BTE and NASAN layers, and finally further improving prediction accuracy and real-time performance on the premise of ensuring model lightweight.

[0011] Further, the forward propagation specifically includes: The input data is added with position encoding by the BTE, and the input data after adding the position encoding is subjected to NASAN processing, and after combined CMRC fusion processing, output data is obtained through time dimension compression processing.

[0012] The above further scheme has the beneficial effects that: The BTE is used to inject absolute and relative position encodings at one time, so that the input data directly obtains explicit time priori; the NASAN completes topology-aware self-attention calculation under the guidance of a multi-order neighborhood matrix, which retains local details and captures global dependencies; the CMRC dynamically fuses different layer features with gradient-modulated gating residual, suppresses redundancy and strengthens long-term dependency modeling; finally, the high-dimensional spatio-temporal representation is aggregated into a compact vector through time dimension compression, which significantly reduces the parameter quantity and the calculation amount, and realizes more than 20% reduction in inference delay under the same accuracy.

[0013] Further, the training process of the target model further includes: The single-step error loss, the multi-scale aggregated error loss, and the adaptive fusion total loss are used as constraint conditions to train the target model to obtain a reference model.

[0014] The above further scheme has the beneficial effects that: A single-step error loss ensures the prediction accuracy at each moment, a multi-scale aggregated error loss synchronously optimizes short, medium and long-term trends, and an adaptive fusion total loss dynamically weights and superimposes regular constraints according to the error of each scale, so that the model automatically focuses on the most difficult time interval and suppresses overfitting during the training process, and finally the reference model obtained maintains the optimal error distribution in global and local prediction, significantly improving the generalization ability and real-time deployment stability.

[0015] 2) In a second aspect, the application further provides a traffic flow prediction system based on a multi-level cross-attention and neighborhood perception mechanism, and the specific technical solutions are as follows: The preprocessing module is configured to preprocess the original traffic data to obtain a node feature matrix and an adjacency matrix, and the preprocessing includes data cleaning, standardization and feature extraction. The construction module is configured to convert the node feature matrix and the adjacency matrix into high-dimensional hidden representations through BTE and NASAN, and to construct the encoding layer and the feature extraction layer in the target model based on the high-dimensional hidden representations. The training module is configured to randomly split the original traffic data to obtain multiple split sets, and to train the target model by combining forward propagation and back propagation to obtain a reference model. The prediction module is configured to sequentially perform validation and evaluation, learning rate scheduling and early stopping checking on the reference model, and when the reference model meets the preset requirements, to obtain target traffic data corresponding to a to-be-predicted area, and to predict the target traffic data through the reference model to obtain a prediction result.

[0016] On the basis of the above-mentioned scheme, the application can also be improved as follows.

[0017] Further, the determination process of the node feature matrix and the adjacency matrix is as follows: The original traffic data is subjected to data cleaning to obtain first traffic data, and the data cleaning includes detection of outliers, removal of outliers and filling of missing values by spatiotemporal interpolation. The first traffic data is subjected to feature extraction to obtain spatial features and temporal features, and the adjacency matrix is constructed based on the spatial features, and the node feature matrix is constructed based on the temporal features.

[0018] Further, the forward propagation is as follows: The input data is added with position encoding through BTE, and the input data with position encoding is subjected to NASAN processing, and after the CMRC fusion processing, the output data is obtained through time dimension compression processing.

[0019] Further, the training process of the target model further includes: The target model is trained using single-step error loss, multi-scale aggregation error loss, and adaptive fusion total loss as constraints to obtain the reference model.

[0020] 3) In a third aspect, the present invention also provides an electronic device, the electronic device including a processor coupled to a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the electronic device to perform any of the methods described above.

[0021] 4) In a fourth aspect, the present invention also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to enable a computer to implement any of the above methods.

[0022] It should be noted that the beneficial effects of the technical solutions of the second to fourth aspects of the present invention and their corresponding possible implementations can be found in the above description of the technical effects of the first aspect and its corresponding possible implementations, and will not be repeated here. Attached Figure Description

[0023] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart illustrating a traffic flow prediction method based on multi-level cross-attention and neighborhood perception mechanism according to an embodiment of the present invention. Figure 2 This is a structural framework diagram of an electronic device according to the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0025] like Figure 1 As shown in the figure, a traffic flow prediction method based on multi-level cross-attention and neighborhood perception mechanism according to an embodiment of the present invention includes the following steps: S1. Preprocess the raw traffic data to obtain the node feature matrix and adjacency matrix. The preprocessing includes data cleaning, standardization and feature extraction. S2, through BTE and NASAN, transforms the node feature matrix and adjacency matrix into high-dimensional implicit representations, and constructs the encoding layer and feature extraction layer in the target model based on the high-dimensional implicit representations; S3, randomly segment the original traffic data to obtain multiple segments, and train the target model by combining forward propagation and back propagation to obtain the reference model; S4, sequentially performing verification evaluation, learning rate scheduling and early stopping check processing on the reference model, and when the reference model meets the preset requirement, obtaining target traffic data corresponding to the to-be-predicted area, predicting the target traffic data through the reference model to obtain a prediction result.

[0026] The traffic flow prediction method based on the multi-level cross attention and neighborhood perception mechanism has the following beneficial effects: By integrating data cleaning, standardization and feature extraction preprocessing, a node feature matrix and an adjacency matrix are obtained, and BTE is combined with NASAN to upgrade the original low-dimensional data to a high-dimensional hidden representation rich in space-time correlation. The model simultaneously captures long-term time dependence and multi-order neighborhood spatial relationship without increasing redundant parameters, significantly reducing the computational complexity. With random split multi-group data sets, forward-backward propagation closed-loop training, and verification-scheduling-early stopping three-level quality control, the model can adaptively adjust the learning rate and stopping time during the training phase, avoiding overfitting and improving the generalization performance. When the model meets the preset requirement, only one forward inference is needed to achieve millisecond-level high-precision prediction of real-time target traffic data. The overall inference speed is improved by more than 30% compared with existing methods, and the attention weight and feature importance score are provided for explainability, taking into account the prediction accuracy, real-time performance and explainability requirements.

[0027] It should be noted that the original traffic data refers to the most bottom-level observation records directly collected from the traffic monitoring system before any cleaning, standardization or feature processing, including but not limited to: instantaneous traffic flow, speed, occupancy, headway reported by each detection point (coil, microwave, video, floating car, mobile phone signaling or RSU roadside unit) in the road network at equal intervals or event triggered time; time stamp (accurate to seconds or milliseconds) and geographic location identifier (node number, latitude and longitude or road segment ID) attached to each record; and optional external synchronization information such as weather conditions, accident events, holiday markers. The data usually exists in the form of CSV, Parquet, database table or real-time Kafka stream, and may have quality problems such as missing values, abnormal jumps, clock drift and duplicate records. All subsequent preprocessing, feature extraction and model training are based on the original traffic data.

[0028] In another embodiment of the present scheme, the process of obtaining the node feature matrix and the adjacency matrix can include: Step 1: After reading the original traffic data, first perform outlier detection processing, identify outliers in flow, speed and occupancy based on the 3σ criterion or IQR boxplot method, and perform outlier removal processing to remove the entire row of data records judged to be abnormal; Step 2: Perform temporal-spatial interpolation to fill in missing values in the sparse matrix formed after the deletion, first use linear interpolation to complete short-time missing within the same node adjacent time window, if the time window missing exceeds the threshold, then use the weighted average of the same period of the spatial neighborhood nodes for interpolation, the weight is generated according to the road distance or correlation coefficient inverse distance decay, and the complete first traffic data is obtained; Step 3: Extract time features from the first traffic data, extract the hour, week, whether it is a weekday, whether it is a holiday, month, and season as the time feature vector, and perform periodic sine and cosine coding on the continuous time variable; Step 4: Synchronously extract spatial features, calculate the degree centrality, betweenness centrality, road length, lane number, historical average speed, and historical standard deviation of each detection point (node) as the spatial feature vector; Step 5: Construct an adjacency matrix, generate a basic adjacency matrix A_base based on the physical distance between nodes or road connectivity, and then calculate the distance decay weight w_ij=exp(-d_ij / σ) multiplied by the centrality weighted sum based on the road length and centrality in the spatial feature, to obtain the weighted adjacency matrix A; Step 6: Construct a node feature matrix, concatenate the time feature vector of each node at the same time with the spatial feature vector of the node to form a node feature vector, then stack all node feature vectors in the time dimension according to the timestamp, and finally obtain the node feature matrix X∈R^(N×D), where N is the number of nodes, D is the time feature dimension plus the spatial feature dimension, and the column of X is standardized by Z-score to complete the synchronous output of the node feature matrix and the adjacency matrix.

[0029] In another embodiment of the present scheme, the process of converting the node feature matrix and the adjacency matrix into high-dimensional hidden representation by BTE and NASA can include: After inputting the node feature matrix X∈R^(N×D) and the adjacency matrix A∈R^(N×N), it first undergoes bidirectional time encoding (BTE): for each time step t of X, the absolute position code P_abs(t,i)=sin(t / 10000^(2i / d)) or cos(t / 10000^(2i / d)) is calculated and fused with the relative position code P_rel(t,Δt)=R_θ(Δt)W_r according to the weight α to obtain P_final. P_final is then added to X to form the time-encoded feature X_te∈R^(N×D); X_te is fed into an L-layer neighborhood-aware self-attention network NASAN. Each layer first maps X_te to a query Q, key K, and value V using learnable matrices W_Q, W_K, and W_V. Then, a topological awareness attention score S = softmax((QK^T) / √d + γA_topo) is constructed using weighted first-order, second-order, and third-order adjacency power matrices A_topo = λ1A + λ2A² + λ3A³ and a modulation factor γ. The residual update output of SV + XG(X) is then completed by the gating function G(X) = sigmoid(XW_g). Interlayer dimension Maintain or increment, and obtain H_L after multi-layer stacking; input H_L into gradient modulation residual and connect to GMRC, first calculate feature importance E=sigmoid(MLP(H_L)) using MLP, and then multiply by the gradient norm. And by scaling η, we get the modulation factor M = ηtanh(E) Finally, the high-dimensional hidden representation F_L=H_L+H_L·M∈R^(N×D_L) is output as a unified representation of the coding layer and the feature extraction layer.

[0030] In another embodiment of this scheme, the process of constructing the encoding layer and feature extraction layer in the target model based on high-dimensional implicit representation includes: After directly using the high-dimensional hidden representation F_L∈R^(B,T,N,D_L) as the output of the encoding layer, (B,T,N,D_L) is first reshaped into (BT,N,D_L) and fed into the first layer of NASAN. In this layer, the input is first mapped to Q, K, and V by W_Q, W_K, and W_V. Then, the attention score S=softmax((QK^T) / √d+γA_topo) is calculated using the first- to third-order neighborhood matrices A_topo weighted by λ1, λ2, and λ3 and the γ modulation factor. The gated residual is then output as H1. H1 is then passed through the second to the Lth layers of NASAN. The above mapping-topological attention-gating update process is repeated in each layer, and the process is maintained in the channel dimension or progressively increased to the next level. This forms an L-layer stacked spatial-temporal feature extraction layer; after each NASAN layer, a GMRC module is inserted: the feature importance E is extracted using MLP and multiplied by the gradient norm. and scaled by η to get modulation factor M, the residual weighted result of current layer output and last layer fusion result is obtained ; finally, the Lth layer GMRC output F_L is compressed in time dimension (average pooling or 1x1 convolution) to get (B, N, D_out) as the unified output of the encoding layer and the feature extraction layer for the subsequent prediction head.

[0031] In another embodiment of the present scheme, the specific process of S3 includes: First, the original traffic data is divided into continuous samples in time sequence by sliding window, and the sample set is divided into training set, validation set and test set in the ratio of 7:2:1 by random shuffling strategy, and further randomly divided into multiple groups of mini-batches in the training set according to batch size B; for each mini-batch, forward propagation is performed: the node feature matrix and the adjacency matrix are encoded by BTE, and the high-dimensional hidden representation is extracted by L-layer NASAN+GMRC, and the prediction value is output by MLP , the single-step error, multi-scale error and adaptive fusion total loss Loss are calculated with the true value Y; back propagation is performed: the gradients of the prediction head, GMRC, each layer NASAN and BTE are calculated in turn from Loss, and all trainable parameters θ are updated using Adam, and at the end of each epoch, val_loss, MAE and RMSE are calculated using the validation set; if the validation loss does not improve for n consecutive epochs, the learning rate is halved, and if it still does not improve for m consecutive epochs, early stopping is triggered, and the θ corresponding to the best loss of the validation set is finally saved as the reference model.

[0032] In another embodiment of the present scheme, the specific implementation process of S4 includes: Load the saved best parameters After restoring the reference model, the validation set reserved in stage 1 is used to pass through the model as a whole, and MAE, RMSE, MAPE and custom threshold are calculated and recorded, if any of the indicators does not meet the requirements, a round of prediction-loss-back propagation-Adam update is performed on the validation set with the current learning rate, after completion, if the validation loss does not improve for n consecutive rounds, the learning rate is multiplied by 0.5 and the optimizer is updated synchronously, if it still does not improve for m consecutive rounds, early stopping is triggered to lock the parameters; when all validation indicators meet the preset requirements, the original traffic data of the area to be predicted is collected in real time, and after abnormal data is removed, spatio-temporal interpolation, feature extraction, standardization, adjacency matrix construction and node feature matrix generation are completed in the same way as stage 1, the reference model is directly input to perform a forward propagation, and the prediction result is output by BTE→NASAN→GMRC→MLP , and finally the standardized Ŷ is transformed back to the true scale to get the final traffic flow prediction value.

[0033] It needs to be further explained that the preset requirements are composed of a set of quantifiable performance indicators and business constraints, specifically including: verification set MAE≤x veh / km, RMSE≤y veh / km, MAPE≤z%, single batch inference delay≤t ms, continuous n epoch verification loss drop amplitude<ε, model parameter quantity≤M, and prediction coverage≥c% and explainability score≥s, wherein x, y, z, t, ε, M, c, and s are set by the actual deployment scene before training.

[0034] Further, the determination process of the node feature matrix and the adjacency matrix is as follows: The original traffic data is subjected to data cleaning processing to obtain first traffic data, and the data cleaning processing includes: detecting and processing abnormal values, removing abnormal values, and filling missing values by spatiotemporal interpolation. The first traffic data is subjected to feature extraction to obtain spatial features and temporal features, the adjacency matrix is constructed based on the spatial features, and the node feature matrix is constructed based on the temporal features.

[0035] Further, the forward propagation is specifically as follows: The input data is added with position encoding by BTE, and the input data after adding position encoding is subjected to NASAN processing, combined with CMRC fusion processing, and then subjected to time dimension compression processing to obtain output data.

[0036] Further, the training process of the target model further includes: The single-step error loss, the multi-scale aggregated error loss, and the adaptive fusion total loss are used as constraint conditions to train the target model to obtain a reference model.

[0037] In the above embodiments, although the steps are numbered S1, S2, etc., it is only a specific embodiment given by the present application, and those skilled in the art can adjust the execution order of S1, S2, etc. according to the actual situation, which is within the protection scope of the present application. It can be understood that in some embodiments, some or all of the above embodiments can be included.

[0038] Embodiment 1, the technical scheme of the present application includes the following modules: I. Data preprocessing module The data preprocessing module is responsible for cleaning, standardizing, and feature extraction of the original traffic data, including: Missing value filling: using spatiotemporal interpolation method to process missing data; Data standardization: using Z-score standardization method to normalize the data; Time feature extraction: extracting features such as hour, week, month, holiday, etc. in the timestamp.

[0039] II. Neighborhood-aware self-attention network layer (NASAN) NASAN is one of the core innovations of the present application. It effectively captures the spatial dependency in the traffic network by fusing local topology information and global self-attention mechanism.

[0040] The calculation process of NASAN is as follows: 1. Input feature matrix X ∈ R^(N×D), where N is the number of nodes and D is the feature dimension 2. Calculate query, key and value matrices: Q = XW_Q K = XW_K V = XW_V where W_Q, W_K, W_V ∈ R^(D×d) are learnable parameter matrices.

[0041] 3. Calculate topology-aware attention matrix: A_topo =λ_1 A +λ_2 (A^2) +λ_3 (A^3); where A is the adjacency matrix, λ_1, λ_2, λ_3 are learnable weight parameters, and A^n represents n-order neighborhood information. 4. Calculate neighborhood-aware self-attention score: S = softmax((QK^T) / sqrt(d) +γ A_topo) where γ is a learnable modulation factor.

[0042] 5. Output feature: Z = SV + X G(X) where G(X) is a gating function, G(X) = sigmoid(X W_g), W_g is a learnable parameter matrix.

[0043] Parameter interpretation of NASAN layer: W_Q, W_K, W_V: Projection matrices that project input features to query, key, and value spaces; λ_1, λ_2, λ_3: Weight coefficients of different order neighborhoods; γ: Modulation factor of topology structure attention; W_g: Parameter matrix of gating function.

[0044] III. Gradient modulation residual connection (GMRC) GMRC is another innovation of the present application, which enhances the modeling ability of long-term dependencies by adaptively adjusting the contribution of different features.

[0045] The calculation process of GMRC is as follows: 1. Input features X and transformed features F(X); 2. Calculate feature importance score: E(X) = sigmoid(MLP(X)); Where MLP is a multi-layer perceptron.

[0046] 3. Calculate gradient modulation factor: M(X) =η tanh(E(X) || ||_2); Where is the gradient of the loss function with respect to X, and η is a learnable scaling factor.

[0047] 4. Output features: Y = F(X) + X M(X) Parameter interpretation of GMRC: MLP: Multi-layer perceptron for calculating feature importance; η: Scaling factor of gradient modulation.

[0048] Four, Bidirectional Time Encoding Mechanism (BTE) BTE enhances the modeling ability of time patterns by combining absolute position encoding and relative position encoding.

[0049] The calculation process of BTE is as follows: 1. Calculate absolute position encoding: P_abs(t, i) = sin(t / 10000^(2i / d)), when i is even; P_abs(t, i) = cos(t / 10000^(2i / d)), when i is odd; Where t is the time step, i is the dimension index, and d is the model dimension.

[0050] 2. Calculate relative position encoding: P_rel(t,Δt) = R_θ(Δt) W_r; Where Δt is the relative time difference, R_θ is the rotation matrix function, and W_r is the learnable parameter matrix.

[0051] 3. Fusion position encoding: P_final = α P_abs + (1-α) P_rel; where α is the adaptive fusion weight.

[0052] 4. Apply position encoding to input features: X_t = X_t + P_final; Parameter interpretation of BTE: W_r: Parameter matrix of relative position encoding. α: Fusion weight of absolute position encoding and relative position encoding.

[0053] Five, multi-scale fusion loss function (MSFL) MSFL is another innovation point proposed by the present application, which improves the generalization ability of the model by optimizing the prediction accuracy of different time scales at the same time.

[0054] The calculation process of MSFL is as follows: 1. For each prediction time step from t+1 to t+H, calculate the prediction error: L_step(t) = β_1 MSE(y_t, ) + β_2 MAE(y_t, ); where y_t is the true value, is the predicted value, and β_1 and β_2 are weight coefficients.

[0055] 2. Calculate the aggregated error of different time scales: L_short = (1 / m) Σ(L_step(t)),t∈[1, m]; L_medium = (1 / n) Σ(L_step(t)),t∈[m+1, m+n]; L_long = (1 / p) Σ(L_step(t)),t∈[m+n+1, H]; where m, n, p are the step lengths of different time scales.

[0056] 3. Calculate the adaptive weight: w_short = exp(-δ L_short) / Z; w_medium = exp(-δ L_medium) / Z; w_long = exp(-δ L_long) / Z; where Z is a normalization factor and δ is a temperature parameter.

[0057] 4. Final loss function: L_final = w_short L_short + w_medium L_medium + w_long L_long +λ R(θ); where R(θ) is a regularization term and λ is a regularization coefficient.

[0058] Parameter interpretation of MSFL: β_1,β_2: Weight coefficients of MSE and MAE; m, n, p: Step sizes of different time scales; δ: Temperature parameter in adaptive weight calculation; λ: Regularization coefficient.

[0059] Six, overall model architecture 1. Input layer: receives spatio-temporal data and features; 2. Encoding layer: encodes temporal features using BTE; 3. Feature extraction layer: composed of multiple layers of NASAN, extracts spatio-temporal features; 4. Fusion layer: uses GMRC to fuse features at different levels; 5. Prediction layer: outputs traffic flow prediction results at future time steps; 6. Loss calculation: uses MSFL to calculate multi-scale prediction loss.

[0060] where 1. Input features X are encoded to obtain time-encoding features X_te; 2. X_te extracts spatio-temporal features through multiple layers of NASAN: H_1 = NASAN_1(X_te) H_2 = NASAN_2(H_1) ... H_L = NASAN_L(H_{L-1}) 3. Use GMRC to fuse features at different levels: F_1 = GMRC_1(H_1, X_te) F_2 = GMRC_2(H_2, F_1) ... F_L = GMRC_L(H_L, F_{L-1}) 4. Generate the prediction of future time step by the prediction layer: = MLP(F_L) 5. Calculate the loss using MSFL: Loss = MSFL(Y, )。

[0061] The application also provides a traffic flow prediction system based on a multi-level cross attention and neighborhood perception mechanism, and the specific technical solutions are as follows: The preprocessing module is used for preprocessing the original traffic data to obtain a node feature matrix and an adjacency matrix, and the preprocessing includes data cleaning processing, standardization processing and feature extraction processing; The construction module is used for converting the node feature matrix and the adjacency matrix into high-dimensional hidden representations through BTE and NASAN, and constructing an encoding layer and a feature extraction layer in the target model based on the high-dimensional hidden representations; The training module is used for randomly splitting the original traffic data to obtain multiple split sets, training the target model by combining forward propagation and back propagation, and obtaining a reference model; The prediction module is used for sequentially performing verification and evaluation, learning rate scheduling and early stopping checking on the reference model, and when the reference model meets the preset requirement, obtaining target traffic data corresponding to a to-be-predicted area, predicting the target traffic data by the reference model, and obtaining a prediction result.

[0062] It should be noted that the beneficial effects of the traffic flow prediction system based on the multi-level cross attention and neighborhood perception mechanism provided in the above embodiment are the same as those of the traffic flow prediction method based on the multi-level cross attention and neighborhood perception mechanism, and will not be repeated here. In addition, when the system provided in the above embodiment realizes its functions, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the system is divided into different functional modules according to actual conditions to complete all or part of the above described functions. In addition, the system and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is shown in the method embodiment, which will not be repeated here.

[0063] As Figure 2As shown, an electronic device 300 according to an embodiment of the present application includes a processor 320 coupled to a memory 310, and the memory 310 stores at least one computer program 330, and the at least one computer program 330 is loaded and executed by the processor 320, so that the electronic device 300 implements any of the above methods, in particular: The electronic device 300 can have a large difference due to different configurations or performances, and can include one or more processors 320 (Central Processing Units, CPUs) and one or more memories 310, wherein the one or more memories 310 store at least one computer program 330, and the at least one computer program 330 is loaded and executed by the one or more processors 320, so that the electronic device 300 implements the traffic flow prediction method based on the multi-level cross attention and neighborhood perception mechanism provided by the above embodiment. Of course, the electronic device 300 can also have a wired or wireless network interface, a keyboard, and an input and output interface, and other components for realizing the functions of the device, which are not described here.

[0064] A computer readable storage medium according to an embodiment of the present application stores at least one computer program, and the at least one computer program is loaded and executed by a processor, so that the computer implements any of the above methods.

[0065] Optionally, the computer readable storage medium can be a Read-Only Memory (ROM), a Random Access Memory (RAM), a Compact Disc Read-Only Memory (CD-ROM), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0066] In the exemplary embodiments, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. The processor of the electronic device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the electronic device executes any of the above methods.

[0067] It should be noted that the terms "first", "second" in the specification and claims of the present application are used to distinguish similar objects, and represent a specific order or sequence. The order of use of similar objects can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described.

[0068] Those skilled in the art will understand that the application can be implemented as a system, method or computer program product. Therefore, the present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.) or a combination of both software and hardware embodiments, which can all generally be referred to herein as a "circuit," "module" or "system." Furthermore, the present disclosure can take the form of a program product on one or more computer-readable medium(s) having computer-readable program code embodied in the medium.

[0069] Any combination of one or more computer-readable medium(s) can be utilized. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0070] Although the embodiments of the present application have been shown and described above, it is to be understood that the above-described embodiments are merely exemplary, and are not to be taken in a limiting sense, but the scope of the present application is not to be understood as being limited to the above-described embodiments, and various changes, modifications, replacements and variations can be made by those skilled in the art within the scope of the present application.

Claims

1. A traffic flow prediction method based on multi-level cross-attention and neighborhood perception mechanism, characterized in that, The method comprises the following steps: Preprocessing the original traffic data to obtain a node feature matrix and an adjacency matrix, wherein the preprocessing comprises data cleaning processing, standardization processing and feature extraction processing; Converting the node feature matrix and the adjacency matrix into high-dimensional hidden representations through BTE and NASA, and constructing an encoding layer and a feature extraction layer in a target model based on the high-dimensional hidden representations; Splitting the original traffic data randomly to obtain multiple split sets, training the target model by combining forward propagation and back propagation, and obtaining a reference model; When the reference model meets the preset requirements, obtaining target traffic data corresponding to a to-be-predicted region, predicting the target traffic data through the reference model, and obtaining a prediction result.

2. The traffic flow prediction method based on multi-level cross-attention and neighborhood perception mechanism according to claim 1, characterized in that, The determination process of the node feature matrix and the adjacency matrix is as follows: Performing data cleaning processing on the original traffic data to obtain first traffic data, wherein the data cleaning processing comprises detecting abnormal value processing, removing abnormal value processing and filling missing value processing through spatiotemporal interpolation; Extracting features from the first traffic data to obtain spatial features and temporal features, constructing the adjacency matrix based on the spatial features, and constructing the node feature matrix based on the temporal features.

3. The traffic flow prediction method based on multi-level cross-attention and neighborhood perception mechanism according to claim 1, characterized in that, The forward propagation is specifically as follows: Adding position encoding to the input data through BTE, performing NASA processing on the input data after adding the position encoding, and obtaining output data through time dimension compression processing after CMRC fusion processing.

4. The traffic flow prediction method based on multi-level cross-attention and neighborhood perception mechanism according to claim 1, characterized in that, The training process of the target model further comprises: Training the target model by taking single-step error loss, multi-scale aggregated error loss and adaptive fusion total loss as constraint conditions to obtain a reference model.

5. A traffic flow prediction system based on multi-level cross-attention and neighborhood perception mechanism, characterized in that, The method comprises the following steps: The preprocessing module is configured to preprocess the original traffic data to obtain a node feature matrix and an adjacency matrix, wherein the preprocessing comprises data cleaning processing, standardization processing and feature extraction processing; The construction module is configured to convert the node feature matrix and the adjacency matrix into high-dimensional hidden representations through BTE and NASA, and construct an encoding layer and a feature extraction layer in a target model based on the high-dimensional hidden representations; The training module is configured to split the original traffic data randomly to obtain multiple split sets, train the target model by combining forward propagation and back propagation, and obtain a reference model; The prediction module is configured to sequentially perform validation evaluation, learning rate scheduling and early stopping checking on the reference model, and when the reference model meets the preset requirements, obtain target traffic data corresponding to a to-be-predicted region, predict the target traffic data through the reference model, and obtain a prediction result.

6. The traffic flow prediction system based on multi-level cross-attention and neighborhood perception mechanism according to claim 5, wherein, The determination process of the node feature matrix and the adjacency matrix is as follows: Performing data cleaning processing on the original traffic data to obtain first traffic data, wherein the data cleaning processing comprises detecting abnormal value processing, removing abnormal value processing and filling missing value processing through spatiotemporal interpolation; Extracting features from the first traffic data to obtain spatial features and temporal features, constructing the adjacency matrix based on the spatial features, and constructing the node feature matrix based on the temporal features. The first traffic data is subjected to feature extraction to obtain spatial features and time features, the adjacency matrix is constructed based on the spatial features, and the node feature matrix is constructed based on the time features.

7. The traffic flow prediction system based on multi-level cross-attention and neighborhood perception mechanism according to claim 5, wherein, The forward propagation specifically comprises: The input data is added with position encoding by the BTE, and the input data added with the position encoding is subjected to NASAN processing, combined with CMRC fusion processing, and then subjected to time dimension compression processing to obtain output data.

8. The traffic flow prediction system based on multi-level cross-attention and neighborhood perception mechanism according to claim 5, wherein, The training process of the target model further comprises: The single-step error loss, the multi-scale aggregated error loss and the adaptive fusion total loss are used as constraint conditions to train the target model to obtain a reference model.

9. An electronic device, comprising: The electronic device comprises a processor coupled with a memory, and the memory stores at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the method of any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one computer program, which is loaded and executed by the processor to enable the computer to implement the method of any one of claims 1 to 4.

Images