Traffic prediction method based on dynamic graph spatiotemporal correlation and adaptive adversarial training

Abstract

The present application relates to a traffic prediction method based on dynamic graph spatiotemporal correlation and adaptive adversarial training. The method includes: using an adaptive graph convolutional network to obtain a dynamic graph spatial embedding representation by learning dynamic graph data capture; capturing dynamic graph data according to a temporal attention network to obtain a dynamic graph temporal embedding representation; obtaining a dynamic graph embedding; using the dynamic graph embedding as the starting embedding and the pre-set last adversarial node embedding as the context node embedding as the input of the policy network, setting an adversarial reward function and a loss function, and then training the policy network; updating the dynamic graph according to the trained policy network output, capturing spatial correlation and temporal correlation on the updated dynamic graph, and obtaining the traffic prediction result corresponding to the current network device. This method can effectively deal with data pollution and node anomalies, and adapt to changing network conditions to achieve traffic prediction.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and data analysis technology, and in particular to a traffic prediction method based on dynamic graph spatiotemporal correlation and adaptive adversarial training. Background Art

[0002] Network traffic forecasting is a technology that uses historical network traffic data to estimate future network traffic volume and trends. It is crucial in network planning, operations, and management, helping network managers more effectively allocate network resources and improve network performance and reliability. By estimating future network loads, network traffic forecasting allows for proactive resolution of overloads before they occur, thereby increasing network availability. This approach is more effective than traditional reactive behavior. Frequent data transmission and information exchange are required between numerous geographically distributed systems. While continuous improvements in network infrastructure generally meet basic communication requirements, bandwidth demands may not be met in certain areas or at certain times due to irrational resource allocation. Therefore, leveraging artificial intelligence and machine learning technologies can help communication systems better predict user network traffic demands, proactively optimize resource deployment and execution strategies, improve overall system performance, effectively respond to unexpected demands, enhance network stability and service quality, reduce costs, and improve efficiency. Network nodes, such as servers, routers, and switches, bear the crucial responsibilities of receiving, processing, and forwarding data. However, in complex network environments, nodes are often subject to interference. These anomalies, such as those caused by erroneous input or malicious attacks, can lead to inaccurate traffic forecasts, resulting in erroneous resource allocation strategies and reduced response times. Therefore, there is an urgent need to design a robust traffic forecasting method that maintains high accuracy under various uncertainties.

[0003] Traditional robust traffic prediction methods are primarily categorized as those based on robust statistics, machine learning, and deep learning. Robust statistical methods, such as the robust autoregressive moving average (ARMA), mitigate the impact of outliers by adjusting their error terms and are suitable for linear time series with outliers. In machine learning, ensemble learning methods, such as random forests and gradient boosting machines, combine the results of multiple prediction models to reduce the impact of individual model biases, thereby enhancing prediction robustness. Deep learning techniques, particularly those employing denoising autoencoders and recurrent neural network variants, such as long short-term memory (LSTM) networks with attention mechanisms, can effectively handle complex nonlinear patterns in data and provide reliable predictions even with unstable data quality. Overall, while traditional robust prediction methods can effectively handle outliers to a certain extent and ensure stable predictions, they often perform poorly in the face of dynamic network environments. With the continuous evolution of network attack patterns and the increase in attack types, traditional prediction methods struggle to adapt quickly, resulting in a decline in prediction accuracy. Furthermore, most adversarial training methods are designed for static environments and have limited effectiveness in dynamic traffic prediction tasks. Fixed, static strategies are ineffective in the face of constantly changing network environments. Summary of the Invention

[0004] Based on this, it is necessary to provide a traffic prediction method based on dynamic graph spatiotemporal correlation and adaptive adversarial training that can effectively deal with data pollution and node anomalies, and adapt to changing network conditions to achieve traffic prediction, in order to address the above technical problems.

[0005] A traffic prediction method based on dynamic graph spatiotemporal correlation and adaptive adversarial training, the method comprising:

[0006] Modeling network devices as dynamic graphs; dynamic graphs include node sets, edge sets, and adjacency matrices; node sets include multiple nodes; nodes include servers, routers, and switches; obtaining dynamic graph data corresponding to network devices at historical moments;

[0007] The adaptive graph convolutional network is used to capture the dynamic changes of nodes and edges in the dynamic graph data to obtain the spatial embedding representation of the dynamic graph; the temporal correlation between nodes in the dynamic graph data is captured by the temporal attention network to obtain the temporal embedding representation of the dynamic graph;

[0008] The dynamic graph spatial embedding representation and the dynamic graph temporal embedding representation input are nonlinearly activated and linearly combined to obtain the dynamic graph embedding; the dynamic graph embedding is used as the starting embedding and the pre-set last adversarial node embedding is defined as the context node embedding as the input of the policy network;

[0009] In the policy network, the context nodes are updated using the multi-head attention mechanism to obtain the updated initial node embedding. The updated initial node embedding is used to calculate the probability of the next node, and the node with the highest probability is selected as the next adversarial node to be sampled. The adversarial node is applied to the policy network, and the adversarial reward function and loss function of the policy network are set according to the change in prediction performance before and after the adversarial node is applied.

[0010] The policy network is trained using an adversarial reward function and a loss function, and an updated adversarial node is output according to the trained policy network. The dynamic graph is updated according to the updated adversarial node, and the spatial correlation and temporal correlation of the updated dynamic graph are captured to obtain the traffic prediction result corresponding to the current network device.

[0011] The above-mentioned traffic prediction method based on spatiotemporal correlation of dynamic graphs and adaptive adversarial training, this application models the network device as a dynamic graph, uses the adaptive graph convolutional network to capture the dynamic changes of nodes and edges in the dynamic graph data to obtain the dynamic graph spatial embedding representation; according to the temporal attention network, the temporal correlation between nodes in the dynamic graph data is captured to obtain the dynamic graph temporal embedding representation, and the adaptive graph convolutional network is used to effectively mine the complex spatial interaction patterns between nodes, while the temporal attention network keenly captures the dynamic evolution characteristics of time series traffic. The two collaborate to understand and predict the dynamic changes of traffic patterns, effectively improving the accuracy of traffic data prediction. Furthermore, the present application designs a strategy generation mechanism, which performs nonlinear activation and linear combination on the input of the dynamic graph spatial embedding representation and the dynamic graph temporal embedding representation to obtain the dynamic graph embedding; the dynamic graph embedding is used as the starting embedding and the pre-set last adversarial node embedding is defined as the context node embedding as the input of the policy network; the context node is updated in the policy network according to the multi-head attention mechanism, the updated initial node embedding is used to calculate the probability of the next node, and the node with the highest probability is selected from all nodes as the next sampled adversarial node to automatically generate an adversarial node sequence, which represents a specific set of nodes used to generate adversarial samples, simulating the response of the network when encountering hypothetical attacks or failures. By applying the projected gradient descent method, the adversarial samples play a "stress test" function in the training cycle, constantly tempering the anti-interference strength of the method. Finally, by setting the policy network loss function to systematically evaluate and fine-tune the performance of the nodes selected by the policy network, the overall adversarial training framework is further iteratively optimized to ensure that the traffic prediction method maintains a high degree of stability and predictability in the rapidly changing network ecology. This application uses a dynamic node selection strategy for adversarial training to address the lack of robustness of current traffic prediction methods in the face of data contamination and node anomalies. It also improves the accuracy and stability of predictions through an adaptive graph convolutional network and a temporal attention mechanism. Furthermore, this application uses real-time traffic data input to output future traffic trends and potential anomaly warnings to guide network operations and policy tuning. This includes flexible bandwidth allocation, intelligent optimization of routing strategies, and pre-emptive identification and mitigation of network risks, ensuring uninterrupted and secure network services. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 1 is a flow chart of a traffic prediction method based on spatiotemporal correlation of dynamic graphs and adaptive adversarial training in one embodiment;

[0013] Figure 2 A schematic diagram of a network traffic monitoring, analysis, and prediction process in one embodiment;

[0014] Figure 3 A schematic diagram of a strategic neural network structure in one embodiment;

[0015] Figure 4 Schematic diagram of the flow structure of traffic prediction based on dynamic graph spatiotemporal correlation and adaptive adversarial training in another embodiment;

[0016] Figure 5 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0018] In one embodiment, Figure 1 As shown, a traffic prediction method based on dynamic graph spatiotemporal correlation and adaptive adversarial training is provided, which includes the following steps:

[0019] Step 102: Model the network device as a dynamic graph; the dynamic graph includes a node set, an edge set, and an adjacency matrix; the node set includes multiple nodes; the nodes include servers, routers, and switches; and obtain dynamic graph data corresponding to the network device at a historical moment.

[0020] In this application, servers, routers, and switches are regarded as nodes in the figure. They are the cornerstones of the network architecture and play a decisive role in the control of data flow. Traffic data, reflected in the amount and frequency of data packet transmission on these nodes, is a direct indicator for evaluating the active status of the network. It is collected in real time through SNMP and NetFlow, and comprehensively covers information such as bandwidth usage, upstream and downstream rates, and data packet size. The robust traffic prediction method proposed in this application uses real-time traffic data input to output future traffic trends and potential anomaly warnings to guide network operation and maintenance and policy tuning, covering flexible allocation of bandwidth configuration, intelligent optimization of routing strategies, and pre-identification and mitigation of network risks to ensure the uninterrupted and secure network services. Figure 2 As shown in the figure, a systematic process of traffic analysis and prediction is demonstrated.

[0021] In step 104, an adaptive graph convolutional network is used to capture the dynamic changes of nodes and edges in the dynamic graph data to obtain a spatial embedding representation of the dynamic graph; and a temporal attention network is used to capture the temporal correlation between nodes in the dynamic graph data to obtain a temporal embedding representation of the dynamic graph.

[0022] Dynamic graphs can reflect changes in network status in real time, including updates to node connectivity and node features, enabling the method to adapt to complex and ever-changing network environments, thereby improving prediction accuracy and method response speed. The traffic data is represented as a dynamic graph G = (V, E, A), where V is a node set representing each node in the network, and |V| = N represents the number of nodes N in the node set V; E is a set of edges representing the connection relationship between nodes. is the adjacency matrix. The dynamic graph can dynamically reflect the information changes of nodes and edges in the network, especially by continuously updating in time series, thus truly simulating the dynamic behavior of the actual network. Define the function F θ , to predict the next τ steps of data based on the past T steps of historical data:

[0023] {X t+1 ,X t+2 ,…,X t+τ}=F θ (X t ,X t-1 ,…,X t-T+1 ; G)

[0024] Among them, X t represents the flow data at time t, {X t+1 ,X t+2 ,...,X t+τ} represents the predicted traffic data from time t+1 to time t+τ. Prediction function F θ Using the traffic data of the past T steps {X t ,X t-1 ,...,X t-T+1} and dynamic graph G to predict the traffic data for the next τ steps, and θ represents the learnable parameters in the method.

[0025] Adaptive graph convolutional neural networks are used to optimize the spatial dependencies between nodes in dynamic graphs and reduce the number of parameters. Adaptive graph convolutional networks capture the dynamic characteristics of network structures by learning the dynamic changes of nodes and edges in dynamic graphs, thereby more accurately reflecting the actual state of the network. Two parameter matrices are learned: the node embedding matrix and the and the weight matrix Where d is the embedding dimension. Through matrix decomposition, the weight matrix is ​​represented as M AS W AS , the formula is as follows:

[0026]

[0027] In order to fully reflect the spatial correlation of the data, the dependencies between nodes in the dynamic graph are adaptively inferred. Randomly initialize the node embedding matrix where R AEach row represents the embedding of a node, and d represents the embedding dimension of the node. The spatial dependencies between nodes in the dynamic graph are inferred by the following formula:

[0028]

[0029] Among them, ReLU is the activation function, represents the Hadamard product, M is used to adjust the weight of each element, B is the bias matrix, and the adaptive graph matrix R A It is automatically updated during training to learn node features in dynamic graphs.

[0030] By optimizing the spatial dependencies between nodes in dynamic graphs through adaptive graph convolutional networks, we can gain a deeper understanding of how nodes in the network influence each other and how these interactions change, thereby more accurately predicting traffic data.

[0031] In dynamic graphs, traffic data not only has spatial dependencies between nodes, but also has strong temporal correlations, especially traffic at similar times often has similar patterns. In order to better capture these spatiotemporal characteristics, this application uses a temporal attention network to capture the temporal correlation of dynamic graphs. The temporal attention network includes an encoding module, a graph attention module, a multi-layer capture module, a feature reconstruction module, an important feature matrix module, and a feature learning module; according to the encoding module, the input dynamic graph data is encoded to obtain a node feature matrix and a timestamp node feature matrix; in the graph attention module, a fused attention matrix is ​​calculated based on the input node feature matrix and the timestamp node feature matrix; in the multi-layer capture module, the node features of the next moment are captured based on the fused attention matrix at the current moment; in the feature reconstruction module, the captured node feature matrix is ​​reconstructed, and the node features of the current moment are supplemented by calculating the difference between the features output by the static attention of the graph and the features output by the dynamic attention of the graph to obtain the reconstructed features; in the important feature matrix module, the reconstructed features are processed according to the self-attention mechanism to obtain an important feature matrix; in the feature learning matrix, the important feature matrix is ​​feature learned to obtain a temporal embedding representation of the dynamic graph.

[0032] Temporal attention mechanisms analyze the temporal correlations between nodes in a dynamic graph to better capture and predict traffic changes in dynamic networks. This process understands and exploits patterns in time series, improving the ability to capture dynamic network behavior.

[0033] Step 106, nonlinearly activate and linearly combine the dynamic graph spatial embedding representation and the dynamic graph temporal embedding representation input to obtain the dynamic graph embedding; the dynamic graph embedding is used as the starting embedding and the pre-set last adversarial node embedding is defined as the context node embedding as the input of the policy network.

[0034] Step 108: Update the context node in the policy network according to the multi-head attention mechanism to obtain the updated initial node embedding; use the updated initial node embedding to calculate the next node probability, and select the node with the highest probability among all nodes as the next sampled adversarial node; apply the adversarial node to the policy network, and set the adversarial reward function and loss function of the policy network according to the change in prediction performance before and after applying the adversarial node.

[0035] In this application, the policy network is designed to dynamically adapt to the time evolution of the graph structure and specifically select key nodes that may significantly affect the prediction performance of the method when attacked or contaminated. The core of the policy network is a multi-head attention mechanism that is optimized to process the real-time features of nodes in the dynamic graph, thereby more accurately evaluating the criticality of each node. Its structure is as follows Figure 3 As shown. The dynamic graph spatial embedding representation and the dynamic graph temporal embedding representation are activated nonlinearly and then linearly combined to obtain the final dynamic graph embedding, namely:

[0036] U=c1·Φ1(O)+(1-c1)·Φ2(h)

[0037] Among them, Φ1 and Φ2 are learnable parameterized activation functions, and c1 balances the proportion of spatial embedding and temporal embedding.

[0038] The input of the policy network is the context node embedding, including the starting embedding and the last adversarial node embedding. The context node embedding is defined as follows:

[0039]

[0040] where v is a learned embedding in the first iteration step, is the embedding of the last selected adversarial node in the k-1th iteration step.

[0041] To update the context node embedding, the multi-head attention mechanism is calculated as follows:

[0042]

[0043] Among them, ∥ represents the serial operation, J is the number of heads of multi-head attention, MHA j It is the multi-head attention mechanism of the j-th head, and the calculation formula is as follows:

[0044]

[0045] Query vector q in attention mechanism (c) , key vector k j Sum value vector v j is defined as follows:

[0046] k j =W j K H′,v j =W j V H′(j=1,…,M)

[0047] in, W j K , W j V is the linear transformation matrix, and H′ is the output of the previous layer.

[0048] To calculate the probability of the next node, we first calculate the query vector q and key vector k based on the initial node embedding i :

[0049] q=W Q U (c) ′,k i =W K H i '

[0050] First, we compute the logit using a single-headed attention, querying all nodes with the updated context node:

[0051]

[0052] Where C is a constant, d k is the dimension of the key vector, r j is the attention score of node j. The selected node is represented by r j = -∞ or the smallest negative number that the computer can express to mask. Then use the softmax function to calculate the final probability of the node:

[0053]

[0054] Among them, p i is the probability of node i, ω k is the current node, s is the current state, ω 1:k-1 is the node sequence selected previously, r i is the attention score of node i. The node with the highest probability is selected as the next sampled adversarial node among all nodes, ensuring that the policy network not only considers the immediate characteristics of the node, but also evaluates its potential threat to network stability during the dynamic evolution of the entire network.

[0055] Through a dynamic graph-based policy network, when the predicted performance may be threatened, the system's defense capability is enhanced by targeted selection of key nodes, thereby improving the robustness of the entire network.

[0056] To evaluate the accuracy of the policy network in generating adversarial nodes, this application uses internal loss (the change in prediction performance before and after applying the adversarial node) as a reward. Since dynamic graphs can reflect the information changes of nodes and edges in the network in real time, this application uses this feature to provide a more accurate evaluation. The larger the loss value, the better the generated adversarial node. The calculation formula is as follows:

[0057]

[0058] in is the mean square error loss function, X′ t is the historical data after adversarial perturbation, and G is a dynamic graph that captures the changes of nodes and edges in the network through dynamic updates.

[0059] As training progresses, the method becomes more robust and the internal loss is expected to decrease. This can lead to incorrect feedback and optimization jitter. To improve adversarial training, knowledge distillation is used to transfer the knowledge of the teacher network to the student network. By leveraging the spatiotemporal characteristics captured by the dynamic graph, the previously trained network is used as the teacher network and the current network to be trained is used as the student network. The knowledge distillation reward is defined as follows:

[0060]

[0061] Among them, L KD is the knowledge distillation loss, is the mean square error loss function, X′ t is the historical data after adversarial disturbance, is the teacher network trained last time.

[0062] In order to enhance the learning of the policy network, the Markov distance between the current network and the teacher network is calculated to increase the training reward. The formula is as follows:

[0063]

[0064] Among them, T K (d) shows the method of calculating distance, which takes into account the dynamic changes of nodes and edges in the dynamic graph.

[0065] Using the Markov distance as part of the reward ensures that the distance between the target policy network and the teacher network is as small as possible. The final adversarial reward is:

[0066]

[0067] where c is a constant, and α1 and α2 are parameters that control the amount of knowledge transferred from the teacher network and the weight of the Markov distance reward.

[0068] By leveraging the dynamic characteristics of dynamic graphs, we can effectively evaluate and select the optimal adversarial nodes, enhance the robustness of the network in the face of data pollution and node anomalies, and ensure the prediction performance and stability of dynamic graphs.

[0069] Finally, the policy network is trained to optimize the dynamic graph prediction performance. The policy network is trained by alternating training of the policy network and the traffic prediction method. The policy network is trained according to the input X. t-τ:t Generate an adversarial node sequence, denoted as Ω. Then calculate the reward of the corresponding operation and use the reward to update the policy network. Next, use the selected adversarial node sequence to calculate the adversarial sample X′ through the projected gradient descent method. t-τ:t′ :

[0070]

[0071] The constraints are:

[0072]

[0073] Among them, X t+1:t+τ ={X t+1 ,X t+2 ,…,X t+τ} is the next τ step data, ∈ is a predefined perturbation budget that limits the maximum change between the adversarial sample and the original data to ensure that the generated adversarial sample is within a reasonable range. The adversarial sample is generated by X at the i-th iteration. t -τ:t ′(i) Indicates that γ is the step size, I t ∈{0,1} n is the final node selection metric obtained from the policy network, is the mean square error loss function, sign is the sign function, indicating the gradient direction.

[0074] The traffic prediction method then uses these adversarial examples to optimize the prediction loss:

[0075]

[0076] To train the policy network, the loss function is defined as follows:

[0077]

[0078] Where c is a constant, s represents a problem instance, which consists of traffic data from time point t-T+1 to time point t. The policy network is optimized by gradient descent using the Adam optimizer. The detailed operation steps of the traffic robust prediction method are detailed in Figure 4, adversarial node sequences are generated through policy networks, adversarial samples are generated using the projected gradient descent method, and the training process is optimized through internal loss and knowledge distillation rewards, so that dynamic graphs can adapt to complex and dynamic application environments and improve the defense capabilities of dynamic graphs against dynamic attacks.

[0079] Step 110: train the policy network using the adversarial reward function and the loss function, and output an updated adversarial node based on the trained policy network; update the dynamic graph based on the updated adversarial node, and capture the spatial correlation and temporal correlation of the updated dynamic graph to obtain the traffic prediction result corresponding to the current network device.

[0080] The policy network is trained using an adversarial reward function and a loss function, and an updated adversarial node is output based on the trained policy network; the dynamic graph is updated based on the updated adversarial node, and the adversarial node in the initial dynamic graph is replaced with the updated adversarial node to obtain an updated dynamic graph, and then the spatial correlation and temporal correlation of the updated dynamic graph are captured, and the captured dynamic graph spatial embedding representation and temporal embedding representation are passed through a two-dimensional convolution layer to obtain the traffic prediction result corresponding to the current network device. In the above-mentioned traffic prediction method based on dynamic graph spatiotemporal correlation and adaptive adversarial training, the present application models the network device as a dynamic graph, and uses an adaptive graph convolutional network to capture the dynamic changes of nodes and edges in the dynamic graph data to obtain a dynamic graph spatial embedding representation; the temporal correlation between nodes in the dynamic graph data is captured based on the temporal attention network to obtain a dynamic graph temporal embedding representation, and the adaptive graph convolutional network effectively mines the complex spatial interaction patterns between nodes, while the temporal attention network keenly captures the dynamic evolution characteristics of time series traffic. The two collaborate to understand and predict the dynamic changes of traffic patterns, effectively improving the accuracy of traffic data prediction. Furthermore, the present application designs a strategy generation mechanism, which performs nonlinear activation and linear combination on the input of the dynamic graph spatial embedding representation and the dynamic graph temporal embedding representation to obtain the dynamic graph embedding; the dynamic graph embedding is used as the starting embedding and the pre-set last adversarial node embedding is defined as the context node embedding as the input of the policy network; the context node is updated in the policy network according to the multi-head attention mechanism, the updated initial node embedding is used to calculate the probability of the next node, and the node with the highest probability is selected from all nodes as the next sampled adversarial node to automatically generate an adversarial node sequence, which represents a specific set of nodes used to generate adversarial samples, simulating the response of the network when encountering hypothetical attacks or failures. By applying the projected gradient descent method, the adversarial samples play a "stress test" function in the training cycle, constantly tempering the anti-interference strength of the method. Finally, by setting the policy network loss function to systematically evaluate and fine-tune the performance of the nodes selected by the policy network, the overall adversarial training framework is further iteratively optimized to ensure that the traffic prediction method maintains a high degree of stability and predictability in the rapidly changing network ecology. This application uses a dynamic node selection strategy for adversarial training to address the lack of robustness of current traffic prediction methods in the face of data contamination and node anomalies. It also improves the accuracy and stability of predictions through an adaptive graph convolutional network and a temporal attention mechanism. Furthermore, this application uses real-time traffic data input to output future traffic trends and potential anomaly warnings to guide network operations and policy tuning. This includes flexible bandwidth allocation, intelligent optimization of routing strategies, and pre-emptive identification and mitigation of network risks, ensuring uninterrupted and secure network services.

[0081] In one embodiment, an adaptive graph convolutional network is used to capture the spatial correlation of a dynamic graph by learning the dynamic changes of nodes and edges in the dynamic graph data, including:

[0082] The adaptive graph convolutional network is used to capture the dynamic changes of nodes and edges in dynamic graph data to obtain the spatial correlation of dynamic graphs.

[0083]

[0084] Where N is the total number of nodes in the dynamic graph, σ is the Sigmoid activation function, i is the current node, j is its neighbor node, k represents all other nodes that may affect node i, I N is the unit matrix, I represents the input dynamic graph data, O represents the output dynamic graph spatial correlation, is the adjacency matrix, D is the degree matrix, is the node embedding matrix, is the weight pool, is the bias pool, where d is the embedding dimension, C is the feature dimension, F is the output feature dimension, and ReLU is the activation function. represents the Hadamard product, M is used to adjust the weight of each element, B is the bias matrix, R A It is an adaptive graph matrix, which is automatically updated during training. exp is an exponential function used to calculate the exponential power of e.

[0085] In one embodiment, the temporal attention network includes an encoding module, a graph attention module, a multi-layer capture module, a feature reconstruction module, an important feature matrix module, and a feature learning module. The temporal correlation between nodes in the dynamic graph data is captured by the temporal attention network to obtain a temporal embedding representation of the dynamic graph, including:

[0086] Encode the input dynamic graph data according to the encoding module to obtain the node feature matrix and the timestamp node feature matrix;

[0087] In the graph attention module, the fused attention matrix is ​​calculated based on the input node feature matrix and the timestamp node feature matrix;

[0088] Update the fused attention matrix in the multi-layer capture module to obtain the updated node matrix;

[0089] In the feature reconstruction module, the updated node matrix is ​​reconstructed. The node features at the current moment are supplemented by calculating the difference between the features output by the static attention graph and the features output by the dynamic attention graph to obtain the reconstructed features.

[0090] In the important feature matrix module, the reconstructed features are processed according to the self-attention mechanism to obtain the important feature matrix;

[0091] Feature learning is performed on the important feature matrix in the feature learning matrix to obtain the dynamic graph time embedding representation.

[0092] In a specific embodiment, to better capture temporal correlations, this application uses a combination of graph static attention and graph dynamic attention to highlight the importance of changes in node attributes at different time points. Graph static attention is used to capture fixed node attributes, while graph dynamic attention is used to capture changes in node attributes over time. This design can simultaneously consider node attributes at a specific point in time and attributes that change over time, thereby improving the performance of the method at different times and enhancing its ability to process dynamic graph data.

[0093] To further capture temporal dependencies, this application uses a multi-layer structure to capture dependencies at different time points:

[0094]

[0095] But in most scenarios, the graph static attention matrix A static is 0, so the above formula can also be written as

[0096]

[0097] Among them, H (l) is the node feature matrix of the lth layer. This design can capture the spatiotemporal dependencies of node features at different times, thereby improving the performance of the prediction method in dynamic graph data processing.

[0098] In order to further enhance the feature representation capability of the method, this application introduces a variable Transformer for feature reconstruction. The variable Transformer is a complex network composed of different modules that can sample features of traffic data at important time points and process these features through a classic multi-head attention layer. First, key time points are selected and feature sampling of traffic data at these time points is performed. Then, a multi-head attention mechanism is used to process these features, splicing the features output by each attention head together and adjusting them through linear transformations and offsets. The core is to compare the difference between the features output by the static attention of the graph and the features output by the dynamic attention of the graph:

[0099] Q=[Q1,Q2,…,Q h ]

[0100] K=[K1,K2,…,K h ]

[0101] V=[V1,V2,…,Vh ]

[0102] head i =Attention(Q i ,K i ,V i )

[0103]

[0104] ΔF=F static -F dynamic

[0105] Among them, F static and F dynamic are the output features of static and dynamic attention respectively, Q, K, V are the matrices of query, key and value respectively, and W O is the output linear transformation matrix. This difference ΔF is assigned to the currently selected time graph data to adjust and compensate for the differences between features, thereby accurately reflecting time dependencies. This approach not only improves prediction capabilities but also enhances the ability to capture and process features at key time points.

[0106] Finally, the relationship matrix, that is, the important feature matrix, is determined by summarizing the key features of the entire sequence into a set to capture the mutual relationship between the features, as follows:

[0107] Q=FW Q ,K=FW K ,V=FW V

[0108]

[0109] F att =LayerNorm(F+A)

[0110] Among them, F represents the input feature, W Q 、W K and W V is the weight matrix of query, key and value, A is the attention matrix, and LayerNorm represents the layer normalization operation.

[0111] The information in these relationship matrices is applied to each query point in a weighted manner. This design effectively fuses information from a categorical perspective and ultimately applies the weighted results to specific tasks, improving the method's predictive accuracy and robustness. This provides higher performance and more precise results, particularly when processing complex time series data.

[0112] To enhance the accuracy of the prediction method, the temporal attention mechanism is used to capture the long-term dynamic temporal relationship of traffic data in the dynamic graph. The formula is as follows:

[0113] E=V e ·σ(((x′ i,t ) T W1)W2(x′ i,t W3)+b e )

[0114] Where V e ,b e ,W1,W2,W3 are all learnable parameters, x′ i,t Represents the input data x at time t i The result after normalization.

[0115] Temporal attention mechanisms analyze the temporal correlations between nodes in a dynamic graph to better capture and predict traffic changes in dynamic networks. This process understands and exploits patterns in time series, improving the ability to capture dynamic network behavior.

[0116] In one embodiment, the graph attention module calculates the fused attention matrix based on the input node feature matrix and the timestamp node feature matrix, including

[0117] In the graph attention module, the graph static attention matrix is ​​calculated based on the input node feature matrix:

[0118] A static =softmax(HW static H T )

[0119] Where H is the node feature matrix, the superscript T represents the transpose operation, and W static Represents the weight matrix of static features, and softmax represents the normalized exponential function, which is used to convert the vector into a probability distribution;

[0120] The graph dynamic attention matrix is ​​calculated based on the input timestamp node feature matrix:

[0121]

[0122] Among them, W dynamic represents the dynamic weight matrix;

[0123] The graph static attention matrix and the graph dynamic attention matrix are fused by weighted addition to obtain the fused attention matrix:

[0124] A combined =αA static +(1-α)A dynamic

[0125] Among them, α is a weight parameter used to control the contribution of static and dynamic attention mechanisms to the fused attention matrix.

[0126] In one embodiment, the fused attention matrix is ​​updated in the multi-layer capture module to obtain an updated node matrix, including:

[0127] The fusion attention matrix is ​​updated in the multi-layer capture module, and the updated node matrix is ​​obtained as

[0128]

[0129] Among them, A combined Represents the fusion attention matrix, ReLu is the activation function, is the normalized degree matrix, H (l) is the node feature matrix of the lth layer, is the feature weight matrix of the lth layer.

[0130] In one embodiment, feature learning is performed on an important feature matrix in a feature learning matrix to obtain a temporal embedding representation of a dynamic graph, including:

[0131] The feature learning matrix is ​​used to learn the important feature matrix, and the dynamic graph time embedding is expressed as

[0132]

[0133] Among them, F att Represents the important feature matrix, R A represents the adaptive graph matrix, E represents the long-term dynamic time relationship of traffic data in the dynamic graph, tanh is the activation function, [·] represents the connection operation, ⊙ represents the element-by-element product operation, z and r are the reset gate and update gate respectively, t represents the current time, σ represents the Sigmoid activation function, represents the candidate hidden state at the current moment, h t-1 Represents the dynamic graph time embedding representation of the previous moment of the current moment, softmax represents the normalized exponential function, which is used to convert the vector into a probability distribution. is an intermediate variable, representing the dependencies between the inferred nodes, W z 、W z 、 b z 、b r as well as Represents different learnable parameters, ReLU is the activation function, and the superscript T represents the transpose operation.

[0134] In one embodiment, updating the context node in the policy network according to the multi-head attention mechanism to obtain an updated node embedding includes:

[0135] In the policy network, the context nodes are updated according to the multi-head attention mechanism, and the updated node embedding is obtained as

[0136]

[0137] Among them, ∥ represents the serial operation, J is the number of heads of multi-head attention, MHA j It is the multi-head attention mechanism of the j-th head, and the calculation formula is as follows:

[0138]

[0139] Query vector in attention mechanism ≤ (c) , key vector k j Sum value vector v j is defined as follows:

[0140] k i =W j K H′,v j =W j V H′(j=1,…,M)

[0141] Among them, U (c) represents the context node embedding, W j K , W j V is the linear transformation matrix, and H′ is the output of the previous layer.

[0142] In one embodiment, calculating the next node probability using the updated initial node embedding includes:

[0143] Calculate the query vector q and the key vector k of node i based on the initial node embedding i , the formula is as follows:

[0144] q=W Q U (c) ′,k i =W K H i '

[0145] Among them, W Q is the weight matrix used to generate the query vector, W K is the weight matrix used to generate the key vector;

[0146] Use single-head attention to calculate logit to query the updated context node and all nodes:

[0147]

[0148] where C is a constant, d k is the dimension of the key vector, r j is the attention score of node j, tanh is the activation function, the superscript T represents the transpose operation, k j is the key vector;

[0149] At each step k, a node ω k is selected, and not only the current graph node embedding but also the outputs ω k′ of all previously selected nodes need to be considered. For all k′ < k, the selected nodes are masked with r j = -∞ or the smallest negative number that the computer can represent, and then softmax is used for

[0150]

[0151] where, p i is the probability that node i is selected, ω k is the current node, s is the current state, ω 1:k-1 is the sequence of previously selected nodes, r i is the attention score of node i, p θ represents the probability distribution under the given prediction model parameters θ.

[0152] In one embodiment, the adversarial reward function of the policy network is set as

[0153]

[0154] where, Ω is the set of selected nodes, r(Ω) represents the internal loss, is the mean squared error loss function, X′ t is the historical data after adversarial perturbation at time t, G is the dynamic graph data, r KD is the knowledge distillation reward, L KD is the knowledge distillation loss, [[ID=5३]]is the teacher network from the previous training, T k (d) represents the method of calculating the distance, represents the Markov distance between the current network and the teacher network, c is a constant, α1 and α2 are parameters that control the amount of knowledge transferred from the teacher network and the Markov distance reward weight, and the max function is used to find the maximum value in a set of values.

[0155] In one embodiment, the loss function of the policy network is set as

[0156]

[0157] Where c is a constant, s represents the problem instance, which consists of the flow data from time point t-T+1 to time point t, and p φ (Ω|s) represents the probability of selecting the node set Ω under the problem instance s, φ is the parameter of the policy network, log is the logarithmic probability, Represents the gradient.

[0158] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0159] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a traffic prediction method based on spatiotemporal correlation of dynamic graphs and adaptive adversarial training is implemented. The display screen of the computer device can be a liquid crystal display or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0160] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0161] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0162] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A traffic prediction method based on dynamic graph spatiotemporal correlation and adaptive adversarial training, characterized by: The method comprises: Modeling network devices as a dynamic graph; the dynamic graph includes a node set, an edge set, and an adjacency matrix; the node set includes multiple nodes; the nodes include servers, routers, and switches; obtaining dynamic graph data corresponding to network devices at historical moments; An adaptive graph convolutional network is used to capture the dynamic changes of nodes and edges in the dynamic graph data to obtain a dynamic graph spatial embedding representation; a temporal attention network is used to capture the temporal correlation between nodes in the dynamic graph data to obtain a dynamic graph temporal embedding representation; Performing nonlinear activation and linear combination on the dynamic graph spatial embedding representation and the dynamic graph temporal embedding representation input to obtain a dynamic graph embedding; defining the dynamic graph embedding as a starting embedding and the pre-set last adversarial node embedding as a context node embedding as inputs to the policy network; The context node is updated in the policy network according to a multi-head attention mechanism to obtain an updated initial node embedding; the probability of the next node is calculated using the updated initial node embedding, and the node with the highest probability is selected from all nodes as the next sampled adversarial node; the adversarial node is applied to the policy network, and the adversarial reward function and loss function of the policy network are set according to the change in prediction performance before and after the application of the adversarial node; The strategy network is trained using the adversarial reward function and loss function, and an updated adversarial node is output according to the trained strategy network; the dynamic graph is updated according to the updated adversarial node, and the spatial correlation and temporal correlation capture are performed on the updated dynamic graph to obtain the traffic prediction result corresponding to the current network device.

2. The method according to claim 1, characterized in that Adaptive graph convolutional networks are used to capture the spatial correlation of dynamic graphs by learning the dynamic changes of nodes and edges in dynamic graph data, including: The adaptive graph convolutional network is used to capture the dynamic changes of nodes and edges in dynamic graph data to obtain the spatial correlation of dynamic graphs. Where N is the total number of nodes in the dynamic graph, σ is the Sigmoid activation function, i is the current node, j is its neighbor node, k represents all other nodes that may affect node i, I N is the unit matrix, I represents the input dynamic graph data, O represents the output dynamic graph spatial correlation, is the adjacency matrix, D is the degree matrix, is the node embedding matrix, is the weight pool, is the bias pool, where d is the embedding dimension, C is the feature dimension, F is the output feature dimension, and ReLU is the activation function. represents the Hadamard product, M is used to adjust the weight of each element, B is the bias matrix, R A It is an adaptive graph matrix, which is automatically updated during training. exp is an exponential function used to calculate the exponential power of e.

3. The method according to claim 1, characterized in that The temporal attention network includes an encoding module, a graph attention module, a multi-layer capture module, a feature reconstruction module, an important feature matrix module and a feature learning module; The temporal correlation between nodes in the dynamic graph data is captured according to the temporal attention network to obtain a temporal embedding representation of the dynamic graph, including: Encode the input dynamic graph data according to the encoding module to obtain a node feature matrix and a timestamp node feature matrix; In the graph attention module, a fused attention matrix is ​​calculated based on the input node feature matrix and the timestamp node feature matrix; Updating the fused attention matrix in the multi-layer capture module to obtain an updated node matrix; Reconstructing the updated node matrix in the feature reconstruction module, supplementing the node features at the current moment by calculating the difference between the features output by the static attention graph and the features output by the dynamic attention graph, and obtaining the reconstructed features; The important feature matrix module processes the reconstructed features according to the self-attention mechanism to obtain an important feature matrix; The feature learning module performs feature learning on the important feature matrix to obtain a dynamic graph temporal embedding representation.

4. The method according to claim 3, characterized in that In the graph attention module, the fusion attention matrix is ​​calculated based on the input node feature matrix and the timestamp node feature matrix, including the graph static attention matrix A calculated based on the input node feature matrix in the graph attention module. static =softmax(HW static H T ) Where H is the node feature matrix, the superscript T represents the transpose operation, and W static Represents the weight matrix of static features, and softmax represents the normalized exponential function, which is used to convert the vector into a probability distribution; The graph dynamic attention matrix is ​​calculated based on the input timestamp node feature matrix: Among them, W dynamic represents the dynamic weight matrix; The graph static attention matrix and the graph dynamic attention matrix are fused by weighted addition to obtain the fused attention matrix: A combined =αA static +(1-a)A dynamic Among them, α is a weight parameter used to control the contribution of static and dynamic attention mechanisms to the fused attention matrix.

5. The method according to claim 3, characterized in that The fused attention matrix is ​​updated in the multi-layer capture module to obtain an updated node matrix, including: The fusion attention matrix is ​​updated in the multi-layer capture module, and the updated node matrix is ​​obtained as Among them, A combined Represents the fusion attention matrix, ReLU is the activation function, is the normalized degree matrix, H (l) is the node feature matrix of the lth layer, is the feature weight matrix of the lth layer.

6. The method according to claim 3, characterized in that The feature learning module performs feature learning on the important feature matrix to obtain a dynamic graph temporal embedding representation, including: In the feature learning module, the important feature matrix is ​​subjected to feature learning, and the dynamic graph time embedding representation is obtained as follows: Among them, F att Represents the important feature matrix, R A represents the adaptive graph matrix, E represents the long-term dynamic time relationship of traffic data in the dynamic graph, tanh is the activation function, [·] represents the connection operation, ⊙ represents the element-by-element product operation, z and r are the reset gate and update gate respectively, t represents the current time, σ represents the Sigmoid activation function, represents the candidate hidden state at the current moment, h t-1 Represents the dynamic graph time embedding representation of the previous moment of the current moment, softmax represents the normalized exponential function, which is used to convert the vector into a probability distribution. is an intermediate variable, representing the dependencies between the inferred nodes, W z 、W z 、 b z 、b r as well as Represents different learnable parameters, ReLU is the activation function, and the superscript T represents the transpose operation.

7. The method according to claim 1, characterized in that The context node is updated in the policy network according to the multi-head attention mechanism to obtain an updated node embedding, including: In the policy network, the context node is updated according to the multi-head attention mechanism, and the updated node embedding is obtained as Among them, || represents the concatenation operation, J is the number of heads of multi-head attention, MHA j It is the multi-head attention mechanism of the j-th head, and the calculation formula is as follows: Query vector q in attention mechanism (c) , key vector k j Sum value vector v j is defined as follows: Among them, U (c) represents the context node embedding, is the linear transformation matrix, and H′ is the output of the previous layer.

8. The method according to claim 1, characterized in that Calculating the next node probability using the updated initial node embedding includes: Calculate the query vector q and the key vector k of node i based on the initial node embedding i , the formula is as follows: q=W Q U (c) ′,k i =W K H i ′ Among them, W Q is the weight matrix used to generate the query vector, W K is the weight matrix used to generate the key vector; Use single-head attention to calculate logit to query the updated context node and all nodes: Where C is a constant, d k is the dimension of the key vector, r j is the attention score of node j, tanh is the activation function, superscript T represents the transposition operation, k j is the key vector; At each step k, a node ω is selected k , and not only the current graph node embedding but also the outputs ω of all previously selected nodes need to be considered k′ , for all k′ < k, the selected nodes are masked with r j = -∞ or the smallest negative number that the computer can represent, and then the softmax function is used to calculate the probability of the next node as Among them, p i is the probability of node i being selected, ω k is the current node, s is the current state, ω 1:k-1 is the node sequence selected previously, r i is the attention score of node i, p θ represents the probability distribution under given prediction model parameters θ.

9. The method according to claim 1, characterized in that Set the adversarial reward function of the policy network to Where Ω is the selected node set, r(Ω) represents the internal loss, is the mean square error loss function, X′ t is the historical data after the adversarial disturbance at time t, G is the dynamic graph data, r KD is the knowledge distillation reward, L KD is the knowledge distillation loss, is the teacher network trained last time, T K (d) represents the distance calculation method, represents the Markov distance between the current network and the teacher network, c is a constant, α1 and α2 are parameters that control the amount of knowledge transferred from the teacher network and the Markov distance reward weight, and the max function is used to find the maximum value in a set of values.

10. The method according to claim 9, characterized in that Set the loss function of the policy network to Where c is a constant, s represents the problem instance, which consists of the flow data from time point t-T+1 to time point t, and p φ (Ω|s) represents the probability of selecting the node set Ω under the problem instance s, φ is the parameter of the policy network, log is the logarithmic probability, Represents the gradient.

Images