Spatio-Temporal Traffic Speed Prediction Method and System Integrating Attention and Multi-Graph Convolution

Abstract

The present invention discloses a spatio-temporal traffic speed prediction method and system that fuses attention and multi-graph convolution. It relates to the field of traffic spatio-temporal prediction, and its technical key points are as follows: obtaining auxiliary data and main data; respectively substituting them into the dilated spatio-temporal graph convolution to obtain the graph adjacency matrix; fusing the graph adjacency matrices to obtain the graph adjacency matrix of the fused features; extracting the channel features of the main data, adding the channel features and the graph adjacency matrix of the fused features point by point to obtain the output of the main data; extracting the channel features of the auxiliary data, adding the channel features of the auxiliary data and the graph adjacency matrix point by point to obtain the output of the auxiliary data; repeating the above steps, stopping when the loop reaches a predetermined number of times, and performing skip connection on the graph adjacency matrix of the obtained fused features to obtain the traffic speed prediction data. The present invention constructs a deep learning spatio-temporal model for traffic speed prediction, and adaptively extracts dynamic spatial features and temporal features.

Description

Technical Field

[0001] The present invention relates to the field of traffic spatio-temporal prediction, and particularly to a spatio-temporal traffic speed prediction method and system that fuses attention and multi-graph convolution. Background Art

[0002] With the acceleration of the urbanization process and the continuous increase in the number of vehicles, the traffic system has been continuously expanding and the urban road conditions have become increasingly complex. Traffic problems such as traffic jams, traffic accidents, and air pollution have emerged in an endless stream. In order to alleviate traffic congestion and improve urban traffic efficiency, traffic management and planning departments need to accurately predict traffic speed in real time.

[0003] Traffic speed prediction is a spatio-temporal data mining task that predicts future demands based on the spatial and temporal characteristics of historical data. Traditional machine learning methods not only require data to meet certain assumptions, but also lack the modeling of urban road network structures and traffic flow relationships. Compared with traditional machine learning methods, deep learning methods can achieve better performance without relying on human intervention. Graph convolutional networks can analyze graph-structured data with non-Euclidean distances and can extract the relationships between graph nodes more accurately compared to traditional convolutions. However, when traditional graph convolutional network models extract spatial correlation features, they cannot assign the same importance coefficients or weights to different neighbor nodes. Spatio-temporal graph neural networks are currently the mainstream method for dealing with spatio-temporal dependence relationships, and their main idea is to combine deep learning-based time series models with graph neural networks. Some scholars have used a combination of graph neural networks and recurrent neural networks to model the complex spatio-temporal dependence relationships of road networks, but this method ignores the spatial information of the data and has poor long-term prediction effects. Some scholars have designed a dynamic graph construction method to learn the time-specific spatial dependencies of road segments and proposed a dynamic graph convolution module that aggregates the hidden states of neighbor nodes to the focal node by passing messages on the dynamic adjacency matrix, but the feature extraction of the original data is not precise enough. By introducing the attention mechanism, spatio-temporal graph convolution traffic prediction models can better handle the influence of multiple factors on traffic speed prediction and improve the model's ability to capture the relationships between remote monitoring points. Therefore, the application of the attention mechanism in spatio-temporal graph convolution traffic prediction has great potential and can help improve the accuracy and practicality of traffic prediction. Summary of the Invention

[0004] From the perspective of spatio-temporal prediction, the present invention invents a spatio-temporal traffic speed prediction method and system that fuses attention and multi-graph convolution, solves the problem of only considering the influence of single factors in spatio-temporal feature extraction for traffic speed prediction and the problem of non-adaptive fusion of feature representations at different time steps; and can further obtain more accurate traffic speed prediction results.

[0005] According to the first aspect of the present invention, a spatio-temporal traffic speed prediction method integrating attention and multi-graph convolution is provided, comprising the following steps:

[0006] Step 1: Obtain auxiliary data and main data; wherein, the main data includes traffic speed, and the auxiliary data includes traffic flow;

[0007] Step 2: Substitute the auxiliary data and the main data into the dilated spatio-temporal graph convolution respectively to obtain the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data;

[0008] Step 3: Fuse the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data to obtain the graph adjacency matrix of the fused features;

[0009] Step 4: Extract the channel features of the main data, add the channel features of the main data and the graph adjacency matrix of the fused features point by point to obtain the output of the main data, and use it as the input of the main data for the next step;

[0010] Step 5: Extract the channel features of the auxiliary data, add the channel features of the auxiliary data and the graph adjacency matrix of the auxiliary data point by point to obtain the output of the auxiliary data, and use it as the input of the auxiliary data for the next step;

[0011] Step 6: Repeat Steps 2-5, and when the loop reaches a predetermined number of times, stop. Perform skip connection on the finally obtained graph adjacency matrix of the fused features to obtain the traffic speed prediction data.

[0012] Based on the above technical solutions, the present invention can also be improved as follows.

[0013] Optionally, the substituting the auxiliary data and the main data into the dilated spatio-temporal graph convolution respectively to obtain the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data includes:

[0014] Substitute the auxiliary data and the main data into the dilated spatio-temporal graph convolution respectively. The dilated spatio-temporal graph convolution extracts the spatio-temporal features of the auxiliary data and the main data respectively, and outputs them as the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data.

[0015] Optionally, the dilated spatio-temporal graph convolution is a spatio-temporal network composed of a temporal convolution layer and a graph convolution network. Dilated convolution and gated convolution are used in the temporal convolution layer; dynamic graph convolution is used in the graph convolution network to update the hidden state of the main node by aggregating the hidden states of neighbor nodes through weighted links.

[0016] Optionally, the expression formulas for obtaining the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data are as follows:

[0017]

[0018] Among them, E t ∈R N×C represents the node feature matrix of time t, N is the number of nodes, K is the number of convolution kernels, and W k ∈R C×C is a learnable weight matrix. is the attention coefficient of node n in the k-th convolution kernel, which is obtained by weighted summation of the features of the node and its neighbor nodes.

[0019] Optionally, the fusion of the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data to obtain the graph adjacency matrix of the fusion features includes:

[0020] Performing pixel-by-pixel addition on the feature vectors with spatio-temporal embedding information obtained by spatio-temporal feature extraction of the auxiliary data and the main data to obtain the fused graph adjacency matrix.

[0021] Optionally, the pixel-by-pixel addition of the feature vectors with spatio-temporal embedding information obtained by spatio-temporal feature extraction of the auxiliary data and the main data is expressed by the formula:

[0022]

[0023] Among them, represents the feature vector with spatio-temporal information embedding, N is the number of nodes, ∑ represents the accumulation operation, W is a learnable weight matrix, is the attention coefficient of node n in the k-th convolution kernel, represents pixel-by-pixel addition.

[0024] Optionally, the extraction of the channel features of the main data, and the pixel-by-pixel addition of the channel features of the main data and the graph adjacency matrix integrating the features of the auxiliary data with spatio-temporal information embedding are substituted into the following formula:

[0025]

[0026] Among them, X n+1 represents the next input, represents the fused feature matrix, represents the channel features of the data.

[0027] Optionally, the jump connection of the graph adjacency matrix of the finally obtained fused features includes: connecting and transmitting the hidden states of the spatio-temporal embedding information from different depth layers into the activation function to obtain the final traffic speed prediction data.

[0028] Optionally, the activation function is expressed by the following formula:

[0029]

[0030] where n ∈ (1, 2, …, N), represents that each layer has a spatio-temporal information embedding vector, ReLU represents a non-linear activation, and Cat represents a concatenation operation.

[0031] According to the second aspect of the present invention, there is provided a spatio-temporal traffic speed prediction system integrating an attention mechanism and multi-graph convolution, including:

[0032] A channel attention module, which is used to extract the channel features of auxiliary data, perform element-wise addition on the extracted channel features of the data and the graph adjacency matrix, output the data, and use it as the input of the next-step data; and each time after channel attention calculation, the channels of the input data remain unchanged;

[0033] An extended spatio-temporal graph convolution module, which consists of a time convolution layer and a graph convolution network, is used to extract the spatio-temporal features of data, and substituting the data into the extended spatio-temporal graph convolution can obtain the graph adjacency matrix of the data;

[0034] A multi-graph fusion module, which is used to fuse multiple external feature data captured by the model with traffic speed data to obtain the graph adjacency matrix of the fused features.

[0035] Technical effects and advantages of the present invention:

[0036] The spatio-temporal prediction model constructed based on the attention mechanism and multi-graph fusion of the present invention effectively solves the problems of only considering the influence of single factors in spatio-temporal feature extraction for traffic speed prediction and non-adaptively fusing feature representations of different time steps, and can obtain more accurate traffic speed prediction results. Description of the Drawings

[0037] Figure 1 It is the network structure diagram of the spatio-temporal traffic speed prediction method integrating attention and multi-graph convolution provided by the embodiment of the present invention;

[0038] Figure 2 It is the structure diagram of the extended spatio-temporal graph convolution provided by the embodiment of the present invention;

[0039] Figure 3 It is the structure diagram of the channel attention provided by the embodiment of the present invention; Detailed Embodiments

[0040] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0041] It is understandable that, based on the deficiencies in the background art, an embodiment of the present invention proposes a spatio-temporal traffic speed prediction method that fuses attention and multi-graph convolution, specifically as follows: Figure 1 The method includes the following steps:

[0042] Step 1: Obtain auxiliary data and main data; among them, the main data includes traffic speed data, and the auxiliary data includes traffic flow data;

[0043] In this embodiment, the auxiliary data is located in the auxiliary unit and serves as other external feature factors affecting traffic speed to assist in traffic speed prediction. Usually, traffic flow data is used.

[0044] The main data is the main part of the prediction, uses traffic speed data, and is located in the main unit.

[0045] Step 2: Substitute the auxiliary data and the main data into the dilated spatio-temporal graph convolution respectively to obtain the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data;

[0046] Specifically, substitute the auxiliary data and the main data into the dilated spatio-temporal graph convolution respectively, extract the spatio-temporal features of the auxiliary data and the main data respectively, and finally obtain the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data. Among them, the spatio-temporal features of the auxiliary data are traffic flow data, and the spatio-temporal features of the main data are traffic speed data.

[0047] It should be noted that, as shown in Figure 2 The dilated spatio-temporal graph convolution is a spatio-temporal network jointly composed of a temporal convolution layer and a graph convolution network, which is used to extract the spatio-temporal features of data. To avoid gradient disappearance and retain non-linearity, dilated convolution and gated convolution are used in the temporal convolution layer, which combines the characteristics of dilated convolution and gated convolution. By increasing the dilations inside the convolutional kernel to expand the receptive field, and at the same time controlling the flow of information through the gated mechanism to reduce redundant features and the risk of overfitting in the network, effectively capturing the long-range dependencies of the input features so that the network has stronger expressive power.

[0048] To capture the diversity of data to a greater extent, the Sigmoid activation function is used after the dilated convolution to map the output data values between 0 and 1. Similarly, the tanh activation function is used after the gated convolution to map the output data values between -1 and 1. Then the output results of the two are multiplied pixel by pixel. The specific calculation formula is as follows:

[0049]

[0050] In the formula, W g and W fLet \(W\) represent two convolutional kernels, \(b\) represent the bias, both of which are learnable parameters, and \(\sigma\) represent the sigmoid function. Denotes element-wise multiplication.

[0051] Furthermore, in order to more accurately capture the changes of road nodes, the embodiment of the present invention adopts dynamic graph convolution, that is, convolving different nodes at different times. Different from traditional graph convolution, dynamic graph convolution can handle the changes of nodes and edges. This operation can be achieved through the multiplication of the adjacency matrix, the hidden state of the nodes, and learnable states and other parameters. In each step of convolution, the hidden state of the neighboring nodes is aggregated through weighted links to update the hidden state of the main node. The specific calculation process is as follows:

[0052]

[0053] Among them, \(E\) t \(\in \mathbb{R}\) N×C represents the node feature matrix at time \(t\), \(N\) is the number of nodes, \(K\) is the number of convolutional kernels, \(W\) k \(\in \mathbb{R}\) C×C is a learnable weight matrix, is the attention coefficient of node \(n\) in the \(k\)-th convolutional kernel, which is obtained by weighted summing the features of the node and its neighboring nodes.

[0054] Step 3: Fuse the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data to obtain the graph adjacency matrix of the fused features;

[0055] In this embodiment, fusing the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data to obtain the graph adjacency matrix of the fused features includes substituting them into the multi-graph fusion module. The multi-graph fusion module is used to fuse the spatio-temporal embedding information of the auxiliary data and the spatio-temporal embedding information of the main data to improve the performance of the model, thereby improving the prediction accuracy of traffic speed. In the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data, not only are there spatio-temporal dependencies between the nodes of the main data, but there are also spatio-temporal dependencies between the nodes of the auxiliary data. Therefore, we use separate spatio-temporal block stacking models for these two types of nodes respectively. After stacking \(l\) layers, we can obtain the spatio-temporal embedding vectors of the main data and the spatio-temporal embedding vectors of the auxiliary data, and perform a fusion operation on the two.

[0056] As shown in formula (3),

[0057]

[0058] When there are \(N\) external node features, formula (3) can be extended to formula (4).

[0059]

[0060] Among them, represents the feature vector with spatio-temporal information embedding, N is the number of nodes, ∑ represents the accumulation operation, W is the learnable weight matrix, is the attention coefficient of node n in the k-th convolutional kernel, represents element-wise addition.

[0061] Step 4: Extract the channel features of the main data, perform element-wise addition on the channel features of the main data and the graph adjacency matrix of the fusion features to obtain the output of the main data, and use it as the input for the next step;

[0062] In this embodiment, the extraction of the channel features of the main data includes: substituting the main data into the channel attention module. The channel attention module is shown as Figure 3 to extract the channel features of the data to improve the model's ability to capture the relationships between remote monitoring points. The channel attention module first performs adaptive average pooling on the spatial dimension of the data, then learns the channel attention through two fully connected layers, normalizes it with the Sigmoid activation function to obtain the channel attention map, and finally multiplies the channel attention map with the original features to obtain the weighted features. Ultimately, the feature enhancement of the input data at the channel layer is realized, effectively improving the prediction accuracy of the model, and the channels of the input data can remain unchanged each time after being calculated by the channel attention module.

[0063] The present invention combines the Tao attention module to improve the model's ability to capture the relationships between remote monitoring points. And the channels of the input data can remain unchanged each time after being calculated by the channel attention module.

[0064] To obtain the attention in the channel dimension, after the data is input, global average pooling is first performed based on the width and height of the data to reduce the spatial features to 1×1, as shown in formula (5); then two fully connected layers and a non-linear activation function are used to establish the connection between channels, as shown in formula (6); after passing through the Sigmoid activation function to obtain the normalized weight, and finally, element-wise weighting is performed on each channel of the original input through multiplication to complete the recalibration of the original features by the channel attention, as shown in formula (7).

[0065]

[0066]

[0067]

[0068] In the formula, represents two fully connected layers, Scale() is the compression operation, and σ represents the Sigmoid activation function.

[0069] Extract the channel features of the main data, and perform element-wise addition on the channel features of the main data and the graph adjacency matrix that combines the auxiliary data features with spatio-temporal information embedding to obtain the next input of the main unit, including the following formula:

[0070]

[0071] X n+1 represents the next input, represents the fused feature matrix, represents the channel features of the data.

[0072] Step 5: Extract the channel features of the auxiliary data, perform element-wise addition on the channel features of the auxiliary data and the graph adjacency matrix of the auxiliary data to obtain the output of the current step, and use it as the input for the next step;

[0073] In this embodiment, extracting the channel features of the auxiliary data includes: substituting the auxiliary data into the channel attention module. The process of extracting the channel features of the auxiliary data can refer to the extraction process of the main data. Details are not elaborated here.

[0074] Furthermore, for the extraction of the channel features of the auxiliary data, perform element-wise addition on the channel features of the auxiliary data and the feature vector with spatio-temporal embedding information obtained through spatio-temporal feature extraction to obtain the next input of the auxiliary unit, including the following formula:

[0075]

[0076] represents the next input, represents the feature vector with spatio-temporal embedding information, represents the channel features of the data.

[0077] Step 6: Repeat Steps 2 - 5, and terminate when the loop reaches a predetermined number of times. Perform skip connection on the feature matrix with spatio-temporal embedding information obtained in the main unit to obtain the final traffic speed prediction data.

[0078] In this embodiment, the predetermined number of times is usually set to 8 times. In places with heavy traffic, the predetermined number of times can be set as needed. Spatio-temporal traffic speed prediction means accurately predicting the current traffic speed in real time.

[0079] It should be noted that perform skip connection on the feature matrix with spatio-temporal embedding information obtained in the main unit. That is, connect and transfer the hidden states of the spatio-temporal embedding information from different depth layers to an activation function to obtain the final traffic speed prediction data. Specifically, substitute into the formula:

[0080]

[0081] Wherein, n ∈ (1, 2, …, N), ReLU represents a non-linear activation, and Cat represents a concatenation operation.

[0082] In summary, the present invention uses multi-source graph convolution to fuse the features of external factors and the main features of traffic speed, uses an attention mechanism to adaptively extract the feature representations of data channels at different time steps, and uses skip connections as the prediction layer to map spatio-temporal features to actual demand values, achieving the stability and accuracy of long-term prediction. Finally, the framework is evaluated on three large-scale real-world road network traffic datasets, and it performs better than advanced baseline models.

[0083] The embodiment of the present invention also provides a spatio-temporal traffic speed prediction system that fuses an attention mechanism and multi-source graph convolution. The system includes:

[0084] A channel attention module, which is used to extract the channel features of auxiliary data, add the extracted channel features of the data and the graph adjacency matrix point by point, output the data, and use it as the input of the next step of data; and each time after channel attention calculation, the channel of the input data remains unchanged;

[0085] An extended spatio-temporal graph convolution module, which consists of a temporal convolution layer and a graph convolution network, is used to extract the spatio-temporal features of data. Substituting the data into the extended spatio-temporal graph convolution can obtain the graph adjacency matrix of the data;

[0086] A multi-source graph fusion module, which is used to fuse multiple external feature data captured by the model with traffic speed data to obtain the graph adjacency matrix of the fused features.

[0087] It should be understood that the parts not elaborated in detail in this specification all belong to the prior art.

[0088] Finally, it should be noted that the above are only the preferred embodiments of the present invention and are not used to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A spatio-temporal traffic speed prediction method that fuses attention and multi-graph convolution, characterized in that, Including the following steps: Step 1: Obtain auxiliary data and main data; among them, the main data includes traffic speed, and the auxiliary data includes traffic flow; Step 2: Substitute the auxiliary data and the main data into the dilated spatio-temporal graph convolution respectively to obtain the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data; Step 3: Fuse the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data to obtain the graph adjacency matrix of the fused features; including: adding the feature vectors with spatio-temporal embedding information obtained by spatio-temporal feature extraction of the auxiliary data and the main data pixel by pixel to obtain the fused graph adjacency matrix; The formula is expressed as: Among them, and represent feature vectors with spatio-temporal information embedding, is the number of nodes, ∑ represents the summation operation, is a learnable weight matrix, is the node at the th attention coefficient in the convolutional kernel, represents element-wise addition; Step 4: Extract the channel features of the main data, add the channel features of the main data and the graph adjacency matrix of the fused features point by point to obtain the output of the main data, and use it as the input of the main data in the next step; among them, add the channel features of the main data and the graph adjacency matrix that fuses the features of the auxiliary data with spatio-temporal information embedding point by point and substitute it into the following formula: Among them, represents the input of the next step, represents the fused feature matrix, represents the channel features of the data; Step 5: Extract the channel features of the auxiliary data, add the channel features of the auxiliary data and the graph adjacency matrix of the auxiliary data point by point to obtain the output of the auxiliary data, and use it as the input of the auxiliary data in the next step; Step 6: Repeat Steps 2-5, stop when the loop reaches a predetermined number of times, and perform skip connection on the finally obtained graph adjacency matrix of the fused features to obtain the traffic speed prediction data.

2. The spatio-temporal traffic speed prediction method based on the fusion of attention and multi-graph convolution according to claim 1, wherein, The substituting the auxiliary data and the main data into the dilated spatio-temporal graph convolution respectively to obtain the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data includes: Substitute the auxiliary data and the main data into the dilated spatio-temporal graph convolution respectively. The dilated spatio-temporal graph convolution extracts the spatio-temporal features of the auxiliary data and the main data respectively to obtain the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data.

3. The spatio-temporal traffic speed prediction method based on the fusion of attention and multi-graph convolution according to claim 2, characterized in that, The dilated spatio-temporal graph convolution is a spatio-temporal network jointly composed of a temporal convolution layer and a graph convolution network. Dilated convolution and gated convolution are used in the temporal convolution layer; dynamic graph convolution is used in the graph convolution network to update the hidden state of the main node by aggregating the hidden states of neighbor nodes through weighted links.

4. The spatio-temporal traffic speed prediction method by fusing attention and multi-graph convolution according to claim 2, characterized in that The expression formulas for obtaining the graph adjacency matrix of the auxiliary data and the graph adjacency matrix of the main data are as follows: Among them, represents the node feature matrix at time t, is the number of nodes, is the number of convolutional kernels, is the learnable weight matrix, is the node at the th convolutional kernel, and is obtained by weighted summation of the features of the node and its neighbor nodes.

5. The spatio-temporal traffic speed prediction method based on the fusion of attention and multi-graph convolution according to claim 1, characterized in that The performing skip connection on the finally obtained graph adjacency matrix of the fused features includes: connecting and transmitting the hidden states of the spatio-temporal embedding information from different depth layers into the activation function to obtain the final traffic speed prediction data.

6. The spatio-temporal traffic speed prediction method by fusing attention and multi-graph convolution according to claim 5, characterized in that The activation function is expressed by the following formula: Among them, , represents that each layer has a spatio-temporal information embedding vector, represents a non-linear activation, represents a concatenation operation.

7. A spatio-temporal traffic speed prediction system that fuses an attention mechanism with multi-graph convolution, characterized in that, Including: A channel attention module, which is used to extract the channel features of the auxiliary data, add the extracted channel features of the data and the graph adjacency matrix point by point, perform data output, and use it as the input of the data in the next step; And each time after channel attention calculation, the channels of the input data remain unchanged; A dilated spatio-temporal graph convolution module, which is composed of a temporal convolution layer and a graph convolution network, is used to extract the spatio-temporal features of the data. Substituting the data into the dilated spatio-temporal graph convolution can obtain the graph adjacency matrix of the data; add the channel features of the main data and the graph adjacency matrix that fuses the features of the auxiliary data with spatio-temporal information embedding point by point and substitute it into the following formula: Among them, represents the input for the next step, represents the fused feature matrix, represents the channel features of the data; A multi-source graph fusion module, which is used to fuse multiple external feature data captured by the model with traffic speed data to obtain a graph adjacency matrix of fused features; it includes: adding the feature vectors with spatio-temporal embedding information obtained by spatio-temporal feature extraction of auxiliary data and main data pixel by pixel to obtain a fused graph adjacency matrix; the formula is expressed as: Among them, , represent feature vectors with spatio-temporal information embedding, is the number of nodes, and ∑ represents the summation operation, is a learnable weight matrix, is the node at the th attention coefficient in the convolution kernel, represents element-wise addition.

Images