A traffic prediction method based on graph transfer learning

Abstract

The present invention belongs to the field of computer data analysis, and provides a traffic prediction method based on graph transfer learning, which can efficiently transfer knowledge from a source domain with sufficient training samples to a target domain with insufficient training samples, and improve the prediction performance of a traffic prediction model based on a graph convolutional neural network in a small sample situation. The method of the present invention can help improve the prediction accuracy of a traffic prediction model based on graph data in a small sample situation. The method of the present invention introduces spatial clustering and temporal clustering regularization terms, balances spatiotemporal pattern matching with prediction accuracy, and completes the matching of the spatiotemporal pattern similarity of nodes in the target domain and the source domain in a data-driven manner, thereby achieving the effect of reducing negative transfer, and greatly reducing the computational overhead of previous methods in the spatiotemporal pattern matching process.

Description

Technical Field

[0001] The present invention belongs to the field of computer data analysis, and in particular to a method for migrating specific temporal and spatial knowledge across domains. Background Art

[0002] Traffic prediction is very important for urban traffic management. It can help traffic management departments predict the motor vehicle flow, driving speed and occupancy rate of urban roads, and conduct real-time guidance to reduce the occurrence of urban road congestion. In recent years, deep learning methods have been widely used in the field of traffic prediction, and have greatly improved the accuracy of prediction compared with traditional time series prediction models. At present, traffic prediction models based on deep learning mainly model problems from two aspects, namely time dependency and space dependency. For capturing time dependency, recurrent neural networks (RNN) or temporal convolutional networks (TCN) are mainly used, while for capturing space dependency, convolutional neural networks (CNN) or graph convolutional neural networks (GCN) are mainly used. Among them, convolutional neural networks are suitable for capturing space dependency on regular grid data, while graph convolutional neural networks are suitable for capturing space dependency on graph data. The essential attribute of road network is graph, and it is more reasonable to model road network using graph rather than grid. In recent years, with the development of graph neural network technology, more and more traffic prediction models are based on graph convolutional neural networks instead of convolutional neural networks to capture space dependency, and have achieved the best prediction effect. However, the following problems exist in previous work: First, when training the prediction model based on deep learning, a large amount of historical traffic data is required, but in the implementation and application, we often face the problem of data shortage, which makes the training of these deep-based models difficult. Second, the current method to improve the accuracy of traffic prediction in small sample situations is mainly based on transfer learning, and the existing transfer learning method for traffic prediction is mainly applicable to regular grid-type traffic data, and is not applicable to graph data of traffic networks. For graph data, due to the different structures and scales of road networks in different regions, it is difficult to find similar areas in the source domain (data-rich) to match the target domain (data-scarce), thereby improving the migration performance. Therefore, a method that can improve the prediction accuracy of traffic prediction models based on graph convolutional neural networks in small sample situations is a technical difficulty to be overcome at present, and is also the motivation for the invention of this invention. Next, the relevant background technology in this field is introduced in detail.

[0003] (1) Traffic Forecast

[0004] In order to improve the accuracy of traffic prediction, people have done a lot of work. Some early studies applied classical statistical and machine learning methods, such as historical average (HA), differential integrated moving average autoregressive model (ARIMA) and support vector machine model (SVM) to predict future traffic conditions. In recent years, models based on deep learning have shown excellent capabilities in traffic prediction. Traffic prediction models based on deep learning usually use convolutional neural networks (CNN) or graph convolutional neural networks (GCN) to capture spatial dependencies, and use recurrent neural networks (RNN) or temporal convolutional networks (TCN) to model nonlinear temporal dependencies. In the latest research, researchers combined several basic modules and combined different types of technologies to jointly model complex dynamic spatiotemporal associations, achieving state-of-the-art traffic prediction performance. Representative works include: ST-ResNet model, DCRNN model, GraphWave Net model and AGCRN model. However, the disadvantage of these works is that a large amount of historical traffic data is required to train the model. When the data used for training is insufficient, the prediction performance of the model is poor.

[0005] (2) Transfer Learning

[0006] The purpose of transfer learning is to improve the performance of the prediction model in the target domain by transferring knowledge learned from different but related source domains. It is a common method to solve the problem of small sample learning. Wei et al. proposed to transfer the semantically related dictionary learned from the data-rich source city to the target city, enrich the feature representation of the target city, and then classify and predict the air quality of the target city. Lin et al. designed a feature-based transfer learning model for traffic speed prediction through various manually selected spatial and temporal features and improved the interpretability of the model. However, both methods are not designed for deep learning and require more prior knowledge as guidance. Wang et al. transferred knowledge from the source city to the target city by considering the potential representation of similar area pairs between cities. Zhang et al. designed a model-based deep transfer learning framework to improve the prediction accuracy by using the spatiotemporal similarity of different types of regional traffic flow data. Yao et al. used the idea of ​​multiple source domains to transfer knowledge from multiple related but different domains, thereby obtaining more comprehensive knowledge covering the target domain and reducing the risk of negative transfer. However, these methods mainly build spatial association models by decomposing the road network into grids. In fact, the road network is essentially a graph structure. In order to capture spatial information more effectively, it is more appropriate to represent the road network as a graph. At present, the transfer learning method of traffic prediction model based on graph data is still in urgent need of research. Summary of the invention

[0007] In order to solve the technical problem that the existing traffic prediction method based on graph convolutional neural network has low prediction accuracy under small sample conditions, the present invention designs a traffic knowledge transfer method based on node spatiotemporal pattern clustering, which can efficiently transfer knowledge from a source domain with sufficient training samples to a target domain with insufficient training samples, and improve the prediction performance of the traffic prediction model based on graph convolutional neural network under small sample conditions.

[0008] A traffic prediction method based on graph transfer learning, the steps are as follows:

[0009] (1) Learning traffic spatiotemporal pattern knowledge in data-rich source domains:

[0010] ①Model input:

[0011] Use multiple sensors at different locations in the road network to record the traffic conditions at each observation point over a period of time. The recorded value at time t is C is the number of traffic characteristics (such as traffic flow, speed, and road occupancy). After data cleaning (removing outliers and filling in missing values), the traffic data is preprocessed based on the sliding window method. Suppose it is necessary to predict the traffic status within H time steps after time t. Then the corresponding model input is the traffic flow record value in the past S time steps Where N is the number of sensors.

[0012] ②Model architecture:

[0013] The spatial and temporal dependencies are captured based on the graph convolutional network (GCN) and the temporal convolutional network (TCN), and the traffic spatiotemporal patterns are obtained based on the clustering algorithm, such as Figure 1 shown.

[0014] When traditional graph convolutional networks extract features from all nodes, the convolution kernel parameters used are the same. They extract the common spatial dependency patterns of all nodes, which can be described by formula (1):

[0015]

[0016] in are the output and input of the convolutional layer, F and C are the output and input traffic feature dimensions, is the identity matrix, is the adjacency matrix of the graph, is the degree matrix of the graph, is the convolution kernel parameter. In order to learn the spatial dependency pattern unique to each node, we need to use a separate convolution kernel for each node, so the final convolution kernel parameter required is On the one hand, when the number of nodes is large, the number of parameters of W′ will become huge, which makes optimization difficult and easily causes the problem of model overfitting; on the other hand, the convolution kernel parameters of each node are independent at this time, which does not take into account the implicit correlation between nodes (for example, nodes with similar spatial dependencies have similar convolution kernel parameters). Inspired by related work, the convolution kernel parameters W of the graph convolution network can be decomposed into the product form of two matrices with fewer parameters based on matrix decomposition. This process can be described by formula (2):

[0017]

[0018] in is the node space embedding matrix, its i-th row E i Represents the embedding representation of the i-th node in the subspace. is the spatial parameter pooling matrix, whose i-th row P i is the i-th component of the convolution kernel parameter. D is the dimension of the embedded subspace, and D<<N. From the perspective of node i, it has a spatial embedding vector Using this spatial embedding vector, we can get the spatial parameter pooling matrix A linear combination of the row vectors of the spatial parameter pooling matrix is ​​obtained: After shape adjustment, the convolution kernel parameters unique to node i are obtained From the above analysis, we can see that: on the one hand, the number of parameters that need to be learned is reduced from N×C×F to D×(N+C×F), which significantly reduces the difficulty of model optimization under the condition that D<<N; on the other hand, the convolution parameters between nodes establish a correlation through the parameter pool matrix, thereby establishing a correlation between nodes. By stacking multiple layers of graph convolutional neural networks, we can take into account neighbor information at a longer distance. For input We use the above-mentioned improved graph convolutional neural network to capture spatial dependencies at each time step.

[0019] In order to capture the node-specific temporal patterns, we design a gating mechanism based on the Temporal Convolutional Network (TCN) to extract the temporal patterns unique to each node. The one-dimensional dilated causal convolution operation can be described by formula (3):

[0020]

[0021] in is a one-dimensional input sequence, is a convolution kernel, K l is the length of the convolution kernel, d f is the expansion coefficient. For high-dimensional input The dilated causal convolution operation with a gating mechanism can be described by formula (4):

[0022]

[0023] where Θ T , b T is the convolution parameter, is the gating matrix, φ(·), σ(·) are the tanh and sigmoid activation functions respectively, is the output, and ⊙ is the Hadamard product. Similarly, on the one hand, the number of parameters of the gating matrix M is positively correlated with the number of nodes. When the number of nodes is large, the optimization of the gating matrix M becomes difficult; on the other hand, directly using the M matrix ignores the implicit correlation between nodes. Based on matrix decomposition, we decompose the M matrix into the product of two matrices with fewer parameters. In this way, formula (4) can be rewritten as formula (5):

[0024]

[0025] in is the node time embedding matrix, is the temporal parameter pooling matrix, D′ is the dimension of the embedding subspace, and D′<<N. By stacking multiple temporal convolutional layers, we can capture longer-term temporal dependencies.

[0026] Finally, based on the features extracted by the previous graph convolution layer and temporal convolution layer, a convolution layer is used to generate the final traffic prediction value. This process can be described by formula (6):

[0027]

[0028] Where W p , b p are the parameters of the convolutional layer.

[0029] Based on the matrix decomposition method, we can learn the unique time and space patterns of each node. However, the time and space patterns of all nodes are not completely different. There are some similarities between them. Based on the above method, we use clustering algorithms to cluster time and space patterns.

[0030] ③Model training method:

[0031] a) Assume that the model has been trained for M rounds, there are K types of spatial patterns, Q types of temporal patterns, and smoothing coefficients γ and θ. We randomly select E i K points in (i=1,2,...,N) are used as the initial centers of the spatial pattern k∈{1,2,...,K}, randomly select R iQ points in (i=1,2,...,N) are used as the initial centers of the time pattern q∈{1,2,...,Q};

[0032] b) For e = 1, 2, ..., Mdo:

[0033] 1. For input in the source domain Use the above model to get the corresponding output prediction value And calculate the prediction loss function:

[0034]

[0035] 2. Assign spatial and temporal categories to each node’s embedding vector:

[0036]

[0037]

[0038] 3. Calculate spatial clustering and temporal clustering losses:

[0039]

[0040]

[0041] 4. Calculate the center vector of each category:

[0042]

[0043]

[0044] Among them I k ,I q They represent the sets of nodes belonging to the kth spatial category and the qth temporal category respectively.

[0045] 5. Smoothly update the category center vector:

[0046]

[0047]

[0048] 6. According to the loss function L = L p +α·L s +β·L t , calculate the gradient of model parameters, and update model parameters Θ = {E, P, R, U, Θ T ,b T ,W p ,b p}, where α, β are hyperparameters;

[0049] (1) Transfer knowledge to the target domain where data is scarce and perform model fine-tuning:

[0050] There are differences in the transportation networks of different cities or regions, which are mainly reflected in the scale and connection mode of the transportation networks. However, although the transportation networks of different cities or regions are different, from the perspective of a certain node in the network, their spatial patterns and temporal patterns are similar. For example: the node flow around the railway station in City A is always proportional to the node flow around the bus station, and the node flow around the railway station and the passenger station in City B also has a similar spatial pattern; the nodes around a school in City A always have a peak in traffic at 8 am and 5 pm every day, and the nodes around a school in City B also have a similar temporal pattern.

[0051] The essence of extracting node features by graph neural networks is to aggregate the representations of neighboring nodes of a node (sometimes including its own node representation) and generate a new representation of the node through some combination. Usually we stack multiple graph neural network layers to obtain information about more distant neighbors. In the lth graph neural network layer, this process can be expressed by formulas (7) and (8):

[0052]

[0053]

[0054] in represents the set of neighbor nodes of node v, They represent the features of node u in the l-1th and lth graph neural network layers respectively, Represents the features obtained by aggregating the neighbor nodes of node v in the lth graph neural network layer, AGGREGATE (l) {·} and COMBINE (l) {·} respectively represent the aggregation and combination operations of the lth graph neural network layer. For graph convolutional neural network (GCN), this process can be expressed by formulas (9) and (10):

[0055]

[0056]

[0057] It can be seen that for a certain node in the graph convolutional neural network, the way it aggregates the features of the surrounding neighboring nodes is fixed, that is, taking the average. The way it combines and generates new features is through a linear transformation W (l) and a nonlinear transformation ReLU(·), where W (l)is equivalent to W in formula (1). Therefore, for graph convolutional neural networks, spatial knowledge is contained in W, and the spatial knowledge we need to transfer is W. In the previous section, we mentioned that the improved graph convolutional neural network model learns a unique convolution kernel parameter W for each node i based on the matrix decomposition method without significantly increasing the computational overhead. i , in order to capture the unique spatial dependency pattern of each node and improve the accuracy of prediction. In order to transfer spatial dependency knowledge, according to the previous analysis, the simplest way is to directly transfer the convolution kernel parameters W i . However, this method faces two major problems: First, how to match the nodes in the source domain and the target domain so that nodes with similar spatial dependencies use similar convolution kernel parameters. The current method to solve the matching problem is mainly to use the similarity (correlation) of historical time series data for matching. For example, in the RegionTrans method, it is based on the historical time series data of the target domain and the source domain. It searches for the most similar region in the source domain for each region in the target domain, and uses the spatiotemporal features of the region in the source domain to help learn the spatiotemporal features of the region in the target domain. The problem with these methods is that they only calculate the spatiotemporal similarity of two regions based on historical time series, ignoring the use of spatial information. Second, the computational overhead of matching nodes in the target domain one-to-one with nodes in the source domain is huge, especially when the number of nodes is large. Analysis W i It can be found that it is a linear combination of multiple convolution kernels: Therefore, we do not need to migrate W directly i , we only need to migrate the multiple convolution kernels that constitute it. In this way, we only need to learn the embedding vectors of the nodes in the target domain, which is much easier than directly learning the convolution kernels when data is scarce. In the source domain, we cluster the embedding vectors of the nodes, so that we can obtain several major spatial patterns and obtain the encoding vectors μ of these spatial patterns. k ,k={1,2,...,K}. In the target domain, we can use μ k To guide the learning of node spatial embedding vectors. The spatial clustering objective function we optimize in the target domain can be described by formula (11):

[0058]

[0059] For the temporal pattern, we learn a unique temporal pattern for each node based on matrix decomposition. In the migration process, we only need to migrate the temporal parameter pool matrix U and use the temporal pattern encoding vector π learned in the source domain. q,q={1,2,...,Q} to guide the learning of node temporal embedding vectors. The temporal clustering objective function we optimize in the target domain can be described by formula (12):

[0060]

[0061] Finally, we fine-tune the model on the target domain using a small amount of traffic time series data. The entire migration process and fine-tuning method are as follows:

[0062] a) Using the model parameters {P,U,Θ} learned in the source domain T ,b T ,W p ,b p} to initialize the model parameters {P′,U′,Θ′ on the target domain T ,b′ T ,W′ p ,b′ p}, randomly initialize {E′, R′}, and encode the spatial and temporal pattern categories learned in the source domain as μ k (k=1,2,...,K),π q (q=1,2,...,Q), let the number of fine-tuning rounds be M′;

[0063] b) For e = 1, 2, ..., M′do:

[0064] 1) For input in the target domain Use the model to get the corresponding output prediction value And calculate the prediction loss function:

[0065]

[0066] 2) Assign spatial and temporal categories to the embedding vector of each node in the target domain:

[0067]

[0068]

[0069] 3) Calculate the spatial clustering and temporal clustering loss functions on the target domain:

[0070]

[0071]

[0072] 4) According to the loss function L′=L′ p +α′·L′ s +β′·L′ t, calculate the gradient of the model parameters, fix {P′,U′}, and use the gradient to update the remaining parameters {E′,R′,Θ′ T ,b′ T ,W′ p ,b′ p}.

[0073] Beneficial effects of the present invention:

[0074] (1) Knowledge transfer based on graph traffic data

[0075] Currently, the most advanced traffic prediction models are based on graph data, but most traffic migration prediction methods are based on regular grid traffic data. This patent proposes a knowledge migration method based on graph traffic data, which can help improve the prediction accuracy of traffic prediction models based on graph data in small sample situations. Specifically, based on parameter matrix decomposition, by introducing a clustering algorithm in the graph node embedding subspace, multiple spatial and temporal pattern representations are learned from the data-rich source domain, and these pattern information is used to help improve the prediction accuracy of the model in the data-scarce data domain.

[0076] (2) Data-driven spatiotemporal pattern matching

[0077] Different from most current traffic migration prediction algorithms that need to explicitly introduce artificially defined similarities to match similar spatiotemporal patterns in the target domain and the source domain, this patent introduces spatial clustering and temporal clustering regularization terms (L s ,L t ,L′ s ,L′ t ), balances spatiotemporal pattern matching with prediction accuracy, and completes the matching of spatiotemporal pattern similarities of nodes in the target domain and the source domain in a data-driven manner, thereby achieving the effect of reducing negative transfer and significantly reducing the computational overhead of previous methods in the spatiotemporal pattern matching process. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a spatiotemporal pattern learning and prediction model architecture.

[0079] Figure 2 It is an ablation experiment. DETAILED DESCRIPTION

[0080] The present invention will be further described in detail below in conjunction with specific implementation modes, but the present invention is not limited to the specific implementation modes.

[0081] A traffic migration prediction method based on graph convolutional neural network includes two parts: pre-training on data-rich domains to acquire knowledge and transferring knowledge to data-scarce domains and fine-tuning:

[0082] 1. Train the network model on a data-rich domain to obtain spatial and temporal pattern information. The steps are as follows:

[0083] a) Assume that the model has been trained for M rounds, there are K types of spatial patterns, Q types of temporal patterns, and smoothing coefficients γ and θ. We randomly select E i K points in (i=1,2,...,N) are used as the initial centers of the spatial pattern k∈{1,2,...,K}, randomly select R i Q points in (i=1,2,...,N) are used as the initial centers of the time pattern q∈{1,2,...,Q};

[0084] b) For e = 1, 2, ..., Mdo:

[0085] 1. For input in the source domain Use the above model to get the corresponding output prediction value And calculate the prediction loss function:

[0086]

[0087] 2. Assign spatial and temporal categories to each node’s embedding vector:

[0088]

[0089]

[0090] 3. Calculate spatial clustering and temporal clustering losses:

[0091]

[0092]

[0093] 4. Calculate the center vector of each category:

[0094]

[0095]

[0096] 5. Smoothly update the category center vector:

[0097]

[0098]

[0099] 6. According to the loss function L = L p +α·L s +β·L t, calculate the gradient of model parameters, and update model parameters Θ = {E, P, R, U, Θ T ,b T ,W p ,b p}.

[0100] c) Get the trained model parameters Θ = {E, P, R, U, Θ T ,b T ,W p ,b p} and spatial pattern category information μ k (k=1,2,...,K), time pattern category information π q (q=1,2,...,Q).

[0101] 2. Transfer knowledge to data-scarce domains and fine-tune the model. The specific steps are as follows:

[0102] a) Using the model parameters {P,U,Θ} learned in the source domain T ,b T ,W p ,b p} to initialize the model parameters {P′,U′,Θ′ on the target domain T ,b′ T ,W′ p ,b′ p}, randomly initialize {E′, R′}, and encode the spatial and temporal pattern categories learned in the source domain as μ k (k=1,2,...,K),π q (q=1,2,...,Q), let the number of fine-tuning rounds be M′;

[0103] b) For e=1,2,...,M′do:

[0104] 1) For input in the target domain Use the model to get the corresponding output prediction value And calculate the prediction loss function:

[0105]

[0106] 2) Assign spatial and temporal categories to the embedding vector of each node in the target domain:

[0107]

[0108]

[0109] 3) Calculate the spatial clustering and temporal clustering loss functions on the target domain:

[0110]

[0111]

[0112] 4) According to the loss function L′=L′ p +α′·L′ s +β′·L′ t , calculate the gradient of the model parameters, fix {P′,U′}, and use the gradient to update the remaining parameters {E′,R′,Θ′ T ,b′ T ,W′ p ,b′ p}.

[0113] c) Get the fine-tuned model parameters Θ′={E′,P′,R′,U′,Θ′ T ,b′ T ,W′ p ,b′ p}, and use it to predict traffic time series data in the target domain.

[0114] Figure 1 It is a spatiotemporal pattern learning and prediction model architecture. The input data is historical traffic time series data and traffic network structure data. The graph convolutional neural network is used to capture spatial dependencies and cluster the spatial embedding matrix of nodes to obtain spatial patterns. The temporal convolutional network is used to capture temporal dependencies and cluster the temporal embedding vectors of nodes to obtain temporal patterns. Finally, the convolutional neural network is used to generate the final traffic time series data prediction results based on the extracted spatiotemporal features.

[0115] Figure 2 This is an ablation experiment. The California highway datasets PEMS4 and PEMS8 were used for the experiment, where the target domain only contains one day's traffic time series data. NodeTrans is the method for migrating spatial and temporal patterns proposed in this patent. NodeTrans-NS does not introduce spatial pattern migration, NodeTrans-NT does not introduce temporal pattern migration, and NodeTrans-NST does not introduce spatial and temporal pattern migration. It can be seen that migrating spatial and temporal patterns at the same time is the most effective, and migrating spatial patterns or temporal patterns alone can also improve prediction performance.

Claims

1. A traffic prediction method based on graph transfer learning, characterized in that: Here are the steps: (1) Learning traffic spatiotemporal patterns in data-rich source domains ①Model input: Use multiple sensors at different locations in the road network to record the traffic conditions at each observation point over a period of time. The recorded value at time t is Where C is the number of traffic features; after data cleaning, the traffic data is preprocessed based on the sliding window method; assuming that the traffic status within H time steps after time t needs to be predicted Then the corresponding model input is the traffic flow record value in the past S time steps Where N is the number of sensors; ②Model architecture: Capture spatial and temporal dependencies based on graph convolutional networks and temporal convolutional networks, and obtain traffic spatiotemporal patterns based on clustering algorithms; When traditional graph convolutional networks extract features from all nodes, the convolution kernel parameters used are the same. They extract the common spatial dependency patterns of all nodes, which can be described by formula (1): in, is the output and input of the convolutional layer, F and C are the number of output and input traffic features, is the identity matrix, is the adjacency matrix of the graph, is the degree matrix of the graph, is the convolution kernel parameter; in order to learn the unique spatial dependency pattern of each node, a separate convolution kernel is needed for each node, and the final convolution kernel parameter required is Based on matrix decomposition, the convolution kernel parameter W of the graph convolutional network is decomposed into the product of two matrices with fewer parameters. This process is described by formula (2): in, is the node space embedding matrix, its i-th row E i represents the embedding representation of the i-th node in the subspace; is the spatial parameter pooling matrix, whose i-th row P i is the i-th component of the convolution kernel parameter; D is the dimension of the embedded subspace, and D<<N; from the perspective of node i, it has a spatial embedding vector Use this spatial embedding vector from the spatial parameter pooling matrix A linear combination of the row vectors of the spatial parameter pooling matrix is ​​obtained: After shape adjustment, the convolution kernel parameters unique to node i are obtained By stacking multiple layers of graph convolutional neural networks, the information of neighbors at longer distances is taken into account; for the input The improved graph convolutional neural network mentioned above is used to capture spatial dependencies at each time step; In order to capture the node-specific temporal patterns, a gating mechanism is designed based on the temporal convolutional network to extract the unique temporal patterns of each node; the one-dimensional dilated causal convolution operation is described by formula (3): in, is a one-dimensional input sequence, is a convolution kernel, K l is the length of the convolution kernel, d f is the expansion coefficient; for high-dimensional input The dilated causal convolution operation with gating mechanism is described by formula (4): Among them, Θ T , b T is the convolution parameter, is the gating matrix, φ(·), σ(·) are the tanh and sigmoid activation functions respectively, is the output, ⊙ is the Hadamard product; based on matrix decomposition, the M matrix is ​​decomposed into the product of two matrices with fewer parameters, so that formula (4) is rewritten as formula (5): in, is the node time embedding matrix, is the time parameter pooling matrix, D′ is the dimension of the embedding subspace, and D′<<N; by stacking multiple time convolution layers, longer-term temporal dependencies are captured; Finally, based on the features extracted by the previous graph convolution layer and temporal convolution layer, a convolution layer is used to generate the final traffic prediction value. This process is described by formula (6): Among them, W p , b p are the parameters of the convolutional layer; Based on the matrix decomposition method, the unique time and space patterns of each node are learned; however, the time and space patterns of all nodes are not completely different, and there are some similarities between them. Based on the above method, the clustering algorithm is used to cluster the time and space patterns; ③Model training method: a) Assume that the number of model training rounds is M, there are K types of spatial patterns, Q types of temporal patterns, and smoothing coefficients are γ and θ; randomly select E i K points in (i=1,2,...,N) are used as the initial centers of the spatial pattern Randomly select R i Q points in (i=1,2,...,N) are used as the initial centers of the time pattern b)Fore=1,2,...,Mdo: 1) For input in the source domain Use the above model to get the corresponding output prediction value And calculate the prediction loss function: 2) Assign spatial and temporal categories to each node’s embedding vector: 3) Calculate spatial clustering and temporal clustering losses: 4) Calculate the center vector of each category: Among them, I k ,I q denote the set of nodes belonging to the kth spatial category and the qth temporal category respectively; 5) Smoothly update the category center vector: 6) According to the loss function L = L p +α·L s +β·L t , calculate the gradient of model parameters, and update model parameters Θ = {E, P, R, U, Θ T ,b T ,W p ,b p }, where α, β are hyperparameters; (2) Transfer knowledge to the target domain where data is scarce and perform model fine-tuning There are differences in the transportation networks of different cities or regions, which are reflected in the scale and connection mode of the transportation networks; however, although the transportation networks of different cities or regions are different, from the perspective of a certain node in the network, their spatial and temporal patterns are similar; The essence of extracting node features using graph convolutional neural networks is to aggregate the representations of a node's neighboring nodes and generate a new representation of the node through some combination. This process is expressed by formulas (7) and (8): in, represents the set of neighbor nodes of node v, They represent the features of node u in the l-1th and lth graph neural network layers respectively, Represents the features obtained by aggregating the neighbor nodes of node v in the lth graph neural network layer, AGGREGATE (l) {·} and COMBINE (l) {·} respectively denote the aggregation and combination operations of the lth graph neural network layer; For graph convolutional neural networks, this process is expressed by formulas (9) and (10): For a node in a graph convolutional neural network, the way it aggregates the features of its neighboring nodes is fixed, that is, taking the mean. i It is composed of a linear combination of multiple convolution kernels: It is only necessary to migrate the multiple convolution kernels that constitute it; in the target domain, only the embedding vector of the node needs to be learned; in the source domain, the embedding vectors of the nodes are clustered, so that several major spatial patterns are obtained, and the encoding vectors μ of these spatial patterns are obtained k , k = {1, 2, ..., K}; in the target domain, use μ k To guide the learning of node spatial embedding vectors; the spatial clustering objective function optimized in the target domain is described by formula (11): For the time pattern, based on the matrix decomposition method, a unique time pattern is learned for each node; in the migration process, similarly, only the time parameter pool matrix U needs to be migrated, and the time pattern encoding vector π learned in the source domain is used q ,q={1,2,...,Q} is used to guide the learning of node time embedding vectors; the time clustering objective function optimized in the target domain is described by formula (12): Finally, a small amount of traffic time series data is used to fine-tune the model in the target domain; the entire migration process and fine-tuning method are as follows: a) Using the model parameters {P,U,Θ} learned in the source domain T ,b T ,W p ,b p } to initialize the model parameters {P′,U′,Θ′ on the target domain T ,b′ T ,W′ p ,b′ p }, randomly initialize {E′, R′}, and encode the spatial and temporal pattern categories learned in the source domain as μ k (k=1,2,...,K),π q (q=1,2,...,Q), let the number of fine-tuning rounds be M′; b)Fore=1,2,...,M′do: 1) For input in the target domain Use the model to get the corresponding output prediction value And calculate the prediction loss function: 2) Assign spatial and temporal categories to the embedding vector of each node in the target domain: 3) Calculate the spatial clustering and temporal clustering loss functions on the target domain: 4) According to the loss function L′=L′ p +α′·L′ s +β′·L′ t , calculate the gradient of the model parameters, fix {P′,U′}, and use the gradient to update the remaining parameters {E′,R′,Θ′ T ,b′ T ,W′ p ,b′ p }.

Images