Multi-mode traffic demand prediction method based on federated learning

Abstract

The invention discloses a multi-mode traffic demand prediction method based on federated learning, and the method is characterized in that an attention federated learning framework is adopted, the attention federated learning framework comprises a fine-grained graph convolution circulation network which is used for capturing the spatial-temporal characteristics of a single mode at each local client, each client independently trains a model and uploads the model parameters to a global server, and the global server is connected with the fine-grained graph convolution circulation network; model parameters are merged through an attention mechanism, the similarity between different modes is calculated, and the generated global prediction model realizes multi-mode traffic demand prediction under the condition of not directly sharing data. Compared with the prior art, the method has the advantages that spatio-temporal data of different traffic modes are integrated, efficient demand prediction of a multi-mode traffic system is realized without directly sharing data, the accuracy of multi-mode traffic demand prediction is further improved on the premise of not sacrificing data privacy, the problems of data privacy and data ownership are solved, and the traffic demand prediction efficiency is improved. The method is expected to be widely applied to an intelligent traffic system, and has good application prospects and commercial development value.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-mode demand forecasting, and in particular to a multi-mode traffic demand forecasting method based on federated learning. Background Art

[0002] With the acceleration of urbanization, traffic congestion, travel time uncertainty and other issues are becoming increasingly serious. Accurate and timely traffic demand forecasting has become a key task in intelligent transportation systems. Multimodal traffic demand forecasting refers to predicting future travel demand by considering historical data of multiple transportation modes (such as public transportation, taxis, shared bicycles, etc.) at the same time. This is of great significance for urban traffic management, resource scheduling and infrastructure planning.

[0003] Most existing traffic demand forecasting models rely on data from a single mode and use traditional statistical or machine learning methods, such as the autoregressive integrated moving average model (ARIMA), linear regression, neural networks, etc. Although these methods can predict the demand of a single mode of transportation to a certain extent, due to the complexity and diversity of urban transportation systems, they often fail to capture the spatial and temporal correlations in multi-modal transportation systems. For example, the demand patterns of bicycles and taxis may be highly correlated in different time periods, and ignoring these potential correlations will lead to a decrease in prediction accuracy.

[0004] With the rapid development of deep learning, models such as convolutional neural networks (CNN), recurrent neural networks (RNN), and long short-term memory networks (LSTM) have performed well in time series data prediction. For example, CNN can extract spatial features through convolutional layers, while RNN is suitable for processing sequence data. However, these models usually require all data to be centralized on a server for training, which can easily lead to data privacy issues, especially when data on different traffic modes are controlled by different institutions or companies. This centralized data training is not only difficult to ensure privacy, but may also involve data security and ownership disputes. In response to this challenge, federated learning, as a new distributed machine learning framework, has been proposed to solve the problem of data privacy protection in multi-party data collaboration. Federated learning allows all parties to conduct collaborative learning by sharing model parameters without sharing the original data, thereby achieving joint modeling of multi-party data. This feature makes federated learning particularly suitable for scenarios where data modeling across multiple institutions is required, such as multi-modal demand prediction in intelligent transportation systems.

[0005] At present, many fields have tried to use federated learning to solve the data sharing problem. For example, in the fields of smartphone keyboards, edge computing, and healthcare, federated learning has demonstrated its potential to protect data privacy while improving model performance. However, these federated learning methods still have limitations, such as the inability to effectively capture the complex spatial and temporal correlations between cross-modal data. Therefore, introducing federated learning into multimodal traffic demand forecasting and improving it is an important topic in current research. In order to further improve the prediction accuracy and reduce the error of the model, graph convolutional networks are widely used in traffic forecasting. This network captures the spatial correlation between nodes (such as traffic stations or areas) through graph structure modeling, but it has certain limitations when processing time series data. To make up for this deficiency, graph convolutional recurrent networks combine the advantages of graph convolutional networks and recurrent neural networks, can capture spatial and temporal characteristics at the same time, and perform well in traffic flow and demand forecasting. However, existing multimodal traffic demand forecasting models based on graph convolutional networks or graph convolutional recurrent networks usually rely on centralized data training and are difficult to apply to actual multi-agency scenarios. Summary of the invention

[0006] The purpose of the present invention is to provide a multi-modal traffic demand forecasting method based on federated learning in response to the deficiencies of the prior art. An attention federated learning framework is used to perform model training on local clients of multiple traffic modes, and the model parameters of each mode are merged using a global server. The model training is combined with a fine-grained graph convolutional recurrent network to generate a global prediction model that realizes multi-modal traffic demand forecasting without directly sharing data. This method uses a fine-grained graph convolutional recurrent network model to independently train local data on each local client to capture the spatiotemporal characteristics of a single mode. Each client independently trains the model and uploads the model parameters to the global server. The server performs weighted merging of the model parameters of each client through a federated learning center, and calculates the similarity between different modes, thereby generating an efficient multi-modal prediction model without sharing the original data. This model can not only effectively protect data privacy and avoid cross-institutional data sharing issues, but also capture the potential correlation between different traffic modes through the sharing and merging of model parameters. In addition, the fine-grained graph convolution layer can more accurately capture the spatiotemporal characteristics of each traffic mode with the help of the dynamic embedding matrix, thereby further improving the accuracy of multimodal traffic demand forecasting without sacrificing data privacy. The method is simple and effective, and it solves the problems of data privacy and data ownership. The introduction of fine-grained graph convolution recurrent networks significantly improves the prediction accuracy. In the future, with the further opening of large-scale multimodal traffic data, it is expected to be widely used in intelligent transportation systems, with good application prospects and commercial development value.

[0007] The specific technical solution for achieving the purpose of the present invention is: a multi-modal traffic demand forecasting method based on federated learning, which is characterized in that the method avoids the problem of data privacy leakage by introducing a federated learning framework, and at the same time utilizes the spatiotemporal correlation information between different traffic modes to improve the accuracy of demand forecasting. Specifically, each institution conducts model training locally, and sends the trained parameters to the processing center for integration through the federated learning mechanism. The processing center performs parameter weighting processing according to the correlation of each mode, and returns the optimized parameters for local prediction. The method specifically includes the following steps:

[0008] Step 1: Data collection and preprocessing: collect spatiotemporal data from different transportation modes, including but not limited to taxis, buses, shared bicycles, etc., and preprocess the raw data to handle noise data, missing values ​​and outliers to ensure data quality.

[0009] Step 2: Initialize the number of hidden units in the recurrent neural network (RNN) to be between 64 and 128, and select 1, 2, or 3 layers for the graph convolution layer; set the input data embedding dimension to be tuned between 5, 10, and 15; and set the learning rate η to be tuned between 0.001, 0.005, or 0.01.

[0010] Step 3: Local client model construction: For each transportation mode (such as bus, taxi, bicycle, etc.), a fine-grained graph convolutional recurrent network is constructed on the local client to capture the spatiotemporal dependency of the mode. The graph convolutional network is used to process spatial correlation, and the recurrent neural network is used to process temporal dependency. The graph convolution is expressed by the following formula (a):

[0011]

[0012] Among them, A is the adjacency matrix, D is the degree matrix, W is the weight matrix, and b is the bias.

[0013] The time-dependent update of the recurrent neural network is expressed by the following formula (b):

[0014] h t =σ(W z ·[h t-1 ,x t ]+b z )(b).

[0015] Among them, h t is the hidden state at time step t, W z and b z are the training weights and biases.

[0016] Step 4: Local client model training. Use local data to train the model on each client and use the mini-batch gradient descent method shown in the following formula (c) to optimize the model parameters:

[0017]

[0018] Where η is the learning rate, is the loss function, Small batch data.

[0019] Step 5: Upload model parameters. After training, each local client uploads the parameters of the graph convolutional recurrent network model to the global server instead of sharing the original data. This ensures data privacy and avoids the leakage of sensitive information.

[0020] Step 6: Similarity calculation of the attention mechanism. To avoid simply weighting all parameters directly, the similarity between model parameters is calculated by the following formula (d) to determine which parameters have higher shared value:

[0021]

[0022] The kth layer parameters after merging Calculated by the following formula (e):

[0023]

[0024] in, is the kth layer parameter after merging; The similarity between patterns j1 and j2 is calculated by the similarity measurement function. The calculation of the similarity score is based on multiple indicators, such as Euclidean distance, cosine similarity or other measurement methods. The similarity is calculated by Euclidean distance or cosine similarity to ensure the validity of the parameters.

[0025] Step 7: Global server parameter merging: The global server merges the model parameters uploaded by different clients through the attention mechanism.

[0026] Step 8: Model parameter update and transmission back to the client. The server generates new global model parameters through the result of parameter merging, and transmits these updated model parameters back to each local client. Each local client uses the updated model parameters for further training to improve prediction accuracy.

[0027] Step 9: Calculate and optimize the model loss function. On the local client, calculate the error between the predicted value and the actual value, and optimize the model parameters using the loss function shown in the following formula (f):

[0028]

[0029] in, is the predicted value, For actual needs.

[0030] Step 10: Continuous iterative update of the model. During the entire process, the federated learning framework continuously updates the global model through periodic iterative training, and adjusts the collaboration between various modes through the attention mechanism to achieve the purpose of improving the overall prediction accuracy. After each round of training, the model parameters will continue to be updated, and the process of steps 4 to 9 will be repeated until the model converges.

[0031] Step 11: Traffic demand prediction. After receiving the updated model parameters, each client performs demand prediction based on the spatiotemporal data of its own mode, using graph convolutional layers to capture spatial correlations and RNN to capture temporal correlations. The predicted time step is set according to the specific scenario and is predicted by the following formula (g):

[0032]

[0033] in, Indicates local traffic patterns The prediction function of is the actual demand value at time step T.

[0034] The global prediction model supports the interdependence between multiple different traffic modes and improves the overall prediction performance by sharing the spatiotemporal features of different modes; the attention mechanism is used to determine which modes' model parameters have higher sharing value, thereby optimizing the prediction of the multi-modal transportation system.

[0035] Compared with the prior art, the present invention integrates distributed data sources from multiple traffic modes, performs multi-modal demand forecasting without directly sharing the original data, and optimizes the accuracy of demand forecasting by introducing a federated learning framework and combining the spatiotemporal characteristics of multi-modal traffic data, while protecting data privacy, and is suitable for large-scale, multi-regional, and multi-modal traffic demand forecasting tasks. Without sacrificing data privacy, the accuracy of multi-modal traffic demand forecasting is improved, the method is simple, the use effect is good, and the data privacy and data ownership issues are well solved. The introduction of a fine-grained graph convolutional recurrent network significantly improves the prediction accuracy, can process large-scale multi-modal traffic system data, and supports effective prediction when the data set format and scale are inconsistent. It can be widely used in smart transportation, urban planning, public transportation scheduling and other fields, and has good application prospects and commercial development value. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A system diagram of the architecture of the present invention;

[0037] Figure 2 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0038] The invention is further described in detail with reference to the following specific embodiments and drawings.

[0039] Example 1

[0040] See also Figure 1 , a multi-modal traffic demand forecasting system using the architecture of the present invention, the system includes: a local client module, a model training module, and a global aggregation optimization module, in which each client represents a traffic mode (such as bus, taxi, shared bicycle, etc.), and each client uses a fine-grained graph convolutional recurrent network to capture the spatiotemporal dependencies of the traffic mode. The local client trains the graph convolutional recurrent network model based on the spatiotemporal data of its traffic mode to generate local model parameters, and each client uploads the locally trained model parameters to the global optimization aggregation module after the training is completed. The global server uses the attention federated learning mechanism to merge the model parameters uploaded from different local clients, and returns these parameters to each local client for further optimization. The present invention can effectively utilize the data of multiple traffic modes for demand forecasting, and ensure the privacy protection of the data through federated learning.

[0041] See also Figure 2 , the following steps are used to predict multi-modal traffic demand, and the specific implementation process is as follows:

[0042] Step 1: Collect spatiotemporal data from different transportation modes (such as shared bikes, taxis, buses, etc.) in multiple cities. In the specific implementation, three real-world datasets are used: shared bike data in a certain city, taxi data in another certain city, and taxi data in another certain city.

[0043] In data preprocessing, the original data was first subjected to noise removal, missing value filling, and outlier removal. Then, the data was normalized using Min-Max normalization to ensure that the model can effectively learn between different traffic modes. After normalization, the data was restored to its original scale to evaluate the prediction effect of the model.

[0044] Step 2: For each traffic mode, a fine-grained graph convolutional recurrent network is built on the local client to capture spatiotemporal dependencies. In the specific implementation: Each local client uses local data to train the model and uses the mini-batch gradient descent method to optimize the model parameters.

[0045] Step 3: Upload and merge model parameters. After training, each local client uploads the parameters of the graph convolutional recurrent network model to the global server. The server merges the model parameters of different traffic modes through the attention mechanism in federated learning.

[0046] Step 4: Send the updated local model parameters to each local client for optimization until the model converges, and use the optimized model to perform multi-modal traffic prediction on three real-world data sets. Experiments show that the global prediction model constructed by the present invention has certain advantages in demand prediction.

[0047] The above is only a further explanation of the present invention and is not intended to limit this patent. Any equivalent implementation of the present invention should be included in the scope of the claims of this patent.

Claims

1. A multi-modal traffic demand forecasting method based on federated learning, characterized in that: The method trains models on local clients of multiple traffic modes and uses a global server to merge model parameters of each mode, thereby realizing multi-modal traffic demand prediction without directly sharing data. Specifically, the method includes the following steps: Step 1: Data collection and preprocessing Collect spatiotemporal data from different transportation modes including taxis, buses and shared bikes, and pre-process the collected raw data for noise data, missing values ​​and outliers; Step 2: Initialize network settings The number of hidden units in the initialized recurrent neural network is set between 64 and 128, the number of graph convolutional layers is selected to be 1, 2, or 3, the embedding dimension of the input data is set to be tuned between 5, 10, and 15, and the learning rate η is set to be tuned at 0.001, 0.005, or 0.01; Step 3: Local client model construction For each transportation mode including buses, taxis and bicycles, a fine-grained graph convolutional recurrent neural network is constructed on the local client to capture the spatiotemporal dependencies of the mode. The graph convolution captures the spatial correlation of the mode by the following formula (a): Among them, A is the adjacency matrix; D is the degree matrix; W is the weight matrix; b is the bias; The time dependency of the recurrent neural network is updated by the following formula (b): h t =σ(W z ·[h t-1 ,x t ]+b z ) (b); Among them, h t is the hidden state at time step t; W z and b z are the weights and biases for training; Step 4: Local client model training The model is trained on each client using local data, and the model parameters are optimized using the mini-batch gradient descent method shown in the following formula (c): Where η is the learning rate; is the loss function, For small batches of data; Step 5: Upload model parameters After training is completed, each local client uploads the parameters of the graph convolutional recurrent neural network model to the global server; Step 6: Similarity calculation of attention mechanism The following formula (d) is used to calculate the similarity between model parameters to determine which parameters have higher shared value: in,; is the similarity between patterns j1 and j2 calculated by the similarity measurement function; is the kth layer parameter after merging; Similarity is the similarity between each layer of the Euclidean distance measurement mode; Normalize is the normalization process; Step 7: Global server parameter merge The global server merges the model parameters uploaded by different clients through the attention mechanism: Step 8: Update model parameters and send back to the client The server generates new global model parameters based on the result of parameter merging and transmits these updated model parameters back to each local client; Step 9: Model loss function calculation and optimization On the local client, the error between the predicted value and the actual value is calculated, and the model parameters are optimized using the loss function shown in the following formula (f): in, is the predicted value; For actual needs; Step 10: Continuous iterative updates of the model The federated learning framework is used to perform periodic iterative training to continuously update the global model, and the attention mechanism is used to adjust the collaboration between various modes to improve the overall prediction accuracy. After each round of training, the model parameters will continue to be updated, and steps 4 to 9 will be repeated until the model converges. Step 11: Traffic Demand Forecast Each client uses the updated model parameters to perform demand forecasting based on the spatiotemporal data of its own model, using the graph convolution layer to capture spatial correlation and the recurrent neural network to capture temporal correlation. The predicted time step is set according to the specific scenario, and the demand forecasting is performed by the following formula (g): in, For local traffic mode The prediction function of Represents the actual demand value at time step T.

2. The multi-modal traffic demand forecasting method based on federated learning according to claim 1 is characterized in that: The similarity score is calculated using the Euclidean distance or cosine similarity method.

3. The multi-modal traffic demand forecasting method based on federated learning according to claim 1, characterized in that: The missing values ​​​​are handled using a generative adversarial network.

4. The multi-modal traffic demand forecasting method based on federated learning according to claim 1, characterized in that: The kth layer parameters after merging Calculated by the following formula (e): in, is the kth layer parameter after merging; is the similarity between patterns j1 and j2 calculated by the similarity measure function.

Images