Cluster type federated learning-based urban traffic flow prediction method

Through clustered federated learning and convolutional recursive neural networks, combined with cross-fusion and Staleness-aware algorithm, the spatiotemporal dependence and data privacy issues in traffic flow prediction are solved, and more efficient and safe traffic flow prediction is achieved.

CN119068664APending Publication Date: 2024-12-03BEIJING INST OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410806117.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-21
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing traffic flow prediction methods are difficult to effectively capture complex space-time dependencies, and there is a risk of data privacy leakage, especially in multiple participants scenarios.

Method used

A cluster-based federated learning method is adopted to capture space-time dependencies by constructing cluster topology and convolutional recursive neural networks, and parameter aggregation is performed using cross-fusion and Staleness-aware global update algorithms to avoid directly sharing the original data.

Benefits of technology

It improves the accuracy and security of traffic flow prediction, ensures data privacy protection, makes full use of local data resources and reduces the impact of cluster asynchronous training on the global model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068664A_ABST
    Figure CN119068664A_ABST
Patent Text Reader

Abstract

The invention relates to an urban traffic flow prediction method based on cluster federal learning, and belongs to the technical field of intelligent traffic. According to the method, a cluster-type federated learning framework is adopted, a space-time dependency relationship is mined on the basis of protecting privacy data, and a cluster-type traffic flow space-time prediction model is constructed; comprising the steps of acquiring urban traffic flow data information, constructing a cluster topological structure, predicting and calculating by a local client, aggregating cluster leader parameters, updating global parameters of a central server and the like. According to the method, the complex dependency relationship of the traffic data in time and space is considered in combination with the actual urban road network structure and the situation that data leakage is likely to happen to multiple participants, the data privacy is protected, and the prediction accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an urban traffic flow prediction method based on cluster federated learning, belonging to the technical field of intelligent transportation. Background Art

[0002] In recent years, with the vigorous development of social economy and the continuous improvement of people's living standards, the number of cars has increased day by day, and urban cars are facing more and more traffic congestion problems. How to accurately and efficiently predict urban traffic flow is of great significance to solving the traffic congestion problem on urban roads, reducing environmental pollution and improving traffic travel efficiency.

[0003] At present, there have been many research works on traffic flow prediction tasks, mainly focusing on how to use centralized machine learning methods to improve the prediction accuracy, such as autoregressive models (AR), long short-term memory networks (LSTM) and graph neural networks (GNN). Some methods only consider time correlation and ignore the spatial dependency between roads, resulting in low prediction accuracy. Most methods only use centralized learning prediction, which may cause the leakage of private traffic data. Therefore, the task of urban traffic flow prediction mainly faces two difficulties: complex time and space dependencies and data security and privacy issues.

[0004] In view of the complex spatiotemporal dependencies, urban traffic flow is not only closely related to the passage of time, but also closely related to the divergence and convergence of roads. In terms of time, the traffic volume on weekdays and holidays will be significantly different, and the rush hour will greatly affect the traffic flow information. In terms of space, the intersection of main roads and auxiliary roads and the connection of road intersections will affect each other's traffic volume. The changes in urban and suburban road traffic caused by rush hour involve complex time and space dependencies. Therefore, how to mine nonlinear spatiotemporal dependencies to accurately predict urban road traffic is a challenging problem.

[0005] Regarding data security and privacy issues, the collection of urban traffic flow information in real life often comes from different institutions, such as government transportation departments and various travel platforms. If traditional centralized learning methods are used to achieve traffic prediction, all data information needs to be shared and aggregated on a central server for training. This process greatly increases the possibility of privacy data leakage. Therefore, how to build a secure and reliable model architecture is a practical challenge to avoid potential privacy leakage issues. Summary of the invention

[0006] The purpose of the present invention is to solve the technical defects of the existing traffic flow prediction methods. In combination with the actual road network structure of the city and the situation where data leakage is prone to occur among multiple participants, the complex dependencies in time and space are considered. In order to protect data privacy and solve security risks, a method for predicting urban traffic flow based on clustered federated learning is proposed.

[0007] In order to achieve the above object, the present invention adopts the following technical scheme:

[0008] A method for predicting urban traffic flow based on cluster federated learning includes the following steps:

[0009] Step 1: Obtain urban traffic flow data information and preprocess it to obtain spatiotemporal series data on urban traffic flow. As shown in the following formula:

[0010]

[0011] Among them, χ is the spatiotemporal series data, n represents the number of traffic roads, and w represents the observation window. Represents a real number set. Input χ into the cluster-based spatiotemporal traffic flow prediction model.

[0012] Step 2: The cluster traffic flow spatiotemporal prediction model constructs a cluster topology structure based on urban traffic flow data to capture the spatial dependency of urban traffic flow data.

[0013] Specifically, building a cluster topology structure includes the following steps:

[0014] Step 2.1: Use variable fuzzy clustering method to divide local clients into k clusters [C1, C2, …, C k ] so that the correlation between variables in the same cluster is as high as possible, while the correlation between variables in different clusters is as low as possible.

[0015] The specific calculation process is as follows:

[0016] Step 2.1.1: Randomly initialize the participation matrix U and cluster center matrix V according to the number of variables n and the number of clusters k; where, μ ij Represents variable x i In cluster C j The level of participation in.

[0017] Step 2.1.2: Calculate the cluster center C j and variable x i In cluster C j Level of participation in Where m is the ambiguity weighting index;

[0018] Step 2.1.3: Continuously update the participation matrix U and the cluster center matrix V to minimize and obtain the minimum value, where 1 < m < ∞. When J(x i , C j ) reaches the minimum value, stop the update.

[0019] Step 2.1.4: According to the participation matrix U, obtain the cluster division result [C1, C2, …, C k of the variables;

[0020] Step 2.2: Based on the participation matrix U, select the local client with the highest participation in the cluster as the leader node CH of this cluster k ;

[0021] Step 3: The local client captures the local and global information in the time pattern through a convolutional recurrent neural network, thereby capturing the time dependence of urban traffic flow data and obtaining the prediction result of the local traffic data;

[0022] Among them, the specific process of constructing the convolutional recurrent neural network prediction algorithm includes:

[0023] Step 3.1: The convolutional neural network uses k filters to obtain the short-term dependence information in the time dimension and calculates the output result o C ;

[0024] Step 3.2: The recurrent neural network adopts the gate learning of the Bi-LSTM network to obtain the long-term dependence information in the time dimension and calculates the output result h t ;

[0025] Step 3.3: Concatenate the output result o C of the convolutional neural network and the output result h t of the recurrent neural network as the result of the local client's convolutional recurrent neural network prediction algorithm.

[0026] Step 4: The local client sends the local model parameters to the leader node CH of its affiliated cluster k , and CH k uses the cross-fusion method to aggregate the model parameters of the local clients in the same cluster to obtain the aggregation result ω k , and sends it to the central server in the federated learning framework.

[0027] Step 5: The central server uses the Staleness-aware global update algorithm to update the parameters of the central server's global model and send the fused and updated parameters to each leader node CH k ;

[0028] The specific process of the Staleness-aware global update algorithm includes:

[0029] Step 5.1: Since the aggregation of each cluster is completed asynchronously, the number of updates of each cluster since the last global update is different. This number of updates is defined as the stale value τ;

[0030] Step 5.2: Based on the stale value τ, calculate the weight value of the parameter ω(k) of cluster k at the training epoch t

[0031] Step 5.3: According to the weight value of parameter ω(k) Calculate the updated result of the global model ω t , send this result to the leader node CH k ;

[0032] Step 6: Leader Node CH k Get the updated parameter ω t , respectively sent to each local client;

[0033] Step 7: After the local client obtains the parameters, it updates the local model parameters;

[0034] Step 8: Repeat steps 3 to 7 until the expected effect is achieved and the final urban traffic flow prediction result is obtained.

[0035] At this point, after steps 1 to 8, the functions of acquiring, processing and predicting urban traffic flow have been completed. In addition, the complex dependencies between traffic roads in spatial and temporal dimensions have been captured, and the security and privacy protection of data information have been guaranteed, thus realizing a cluster-based federated learning method for predicting urban traffic flow.

[0036] Beneficial Effects

[0037] Compared with the prior art, the method of the present invention has the following advantages:

[0038] 1. The present invention captures the spatial dependency of urban traffic flow data by constructing a cluster topology structure, and uses a convolutional recurrent neural network to capture the local and global information in the time pattern, thereby capturing the time dependency of urban traffic flow data. Furthermore, the cross-fusion method of the cluster leader node effectively mines the implicit spatiotemporal dependency in urban traffic flow, thereby effectively improving the accuracy of urban traffic flow prediction.

[0039] 2. The present invention adopts a clustered federated learning framework and updates the urban traffic flow prediction model through a parameter transfer aggregation mechanism, rather than directly sharing the original data between multiple clients. This can prevent the privacy information between different data sources from being attacked and leaked, and effectively improve the security of urban traffic flow prediction.

[0040] 3. The present invention designs a staleness-aware global update algorithm to implement the aggregate update of security parameters. This algorithm determines the update of the global model through stale values, and can realize that all clusters are trained simultaneously and asynchronously. Through these trainings, the data resources of the local client can be fully utilized and good convergence performance can be achieved. At the same time, this algorithm can also reduce the impact of the local client disconnection on the stagnation of the global model by updating the weights with stale values. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the process of the present invention;

[0042] Figure 2 It is a schematic diagram of the framework of the present invention;

[0043] Figure 3 It is a schematic diagram of the training process in the present invention;

[0044] Figure 4 Schematic diagram of the convolutional recurrent neural network prediction algorithm in the present invention. DETAILED DESCRIPTION

[0045] The present invention is further described in detail below with reference to the accompanying drawings.

[0046] like Figure 1 As shown, a method for predicting urban traffic flow based on cluster federated learning includes obtaining urban traffic flow data information, building a cluster topology structure, local client prediction calculation, cluster leader parameter aggregation, and central server global parameter update.

[0047] The objects involved in this method include the central server, cluster leader and local client, which form a three-layer structure. Figure 2 As shown, in the three-layer structure, the local client communicates with the cluster leader to transmit parameters, and the cluster leader communicates with the central server to transmit parameters. Among them, the cluster leader is selected from the cluster topology after the local client builds the cluster topology.

[0048] Specifically, the training process of this method is as follows Figure 3As shown in the figure. The local client uploads parameters to the cluster leader, and the cluster leader aggregates the parameters and sends them to the central server. The central server performs aggregation and update, and then sends the updated parameters to the cluster leader, and the cluster leader further sends them to the local client. The specific steps are as follows:

[0049] Step 1: Obtain the urban traffic flow data information, preprocess it to obtain the spatio-temporal sequence data PeMSD7 of urban traffic flow. Among them, the number of traffic roads n = 228, and the observation window w = 12672. Input it into the cluster-based traffic flow spatio-temporal prediction model.

[0050] Step 2: The cluster-based traffic flow spatio-temporal prediction model constructs a cluster topology structure according to the urban traffic flow data, so as to capture the spatial dependence relationship of the urban traffic flow data.

[0051] Among them, the specific process of the algorithm for constructing the cluster topology structure is as follows:

[0052] Step 2.1: Adopt the variable fuzzy clustering method to divide the local clients into k = 8 clusters [C1, C2,..., C8], so that the correlation between variables in the same cluster is as high as possible, while the correlation between variables in different clusters is as low as possible. The specific calculation process is as follows:

[0053] Step 2.1.1: According to the number of variables n = 228 and the number of clusters k = 8, randomly initialize the membership matrix and the cluster center matrix V = [C1,..., C8]; where μ ij represents the degree of participation of variable x i in cluster C j .

[0054] Step 2.1.2: Calculate the cluster center C and the degree of participation of variable x in cluster C j through i and j . Among them, m is the fuzzy weighted exponent, and the calculation limit condition is and μ ij ∈[0,1]. C l represents the center of the l-th cluster, l ∈ [1, k].

[0055] Step 2.1.3: Continuously update the membership matrix U and the cluster center matrix V so that obtains the minimum value, 1 < m < ∞. When J(x i , C j ) reaches the minimum value, stop the update.

[0056] Step 2.1.4: Obtain the cluster division results of the variables [C1, C2, …, C8] according to the participation matrix U;

[0057] Step 2.2: Based on the participation matrix U, the local client with the largest participation in the cluster is selected as the leader node of this cluster, and a total of 8 leader nodes [CH1,...,CH8] are determined;

[0058] Step 3: The local client captures the local and global information in the time pattern through a convolutional recurrent neural network, thereby capturing the time dependency of urban traffic flow data and obtaining the prediction results of local traffic data;

[0059] Among them, the specific process of constructing the convolutional recurrent neural network prediction algorithm is as follows:

[0060] Step 3.1: The convolutional neural network uses k filters to obtain short-term dependency information in the time dimension, which is calculated as: Calculate the output result o C , where * represents the convolution operation, RELU represents the activation function, and W i represents the weight matrix, E C represents the characterization result of local data, b i Represents the bias term.

[0061] Step 3.2: The recurrent neural network uses the gate learning of the Bi-LSTM network to obtain the long-term dependency information in the time dimension. The calculation process is as follows:

[0062] Step 3.2.1: The LSTM network loop function is expressed as:

[0063] h t , c t =LSTM(h t-1 , c t-1 , x t )

[0064] Where t represents the observation time point; h t represents the hidden state, c t represents the cell state, x t Represents the input value at time point t.

[0065] Step 3.2.2: The calculation process of the LSTM network is as follows:

[0066] r t =σ(x t W xr +h t-1 W hr +b r )

[0067] u t =σ(x t W ur +h t-1 W hu +b r )

[0068] c t =RELU(x t W xc +r t ☉(h t-1 W hc )+b c )

[0069] h t =(1-u t )⊙h t-1 +u t ⊙c t )

[0070] Among them, x t represents the input value at time point t, σ represents the sigmoid function, ⊙ represents the element-wise product, r t represents the output of the forget gate, u t Represents the input gate output; W xr , W hr , W ur , W hu , W xc , W hc are all weight matrices; b r , b c Represents the bias term.

[0071] Through the calculation of the LSTM network, the hidden state h is obtained t .

[0072] Step 3.2.3: Bi-LSTM combines two ordinary LSTM networks to capture both forward and reverse dependencies and By splicing Get the output result of the prediction generation module. represents the forward hidden state, Represents the inverted hidden state.

[0073] Step 3.3: The output of the convolutional neural network C And the output of the recurrent neural network h t Perform stitching as the result of the local client-side convolutional recurrent neural network prediction algorithm;

[0074] Step 4: The local client sends the local model parameters to the leader node CH of the cluster to which it belongsk , CH k The cross-fusion method is used to aggregate the model parameters of local clients in the same cluster to obtain the aggregated result ω k , and send it to the central server in the federated learning framework.

[0075] The polymerization process is as follows:

[0076] Step 4.1: The local client of each cluster sends parameters to the leader node CH k ;

[0077] Step 4.2: Leader Node CH k The variable cross-fusion method is used to obtain cluster aggregation results by simulating the interaction between each pair of variables given a real-valued feature vector. In each cluster, apply The correlation between the parameters of each client in the cluster is calculated, where VIR represents the function of obtaining the cluster aggregation result, ω0 represents the overall deviation, and ω i represents the weight of the i-th variable, i ,u j > represents the variable x i and x j The pairwise interactions between Represents the dot product. A total of 8 cluster aggregation results are obtained

[0078] Step 4.3: Leader Node CH k Send the aggregated results to the central server;

[0079] Step 5: The central server uses the staleness-aware global update algorithm to update the parameters of the central server global model and sends the updated parameters to each leader node CH k .

[0080] Among them, the specific process of the Staleness-aware global update algorithm is as follows:

[0081] Step 5.1: Since the aggregation of each cluster is done asynchronously, the number of updates for each cluster since the last global update is different. This number of updates is defined as the stale value τ.

[0082] Step 5.2: According to the stale value τ, Calculate the weight value of the parameter ω(k) of cluster k when the number of training epochs is t Where a>0, b>0; as the weights of the initial model.

[0083] ​According to the calculation of α value, the weight value of parameter ω(k) when the number of training epochs is t The calculation method is expressed as:

[0084] Step 5.3: According to the weight value of parameter ω(k) By formula Calculate the updated result of the global model ω t , send this result to the leader node CH k ;

[0085] Step 6: Leader Node CH k Get the updated parameter ω t , respectively sent to each local client;

[0086] Step 7: After obtaining the parameters, the local client updates the local model parameters;

[0087] Step 8: Repeat steps 3-7 until the expected effect is achieved and the final urban traffic flow prediction result is obtained.

[0088] At this point, after steps 1 to 8, a method for predicting urban traffic flow based on cluster federated learning has been completed.

[0089] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for predicting urban traffic flow based on cluster federated learning, characterized in that: The following steps are involved: Step 1: Obtain urban traffic flow data information and preprocess it to obtain spatiotemporal series data on urban traffic flow; Step 2: The cluster traffic flow spatiotemporal prediction model constructs a cluster topology structure based on urban traffic flow data to capture the spatial dependency of urban traffic flow data; Step 3: The local client uses a convolutional recurrent neural network to capture the local and global information in the time pattern, thereby capturing the time dependency of urban traffic flow data and obtaining the prediction results of local traffic data; Among them, building a convolutional recursive neural network prediction algorithm includes the following steps: Step 3.1: The convolutional neural network uses k filters to obtain short-term dependency information in the time dimension and calculates the output result o C ; Step 3.2: The recurrent neural network uses the gate learning of the Bi-LSTM network to obtain the long-term dependency information in the time dimension and calculate the output result h t ; Step 3.3: The output of the convolutional neural network C And the output of the recurrent neural network h t Perform stitching as the result of the local client-side convolutional recurrent neural network prediction algorithm; Step 4: The local client sends the local model parameters to the leader node CH of the cluster to which it belongs k , CH k The cross-fusion method is used to aggregate the model parameters of local clients in the same cluster to obtain the aggregated result ω k , and send it to the central server in the federated learning framework; Step 5: The central server uses the staleness-aware global update algorithm to update the parameters of the central server global model and sends the updated parameters to each leader node CH k ; Among them, the Staleness-aware global update algorithm includes: Step 5.1: Define the number of updates as the stale value τ; Step 5.2: Based on the stale value τ, calculate the weight value of the parameter ω(k) of cluster k at the training epoch t Step 5.3: According to the weight value of parameter ω(k) Calculate the updated result of the global model ω t , send this result to the leader node CH k ; Step 6: Leader Node CH k Get the updated parameter ω t , respectively sent to each local client; Step 7: After the local client obtains the parameters, it updates the local model parameters; Step 8: Repeat steps 3 to 7 until the expected effect is achieved and the final urban traffic flow prediction result is obtained.

2. The urban traffic flow prediction method based on cluster federated learning as claimed in claim 1, characterized in that: In step 1, the processing method is as follows: Among them, χ is the spatiotemporal series data, n represents the number of traffic roads, and w represents the observation window. represents a set of real numbers; χ is input into the cluster-based spatiotemporal traffic flow prediction model.

3. The urban traffic flow prediction method based on cluster federated learning as claimed in claim 1, characterized in that: In step 2, the cluster topology is constructed, including the following steps: Step 2.1: Use variable fuzzy clustering method to divide local clients into k clusters [C1, C2, …, C k ] so that the correlations between variables in the same cluster are as high as possible, and the correlations between variables in different clusters are as low as possible; Step 2.1.1: Randomly initialize the participation matrix U and cluster center matrix n according to the number of variables n and the number of clusters k; Step 2.1.2: Calculate the cluster center C j and variable x i In cluster C j Level of participation in Where m is the ambiguity weighting index; Step 2.1.3: Continuously update the participation matrix U and the cluster center matrix V to minimize obtaining the minimum value, where 1 < m < ∞; when J(x i , C j ) reaches the minimum value, stop the update; Step 2.1.4: According to the participation matrix U, obtain the clustering results of the variables [C1, C2, …, C k ]; Step 2.2: Based on the participation matrix U, select the local client with the highest participation in the cluster as the leader node CH of this cluster k .

4. The urban traffic flow prediction method based on cluster federated learning as claimed in claim 3, characterized in that: Among them, μ ij Represents variable x i In cluster C j level of participation in.

5. The urban traffic flow prediction method based on cluster federated learning as claimed in claim 3, characterized in that: pass and Calculate the cluster center C j and variable x i In cluster C j Level of participation in Where m is the ambiguity weighting index, and the calculation limit is And μ ij ∈[0,1]; C l represents the lth cluster center, l∈[1,k].

6. The urban traffic flow prediction method based on cluster federated learning as claimed in claim 1, characterized in that: In step 3, Where * represents the convolution operation, RELU represents the activation function, and W i represents the weight matrix, E C represents the characterization result of local data, b i represents the bias term, and k represents the number of filters.

7. The urban traffic flow prediction method based on cluster federated learning as claimed in claim 1, characterized in that: In step 3, the long-term dependency information in the time dimension is calculated as follows: Step 3.2.1: The LSTM network loop function is expressed as: h t ,c t =LSTM(h t-1 ,c t-1 ,x t ) Where t represents the observation time point; h t represents the hidden state, c t represents the cell state, x t represents the input value at time point t; Step 3.2.2: The calculation process of the LSTM network is as follows: r t =σ(x t W xr +h t-1 W hr +b r ) u t =σ(x t W ur +h t-1 W hu +b r ) c t =RELU(x t IN xc +r t ⊙(h t-1 IN hc )+b c ) h t =(1-u t )⊙h t-1 +u t ⊙c t ) Among them, x t represents the input value at time point t, σ represents the sigmoid function, ⊙ represents the element-wise product, r t represents the output of the forget gate, u t Represents the input gate output; W xr , W hr , W ur , W hu , W xc , W hc are all weight matrices; b r 、b c represents the bias term; Through the calculation of the LSTM network, the hidden state h is obtained t ; Step 3.2.3: Bi-LSTM combines two ordinary LSTM networks to capture both forward and reverse dependencies and By splicing Get the output result of the prediction generation module; represents the forward hidden state, Represents the inverted hidden state.

8. The urban traffic flow prediction method based on cluster federated learning as claimed in claim 1, characterized in that: In step 4, the aggregation process is as follows: Step 4.1: The local client of each cluster sends parameters to the leader node CH k ; Step 4.2: Leader Node CH k The variable cross-fusion method is used to obtain the cluster aggregation results by simulating the interaction between each pair of variables given a real-valued feature vector; in each cluster, the application The correlation between the parameters of each client in the cluster is calculated, where VIR represents the function of obtaining the cluster aggregation result, ω0 represents the overall deviation, and ω i represents the weight of the i-th variable, <υ i ,υ j > represents the variable x i and x j The pairwise interactions between , ° represents the dot product; Step 4.3: Leader Node CH k The aggregated results are sent to the central server.

9. The urban traffic flow prediction method based on cluster federated learning as claimed in claim 1, characterized in that: In step 5.2, according to the stale value τ, Calculate the weight value of the parameter ω(k) of cluster k when the number of training epochs is t Where a>0, b>0; α=1- As the weight of the initial model; According to the calculation of α value, the weight value of parameter ω(k) when the number of training epochs is t The calculation method is expressed as: In step 5.3, according to the weight value of parameter ω(k) By formula Calculate the updated result of the global model ω t , send this result to the leader node CH k .