Cross-travel platform supply and demand prediction method based on graph federal learning

Abstract

The invention discloses a cross-travel platform graph federal learning prediction method, and belongs to the field of federal calculation. The central server is responsible for coordinating each travel platform, decoupling operators, aggregating models and updating the models; the travel platform side mainly cooperates with the server to carry out local space-time diagram modeling and local space-time feature extraction. Each travel platform firstly divides a service area into different grid areas; then, calculating supply and demand data of different regions according to historical data, and taking historical supply and demand in the regions as prediction features; and then, modeling the association relationship among different regions into a graph federation, and carrying out cross-platform feature aggregation under the coordination of a central server to realize cross-platform space-time supply and demand prediction. According to the invention, the calculation overhead is reduced, and both privacy and efficiency are considered.

Description

Technical Field

[0001] The present invention belongs to the field of federated computing and relates to a supply and demand forecasting method based on graph federated learning, and specifically to a cross-travel platform supply and demand forecasting method based on graph federated learning. Background Art

[0002] The technical fields related to the present invention mainly include two: a supply and demand forecasting method for large-scale shared travel and a privacy protection method for federated spatiotemporal graph forecasting.

[0003] Most of the previous supply and demand forecasting methods for shared mobility only use the domain of a single platform for supply and demand forecasting. These methods usually model spatial regions as graph nodes and then combine them with time series models for forecasting. These methods are limited by the data of a single platform and cannot use the data of various travel platforms to form a global perspective. Therefore, it is easy to lead to limited forecasting accuracy in supply and demand forecasting, that is, there is "forecast bias". Recently, some methods use federated learning methods to train machine learning models using data from multiple institutions, which usually has significant improvements over models trained with data from a single institution.

[0004] There are two main types of privacy protection technologies for federated spatiotemporal graph prediction: graph embedding vector technology and homomorphic encryption technology. In the graph embedding vector technology, each federation member represents its private spatiotemporal graph as an embedding vector, and believes that the vector can effectively protect privacy. After obtaining the graph embedding vectors of each party, the federation central server can directly adopt the same spatiotemporal graph prediction method as in the single platform scenario. However, the privacy of graph embedding vectors is not guaranteed in theory and practice, and there is a privacy risk of original data leakage. In the data based on homomorphic encryption technology, some works allow all federation parties to obtain the same set of encryption public keys to encrypt various intermediate results transmitted in the federated prediction. In this case, the spatiotemporal graph prediction method of a traditional single travel platform can be simply extended to a cross-platform federated prediction scenario under the constraints of data privacy protection, so as to achieve supply and demand prediction using historical supply and demand data in different regions of each shared travel platform under the constraints of data privacy protection. However, this technology will lead to excessive communication volume and computing time overhead, making it difficult to achieve high-time supply and demand prediction across shared travel platforms.

[0005] In summary, existing methods still lack a federated supply and demand forecasting method for cross-shared travel platforms. Moreover, for commonly used spatiotemporal forecasting technologies, especially the frequently used spatiotemporal graph forecasting methods, there is still a lack of a federal privacy protection framework that takes into account both data privacy security and the timeliness of supply and demand forecasting. Summary of the invention

[0006] By analyzing and comparing the existing technologies, the present invention designs a cross-travel platform supply and demand prediction method based on graph federated learning, which allows multiple shared travel platforms to aggregate the supply and demand characteristics of each party in different time and space regions under the data privacy requirement of "original data does not leave the domain, data is available but not visible", avoiding the "prediction bias" caused by being limited by the data of a single platform, and realizing high-efficiency and accurate prediction of global driver and passenger supply and demand.

[0007] A method for cross-travel platform supply and demand prediction based on graph federated learning. The specific steps are as follows:

[0008] Step 1: divide the data of m travel platforms into time-space supply and demand areas, and build a space-time graph federation model;

[0009] In the spatial dimension, each travel platform is divided into grid areas of different sizes according to its geographical location; the grid areas are used as nodes of the space-time graph, and the connecting edges of different grid areas are constructed through the space-time learning method.

[0010] In the time dimension, the maximum sampling frequency of each travel platform is selected as the federated time window size, and the time window is processed as the historical supply and demand forecast feature of the sliding time window;

[0011] The maximum sampling frequency of the travel platform o is: l = max{l o},o∈O, where set O represents the set of travel platforms, lo represents the GPS sampling frequency of the capacity of travel platform o;

[0012] Finally, through time and space division, a local time and space diagram of supply and demand forecast of each travel platform is constructed Indicates that travel platform o is in time window t T The local space-time graph of each travel platform G o Together they form a spatiotemporal graph federation model G = {G1,…,G m}.

[0013] Step 2: Extract local spatiotemporal features of each travel platform to obtain spatial correlation matrix and temporal features;

[0014] In the time dimension, the time series model is used to process the time series features of each travel platform data in the local space-time graph, and the time window tT of the travel platform o is his The supply and demand characteristics within t are expressed as The time characteristics of m travel platforms are expressed as

[0015] The time series models include threshold recurrent neural networks, long short-term memory neural networks, etc.

[0016] In the spatial dimension, the local spatiotemporal graph federation association weights are learned based on historical data, E = [E1,…,E m ], where E o Represents the association of the grid area of ​​travel platform o.

[0017] Step 3: The central server uses the local spatiotemporal features of each travel platform to decouple the spatiotemporal prediction operators and uses them for local calculations on each travel platform.

[0018] The decoupling of the spatiotemporal prediction operator refers to the central server decoupling the distributed graph convolution operator into operators that can be executed independently by each platform, and then each travel platform completes the local calculation. The specific description is as follows:

[0019] Solve the optimization problem and obtain a quadratic function approximation to the ReLU function:

[0020] The optimization problem is:

[0021]

[0022] When p=1 is selected, based on the optimization objective of the L1 loss function, the approximate activation function of ReLU(x) is solved as: ReLU2(x)=0.867x 2 +0.499x+0.0395.

[0023] After using the ReLU2(x) activation function, the auxiliary spatiotemporal graph convolution operator A = ReLU2(E·E T )h t The calculation decoupling is as follows:

[0024] Transform the activation function ReLU2(x·y) into ReLU equivalently 2,2 (x) y 2 +ReLU 2,1 (x)y+ReLU 2,0 (x), where ReLU 2,k (x), k = 0, 1, 2 represents the kth term of the ReLU2(x) activation function. Therefore, the spatiotemporal graph convolution operator ReLU (EE T )h t The decoupling transformation is as follows:

[0025]

[0026] in, Represents the matrix E o The tensor product with itself, E o ·E p Represents the inner product of matrices.

[0027] After decoupling transformation, each travel platform only needs local calculation The internal correlation matrix tensor product is sufficient.

[0028] Step 4: Aggregate the local calculation results of each travel platform and input them into the global spatiotemporal graph federation model to perform global spatiotemporal supply and demand forecasting;

[0029] The various calculation results are aggregated through a safe aggregation operation based on random masks and input into the global model as acceptable historical features of the sliding window in the global spatiotemporal graph prediction to complete the global spatiotemporal supply and demand prediction.

[0030] Aggregation is completed based on the random mask secure summation method. The calculation method is as follows:

[0031] For m travel platforms, each platform holds a privacy value v o ; Secure summation over random masks allows parties to generate masks:

[0032]

[0033] Among them, sd o With sd o' are the random number seeds of platform o and o' respectively, is a pseudo-random generation function. Under this mask, the sum of the masked privacy values ​​of all parties is allowed to be equal to the sum of the original privacy values, that is,

[0034] Therefore, the quadratic and linear terms of ReLU2 in step 3 can both be securely summed using random masks to complete the calculation of the association matrix across travel platforms for time window t.

[0035] Then, As historical features, it is input into the global spatiotemporal graph federation model to predict the future time window from t+1 to t+T fut The travel supply and demand situation in different regions within the country.

[0036] In step 5, the central server updates the global space-time graph federation model based on the global space-time supply and demand forecast results, and distributes the global space-time supply and demand forecast and the new global space-time graph federation model to each travel platform. Each travel platform updates the local space-time graph prediction model and returns to step 1 to iterate the local calculation and global federation calculation until the supply and demand model of the space-time graph federation learning converges.

[0037] Through the loss function Update global model parameters Get a new global spatiotemporal graph federation model G θ′ .

[0038] Step six: predict the capacity supply and user demand of each travel platform in different regions based on the global prediction results and local historical data.

[0039] In summary, the advantages of the present invention are as follows:

[0040] (1) This method establishes a spatiotemporal graph federation as a model for cross-travel platform supply and demand forecasting. Different grid areas are divided spatially as nodes of the spatiotemporal graph, and a regional correlation matrix driven by historical supply and demand data is constructed through machine learning methods. In terms of time, a sliding window method is used to receive the historical characteristics and dynamic correlation matrix of each platform.

[0041] (2) This method designs a quadratic function approximate calculation form of the ReLU(x) activation function in the spatiotemporal graph convolution, and innovatively uses the L1 loss function as the optimization objective to measure the approximation effect, and finally obtains the quadratic approximate activation function ReLU2(x) when the mean is close to 0.

[0042] (3) With the assistance of the approximate activation function ReLU2(x), the binary function ReLU(xy) is computationally decoupled, allowing each platform to decouple the association matrix into a form that can be securely aggregated with a mask with low computational overhead, avoiding the high computational overhead of traditional homomorphic encryption methods and taking into account both privacy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a workflow diagram of the cross-travel platform supply and demand forecasting method based on graph federated learning of the present invention;

[0044] Figure 2 The experimental results of different methods for predicting the supply and demand of shared travel in the embodiments of the present invention are as follows;

[0045] Figure 3 This is the experimental result of the approximation error of the quadratic activation function used in the present invention. DETAILED DESCRIPTION

[0046] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0047] The spatiotemporal supply and demand forecasts in different regions can adopt the commonly used sliding time window mode, that is, each platform uses the past T his The historical data in the time window is used as input to predict the future T fut The supply and demand situation of a time window. Assuming the current time is t, the prediction task can be expressed as:

[0048]

[0049] Among them, G θrepresents the spatiotemporal graph model for supply and demand forecasting, and θ represents the learnable parameters of the model.

[0050] For cross-platform shared travel, the spatiotemporal features are dispersed among the platforms, that is, X = {X1,…,X m However, due to the constraints of data privacy, each platform cannot directly share its historical features outside the domain; therefore, it is necessary to adopt a privacy-preserving method to complete the training and reasoning of the spatiotemporal graph model while utilizing the data from all parties.

[0051] Taking into account the spatiotemporal data characteristics across shared travel platforms, the present invention designs a cross-travel platform supply and demand prediction method based on spatiotemporal graph federation that takes into account both computational efficiency and data privacy.

[0052] The implementation platform of the present invention is a supply and demand forecasting platform across shared travel. Here, we take the online car-hailing platform as an example to introduce the spatiotemporal forecasting methods for different regions. Figure 1 As shown in the figure, the central server is responsible for coordinating various travel platforms, operator decoupling, model aggregation and model updating, etc.; while the travel platform side mainly cooperates with the server to perform local spatiotemporal graph modeling and local spatiotemporal feature extraction. Each travel platform first divides the service area into different grid areas (such as hexagons, squares, etc.); then, the supply and demand data of different regions are calculated based on historical data, and the historical supply and demand in the region are used as prediction features; then, the relationship between different regions can be modeled into a graph federation, and cross-platform feature aggregation is performed under the coordination of the central server to achieve cross-platform spatiotemporal supply and demand prediction. It can be divided into the following steps:

[0053] Step 1: Space-time supply and demand area division:

[0054] Different from the method of constructing the road network (intersections, roads) into a time-space graph in most previous prediction technologies, the present invention adopts a new time-space graph federation construction method in time and space. Specifically:

[0055] In the spatial dimension, each travel platform loads historical capacity supply and user travel data for cluster analysis, and divides them into grid areas of different sizes (such as squares with a side length of 5 km, hexagonal grids, etc.), and calculates the historical supply and demand data of different areas accordingly. Then, different spatial areas can be used as nodes to build a federated space-time graph. The connecting edges of different grid areas are constructed through spatiotemporal learning methods. Since the place where users initiate travel in shared travel is often not on the road, using grids as the basic unit for supply and demand forecasting can better fit the supply and demand forecasting task of shared travel.

[0056] In the time dimension, in order to avoid the problem of "ghost cars" (part of the capacity may suddenly appear in a certain time window due to inappropriate sampling frequency) caused by inconsistent sampling frequencies of various platforms, the maximum sampling frequency of the travel platform is selected as the federation time window size, l = max{l o},o∈O, where set O represents the set of travel platforms, lo represents the GPS sampling frequency of the capacity of travel platform o;

[0057] Finally, different spatial regions are used as nodes of the graph, and time regions are used as node features of the graph to construct a spatiotemporal graph federation model. Local spatiotemporal graph of supply and demand forecasts for each travel platform Local spatiotemporal graph G of each travel platform o Together they constitute the spatiotemporal graph federation model G = {G1,…,G m}.

[0058] Step 2: Local spatiotemporal feature extraction:

[0059] In the time dimension, we use commonly used threshold recurrent neural networks or long short-term memory neural networks to process the temporal features in the local space-time graph and convert the time window tT of the travel platform o into his The supply and demand characteristics within t are expressed as The time characteristics of m travel platforms are expressed as

[0060] In the spatial dimension, the local linkage graph federation association weights are learned based on historical data, E = [E1,…,E m ], where E o Represents the association of the grid area of ​​travel platform o.

[0061] Step 3: Decoupling of spatiotemporal prediction operators:

[0062] The central server that assists each platform in supply and demand forecasting needs to jointly utilize the temporal and spatial features of the cross-platforms to complete the cross-platform supply and demand forecasting. It decouples the distributed graph convolution operators into operators that can be executed separately by each platform; then, each shared travel platform completes the local calculation.

[0063] In order to realize the cross-platform spatiotemporal supply and demand forecasting, it is necessary to first calculate the cross-platform spatiotemporal correlation matrix (that is, the impact of travel platform o on another travel platform p), which relies on the activation operator ReLU (EE T )h t However, platform o only holds its own spatial incidence matrix E o With time characteristics In order to solve the above problems, traditional methods usually use the same set of homomorphic encryption public keys to encrypt each E o and Encryption requires two-to-one communication between platforms However, this method has a huge computational overhead, and the time required for prediction exceeds the time window size, making it impossible to apply in practice.

[0064] In response to the above problems, the present invention proposes an activation operator approximation method based on quadratic functions: each travel platform calculates the ReLU activation function under the approximation of the quadratic function, and with the assistance of the ReLU activation function, decouples the intermediate results of the cross-domain correlation matrix required by each platform; then, a random number seed that can generate a mask is generated, and a random mask is added to the local intermediate results under the protocol to prepare for model aggregation.

[0065] By solving the optimization problem, a quadratic function approximation to the ReLU function is obtained.

[0066] The optimization problem is:

[0067]

[0068] Different from the L2 loss function in the traditional method, the present invention innovatively selects an optimization target based on the L1 loss function (ie, p=1), which can obtain a better approximation effect when the mean value of the calculated value x is near 0.

[0069] By solving the above optimization objectives, we can get the approximate activation function of ReLU(x) ReLU2(x) = 0.867x 2 +0.499x+0.0395.

[0070] After using the ReLU2(x) activation function, the spatiotemporal graph convolution operator A=ReLU2(E·E T )h t Decouple the calculation. Transform ReLU2(x·y) equivalently to ReLU 2,2 (x) y 2 +ReLU 2,1 (x)y+ReLU 2,0 (x), where ReLU2,k(x) represents the kth term of the ReLU2(x) activation function. Therefore, the spatiotemporal graph convolution operator ReLU(EE T )h t It can be decoupled and transformed into the following form:

[0071]

[0072] in, Represents the matrix E o The tensor product with itself, E o ·E p Represents the inner product of matrices.

[0073] After decoupling transformation, each shared travel platform only needs to calculate locally The internal correlation matrix tensor product is sufficient.

[0074] Step 4: Aggregate the spatiotemporal prediction results. Aggregate the intermediate results of the cross-domain association matrix of each travel platform to obtain the spatiotemporal association matrix under global supply and demand, and complete the cross-platform supply and demand prediction based on this matrix.

[0075] After each shared travel platform executes the decoupled operators locally, it aggregates the calculation results through a safe aggregation operation based on random masks, and inputs them into the global model as acceptable historical features of the sliding window in the global spatiotemporal graph prediction to complete the global spatiotemporal supply and demand prediction.

[0076] The aggregation of spatiotemporal prediction results is completed based on the random mask secure summation method. The calculation method is as follows:

[0077] For m travel platforms, each platform holds a privacy value v o ; Secure summation over random masks allows parties to generate masks:

[0078]

[0079] Among them, sd o With sd o' are the random number seeds of platform o and o' respectively, is a pseudo-random generation function. Under this mask, the sum of the masked privacy values ​​of all parties is allowed to be equal to the sum of the original privacy values, that is,

[0080] Therefore, the quadratic and linear terms of ReLU2 in step 3 can both be securely summed using random masks to complete the calculation of the association matrix across travel platforms for time window t.

[0081] Then, you can As historical features, it is input into the global spatiotemporal graph federation model to predict the future time window from t+1 to t+T fut Travel supply and demand forecasts for different regions within the country.

[0082] Step 5: Federated Model Update Aggregation:

[0083] The accuracy of the prediction results is calculated based on the root mean square error. Where |G o | represents the spatial area of ​​each platform, y g and They represent the real supply and demand and the predicted supply and demand of the spatial region g in the platform respectively;

[0084] Through the loss function Update global model parameters To obtain a new global spatiotemporal graph federation model G θ′ The global spatiotemporal supply and demand forecast results and the new spatiotemporal graph federation model are distributed to each platform, and each platform updates the local spatiotemporal graph forecast model based on the federation model.

[0085] Finally, repeat steps 1 to 5, and iterate local computing and global federated computing until the supply and demand model of spatiotemporal graph federated learning converges.

[0086] This embodiment is verified on a real data set of four taxi companies in Beijing. Figure 2 As shown, it can be seen that after using this method, the cross-platform supply and demand prediction error is significantly lower than the single-platform prediction, and is close to the ideal prediction error when ignoring data privacy constraints and allowing data to be directly integrated out of the domain. Figure 3 This is the approximate effect diagram of the proposed ReLU2(x) function. It can be seen that it better approximates the ReLU(x) activation function in the range of [-0.5, 0.5].

Claims

1. A cross-travel platform supply and demand forecasting method based on graph federated learning, characterized in that: The specific steps are as follows: Step 1: divide the data of m travel platforms into time-space supply and demand areas, and build a space-time graph federation model; In the spatial dimension, each travel platform is divided into grid areas of different sizes according to its geographical location. The grid areas are used as nodes of the space-time graph, and the connection edges between different grid areas are constructed through the space-time learning method. In the time dimension, the maximum sampling frequency of each travel platform is selected as the federated time window size, and the time window is processed as the historical supply and demand forecast feature of the sliding time window; Finally, through time and space division, a local time and space diagram of supply and demand forecast of each travel platform is constructed Indicates that travel platform o is in time window t T The local space-time graph of each travel platform G o Together they form a spatiotemporal graph federation model G = {G1,…,G m }; Step 2: Extract local spatiotemporal features of each travel platform to obtain spatial correlation matrix and temporal features; In the time dimension, the time series model is used to process the time series features of each travel platform data in the local space-time graph, and the time window tT of the travel platform o is his The supply and demand characteristics within t are expressed as The time characteristics of m travel platforms are expressed as In the spatial dimension, the local spatiotemporal graph federation association weights are learned based on historical data, E = [E1,…,E m ], where E o Represents the relevance of the grid area of ​​the travel platform o; Step 3: The central server uses the local spatiotemporal features of each travel platform to decouple the spatiotemporal prediction operators and uses them for local calculations on each travel platform. The decoupling of the spatiotemporal prediction operator refers to the central server decoupling the distributed graph convolution operator into operators that can be executed independently by each platform, and then each travel platform completes the local calculation; Solve the optimization problem and obtain a quadratic function approximation to the ReLU function: The optimization problem is: When p=1 is selected, based on the optimization objective of the L1 loss function, the approximate activation function of ReLU(x) is solved as: ReLU2(x)=0.867x 2 +0.499x+0.0395; After using the ReLU2(x) activation function, the auxiliary spatiotemporal graph convolution operator A = ReLU2(E·E T )h t The calculation decoupling is as follows: Transform the activation function ReLU2(x·y) into ReLU equivalently 2,2 (x) y 2 +ReLU 2,1 (x)y+ReLU 2,0 (x), where ReLU 2,k (x), k = 0, 1, 2 represents the kth term of the ReLU2(x) activation function; therefore, the spatiotemporal graph convolution operator ReLU (EE T )h t The decoupling transformation is as follows: in, Represents the matrix E o The tensor product with itself, E o ·E p represents the inner product of a matrix; After decoupling transformation, each travel platform only needs local calculation The internal correlation matrix tensor product is sufficient; Step 4: Aggregate the local calculation results of each travel platform and input them into the global spatiotemporal graph federation model to perform global spatiotemporal supply and demand forecasting; The various calculation results are aggregated through a safe aggregation operation based on random masks and input into the global model as acceptable historical features of the sliding window in the global spatiotemporal graph prediction to complete the global spatiotemporal supply and demand prediction; The quadratic and linear terms of ReLU2 in step 3 can be safely summed using random masks to complete the calculation of the correlation matrix for time window t across travel platforms; Then, As historical features, it is input into the global spatiotemporal graph federation model to predict the future time window from t+1 to t+T fut The travel supply and demand in different regions within the country; Step 5: The central server updates the global spatiotemporal graph federation model based on the global spatiotemporal supply and demand forecast results, and distributes the global spatiotemporal supply and demand forecast and the new global spatiotemporal graph federation model to each travel platform. Each travel platform updates the local spatiotemporal graph forecast model and returns to step 1 to iterate the local calculation and global federation calculation until the supply and demand model of the spatiotemporal graph federation learning converges. Step six: predict the capacity supply and user demand of each travel platform in different regions based on the global prediction results and local historical data.

2. According to claim 1, a method for predicting supply and demand across travel platforms based on graph federated learning is characterized in that: The maximum sampling frequency of the travel platform o is: l = max{l o },o∈O, where set O represents the set of travel platforms, l o Indicates the GPS sampling frequency of the transportation capacity of travel platform o.

3. According to claim 1, a method for predicting supply and demand across travel platforms based on graph federated learning is characterized in that: Aggregation is completed based on the random mask secure summation method. The calculation method is as follows: For m travel platforms, each platform holds a privacy value v o ; Secure summation over random masks allows parties to generate masks: Among them, sd o With sd o' are the random number seeds of platform o and o' respectively, is a pseudo-random generation function; under this mask, the sum of the masked privacy values ​​of all parties is allowed to be equal to the sum of the original privacy values, that is, 4. According to claim 1, a method for predicting supply and demand across travel platforms based on graph federated learning is characterized in that: The specific updating of the global spatiotemporal graph federation model is as follows: Through the loss function Update global model parameters Get a new global spatiotemporal graph federation model G θ′ .

Images