Parking flow prediction model training method based on federated learning
By calculating model similarity and weighted average method verification in parking traffic prediction model, malicious model attack problems in data sharing among multiple parking lots are solved, model accuracy and generalization capabilities are improved, and data privacy and security are ensured.
Patent Information
- Application Number
- CN202111499469.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2021-05-13
- Filing Date
- 2021-12-09
- Publication Date
- 2025-01-17
- Estimated Expiration
- 2041-12-09
AI Technical Summary
The existing parking traffic prediction model has data privacy issues and malicious model attacks in data sharing between multiple parking lots, resulting in low model accuracy and slow convergence speed.
By calculating the similarity between local models and the similarity between local models and the global models, the improved cosine similarity and weighted average method are used to verify and aggregate model parameters to prevent malicious model attacks and perform data transmission under homomorphic encryption. Ensure data security.
It improves the generalization ability and accuracy of the model, reduces the accuracy deviation of the global model, prevents malicious model attacks, and ensures data privacy and security.
Smart Images
Figure CN114372046B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer application technology, and in particular to a parking flow prediction model training method based on federated learning. Background Art
[0002] With the development of computer technology, more and more technologies are applied in the field of parking lots. Users will have certain periodic behaviors when parking. Mining the parking behavior patterns of users and predicting the parking flow of users in each period can better manage parking space resources. In the existing technology, due to the privacy issues of user parking data, the parking flow prediction model mainly uses the historical parking flow data of a single parking lot to train the long short-term memory artificial neural network, and adjusts the parameters of the long short-term memory artificial neural network by increasing the number of training times and adjusting the training parameters to improve the accuracy of the parking flow prediction model. However, this parking flow model can only meet the parking flow prediction needs of a single parking lot, and cannot meet the parking flow prediction needs of multiple parking lots.
[0003] In order to improve the generalization ability of the parking flow prediction model for different parking lots, a large amount of high-quality parking lot data is needed to train the parking flow prediction model. However, since parking data involves user privacy, it is impossible to obtain data from multiple parking lots for centralized model training, which makes the traditional model training method face great challenges. Federated learning, as a core technology of artificial intelligence, is an effective way to solve this challenge. This method allows local participants to train local models locally with data, upload model parameters to the aggregation server for model parameter aggregation, and then send them to local participants to update the local model and continue training. The data does not leave the local training and optimization model. However, federated learning is vulnerable to malicious model poisoning attacks. When local participants have malicious behavior, the model parameters they upload are often false. When the model is aggregated, the accuracy of the aggregated global model will be reduced. In the prior art, the cosine similarity between the local model and the global model can be calculated to determine whether the local model is a malicious model and discard it. However, judging the malicious model only from the similarity between the local model and the global model is too one-sided. It does not take into account that when there is a large deviation in the global model, the normal local model will have a low similarity with the global model, so the normal local model will be discarded, resulting in slow convergence of the aggregated global model and low model accuracy, which puts higher requirements on the technology. Summary of the invention
[0004] In view of the fact that the federated learning method is easily attacked maliciously in the prediction of parking flow in parking lots, and the existing method for judging malicious models has the disadvantage of easily abandoning normal local models, resulting in the above-mentioned problems of slow convergence speed of the aggregated global model and low model accuracy, the present invention provides a parking flow prediction model training method based on federated learning, which reduces the accuracy deviation of the global model and improves the performance of the model by calculating the similarity values between local models and the similarity values between the local model and the global model under the condition of ensuring data privacy and security.
[0005] The present invention is achieved through the following technical solutions:
[0006] A parking flow prediction model training method based on federated learning, the method comprising the following steps:
[0007] Step S101, collecting historical parking data of a parking lot and preprocessing the historical parking data of the parking lot;
[0008] Step S102, counting the number of vehicles arriving in each period of the preprocessed historical parking data to construct parking flow time series data;
[0009] Step S103, normalizing the parking flow time series data;
[0010] Step S104: performing federated learning based on the normalized parking flow time series data, with each participant using local data to train a local parking flow prediction model, and encrypting the trained local model parameters and uploading them to the aggregation server;
[0011] Step S105: the aggregation server verifies all received local model parameters, decrypts and verifies all local model parameters. The local model parameters are verified by comparing the similarity values between the local model parameters and the local model parameters with the global model parameters. The similarity values are obtained based on the improved cosine similarity calculation. For n-dimensional vectors A and B, the cosine similarity calculation formula between the two is: Assume that the model parameters have a total of p layers, each layer has a dimension of m×n, and calculate two model parameters w 1 and w 2 Similarity value in, is the model parameter w 1 The parameter at the position of row j and column k in the i-th layer, is the model parameter w 2The parameters at the position of the kth column in the jth row of the i-th layer are similar if the cosine similarity of the two local models is greater than 0.5, otherwise they are dissimilar; the dissimilar local model parameters are discarded, and all similar local model parameters are aggregated based on the weighted average method to obtain the global model parameters and establish the global model;
[0012] Step S106, determining whether the global model has converged. If not, sending the global model parameters to each participant to continue iterative training until the global model converges;
[0013] Step S107: using the converged global model as a parking flow prediction model.
[0014] Furthermore, the pre-processing in step S101 includes: cleaning the historical parking data of the parking lot, and removing sample data with abnormal parking time and missing data.
[0015] Furthermore, the step S102 includes: counting the number of arrivals in each time period for the pre-processed historical parking data, and constructing parking flow time series data in chronological order.
[0016] Furthermore, the normalization method of step S103 is: obtaining the maximum and minimum values in the parking flow time series data; normalizing the parking flow time series data of each period between 0 and 1, and the normalization formula is: Where x' is the normalized value, x is the actual value of parking flow, and x min is the minimum value in the parking flow time series data, x max It is the maximum value in the parking flow time series data.
[0017] Furthermore, the encryption method in step S104 adopts a homomorphic encryption algorithm, and each participant and the aggregation server have the same public and private keys. By applying the homomorphic encryption algorithm, even if others obtain the encrypted model parameters, they cannot know any original data information due to the lack of key information, which further ensures the security of the data.
[0018] Furthermore, the process of aggregating all similar local model parameters in step S105 is as follows: in the first round of federated learning, the local model parameter with the largest number of similarities to other local model parameters is selected as the benchmark model parameter. If there are multiple local model parameters with the largest number, the Euclidean distances of these local model parameters and the local model parameters similar to them are calculated respectively, and the local model parameter with the smallest Euclidean distance to other local model parameters is selected as the benchmark model parameter. The calculation formula for the Euclidean distance of the model parameters in this round is: The baseline model parameters and other local model parameters similar to them are used as local model parameters for aggregation in the first round; before aggregating model parameters from the second round of federated learning, the global model parameters of the previous round are first used as the first baseline model parameters, and all local models similar to the first baseline model parameters are selected. Then, from these local model parameters, the local model parameters with the largest number of similarities to other local models are selected as the second baseline model parameters. Similarly, if there are multiple local model parameters with the largest number, the second baseline model parameters are selected by calculating the Euclidean distance, and all local models similar to the second baseline model parameters are selected for model parameter aggregation in this round. The weighted average method formula for aggregating local model parameters is: Among them, w is the global model parameter after aggregation, w i is the local model parameter of participant i, p i is the model parameter w i The weight value, n i is the number of local training data of participant i, and n is the total number of local training data of all participants.
[0019] Furthermore, the specific process of step S106 is: testing the established global model, calculating the prediction error value through test data, and then judging whether the global model has converged. If not, the aggregation server encrypts the global model parameters and sends them to each participant. Each participant decrypts the global model parameters and updates the local model parameters, and continues iterative training until the global model converges.
[0020] The present invention provides a parking flow prediction model training method based on federated learning. On the one hand, the parking flow prediction model is trained under the condition of ensuring user data privacy and security. On the other hand, model poisoning attacks during model training are prevented. The trained parking flow prediction model has good generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 This is a flowchart of the parking flow prediction model training based on federated learning of the present invention;
[0022] Figure 2 It is a local model verification process in the parking flow prediction model training method based on federated learning of the present invention;
[0023] Figure 3 This is a diagram of the parking flow prediction model training architecture based on federated learning in an embodiment;
[0024] Figure 4 The prediction result of parking lot 1 in the embodiment;
[0025] Figure 5 The prediction result of parking lot 2 in the embodiment;
[0026] Figure 6 The prediction result of parking lot 3 in the embodiment;
[0027] Figure 7 It is a graph showing the error variation trend during model training of the existing method and the method of the present invention. DETAILED DESCRIPTION
[0028] The present invention is further described in detail below with reference to specific embodiments and accompanying drawings.
[0029] Example
[0030] The present invention selects the historical parking data of three parking lots as an example, wherein each parking lot is a participant. Figure 3 As shown, there are three participants and one aggregation server. The specific process is as follows:
[0031] First, each participant uses the local parking flow time series data to perform 100 rounds of local model training, then extracts the local model parameters and sends them to the aggregation server through homomorphic encryption. The aggregation server decrypts the local model parameters received from all participants, and aggregates the local model parameters by weighted average according to the amount of local parking flow time series data of each participant. Finally, the global model parameters of this round of federated learning are obtained, and the global model is established and the convergence of the global model is tested through the test data set. If the global model does not converge, the global model parameters are encrypted and sent to each participant. Each participant decrypts the global model parameters and updates the local model parameters, and continues to use the local parking flow time series data for 100 rounds of local model training. The above process is repeated until the global model converges. Finally, the converged global model is used as the final parking flow prediction model.
[0032] The specific method of this embodiment is as follows:
[0033] Step 1: Select the historical parking data of three parking lots for four weeks respectively, where parking lot 1 is the all-day historical parking data from 2020.8.24 to 2020.9.18, parking lot 2 is the all-day historical parking data from 2020.10.5 to 2020.10.31, and parking lot 3 is the all-day historical parking data from 2020.8.3 to 2020.8.29. The original field information of the parking lot historical data is shown in Table 1.
[0034] Table 1 Original field information of parking lot historical data
[0035] Data Fields Specific Information User Information User type, license plate number, fee type, account balance Vehicle entry and exit information Entry time, exit time, parking duration Parking fee information Payment amount (RMB), exemption amount (RMB), fee type, fee method Operational Information Operator
[0036] Step 2: In the embodiment, the vehicle data selection time period is 0:00-24:00, the license plate number, entry time and exit time in the original fields of the parking lot historical data are retained, and the number of vehicles arriving in each time period is counted. For example, the number of vehicles arriving in each time period on August 24, 2020 for parking lot 1 is shown in Table 2.
[0037] Table 2 Number of vehicles arriving in each period on August 24, 2020
[0038] time Number of vehicles arriving time Number of vehicles arriving 2020 / 8 / 24 0:00 0 2020 / 8 / 24 12:00 8 2020 / 8 / 24 1:00 0 2020 / 8 / 24 13:00 17 2020 / 8 / 24 2:00 0 2020 / 8 / 24 14:00 8 2020 / 8 / 24 3:00 0 2020 / 8 / 24 15:00 9 2020 / 8 / 24 4:00 1 2020 / 8 / 24 16:00 9 2020 / 8 / 24 5:00 1 2020 / 8 / 24 17:00 2 2020 / 8 / 24 6:00 16 2020 / 8 / 24 18:00 0 2020 / 8 / 24 7:00 90 2020 / 8 / 24 19:00 4 2020 / 8 / 24 8:00 47 2020 / 8 / 24 20:00 0 2020 / 8 / 24 9:00 10 2020 / 8 / 24 21:00 1 2020 / 8 / 24 10:00 8 2020 / 8 / 24 22:00 1 2020 / 8 / 24 11:00 12 2020 / 8 / 24 23:00 0
[0039] Step 3: Construct time series data based on the number of vehicles arriving in each period, with a total of 624 data, and perform normalization. The data from the first three weeks are used as the historical data set, and the data from the last week are used as the test data set. The historical data set is used to train the local model, and the test data set is used to test the performance of the global model.
[0040] Step 4: Conduct the federated learning model training process. Each participant uses the historical data set of the previous three weeks and the long short-term memory artificial neural network to conduct 100 rounds of model training locally. After the training, the model parameters are obtained from the local model and encrypted through the homomorphic encryption algorithm. Each participant and the aggregation server have the same public and private keys, and then sent to the aggregation server through the network; the aggregation server decrypts and verifies all received local model parameters. The verification of local model parameters is to prevent malicious participants from providing false model parameters to attack and destroy the training process of federated learning, affecting the training quality of the overall model and causing training failure. The verification method is to compare the similarity values between local model parameters and between local model parameters and global model parameters. The similarity values are obtained based on the improved cosine similarity calculation. For n-dimensional vectors A and B, the cosine similarity calculation formula between the two is Assume that the model parameters have a total of p layers, each layer has a dimension of m×n, and calculate two model parameters w 1 and w 2 Similarity value in, is the model parameter w 1 The parameter at the position of row j and column k in the i-th layer, is the model parameter w 2 The parameters at the kth column of the jth row of the i-th layer are similar if the cosine similarity of the two local models is greater than 0.5, otherwise they are dissimilar. The dissimilar local model parameters are discarded, and all similar local model parameters are aggregated based on the weighted average method to obtain the global model parameters and establish the global model. The specific weighted average formula is: Where w is the global model parameter after aggregation, w i are the local model parameters of the verified participant i.
[0041] Step 5: The aggregation server verifies all received local model parameters, decrypts and verifies all local model parameters. The verification method of local model parameters is to compare the similarity values between local model parameters and between local model parameters and global model parameters. The similarity values are obtained based on the improved cosine similarity calculation. For n-dimensional vectors A and B, the cosine similarity calculation formula between the two is Assume that the model parameters have a total of p layers, each layer has a dimension of m×n, and calculate two model parameters w 1 and w 2 Similarity value in, is the model parameter w 1 The parameter at the position of row j and column k in the i-th layer, is the model parameter w 2 The parameters at the position of the kth column in the jth row of the i-th layer are similar if the cosine similarity of the two local models is greater than 0.5, otherwise they are dissimilar; the dissimilar local model parameters are discarded, and all similar local model parameters are aggregated based on the weighted average method to obtain the global model parameters and establish the global model;
[0042] The process of aggregating all local model parameters is as follows: In the first round of federated learning, the local model parameter with the largest number of similarities to other local model parameters is selected as the benchmark model parameter. If there are multiple local model parameters with the largest number, the Euclidean distances between these local model parameters and their similar local model parameters are calculated respectively, and the local model parameter with the smallest Euclidean distance to other local model parameters is selected as the benchmark model parameter. The calculation formula for the Euclidean distance of the model parameters in this round is: The baseline model parameters and other local model parameters similar to them are used as local model parameters for aggregation in the first round; before aggregating model parameters from the second round of federated learning, the global model parameters of the previous round are first used as the first baseline model parameters, and all local models similar to the first baseline model parameters are selected. Then, from these local model parameters, the local model parameters with the largest number of similarities to other local models are selected as the second baseline model parameters. Similarly, if there are multiple local model parameters with the largest number, the second baseline model parameters are selected by calculating the Euclidean distance, and all local models similar to the second baseline model parameters are selected for model parameter aggregation in this round. The weighted average method formula for aggregating local model parameters is: Among them, w is the global model parameter after aggregation, w i is the local model parameter of participant i, p i is the model parameter w i The weight value, n i is the number of local training data of participant i, and n is the total number of local training data of all participants.
[0043] Step 6: Test the established global model, calculate the prediction error value through test data, and then determine whether the global model has converged. If not, the aggregation server encrypts the global model parameters and sends them to each participant. Each participant decrypts the global model parameters and updates the local model parameters, and continues iterative training until the global model converges.
[0044] Step 7: Use the converged global model as the final parking flow prediction model.
[0045] The parking flow prediction model is tested using the test data set of each parking lot. The prediction results for parking lot 1 are as follows: Figure 4 As shown in Figure 2, the prediction results for parking lot 2 are as follows: Figure 5 As shown, the results for parking lot 3 are as follows Figure 6 As shown in the figure, it can be seen that the parking flow prediction model has a good prediction performance for each parking lot. The root mean square error between the prediction result and the true value of each parking lot is 5.28895, 4.10479, and 2.11413, respectively. Therefore, the obtained parking flow prediction model has a strong generalization ability. Figure 7 This is the error change during model training between the existing method and the proposed method. It can be seen from the figure that the model error of the proposed method decreases faster than the existing method, and it also verifies that the proposed method improves the convergence speed of the model.
[0046] The embodiments described above are only descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the protection scope determined by the claims of the present invention.
Claims
1. A parking flow prediction model training method based on federated learning, characterized in that: The method comprises the following steps: Step S101, collecting historical parking data of a parking lot and preprocessing the historical parking data of the parking lot; Step S102, counting the number of vehicles arriving in each period of the preprocessed historical parking data to construct parking flow time series data; Step S103, normalizing the parking flow time series data; Step S104: performing federated learning based on the normalized parking flow time series data, with each participant using local data to train a local parking flow prediction model, and encrypting the trained local model parameters and uploading them to the aggregation server; Step S105: The aggregation server verifies all received local model parameters, decrypts and verifies all local model parameters. The local model parameters are verified by comparing the similarity values between local model parameters and between local model parameters and global model parameters. The similarity values are obtained based on the improved cosine similarity calculation. For n-dimensional vectors A and B, the cosine similarity calculation formula between the two is: Assume that there are p layers of model parameters, each layer has a dimension of m×N, and calculate two model parameters w 1 and w 2 Similarity value in, is the model parameter w 1 The parameter at the position of row j and column k in the i-th layer, is the model parameter w 2 The parameters at the position of the kth column in the jth row of the i-th layer are similar if the cosine similarity of the two local models is greater than 0.5, otherwise they are dissimilar; the dissimilar local model parameters are discarded, and all similar local model parameters are aggregated based on the weighted average method to obtain the global model parameters and establish the global model; Step S106, determining whether the global model has converged. If not, sending the global model parameters to each participant to continue iterative training until the global model converges; Step S107: using the converged global model as a parking flow prediction model.
2. The parking flow prediction model training method based on federated learning according to claim 1 is characterized in that: The pre-processing in step S101 includes: cleaning the historical parking data of the parking lot and removing sample data with abnormal parking time and missing data.
3. The parking flow prediction model training method based on federated learning according to claim 1 is characterized in that: The step S102 includes: counting the number of arrivals in each time period for the pre-processed historical parking data, and constructing parking flow time series data in chronological order.
4. The parking flow prediction model training method based on federated learning according to claim 1 is characterized in that: The method of normalization processing in step S103 is: obtaining the maximum value and the minimum value in the parking flow time series data; The parking flow time series data of each period is normalized between 0 and 1. The normalization formula is: Where x' is the normalized value, x is the actual value of parking flow, and x min is the minimum value in the parking flow time series data, x max It is the maximum value in the parking flow time series data.
5. The parking flow prediction model training method based on federated learning according to claim 1 is characterized in that: The encryption method in step S104 adopts a homomorphic encryption algorithm, and each participant and the aggregation server have the same public and private keys.
6. The parking flow prediction model training method based on federated learning according to claim 1 is characterized in that: The process of aggregating all similar local model parameters in step S105 is as follows: in the first round of federated learning, the local model parameter with the largest number of similarities to other local model parameters is selected as the benchmark model parameter. If there are multiple local model parameters with the largest number, the Euclidean distances between these local model parameters and the local model parameters similar to them are calculated respectively, and the local model parameter with the smallest Euclidean distance to the other local model parameters is selected as the benchmark model parameter. The calculation formula for the Euclidean distance of the model parameters in this round is: The baseline model parameters and other local model parameters similar to them are used as local model parameters for aggregation in the first round; before aggregating model parameters from the second round of federated learning, the global model parameters of the previous round are first used as the first baseline model parameters, and all local models similar to the first baseline model parameters are selected. Then, from these local model parameters, the local model parameters with the largest number of similarities to other local models are selected as the second baseline model parameters. Similarly, if there are multiple local model parameters with the largest number, the second baseline model parameters are selected by calculating the Euclidean distance, and all local models similar to the second baseline model parameters are selected for model parameter aggregation in this round. The weighted average method formula for aggregating local model parameters is: Among them, w is the global model parameter after aggregation, w i is the local model parameter of participant i, p i is the model parameter w i The weight value, n i is the number of local training data of participant i, and n is the total number of local training data of all participants.
7. The parking flow prediction model training method based on federated learning according to claim 1 is characterized in that: The specific process of step S106 is: testing the established global model, calculating the prediction error value through test data, and then judging whether the global model has converged. If not, the aggregation server encrypts the global model parameters and sends them to each participant. Each participant decrypts the global model parameters and updates the local model parameters, and continues iterative training until the global model converges.