Federal learning efficient communication method and system for traffic flow prediction

Abstract

The invention relates to a federated learning efficient communication method and system for traffic flow prediction, and belongs to the field of traffic flow prediction. And each terminal compresses the collected streaming trajectory data, screens out key trajectory fragments and important time points, and generates a compressed trajectory data set. And deploying a small sparse LSTM model to carry out local model training to obtain local model update. And each terminal uploads a local model update to the cloud, and the cloud executes global aggregation and then adaptively removes low-weight connection from the model and dynamically evolves the model structure in combination with model update parameter importance evaluation, and issues the evolved model to the terminal. And repeating the steps until the global model converges. And each terminal predicts the real-time traffic flow data by using the convergent global model, and outputs the traffic flow change in the future time period, thereby realizing the dynamic distributed prediction of the traffic flow. According to the method, the problem of high communication cost of federal learning is solved, and the real-time response capability is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of traffic flow prediction, and relates to a federated learning efficient communication method and system for traffic flow prediction. BACKGROUND

[0002] Precise and efficient traffic flow prediction is of great significance for relieving traffic congestion, improving travel efficiency, and assisting management departments in planning traffic routes. At present, researchers attempt to use two ways of model-based and data-driven to realize traffic flow prediction. With the enhancement of massive data collection technology, the method of machine learning (ML) has become the mainstream method for current traffic flow prediction. This method mainly adopts a centralized cloud-based way to train the prediction model, which requires terminals (such as mobile phones, cameras, radars, etc.) to upload all the sensor data captured by themselves to the cloud, and the cloud completes the model training and inference.

[0003] Although this way can train a high-precision training model, it still faces the disadvantages of high transmission delay, high network bandwidth pressure, and privacy leakage. Federated learning (FL) allows multiple distributed edge devices to collaboratively complete the training of a global model under the unified coordination of a cloud server without transmitting the original data collected by themselves. The federated learning method effectively protects the privacy information of users while not affecting the training of available models. However, the communication efficiency problem has always been an important bottleneck restricting the implementation of federated learning. Specifically, most current traffic flow prediction scenarios based on federated learning usually include a large number of terminals with limited computing and network bandwidth resources. In order to obtain a traffic flow prediction model with satisfactory accuracy for users, the terminal needs to interact with the cloud through multiple rounds of communication. Traffic flow prediction models are mostly trained using complex network models (such as LSTM), and each round of communication of the model update parameters usually contains millions of sub-update parameters. Therefore, directly transmitting the high-dimensional gradient obtained by the terminal through each round of communication to the cloud through limited network bandwidth resources will consume huge communication overhead. Moreover, in the intelligent transportation system, data collection nodes (such as RSU) have unstable network conditions (such as WIFI), which even become an important bottleneck affecting the training efficiency and performance of the model.

[0004] Most of the existing researches reduce the transmission bit number of each round of communication by selecting part of the local original update parameters or quantized updates from the model compression (such as model quantization, model sparsification) to the cloud. However, these methods only consider removing part of the redundant update parameters from the perspective of model update parameters to reduce the communication volume, they do not essentially analyze the reason for the generation of redundant update parameters, so they do not fundamentally improve the communication efficiency of federated learning. In addition, most of the existing researches are for static image data scenarios and do not consider dynamic streaming traffic data scenarios. In fact, redundant data may be an essential reason for high communication cost. Specifically, traffic data has significant redundancy, and the multi-source perception system performs high-frequency and dense sampling in time and space, which will lead to a large amount of repeated or similar information being recorded. For example, the trajectory of the same vehicle may be repeatedly collected by multiple sensors, or almost unchanged positioning data is continuously generated when the vehicle is stationary, thereby introducing a large amount of redundant content. Therefore, in combination with the streaming and redundant characteristics in the traffic flow prediction scenario, it is a key problem to be solved in the current federated learning efficient communication for traffic flow prediction to propose an actual communication efficient algorithm to reduce the communication overhead required for training while ensuring the accuracy of the prediction model.

[0005] In summary, in the intelligent transportation system, in order to protect the terminal privacy information while ensuring the availability of the traffic flow prediction model, federated learning emerges as the times require. However, in the intelligent transportation system based on federated learning, the terminal network bandwidth resource is limited and the network condition is unstable, and the transmission of the high-dimensional gradient information obtained by training to the cloud in each round of iteration will consume huge communication overhead, and even become a bottleneck problem of model training. Most of the existing communication efficient methods use model compression or selection of training sub-updates to reduce the transmission of redundant gradients in the training process. However, they do not essentially analyze the essential reason for the generation of redundant gradients, and ignore the streaming and redundant characteristics of traffic data, which cannot adapt to large-scale real-time real traffic prediction scenarios. Therefore, designing a communication efficient method that meets the traffic flow prediction scenario to meet the real-time and accurate prediction of the traffic scenario is a key problem to be solved in the current intelligent transportation field. SUMMARY

[0006] Therefore, the purpose of the present application is to provide a federated learning efficient communication method and system for traffic flow prediction, which fully considers the redundancy of streaming trajectory data and the complexity of the model, and combines data compression and model pruning mechanism, effectively solves the communication bottleneck caused by high-dimensional update parameters in traditional federated learning in traffic flow prediction, improves its real-time response ability in real scenarios, and lays a foundation for solving the communication efficiency problem in real traffic flow prediction scenarios.

[0007] To achieve the above purpose, the technical scheme provided by the present application is as follows: A federated learning efficient communication method for traffic flow prediction, comprising the following steps: S1: Each terminal compresses the streaming trajectory data collected locally, filters out key trajectory segments and important time points, and generates a compressed trajectory data set; S2: Based on the compressed trajectory data set, each terminal deploys a small sparse LSTM model for local model training, obtains local model update parameters and uploads them to the cloud; S3: The cloud performs global aggregation to obtain a global model; S4: The cloud combines model update parameter importance evaluation, adaptively removes low weight connections from the global model, dynamically evolves the model structure, and distributes the evolved global model and corresponding model parameters to the terminals; S5: Repeat steps S2 to S4 until the global model converges; S6: Each terminal uses the converged global model to predict real-time traffic flow data and output traffic flow changes in the future period, achieving dynamic distributed prediction of traffic flow.

[0008] Further, in step S1, the compression includes original trajectory data entity compression and original trajectory data time interval compression; The original trajectory data entity compression is specifically: given a terminal set , a data set , is the trajectory sequence data collected by the terminal , where , and , represents the th trajectory data collected by the device , the trajectory is composed of a point sequence of time points, and each point represents dimensional feature space vector; for the data set , a Gaussian mixture model is used for modeling compression, the goal is to select a compressed subset from the entities of the data set , so that the compression error before and after compression is minimized, that is, the cost function before and after compression satisfies the following compression constraint, then the compression is considered acceptable;

[0009] where and respectively represent the cost functions before and after compression, which are constructed based on the log-likelihood function of the Gaussian mixture model, and are respectively defined as:

[0010]

[0011] in, Represents a compressed subset The Middle The weight of each trajectory data entity; For compression tolerance parameters, ; For the allowable error rate parameter, ; and These represent the number of Gaussian components in the Gaussian mixture model and the weight of each component, respectively. , ,and ; Let be the parameter space of the Gaussian mixture model. This represents the mean of the Gaussian components. and Let these represent the covariance matrix and the autocorrelation matrix, respectively. This represents a specific instance of a Gaussian mixture model. Representing data With the The distance function between Gaussian components; The original trajectory data time interval compression specifically involves compressing the trajectory data at the time interval level, that is, compressing it from the perspective of each device. Data Select from all time intervals A weighted compressed subset of data time intervals is obtained by considering several important time intervals. This ensures that the weighted time interval cost function of the datasets before and after compression satisfies the following constraints:

[0012] in, , , and These represent the weighted time interval costs before and after compression, respectively. Represents a set China Data In time interval The weighted weights; Representing data In time interval , the distance function between the Gaussian component; Each dataset is obtained by compressing the original trajectory data entities and the original trajectory data time intervals. Compressed dataset ,in The original trajectory data entity compression set and time interval compression set , that is .

[0013] Further, in step S1, the original trajectory data entity compression specifically includes the following steps: S101: Calculate the entity mean value of each data in the data set , as follows:

[0014] S102: According to the mean value calculated in step S101 , use the k-means clustering algorithm to divide into clusters, the goal is to minimize the distance between each data and the cluster center of any th cluster , to get the final cluster division result , that is, to satisfy the following calculation formula:

[0015] wherein represents a set consisting of the k cluster centers currently used in the k-means clustering algorithm; S103: For each data in the data set , count the cluster index number of the cluster to which it is divided with the smallest distance, that is:

[0016] S104: Calculate the influence of each data in the data set on all data in its cluster, that is, calculate the data entity intra-cluster sensitivity boundary , as follows:

[0017] wherein represents the number of data entities in cluster ; and calculate the sum of distances between each data in the data set and its cluster center point cluster , as follows:

[0018] calculate the sum of distances between each data​​​​ entity of each data in the entity , as follows:

[0019] S105: for each data in the data set , calculate the data entity intermediate sensitivity boundary when the parameter space of the Gaussian mixture model does not consider the influence of the covariance and autocorrelation function, that is, , as follows:

[0020] S106: for each data in the data set , according to the entity intermediate sensitivity boundary calculated in step S105, calculate the data entity sensitivity boundary , as follows:

[0021] wherein the parameter ; the constant , which represents the gap between the maximum eigenvalue of the covariance matrix and the minimum eigenvalue of the covariance matrix in all clusters . S107: for each data in the data set , according to the entity sensitivity boundary obtained in step S106, calculate the data entity sampling probability , as follows:

[0022] Calculate the average value of the entity sensitivity boundary of all data in the data set , as follows:

[0023] S108: according to the compression tolerance parameter and the allowed error rate of the device previously set, and the average value of the entity sensitivity boundary obtained in step S107 , calculate the size of the weighted data entity subset that meets the condition , as follows: ​

[0024] S109: Sampling probability obtained from step S107 From the original trajectory dataset The size of the sampled data entity set is Weighted entity subset ; S110: Data set obtained from step S107 Each data Average value of entity sensitivity boundary Calculate the set Each data Entity weight The formula is:

[0025] Repeat steps S101 to S110, set All devices in the middle are based on the sampling probability In parallel from their respective datasets The size of the sampled entity set is Weighted entity subset This yields a weighted subset of entities for all devices. .

[0026] Furthermore, in step S1, the compression of the original trajectory data time interval specifically includes the following steps: S201: Calculate the dataset Each data Each time interval data entities with data entity mean The total distance The formula is as follows:

[0027] S202: For the dataset Each data Calculate its division into clusters , Medium time interval Cluster The impact of all data in the cluster, i.e., the intra-cluster time sensitivity boundary. The formula is:

[0028] S203: For the dataset Each data Calculate the parameter space of the Gaussian mixture model. Medium time interval without considering the influence of covariance and autocorrelation function, i.e. the time intermediate sensitivity boundary , which is given by

[0029] S204: According to step S202 and step S203, the time sensitivity boundary of each data in the data set is calculated, which is given by the time sensitivity boundary of the time interval

[0030] wherein, denotes the distance function in the Gaussian mixture model parameter space without considering the influence of covariance and autocorrelation function; denotes the time lag step, i.e. the index of the time interval that is backtracked from the current time by q q S205: According to the time sensitivity boundary calculated in step S204, the sampling probability of each data in the data set with respect to the time interval is calculated, which is given by

[0031] According to the time sensitivity boundary calculated in step S204, the average value of the time sensitivity boundary is calculated, which is given by

[0032] S206: According to the compression tolerance parameter of the device , the allowed error rate parameter and the average value of the time sensitivity boundary obtained in step S205, the number of important time intervals that meet the condition size is calculated, which is given by

[0033] S207: According to the sampling probability obtained in step S205 and the number of time interval samplings obtained in step S206, the time interval ​​​​​​​​​​​​​​​Sampling a set of important time interval data ; S208: updating the dataset each piece of data time interval weight , the formula is as follows:

[0034] Repeat steps S201 to S208, and all devices in the set according to the sampling probability , sample from their local dataset the time interval an important weighted time interval subset , get the weighted time interval subset set of all devices .

[0035] Further, in step S2, the deployment of the small sparse LSTM model specifically includes: according to the model sparsification parameters , sparsifying the initialized LSTM model, and deploying the initialized sparse LSTM to each terminal, including the following specific steps: S201: according to the model training task, initialize a fully connected LSTM model containing input layer, hidden layer and output layer, the hidden layer has layers and the number of neurons in the first layer is ; S202: according to the set model sparsification parameters , calculate the sparsification connection probability of the neuron nodes in the layer and the neuron nodes in the layer of the fully connected LSTM model , the formula is as follows:

[0036] wherein, and respectively represent the number of neurons in the first layer and the first layer of the LSTM fully connected network model, denotes the connection probability of the neuron in the first layer and the neuron in the first layer; S203: according to the fully connected LSTM model initialized in step S201, initialize the model parameters ; S204: Based on the sparsification connection probability obtained in step S202 Calculate the mask matrix ; S205: Mask matrix obtained from S204 The initial LSTM sparse network model parameters were calculated. The formula is as follows:

[0037] S206: Apply the initial sparsification model parameters obtained in step S205 Distribute to each terminal.

[0038] Furthermore, in step S4, the dynamic evolution model structure specifically includes: cloud-based global model parameters. Each connection in Conduct an importance assessment to determine its importance. Meanwhile, based on the given initial model pruning rate... Calculate the current iteration round Model pruning rate and based on The structure of the global model is evolved, and the evolved model and its parameters are distributed to each terminal. This process includes the following steps: Step S401: Evaluate global model parameters Each connection Importance The formula is as follows:

[0039] in, express The first in Individual parameter updates express The absolute value; Step S402: The cloud prunes the model according to the given initial model pruning rate. Calculate the current iteration round Model pruning rate :

[0040] Among them, parameters The decay rate of the exponential function; Step S403: Cloud-based Evolution of the global model structure, i.e., from Remove from Unimportant network connections, get the evolved network model and the corresponding model parameters , and the global model structure and model parameters after evolution are distributed to each terminal, and the next round of iterative training begins; Repeat the above steps until round of iteration, the model training is completed, and the final cumulative communication transmission bit number and global model parameters are obtained.

[0041] Scheme 2: An efficient communication system for traffic flow prediction-oriented federated learning, comprising: A compression module, comprising an entity compression module, a time interval compression module, and an entity-time pair merging module; the entity compression module is used for each terminal to perform data entity compression from the respective data set in parallel, obtaining a weighted compressed entity subset , wherein , N is the number of terminals; the time interval compression module is used to select important time intervals from the data set of each terminal, obtaining a weighted data time interval set ; the entity-time pair merging module is used to merge the compressed data entity subset and the time interval subset of each terminal's original data set, obtaining a compressed data set of the original data set ; A sparse LSTM model deployment module is used to perform sparse operation on the LSTM model according to the model sparsification parameters , and deploy the initialized sparse LSTM model to each terminal; A global initialization module is used to initialize the cumulative communication transmission bit number , the number of global communication iterations , the training block size , the local training number , the learning rate , the pruning rate , and the exponential decay rate parameter ; A local model training module is used for each terminal to perform local model training according to the compressed data set , obtaining the local model parameters after rounds of local iterative training , and transmitting to the cloud; A model aggregation module is used to aggregate all the terminals uploaded in the current iteration round Local model update parameters This yields updated global model parameters and updates the current iteration. The cumulative number of communication transmission bits; The model structure evolution module is used to evolve the structure of the global model in the cloud, remove some unimportant model connections, and distribute the evolved global model and corresponding model parameters to each terminal. The traffic flow prediction module is used to predict the traffic flow data collected in real time using the final global model and corresponding model parameters received by the terminal, and to obtain the traffic flow prediction results for future time intervals.

[0042] Preferably, the entity compression module includes: The entity mean calculation submodule is used to calculate the dataset. Each data entity mean ; The clustering partitioning submodule is used to partition the dataset. Each data entity mean The k-means clustering algorithm was used to divide them into clusters with the smallest distance. l In this process, the final cluster partitioning result is obtained. , ; The index label retrieval submodule is used to retrieve the dataset. Each data It is divided into the cluster with the smallest distance. l index number ; The intra-cluster entity sensitivity boundary calculation submodule is used to calculate the dataset. Each data , divided into clusters Intra-cluster data entity sensitivity boundary ; The entity-cluster distance calculation submodule is used to calculate the dataset. Each data entity mean Its cluster Total distance from the center point ; The entity 2 norm calculation submodule is used to calculate the dataset. Each data 2-norm distance ; The distance function calculation submodule is used to calculate the parameter space of the Gaussian mixture model. Distance function at time The value; The entity intermediate sensitivity boundary calculation submodule is configured to calculate the intermediate sensitivity boundary of each data entity in the data set The intermediate sensitivity boundary of each data entity in the data set The intermediate sensitivity boundary of each data entity in the data set The intermediate sensitivity boundary of each data entity in the data set The intermediate sensitivity boundary of each data entity in the data set The feature value gap calculation submodule is configured to calculate the gap between the maximum eigenvalue and the minimum eigenvalue of the covariance matrix in the cluster of the maximum eigenvalue and the cluster of the minimum eigenvalue of the covariance matrix The feature value gap calculation submodule is configured to calculate the gap between the maximum eigenvalue and the minimum eigenvalue of the covariance matrix in the cluster of the maximum eigenvalue and the cluster of the minimum eigenvalue of the covariance matrix The entity sensitivity boundary calculation submodule is configured to calculate the sensitivity boundary of each data entity in the data set according to the entity intermediate sensitivity boundary The entity sensitivity boundary calculation submodule is configured to calculate the sensitivity boundary of each data entity in the data set according to the entity intermediate sensitivity boundary The entity sensitivity boundary calculation submodule is configured to calculate the sensitivity boundary of each data entity in the data set according to the entity intermediate sensitivity boundary The entity sensitivity boundary mean value calculation submodule is configured to calculate the mean value of the sensitivity boundary of all data entities in the data set The entity sensitivity boundary mean value calculation submodule is configured to calculate the mean value of the sensitivity boundary of all data entities in the data set The entity sampling probability calculation submodule is configured to calculate the sampling probability of all data entities in the data set The entity sampling probability calculation submodule is configured to calculate the sampling probability of all data entities in the data set The entity sampling probability calculation submodule is configured to calculate the sampling probability of all data entities in the data set The compressed set entity size calculation submodule is configured to calculate the size of the subset of the original data set after compression according to the compression tolerance parameter of the terminal The compressed set entity size calculation submodule is configured to calculate the size of the subset of the original data set after compression according to the compression tolerance parameter of the terminal The compressed set entity size calculation submodule is configured to calculate the size of the subset of the original data set after compression according to the compression tolerance parameter of the terminal The compressed set entity size calculation submodule is configured to calculate the size of the subset of the original data set after compression according to the compression tolerance parameter of the terminal The entity sampling submodule is configured to sample a subset of data entities that meet the conditions from the original data set according to the sampling probability and the size of the subset of data entities The entity sampling submodule is configured to sample a subset of data entities that meet the conditions from the original data set according to the sampling probability and the size of the subset of data entities The entity sampling submodule is configured to sample a subset of data entities that meet the conditions from the original data set according to the sampling probability and the size of the subset of data entities The entity weight calculation submodule is configured to calculate the entity weight of each data entity in the data set The entity weight calculation submodule is configured to calculate the entity weight of each data entity in the data set The entity weight calculation submodule is configured to calculate the entity weight of each data entity in the data set The entity parallel sampling submodule is configured to execute the above submodules in parallel to obtain the weighted data entity compression subset of each terminal The entity parallel sampling submodule is configured to execute the above submodules in parallel to obtain the weighted data entity compression subset of each terminal The time interval compression module preferably comprises: The entity mean distance and calculation submodule is configured to calculate the sum of the distances of each data entity in the data set from the entity mean value The entity mean distance and calculation submodule is configured to calculate the sum of the distances of each data entity in the data set from the entity mean value The entity mean distance and calculation submodule is configured to calculate the sum of the distances of each data entity in the data set from the entity mean value The intra-cluster time sensitivity boundary calculation submodule is configured to calculate the time sensitivity boundary of each data entity in the data set The intra-cluster time sensitivity boundary calculation submodule is configured to calculate the time sensitivity boundary of each data entity in the data set The intra-cluster time sensitivity boundary calculation submodule is configured to calculate the time sensitivity boundary of each data entity in the data set The intra-cluster time sensitivity boundary calculation submodule is configured to calculate the time sensitivity boundary of each data entity in the data set The intra-cluster time sensitivity boundary calculation submodule is configured to calculate the time sensitivity boundary of each data entity in the data set The intra-cluster time sensitivity boundary calculation submodule is configured to calculate the time sensitivity boundary of each data entity in the data set The intra-cluster time sensitivity boundary calculation submodule is configured to calculate the time sensitivity boundary of each data entity in the data set The intra-cluster time sensitivity boundary calculation submodule is configured to calculate the time sensitivity boundary of each data entity in the data set The intra-cluster time sensitivity boundary calculation submodule is configured to calculate the time sensitivity boundary of each data entity in the data set The intra-cluster time sensitivity boundary calculation submodule is configured to calculate the time sensitivity boundary of each data entity in the data set each piece of data in partitioning into clusters , time sensitivity boundary within cluster about time interval ; ; time interval distance function calculation submodule, configured to calculate each piece of data in , value of distance function when parameter space ; ; time intermediate sensitivity boundary calculation submodule, configured to calculate each piece of data in , time intermediate sensitivity boundary when influence of covariance and autocorrelation function is not considered, i.e. ; ; time sensitivity boundary calculation submodule, configured to calculate each piece of data in , time sensitivity boundary about time interval ; ; time interval sampling probability calculation submodule, configured to calculate each piece of data in , sampling probability about time interval ; ; time sensitivity boundary mean value calculation submodule, configured to calculate mean value of time sensitivity boundary ; time interval sampling scale calculation submodule, configured to calculate number of compression time intervals satisfying conditions according to compression tolerance parameter and allowed error rate parameter of device , and time sensitivity boundary mean value ; ; time interval sampling submodule, configured to sample subset satisfying conditions from time intervals according to sampling probability and number of compression time intervals ; ; ; time interval parallel sampling submodule, configured to extract a weighted time interval subset from dataset of terminal in parallel; ; time interval weight calculation submodule, configured to calculate each piece of data in subset ; Regarding time intervals weight .

[0043] Preferably, the sparse LSTM model deployment module includes: The fully connected LSTM model building submodule is used to build a hidden layer in the cloud. Layer and the first The number of neurons in the layer is A fully connected LSTM model was constructed, and the model parameters were initialized. ; The sparse probability calculation submodule is used to calculate the sparsity based on the model sparsification parameters. Calculate the layers in the fully connected LSTM model neuron nodes With layers neuron nodes sparse connection probability ; The sparse LSTM model construction submodule is used to calculate the sparsified connection probabilities. Calculate the mask matrix for sparse connections. And according to the mask matrix Calculate the sparsity of the model parameters It is then distributed to each terminal.

[0044] Preferably, the local model training module includes: The local model initialization submodule is used by the terminal to initialize the current iteration round based on the data sent from the cloud. global model parameters Initialize local model parameters ; The compressed dataset training submodule is used by the terminal to train the compressed subset. Perform local model training to obtain local model update parameters. And upload it to the cloud; Parallel training submodule, for each terminal The above sub-modules are executed in parallel to obtain the local model update parameter set for all devices. .

[0045] Preferably, the model structure evolution module includes: The connection importance evaluation submodule is used to evaluate the current iteration round. In the global model, arbitrary connections Importance ; The pruning rate acquisition submodule is used to obtain the pruning rate from the initial model. Calculate the current iteration round Model pruning rate , The connection importance ranking submodule is used to rank the importance of each connection in the global model; The unimportant connection removal submodule is used to remove connections from the global model structure. %Unimportant connections, and the evolved global model and corresponding model parameters. Distribute to each terminal; The iterative training submodule is used for global model iterative training to obtain the final result. Global model parameters during rounds of iterative training and cumulative communication bits .

[0046] The beneficial effects of this invention are as follows: In federated learning-based traffic flow prediction scenarios, the high communication cost arises because the network bandwidth resources of intelligent transportation devices are limited, requiring the transmission of high-dimensional update parameters obtained from complex model training to the cloud in each iteration. To address this issue, this invention starts with the real-time massive and redundant characteristics of trajectory sequence data in traffic scenarios. It compresses the real-time massive trajectory entity-time interval data and deploys a sparse LSTM model structure on this basis. By transmitting only the local updates obtained from the sparse LSTM model training to the cloud in each iteration, it can reduce the amount of data transmission between the terminal and the cloud while ensuring the performance of the trained model. Compared with existing technologies, it has the following specific beneficial effects: (1) Unlike existing effective federated learning communication methods that improve model training communication efficiency by reducing redundant gradient transmission through model compression (such as model quantization and model sparsification), this invention comprehensively optimizes both the redundancy of dynamic streaming training data and the complexity of the LSTM model in traffic flow prediction scenarios. It deeply analyzes the mechanism of redundant gradient generation, fundamentally reducing the transmission of invalid information and significantly improving the communication efficiency of federated learning based on streaming traffic data scenarios. The method and system proposed in this invention can provide new ideas for solving the communication efficiency problem in traffic flow prediction scenarios. (2) This invention combines the temporal correlation of trajectory sequence data with the weight of data entities, redefines the sensitivity of data entities / time intervals and the boundary of data entities / time intervals, and proposes a streaming trajectory data entity and time interval compression algorithm to effectively eliminate duplicate, similar and low-contribution data entities / time intervals, thereby reducing communication overhead from the data source.

[0047] (3) Based on the importance measurement of the LSTM model connection, the distributed model evolution algorithm suitable for the traffic flow prediction scene is proposed, the model structure and scale can be evolved adaptively under the premise of ensuring the prediction accuracy, the calculation and communication overhead is reduced, and the real-time prediction demand is met.

[0048] (4) The sparse LSTM is used for training on the compressed streaming trajectory data, the sparse gradient is transmitted and the model scale is dynamically adjusted in each iteration, the balance between low calculation load and high prediction accuracy is realized, and the method has good universality and can be popularized to the federated learning scene of multi-source streaming high-redundancy data such as smart city and Internet of Vehicles.

[0049] Other advantages, objects, and features of the application will be set forth in part in the following specification, and in part will become apparent to those skilled in the art from a consideration of the following specification, or can be learned from practice of the application. The goals and other advantages of the application can be realized and attained by the structure particularly pointed out in the specification. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to make the objects, technical solutions and advantages of the application clearer, the preferred detailed description of the application will be made below in combination with the drawings, in which: Figure 1 is the overall flowchart of the federated learning efficient communication method provided by example 1; Figure 2 is the specific step flowchart of the federated learning efficient communication method provided by example 1; Figure 3 is the schematic diagram of the federated learning efficient communication system module design for traffic flow prediction provided by example 1. DETAILED DESCRIPTION

[0051] The embodiments of the application are described below through specific and concrete examples, and those skilled in the art can easily understand other advantages and effects of the application from the disclosure of the specification. The application can also be implemented or applied through other different specific embodiments, and each detail in the specification can be modified or changed based on different views and applications without departing from the spirit of the application. It should be noted that the diagrams provided in the following examples only illustrate the basic concept of the application in a schematic manner, and the following examples and features in the examples can be combined with each other without conflict.

[0052] The following takes 100 terminals to simulate multiple users of a smart traffic scenario based on federated learning, uses a long short-term memory network (LSTM) commonly used in traffic flow prediction scenarios to train the METR-LA dataset as an example to illustrate the specific implementation steps of the federated learning efficient communication method for traffic flow prediction provided by the application, and the overall goal is to reduce the cumulative communication bits of the terminal and the cloud , to improve the communication efficiency of federated learning.

[0053] Embodiment 1 This embodiment uses computer software technology to realize the automatic operation process, Figure 1 is a general method flowchart, see Figure 1 , in combination with Figure 2 the specific step flowchart, the specific steps of an embodiment of the federated learning efficient communication method for traffic flow prediction provided by the application include: Step S1: original trajectory data entity compression.

[0054] Taking a smart traffic scenario based on federated learning as an example, consider 100 terminals Each terminal is deployed on a road perception device to collect traffic trajectory sequence data of the corresponding sensor in the METR-LA dataset. It is assumed that The collected trajectory dataset is , wherein the first trajectory is , where indicates that the trajectory consists of a sequence of 12 time points, and each time point contains three types of features, i.e. . To compress and model the trajectory, a Gaussian mixture model containing Gaussian components is used, and its parameter set is , satisfying and . Let the cost of uncompressed data be , and the corresponding cost after compression be . To ensure the acceptability of compression, the compression tolerance is taken as , the error rate , and the cost function should satisfy the relative error constraint: . The specific implementation steps are as follows: Step S101: For any device , according to , calculate the entity mean of any data .

[0055] Step S102: According to the mean The k-means clustering algorithm is used to... Divide the data into 20 clusters, and the goal is to minimize the correlation between each data point and any arbitrary cluster. Cluster center of each cluster The distance is used to obtain the final cluster partitioning result. That is, it satisfies the following calculation formula:

[0056] Step S103: Statistics Each data Divided into clusters Which cluster does it belong to, and how do you determine its cluster number? ,Right now:

[0057] Step S104: Calculate the dataset Each data Cluster The impact on all data in the cluster, i.e., based on each data item Divided into clusters number of elements Calculate the sensitivity boundary within the data entity cluster. The formula is as follows:

[0058] in, Cluster The number of data entities in the data.

[0059] Step S105: Calculate the dataset Each piece of data entity mean Its division into clusters The sum of the 2-norm distances between the center points The formula is as follows:

[0060] Step S106: Calculate the dataset Each data entity 2-norm distance The formula is as follows:

[0061] Step S107: For the dataset Each data Calculate the parameter space of the Gaussian mixture model. The effects of covariance and autocorrelation function are not considered in this context. Data entity intermediate sensitivity boundary The formula is as follows:

[0062] Step S108: For the dataset Each data The entity intermediate sensitivity boundary calculated according to step S107 Calculate the sensitivity boundary of data entities The formula is as follows:

[0063] Among them, parameters ,constant It represents all clusters. The difference between the largest and smallest eigenvalues ​​of the covariance matrix in the clusters with the largest and smallest eigenvalues ​​of the covariance matrix; In the embodiment, for each data Set parameters ,according to Calculate the sensitivity boundary The value of , where for constants It is the ratio of the largest eigenvalue to the smallest eigenvalue in the covariance matrix of all clusters; Step S109: For the dataset Each data Based on the entity sensitivity boundary obtained in step S108 Calculate the sampling probability of its data entities. The formula is as follows:

[0064] In the embodiment, for each data Calculate the probability that it will be sampled. ; Step S110: Based on each data obtained in step S108 entity sensitivity boundary Calculate the dataset The average value of the entity sensitivity boundary for all data in the dataset. The formula is as follows:

[0065] In the embodiment, for each data Calculate the average of the entity sensitivity boundaries for all data. .

[0066] Step S111: According to the preset equipment Compression tolerance parameter and the allowed error rate and the entity sensitivity boundary mean obtained in step S110 , calculate the weighted data entity subset size satisfying the condition , the formula is as follows:

[0067] In the embodiment, according to the pre-set device allowed error rate , calculate the weighted data entity subset size satisfying the condition in the data set .

[0068] Step S112: according to the sampling probability obtained in step S109 , sample a weighted entity subset with a data entity collection size of from the original trajectory data set .

[0069] In the embodiment, according to the sampling probability of each data and the sampled data entity subset size , sample a weighted entity subset with a data entity collection size of from the data set .

[0070] Step S113, according to the entity sensitivity boundary mean of each data in the data set obtained in step S110 , calculate the entity weight of each data in the collection , the formula is:

[0071] Repeat steps S101 to S113, and all devices in the collection sample a weighted entity subset with an entity collection size of from their respective data sets according to the sampling probability , to obtain a weighted entity subset collection of all devices .

[0072] Step S2: original trajectory data time interval compression.

[0073] Trajectory data compression is performed from the time interval level, that is, from each device​​​​​Data of Selecting important time intervals, obtaining a weighted data time interval compression subset , such that the weighted time interval cost function of the data set before and after compression satisfies: , wherein , , represents the set Data in time interval weighted weight. The specific implementation steps are as follows: Step S201: Calculate the distance sum of each data , data entity , and data entity mean in each time interval , the formula is as follows:

[0074] In the embodiment, the distance sum of each data , data entity , and data entity mean in each time interval is calculated.

[0075] Step S202: For each data in the data set , calculate its division into cluster , The influence of time interval on all data in the cluster , that is, the intra-cluster time sensitivity boundary , the formula is as follows:

[0076] In the embodiment, for each data , the influence of time interval of the data when it is divided into cluster on all data in the cluster , that is, the intra-cluster time sensitivity boundary .

[0077] Step S203: For each data in the data set , calculate the influence of time interval in the parameter space of the Gaussian mixture model on all data in the cluster​​without considering the influence of the covariance and the autocorrelation function, i.e. the time interval sensitivity boundary of each data in the data set when the parameter space of the Gaussian mixture model is .

[0078] In the embodiment, the time interval sensitivity boundary of each data is calculated.

[0079] Step S204: According to step S202 and step S203, the time interval sensitivity boundary of each data in the data set is calculated. The formula is as follows:

[0080] wherein, denotes the distance function of the time interval in the parameter space of the Gaussian mixture model without considering the influence of the covariance and the autocorrelation function.

[0081] In the embodiment, the sensitivity boundary of each data about the time interval is calculated.

[0082] Step S205: According to the time interval sensitivity boundary calculated in step S204, the sampling probability of each data about the time interval is calculated, and the formula is as follows: In the embodiment, the sampling probability

[0083] about the time interval is calculated for each data .

[0084] Step S206: According to the time interval sensitivity boundary calculated in step S204, the average value of the time interval sensitivity boundary is calculated, and the formula is as follows:

[0085] In the embodiment, the average value of the time interval sensitivity boundary is calculated.

[0086] Step S207 calculates the number of important time intervals satisfying the condition size according to the compression tolerance parameter of the device , the allowed error rate parameter , and the time sensitivity boundary average value of step S206 .

[0087] In an embodiment, the number of important time intervals satisfying the condition size is calculated .

[0088] Step S208 samples important time interval data from the time intervals of the data set according to the sampling probability of step S205 and the number of time interval samples of step S207 , to obtain a time interval sample set .

[0089] In an embodiment, the number of important time interval data is sampled from the time intervals of the data set according to the sampling probability and the number of time interval samples , to obtain a time interval sample set .

[0090] Step S209: the weight of each data time interval in the new data set is calculated as follows:

[0091] In an embodiment, the weight of each data time interval in the new data set is calculated.

[0092] Repeat steps S201 to S209, and all devices in the set sample important weighted time interval subsets from the time intervals of their local data sets according to the sampling probability in parallel, to obtain a set of weighted time interval subsets of all devices .

[0093] In an embodiment, the set​​​​​ All devices in the process execute steps in parallel. All sub-steps from their local dataset Sampling in time interval A significant subset of weighted time intervals This yields a weighted subset of time intervals for all devices. .

[0094] Each dataset is obtained through steps S1 and S2. Compressed dataset ,in Compressed sets of entities and time interval compression set Composition, that is .

[0095] In the embodiment, the steps are... and steps The resulting compressed set of data entities and time interval compression set The subsets are combined to obtain each dataset. Compressed dataset .

[0096] Step S3: Sparsify global model deployment. Based on the model sparsification parameters... The process involves sparsifying the initialized LSTM model and deploying the sparsely initialized LSTM to each terminal, including the following sub-steps: In the embodiment, based on the model sparsity parameters The initialized LSTM model is sparsified, and the sparsely initialized LSTM is deployed to each terminal. The specific implementation is as follows. Step S301: Based on the model training task, initialize a model containing an input layer, hidden layers, and an output layer. The hidden layers have... Layer and the first The number of neurons in the layer is A fully connected LSTM model; In this embodiment, a fully connected LSTM model is constructed with 207, 200, and 207 neurons in the input layer, hidden layer, and output layer, respectively, and each neuron in the input layer is connected to all neurons in the previous layer.

[0097] Step S302: According to the set model sparsity parameters Calculate the layers in the fully connected LSTM model neuron nodes With layers neuron nodes sparse connection probability , the formula is as follows:

[0098] In the embodiment, full connection of each layer is sparsified, so that the connection between the neurons of the i-th layer and the neurons of the j-th layer satisfies the probability . .

[0099] Step S303: According to the full connection LSTM model initialized in step S301, the model parameters are initialized. In the embodiment, the model parameters are initialized according to the full connection LSTM model structure.

[0100] Step S304: According to the sparsification connection probability obtained in step S302, the mask matrix is calculated. In the embodiment, the mask matrix is calculated according to the sparsification connection probability .

[0101] Step S305, according to the mask matrix obtained in S304, the initial LSTM sparsification network model parameters are calculated, and the formula is as follows:

[0102] In the embodiment, the full connection LSTM model is sparsified according to the mask matrix , and the sparsified model parameters are obtained.

[0103] Step S306: The initial sparsified model parameters obtained in step S305 are distributed to each terminal.

[0104] In the embodiment, the sparsified model parameters are distributed to each terminal .

[0105] Step S4: Global initialization. Initialize the cumulative communication transmission bit number , the global communication iteration round number , the training block size , the local training number , the learning rate , the pruning rate , and the exponential decay rate parameter . ​​​

[0106] In an embodiment, the global communication iteration round number is initialized , and the cumulative communication transmission bit number is accumulated , the minimum block size of model training is 50 , the local training number is 5 , the learning rate is 0.01 , the number function decay rate is 0.01 , and the pruning rate is .

[0107] Step S5: Local model acquisition. In each global model iteration round , each device performs local model training according to the trajectory compressed data set using the issued LSTM model parameters , obtains local model parameters after local iteration training for rounds , and transmits to the cloud, including the following sub-steps: In an embodiment, each device performs local model training according to the trajectory compressed data set using the issued LSTM model parameters in parallel, obtains local model parameters after local iteration training for rounds , and uploads to the cloud, and the specific implementation is as follows. Step S501: Each terminal initializes local model parameters according to the global model parameters of the current iteration round , and the formula is as follows:

[0108] In an embodiment, each terminal initializes local model updates according to the global model parameters of the current iteration round , and sets .

[0109] Step S502: Each terminal performs local model training according to the trajectory compressed data set , obtains updated local model parameters , and the calculation formula is as follows:

[0110] In an embodiment, each terminal uses the trajectory compressed data set according to the formula ​​​​The local model training is performed to obtain the updated local model parameters . .

[0111] Step S503: Each terminal uploads the updated local model parameters to the cloud.

[0112] In an embodiment, each terminal uploads the local model update to the cloud.

[0113] Step S6: Global model weighted aggregation. The cloud calculates the global model parameters of the current iteration round and the total data transmission bit number according to the updated local model parameters uploaded by the terminals, including the following sub-steps: In an embodiment, the cloud calculates the global model parameters of the current iteration round and the total data transmission bit number according to the local model parameters uploaded by the terminals, and the specific implementation is as follows, Step S601: The cloud aggregates the updated local model parameters uploaded by the terminals , calculates the global model parameters of the current iteration round , and the formula is as follows:

[0114] In an embodiment, the cloud aggregates the uploaded updated local model parameters , and calculates the global model parameters of the round according to the formula .

[0115] Step S602: Update the cumulative communication transmission bit number , and the formula is as follows:

[0116] In an embodiment, the cumulative communication transmission bit number is calculated according to the formula .

[0117] Step S7: Global model adaptation. The cloud evaluates the importance of each connection in the global model parameters to obtain its importance . At the same time, according to the given initial model pruning rate , the current iteration round​ Model pruning rate and based on The process of evolving the structure of the global model and distributing the evolved model and its parameters to each terminal includes the following sub-steps: In this embodiment, the global model parameters are evaluated in the cloud. Each connection Importance And calculate the current iteration round. Model pruning rate ,in accordance with The global model structure is evolved, and the evolved model and its parameters are distributed to each terminal. The specific implementation is as follows: Step S701: Evaluate global model parameters Each connection Importance The formula is as follows:

[0118] In this embodiment, the LSTM model parameters are calculated. Each sub-parameter The absolute value of the value is used as the basis for each connection. The importance of.

[0119] Step S702: The cloud prunes the model according to the given initial model pruning rate. Calculate the current iteration round Model pruning rate ;

[0120] In this embodiment, an exponential decay rate parameter is set. =0.01, according to the formula Get the previous iteration rounds Model pruning rate .

[0121] Step S703: Cloud-based Evolution of the global model structure, i.e., from Remove from Unimportant network connections are used to obtain the evolved network model and its corresponding parameters. And the evolved global model structure and model parameters Distribute to each terminal and begin the next round of iterative training.

[0122] In the embodiment, the cloud relies on Remove the network connections from the LSTM model. Unimportant network connections, get the LSTM model evolved through structure evolution and the corresponding model parameters , and distribute the evolved global model structure and model parameters to each terminal .

[0123] Repeat steps S5-S7 until the model training is completed, and the final cumulative communication bit number and global model parameters are obtained.

[0124] In the embodiment, steps S5-S7 are repeated until the model training is completed, and the final total communication volume and global model parameters are obtained.

[0125] Step S8: traffic flow prediction output. The cloud end downloads the finally converged global LSTM model to each terminal device, and each terminal uses the model to predict the traffic flow input data collected at time , and outputs the traffic flow prediction result in the future time interval , and the calculation formula is as follows:

[0126] wherein, represents the LSTM model function based on the global parameters , and is the length of the future prediction time interval.

[0127] In the embodiment, the cloud end downloads the global model obtained through iterations to each terminal , and each terminal uses the received global model to predict the traffic flow data collected at time in the past 12 time stamps (i.e. past 60 minutes) and the traffic flow of each node in the future 9 time stamps (i.e. future 45 minutes) .

[0128] Embodiment 2: The embodiment provides a federated learning efficient communication system for traffic flow prediction, as shown in Figure 3 , specifically comprising: An entity compression module is used for device data entity compression is performed on the respective data sets in parallel to obtain a weighted compressed entity subset , comprising the following sub-modules: an entity mean calculation submodule for calculating the entity mean of each data in the data set .

[0129] a cluster division submodule for dividing each data in the data set into a suitable cluster according to the entity mean of each data in the data set using the k-means clustering algorithm to obtain the final cluster division result .

[0130] an index label acquisition submodule for acquiring the index label of each data in the data set divided into a cluster .

[0131] an intra-cluster entity sensitivity boundary calculation submodule for calculating the intra-cluster data entity sensitivity boundary of each data in the data set divided into a cluster .

[0132] an entity-cluster distance calculation submodule for calculating the distance sum of the entity mean of each data in the data set and the cluster center point .

[0133] an entity two-norm calculation submodule for calculating the two-norm distance of each data in the data set .

[0134] a distance function calculation submodule for calculating the value of the distance function when the parameter space of the Gaussian mixture model .

[0135] an entity intermediate sensitivity boundary calculation submodule for calculating the entity intermediate sensitivity boundary of each data in the data set when the parameter space of the Gaussian mixture model .

[0136] ​​​​​​​The feature value difference calculation submodule is configured to calculate the difference between the maximum eigenvalue and the minimum eigenvalue of the covariance matrix in the cluster of the maximum eigenvalue and the cluster of the minimum eigenvalue of the covariance matrix .

[0137] The entity sensitivity boundary calculation submodule is configured to calculate the entity intermediate sensitivity boundary according to the entity intermediate sensitivity boundary The entity sensitivity boundary calculation submodule is configured to calculate the entity intermediate sensitivity boundary according to the entity intermediate sensitivity boundary .

[0138] The entity sensitivity boundary mean value calculation submodule is configured to calculate the mean value of the sensitivity boundary of all data in the data set The entity sensitivity boundary mean value calculation submodule is configured to calculate the mean value of the sensitivity boundary of all data in the data set .

[0139] The entity sampling probability calculation submodule is configured to calculate the sampling probability of all data entities in the data set The entity sampling probability calculation submodule is configured to calculate the sampling probability of all data entities in the data set .

[0140] The compressed set entity scale calculation submodule is configured to calculate the scale of the subset of the original data set after compression according to the compression tolerance parameter of the device The compressed set entity scale calculation submodule is configured to calculate the scale of the subset of the original data set after compression according to the compression tolerance parameter of the device The compressed set entity scale calculation submodule is configured to calculate the scale of the subset of the original data set after compression according to the compression tolerance parameter of the device The compressed set entity scale calculation submodule is configured to calculate the scale of the subset of the original data set after compression according to the compression tolerance parameter of the device The compressed set entity scale calculation submodule is configured to calculate the scale of the subset of the original data set after compression according to the compression tolerance parameter of the device .

[0141] The entity sampling submodule is configured to sample a subset of data entities that meet the conditions from the original data set The entity sampling submodule is configured to sample a subset of data entities that meet the conditions from the original data set The entity sampling submodule is configured to sample a subset of data entities that meet the conditions from the original data set .

[0142] The entity weight calculation submodule is configured to calculate the entity weight of each piece of data in the set The entity weight calculation submodule is configured to calculate the entity weight of each piece of data in the set The entity weight calculation submodule is configured to calculate the entity weight of each piece of data in the set .

[0143] The entity parallel sampling submodule is configured to perform the above submodules in parallel to obtain the weighted data entity compression subset of each device The entity parallel sampling submodule is configured to perform the above submodules in parallel to obtain the weighted data entity compression subset of each device .

[0144] The time interval compression module is configured to select K important time intervals from the data set of each device The time interval compression module is configured to select K important time intervals from the data set of each device The time interval compression module is configured to select K important time intervals from the data set of each device The time interval compression module is configured to select K important time intervals from the data set of each device , comprising the following submodules: The entity mean distance and calculation submodule is configured to calculate the distance between the mean value of each piece of data in the set The entity mean distance and calculation submodule is configured to calculate the distance between the mean value of each piece of data in the set ​entity and its entity mean distance sum .

[0145] intra-cluster time sensitivity boundary calculation submodule, configured to calculate the distance of each piece of data in the parameter space to the cluster , and the intra-cluster time sensitivity boundary about the time interval . .

[0146] time interval distance function calculation submodule, configured to calculate the value of the distance function of each piece of data in the parameter space when the parameter space . time intermediate sensitivity boundary calculation submodule, configured to calculate

[0147] the time intermediate sensitivity boundary of each piece of data in the parameter space when the influence of the covariance and the autocorrelation function is not considered, that is . time sensitivity boundary calculation submodule, configured to calculate the time sensitivity boundary of each piece of data in the parameter space

[0148] about the time interval .

[0149] time interval sampling probability calculation submodule, configured to calculate the sampling probability of each piece of data in the parameter space about the time interval according to the time sensitivity boundary .

[0150] time sensitivity boundary mean calculation submodule, configured to calculate the mean value of the time sensitivity boundary .

[0151] time interval sampling scale calculation submodule, configured to calculate the number of compression time intervals satisfying the condition according to the compression tolerance parameter and the allowed error rate parameter of the device and the time sensitivity boundary mean .

[0152] time interval sampling submodule, configured to sample the time interval from the time interval according to the sampling probability and the number of compression time intervals .​​​​Mid-sampling yields subsets that meet the conditions .

[0153] The time-interval parallel sampling submodule is used to sample data in parallel from... dataset Extract a weighted subset of time intervals. .

[0154] The time interval weight calculation submodule is used to calculate subsets. Each data point Regarding time intervals weight .

[0155] Entity-time pair merging module, used to merge each device Original dataset compressed data entity subset and time interval subset To obtain the original dataset Compressed dataset .

[0156] The sparse LSTM model deployment module is used to deploy models based on sparsification parameters. The LSTM model is sparsified, and the initialized sparse LSTM model is deployed to each terminal. It contains the following sub-modules: The fully connected LSTM model building submodule is used to build a hidden layer in the cloud. Layer and the first The number of neurons in the layer is A fully connected LSTM model was constructed, and the model parameters were initialized. .

[0157] The sparse probability calculation submodule is used to calculate the sparsity based on the model sparsification parameters. Calculate the layers in the fully connected LSTM model neural nodes With layers neural nodes sparse connection probability .

[0158] The sparse LSTM model construction submodule is used to calculate the sparsified connection probabilities. Calculate the mask matrix for sparse connections. And according to the mask matrix Calculate the sparsity of the model parameters It is then distributed to each terminal.

[0159] The global initialization module is used to initialize the cumulative number of communication transmission bits in the cloud. Global communication iteration rounds Training block size Local training data Learning rate pruning rate and exponential decay parameter .

[0160] Local model training module, used for each terminal Based on compressed subsets Perform local model training to obtain... Local model trained in rounds of local iterations and will The data is transmitted to the cloud and includes the following sub-modules: The local model initialization submodule is used by users. According to the current iteration round issued by the cloud global model parameters Initialize local model parameters .

[0161] The compressed dataset training submodule is for users. Based on compressed subsets Perform local model training to obtain local model update parameters. And upload it to the cloud.

[0162] Parallel training submodule, for each terminal The above sub-modules are executed in parallel to obtain the local model update parameter set for all devices. .

[0163] The model aggregation module is used to aggregate the current iteration round number uploaded from all terminals in the cloud. Local model update parameters To obtain updated global model parameters At the same time, update the current iteration round. Cumulative number of communication transmission bits .

[0164] The model structure evolution module is used to evolve the structure of the global LSTM model in the cloud, removing some unimportant model connections, and distributing the evolved LSTM model to each terminal. It contains the following submodules.

[0165] The connection importance evaluation submodule is used to evaluate the current iteration round. global model In, any connection Importance .

[0166] The pruning rate obtaining submodule is configured to obtain a pruning rate of the initial model according to the initial model The pruning rate of the model in the current iteration round is calculated .

[0167] The connection importance ranking submodule is configured to rank the importance of each connection in the global model.

[0168] The unimportant connection removing submodule is configured to remove the unimportant connections from the structure of the global model , and distribute the global model after evolution of the structure and corresponding model parameters to each terminal . .

[0169] The iterative training submodule is configured to iteratively train the global model to obtain the global model parameters after iterative training for a final round and the cumulative number of communication bits . .

[0170] The traffic flow prediction module is configured to use the final global model and corresponding model parameters received by the terminal to predict the real-time collected traffic flow data , and obtain a traffic flow prediction result in a future time interval . .

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, and all should be covered in the scope of the claims of the present application.

Claims

1. A federated learning-based efficient communication method for traffic flow prediction, characterized in that, Includes the following steps: S1: Each terminal compresses the streaming trajectory data it collects locally, filters out key trajectory segments and important time points, and generates a compressed trajectory dataset; S2: Based on the compressed trajectory dataset, each terminal deploys a small sparse LSTM model for local model training, obtains local model update parameters, and uploads them to the cloud. S3: Perform global aggregation in the cloud to obtain a global model; S4: The cloud combines the importance assessment of model update parameters, adaptively removes low-weight connections from the global model, dynamically evolves the model structure, and sends the evolved global model and corresponding model parameters to the terminal. S5: Repeat steps S2 to S4 until the global model converges; S6: Each terminal uses the converged global model to predict real-time traffic flow data and outputs the traffic flow changes for future periods, realizing dynamic distributed prediction of traffic flow.

2. The efficient communication method for federated learning in traffic flow prediction according to claim 1, characterized in that, In step S1, the compression includes compression of the original trajectory data entities and compression of the original trajectory data time intervals; The compression of the original trajectory data entities specifically involves: given a set of terminals Data set , for The collected trajectory sequence data, among which, ,and , Indicates device The collected number Trajectory data, the trajectory is composed of It consists of a sequence of points at each time point, each point express 3D feature space vector; for dataset Gaussian mixture models are used for modeling compression, with the goal of extracting data from the dataset. Select a compressed subset from the entities. If the compression error before and after compression is minimized, that is, if the cost function before and after compression satisfies the following compression constraints, then the compression is considered acceptable. in, and Let these represent the cost functions before and after compression, respectively. Both are constructed based on the log-likelihood function of the Gaussian mixture model and are defined as follows: in, Represents a compressed subset The Middle The weight of each trajectory data entity; For compression tolerance parameters, ; For the allowable error rate parameter, ; and These represent the number of Gaussian components in the Gaussian mixture model and the weight of each component, respectively. , ,and ; Let be the parameter space of the Gaussian mixture model. This represents the mean of the Gaussian components. and Let these represent the covariance matrix and the autocorrelation matrix, respectively. This represents a specific instance of a Gaussian mixture model. Representing data With the The distance function between Gaussian components; The original trajectory data time interval compression specifically involves compressing the trajectory data at the time interval level, that is, compressing it from the perspective of each device. Data Select from all time intervals A weighted compressed subset of data time intervals is obtained by considering several important time intervals. This ensures that the weighted time interval cost function of the datasets before and after compression satisfies the following constraints: in, , , and These represent the weighted time interval costs before and after compression, respectively. Represents a set China Data In time interval The weighted weights; Representing data In time interval , the distance function between the Gaussian component; Each dataset is obtained by compressing the original trajectory data entities and the original trajectory data time intervals. Compressed dataset ,in Compressed sets of entities and time interval compression set Composition, that is .

3. The efficient communication method for federated learning in traffic flow prediction according to claim 2, characterized in that, In step S1, the compression of the original trajectory data entity specifically includes the following steps: S101: Calculate the dataset Each data entity mean The formula is as follows: S102: The mean value calculated based on step S101 The k-means clustering algorithm is used to... Classified to In the n clusters, the goal is to minimize the relationship between each data point and any nth cluster. Cluster center of each cluster The distance is used to obtain the final cluster partitioning result. That is, it satisfies the following calculation formula: in, This represents the set of k cluster centers currently used in the k-means clustering algorithm; S103: For the dataset Each piece of data Calculate the cluster index number of the cluster that is closest to the cluster from which it is assigned. ,Right now: S104: Calculate the dataset Each data Cluster The impact on all data in the cluster, i.e., calculating the sensitivity boundary of the data entity within the cluster. The formula is as follows: in, Cluster The number of data entities in the data; Compute dataset Each piece of data With its cluster center cluster The total distance The formula is as follows: Compute dataset Each data entity 2-norm distance The formula is as follows: S105: For the dataset Each data Calculate the parameter space of the Gaussian mixture model. The effects of covariance and autocorrelation function are not considered in this context. Data entity intermediate sensitivity boundary The formula is as follows: S106: For the dataset Each data The entity intermediate sensitivity boundary calculated according to step S105 Calculate the sensitivity boundary of data entities The formula is as follows: Among them, parameters ;constant It represents all clusters The difference between the largest and smallest eigenvalues ​​of the covariance matrix in the clusters with the largest and smallest eigenvalues ​​of the covariance matrix; S107: For the dataset Each data Based on the entity sensitivity boundary obtained in step S106 Calculate the sampling probability of its data entities. The formula is as follows: Compute dataset The average value of the entity sensitivity boundary for all data in the dataset. The formula is as follows: S108: According to the preset device Compression tolerance parameter and allowable error rate and the mean value of the entity sensitivity boundary obtained in step S107 Calculate the size of the weighted subset of data entities that meet the conditions. The formula is as follows: S109: Sampling probability obtained from step S107 From the original trajectory dataset The size of the sampled data entity set is Weighted entity subset ; S110: Data set obtained from step S107 Each data Average value of entity sensitivity boundary Calculate the set Each data Entity weight The formula is: Repeat steps S101 to S110, set All devices in the middle are based on the sampling probability In parallel from their respective datasets The size of the sampled entity set is Weighted entity subset This yields a weighted subset of entities for all devices. .

4. The efficient communication method for federated learning in traffic flow prediction according to claim 2, characterized in that, In step S1, the compression of the original trajectory data time interval specifically includes the following steps: S201: Calculate the dataset Each data Each time interval data entities with data entity mean The total distance The formula is as follows: S202: For the dataset Each data Calculate its division into clusters , Medium time interval Cluster The impact of all data in the cluster, i.e., the intra-cluster time sensitivity boundary. The formula is: S203: For the dataset Each data Calculate the parameter space of the Gaussian mixture model. Medium time interval Ignoring the effects of covariance and autocorrelation function, i.e. Time intermediate sensitivity boundary The formula is as follows: S204: Calculate the dataset based on steps S202 and S203. Each data Regarding time intervals Time sensitivity boundary The formula is as follows: in, Represents the parameter space of the Gaussian mixture model Medium time interval Distance function without considering covariance and autocorrelation; q This represents the time lag step, which is relative to the current time. The first step of backward tracing q Index of time intervals; S205: The time sensitivity boundary calculated based on step S204 Calculate the dataset Each data Sampling probability of time interval The formula is as follows: The time sensitivity boundary is calculated based on step S204. Calculate the average value of the time sensitivity boundary. The formula is as follows: S206: According to the equipment Compression tolerance parameter Allowable error rate parameter and the time sensitivity boundary mean obtained in step S205 Calculate the size that satisfies the condition. The number of important time intervals is calculated using the following formula: S207: Sampling probability obtained from step S205 The number of time interval samples obtained in step S206 From the dataset time interval Medium sampling From the data of several key time intervals, a time interval sampling set is obtained. ; S208: Update the dataset The time interval between each data entry weight The formula is as follows: Repeat steps S201 to S208, set All devices in the middle are based on the sampling probability Parallel from its local dataset Sampling in time interval A significant subset of weighted time intervals This yields a weighted subset of time intervals for all devices. .

5. The efficient communication method for federated learning in traffic flow prediction according to claim 1, characterized in that, In step S2, deploying a small-scale sparse LSTM model specifically includes: according to the model sparsity parameters The initial LSTM model is sparsified, and the sparsely initialized LSTM is deployed to each terminal, including the following specific steps: S201: Based on the model training task, initialize a model containing an input layer, hidden layers, and an output layer. The hidden layer has... Layer and the first The number of neurons in the layer is A fully connected LSTM model; S202: Based on the set model sparsity parameters Calculate the layers in the fully connected LSTM model neuron nodes With layers neuron nodes sparse connection probability The formula is as follows: in, and These represent the first and second lines in the LSTM fully connected network model, respectively. Layer and first The number of neurons in a layer. Then it means the first neurons in the layer With the neurons in the layer The connection probability; S203: Initialize the model parameters based on the fully connected LSTM model initialized in step S201. ; S204: Based on the sparsification connection probability obtained in step S202 Calculate the mask matrix ; S205: Mask matrix obtained from S204 The initial LSTM sparse network model parameters were calculated. The formula is as follows: S206: Apply the initial sparsification model parameters obtained in step S205 Distribute to each terminal.

6. The efficient communication method for federated learning in traffic flow prediction according to claim 1, characterized in that, In step S4, the dynamic evolution model structure specifically includes: cloud-based global model parameters. Each connection in Conduct an importance assessment to determine its importance. Meanwhile, based on the given initial model pruning rate... Calculate the current iteration round Model pruning rate and based on The structure of the global model is evolved, and the evolved model and its parameters are distributed to each terminal. This process includes the following steps: Step S401: Evaluate global model parameters Each connection Importance The formula is as follows: in, express The first in Individual parameter updates express The absolute value; Step S402: The cloud prunes the model according to the given initial model pruning rate. Calculate the current iteration round Model pruning rate : Among them, parameters The decay rate of the exponential function; Step S403: Cloud-based Evolution of the global model structure, i.e., from Remove from Unimportant network connections are used to obtain the evolved network model and its corresponding parameters. And the evolved global model structure and model parameters Distribute to each terminal and begin the next round of iterative training; Repeat the above steps until... The model training ends after one round of iterations, yielding the final cumulative number of communication transmission bits. and global model parameters .

7. A federated learning-based efficient communication system for traffic flow prediction, characterized in that, include: The compression module includes an entity compression module, a time interval compression module, and an entity-time pair merging module; The entity compression module is used by each terminal to compress data entities in parallel from their respective datasets, obtaining a weighted compressed entity subset. ,in , N The number of terminals; the time interval compression module is used to select from the dataset of each terminal. A weighted data time interval set is obtained by considering several important time intervals. The entity-time pair merging module is used to merge compressed data entity subsets of the original datasets from each terminal. and time interval subset To obtain the original dataset Compressed dataset ; The sparse LSTM model deployment module is used to deploy models based on sparsification parameters. The LSTM model is sparsified, and the initialized sparse LSTM model is deployed to each terminal. The global initialization module is used to initialize the cumulative number of communication transmission bits in the cloud. Global communication iteration rounds Training block size Local training data Learning rate pruning rate and exponential decay parameter ; The local model training module is used by each terminal to train the model based on the compressed dataset. Perform local model training to obtain... Local model parameters during rounds of local iterative training and will Transmitted to the cloud; The model aggregation module is used to aggregate the current iteration round number uploaded from all terminals in the cloud. Local model update parameters This yields updated global model parameters and updates the current iteration. The cumulative number of communication transmission bits; The model structure evolution module is used to evolve the structure of the global model in the cloud, remove some unimportant model connections, and distribute the evolved global model and corresponding model parameters to each terminal. The traffic flow prediction module is used to predict the traffic flow data collected in real time using the final global model and corresponding model parameters received by the terminal, and to obtain the traffic flow prediction results for future time intervals.

8. The efficient communication system for federated learning in traffic flow prediction according to claim 7, characterized in that, The entity compression module includes: The entity mean calculation submodule is used to calculate the dataset. Each data entity mean ; The clustering partitioning submodule is used to partition the dataset. Each data entity mean The k-means clustering algorithm was used to divide them into clusters with the smallest distance. l In this process, the final cluster partitioning result is obtained. , ; The index label retrieval submodule is used to retrieve the dataset. Each data It is divided into the cluster with the smallest distance. l index number ; The intra-cluster entity sensitivity boundary calculation submodule is used to calculate the dataset. Each data , divided into clusters Intra-cluster data entity sensitivity boundary ; The entity-cluster distance calculation submodule is used to calculate the dataset. Each data entity mean Its cluster Total distance from the center point ; The entity 2 norm calculation submodule is used to calculate the dataset. Each data 2-norm distance ; The distance function calculation submodule is used to calculate the parameter space of the Gaussian mixture model. Distance function at time The value; The entity intermediate sensitivity boundary calculation submodule is used for datasets. Each data When the parameter space of the Gaussian mixture model The sensitivity boundary of the entity at that time ; The eigenvalue difference calculation submodule is used to calculate the difference between the largest and smallest eigenvalues ​​of the covariance matrix within the cluster with the largest eigenvalue and the cluster with the smallest eigenvalue of the covariance matrix. ; The entity sensitivity boundary calculation submodule is used to calculate the sensitivity boundary of an entity. Calculate the sensitivity boundary of data entities ; The entity sensitivity boundary mean calculation submodule is used to calculate the dataset. Sensitivity boundary average of all data ; The entity sampling probability calculation submodule is used to calculate the dataset. Sampling probability of all data entities ; The compressed set entity size calculation submodule is used to calculate the size of the terminal. Compression tolerance parameter and allowed error rate parameter and the mean of the entity sensitivity boundary Calculate the size of the subset after entity compression of the original dataset. ; The entity sampling submodule is used to determine the sampling probability. and the size of data entity subsets From the original dataset Data entity subsets that meet the sampling conditions ; The entity weight calculation submodule is used to calculate the set. Each data Entity weight ; The entity parallel sampling submodule is used to execute the above submodules in parallel to obtain the sampling results for each terminal. Weighted data entity compression subset ; The time interval compression module includes: The entity mean distance and calculation submodule is used to calculate the dataset. Each data Entity and its mean The total distance ; The intra-cluster time sensitivity boundary calculation submodule is used to calculate... Each data Divided into clusters Regarding time intervals Intra-cluster time sensitivity boundary ; The time interval distance function calculation submodule is used to calculate... Each data When the parameter space Distance function at time The value; The time intermediate sensitivity boundary calculation submodule is used to calculate... Each data When the effects of covariance and autocorrelation function are not considered, i.e. Time intermediate sensitivity boundary ; The time sensitivity boundary calculation submodule is used to calculate... Each data Regarding time intervals Time sensitivity boundary ; The time interval sampling probability calculation submodule is used to calculate the probability based on the time sensitivity boundary. ,calculate Each data Sampling probability of time interval ; The time sensitivity boundary mean calculation submodule is used to calculate the average value of the time sensitivity boundary. ; The time interval sampling scale calculation submodule is used to calculate the sampling scale based on the device. Compression tolerance parameter and allowed error rate parameter and the mean of the time sensitivity boundary Calculate the number of compression time intervals that meet the conditions. ; The time interval sampling submodule is used to determine the sampling probability. and compression time interval ,from time interval Mid-sampling yields subsets that meet the conditions ; The time-interval parallel sampling submodule is used for parallel sampling from the terminal. dataset Extract a weighted subset of time intervals. ; The time interval weight calculation submodule is used to calculate subsets. Each data point Regarding time intervals weight .

9. The efficient communication system for federated learning in traffic flow prediction according to claim 7, characterized in that, The sparse LSTM model deployment module includes: The fully connected LSTM model building submodule is used to build a hidden layer in the cloud. Layer and the first The number of neurons in the layer is A fully connected LSTM model was constructed, and the model parameters were initialized. ; The sparse probability calculation submodule is used to calculate the sparsity based on the model sparsification parameters. Calculate the layers in the fully connected LSTM model neuron nodes With layers neuron nodes sparse connection probability ; The sparse LSTM model construction submodule is used to calculate the sparsified connection probabilities. Calculate the mask matrix for sparse connections. And according to the mask matrix Calculate the sparsity of the model parameters It is then distributed to each terminal.

10. The efficient communication system for federated learning in traffic flow prediction according to claim 7, characterized in that, The model structure evolution module includes: The connection importance evaluation submodule is used to evaluate the current iteration round. In the global model, arbitrary connections Importance ; The pruning rate acquisition submodule is used to obtain the pruning rate from the initial model. Calculate the current iteration round Model pruning rate , The connection importance ranking submodule is used to rank the importance of each connection in the global model; The unimportant connection removal submodule is used to remove connections from the global model structure. %Unimportant connections, and the evolved global model and corresponding model parameters. Distribute to each terminal; The iterative training submodule is used for global model iterative training to obtain the final result. Global model parameters during rounds of iterative training and cumulative communication bits .

Images