Method for improving federal learning robustness in Internet of Vehicles

Abstract

A traffic prediction method based on distributed machine learning is provided for predicting road traffic. In this case, the traffic prediction method based on distributed machine learning comprises the following steps: a learning server distributes a global multi-task traffic model to a learning agent to locally train the traffic model; the learning agent uploads the locally trained traffic model to a learning server; updating, by the learning server, the global multi-task traffic model using locally trained traffic model parameters acquired from the learning agent; the learning server generates a time-dependent global traffic map by using the trained global multi-task traffic model; distributing the time-dependent global traffic map to vehicles running on the road; and calculating, by the vehicle, an optimal travel route with minimal travel time based on the driving plan usage time dependent global traffic map.

Description

Technical Field

[0001] The present invention relates generally to distributed machine learning for vehicular traffic systems and, more particularly, to methods and apparatus for federated learning in vehicular networks. Background Art

[0002] Modern vehicles are equipped with a variety of onboard sensors to achieve a higher level of automation. Unlike traditional vehicles, modern vehicles are much smarter. They not only collect a variety of vehicle and traffic data, but also run advanced machine learning algorithms to guide their movements.

[0003] However, achieving intelligent transportation is an extremely difficult problem. Physical roads form a complex road network. Most importantly, traffic conditions, such as congestion at one location, can spread and impact traffic conditions at other locations. Furthermore, unexpected events, such as traffic accidents and driver behavior, make traffic conditions even more dynamic and uncertain. All of these factors influence the movement of individual vehicles. Therefore, accurately predicting vehicle parameters, such as speed and trajectory, and applying these predictions to optimize vehicle operation is extremely challenging.

[0004] Data-driven machine learning technology has become an inevitable solution for learning and analyzing vehicle data. However, due to the unique characteristics of vehicle networks, including high mobility, data privacy, communication costs, and high security requirements, applying machine learning to vehicle applications still faces challenges.

[0005] Although vehicles can independently train machine learning models using their own data, the data collected by individual vehicles may be imperfect, which may result in non-robust models whose prediction accuracy may not be robust enough for vehicle applications that require high accuracy, or may even lead to incorrect decisions. Therefore, non-robust machine learning models trained on imperfect data may be unacceptable in vehicle applications. In addition, the data collected by individual vehicles may not be sufficient to train large-scale machine learning models that can be used by vehicles on the road. For example, a vehicle cannot train a machine learning model that is applicable in locations where the vehicle has not traveled. Therefore, training machine learning models independently by individual vehicles is not a practical solution.

[0006] However, uploading the data collected by vehicles to a central server for centralized machine learning model training is impractical due to the enormous communication bandwidth requirements and, most importantly, the widespread threat of sharing private information. Furthermore, different vehicles are equipped with different sensors based on their make, model, size, weight, age, and computing resources. Therefore, the data collected by different vehicles can be highly heterogeneous. Consequently, a central server may not have the capacity to process such heterogeneous data. For example, high-end GPS receivers provide more accurate measurements than low-end GPS receivers. For the same GPS receiver, its accuracy is higher in open areas than in urban areas.

[0007] Recent advances in privacy-preserving federated learning (FL) offer a promising solution. FL is a distributed machine learning technique that allows machine learning models to be trained locally based on the trainer's local data. This ensures data privacy and also addresses communication costs due to zero raw data transfer. Most importantly, FL incorporates data features from collaborative datasets, which allows for robust machine learning model training by eliminating data imperfections contained in individual datasets. Pre-trained robust models can be distributed to distributed devices, such as road vehicles, for use in their prediction tasks.

[0008] FL aims to address two key challenges that distinguish it from traditional machine learning: (1) significant variability in characteristics across vehicles in the network (device heterogeneity) and (2) data that is not identically distributed across the network (statistical heterogeneity).

[0009] FL can be categorized into the vanilla FedAvg algorithm and enhanced FL algorithms, such as FedProx and SCAFFOLD. FedAvg is an iterative learning method. At each iteration, FedAvg first performs model training for E epochs locally on K distributed devices. The devices then communicate their model updates to a central server, where the locally trained models are averaged. Although FedAvg has demonstrated empirical success in homogeneous settings, it does not fully address the underlying challenges associated with heterogeneity. In the context of device heterogeneity, FedAvg does not allow participating devices to perform a variable number of local iterations based on their underlying system constraints; instead, it is common to simply discard devices that fail to complete E epochs within a specified time window. From a statistical perspective, FedAvg has been shown to empirically diverge in settings where data is not identically distributed across devices. Therefore, enhanced FL algorithms, such as FedProx and SCAFFOLD, have been proposed. FedProx is a federated optimization algorithm that addresses the heterogeneity challenge. It adds an additional regularization term to the local objective function to account for heterogeneity. FedProx allows participating devices to perform different iterations of model training. FedProx demonstrates better convergence rates than vanilla FedAvg on non-identically distributed datasets. SCAFFOLD has also been proposed to improve the convergence rate of federated learning. Instead of adding additional terms to the objective function, SCAFFOLD uses control variables to correct for client drift in local updates. SCAFFOLD requires significantly fewer communication rounds and is unaffected by data heterogeneity or client sampling. Furthermore, SCAFFOLD can exploit similarities in client data, resulting in even faster convergence.

[0010] While fluid modeling (FL) offers numerous benefits, its application to vehicular networks still requires addressing numerous challenges. For example, how can locally trained machine learning models be aggregated to achieve robust vehicle trajectory prediction? While existing FL algorithms such as FedProx and SCAFFOLD consider device and data heterogeneity when training machine learning models, model aggregation in FedProx employs the vanilla FedAvg approach, simply averaging locally trained models to produce a global model, while model aggregation in SCAFFOLD uses data size-based averaging. Consequently, FedProx model aggregation fails to consider data at all. Even when SCAFFOLD model aggregation considers data size, it fails to fully exploit the characteristics of different datasets. For example, consider the case of two datasets. Suppose that dataset 1 contains more data samples collected at midnight, while dataset 2 contains fewer data samples collected during the morning rush hour. In this case, dataset 2 is clearly more important than dataset 1 for training the morning rush hour traffic model. However, data size-based model aggregation gives greater weight to dataset 1, preventing it from making accurate decisions. Prediction accuracy is crucial for machine learning models. Even though FedProx and SCAFFOLD exhibit faster convergence rates, they do not guarantee prediction accuracy. Therefore, to obtain robust FL models, new algorithms are needed for both the learning server and the learning agent (the distributed device chosen to train the machine learning model).

[0011] Therefore, there is a need to provide a robust federated learning framework in which both the learning server and the learning agent are provided with the required algorithms to train robust machine learning models for vehicle tasks such as trajectory prediction, and to apply the trained models to road vehicles for their operation optimization, especially with the rising demand for higher automation. Summary of the Invention

[0012] Some embodiments are based on the recognition that modern vehicles are equipped with various sensors to collect data to improve vehicle operation. On the one hand, due to factors such as communication bandwidth limitations, data privacy protection, and security, it is impractical to transmit raw data from all vehicles to a central server for centralized data processing and analysis. On the other hand, the limited amount of data collected by individual vehicles is insufficient to train robust and large-scale machine learning models in a city or state, for example, the vehicle does not know the traffic conditions in places where the vehicle has not traveled. In addition, the data collected by individual vehicles may be imperfect, which may lead to non-robust model training. Therefore, it is necessary to provide a collaborative machine learning method by avoiding raw data transmission and ensuring data privacy.

[0013] To this end, some embodiments of the present invention provide a vehicle federated learning method to train robust machine learning models for accurate motion prediction, wherein a centralized learning server such as a 5G base station (BS) coordinates the federated learning model training and distributes the well-trained machine learning models to road vehicles for their prediction tasks.

[0014] One object of some embodiments is to provide a robust vehicle federated learning method for both a learning server and a learning agent by taking into account data heterogeneity, vehicle heterogeneity, and communication resource heterogeneity. Additionally, another object of some embodiments is to provide accurate vehicle trajectory prediction to optimize vehicle operation.

[0015] Some embodiments are based on the recognition that, unlike traditional vehicle traffic metrics that describe general traffic information such as traffic flow, traffic density, and average traffic speed, vehicle trajectories describe individual vehicle movements. Prediction of vehicle trajectories is crucial for optimal vehicle operation, particularly for automated and autonomous driving.

[0016] Some embodiments are based on the recognition that federated learning is a multi-round machine learning model training process. However, due to high mobility, vehicles may be connected to a connection point (e.g., a 3GPP C-V2X gNodeB or an IEEE DSRC / WAVE roadside unit) for a short period of time. In other words, vehicles may not have time to complete the entire model training process. In addition, due to data heterogeneity, some vehicles may train machine learning models in more iterations, while other vehicles may train machine learning models in fewer iterations. Therefore, the learning server must account for local model heterogeneity in model aggregation.

[0017] Therefore, some embodiments of the present invention apply generalization error, defined as the difference between the true value and the federated learning prediction, as a metric to measure the accuracy of the federated learning algorithm.

[0018] To this end, some embodiments of the present invention provide a variance-based model aggregation method for a learning server by applying an optimal weight simplex to aggregate local models. The weight simplex provides weights for each local model. The weights are calculated using the local data variance (rather than the traditional data size). An optimal weight simplex solution is provided to minimize generalization error, thereby maximizing model accuracy.

[0019] Some embodiments are based on the recognition that the federated learning process uses multiple model parameters, such as the number of local training iterations and the local training time window. These model parameters can be categorized into two types: homogeneous parameters and heterogeneous parameters. Homogeneous parameters describe common characteristics across all tasks. For example, a road map is a common parameter across tasks such as trajectory prediction, speed prediction, and travel time prediction. However, heterogeneous parameters describe specific characteristics of a particular task. For example, a vehicle route is a parameter specific to tasks such as trajectory prediction, speed prediction, and travel time prediction.

[0020] To this end, some embodiments of the present invention adopt a three-module structure within a federated learning framework, where the federated learning framework consists of three interacting modules, each with a unique purpose. First, the graph encoder module encodes the map and vehicle information into a directed graph. Then, the policy head module learns a discrete policy, and the trajectory decoder module decodes the sampled paths into predicted trajectories.

[0021] Therefore, some embodiments of the present invention provide a structure-aware model update method for learning agents to maximize the advantages of heterogeneous updates and minimize the disadvantages. At the beginning of learning, model parameters are divided into a homogeneous set and a heterogeneous set. After each round of global learning, each learning agent performs a homogeneous update on the homogeneous set using the FedAvg algorithm and a heterogeneous update on the heterogeneous set using an algorithm such as FedProx.

[0022] Some embodiments are based on the recognition that the data collected by vehicles depends on location, time, weather, road conditions, special events, etc. Traffic conditions at the same location vary depending on the time of day, weather, etc. Peak traffic conditions are different from off-peak traffic conditions. Traffic conditions on snowy days are different from traffic conditions on sunny days.

[0023] To this end, it is desirable for the selected vehicle agents to group their data into different clusters based on collection location, time, weather, and so on. Consequently, the vehicle agents use different data clusters to train different machine learning models. Vehicle agents do not train models for which they do not have appropriate data. Therefore, vehicle agents only upload trained models to the learning server.

[0024] Therefore, the learning server builds a global model by aggregating locally trained models by considering information including location, time, weather, etc.

[0025] Some embodiments are based on the recognition that the data size, computing resources, and time required for vehicle agents to receive the global model are different. Therefore, the learning server does not require the vehicle agents to perform model training with the same requirements.

[0026] To this end, some embodiments of the present invention allow the learning server to employ a partially trained local model so that some vehicle agents can train the model with more iterations and other vehicle agents can train the model with fewer iterations.

[0027] Some embodiments are based on the recognition that uncertainty exists in the vehicle environment. Therefore, federated learning models must be trained to handle unexpected events such as traffic accidents captured by vehicles on the road.

[0028] According to some embodiments of the present invention, a learning server for training a global machine learning model using vehicle agents via roadside units (RSUs) in a network is provided. The learning server includes: at least one processor; and a memory having stored thereon instructions for a vehicle federated learning method, the instructions causing the at least one processor to execute: selecting a vehicle agent from road vehicles traveling on a road associated with a road map for a global machine learning model; distributing the global machine learning model to the selected vehicle agents via the RSUs, wherein the RSUs are respectively associated with the vehicle agents, wherein the vehicle agents include an onboard computer unit and an onboard sensor configured to collect local data while the vehicle agents are traveling on a current trajectory of the road, wherein the selected vehicle agents locally train the global machine learning model using the onboard computer unit and the collected local data via a structure-aware model training method, wherein the locally trained model is stored as a trained local model; aggregating the trained local models from the selected vehicle agents via a variance-based model aggregation method; and updating the global machine learning model using the aggregated trained local model, wherein the at least one processor continues selecting, distributing, aggregating, and updating until a predetermined number of global training rounds are reached or the learning error stabilizes.

[0029] In addition, another embodiment provides a computer-implemented method for training a global machine learning model using a learning server and vehicle agents via roadside units (RSUs) in a network. The method includes the following steps: selecting a vehicle agent for the global machine learning model from road vehicles traveling on a road associated with a road map; distributing the global machine learning model to the selected vehicle agents via the RSUs, wherein the vehicle agents include an onboard computer unit and an onboard sensor configured to collect local data while the vehicle agents are traveling on a current trajectory on the road, wherein the selected vehicle agents locally train the global machine learning model using the onboard computer unit and the collected local data via a structure-aware model training method, wherein the locally trained model is stored as a trained local model; aggregating the trained local models from the selected vehicle agents via a variance-based model aggregation method; and updating the global machine learning model using the aggregated trained local model, wherein at least one processor continues selecting, distributing, aggregating, and updating until a predetermined number of global training rounds are reached or the learning error stabilizes.

[0030] Therefore, the learning server and the vehicle can interact with each other for model enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The embodiments of the present disclosure will be further described with reference to the accompanying drawings, which are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the embodiments of the present disclosure.

[0032] [ Figure 1 ] Figure 1 Components of a federated learning framework in an Internet of Vehicles according to some embodiments of the present invention are shown;

[0033] [ Figure 2A ] Figure 2A A vehicle federated learning architecture for vehicle mission prediction according to some embodiments of the present invention is shown;

[0034] [ Figure 2B ] Figure 2B An example of functional components of a learning server, a roadside unit, and a vehicle agent in a distributed machine learning platform according to an embodiment of the present invention is shown;

[0035] [ Figure 3 ] Figure 3 illustrating a model aggregation method in a traditional federated learning method according to some embodiments of the present invention;

[0036] [ Figure 4 ] Figure 4 Describing a model aggregation method for vehicle federated learning provided by the present invention according to some embodiments of the present invention;

[0037] [ Figure 5 ] Figure 5 An example of a road segmentation method for dividing a road into different segments according to some embodiments of the present invention is shown;

[0038] [ Figure 6 ] Figure 6 An example of a road segment-based data clustering method for dividing data at a road vehicle agent into clusters according to some embodiments of the present invention is shown;

[0039] [ Figure 7 ] Figure 7 The present invention illustrates a variance-based model aggregation method of a learning server in a vehicle federated learning framework according to some embodiments of the present invention;

[0040] [ Figure 8 ] Figure 8 A method for updating structure-aware model parameters of a learning agent in a vehicle federated learning framework according to some embodiments of the present invention is shown; and

[0041] [ Figure 9 ] Figure 9 The functional blocks of the federated learning model training phase and application phase according to some embodiments of the present invention are shown. DETAILED DESCRIPTION

[0042] The following description provides only exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the present disclosure. Instead, the following description of exemplary embodiments will provide those skilled in the art with a feasible description for implementing one or more exemplary embodiments. It is contemplated that various changes may be made to the function and arrangement of elements without departing from the spirit and scope of the disclosed subject matter set forth in the appended claims.

[0043] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, it will be understood by those skilled in the art that the embodiments may be practiced without these specific details. For example, the systems, processes, and other elements of the disclosed subject matter may be shown as components in block diagram form to avoid obscuring the embodiments in terms of unnecessary details. In other cases, well-known processes, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments. In addition, similar numbers and designations in the various figures indicate similar elements.

[0044] In addition, various embodiments may be described as processes, which may be depicted as flow charts, data flow diagrams, structure diagrams, or block diagrams. Although a flow chart may describe operations as a sequential process, many operations may be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process may terminate when its operations are completed, but may have additional steps not discussed or included in the accompanying drawings. Furthermore, not all operations in any particular described process may appear in all embodiments. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. When a process corresponds to a function, the termination of the function may correspond to the function returning to the calling function or the main function.

[0045] Furthermore, embodiments of the disclosed subject matter may be implemented at least in part manually or automatically. Manual or automatic implementation may be performed or at least assisted by the use of a machine, hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. When implemented in software, firmware, middleware, or microcode, program code or code segments that perform the desired tasks may be stored in a machine-readable medium. A processor may perform the desired tasks.

[0046] Accurate motion prediction is essential to facilitate the development of automated and autonomous vehicles. This knowledge helps drivers make effective driving decisions, alleviating traffic congestion, increasing fuel efficiency, and reducing air pollution. These promising benefits make motion prediction a crucial component of commercial vehicle operations, enabled by advanced driver assistance systems (ADAS), advanced traffic management systems, and intelligent transportation systems (ITS).

[0047] To reap all of the aforementioned benefits, motion prediction must process real-time and historical vehicle data, as well as observations collected by vehicles. For example, onboard global positioning systems (GPS) enable mobility data to be used for motion prediction. This emerging big data can significantly increase data availability in terms of coverage and fidelity, significantly improving data-driven motion prediction.

[0048] The existing technologies in traffic forecasting can be mainly divided into two categories. The first category focuses on parametric methods using models such as autoregressive integrated moving average (ARIMA) models and Kalman filter models. When dealing with traffic that only presents regular changes (for example, recurrent traffic congestion occurring during the morning and evening rush hours), parametric methods can achieve promising forecasting results. However, due to the randomness and nonlinear nature of road traffic, traffic forecasts using parametric methods will deviate from the actual values, especially in sudden traffic. Therefore, instead of fitting traffic data into a mathematical model like parametric methods, an alternative is to use a data-driven machine learning (ML) method. For example, a stacked autoencoder model can be used to learn general traffic flow features for prediction. Long short-term memory (LSTM) recursive neural networks (RNNs) can be used to predict traffic flow, speed and occupancy based on data collected by data collectors. Together with the use of RNNs, convolutional neural networks (CNNs) can also be used to capture potential traffic evolution patterns within the underlying road network.

[0049] While existing technologies focus on using advanced deep learning models for traffic prediction, they all utilize standalone learning models that cannot capture large-scale observations to study traffic variations. In reality, traffic patterns on roads can vary significantly under different circumstances due to changing weather, varying road conditions, and special events. Therefore, standalone models are unable to capture these diverse and complex traffic situations. Furthermore, due to the limited onboard processor power and on-chip memory in vehicles, local training data can be extremely insufficient to achieve promising prediction performance. Most importantly, the data collected by individual vehicles can be incomplete, which can lead to non-robust model training. Furthermore, the collected data can contain personal information. In this case, transmitting data to a centralized server raises privacy concerns. Furthermore, communication costs are another major concern. Therefore, it is necessary to provide a collaborative machine learning architecture that avoids data transmission, considers communication capabilities, and integrates onboard computing resources and local data heterogeneity.

[0050] Figure 1 The components of a federated learning framework 100 in a connected vehicle network are shown. The framework 100 includes a learning server 110, distributed roadside units 120, and road vehicles 130, which serve as potential learning agents. The learning server 110 is connected to the distributed roadside units 120 via a high-speed, reliable communication link 112. The learning server 110 can be located remotely or along a roadside. The learning server 110 configures a machine learning model 115 (stored in memory), referred to as the global model, and aggregates locally trained models 118. The learning server 110 distributes the global model to selected road vehicles for training. Distributed roadside units (RSUs) 120 form the core communication network, associate (connect) road vehicles 125 for service provision, and allocate communication resources 128 to vehicles for model transmission. Most importantly, the RSUs relay communication traffic between the learning server and the vehicles. Road vehicles 130 use their sensors to collect data 136, use their computing resources 135 and local data (local datasets) to train machine learning models 138, and upload the locally trained models to the learning server to construct the global model. The learning server 110 distributes the well-trained machine learning model to all road vehicles 130 via the distributed RSUs 120 for their prediction tasks, such as speed prediction and vehicle-specific power prediction. In this case, the road vehicles 130 and the distributed RSUs 120 communicate wirelessly using downlink communication links 123 and uplink communication links 132.

[0051] Figure 2AA two-tiered vehicle federated learning architecture 200 for vehicle mission prediction is shown, in which a learning server 110 selects an initial machine learning model (e.g., a neural network) and hyperparameters (e.g., a time threshold for completing local training and a time threshold for uploading the locally trained model), selects an initial learning agent to train the model, and distributes the machine learning model and hyperparameters to selected vehicle agents 130 via the RSU 120 for training. At the beginning of each training round, the server distributes the model and hyperparameters to the learning agents for model training. At the end of each training round, the learning server receives the locally trained model and feedback (e.g., the number of local training iterations and communication link quality) from the learning agents. The learning server then aggregates the received local models using a method such as averaging and selects hyperparameters for the next training round. The learning server then selects learning agents and distributes the intermediate models and hyperparameters to the agents for training. In each training round, the selected vehicle agent 130 determines its local training iterations based on the hyperparameters, its computing resources, and its local data size, and then trains the model for the determined number of iterations using its local dataset. Upon completion of local training, the learning agent uploads the trained model to the learning server via the RSU.

[0052] Figure 2B An example of a federated learning platform 201 is shown, which includes functional components 210, 220, and 230 of a learning server, a roadside unit, and a vehicle agent in the distributed machine learning platform 100. The learning server 110 may include an interface (or transceiver) 211 configured to communicate with the learning agent 130 via the RSU 120; one or more processors 212; and a memory / storage 213 configured to store hyperparameters 214, a model aggregation algorithm 215, and a global machine learning model 216. The RSU 120 may include two interfaces (or transceivers) 221 configured to communicate with the learning server 110 via a high-speed reliable link and with the vehicle 130 via a wireless link; one or more processors 222; and a memory / storage 223 configured to store a radio resource allocation algorithm 224, a vehicle-RSU association algorithm 225, and a communication algorithm 226. The vehicle 130 may include: an interface (or transceiver) 231 configured to communicate with the learning server 110 via the RSU 120 via a wireless link; one or more processors 232; sensors 233; and a memory / storage unit 234 configured to store a local data set 235, a machine learning algorithm 236, a machine learning model 237, a machine learning objective function 238, and hyperparameters 239.

[0053] The vehicle agent can complete model training based on different criteria, including (1) the time specified by the learning server, (2) a predetermined number of local training iterations, (3) the local model training error reaching a predetermined threshold, and (4) a stable local model training error.

[0054] Machine learning models can be represented in different ways, for example using a set of model parameters x. For neural network-based machine learning, the model parameters can be represented by a set of neural network weights as x = (x1, x2, ..., x k ).

[0055] Centralized Learning, Traditional Federated Learning, and Problems

[0056] Assume that the network consists of a central server and n distributed clients. The dataset owned by the i-th client is represented as S i , local datasets often differ across clients, global datasets It is defined as the sum of all datasets available to the centralized learning algorithm. Centralized learning aims to find a set of model parameters x that minimizes the loss function or objective function l(x,S) for all clients.

[0057]

[0058] The centralized optimization problem (1) requires uploading all local datasets to a central server, which has two key problems: 1) it requires huge communication bandwidth to upload data and 2) it poses risks in terms of data privacy. Therefore, it is not practical.

[0059] Therefore, Federated Learning (FL) is introduced as a communication-efficient and privacy-preserving framework to solve the optimization problem (1) in a distributed manner. In the decentralized framework, each local client optimizes the loss function on its own local version of the variable, while the central server attempts to seek consensus among all clients. The equivalent decentralized version of Problem (1) can be written as

[0060]

[0061] A round of FL is performed as follows. The global model parameters of the previous round are Sent from the server to all clients, each client tries to use the global model parameters As a starting point to find an algorithm To reduce communication costs, the FL client usually uses the local target as an approximation of the global target to perform multiple optimization steps. After the local calculation is completed, the server collects the updated model parameters from the client and aggregates the collected model parameters to obtain the t-round global model parameters In round t, the server chooses n t A subset of clients to participate in model training. Server aggregation often takes a simplex form The form of weighted average on

[0062]

[0063] Then, the algorithm proceeds to the next round. It can be seen that p i The determination of becomes the key in FL model aggregation.

[0064] The learning server can apply different methods to select vehicle agents, including (1) randomly selecting vehicle agents, (2) selecting vehicle agents that have been connected to the network for longer than a predetermined period of time, (3) selecting vehicles with better link quality to their associated RSUs, (4) selecting vehicles with better performance in the previous round of training, (5) selecting vehicles with a larger dataset, (6) selecting vehicles based on communication resources, and (7) selecting vehicles based on distance from the collected RSUs.

[0065] Federated learning is performed through two main components: a learning server and a learning agent. One of the key functions performed by the learning server is to aggregate locally trained machine learning models with learning agents. However, the previous FedAvg algorithm used simple model averaging aggregation, that is, simply averaging the local models. Figure 3 The FedAvg model aggregation 300 is shown. In round t, the locally trained model 310 is uploaded to the central server, and the central server takes the aggregation weight as 330 to apply the FedAvg model aggregation 320 parameters to obtain the t+1 round global model 340. This aggregation method does not consider the characteristics of the dataset at all, and therefore does not take into account data heterogeneity.

[0066] As a result, the SCAFFOLD algorithm proposes a model aggregation method based on data size to set the aggregation weight to Consider the dataset size. When the clients participating in FL are homogeneous, that is, the dataset S i When all i follow the same distribution, the aggregation weight based on data size produces the best results in terms of excess risk. However, when the dataset S i (i=1,2,…,n t ) do not follow the same distribution, aggregating weights based on data size will not produce optimal results. For example, consider two datasets: dataset 1 contains more data samples collected at midnight, while dataset 2 contains fewer data samples collected during the morning rush hour. To train a model for morning rush hour traffic, dataset 2 is clearly more important than dataset 1. However, aggregating the model based on data size gives a greater weight to dataset 1, which does not produce appropriate aggregation weights.

[0067] Therefore, FL requires new model aggregation methods to find the optimal aggregation weights on heterogeneous datasets.

[0068] Variance-based aggregation of federated learning models

[0069] Since FL algorithms, especially those using momentum-based solvers (e.g., Adam), are difficult to analyze directly, we can simplify the problem by considering the problem of finding the mean of a Gaussian random vector using data from clients. Assume that the total number of clients is n, and the local dataset of client i is denoted as S i , the number of data points in each local client is represented as |S i |. Individual data is represented as where μ is the expected distribution, which can be assumed to be the same across all clients, and the parameter σ i is the standard deviation and the variance of the distribution is The goal of the learning server is to run the FL algorithm to find the best estimate of μ.

[0070] When the data distribution on the client is heterogeneous, finding the optimal aggregation weight is a challenge. Assume that for client i, dataset |S i Distribution of | Meet the following conditions

[0071]

[0072] Assumption (4) assumes that the gradients evaluated at different clients i share the same expectation, while the variance of the gradient varies across agents. This assumption is particularly common in vehicle data, as the traffic dynamics on the road typically remain the same for all vehicles, while the data captured by different vehicles often differ, resulting in different variances in the data.

[0073] The goal is to make the squared error Minimize, where the estimated mean is represented by x. By the simplex p=(p1,...,p n ) to calculate the global estimate. Let x be the global estimate x global =∑ i＝1,...,n p i x i , so the optimal solution to this problem is given by

[0074]

[0075] In this case, algorithm stability is calculated by bounding the generalization error defined as the difference between the true value and the federated learning prediction.

[0076] Theorem 1 For tasks satisfying assumption (4), where the estimated mean is computed by (5), when the weight simplex p = (p1, ..., p n ) takes the following values, the generalization error gen(μ,x global |{S i})minimize

[0077]

[0078] The theorem states that to minimize the generalization error, the optimal aggregation weight is proportional to the size of the local dataset and inversely proportional to the variance of the local dataset.

[0079] Using the optimal aggregation weight given by formula (6), Figure 4 , a variance-based model aggregation method 400 is shown, wherein the variance-based model aggregation 410 uses the model 310 of round t to obtain the global model 420 of round t+1, wherein the variance-based aggregation weight 430 is calculated according to formula (6).

[0080] The results are also consistent with intuition. A dataset with a smaller variance in the data distribution appears more stable and relatively more trustworthy. In this case, the dataset will have a larger aggregation weight.

[0081] For the case of Gaussian variables with a given variance, Theorem 1 guarantees the best-case aggregation weights, which ensures the best possible algorithm stability. In the context of the FL algorithm, the analysis becomes more difficult. Motivated by the theoretical argument of Theorem 1, an estimate of the variance of the dataset can be found.

[0082] Early FL work often used gradient descent on the local client. Recently, driven by the success of momentum and adaptive optimizers in centralized machine learning, FL algorithms have adopted similar approaches, either server-side or client-side, or even on both sides. This paper uses the Adam optimizer as an example to illustrate variance estimation.

[0083] For the Adam optimizer, the variance of the gradient can be calculated as follows. The k-th iteration of Adam is calculated as follows,

[0084]

[0085] Where α represents the step size, β1, β2 represent the exponential decay rate of the moment estimate, and ∈ is the term used in Adam to increase the stability of the algorithm. t Considered as gradient g t The first-order moment of , the variance of the gradient can be estimated as

[0086]

[0087] Use (8) as an estimate of the gradient variance. Figure 7 A variance-based FL model aggregation algorithm is provided in

[15] , which aggregates FL models using optimal aggregation weights.

[0088] Client-side structure-aware federated learning model update

[0089] Variance-based model aggregation allows the FL server to increase algorithmic stability during training. However, FL is a collaborative learning process between the learning server and the learning client. To train robust machine learning models, it is desirable to provide a client-side model update solution for heterogeneous clients.

[0090] To address heterogeneity—that is, device heterogeneity and statistical heterogeneity—and increase learning stability, the FedProx and Scaffold model updating methods have been proposed as modifications to the vanilla FedAvg update. However, these algorithms treat all model parameters as heterogeneous. Extensive empirical experiments show that for homogeneous parameters, these algorithms actually exhibit worse performance than vanilla FedAvg. Furthermore, FedProx uses a simple model aggregation approach, while Scallfold applies data size-based model aggregation. In other words, these algorithms do not use optimal model aggregation weights.

[0091] Therefore, it is desirable to provide a new model updating method that treats model parameters differently, that is, to classify homogeneous model parameters and heterogeneous model parameters in the FL process. Considering the structure of ML models, different layers of complex models often serve different purposes. Taking the convolutional neural network (CNN) in computer vision tasks as an example, it is generally believed that the lower layers of CNN are used as common feature detectors, which can remain unchanged across different tasks, and the last layers are used to learn specific tasks. For vehicle federated learning, the road network is the same for all vehicles, and the traffic flow is also the same for vehicles on the same road. However, the vehicle trajectories, sensors used to collect data, vehicle computing resources, driving destinations, and driver behaviors are different.

[0092] To perform structure-aware model updates (structure-aware model training methods), a three-module architecture is employed in federated learning, with three interacting modules, each with a unique purpose. First, the graph encoder module encodes the map and the vehicles in the vicinity of the learning vehicle into a directed graph. Then, the policy head module learns a discrete policy for each vehicle under consideration, and the trajectory decoder module decodes the sampled paths into predicted trajectories for the learning vehicle.

[0093] In order to maximize the advantages and minimize the disadvantages in heterogeneous FL, such as Figure 8 The client-side structure-aware FL model update method (structure-aware model training method) is shown. At the beginning of the FL process, the model parameters are classified into isomorphic sets or heterogeneous sets After each round of communication, each client uses the FedAvg algorithm to Perform homogeneous model updates using heterogeneous FL algorithms such as FedProx or Scaffold algorithms on the set Perform heterogeneous model updates.

[0094] In order to classify the model parameters, the parameters including road map and traffic flow are classified into sets Road as Figure 5 500 is divided into segments, where the road 500 is divided into three segments 510, 520, and 530. Lane centerlines 540 capture both the direction of traffic flow and the legal routes that individual drivers can follow. A road segment is represented as (x, y, θ, Ι), where x, y are locations, θ is yaw, and Ι is a 2-D binary vector indicating whether the road segment falls on a stop line or a crosswalk. Thus, the road segment representation captures both the geometry as well as the traffic control elements along the lane centerlines. Traffic flow is represented as the number of vehicles on the road segment. Parameters including vehicle trajectories, vehicle destinations, and near-end vehicles are categorized into sets. The trajectory is represented as (x, y, l, v, a, w, Ι), where x, y are the location coordinates, l is the lane number, v, a, w are the velocity, acceleration and yaw rate, and Ι is an indicator with a value of 1 for pedestrians and 0 for vehicles.

[0095] To facilitate federated learning in the Internet of Vehicles, the training data of the learning client can be divided into different clusters, so that each cluster corresponds to a learning model. For example, peak data is used to train a peak model. Data clustering is important for many reasons. For example, off-peak data is not suitable for training a peak traffic model, and local traffic data is not suitable for training a highway traffic model. There are different ways to cluster data. Figure 6 A data clustering method for dividing data at each road vehicle into clusters is shown, wherein the local data 600 of the vehicle is first divided 610 based on road segments and then further divided 620 based on time.

[0096] Figure 9906 , the vehicle clients use their local datasets 908 and the hyperparameters 909 to train the global model and apply the model to the selected vehicle clients. Figure 7 and Figure 8 The variance-based or structure-aware federated learning algorithm provided in

[15] is used to locally train the machine learning model 907. When the local training is completed, the vehicle client uploads the locally trained model to the learning server 909 via the RSU, and the RSU relays the locally trained model to the learning server 910. Upon receiving the locally trained model 911, the learning server uses Figure 4 The variance-based model aggregation shown aggregates the local models and coordinates the next round of training 902.

[0097] Once the machine learning model is well trained, the learning server distributes the model 921 to all road vehicles, which use the trained model to make their multi-level predictions 922. The road vehicles then apply their predictions to their vehicle operations. In addition, the road vehicles can feed their experience back to the learning server for model enhancement.

[0098] The federated learning process can be initiated in different ways 930, such as 1) periodic model training 931: where the learning server initiates periodic model training every day or every week or at other time intervals; 2) event-based model training 932: where the learning server learns information from city management departments about large buildings or large sporting events; and 3) feedback-based model training 933: where the difference between the road vehicle recognition model prediction and the observed true value.

Claims

1. A learning server for training a global machine learning model using vehicle agents via roadside units (RSUs) in a network, the learning server comprising: at least one processor; and a memory storing instructions for a vehicle federated learning method, wherein the instructions cause the at least one processor to execute: selecting the vehicle agent from road vehicles traveling on a road associated with a road map for the global machine learning model; distributing the global machine learning model to selected vehicle agents via the RSUs, wherein the RSUs are associated with the vehicle agents, respectively, wherein the vehicle agents include onboard computer units and onboard sensors configured to collect local data while the vehicle agents travel on a current trajectory on the road, wherein the selected vehicle agents locally train the global machine learning model using the onboard computer units and the collected local data via a structure-aware model training method, wherein the locally trained model is stored as a trained local model; aggregating the trained local models from the selected vehicle agents via a variance-based model aggregation method; and The global machine learning model is updated using the aggregated trained local models, wherein the at least one processor continues the selecting, the distributing, the aggregating, and the updating until a global training round reaches a predetermined number of rounds or a learning error stabilizes.

2. The learning server according to claim 1, wherein In the global training round t, the global machine learning model is represented as a set of global model parameters Among them, the set of global model parameters is distributed to the selected vehicle agents for locally training a distributed global machine learning model using the local dataset of the vehicle agents, wherein the model locally trained by vehicle agent i is represented as 3. The learning server according to claim 2, wherein the selection is performed based on one or a combination of the following: (1) randomly selecting a vehicle agent, (2) selecting a vehicle agent that has been connected to the network for longer than a predetermined period of time, (3) selecting a vehicle agent with better link quality with the associated RSU, (4) selecting a vehicle agent with better performance in a previous global training round, (5) selecting a vehicle agent with a larger data set, (6) selecting a vehicle agent based on communication resources, and (7) selecting a vehicle agent based on a distance from the associated RSU.

4. The learning server according to claim 3, wherein the learning server transmits the set of global model parameters to the associated RSU. distributed to the selected vehicle agents, where The learning server sets the set of global model parameters broadcast to the RSU, which then receives a set of global model parameters Relayed to the associated vehicle agent respectively.

5. The learning server according to claim 1, wherein In the global training round t, the learning server uses the weight simplex p=(p1,...,p n ) The trained local model Aggregate into Where n is the number of selected vehicle agents.

6. The learning server according to claim 5, wherein: While aggregating the trained local models, the learning server applies the variance-based optimal weight simplex p=(p1, ..., p n ) where n is the number of the selected vehicle agents, N i is the number of data samples of vehicle agent i, and is the variance of vehicle agent i.

7. The learning server according to claim 1, wherein After receiving a set of global model parameters When the vehicle agent uses As a starting point, the structure perception model training method is performed, wherein the vehicle agent takes the Partition into isomorphic sets and heterogeneous sets 8. The learning server according to claim 7, wherein: Isomorphism Set The set of global model parameters in is updated using a homogeneous federated learning algorithm such as FedAvg, and the heterogeneous set The set of global model parameters in are updated using a heterogeneous federated learning algorithm such as FedProx.

9. The learning server according to claim 7, wherein: The structure perception model training method uses a graph encoder module, a policy head module and a trajectory decoder module, wherein the graph encoder module is configured to encode the road map, each of the vehicle agents and the proximal vehicle into a directed graph, the policy head module is configured to learn a discrete policy for each of the proximal vehicle and the vehicle agent, and the trajectory decoder module is configured to predict the trajectory of the vehicle agent by decoding the sampled path of the vehicle agent.

10. The learning server according to claim 1, wherein The selected vehicle agent uploads the trained local model to the learning server via the RSU, wherein the selected vehicle agent uploads the trained local model to the currently connected RSU, wherein the RSU relays the received trained local model to the learning server.

11. The learning server according to claim 10, wherein: The selected vehicle agent uploads the trained local model to the learning server based on one or a combination of the following criteria: (1) a time specified by the learning server, (2) a predetermined number of local training iterations, (3) a local model training error reaching a predetermined threshold, and (4) a stable local model training error.

12. The learning server according to claim 1, wherein The selected vehicle agent divides the local data set into different clusters so that each cluster is used to train a specific machine learning model, wherein the local data collected at different locations and different times are used to train the corresponding specific learning model.

13. The learning server according to claim 1, wherein: At least two of the selected vehicle agents collect the local data using two sensors of different types respectively equipped on the at least two of the selected vehicle agents.

14. The learning server according to claim 13, wherein: The two sensors are a high-end GPS receiver and a low-end GPS receiver, wherein the high-end GPS receiver provides measurements that are more accurate than measurements of the low-end GPS receiver.

15. The learning server according to claim 1, wherein The global machine learning model is trained by using a neural network with an adaptive momentum optimizer.

16. The learning server according to claim 1, wherein The training of the global machine learning model is initiated by one or a combination of the following: 1) periodic model training, 2) event-based model training, and 3) feedback-based model training.

17. The learning server according to claim 1, wherein: The learning server distributes the well-trained global machine learning model to all road vehicles for their application, wherein the road vehicles apply the well-trained global machine learning model to the corresponding tasks of the road vehicles, such as trajectory prediction, speed prediction, energy consumption prediction and ADAS / AD parameter calibration.

18. A computer-implemented method for training a global machine learning model using a learning server and vehicle agents via roadside units (RSUs) in a network, the computer-implemented method comprising the following steps: selecting a vehicle agent for the global machine learning model from road vehicles traveling on a road associated with a road map; distributing the global machine learning model to selected vehicle agents via the RSU, wherein the vehicle agents include an onboard computer unit and onboard sensors configured to collect local data while the vehicle agents travel on a current trajectory on the road, wherein the selected vehicle agents locally train the global machine learning model using the onboard computer unit and the collected local data via a structure-aware model training method, wherein the locally trained model is stored as a trained local model; aggregating the trained local models from the selected vehicle agents via a variance-based model aggregation method; and The global machine learning model is updated using the aggregated trained local models, wherein at least one processor continues the selecting, the distributing, the aggregating, and the updating until a global training round reaches a predetermined number of rounds or a learning error stabilizes.

19. The computer-implemented method of claim 18, wherein: In the global training round t, the learning server uses the weight simplex p=(p1,...,p n ) The trained local model Aggregate into Where n is the number of selected vehicle agents.

20. The computer-implemented method of claim 19, wherein: While aggregating the trained local models, the learning server applies the variance-based optimal weight simplex p=(p1, ..., p n ) where n is the number of the selected vehicle agents, N i is the number of data samples of vehicle agent i, and is the variance of vehicle agent i.

Images