How to improve the robustness of federated learning in the Internet of Vehicles

Abstract

A distributed machine learning-based traffic prediction method for predicting traffic on a road is provided, comprising the steps of: a learning server distributing a global multi-task traffic model to learning agents to locally train a traffic model; the learning agents uploading the locally trained traffic model to the learning server; the learning server updating the global multi-task traffic model using locally trained traffic model parameters obtained from the learning agents; the learning server generating a time-dependent global traffic map using the fully trained global multi-task traffic model; distributing the time-dependent global traffic map to vehicles traveling on a road; and calculating, based on a driving plan, an optimal driving route by the vehicles that minimizes driving time using the time-dependent global traffic map.

Description

[Technical Field]

[0001] The present invention relates generally to distributed machine learning for vehicular traffic systems, and more particularly to a method and apparatus for federated learning in a vehicular network. [Background technology]

[0002] To achieve higher levels of automation, modern vehicles are packed with a variety of on-board sensors. Unlike traditional vehicles, modern vehicles are much smarter. Modern vehicles are not only capable of collecting a variety of vehicle and traffic data, but also capable of running advanced machine learning algorithms to guide their movements.

[0003] However, achieving intelligent transportation is an extremely challenging problem. Physical roads form complex road networks. Most importantly, traffic conditions, such as congestion in one location, can propagate and affect traffic conditions in other locations. Furthermore, unexpected events, such as traffic accidents and driver behavior, can make traffic conditions even more dynamic and uncertain. All these factors can affect the behavior of individual vehicles. Therefore, how to accurately predict vehicle parameters, such as speed and trajectory, and then apply this prediction to optimize vehicle operation is extremely challenging.

[0004] Data-driven machine learning techniques have become an inevitable solution for learning and analyzing vehicular data. However, applying machine learning to vehicular applications still faces challenges due to the inherent characteristics of vehicular networks, including high mobility, data privacy, communication costs, and high safety requirements.

[0005] Although vehicles can independently train machine learning models using their own data, the data collected by individual vehicles may contain imperfections, resulting in non-robust models, whose prediction accuracy may not be robust for vehicle applications requiring high accuracy and may even result in incorrect decision-making. Therefore, non-robust machine learning models trained based on imperfect data are unacceptable for vehicle applications. In addition, the data collected by individual vehicles may not be sufficient to train large-scale machine learning models that can be used by vehicles on the road. For example, a vehicle cannot train a machine learning model that can be applied in a location it has never driven in. Therefore, independently training machine learning models by individual vehicles is not a practical solution.

[0006] However, uploading data collected by vehicles to a central server for centralized machine learning model training is impractical due to the enormous communication bandwidth requirements and, most importantly, the widespread threat of sharing personal information. Additionally, different vehicles are equipped with different sensors based on their make, model, size, weight, lifespan, and computational resources. Therefore, the data collected by different vehicles can be highly heterogeneous. As a result, the central server may not have the capacity to process such heterogeneous data. For example, a high-end GPS receiver provides more accurate measurements than a low-end GPS receiver. The same GPS receiver is more accurate in open areas than in urban areas.

[0007] Recent advances in privacy-preserving federated learning (FL) can provide a promising solution. FL is a distributed machine learning technique that enables machine learning models to be trained locally based on the trainer's local data. FL therefore ensures data privacy and addresses the communication cost issue by requiring zero raw data transfer. Most importantly, FL incorporates data features from collaborative datasets, thereby enabling robust machine learning model training by eliminating data imperfections contained in individual datasets. Pre-trained robust models can be distributed to distributed vehicles, such as road vehicles, for prediction tasks.

[0008] FL aims to address two key challenges that distinguish it from traditional machine learning: (1) significant variation in terms of characteristics for each vehicle in the network (device heterogeneity) and (2) non-uniformly distributed data across the network (statistical heterogeneity).

[0009] FL can be divided into the general FedAvg algorithm and enhanced FL algorithms such as FedProx and SCAFFOLD. FedAvg is an iterative learning method. In each iteration, FedAvg first performs E epochs of model training locally on K distributed devices. These devices then communicate model updates to a central server, and the locally trained models are averaged. While FedAvg has demonstrated empirical success in homogeneous settings, it does not adequately address the fundamental challenges associated with heterogeneity. In the context of device heterogeneity, FedAvg does not allow participating devices to perform a variable amount of local iterations based on their underlying system constraints. Instead, it is common to simply exclude devices that cannot complete E epochs within a given time window. From a statistical perspective, FedAvg has been empirically found to diverge in settings where data is non-identically distributed across devices. Therefore, enhanced FL algorithms such as FedProx and SCAFFOLD have been proposed. FedProx is a federated optimization algorithm that addresses the challenge of heterogeneity. FedProx adds an additional regularization term to the local objective function to account for heterogeneity. FedProx allows participating devices to perform different iterations of model training. FedProx exhibits better convergence rates on non-uniformly distributed datasets than the popular FedAvg. SCAFFOLD is also proposed to improve the convergence rate of federated learning. Instead of adding an additional term to the objective function, SCAFFOLD uses control variables to correct client drift in local updates. SCAFFOLD requires significantly fewer communication rounds and is not affected by data heterogeneity or client sampling. Furthermore, SCAFFOLD can exploit similarities in clients' data, resulting in even faster convergence.

[0010] While FL can indeed bring benefits to manifolds, applying it to vehicular networks still requires addressing many issues. For example, how do we aggregate locally trained machine learning models to achieve robust vehicle trajectory prediction? Existing FL algorithms, such as FedProx and SCAFFOLD, train machine learning models while taking device and data heterogeneity into account. However, model aggregation in FedProx still applies the common FedAvg approach, which simply averages locally trained models to obtain a global model, while model aggregation in SCAFFOLD uses data-size-based averaging. As a result, FedProx model aggregation does not consider data at all. While SCAFFOLD model aggregation does consider data size, this method does not fully explore the characteristics of non-identical datasets. As an example, consider the case of two datasets. Dataset 1 contains more data samples collected at night, while dataset 2 contains fewer data samples collected during the morning rush hour. In this case, Dataset 2 is clearly more important than Dataset 1 for training a morning rush-hour traffic model. However, data-size-based model aggregation gives a larger weight to Dataset 1, which is incorrect. Prediction accuracy is key for machine learning models. FedProx and SCAFFOLD show faster convergence rates, but do not guarantee prediction accuracy. Therefore, to obtain a robust FL model, novel algorithms are required for both the learning server and the learning agent, which are the distributed devices selected to train the machine learning model.

[0011] Therefore, especially with the increasing demand for greater automation, there is a need to provide a robust federated learning framework in which the learning server and learning agents both have the necessary algorithms to train robust machine learning models for vehicle tasks such as trajectory prediction and apply the trained models to on-road vehicles for performance optimization. Summary of the Invention

[0012] Some embodiments recognize that modern vehicles are equipped with various sensors to collect data to improve vehicle operation. However, due to factors such as communication bandwidth limitations, data privacy protection, and security, it is impractical to transfer raw data from all vehicles to a central server for centralized data processing and analysis. Meanwhile, the limited amount of data collected by individual vehicles is not sufficient to train robust, large-scale machine learning models in a city or a country. For example, vehicles do not know the traffic conditions in places they have not yet traveled. In addition, the data collected by individual vehicles may contain imperfections, which may result in non-robust model training. Therefore, there is a need to provide a collaborative machine learning method by avoiding raw data transfer and ensuring data privacy.

[0013] To that end, some embodiments of the present invention provide a vehicular federated learning method for training robust machine learning models for accurate motion prediction, in which a centralized learning server, such as a 5G base station (BS), coordinates the federated learning model training and distributes well-trained machine learning models to on-road vehicles for prediction tasks.

[0014] One objective of some embodiments is to provide a robust vehicle federation learning method for both a learning server and a learning agent that takes into account data heterogeneity, vehicle heterogeneity, and communication resource heterogeneity. Furthermore, another objective of some embodiments is to provide accurate vehicle trajectory prediction for optimizing vehicle operation.

[0015] Some embodiments are based on the recognition that, unlike traditional vehicle traffic metrics that represent general traffic information such as traffic flow, traffic density, and average traffic speed, vehicle trajectories represent individual vehicle movements. Predicting vehicle trajectories is crucial to achieving optimal vehicle operation, especially in automated and autonomous driving.

[0016] Some embodiments recognize that federated learning is a multi-round machine learning model training process. However, due to high mobility, vehicles may only connect to a connection point, such as a 3GPP C-V2X gNodeB or an IEEE DSRC / WAVE roadside unit, for a short time. In other words, vehicles may not have time to complete the entire model training process. In addition, due to data heterogeneity, some vehicles may train their machine learning models with more iterations, while others may train their machine learning models with fewer iterations. Therefore, the learning server must consider local model heterogeneity when aggregating models.

[0017] Therefore, some embodiments of the present invention apply generalization error, defined as the difference between the ground truth and the federated learning prediction, as a metric to measure the accuracy of a federated learning algorithm.

[0018] To this end, some embodiments of the present invention provide a distribution-based model aggregation method to the training server by applying an optimal weight simplex to aggregate local models. The weight simplex provides weights for each local model. The weights are calculated using local data distributions instead of traditional data sizes. The optimal weight simplex solution is provided to maximize model accuracy by minimizing generalization error.

[0019] Some embodiments are based on the recognition that the associative learning process takes multiple model parameters, such as the number of local training iterations and the local training time window. These model parameters can be classified into two categories: homogeneous parameters and heterogeneous parameters. Homogeneous parameters represent common features for all tasks. For example, a road map is a common parameter for tasks such as trajectory prediction, speed prediction, and travel time prediction. However, heterogeneous parameters represent specific features for a particular task. For example, a vehicle route is a specific parameter for tasks such as trajectory prediction, speed prediction, and travel time prediction.

[0020] To that end, some embodiments of the present invention incorporate a three-module structure into the federated learning framework, in which the federated learning framework is composed of three interacting modules, each with a unique purpose: First, a graph encoder module encodes map and vehicle information as a directed graph, then a policy header module learns a discrete policy, and the sampled path is decoded into a predicted trajectory by a trajectory decoder module.

[0021] Therefore, some embodiments of the present invention provide a structure-aware model updating method to learning agents to maximize the benefits and minimize the disadvantages of heterogeneous updating. At the start of learning, the model parameters are divided into homogeneous and heterogeneous sets. After each global learning round, each learning agent performs homogeneous updating using the FedAvg algorithm on the homogeneous set and heterogeneous updating using an algorithm such as FedProx on the heterogeneous set.

[0022] Some embodiments are based on the recognition that data collected by vehicles depends on location, time, weather, road conditions, special events, etc. Even in the same location, traffic conditions vary based on different times, different weather, etc. Rush hour traffic conditions are different from off-hour traffic conditions. Traffic conditions on a snowy day are different from traffic conditions on a sunny day.

[0023] To this end, it is desirable for selected vehicle agents to separate their data into different clusters based on collection location, time, weather, etc. As a result, the vehicle agents use different data clusters to train different machine learning models. The vehicle agents will not train models for which they do not have suitable data. Therefore, the vehicle agents simply upload the trained models to the learning server.

[0024] Thus, the learning server builds a global model by aggregating locally trained models taking into account information including location, time, weather, etc.

[0025] Some embodiments are based on the recognition that the data size, computational resources, and time at which the vehicle agents receive the global model are different, and therefore the learning server does not require the vehicle agents to perform model training with identical requirements.

[0026] To that end, some embodiments of the present invention allow the learning server to incorporate partially trained local models, such that some vehicle agents train the model with more iterations and other vehicle agents train the model with fewer iterations.

[0027] Some embodiments are based on the recognition that the vehicular environment is uncertain, and therefore federated learning models must be trained to deal with unexpected events, such as traffic accidents, captured by road vehicles.

[0028] According to some embodiments of the present invention, a learning server for training a global machine learning model using vehicle agents via Roadside Units (RSUs) in a network is provided, the learning server including at least one processor and a memory storing instructions for a vehicle federation learning method, the instructions causing the at least one processor to execute the following: selecting vehicle agents from road vehicles traveling on a road associated with a road map for the global machine learning model; and distributing the global machine learning model to the selected vehicle agents via the RSUs, the RSUs being associated with the vehicle agents, each of the vehicle agents including an on-board computer unit and an on-board sensor configured to collect local data while the vehicle agent travels on a current trajectory of the road, the selected vehicle agents communicating with the on-board computer unit and the collecting data. and locally train the global machine learning model using the selected local data according to a structure-aware model training method, wherein the locally trained model is stored as a trained local model; the instructions further cause the at least one processor to aggregate the trained local models from the selected vehicle agents according to a distribution-based model aggregation method and update the global machine learning model using the aggregated trained local model, wherein the at least one processor continues the selecting, distributing, aggregating, and updating until a predetermined number of global training rounds has been reached or a learning error has stabilized.

[0029] Yet another embodiment provides a computer-implemented method for training a global machine learning model using a learning server and vehicle agents via roadside units (RSUs) in a network. The method includes selecting vehicle agents from road vehicles traveling on a road associated with a road map for the global machine learning model; and distributing the global machine learning model to the selected vehicle agents via the RSU, the vehicle agents including an on-board computing unit and an on-board sensor configured to collect local data while the vehicle agents travel on a current trajectory of the road, the selected vehicle agents locally training the global machine learning model using the on-board computing unit and the collected local data by a structure-aware model training method, and the locally trained model is stored as a trained local model, the method further includes aggregating the trained local models from the selected vehicle agents by a distribution-based model aggregation method, and updating the global machine learning model using the aggregated trained local model, the at least one processor continuing the selecting, distributing, aggregating, and updating until a predetermined number of global training rounds is reached or a learning error stabilizes.

[0030] Thus, the learning server and the vehicle can interact with each other for model enhancement. [Brief explanation of the drawings]

[0031] The presently disclosed embodiments are further described with reference to the accompanying drawings, in which: The drawings shown are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the presently disclosed embodiments.

[0032] [Figure 1]FIG. 1 illustrates components of a federated learning framework in an Internet of Vehicles, according to some embodiments of the present invention. [Figure 2A] FIG. 1 illustrates a vehicle federated learning architecture for vehicle task prediction, according to some embodiments of the present invention. [Figure 2B] FIG. 1 is a diagram illustrating an example of functional components of a learning server, a roadside device, and a vehicle agent in a distributed machine learning platform according to an embodiment of the present invention. [Figure 3] FIG. 1 illustrates a model aggregation approach in a conventional federated learning method, according to some embodiments of the present invention. [Figure 4] FIG. 1 illustrates a model aggregation approach for vehicular federated learning provided by the present invention, according to some embodiments of the present invention. [Figure 5] FIG. 2 illustrates an example of a road segmentation method for dividing a road into different sections according to some embodiments of the present invention. [Figure 6] FIG. 2 illustrates an example of a road segment-based data clustering method for clustering data at on-road vehicle agents, according to some embodiments of the present invention. [Figure 7] FIG. 1 illustrates a distribution-based model aggregation method for a learning server in a vehicular federated learning framework, according to some embodiments of the present invention. [Figure 8] FIG. 1 illustrates a structure-aware model parameter update method for a learning agent in a vehicle federation learning framework, according to some embodiments of the present invention. [Figure 9] FIG. 2 illustrates functional blocks for federated learning model training and application phases according to some embodiments of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0033] The following description provides exemplary embodiments only and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with an enabling description for implementing one or more exemplary embodiments. Contemplated are various changes that may be made in the function and arrangement of elements without departing from the spirit and scope of the disclosed subject matter as set forth in the appended claims.

[0034] Specific details are set forth in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that the embodiments may be practiced without these specific details. For example, systems, processes, and other elements of the disclosed subject matter may be shown as components in block diagram form so as not to obscure the embodiments in unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail so as to avoid obscuring the embodiments. Furthermore, like reference numbers and names in the various drawings indicate like elements.

[0035] Also, particular embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. While a flowchart may describe operations as a sequential process, many of these operations can be performed in parallel or simultaneously. Additionally, the order of these operations may be rearranged. A process may terminate when its operations are completed, but may have additional steps not discussed or included in the diagram. Moreover, not all operations in any specifically described process are performed in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, etc. When a process corresponds to a function, the end of the function may correspond to the function returning to the calling function or the main function.

[0036] Furthermore, embodiments of the disclosed subject matter may be implemented at least in part manually or automatically. The manual or automatic implementation may be performed or at least assisted by the use of machines, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored on a machine-readable medium. A processor may perform the necessary tasks.

[0037] To facilitate the development of automated and autonomous vehicles, having accurate motion prediction is essential due to the fact that such knowledge can help drivers make effective driving decisions to alleviate traffic congestion, improve fuel efficiency, and reduce air pollution. These promising benefits enable motion prediction to play a major role in advanced driver-assistance systems (ADAS), advanced traffic management systems, and commercial vehicle operations that Intelligent Transportation Systems (ITS) aim to enable.

[0038] To take advantage of all the above benefits, motion prediction must process real-time and historical vehicle data and observations collected by vehicles. For example, in-vehicle global positioning systems enable mobility data to be used for motion prediction. Such emerging big data can substantially improve data availability in terms of coverage and fidelity, greatly enhancing data-driven motion prediction.

[0039] Prior art on traffic forecasting can be grouped into two main categories. The first category focuses on using parametric approaches, such as the autoregressive integrated moving average (ARIMA) model and the Kalman filtering model. Parametric approaches can achieve promising prediction results when dealing with traffic that only exhibits regular fluctuations, such as recurrent traffic congestion that occurs during morning and evening rush hours. However, due to the stochastic and nonlinear nature of road traffic, traffic forecasts using parametric approaches may deviate from actual values, especially for sudden traffic events. Therefore, instead of fitting traffic data to mathematical models as is done by parametric approaches, an alternative method is to use data-driven machine learning (ML)-based methods. For example, a stacked autoencoder model can be used to learn general traffic flow features for prediction. A long short-term memory (LSTM) recurrent neural network (RNN) can be used to predict traffic flow, speed, and occupancy based on data collected by data collectors. Along with the use of RNNs, Convolutional Neural Networks (CNNs) can also be utilized to capture potential traffic evolution patterns within the underlying road network.

[0040] Prior art focuses on using advanced deep learning models for traffic prediction, but they all study traffic fluctuations using independent learning models that cannot incorporate large-scale observations. In reality, traffic patterns on roads can vary significantly under different circumstances due to changing weather, changing road conditions, and special events. Therefore, using independent models cannot incorporate such diverse and complex traffic situations. Furthermore, due to the limited on-board processor power and on-chip memory in vehicles, local training data may be highly insufficient, preventing promising prediction performance. Most importantly, data collected by individual vehicles may contain imperfections, resulting in non-robust model training. Meanwhile, collected data may contain personal information. In this case, transferring data to a central server may raise privacy concerns. Communication costs, on the other hand, are another major concern. Therefore, it is necessary to provide a collaborative machine learning architecture by avoiding data transfer, considering communication capabilities, and integrating on-board computing resources with local data heterogeneity.

[0041] FIG. 1 illustrates components of a federated learning framework 100 for the Internet of Vehicles. The framework 100 includes a learning server 110, distributed roadside units 120, and road vehicles 130, which are potential learning agents. The learning server 110 is connected to the distributed roadside units 120 via a reliable, high-speed communication link 112. The learning server 110 may be located remotely or along the roadside. The learning server 110 constructs (115) machine learning models (stored in memory), termed global models, and aggregates (118) locally trained models. The learning server 110 distributes these global models to selected road vehicles for training. Distributed roadside units (RSUs) 120 form a core communication network, associate (connect) (125) road vehicles for service provision, and allocate communication resources to the vehicles for model transmission (128). Most importantly, the RSUs relay communication traffic between the learning server and the vehicles. The road vehicles 130 use sensors 136 to collect data, train 138 machine learning models using computational resources 135 and local data (local datasets), and upload the locally trained models to the learning server 110 to build a global model. The learning server 110 distributes the fully trained machine learning models to all the road vehicles 130 via the distributed RSUs 120 for prediction tasks such as speed prediction and vehicle-specific power prediction. In this case, the road vehicles 130 and the distributed RSUs 120 communicate wirelessly using downlink communication links 123 and uplink communication links 132.

[0042] FIG. 2A illustrates a two-tiered vehicle federation learning architecture 200 for vehicle task prediction. The learning server 110 selects initial machine learning models, such as neural networks, and hyperparameters, such as a time threshold for terminating local training and a time threshold for uploading the locally trained model. The learning server 110 selects initial learning agents to train these models and distributes the machine learning models and hyperparameters to the selected vehicle agents 130 via the RSU 120 for training. At the beginning of each training round, the server distributes the models and hyperparameters to the learning agents for model training. At the end of each training round, the learning server receives the locally trained models and feedback, such as the number of local training iterations and communication link quality, from the learning agents. The learning server then aggregates the received local models using a method such as averaging and selects hyperparameters for the next training round. The learning server then selects learning agents and distributes the provisional models and hyperparameters to the agents for training. In each training round, the selected vehicle agents 130 determine their local training iterations based on the hyperparameters, their computational resources, and their local data size, and then use their local datasets to train the model for the determined number of iterations. Upon finishing the local training, the learning agents upload the trained model to the learning server via the RSU.

[0043] 2B illustrates an example of a federated learning platform 201 including functional components 210, 220, and 230 of a learning server, a roadside unit, and a vehicle agent, respectively, in a distributed machine learning platform 100. The learning server 110 may include an interface (or transceiver) 211 configured to communicate with the learning agent 130 via the RSU 120, one or more processors 212, and memory / storage 213 configured to store hyperparameters 214, a model aggregation algorithm 215, and a global machine learning model 216. The RSU 120 may include two interfaces (or transceivers) 221 configured to communicate with the learning server 110 via a reliable high-speed link and with the vehicle 130 via a wireless link, one or more processors 222, and memory / storage 223 configured to store a wireless resource allocation algorithm 224, a vehicle-RSU association algorithm 225, and a communication algorithm 226. The vehicle 130 may include an interface (or transceiver) 231 configured to communicate with the learning server 110 via the RSU 120 over a wireless link, one or more processors 232, sensors 233, and memory / storage 234 configured to store a local dataset 235, a machine learning algorithm 236, a machine learning model 237, a machine learning objective function 238, and hyperparameters 239.

[0044] The vehicle agent can complete model training based on different criteria, including (1) a time specified by the learning server, (2) a predetermined number of local training iterations, (3) the local model training error reaching a predetermined threshold, and (4) the local model training error stabilizing.

[0045]

number

[0046]

number

[0047] The centralized optimization problem (1) requires uploading all local datasets to a central server, which has two important problems: 1) it requires huge communication bandwidth to upload data, and 2) it puts data privacy at risk. Therefore, it is not practical.

[0048] Therefore, federated learning (FL) is introduced as a communication-efficient and privacy-preserving framework for solving the optimization problem (1) in a distributed manner. In a decentralized framework, each of the local clients optimizes a loss function over their own local versions of the variables, while a central server tries to find consensus among all clients, and the equivalent decentralized version of problem (1) can be written as follows:

number

[0049]

number

[0050]

number

[0051] The learning server can apply different methods to select vehicle agents, including (1) randomly selecting vehicle agents, (2) selecting vehicle agents that have been connected to the network for longer than a predetermined period of time, (3) selecting vehicles with better link quality to their associated RSUs, (4) selecting vehicles with better performance in the previous training round, (5) selecting vehicles with a larger dataset, (6) selecting vehicles based on commutation resources, and (7) selecting vehicles based on the distance to the collected RSUs.

[0052]

number

[0053]

number

[0054] Therefore, a new model aggregation method is needed for FL to find optimal aggregation weights across heterogeneous datasets.

[0055] Distribution-based federated learning model aggregation

[0056]

number

[0057]

number

[0058] Assumption (4) assumes that gradients evaluated at different clients i share the same expected value, but the variance of the gradients differs across agents. This assumption is particularly common for vehicle data because while the traffic dynamics on the road generally remain the same for all vehicles, the data captured by different vehicles tend to be different, resulting in different variances in the data.

[0059]

number

[0060] In this case, algorithmic stability can be calculated by bounding the generalization error, defined as the difference between the ground truth and the federated learning prediction.

[0061]

number

[0062] According to this theorem, to minimize the generalization error, the optimal aggregation weights are proportional to the local dataset size and inversely proportional to the variance of the local dataset.

[0063] Using the optimal aggregation weights given by equation (6), a distribution-based model aggregation method 400 is shown in FIG. 4, where the model 310 at round t is used by distribution-based model aggregation 410 to obtain a round t+1 global model 420, in which distribution-based aggregation weights 430 are calculated according to equation (6).

[0064] The results are also intuitive: a dataset with less variance in the data distribution will appear more stable and relatively reliable, in which case it will have a heavier aggregation weight.

[0065] For Gaussian variables with a given variance, Theorem 1 guarantees the best possible aggregation weights that guarantee the best possible algorithmic stability. In the context of the FL algorithm, the analysis becomes much more difficult. Motivated by the theoretical validity of Theorem 1, we can find an estimate of the variance in a dataset.

[0066] Previous FL research often uses gradient descent for local clients. Recently, motivated by the success of momentum and adaptive optimizers in centralized machine learning, FL algorithms have also adapted similar methods for either server-side updates or client-side updates, or even both. This invention uses the Adam optimizer as an example to illustrate variance estimation.

[0067]

number

[0068] (8) is used as an estimate of the gradient variance.,The variance-based FL model aggregation algorithm, which,aggregates FL models using optimal aggregation weights,is shown in Fig. 7.

[0069] Client-side structure-aware federated learning model updating

[0070] Distributed model aggregation allows the FL server to improve the algorithmic stability of the training process. However, FL is a collaborative learning process between the learning server and the learning clients. To train robust machine learning models, it is desirable to provide a client-side model update scheme for heterogeneous clients.

[0071] To address heterogeneity, i.e., device heterogeneity and statistical heterogeneity, and to improve training stability, the FedProx and Scaffold model update methods have been proposed, which serve as an improved version of the general FedAvg update. However, these algorithms consider all model parameters as heterogeneous. Extensive empirical experiments have shown that for homogeneous parameters, these algorithms actually perform worse than the general FedAvg. In addition, FedProx uses a simple model aggregation method, and Scaffold applies data size-based model aggregation. In other words, these algorithms do not use optimal model aggregation weights.

[0072] Therefore, it is desirable to provide a new model updating method that treats model parameters differently, i.e., classifies homogeneous and heterogeneous model parameters in the FL process. Considering the structure of ML models, different layers of a complex model often serve different purposes. Taking convolutional neural networks (CNNs) in computer vision tasks as an example, it is commonly believed that the lower layers of a CNN serve as common feature detectors that can remain invariant across different tasks, while the final layers are used to learn specific tasks. In vehicle federation learning, the road network is the same for all vehicles, and the traffic flow is also the same for vehicles on the same road. However, the vehicle trajectories, sensors used to collect data, vehicle computational resources, driving destinations, and driver behaviors are different.

[0073] To perform structure-aware model updating (structure-aware model training method), a three-module structure with three interacting modules, each with a unique purpose, is adopted in federated learning. First, a graph encoder module encodes the map and nearby vehicles of the training vehicle as a directed graph, then a policy header module learns a discrete policy for each vehicle under consideration, and the sampled paths are decoded into predicted trajectories of the training vehicle by a trajectory decoder module.

[0074]

number

[0075]

number

[0076] To facilitate federated learning in the Internet of Vehicles, the training data of a learning client can be divided into different clusters, with each cluster corresponding to a learning model, e.g., rush-hour data being used to train a rush-hour model. Data clustering is important for many reasons. For example, off-hour data is undesirable for training a rush-hour traffic model, and local traffic data is unsuitable for training a highway traffic model. There are various methods for clustering data. Figure 6 illustrates a data clustering method used to divide the data for each on-road vehicle into clusters, where the vehicle's local data 600 is first divided based on road segment (610) and then further divided based on time (620).

[0077] FIG. 9 is a diagram illustrating functional blocks of the federated learning training phase and application phase according to some embodiments of the present invention, where block 900 illustrates a model training process and block 920 illustrates a model application process. For model training, the learning server initiates the learning process by selecting a machine learning model (901). The learning server then coordinates multi-round distributed model training (902). To do so, the learning server selects vehicle clients and model training hyperparameters (903). The learning server distributes the global machine learning model and hyperparameters to the selected vehicle clients via the RSUs (904), which then relay the model and hyperparameters to the selected vehicle clients (905). Upon receiving the global model and hyperparameters (906), the vehicle clients locally train the machine learning model using their local dataset 908 and the distribution-based federated learning algorithm or the structure-aware federated learning algorithm provided in FIGS. 7 and 8 (907). Once local training is complete, the vehicle client uploads the locally trained model to the learning server via the RSU (909), and the RSU relays the locally trained model to the learning server (910). Upon receiving the locally trained model (911), the learning server aggregates the local model using the distribution-based model aggregation shown in FIG. 4 and coordinates the next training round (902).

[0078] Once the machine learning models are sufficiently trained, the learning server distributes the models to all road vehicles 921, which use these trained models to make multi-horizon predictions 922. The road vehicles can then feed their experience back to the learning server to strengthen the models.

[0079] The federated learning process can begin in various ways 930, such as: 1) periodic model training 931: the learning server initiates periodic model training daily or weekly or at other intervals; 2) event-based model training 932: the learning server learns information from city management about large buildings or major sporting events; and 3) feedback-based model training 933: on-road vehicles identify differences between model predictions and observed ground truth.

Claims

1. A learning server for training a global machine learning model using vehicle agents via roadside units (RSUs) in a network, comprising: at least one processor; and a memory having stored thereon instructions for a vehicle association learning method, the instructions causing the at least one processor to: selecting the vehicle agent from road vehicles traveling on roads associated with a road map with respect to the global machine learning model; and distributing the global machine learning model to the selected vehicle agents via the RSUs, wherein the RSUs are respectively associated with the vehicle agents, the vehicle agents including an on-board computer unit and an on-board sensor configured to collect local data while the vehicle agents travel on a current trajectory of the road, the selected vehicle agents locally training the global machine learning model using the on-board computer unit and the collected local data by a structure-aware model training method, and the locally trained model is stored as a trained local model, and the instructions further cause the at least one processor to: aggregating the trained local models from the selected vehicle agents by a distribution-based model aggregation method; and updating the global machine learning model using the aggregated trained local models, wherein the at least one processor continues the selecting, distributing, aggregating, and updating until a global training round reaches a predetermined number of rounds or a learning error stabilizes. 【Request Item 2】 【Number 1】

3. 3. The learning server of claim 2, wherein the selecting is performed based on one or a combination of: (1) randomly selecting a vehicle agent; (2) selecting a vehicle agent that has been connected to the network for longer than a predetermined period of time; (3) selecting a vehicle agent with better link quality to the associated RSU; (4) selecting a vehicle agent with better performance in a previous global training round; (5) selecting a vehicle agent with a larger dataset; (6) selecting a vehicle agent based on commutation resources; and (7) selecting a vehicle agent based on distance to the associated RSU. [Request Item 4] [Number 2] [Request Item 5] [Number 3] [Request Item 6] [Number 4] [Request Item 7] [Number 5] [Request Item 8] [Number 6]

9. 8. The learning server of claim 7, wherein the structure-aware model training method uses a graph encoder module configured to encode the road map, each of the vehicle agents and proximal vehicles into a directed graph, a policy header module configured to learn a discrete policy for each of the vehicle agents and proximal vehicles, and a trajectory decoder module configured to predict a trajectory of the vehicle agent by decoding a sampled path of the vehicle agent.

10. 2. The learning server of claim 1, wherein the selected vehicle agent uploads the trained local model to the learning server via the RSU, the selected vehicle agent uploads the trained local model to a currently connected RSU, and the RSU relays the received trained local model to the learning server.

11. 11. The learning server of claim 10, wherein the selected vehicle agent uploads the trained local model to the learning server based on one or a combination of the following criteria: (1) a time specified by the learning server; (2) a predetermined number of local training iterations; (3) a local model training error reaching a predetermined threshold; and (4) a local model training error stabilizing.

12. 2. The learning server of claim 1, wherein the selected vehicle agent divides the local dataset into different clusters such that each cluster is used to train a specific machine learning model, and the local data collected at different locations and different times is used to train the corresponding specific machine learning model.

13. 2. The learning server of claim 1, wherein at least two of the selected vehicle agents collect the local data using two sensors of different types respectively equipped on the at least two of the selected vehicle agents.

14. 14. The learning server of claim 13, wherein the two sensors are a high-end GPS receiver and a low-end GPS receiver, the high-end GPS receiver providing more accurate measurements than the low-end GPS receiver.

15. The learning server of claim 1 , wherein the global machine learning model is trained using a neural network with an adaptive momentum optimizer.

16. 2. The learning server of claim 1, wherein training of the global machine learning model is initiated by one or a combination of: 1) periodic model training; 2) event-based model training; and 3) feedback-based model training.

17. 2. The learning server of claim 1, wherein the learning server distributes a well-trained global machine learning model to all road vehicles according to their applications, and the road vehicles apply the well-trained global machine learning model to their respective tasks, such as trajectory prediction, speed prediction, energy consumption prediction, and ADAS / AD parameter calibration.

18. 1. A computer-implemented method for training a global machine learning model using a learning server and vehicle agents via roadside units (RSUs) in a network, comprising: selecting a vehicle agent from road vehicles traveling on roads associated with a road map with respect to the global machine learning model; and distributing the global machine learning model to the selected vehicle agents via the RSU, the vehicle agents including an on-board computer unit and an on-board sensor configured to collect local data while the vehicle agents travel on a current trajectory of the road, the selected vehicle agents using the on-board computer unit and the collected local data to locally train the global machine learning model by a structure-aware model training method, and the locally trained model is stored as a trained local model, the method further comprising: aggregating the trained local models from the selected vehicle agents by a distribution-based model aggregation method; and updating the global machine learning model using the aggregated trained local models, wherein the at least one processor continues the selecting, distributing, aggregating, and updating until a predetermined number of global training rounds has been reached or a learning error has stabilized. [Request Item 19] [Number 7] [Request Item 20] [Number 8]

Images