Communication- and Computation-Aware Distributed Machine Learning for Vehicular Networks

Abstract

A computer-implemented method is provided for training a global machine learning model using a learning server and a set of vehicle agents connected to roadside units (RSUs). The method includes selecting a vehicle agent from a pool of vehicle agents connected to the RSUs and associating the selected vehicle agent with the RSU based on a distance from the selected vehicle agent to the RSU, the RSU being configured to provide a measurement of the distance to the learning server, the method further includes transmitting a global model, a set of selected agents, and a deadline threshold to the RSU in each global training round, the RSU being configured to transmit the global model and a training deadline to the selected vehicle agent. The associated RSU calculates the training deadline for the corresponding selected vehicle agent, and the selected vehicle agent independently locally trains the global model using the local dataset collected by the on-board sensor of the selected vehicle agent to generate a locally trained model. The method further includes aggregating the locally trained models from the selected vehicle agents via the associated RSUs to update the global model until the global model reaches an expected level of accuracy.

Description

[Technical field]

[0001] The present invention relates generally to machine learning for vehicular traffic systems, and more particularly to a method and apparatus for distributed machine learning for prediction in vehicular networks. [Background technology]

[0002] Modern vehicles are packed with a variety of on-board sensors to collect data as the vehicle moves along the road, and on-board processors can use the collected data to train machine learning models to achieve higher levels of automation. Unlike traditional vehicles, modern vehicles are much more intelligent. Not only can modern vehicles collect a variety of vehicle and traffic data, but they can also run advanced algorithms to guide the vehicle's movements.

[0003] However, achieving intelligent traffic is an extremely challenging problem. Physical roads form a complex road network. Most importantly, traffic conditions such as congestion in one place may propagate and affect traffic conditions in other places. In addition, unexpected events such as traffic accidents and driver behaviors may make the traffic situation even more dynamic and uncertain. All these factors may affect the movement of individual vehicles. Therefore, how to accurately predict vehicle movements and apply the predictions to optimize vehicle operation is very challenging.

[0004] Data-driven machine learning techniques have become a solution for many applications, such as image processing and voice recognition. However, applying machine learning to vehicular applications remains challenging due to the unique characteristics of the vehicular environment, including high mobility, communication costs, data privacy, high safety requirements, etc.

[0005] On the one hand, vehicles can train independent machine learning models such as Long Short-Term Memory (LSTM), but the data collected by individual vehicles may contain imperfections, which may result in non-robust models whose predictive performance may not be sufficient for vehicular applications that require high accuracy, or may even result in incorrect decision-making. Therefore, non-robust machine learning models trained on imperfect data are not acceptable for vehicular applications. Also, the data collected by individual vehicles is insufficient to train large-scale machine learning models that can be used by vehicles on the road. For example, due to limited on-board computing power and memory, vehicles cannot quickly train machine learning models that may be applied in places where the vehicles have not traveled. Therefore, training independent machine learning models by individual vehicles is not a practical solution.

[0006] On the other hand, transferring data collected by vehicles to a central server for centralized machine learning model training is impractical due to the huge communication bandwidth requirements and the widespread threat of sharing private information. Also, different vehicles are equipped with different sensors based on their model, size, weight, age and computational resources. Thus, the data collected by vehicles is highly heterogeneous. As a result, the central server may not have the necessary information to process such heterogeneous data. For example, a high-end GPS receiver provides more accurate measurements than a low-end GPS receiver. Moreover, the same GPS receiver is more accurate in open areas than in urban areas. Therefore, new solutions are urgently needed.

[0007] Recent advances in privacy-preserving distributed machine learning such as Federated Learning (FL) may bring a promising solution. This distributed machine learning is an advanced machine learning technique that allows machine learning models to be trained locally based on the trainer's local data. Thus, this distributed machine learning can guarantee user privacy protection and also effectively address the communication cost issue without data transfer. Most importantly, this distributed machine learning allows training of robust machine learning models by eliminating data imperfections contained in individual datasets since it incorporates data features from collaborative datasets. Well-trained robust models can be distributed to vehicles on the road for prediction tasks anywhere and at any time. Thus, with the growing demand for higher automation, distributed machine learning that incorporates mobility, communication and computation seems inevitable.

[0008] Although distributed vehicular machine learning can indeed bring manifold benefits, it also faces new challenges in vehicular networks. For example, high mobility and latency are two of the main concerns of distributed machine learning in vehicular environments. On the one hand, due to high mobility, the time for vehicles to connect to connection points will be short. Thus, vehicles will have limited time to finish model training. On the other hand, training machine learning models, especially multi-round distributed training, takes time. Global distribution time, local model upload time, local model training time, model queuing time, etc. contribute to latency in distributed machine learning model training. Also, due to heterogeneous computational resources and datasets, training times from different vehicles may vary greatly. Thus, distributed machine learning for vehicular networks must address these issues.

[0009] Therefore, there is a need to provide a robust communication-computation aware distributed machine learning platform for vehicular networks. Summary of the Invention

[0010] One objective of some embodiments is to provide a communication-computation aware distributed machine learning platform to incorporate mobility, communication, computation and data heterogeneity for accurate vehicle metric prediction. Additionally, another objective of some embodiments is to make Vehicle Specific Power (VSP) an exemplary metric for prediction.

[0011] Some embodiments are based on the recognition that, unlike general traffic metrics such as traffic flow, traffic density and average traffic speed, individual vehicle metrics such as location, speed, acceleration and VSP are more useful for optimizing vehicle operation. To achieve optimal vehicle operation, prediction of vehicle metrics is essential, especially in automated and autonomous driving.

[0012] Thus, some embodiments of the present invention provide distributed machine learning techniques for accurately predicting individual vehicle metrics to optimize vehicle operation.

[0013] Some embodiments are based on the recognition that modern vehicles are equipped with various sensors for collecting data. On the one hand, due to facts such as communication bandwidth limitations, privacy protection and security, it is impractical to transfer data from all vehicles to a central server for centralized data processing and analysis. On the other hand, the limited amount of data collected by individual vehicles is not suitable for training machine learning models for large-scale prediction in a city or a state. For example, the vehicles do not know the traffic conditions in places where the vehicles have not yet traveled. Also, the data collected by individual vehicles may contain deficiencies, which may result in training of non-robust models. Therefore, there is a need to provide a collaborative machine learning platform by avoiding local data transfers and taking into account communication capabilities, integrating the heterogeneity of on-board computing resources with the heterogeneity of local data.

[0014] To this end, some embodiments of the present invention utilize distributed machine learning techniques such as federated learning to build robust predictive models for accurate vehicle metrics prediction, and a centralized learning server coordinates the distributed model training by taking into account communication capabilities, heterogeneity of on-board computing resources and heterogeneity of local data, and distributes well-trained machine learning models to vehicles on the road for prediction tasks.

[0015] Some embodiments are based on the recognition that multi-round distributed machine learning model training is time-consuming. However, due to high mobility, the time that a vehicle connects to a roadside unit (RSU), such as a 3GPP C-V2X gNodeB or an IEEE DSRC / WAVE roadside unit, will be short. In other words, it may happen that a vehicle does not have enough time to complete the entire model training process. Therefore, it is necessary to provide a realistic communication network architecture for vehicular distributed machine learning.

[0016] Therefore, some embodiments of the present invention provide a hierarchical communication network architecture including a set of distributed RSUs to cooperatively relay data traffic between a learning server and a set of learning agents, with vehicles playing the role of learning agents. The RSUs form a core communication network, in which the RSUs connect to the learning server via reliable communication links, such as wired communication links, and connect to the vehicles on the road via wireless communication links. This architecture not only increases the coverage area, but also extends the connection time between the learning server and the learning agents. The vehicles may be associated with different RSUs at different times along the road. The communication between the learning server and the vehicle agents is relayed by the core communication network. This architecture not only increases the coverage area, but also extends the connection time between the learning server and the learning agents.

[0017] To that end, the learning server first selects vehicle agents and then distributes the global machine learning model to the selected vehicle agents during a first round of distributed training. Each selected vehicle then uses its own data to independently train the received model without sharing its data with other vehicles or the learning server. After a certain iteration of training, each vehicle agent uploads the trained model to the learning server via the core communication network. The learning server then aggregates the received models from the selected vehicle agents to build an updated global model. Once model aggregation is complete, the learning server selects vehicle agents to redistribute the aggregated model to the selected vehicle agents during a second round of training. This training and aggregation process continues until a robust model is built.

[0018] Some embodiments are based on the recognition that downlink / uplink model transmission times, computational resources, model queuing times, and data types / sizes vary from vehicle to vehicle. Thus, the time at which the vehicle agents receive the global model varies. As a result, some vehicle agents have more time to train machine learning models locally, while other vehicle agents have less time to train machine learning models locally. Therefore, it is unrealistic to require all vehicle agents to perform a uniform amount of training work, such as performing the same number of local iterations.

[0019] To that end, some embodiments of the present invention allow vehicle agents to perform non-uniform model training such that each agent determines its own number of training iterations, and allow the learning server to obtain partially trained local models such that some vehicle agents train the model with more iterations and other vehicle agents train the model with fewer iterations.

[0020] Some embodiments are based on the recognition that the learning server and the selected vehicle agents communicate over a core communication network, and the communication between the RSUs and the vehicles is over unreliable wireless links. Thus, the distribution of the global model may not reach the vehicle agents, and similarly, the upload of the local model may not reach the learning server. Therefore, a time threshold needs to be defined so that the learning server does not wait forever for the local model.

[0021] To that end, some embodiments of the present invention define a time threshold for each global training round of distributed model training, such that the time difference between the time the learning server sends the global model to the vehicle agents and the time the learning server receives the local models from the vehicle agents must be less than the specified threshold. If the learning server has not received local models from some vehicle agents by the time threshold, the learning server aggregates the models without waiting for the unreceived local models.

[0022] Therefore, the total time consists of global model queuing time, global model transmission time, local model training time, local model queuing time, and local model transmission time. Global model queuing time is the time that the global model stays in the transmission queue of the RSU before transmission. Global model transmission time includes the time to transmit the global model from the learning server to the RSU plus the time to transmit the global model from the RSU to the vehicle agent. Local model training time is the time required to train the model locally. The local model training time is determined by the resources of the vehicle agent including processor power, amount of local data, etc. Local model queuing time is the time that the trained local model is queued in the transmission queue of the vehicle agent before transmission, assuming there is no queuing at the RSU due to the high-speed communication link between the RSU and the learning server. Local model transmission time is the time to transmit the local model from the vehicle agent to the RSU plus the time to transmit the local model from the RSU to the learning server.

[0023] Some embodiments are based on the recognition that model queuing time and model transmission time are overheads that need to be minimized so that the vehicle agents have more time for model training.

[0024] To that end, some embodiments of the present invention eliminate downlink model queuing time by multicasting the global model. Each RSU solves a multicast beamforming problem to maximize the data rate for global model distribution in the downlink. To upload the locally trained model, each RSU solves a complex combinatorial problem to allocate optimal radio resources to the learning agents. The optimal resource allocation minimizes the model transmission time. Thus, distributed machine learning is formulated as an optimization problem.

[0025] Some embodiments are based on the recognition that data collected by a vehicle depends on location, time, weather, road conditions, special events, etc. Even in the same location, traffic conditions vary based on different times of day, different weather, etc. Traffic conditions during rush hour are different from traffic conditions during off-hours. Traffic conditions on a snowy day are different from traffic conditions on a sunny day.

[0026] To that end, the vehicle agents preferably divide their data into different clusters based on collection location, time, weather, etc. As a result, the vehicle agents train different models by using different data clusters. The vehicle agents will not train a model for which the vehicle agents do not have suitable data. Thus, the vehicle agents upload only trained models to the learning server.

[0027] Therefore, the learning server builds a global model by aggregating locally trained models taking into account information including location, time, weather, etc.

[0028] Some embodiments are based on the recognition that the VSP is the power demand by a running engine. The VSP is used to calculate fuel consumption and corresponding emissions. Thus, predicting the VSP is important.

[0029] Therefore, some embodiments of the present invention provide multi-horizon VSP prediction, where the prediction time horizon is made up of multiple prediction periods: a longer time horizon provides more predictions, and a shorter time horizon provides more accurate predictions.

[0030] Some embodiments are based on the recognition that uncertainties exist in the vehicular environment, and therefore machine learning models must be trained to deal with unexpected events, such as traffic accidents, captured by vehicles on the road.

[0031] Therefore, the learning server and the vehicle can interact with each other for model enhancement.

[0032] According to some embodiments of the present invention, there is provided a computer-implemented method for training a global machine learning model using a learning server and a set of vehicle agents connected to road side units (RSUs), the method using a processor coupled to a memory storing instructions for implementing the method, the instructions performing steps of the method when executed by the processor, the method comprising: selecting a vehicle agent from a pool of vehicle agents connected to the RSUs, the vehicle agents comprising an on-board computer unit and an on-board sensor configured to collect a local dataset through a trajectory of the selected vehicle agent on a road, the method further comprising associating the selected vehicle agent with each of the RSUs based on a distance from the selected vehicle agent to the RSU, the RSUs being configured to provide the distance measurements to the learning server, the method further comprising:

number

[0033] Additionally, some embodiments of the present invention provide a computer-implemented communication method for having a learning server update a global model by providing a locally trained model from a vehicle agent selected by said learning server, said method using a processor coupled to a memory storing instructions implementing said method, said instructions, when executed by said processor, performing steps of said method, said method comprising:

number

[0034] Further, in accordance with some embodiments of the present invention, there is provided a communication and computation aware distributed machine learning system for a vehicular network including a set of road side units (RSUs) and a learning server in communication with on-road vehicles for training a global machine learning model in a distributed manner, the system including a processor coupled to a memory storing instructions implementing a method, the instructions being executed by the processor to perform steps of the method, the method including: selecting a set of on-road vehicles from the on-road vehicles as learning agents; determining a training deadline threshold for the learning agents to finish training a local model; determining an upload deadline threshold for the learning agents to finish uploading the locally trained model; and providing an always-on connection between the learning server and the learning agents, the method including (1) randomly selecting vehicle agents, (2) selecting vehicles that stay connected longer to the associated RSUs, (3) selecting vehicles that have better link quality to the associated RSUs, (4) selecting vehicles that had better performance in previous training rounds, (5) selecting vehicles with a larger dataset, and (6) selecting vehicles that have more commuting data to provide an always-on connection between the learning server and the learning agents. and distributing a global machine learning model to the selected learning agents via the associated RSUs by performing downlink multicast beamforming to minimize a global machine learning model distribution delay using the associated RSUs; the learning agents including an on-board processing unit collect local data along a road using an on-board sensor and cluster the collected local data based on a vehicular environment of the learning agent; each of the learning agents determines a local model training iteration based on a computation power and a data size to satisfy a sum of the training deadline threshold and the upload deadline threshold; and each of the learning agents performs the local model training iteration based on a computation power and a data size to satisfy a sum of the training deadline threshold and the upload deadline threshold during the determined local model training iteration.The collected local data is used to locally train the global machine learning model, and the learning agent reports channel measurements to the associated RSU and neighboring unassociated RSUs to perform a best handover from the currently associated RSU, and the method further includes allocating optimal uplink radio resources of physical resource blocks (pRBs) of the associated RSU to the associated learning agent to minimize a local model upload delay and a queuing delay of the learning agent, and aggregating the locally trained model received from the selected vehicle agent via the allocated optimal uplink radio resources of the RSU to update the global model.

[0035] The presently disclosed embodiments will be further described with reference to the accompanying drawings, in which the drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the presently disclosed embodiments. [Brief description of the drawings]

[0036] [Figure 1] FIG. 1 illustrates a communication-computation aware distributed learning platform for vehicular networks according to some embodiments of the present invention. [Diagram 2] FIG. 2 illustrates an example of association of vehicles and road side units (RSUs) on a road as the vehicles travel along the road, according to some embodiments of the present invention. [Figure 3A] FIG. 1 illustrates a two-tier communication-computation aware distributed machine learning architecture according to some embodiments of the present invention. [Figure 3B] FIG. 2 is a diagram illustrating an example of functional components of a learning server, a roadside device, and a vehicle agent in a distributed machine learning platform according to an embodiment of the present invention. [Figure 4]FIG. 2 illustrates an example of a data clustering method used to divide data for each vehicle on a road into clusters according to some embodiments of the present invention. [Diagram 5] FIG. 2 illustrates a two-dimensional physical resource block in a 3GPP C-V2X communication network according to some embodiments of the present invention. [Figure 6] FIG. 2 illustrates an algorithm for optimally allocating radio resources to associated vehicles according to some embodiments of the present invention. [Figure 7] FIG. 4 illustrates an algorithm for associating vehicles with roadside units according to some embodiments of the present invention. [Figure 8] FIG. 2 illustrates a vehicle federation learning (FL) algorithm according to some embodiments of the present invention. [Figure 9] FIG. 2 illustrates a multi-horizon prediction with six predictions by using seven historical data samples according to some embodiments of the present invention. [Figure 10] FIG. 2 illustrates functional blocks for distributed machine learning training and application phases according to some embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] The following description provides exemplary embodiments only and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the following description of exemplary embodiments will provide those skilled in the art with an enabling description for implementing one or more exemplary embodiments. It is contemplated that various changes may be made to the function and arrangement of elements without departing from the spirit and scope of the disclosed subject matter as set forth in the appended claims.

[0038] Specific details are given in the following description to provide a thorough understanding of the embodiments. However, those skilled in the art will understand that the embodiments can be practiced without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in block diagram form in order to avoid obscuring the embodiments in unnecessary detail. In other instances, well-known processes, structures, and techniques may be shown without unnecessary detail in order to avoid obscuring the embodiments. Additionally, like reference numbers and names in the various drawings refer to like elements.

[0039] Also, individual embodiments may be described as a process that is depicted as a flowchart, a flow diagram, a data flow diagram, a structure diagram, or a block diagram. Although a flowchart may describe operations as a sequential process, many of the operations may be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process may be terminated when its operations are completed, but may have additional steps not discussed or included in the diagram. Moreover, not all operations in any process that are specifically described may occur in all embodiments. A process may correspond to a method, a function, a procedure, a subroutine, a subprogram, or the like. When a process corresponds to a function, the end of the function may correspond to a return of the function to the calling function or to the main function.

[0040] Additionally, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. The manual or automated implementation may be performed or at least assisted through machine, hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented in software, firmware, middleware, or microcode, the program code or code segments to perform the necessary tasks may be stored on a machine-readable medium. A processor(s) may perform the necessary tasks.

[0041] To achieve higher automation levels, modern vehicles are packed with a variety of on-board sensors to detect diverse data. To facilitate the development of automated and autonomous vehicles, it is essential to have accurate metrics predictions. This is due to the fact that such knowledge can help drivers make effective driving decisions to ease traffic congestion, improve fuel efficiency and reduce air pollution. These promising benefits enable vehicle metrics predictions to play a key role in Advanced Driver-Assistance Systems (ADAS), Advanced Traffic Management Systems and commercial vehicle operations that Intelligent Transportation Systems (ITS) aim to realize.

[0042] Machine Learning (ML) techniques can be used for prediction tasks in vehicular networks. For example, stacked autoencoder models can be used to learn general traffic flow features for prediction. Long Short-Term Memory (LSTM) Recurrent Neural Networks (RNNs) can be used to predict traffic flows. Convolutional Neural Networks (CNNs) can also be used along with the use of RNNs to capture latent traffic patterns in the underlying road network.

[0043] Although the prior art focuses on using advanced deep learning models for vehicle traffic prediction, all of those prior arts use independent learning models to study traffic variations. Due to weather changes, road condition changes, and special events, traffic patterns on roads may vary greatly under different circumstances. Therefore, using independent models cannot capture such diverse and complex traffic situations. Most importantly, the data collected by individual vehicles may be non-IID (independent identically distributed) and may contain imperfections. Simply training independent ML models on individual vehicle data may result in training of non-robust models. However, transferring data to a central server raises privacy issues and increases communication costs. Therefore, there is a need to provide a cooperative machine learning architecture by integrating the heterogeneity of on-board computing resources and the heterogeneity of local data, avoiding data transfer and taking communication efficiency into account.

[0044] Privacy-preserving distributed machine learning such as federated learning (FL) may provide a solution. FL is an advanced ML technique that allows ML models to be trained locally based on trainer's local data. Thus, FL can guarantee user privacy protection and also effectively reduce communication overhead. Most importantly, FL can incorporate data features into collaborative heterogeneous datasets, which enables training of robust traffic models by eliminating data imperfections contained in individual datasets.

[0045] FIG. 1 illustrates a communication-computation aware distributed learning platform 100 for vehicular networks and its components and interactions between them according to some embodiments of the present invention. The platform 100 includes a learning server 110, distributed roadside units 120, and on-road vehicles 130 that are learning agents. In this case, the learning server 110 is connected to the distributed roadside units 120 via a reliable high-speed communication link 112. The learning server 110 can be located remotely or along the roadside. The learning server 110 selects a machine learning model 115, named a global model, to aggregate the locally trained models (118). The learning server 110 distributes the global model to the selected on-road vehicles for training. Distributed roadside units (RSUs) 120 form the core communication network and are associated (connected) (125) with on-road vehicles for servicing and allocate communication resources to those vehicles (128). Most importantly, the RSUs relay communication traffic between the learning server and the vehicles. The vehicles 130 on the road use sensors 136 to collect data, train 138 machine learning models using computational resources 135 and local data, and upload the locally trained models to the learning server to build a global model. The learning server 110 distributes well-trained machine learning models to the vehicles 130 on the road via the distributed RSUs 120 for prediction tasks such as speed prediction and vehicle-specific power prediction. In this case, the vehicles 130 on the road and the distributed RSUs 120 communicate wirelessly using the downlink communication link 123 and the uplink communication link 132.

[0046] Unlike traditional mobile devices such as smartphones, vehicles have high mobility and can quickly switch connection points at any time. Figure 2 illustrates an example of a connection between a vehicle and an RSU on a road when the vehicle runs along the road, and RSU1 201 and RSU2 202 connect to the learning server 110 by a high-speed wired link. A vehicle 200 on the road connects to RSU1 at time t1 and connects to RSU2 at time t2.

[0047] To address such short-term connection issues, an embodiment of the present invention provides a two-tiered communication-computation-aware distributed machine learning architecture 300 as shown in FIG. 3A. FIG. 3A illustrates a two-tiered communication-computation-aware distributed machine learning architecture in which a learning server selects an initial ML model, such as a neural network, and hyperparameters, such as a time threshold for finishing local training and a time threshold for uploading a locally trained model, selects an initial learning agent for training the model, and distributes the ML model and hyperparameters to the selected vehicle agents via RSUs for training. When a training round starts, the learning server receives the locally trained model and feedback, such as the number of local training iterations and communication link quality, from the learning agent. The learning server then aggregates the received local models using a method such as averaging, and selects hyperparameters for the next training round. The learning server then selects a learning agent and distributes the interim model and hyperparameters to the agent for training. In each training round, the selected vehicle agents determine their local training iterations based on the hyperparameters, their computational resources and their local data size, and then train the model using their local dataset for the determined number of iterations. After finishing the local training, the learning agents upload the trained model to the learning server via the RSU.

[0048] 3B is a diagram illustrating an example of functional components 310, 320, and 330 of a learning server, a roadside unit, and a vehicle agent in a distributed machine learning platform 100. The learning server 110 may include an interface (or transceiver) 311 configured to communicate with the learning agent 130 via the RSU 120, one or more processors 312, and a memory / storage 313 configured to store hyperparameters 314, a model aggregation algorithm 315, and a global machine learning model 316. The RSU 120 may include two interfaces (or transceivers) 321 configured to communicate with the learning server 110 via a reliable high-speed link and with the vehicle 130 via a wireless link, one or more processors 322, and a memory / storage 323 configured to store a wireless resource allocation algorithm 324, a vehicle-RSU association algorithm 325, and a communication algorithm 326. The vehicle 130 may include an interface (or transceiver) 331 configured to communicate with the learning server 110 via the RSU 120 over a wireless link, one or more processors 332, sensors 333, and memory / storage 334 configured to store local data 335, machine learning algorithms 336, machine learning models 337, machine learning objective functions 338 and hyperparameters 339.

[0049] To facilitate distributed machine learning, the training data of the learning agents can be divided into different clusters, with each cluster corresponding to a learning model, for example, using rush hour data to train a rush hour model. Data clustering is important for many reasons, for example, off-hour data is not desirable for training a rush hour traffic model, and local traffic data is not suitable for training a highway traffic model. There are multiple ways to cluster data. Figure 4 illustrates a data clustering method used to divide the data of each vehicle on the road into clusters, where the vehicle's local data 400 is first divided based on location (410) and then further divided based on time (420). System model and downlink / uplink communication model

[0050]

number

[0051] To perform such distributed model training in a vehicular environment, we need w and w (v)An efficient I2V / V2I communication platform is needed to share the I2V / V2I network. Therefore, a realistic dense heterogeneous network architecture is provided where multiple RSUs are deployed across a Region Of Interest (RoI) to connect vehicles to each other. These RSUs are connected to a learning server by reliable high-speed communication links. As shown in Fig. 1, vehicles travel on roads and are connected to the learning server through these RSUs. The motivation for adopting this architecture is three-fold. First, the on-board computational power of vehicles is limited. It may take a long time to finish local model training. Thus, vehicles may move away from the RSUs where they received the global model from the learning server. This two-tiered architecture allows vehicles to be covered by other RSUs when they finish model training. Second, it aims to guarantee V2I / I2V connectivity in a larger area for comprehensive observation and robust model training. Third, it may be impossible to establish a direct vehicle-server communication link due to limited coverage of wireless links.

[0052]

number

[0053]

number

[0054]

number

[0055]

number

[0056] For this purpose, pRB zb The downlink signal-to-noise ratio (SNR) over the entire signal path is calculated as follows:

number

[0057]

number

[0058]

number

[0059]

number

[0060] The present invention provides an extended FedProx-based distributed machine learning to incorporate communication delays, queuing delays, model training delays and dataset heterogeneity in vehicular networks.

[0061]

number

[0062]

number

[0063]

number

[0064]

number

[0065]

number

[0066]

number

[0067] Similarly, the time to upload the trained model of vehicle v is calculated as follows:

number

[0068]

number

[0069]

number

[0070]

number

[0071] The total delay of agent v is calculated as follows:

number

[0072] The learning server is thrWe set a time threshold, denoted as , in each global training round, by which the learning server needs to distribute the global model and the agents need to train models locally and upload the trained models back to the learning server. In other words, the following constraints must hold:

number

[0073]

number

[0074] The goal is to ensure that constraint (12) holds, and to minimize the model transmission and queuing times so that model training time is dominated in each global training round. Optimizing model distribution delays

[0075]

number

[0076]

number

[0077]

number

[0078]

number

[0079]

number

[0080] Note that the optimization problem (16) is a mixed combinatorial problem and is NP-hard. In the present invention, the SNRs over all pRBs are stacked to obtain the gain matrix G b,z Then, the widely used Hungarian algorithm is used to find the optimal pRB allocation. This process is illustrated in Figure 6, which shows an algorithm for optimally allocating radio resources to associated vehicles. Local model training

[0081]

number

[0082] Therefore, agent v determines its local model training iterations as follows:

number

[0083]

number

[0084]

number

[0085] Vehicles can be associated with RSUs using various methods. Figure 7 illustrates an algorithm for associating vehicles with RSUs based on distance according to some embodiments of the present invention. In general, when a vehicle is close to an RSU, the link quality should be excellent. Therefore, the model transmission delay between the vehicle and the RSU should be short. Vehicle Agent Selection

[0086] Selecting a learning agent in a dynamic vehicle pool is challenging because vehicles come and go. Thus, the vehicle pool changes dynamically. On the one hand, the learning server does not directly communicate with the vehicles on the road. The vehicles on the road connect to the RSUs and report link quality measurements of Reference Signal Received Powers (RSRPs) only to the associated RSUs. On the other hand, the RSUs have link quality information only for the associated vehicles and do not have link quality information of vehicles associated with other RSUs. Hence, the learning server cooperates with the RSUs in selecting the learning agent. The following are methods that can be used to select a vehicle agent: 2) Selection method-1 (SM1): Randomly select a vehicle agent. 3) Selection Method-2 (SM2): Select a vehicle that is likely to remain connected to the associated RSU for a longer period of time, e.g., a vehicle that is close to the associated RSU. 4) Selection Method-3 (SM3): Select a vehicle with a better link quality to the associated Rus. 5) Selection Method-4 (SM4): Select the vehicle that performed better in the previous training round, e.g., the vehicle with smaller training loss. 6) Selection Method-5 (SM5): Select vehicles with a larger dataset 7) Selection Method 6 (SM6): Select the vehicle with more commutation resources. Summary of distributed learning solutions offered

[0087]

number

[0088] A well-trained model can be applied to a prediction task by vehicles on the road. The machine learning model provided by the embodiment of the present invention allows multi-horizon prediction, i.e., making multiple predictions at a certain prediction time. Figure 9 illustrates multi-horizon prediction by making six predictions 900 using seven historical data samples 910 at a prediction time 930. The data sampling period 940 is Δt s and the forecast period is Δt p It is.

[0089] FIG. 10 is a diagram illustrating functional blocks of distributed machine learning training and application phases according to some embodiments of the present invention, where block 1000 illustrates a model training process and block 1020 illustrates a model application process. For model training, the learning server starts the learning process by selecting a machine learning model (1001). The learning server then coordinates multi-round distributed model training (1002). To do so, the learning server selects vehicle agents and model training hyperparameters (1003). The learning server distributes the global machine learning model and hyperparameters to the selected vehicle agents via the RSUs (1004), which then relay the model and hyperparameters to the selected vehicle agents (1005). Upon receiving the global model and hyperparameters (1006), the vehicle agents locally train the machine learning model using their local dataset 1008 and the FedProx-based algorithm provided in FIG. 8 (1007). When the local training time expires, the vehicle agent uploads the locally trained model to the learning server via the RSU (1009), and the RSU relays the locally trained model to the learning server (1010). Upon receiving the locally trained model (1011), the learning server aggregates the local models and coordinates the next training round (1002).

[0090] Once the machine learning model is well trained, the learning server distributes the model to vehicles on the road (1021), and the vehicles on the road use the trained model to make multi-horizon predictions (1022). The vehicles on the road then apply those predictions to their actions (1023). The vehicles on the road can also feed back experience to the learning server to enhance the model (1024). 4.Applications

[0091]

number

Claims

1. 1. A computer-implemented method for training a global machine learning model using a learning server and a set of vehicle agents connected to Roadside Units (RSUs), the method using a processor coupled to a memory storing instructions implementing the method, the instructions, when executed by the processor, perform steps of the method, the method comprising: selecting a vehicle agent from the pool of vehicles connected to the RSU, the vehicle agent including an on-board computer unit and an on-board sensor configured to collect local data along a trajectory of the selected vehicle agent on a roadway, the method further comprising: and associating each of the selected vehicle agents with the RSUs based on a distance from the selected vehicle agents to the RSUs, the RSUs being configured to provide the distance measurements to the learning server, the method further comprising: [0010] aggregating the locally trained models from the selected vehicle agents via the associated RSUs to update the global model until the global training round reaches a predetermined threshold.

2. The deadline threshold d ｃｍｐ and d ｔｈｒ , d ｃｍｐ <d ｔｈｒ and the deadline threshold d ｃｍｐ is the deadline for the selected vehicle agent to finish training the global model locally, and the deadline threshold d ｔｈｒ The method of claim 1 , wherein x is a deadline for the selected vehicle agent to finish uploading the locally trained model. 【Request 3】 【Number 2】 【Request 4】 【Number 3】 【Request 5】 【Number 4】 【Request 6】 【Number 5】 【Request 7】 【Number 6】 【Request 8】 【Number 7】 【Request 9】 【Number 8】

10. The method of claim 1 , wherein the learning server and the RSU are connected via a reliable high speed communication link.

11. The method of claim 1 , wherein the training data includes at least one or a combination of three or more of longitude along a road, latitude, speed, acceleration, and weather information for each of the vehicle agents.

12. 2. The method of claim 1, wherein the vehicle agents are selected using one or a combination of the following methods: (1) randomly selecting vehicle agents; (2) selecting vehicles that remain connected to the associated RSU longer; (3) selecting vehicles with better link quality to the associated RSU; (4) selecting vehicles that had better performance in previous training rounds; (5) selecting vehicles with a larger dataset; and (6) selecting vehicles with more commutation resources.

13. The method of claim 1 , wherein the global model is parameterized by w, which represents the weights of a neural network of the machine learning model.

14. A vehicle agent v satisfies the following FedProx-based objective function: [0097] The method of claim 1 , wherein: 【Request 15】 【Number 10】

16. 1. A computer-implemented communication method for having a learning server update a global model by providing a locally trained model from a vehicle agent selected by the learning server, the method using a processor coupled to a memory storing instructions implementing the method, the instructions, when executed by the processor, perform steps of the method, the method comprising: ##EQU00011## receiving the locally trained model uploaded from the associated vehicle agent and sending it to the learning server.

17. 17. The method of claim 16, wherein obtaining and transmitting the global model and the locally trained models between the learning server and the RSUs is performed over a reliable high-speed communication link.

18. The method of claim 16 , wherein the steps of receiving and transmitting the global model and the locally trained model between the RSU and the vehicle agent are performed over a wireless link.

19. 17. The method of claim 16, wherein the uplink resource allocation is performed for each physical resource block (pRB) located at the associated vehicle agent to maximize a throughput of the uplink communication.

20. The local training time is determined by the sum of a global model distribution delay from the learning server to the associated vehicle agents and a local model training delay corresponding to each vehicle agent being within the deadline threshold d ｃｍｐ and a sum of a global model distribution delay from the learning server to the associated vehicle agents, a local model training delay corresponding to the training deadline of each vehicle agent, an uplink queuing delay, and a local model upload delay from the selected vehicle agent is determined to be less than the deadline threshold d. ｔｈｒ The method of claim 16, wherein the .lambda.

21. A communication and computation aware distributed machine learning system for a vehicular network including a set of Road Side Units (RSUs) and a learning server in communication with vehicles on a road for training a global machine learning model in a distributed manner, the system including a processor coupled to a memory storing instructions implementing a method, the instructions, when executed by the processor, perform steps of the method, the method including: selecting a set of on-road vehicles as learning agents from the on-road vehicles; determining a training deadline threshold for the learning agent to finish training a local model; determining an upload deadline threshold for the learning agent to finish uploading the locally trained model; To provide a constant connection between the learning server and the learning agents, the learning agents are associated with the RSUs based on one or a combination of the following methods: (1) randomly selecting vehicle agents; (2) selecting vehicles that remain connected to the associated RSUs longer; (3) selecting vehicles with better link quality to the associated RSUs; (4) selecting vehicles that performed better in previous training rounds; (5) selecting vehicles with larger datasets; and (6) selecting vehicles with more commutation resources. and distributing a global machine learning model to the selected learning agents via the associated RSUs by performing downlink multicast beamforming to minimize a global machine learning model distribution delay using the associated RSUs, wherein the learning agents including an on-board processing unit collect local data along a road using on-board sensors and cluster the collected local data based on a vehicular environment of the learning agents, each of the learning agents determines a local model training iteration based on a computational power and a data size to satisfy a sum of the training deadline threshold and the upload deadline threshold, each of the learning agents locally trains the global machine learning model using the collected local data during the determined local model training iteration, and the learning agents report channel measurements to the associated RSUs and neighboring unassociated RSUs to perform a best handover from the currently associated RSU, and the method further comprises: allocating optimal uplink radio resources of physical resource blocks (pRBs) of the associated RSU to the associated learning agent to minimize local model upload delay and queuing delay of the learning agent; and aggregating the locally trained models received from the selected vehicle agents via the assigned optimal uplink radio resources of the RSU to update the global model.

22. 22. The system of claim 21, wherein the learning server determines two time thresholds in each global training round of the steps of distributing the global model and aggregating the locally trained models to limit uplink queuing delays through the learning agent, local model training delays of the on-board processing unit of the learning agent, downlink delays of transmission of the global machine learning model, and uplink delays of transmission of the locally trained models.

23. 23. The system of claim 22, wherein the learning server is configured to repeatedly perform the distributing and aggregating steps until the global training round reaches a predetermined threshold.

24. 22. The system of claim 21, wherein the associated RSUs are located at a closest distance from the learning agent compared to non-associated RSUs.

Images