Distributed machine learning with communication and computational awareness for vehicle networks

Abstract

A computer-implemented method is provided for training a global machine learning model using a learning server and a set of vehicle agents connected to roadside units (RSUs). The method includes selecting a vehicle agent from a pool of vehicle agents connected to the RSUs and associating the selected vehicle agent with the RSU based on a distance from the selected vehicle agent to the RSU, the RSU being configured to provide a measurement of the distance to the learning server, the method further includes transmitting a global model, a set of selected agents, and a deadline threshold to the RSU in each global training round, the RSU being configured to transmit the global model and a training deadline to the selected vehicle agent. The associated RSU calculates the training deadline for the corresponding selected vehicle agent, and the selected vehicle agent independently locally trains the global model using the local dataset collected by the on-board sensor of the selected vehicle agent to generate a locally trained model. The method further includes aggregating the locally trained models from the selected vehicle agents via the associated RSUs to update the global model until the global model reaches an expected level of accuracy.

Description

Technical Field

[0001] The present invention generally relates to machine learning for vehicle traffic systems, and more particularly to a method and apparatus for distributed machine learning for prediction in vehicle networks.

Background Art

[0002] Modern vehicles are packed with various on-board sensors for collecting data when the vehicle moves along the road, and the on-board processor can use the collected data to train a machine learning model to achieve a higher level of automation. Different from conventional vehicles, modern vehicles are much more intelligent. Modern vehicles can not only collect various vehicle data and traffic data, but also execute advanced algorithms to guide the movement of the vehicle.

[0003] However, realizing intelligent transportation is an extremely difficult problem. Physical roads form a complex road network. Most importantly, traffic situations such as congestion at one location may spread and affect traffic situations at other locations. Furthermore, unexpected events such as traffic accidents and driver behavior may make the traffic situation even more dynamic and uncertain. All these factors can affect the movement of individual vehicles. Therefore, it is very difficult to accurately predict the movement of vehicles and apply the prediction to optimize vehicle operations.

[0004] Data-driven machine learning technologies have become solutions for many applications such as image processing and speech recognition. However, applying machine learning to vehicle applications is still difficult due to the unique characteristics of the vehicle environment, including high mobility, communication costs, data privacy, and high safety requirements.

[0005] On the one hand, vehicles can train independent machine learning models, such as Long Short-Term Memory (LSTM). However, the data collected by individual vehicles may contain deficiencies, which can lead to non-robust models whose predictive performance is insufficient for vehicle applications requiring high accuracy, or even to lead to erroneous decisions. Therefore, non-robust machine learning models trained on deficiency data are unacceptable for vehicle applications. Furthermore, the data collected by individual vehicles is insufficient to train large-scale machine learning models that can be used by vehicles on the road. For example, due to limited onboard computing power and memory, vehicles cannot quickly train machine learning models that can be applied in locations where the vehicle has never traveled before. Therefore, training independent machine learning models by individual vehicles is not a practical solution.

[0006] On the other hand, transferring data collected by vehicles to a central server for training centralized machine learning models is impractical due to the enormous communication bandwidth requirements and the widespread threat of sharing private information. Furthermore, different vehicles are equipped with different sensors based on their model, size, weight, age, and computing resources. Therefore, the data collected by vehicles is highly heterogeneous. As a result, a central server may not have the necessary information to process such heterogeneous data. For example, a high-end GPS receiver provides more accurate measurements than a low-end GPS receiver. Moreover, even with the same GPS receiver, accuracy is higher in open areas than in urban areas. Therefore, a new solution is urgently needed.

[0007] Recent advances in privacy-preserving distributed machine learning, such as Federated Learning (FL), offer promising solutions for the future. This distributed machine learning is an advanced machine learning technique that allows machine learning models to be trained locally based on the trainer's local data. Thus, this distributed machine learning can also guarantee user privacy and effectively address communication cost issues without data transfer. Most importantly, this distributed machine learning incorporates data features from collaborative datasets, enabling the training of robust machine learning models by eliminating data deficiencies in individual datasets. A well-trained, robust model can be deployed to vehicles on the road anytime, anywhere for predictive tasks. Therefore, with the increasing demand for higher levels of automation, distributed machine learning incorporating mobility, communication, and computation seems inevitable.

[0008] While distributed vehicle machine learning can indeed offer a wide range of benefits, it also faces new challenges in vehicle networks. For example, high mobility and latency are two of the main concerns for distributed machine learning in a vehicle environment. On the one hand, due to high mobility, the time a vehicle spends connecting to a connection point will be short. Therefore, vehicles will have limited time to complete model training. On the other hand, training machine learning models, especially multi-round distributed training, is time-consuming. Global distribution time, local model upload time, local model training time, and model queuing time are all contributing factors to latency in distributed machine learning model training. Furthermore, due to heterogeneous computing resources and datasets, training times from different vehicles can vary significantly. Therefore, distributed machine learning for vehicle networks must address these issues.

[0009] Therefore, there is a need to provide a robust, communication- and computationally conscious distributed machine learning platform for vehicle networks. [Overview of the project]

[0010] One objective of some embodiments is to provide a communication-computation-aware distributed machine learning platform that incorporates mobility, communication, computation, and data heterogeneity for accurate vehicle metric prediction. Furthermore, another objective of some embodiments is to use vehicle-specific power (VSP) as an exemplary metric for prediction.

[0011] Some embodiments are based on the recognition that individual vehicle metrics such as location, speed, acceleration, and VSP are more useful for optimizing vehicle behavior than general traffic metrics such as traffic flow, traffic density, and average traffic speed. Predicting vehicle metrics is essential to achieving optimal vehicle behavior, especially in automated and autonomous driving.

[0012] Therefore, some embodiments of the present invention provide distributed machine learning techniques for accurately predicting individual vehicle metrics and optimizing vehicle operation.

[0013] Some embodiments are based on the recognition that modern vehicles are equipped with various sensors for collecting data. On the one hand, due to factors such as communication bandwidth limitations, privacy protection, and security, transferring data from all vehicles to a central server for centralized data processing and analysis is impractical. On the other hand, the limited amount of data collected by individual vehicles is not suitable for training machine learning models for large-scale predictions in a city or state. For example, a vehicle does not know the traffic conditions in places it has not yet traveled. Also, the data collected by individual vehicles may contain imperfections, which can lead to the training of inrobust models. Therefore, there is a need to provide a collaborative machine learning platform that avoids local data transfer and integrates the heterogeneity of onboard computing resources with the heterogeneity of local data, taking communication capabilities into account.

[0014] To this end, some embodiments of the present invention utilize distributed machine learning techniques such as federative learning to build robust predictive models for accurate vehicle metric prediction, and a centralized learning server coordinates distributed model training by taking into account heterogeneity of communication capabilities, onboard computing resources, and local data, and distributes the well-trained machine learning models to vehicles on the road for predictive tasks.

[0015] Some embodiments are based on the recognition that multi-round distributed machine learning model training is time-consuming. However, due to high mobility, the time a vehicle has to connect to a roadside unit (RSU), such as a 3GPP C-V2X gNodeB or an IEEE DSRC / WAVE roadside unit, will be short. In other words, a vehicle may not have enough time to complete the entire model training process. Therefore, a practical communication network architecture needs to be provided for distributed machine learning in vehicles.

[0016] Accordingly, some embodiments of the present invention provide a hierarchical communication network architecture including a set of distributed RSUs to cooperatively relay data traffic between a learning server and a set of learning agents, with vehicles acting as learning agents. The RSUs form a core communication network, in which the RSUs connect to the learning server via reliable communication links such as wired communication links and to vehicles on the road via wireless communication links. This architecture not only increases the coverage area but also extends the connection time between the learning server and the learning agents. Vehicles may be associated with different RSUs at different times along the road. Communication between the learning server and vehicle agents is relayed by the core communication network. This architecture not only increases the coverage area but also extends the connection time between the learning server and the learning agents.

[0017] To achieve this, the learning server first selects vehicle agents and then distributes the global machine learning model to the selected vehicle agents during the first round of distributed training. Each selected vehicle then independently trains the received model using its own data, without sharing that data with other vehicles or the learning server. After a certain number of training iterations, each vehicle agent uploads its trained model to the learning server via the core communication network. The learning server then aggregates the models received from the selected vehicle agents to build an updated global model. Once model aggregation is complete, the learning server selects vehicle agents and redistributes the aggregated model to the selected vehicle agents during the second round of training. This training and aggregation process continues until a robust model is built.

[0018] Some embodiments are based on the understanding that downlink / uplink model transmission times, computational resources, model queuing times, and data types / sizes differ from vehicle to vehicle. Therefore, the time at which vehicle agents receive the global model will vary. As a result, some vehicle agents will have more time to train the machine learning model locally, while others will have less time. Therefore, it is impractical to require all vehicle agents to perform a uniform amount of training work, such as performing the same number of local iterations.

[0019] To this end, some embodiments of the present invention enable vehicle agents to perform non-uniform model training so that each agent determines its own number of training iterations, and enable the learning server to acquire partially trained local models such that some vehicle agents train the model with more iterations, while others train the model with fewer iterations.

[0020] Some embodiments are based on the understanding that the learning server and selected vehicle agents communicate via a core communication network, while communication between the RSU and the vehicle is via an unreliable wireless link. Therefore, the distribution of global models may not reach the vehicle agents, and similarly, the upload of local models may not reach the learning server. Thus, a time threshold must be defined to prevent the learning server from waiting indefinitely for local models.

[0021] To this end, some embodiments of the present invention define a time threshold for each global training round of distributed model training so that the time difference between the time the learning server sends the global model to the vehicle agents and the time the learning server receives the local models from the vehicle agents must be less than this specified threshold. If the learning server does not receive local models from some vehicle agents by this time threshold, the learning server aggregates the models without waiting for the unreceived local models.

[0022] Therefore, the total time consists of global model queuing time, global model transmission time, local model training time, local model queuing time, and local model transmission time. Global model queuing time is the time the global model remains in the RSU's transmission queue before transmission. Global model transmission time includes the time it takes to send the global model from the training server to the RSU and the time it takes to send the global model from the RSU to the vehicle agent. Local model training time is the time required to train the model locally. Local model training time is determined by the vehicle agent's resources, including processor power and the amount of local data. Local model queuing time is the time the trained local model is queued in the vehicle agent's transmission queue before transmission, assuming there is no queuing on the RSU due to the high-speed communication link between the RSU and the training server. Local model transmission time is the time it takes to send the local model from the vehicle agent to the RSU and the time it takes to send the local model from the RSU to the training server.

[0023] Some embodiments are based on the recognition that model queuing time and model transmission time are overheads that need to be minimized so that vehicle agents have more time to train models.

[0024] To this end, some embodiments of the present invention eliminate downlink model queuing time by multicasting global models. Each RSU solves a multicast beamforming problem to maximize the data rate for global model distribution on the downlink. To upload locally trained models, each RSU solves a complex combinatorial problem to allocate optimal radio resources to the learning agents. Optimal resource allocation minimizes model transmission time. Thus, distributed machine learning is formulated as an optimization problem.

[0025] Some embodiments are based on the recognition that data collected by a vehicle depends on factors such as location, time, weather, road conditions, special events, etc. Even at the same location, traffic conditions vary based on different times, different weather, etc. Rush hour traffic conditions are different from off-peak traffic conditions. Traffic conditions on a snowy day are different from those on a sunny day.

[0026] Therefore, it is desirable for the vehicle agent to divide those data into different clusters based on the collection location, time, weather, etc. As a result, the vehicle agent trains different models by using different data clusters. The vehicle agent does not train a model for which the vehicle agent does not have appropriate data. Thus, the vehicle agent uploads only the trained models to the learning server.

[0027] Therefore, the learning server constructs a global model by aggregating locally trained models taking into account information including location, time, weather, etc.

[0028] Some embodiments are based on the recognition that VSP is the power demand by the engine during operation. VSP is used to calculate fuel consumption and corresponding emissions. Therefore, prediction of VSP is important.

[0029] Therefore, some embodiments of the present invention provide multi-horizon VSP prediction. The prediction time horizon is composed of a plurality of prediction periods. The longer the time horizon, the more predictions are provided, and the shorter the time horizon, the more accurate the predictions are.

[0030] Some embodiments are based on the recognition that there is uncertainty in the vehicle environment. Therefore, the machine learning model must be trained to handle unexpected events such as traffic accidents captured by vehicles on the road.

[0031] Therefore, the learning server and the vehicle can interact with each other to enhance the model.

[0032] According to some embodiments of the present invention, a computer-implemented method is provided for training a global machine learning model using a set of vehicle agents connected to a learning server and a roadside unit (RSU), wherein the method uses a processor coupled with memory storing instructions for implementing the method, the instructions, when executed by the processor, perform the steps of the method, the method includes the step of selecting a vehicle agent from a pool of vehicle agents connected to the RSU, the vehicle agent includes an onboard computer unit and an onboard sensor configured to collect a local dataset through the trajectory of the selected vehicle agent on the road, the method further includes the step of associating the selected vehicle agent and the RSU, respectively, based on the distance from the selected vehicle agent to the RSU, the RSU is configured to provide the distance measurement to the learning server, and the method further includes

number

[0033] Furthermore, some embodiments of the present invention provide a computer-implemented communication method for causing a learning server to update a global model by providing a model locally trained from a vehicle agent selected by the learning server. The method uses a processor coupled with memory that stores instructions for implementing the method, and when the instructions are executed by the processor, they perform the steps of the method, and the method

number

[0034] Furthermore, according to some embodiments of the present invention, a communication and computation-aware distributed machine learning system is provided for a vehicle network including a set of roadside units (RSUs) and a learning server that communicates with vehicles on the road, for training a global machine learning model in a distributed manner, wherein the system includes a processor coupled with memory that stores instructions for implementing the method. In this case, the instructions, when executed by the processor, perform the steps of the method, which include: selecting a set of vehicles on the road as learning agents from the vehicles on the road; determining a training deadline threshold for the learning agents to finish training a local model; determining an upload deadline threshold for the learning agents to finish uploading the locally trained model; and providing a constant connection between the learning server and the learning agents by (1) a method for randomly selecting vehicle agents; (2) a method for selecting vehicles that will stay connected longer to the associated RSU; (3) a method for selecting vehicles that have better link quality to the associated RSU; (4) a method for selecting vehicles that have performed better in previous training rounds; (5) a method for selecting vehicles with larger datasets; and (6) a method for providing a constant connection between the learning server and the learning agents by (1) a method for randomly selecting vehicle agents; (2) a method for selecting vehicles that will stay connected longer to the associated RSU; (3) a method for selecting vehicles that have better link quality to the associated RSU; (4) a method for selecting vehicles that have performed better in previous training rounds; (5) a method for selecting vehicles with larger datasets. The process includes the steps of associating the learning agent with the RSU based on one or a combination of methods for selecting a vehicle having the RSU, and distributing the global machine learning model to the selected learning agent via the associated RSU by performing downlink multicast beamforming using the associated RSU to minimize the global machine learning model distribution delay, wherein the learning agent, including an onboard processing unit, collects local data along the road using onboard sensors, clusters the collected local data based on the vehicle environment of the learning agent, each learning agent determines local model training iterations based on computing power and data size to satisfy the sum of the training deadline threshold and the upload deadline threshold, and each learning agent performs local model training iterations during the determined local model training iterations.The method further includes the steps of: using the locally collected data described above to locally train the global machine learning model; the learning agent reporting channel measurements to the associated RSU and adjacent unassociated RSUs to perform the best handover from the currently associated RSU; the method further includes the steps of: allocating the associated learning agent to the best uplink radio resources of the associated RSU's physical resource block (pRB) to minimize the learning agent's local model upload and queuing delays; and aggregating the locally trained model received from the selected vehicle agent via the RSU's allocated best uplink radio resources to update the global model.

[0035] The embodiments disclosed herein will be further described with reference to the accompanying drawings. The drawings shown are not necessarily to scale, and instead, the emphasis is generally on illustrating the principles of the embodiments disclosed herein. [Brief explanation of the drawing]

[0036] [Figure 1] This figure shows a communication-computation-aware distributed learning platform for a vehicle network, according to some embodiments of the present invention. [Figure 2] This figure shows an example of the association between a vehicle on a road and a roadside unit (RSU) when the vehicle is traveling along the road, according to some embodiments of the present invention. [Figure 3A] This figure shows a two-layer communication-computation-aware distributed machine learning architecture according to some embodiments of the present invention. [Figure 3B] This figure shows an example of functional components for a learning server, a roadside machine, and a vehicle agent in a distributed machine learning platform according to an embodiment of the present invention. [Figure 4]This figure shows an example of a data clustering method used to divide data from each vehicle on a road into clusters, according to some embodiments of the present invention. [Figure 5] This figure shows a two-dimensional physical resource block in a 3GPP C-V2X communication network according to some embodiments of the present invention. [Figure 6] This figure shows an algorithm for optimally allocating wireless resources to associated vehicles, according to some embodiments of the present invention. [Figure 7] This figure shows an algorithm for associating a vehicle with a roadside unit, according to some embodiments of the present invention. [Figure 8] This figure shows a vehicle-associated learning (FL) algorithm according to some embodiments of the present invention. [Figure 9] This figure shows a multi-horizon prediction having six predictions by using seven historical data samples, according to some embodiments of the present invention. [Figure 10] This figure shows functional blocks for the distributed machine learning training phase and application phase according to some embodiments of the present invention. [Modes for carrying out the invention]

[0037] The following description provides only specific embodiments and is not intended to limit the scope, applicability, or configuration of the disclosure. Rather, the following description of specific embodiments will provide a description that enables the realization of one or more specific embodiments for those skilled in the art. Various modifications are intended to be made to the function and configuration of the elements without departing from the spirit and scope of the subject matter disclosed in the appended claims.

[0038] Specific details are provided in the following description to ensure a full understanding of the embodiments. However, those skilled in the art will understand that the embodiments can be carried out even without these specific details. For example, systems, processes, and other elements in the disclosed subject matter may be shown as components in the form of block diagrams to avoid obscuring the embodiments with unnecessary details. In other examples, well-known processes, structures, and techniques may be shown without unnecessary details to avoid obscuring the embodiments. Furthermore, similar reference numbers and names in different drawings refer to similar elements.

[0039] Furthermore, individual embodiments may be described as processes shown as flowcharts, flow diagrams, data flow diagrams, structural diagrams, or block diagrams. While flowcharts may describe operations as sequential processes, many operations can be performed in parallel or simultaneously. In addition, the order of operations may be reordered. A process may terminate when its operations are complete, but it may have additional steps that are not discussed or included in the diagrams. Moreover, not all operations in any specifically described process can occur in all embodiments. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. If a process corresponds to a function, the termination of the function may correspond to returning the function to the calling function or the main function.

[0040] Furthermore, embodiments of the disclosed subject matter may be implemented, at least in part, either manually or automatically. Manual or automatic implementation may be performed, or at least assisted, through a machine, hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. If implemented through software, firmware, middleware, or microcode, the program code or code segments for performing the required tasks may be stored on a machine-readable medium. A processor(s) may perform the required tasks.

[0041] To achieve higher levels of automation, modern vehicles are packed with various onboard sensors to detect diverse data. Accurate metric predictions are essential for accelerating the development of automated and autonomous vehicles. This is because such knowledge can help drivers make effective driving decisions to alleviate traffic congestion, improve fuel efficiency, and reduce air pollution. These promising benefits enable vehicle metric predictions to play a major role in advanced driver-assistance systems (ADAS), advanced traffic management systems, and commercial vehicle operations, which are the goals of intelligent transportation systems (ITS).

[0042] Machine learning (ML) techniques can be used for prediction tasks in vehicle networks. For example, stacked autoencoder models can be used to learn general traffic flow features for prediction. Long-shorter-term memory (LSTM) recurrent neural networks (RNNs) can be used to predict traffic flow. In addition to RNNs, convolutional neural networks (CNNs) can also be used to incorporate latent traffic patterns within the underlying road network.

[0043] Prior technologies focus on using advanced deep learning models for vehicle traffic prediction, but all of these prior technologies study traffic fluctuations using independent learning models. Due to weather changes, changes in road conditions, and special events, traffic patterns on roads can vary significantly under different circumstances. Therefore, using independent models cannot capture such diverse and complex traffic conditions. Most importantly, data collected by individual vehicles is non-IID (identical independent and identically distributed) and may contain imperfections. Simply training independent ML models on individual vehicle data may result in training non-robust models. However, transferring data to a central server creates privacy issues and increases communication costs. Therefore, it is necessary to provide a collaborative machine learning architecture that avoids data transfer and integrates the heterogeneity of onboard computing resources with the heterogeneity of local data, taking communication efficiency into account.

[0044] Distributed machine learning techniques that protect privacy, such as federative learning (FL), can offer a solution. FL is an advanced ML technique that allows ML models to be trained locally based on the trainer's local data. Therefore, FL can guarantee user privacy and effectively reduce communication overhead. Most importantly, FL can incorporate data features into collaborative, heterogeneous datasets, thereby enabling the training of robust traffic models by eliminating data deficiencies present in individual datasets.

[0045] Figure 1 shows a communication-computation-aware distributed learning platform 100 and its components for a vehicle network according to some embodiments of the present invention, as well as the interactions between those components. The platform 100 includes a learning server 110, distributed roadside units 120, and road vehicles 130 which are learning agents. In this case, the learning server 110 is connected to the distributed roadside units 120 via a reliable high-speed communication link 112. The learning server 110 can be located far away or along the roadside. The learning server 110 selects a machine learning model 115, named the global model, to aggregate locally trained models (118). The learning server 110 distributes the global model to selected road vehicles for training. The distributed roadside units (RSUs) 120 form a core communication network and are associated with (connected to) road vehicles for service provision (125) and allocate communication resources to those vehicles (128). Most importantly, the RSUs relay communication traffic between the learning server and the vehicles. Vehicles 130 on the road use sensors 136 to collect data, train machine learning models using computing resources 135 and local data (138), and upload the locally trained models to a learning server to build a global model. The learning server 110 distributes the well-trained machine learning models to the vehicles 130 on the road via distributed RSUs 120 for prediction tasks such as speed prediction and vehicle-specific power prediction. In this case, the vehicles 130 on the road and the distributed RSUs 120 communicate wirelessly using downlink communication links 123 and uplink communication links 132.

[0046] Unlike conventional mobile devices such as smartphones, vehicles have high mobility and can quickly switch connection points at any time. Figure 2 shows an example of the connection between a vehicle on the road and an RSU when the vehicle is traveling along the road, with RSU1201 and RSU2202 connecting to the learning server 110 via a high-speed wired link. Vehicle 200 on the road connects to RSU1 at time t1 and to RSU2 at time t2.

[0047] To address such short-term connectivity issues, embodiments of the present invention provide a two-layer communication-computation-aware distributed machine learning architecture 300, as shown in Figure 3A. Figure 3A illustrates a two-layer communication-computation-aware distributed machine learning architecture in which a learning server selects an initial ML model, such as a neural network, and hyperparameters such as a time threshold for completing local training and a time threshold for uploading the locally trained model, selects an initial learning agent to train the model, and distributes the ML model and hyperparameters to selected vehicle agents via RSU for training. When a training round begins, the learning server receives the locally trained model and feedback from the learning agent, such as the number of local training iterations and communication link quality. The learning server then aggregates the received local models using methods such as averaging and selects hyperparameters for the next training round. The learning server then selects a learning agent and distributes the provisional model and hyperparameters to the agent for training. In each training round, the selected vehicle agents determine their local training iterations based on their hyperparameters, computational resources, and local data size, and then train their models using their local datasets for the determined number of iterations. Once local training is complete, the training agents upload the trained models to the training server via RSUs.

[0048] Figure 3B shows examples of functional components 310, 320, and 330 of the learning server, roadside machine, and vehicle agent, respectively, in the distributed machine learning platform 100. The learning server 110 may include an interface (or transceiver) 311 configured to communicate with the learning agent 130 via the RSU 120, one or more processors 312, and memory / storage 313 configured to store hyperparameters 314, a model aggregation algorithm 315, and a global machine learning model 316. The RSU 120 may include two interfaces (or transceivers) 321 configured to communicate with the learning server 110 via a reliable high-speed link and with the vehicle 130 via a wireless link, one or more processors 322, and memory / storage 323 configured to store a wireless resource allocation algorithm 324, a vehicle-RSU association algorithm 325, and a communication algorithm 326. The vehicle 130 may include an interface (or transceiver) 331 configured to communicate with the learning server 110 via a wireless link through an RSU 120, one or more processors 332, a sensor 333, and a memory / storage 334 configured to store local data 335, a machine learning algorithm 336, a machine learning model 337, a machine learning objective function 338, and hyperparameters 339.

[0049] To facilitate distributed machine learning, the training data of a learning agent can be divided into different clusters so that each cluster corresponds to a learning model, for example, using rush hour data to train a rush hour model. Data clustering is important for many reasons, such as off-hour data being undesirable for training a rush hour traffic model, or local traffic data being unsuitable for training a highway traffic model. There are several ways to cluster data. Figure 4 shows a data clustering method used to divide the data of each vehicle on a road into clusters, where the vehicle's local data 400 is first divided based on location (410) and then further divided based on time (420). System models and downlink / uplink communication models

[0050]

number

[0051] To perform such distributed model training in a vehicle environment, w and w (v)An efficient I2V / V2I communication platform is needed to share data. Therefore, a realistic high-density heterogeneous network architecture is provided in which multiple RSUs are deployed across the Region of Interest (RoI) to connect vehicles to each other. These RSUs are connected to the learning server by reliable high-speed communication links. As shown in Figure 1, vehicles travel on the road and connect to the learning server via these RSUs. The motivation for adopting this architecture consists of three parts. First, the onboard computing power of vehicles is limited. It may take a long time to complete local model training. Therefore, vehicles may move away from the RSUs that have received the global model from the learning server. This two-tier architecture allows vehicles to be covered by other RSUs when they have completed model training. Second, it aims to ensure V2I / I2V connectivity over a larger area for comprehensive observation and robust model training. Third, it may be impossible to establish a direct vehicle-server communication link due to limited radio link coverage.

[0052]

number

[0053]

number

[0054]

number

[0055]

number

[0056] For this reason, pRB zb The downlink signal-to-noise ratio (SNR) over the specified range is calculated as follows:

number

[0057]

number

[0058]

number

[0059]

number

[0060] This invention provides an enhanced FedProx-based distributed machine learning solution for incorporating communication delay, queuing delay, model training delay, and dataset heterogeneity into vehicle networks.

[0061]

number

[0062]

number

[0063]

number

[0064]

number

[0065]

number

[0066]

number

[0067] Similarly, the time required to upload the trained model for vehicle v is calculated as follows:

number

[0068]

number

[0069]

number

[0070]

number

[0071] The total delay of agent v is calculated as follows:

number

[0072] The learning server is d thrA time threshold is set for each global training round, and by that time threshold, the training server must distribute the global model, and the agents must train the model locally and upload the trained model back to the training server. In other words, the following constraints must be met.

number

[0073]

number

[0074] The goal is to minimize model submission time and queuing time so that model training time is maximized in each global training round, while ensuring that constraint (12) is met. Optimization of model distribution delay

[0075]

number

[0076]

number

[0077]

number

[0078]

number

[0079]

number

[0080] Note that the optimization problem (16) is a mixed combinatorial problem and is NP-hard. The present invention stacks the SNRs over all pRBs to form a gain matrix G b,z Next, the optimal pRB assignment is found using the widely used Hungarian algorithm. This process is explained in Figure 6. Figure 6 shows the algorithm for optimally allocating radio resources to associated vehicles. Local model training

[0081]

number

[0082] Therefore, agent v decides on the local model training iterations as follows:

number

[0083]

number

[0084]

number

[0085] Vehicles can be associated with RSUs using various methods. Figure 7 shows an algorithm for associating a vehicle with an RSU based on distance, according to some embodiments of the present invention. Generally, the closer the vehicle is to the RSU, the better the link quality should be. Therefore, the model transmission delay between the vehicle and the RSU should be short. Vehicle Agent Selection

[0086] Selecting a learning agent in a dynamic vehicle pool is challenging because vehicles are constantly moving. Therefore, the vehicle pool changes dynamically. On the one hand, the learning server does not communicate directly with vehicles on the road. Vehicles on the road connect to RSUs and report link quality measurements of Reference Signal Received Powers (RSRP) only to their associated RSU. On the other hand, an RSU only has link quality information for the vehicle it is associated with and not for vehicles associated with other RSUs. Therefore, the learning server coordinates with the RSUs when selecting a learning agent. The following are methods that can be used to select a vehicle agent. 2) Selection Method-1 (SM1): Randomly select a vehicle agent. 3) Selection Method-2 (SM2): Select a vehicle that is likely to remain connected for a longer period due to its association with the RSU, for example, a vehicle that is close to the association with the RSU. 4) Selection method-3 (SM3): Associated RSU Select a vehicle with better link quality. 5) Selection Method-4 (SM4): Select a vehicle that performed better in a previous training round, for example, a vehicle with less training loss. 6) Selection Method-5 (SM5): Select vehicles with larger datasets. 7) Selection Method-6 (SM6): Select a vehicle with more communication resources. Summary of the distributed learning solutions provided

[0087]

number

[0088] A well-trained model can be applied to a prediction task by vehicles on a road. The machine learning model provided by embodiments of the present invention enables multi-horizon prediction, i.e., making multiple predictions within a given prediction time. Figure 9 shows a multi-horizon prediction by making six predictions 900 using seven historical data samples 910 at a prediction time 930. The data sampling period 940 is Δt s Therefore, the forecast period of 920 is Δt p That is the case.

[0089] Figure 10 is a diagram showing functional blocks of the distributed machine learning training phase and application phase according to some embodiments of the present invention, where block 1000 shows the model training process and block 1020 shows the model application process. For model training, the learning server initiates the learning process by selecting a machine learning model (1001). The learning server then coordinates multi-round distributed model training (1002). To do so, the learning server selects vehicle agents and model training hyperparameters (1003). The learning server distributes the global machine learning model and hyperparameters to the selected vehicle agents via RSUs (1004), and the RSUs then relay the model and hyperparameters to the selected vehicle agents (1005). Upon receiving the global model and hyperparameters (1006), the vehicle agents locally train the machine learning model using their local datasets 1008 and the FedProx-based algorithm provided in Figure 8 (1007). When the local training time expires, the vehicle agent uploads the locally trained model to the learning server via the RSU (1009), and the RSU relays the locally trained model to the learning server (1010). Upon receiving the locally trained model (1011), the learning server aggregates the local models and coordinates the next training round (1002).

[0090] Once the machine learning model is well trained, the training server distributes the model to vehicles on the road (1021), and the vehicles on the road use the trained model to make multi-horizon predictions (1022). The vehicles on the road then apply those predictions to their actions (1023). The vehicles on the road can also feed their experience back to the training server to improve the model (1024). 4.Applications

[0091]

number

Claims

1. A computer-implemented method for training a global machine learning model using a learning server and a set of vehicle agents connected to a Roadside Unit (RSU), wherein the method uses a processor coupled with memory storing instructions for implementing the method, the instructions, when executed by the processor, perform the steps of the method, and the method The method further includes the step of selecting a vehicle agent from a pool of vehicle agents connected to the RSU, wherein the vehicle agent includes an onboard computer unit and an onboard sensor configured to collect local data through the trajectory of the selected vehicle agent on the road, and the method further includes The steps include associating the selected vehicle agent with the RSU based on the distance from the selected vehicle agent to the RSU, [Math 1] The steps include aggregating locally trained models from the selected vehicle agents via the associated RSU in order to update the global model until the global training round reaches a predetermined threshold, The aforementioned deadline threshold d ｃｍｐ The deadline for the selected vehicle agent to finish locally training the global model is the deadline threshold d. ｔｈｒ A method in which the selected vehicle agent finishes uploading the locally trained model.

2. The aforementioned deadline threshold d ｃｍｐ and d ｔｈｒ d ｃｍｐ <d ｔｈｒ The method according to claim 1, wherein it is determined in each global training round to be such. [Request Item 3] [Number 2] [Request Item 4] [Number 3] [Request Item 5] [Number 4] [Request Item 6] [Number 5] [Request Item 7] [Number 6] [Request Item 8] [Number 7] [Request Item 9] [Number 8]

10. The method according to claim 1, wherein the learning server and the RSU are connected via a highly reliable high-speed communication link.

11. The method according to claim 1, wherein the vehicle agent is selected using one or a combination of the following methods: (1) a method of randomly selecting a vehicle agent; (2) a method of selecting a vehicle that will remain connected to the associated RSU for a longer period of time; (3) a method of selecting a vehicle that has better link quality to the associated RSU; (4) a method of selecting a vehicle that has performed better in a previous training round; (5) a method of selecting a vehicle that has a larger dataset; and (6) a method of selecting a vehicle that has more communication resources.

12. The method according to claim 1, wherein the global model is parameterized by w, which represents the weights of the neural network of the machine learning model.

13. Vehicle agent v uses the following FedProx-based objective function [Number 9] The method according to claim 1, wherein the global model is used for training. [Request Item 14] [Number 10]

15. A computer-implemented communication method for updating a global model by providing a learning server with a model locally trained from a vehicle agent selected by the learning server, wherein the communication method uses a processor coupled with a memory that stores instructions for implementing the communication method, and the instructions, when executed by the processor, perform steps of the communication method, and the communication method [Math 11] The steps include receiving the locally trained model uploaded from the selected vehicle agent and sending it to the learning server, The aforementioned deadline threshold d ｃｍｐ The deadline for the selected vehicle agent to finish locally training the global model is the deadline threshold d. ｔｈｒ A method in which the selected vehicle agent finishes uploading the locally trained model.

16. The method according to claim 15, wherein the step of acquiring and transmitting the global model and the locally trained model between the learning server and the RSU is performed over a reliable high-speed communication link.

17. The method according to claim 15, wherein the steps of receiving and transmitting the global model and the locally trained model between the RSU and the vehicle agent are performed via a radio link.

18. The method according to claim 15, wherein the uplink resource allocation is performed for each physical resource block (pRB) located in the selected vehicle agent in order to maximize the throughput of uplink communication.

19. The local training time is determined such that the sum of the global model distribution delay from the learning server to the selected vehicle agent and the local model training delay corresponding to each vehicle agent is less than the deadline threshold d ｃｍｐ , and the sum of the global model distribution delay from the learning server to the selected vehicle agent, the local model training delay corresponding to the training deadline of each vehicle agent, the uplink queuing delay, and the local model upload delay from the selected vehicle agent is determined to be less than the deadline threshold d ｔｈｒ The method according to claim 15, wherein the method is determined to be less than

20. A communication and computationally aware distributed machine learning system for a vehicle network including a set of roadside units (RSUs) and a learning server communicating with vehicles on the road, for training a global machine learning model in a distributed manner, wherein the system includes a processor coupled with memory storing instructions for implementing a method, the instructions, when executed by the processor, perform steps of the method, and the method The steps include selecting a set of vehicles on the road as a learning agent from among the vehicles on the road, The steps include determining a training deadline threshold for when the learning agent will finish training the local model, The steps include determining an upload deadline threshold for when the learning agent finishes uploading the locally trained model, To provide a constant connection between the learning server and the learning agent, the learning agent is associated with the RSU based on one or a combination of the following methods: (1) a method for randomly selecting a vehicle agent; (2) a method for selecting a vehicle that will remain connected to the associated RSU for a longer period of time; (3) a method for selecting a vehicle that has better link quality to the associated RSU; (4) a method for selecting a vehicle that performed better in a previous training round; (5) a method for selecting a vehicle with a larger dataset; and (6) a method for selecting a vehicle with more communication resources. The method includes the steps of distributing a global machine learning model to selected learning agents via the associated RSU by performing downlink multicast beamforming to minimize the global machine learning model distribution delay using the associated RSU, wherein the learning agent, including an onboard processing unit, collects local data along the road using onboard sensors, clusters the collected local data based on the vehicle environment of the learning agent, each learning agent determines local model training iterations based on computing power and data size to satisfy the sum of the training deadline threshold and the upload deadline threshold, each learning agent locally trains the global machine learning model using the collected local data during the determined local model training iterations, currently measures the reference signal reception power from the associated RSU and adjacent unassociated RSUs, the measured values ​​are reported to the associated RSU, and the method further includes To minimize the local model upload delay and queuing delay of the learning agent, the steps include: allocating the associated learning agent to the optimal uplink radio resources of the associated RSU's physical resource block (pRB); The steps include updating the global model by aggregating the locally trained models received from the selected vehicle agent via the RSU's allocated optimal uplink radio resources, The total time, consisting of global model queuing time, global model transmission time, local model training time, local model queuing time, and local model transmission time, is less than a predetermined deadline threshold in the system.

21. The system according to claim 20, wherein the learning server determines the training deadline threshold and the upload deadline threshold in each global training round of the steps of distributing the global model and aggregating the locally trained model in order to limit the uplink queuing delay via the learning agent, the local model training delay of the onboard processing unit of the learning agent, the downlink delay of the transmission of the global machine learning model, and the uplink delay of the transmission of the locally trained model.

22. The system according to claim 21, wherein the learning server is configured to repeatedly perform the distribution step and the aggregation step until the global training round reaches a predetermined threshold.

23. The system according to claim 20, wherein the associated RSU is located at the closest distance from the learning agent compared to the unassociated RSU.

Images