Methods, application and vehicles for managing a cluster of vehicles

Abstract

The disclosure relates to a computer implemented method for managing a cluster of vehicles. The method comprises receiving a request from a first vehicle, through a mobile communications network, for establishing a cluster of vehicles. The method comprises assigning the first vehicle as cluster head and authorizing the first vehicle to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles. The method comprises receiving a request from at least one second vehicle, through the cluster head, for joining the cluster. The method comprises assigning a standby node, selected among the at least one second vehicle, the standby node having mobile communications network capacity. The method comprises periodically assigning a new cluster head, to the cluster of vehicles, using a reinforcement learning model.

Description

METHODS, APPLICATION AND VEHICLES FOR MANAGING A CLUSTER OF VEHICLES TECHNICAL FIELD

[0001] The present disclosure relates to Artificial Intelligence (AI) / Machine Learning (ML) methods and means for enabling and coordinating heterogeneous connected clusters of vehicles in fifth generation (5G) networks and beyond. BACKGROUND

[0002] 5G penetration with Ultra Reliable Low Latency Communications (URLLC), massive Machine-Type Communication (mMTC) and Mobile Broadband (MBB) capabilities offers a great opportunity to develop new applications. These applications are deployed mainly with traction from edge computing. Next phases in the connected vehicles industry and drones will be defined by smart city and consumer centric requirements.

[0003] In a potentially long transition period, fully connected vehicles (vehicles supporting cellular and dedicated short-range connectivity or ad-hoc connectivity, Cellular and Device to Device (C&D2D)) will coexist with partially connected vehicles (vehicles with only dedicated short-range connectivity or ad-hoc Device to Device (D2D)) and unconnected vehicles. The concept of collaborative Vehicle-to- Vehicle (V2V) communication applications will respond to the needs of digital consumers with modern sharing-based architecture, such as platooning or flocking. SUMMARY

[0004] A framework, system and application that can be used to manage network access and data traffic of a cluster of heterogenous vehicles that are moving independently from one location to another is proposed herein. The technique provides a method using Multiarmed Bandit (MAB) to select a cluster head and a standby node dynamically. The vehicles evolve cooperatively with the ability to join and exit the cluster. The principle of vertical communication is adapted to horizontal D2D communication.

[0005] There is provided a computer implemented method, executed by an application running in a cloud computing network, for managing a cluster of vehicles. The method comprises receiving a request from a first vehicle, through a mobilecommunications network, for establishing a cluster of vehicles. The method comprises assigning the first vehicle as cluster head and authorizing the first vehicle to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles. The method comprises receiving a request from at least one second vehicle, through the cluster head, for joining the cluster. The method comprises assigning a standby node, selected among the at least one second vehicle, the standby node having mobile communications network capacity. The method comprises periodically assigning a new cluster head, to the cluster of vehicles, using a reinforcement learning model.

[0006] There is provided a computer implemented method, executed by a vehicle. The method comprises sending a request to an application running in a cloud computing network, through a mobile communications network, for establishing a cluster of vehicles. The method comprises receiving an assignment as cluster head and an authorization to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles. The method comprises forwarding a request for joining the cluster, from at least one second vehicle, to the application. The method comprises forwarding a standby node assignment, to a vehicle selected among the at least one second vehicle, the standby node having mobile communications network capacity. The method comprises periodically receiving a new cluster head assignment, made using a reinforcement learning model, and initiating assignment of the cluster head.

[0007] There is provided an apparatus, in a cloud computing network, comprising processing circuits and a memory. The memory contains instructions executable by the processing circuits whereby the apparatus is operative to receive a request from a first vehicle, through a mobile communications network, for establishing a cluster of vehicles. The apparatus is operative to assign the first vehicle as cluster head and authorize the first vehicle to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles. The apparatus is operative to receive a request from at least one second vehicle, through the cluster head, for joining the cluster. The apparatus is operative to assign a standby node, selected among the at least one second vehicle, the standby node having mobile communications network capacity. The apparatus is operative to periodically assign a new cluster head, to the cluster of vehicles, using a reinforcement learning model.

[0008] There is provided a vehicle, comprising processing circuits and a memory. The memory contains instructions executable by the processing circuits whereby the vehicle is operative to send a request to an application running in a cloud computing network, through a mobile communications network, for establishing a cluster of vehicles. The vehicle is operative to receive an assignment as cluster head and an authorization to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles. The vehicle is operative to forward a request for joining the cluster, from at least one second vehicle, to the application. The vehicle is operative to forward a standby node assignment, to a vehicle selected among the at least one second vehicle, the standby node having mobile communications network capacity. The vehicle is operative to periodically receive a new cluster head assignment, made using a reinforcement learning model, and initiating assignment of the cluster head.

[0009] There is also provided a non-transitory computer readable media storing any of the instructions provided herein.

[0010] The method system and application provided herein present improvements to the way coordination of heterogeneous connected clusters of vehicles operate. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a schematic illustration showing an overview of the engagement between D2D clusters and a mobile network.

[0012] Figure 2 is a schematic illustration of the connectivity landscape in an advanced transportation system, showing a variety of vehicles-to-everything (V2X) communication supporting connected vehicle.

[0013] Figure 3 is a schematic illustration of a typical scenario where heterogenous vehicles are moving cooperatively, where the application is restricted by a specific area of the roadway.

[0014] Figure 4 is a flowchart illustrating the adaptation of cluster head selection into the MAB algorithm when a cluster of vehicles is moving from one location to another.

[0015] Figure 5 is a schematic illustration of cluster growth from location A to location B, where a cluster is already formed with a cluster head (Vehicle I) and a standby node (Vehicle II) and where two new vehicles C&D2D join the cluster at location B.

[0016] Figure 6 is a schematic illustration showing the switching of the standby node occurring at location C and showing the result of using the MBA to operate the replacement of the cluster head (Vehicle I) with the new cluster head (Vehicle III).

[0017] Figure 7 is a flowchart of a method executed by an application running in a cloud computing network, for managing a cluster of vehicles.

[0018] Figure 8 is a flowchart of a method executed by a vehicle.

[0019] Figure 9 is a schematic illustration of a hardware in which steps and / or method described herein can be executed.

[0020] Figure 10 is a schematic illustration of a virtualization environment in which the different steps and hardware components described herein can be deployed. DETAILED DESCRIPTION

[0021] Various features will now be described with reference to the drawings to fully convey the scope of the disclosure to those skilled in the art.

[0022] Sequences of actions or functions may be used within this disclosure. It should be recognized that some functions or actions, in some contexts, could be performed by specialized circuits, by program instructions being executed by one or more processors, or by a combination of both.

[0023] Further, computer readable carrier or carrier wave may contain an appropriate set of computer instructions that would cause a processor to carry out the techniques described herein.

[0024] The functions / actions described herein may occur out of the order noted in the sequence of actions or simultaneously. Furthermore, in some illustrations, some blocks, functions or actions may be optional and may or may not be executed; these are generally illustrated with dashed lines.

[0025] At least some aspects of the techniques described herein may be implemented using artificial intelligence, which comprises a variety of techniques as would be apparent to a person skilled in the art, including machine learning techniques. Machine learning techniques include Neural Network (NN), or Artificial Neural Network (ANN), and both terms may be used interchangeably herein. In some contexts, an Artificial Neural Network could include biological portions.

[0026] In this disclosure, problems are addressed which relate with high availability of network connectivity and reliable data exchange when clusters of heterogenousconnected vehicles (non-stationary as opposite to platoon) move from distinct locations to other locations.

[0027] Three operations are performed when a cluster of connected vehicles is deployed. - Data Generation and Collection: This operation is achieved via vehicular sensors or external sensors such as smart devices or environmental sensors. Data collection involves strategies such as data fusion, synchronization, transfer, and cleaning. - Data Analysis: The data is processed, managed, stored and eventually used for intelligence (learning) and inference. - Control and Mobility: The control is carried out via actuation that handles mobility of the vehicle in the field. As an example, when a vehicle comes from a region having no coverage, automatically switching on a base station (at the edge) of the moving vehicle when the base station is detected. Mobility is a case of interest in a vehicle environment as it refers to the propriety of the network to provide a seamless and continuous connection when a vehicle is moving (up to 500km / h).

[0028] The operations mentioned above will fail if the access link (connectivity) between the moving vehicles and the mobile network is broken or is not available. The operations may also fail if excessive access to the network is not allowed. Excessive access (or Wireless Network Overload) is one of the major challenges in V2X, due to the rising number of devices used for traffic monitoring and management. Advanced (adaptive) routing protocol for resource allocation and prioritization is needed to overcome this issue. In case of excessive access, due to bandwidth limitation (capacity), the mobile network rejects the connection request from the devices when a certain threshold is reached. There are several factors that could complicate connectivity such as congestion, data collision, low coverage, and low availability of the mobile network. To achieve a properly managed connectivity, data management and analysis are challenges that should be addressed to improve application latency and performance.

[0029] There are several possible uses for vehicle clustering. A non-exhaustive list is provided next.- Route Optimization: D2D and C&D2D vehicles can benefit from inter cluster communication to receive early notifications such as to redirect traffic away from congestion, enabling route planning. - Smart city lights and controlled light traffic, route anomalies: automatic update of road signage according to traffic or conditions as well as automatic reporting of traffic. - Wireless Network Coverage: the users in different locations (cluster participants) can receive alerts concerning network quality in a given region, leading to proactive actions to manage the traffic flow. - Driving Safety and Reliability: early detection of unexpected situations and allowing cluster participants to experience improved driving capability in term of controlled speed and awareness. - Parking Application: eliminating the need to hunt parking lots in search of an open spot. The solution may be used for moving cluster of vehicles and static cluster. - Emergency Vehicle Prioritization: vehicles in the cluster can be notified, using a warning system, when emergency vehicles are nearby and need to pass.

[0030] The solution presented herein addresses the heterogeneous connectivity scenario where two types of vehicles (vehicles with only D2D capability and vehicles with Cellular and D2D capability labelled ad C&D2D) can participate in the clustering. Only vehicles of the C&D2D type can start a cluster.

[0031] The selection of the cluster head is based on Reinforcement Learning, with the introduction of the concept of standby cluster head to ensure high availability.

[0032] The cluster is formed dynamically; in other words, the vehicles can join or leave the cluster in an ad-hoc fashion based on their position related to the other vehicles in the cluster, as well as their destinations.

[0033] When the cluster head leaves the cluster, the application in the cloud finds the replacement of cluster head and standby pair. The application in the cloud should be able to predict the next cluster head or, if informed in advance, it can decide the replacement before the cluster runs into trouble. In a case where the vehicle (cluster head) is totally out of service, e.g., due to an accident, the remaining vehicles in the cluster should become standby until the application becomes aware of the situation and finds a cluster head replacement. The concept of High Availability is enforced bya pair of vehicles that have the capability of C&D2D, one being the cluster head or lead node and the other being a standby node. In the cluster, all vehicles that have C&D2D capability are candidates for both head and standby roles. The selection of head and standby nodes is done through a simple MAB, (reinforcement learning) by considering the destination, signal strength, the speed of the vehicle, and the Central Processing Unit (CPU) load of the vehicle as four factors of the reward function.

[0034] The solution presented herein is designed as an application where participant subscribe to benefit from integrating a network (cluster) of vehicles. From the vehicles supporting Cellular and D2D, at minimum two are expected to access the cellular network via a head / standby mechanism. An AI / ML (Reinforcement Learning) model is introduced to select which two vehicles on the move will be selected as the head and standby nodes.

[0035] Unnecessary cellular connectivity, such as excessive access, is avoided by allowing a unique access link to the cellular network via the cluster head. All other vehicles rely on the cluster head for communications out of the cluster (this applies to communications related to formation, use, and exit of the cluster through a dedicated application). For that purpose, the cluster head is assisted by the standby node to avoid abrupt drops and to maintain high availability of the application.

[0036] To ensure high availability, a pair of vehicles that have C&D2D capability are selected to manage the cluster. After the cluster is formed, devices and / or vehicles that have only D2D capabilities can also join the cluster. Communication between the devices and vehicles that have only D2D capability and the application in the cloud rely on the cluster head. On the other hand, the devices and vehicles that have C&D2D are allowed to join the cluster too and will become part of a selection pool of candidates for head or standby nodes, from which the head and standby can be picked up when there are changes in the cluster.

[0037] When the head or standby vehicle is foreseen to leave or is detected to leave the cluster, the procedure to find the replacement for the current head or / and standby vehicle is triggered. As a vehicle joins the cluster, speed, destination, position, and capability are used to estimate the time to stay in the cluster. If the vehicle is assigned as cluster head or standby, the system, based on MAB will trigger and anticipate the switch or exit accordingly. The selection from the pool of candidates is done by applying Multiarmed Bandit (reinforcement learning).

[0038] In the case where the cluster head and the standby vehicle would both leave at the same time, a few things may happen. The application could dismantle the cluster if only D2D vehicles remain in the cluster. Alternatively, if at least one of the remaining vehicles had cellular&D2D capabilities, it could start a new cluster. In this case, the application could reassign the same vehicles to the cluster, but it could also assign vehicles to neighbouring clusters.

[0039] A list of neighbouring clusters may be kept as a backup for each cluster. If no vehicle in the cluster that lost both its head and standby vehicles is qualified for head and standby, the application could select the other clusters for the remaining vehicles in the current cluster. The strategy to assign the cluster head and standby vehicles / nodes exploits two concepts: - The characteristics (destination, speed, cellular capability) of the C&D2D vehicles joining or leaving the cluster in an ad-hoc fashion. - The Multiarmed bandit (reinforcement learning) approach to decide the best strategy for selecting the cluster head and the standby vehicles / nodes and applying it in the cluster accordingly.

[0040] Referring to figure 1, as previously explained, to keep High Availability (HA) of the collaborative driving application 105 and avoid unnecessary communication with the 5G access nodes 110, a pair of moving vehicles are selected as head vehicle 115 and standby vehicle 120. The selection of these two moving vehicles is done via reinforcement learning (e.g., MAB) to maximize the rewards, which is a function of the following factors: - Destination of the moving vehicle; - Signal strength for the communication between the moving vehicle and the mobile network; - Speed of the moving vehicle; - CPU load of the moving vehicle; - Network bandwidth of the moving vehicle.

[0041] Figure 1 presents a concrete example for the journey from location A to location C.

[0042] At location A, a team of three trucks and two cars are formed into a single cluster 130. A team of three trucks moves from location A to location B. Since all three trucks have C&D2D capability, two of them are selected to be head 115 andstandby 120, to communicate with the driving application (App) 105 deployed in the mobile network 125 (or edge cloud). Here, the two cars in cluster 130 have only D2D capacity. Hence, they cannot be the candidates for head and standby selection.

[0043] According to the conditions, the head and standby can be switched from time to time as indicated in the left two boxes (time t1 and time t2) in Figure 1.

[0044] Before arriving at location B, two cars left the cluster (one at time t1 and another at time t2) and a team of two other trucks that are heading for location C joined the cluster 130. Those two new trucks have C&D2D capability.

[0045] Now a pool of five trucks is used for head and standby selection. After applying Reinforcement Learning (RL) model, the two new trucks are picked as head and standby at time t3.

[0046] At location B, a team of three trucks left the cluster. One car (only D2D capability) joined the cluster at time t4. Now the two remaining trucks are responsible for the communication to the driving application 105 deployed in the cloud via mobile the network 125. The journey for this cluster ends at location C.

[0047] The solution presented herein deals with heterogeneous vehicle networks or clusters. It is assumed that the participants vehicles may be equipped with cellular connectivity or Direct Short-Range Communication (DSRC) capabilities. The solution is fully autonomous and relies on 5G capability and the cloud to manage the cluster. The solution may provide several advantages, such as: - It can be a bridge or transition element to accelerate V2X adoption. - It can improve data transfer by preventing the access node from being exhausted. This is done by avoiding unnecessary communication over 5G. Only the cluster head is maintained connected and is expected to send data, especially control plane data, over the 5G network. In inter-cluster scenario, data could be transmitted using DSRC capable vehicle in the appropriate range. - It is communications service provider (CSP) agnostic. - It can enhance driving capability and experience of the participants by disseminating data such as real-time position, heading and speed of the participant vehicles, using either 5G or DSRC capabilities. These information statuses are aggregated in the cloud for model training, updating and inference, allowing the participant vehicles to be aware of alarms and all other driving conditions. - The concept of high availability of the application is materialized with the setting of the cluster head node / standby node, to maintain the 5G connectivity andavailability. This allows the cluster to be always connected to 5G either via the cluster head or the standby node. - The solution provides a soft alternative to platoons, with enhanced capabilities: by allowing vehicles with distinct destinations to use the application. Vehicles are free to leave or join the clusters at any moment. - The solution can be extended to drones to collaboratively achieve a given task or fight plan.

[0048] Figure 2 illustrates a scenario where a dynamic cluster of heterogenous connected vehicles are collaboratively interacting and moving from location A towards location B. It is required in such scenario to properly handle data transfer through reliable connectivity. For this, only one vehicle with C&D2D capability manages the cluster, the cluster head. A standby vehicle or node is also selected to avoid drops and enable smooth transition of the cluster head exiting the cluster.

[0049] The landscape of the network access in V2X scenario is shown in Figure 2. Some vehicles only have vehicle to vehicle communication capabilities, some have vehicle to infrastructure (V2I) capabilities, some have vehicle to device capabilities, etc.

[0050] Turning to figure 3, the application 105 can be executed to control clusters of vehicles in a smart city or on highways with reliable 5G coverage and Road Side Units (RSUs). Let’s consider the network of vehicles (cluster) as shown in Figure 3. At least one 5G / Long Term Evolution (LTE) cellular capable vehicle (C&D2D) is mandatory to form or start the cluster. It should be noted that although examples are provided in the context of 5G networks, the solution described herein can be applied in networks of the future. As shown in Figure 3, only one vehicle in the cluster is expected to orchestrate the application (also known as the cluster head 115, lead node or active node). A back-up vehicle (standby node120) is assigned to the cluster and works side-by-side with the cluster head, with the ability to switch the role with the cluster head.

[0051] Figure 3 applies in general roadways with 5G cellular infrastructure enabled. Therefore, the application is deployed in an area with 5G cellular coverage. DSRC links between connected vehicles are also indicated with lines between the vehicles. The reader will note that standard vehicles (without connectivity) cannot participate in collaborative driving.

[0052] Initialization of the cluster of vehicles, preliminary phase

[0053] The application 105 resides in the cloud and starts with setting up the cluster. Some conditions are required to start a cluster of vehicles. It is assumed that more vehicles only support D2D than C&D2D. Only vehicles with C&D2D capability will start a cluster or network of vehicles. D2D vehicles cannot start a cluster. Only vehicles that have subscribed to the application can join the cluster.

[0054] It is also required that each C&D2D vehicles joining the cluster should be included in a pool of potential devices that can be elected cluster head node 115 or standby node 120, while maintaining their cellular 5G connectivity on standby and relying only on the cluster head for communication with the application 105. The cluster head node 115 and standby node 120 should be selected by the Reinforcement Learning (RL) algorithm among the C&D2D vehicles.

[0055] A vehicle at a distance above a certain configurable distance threshold (for example 1km, which is a distance restricted by the DSRC range), from the cluster head back and front cannot join the cluster. Such a vehicle will be routed to the nearest cluster. If the vehicle is C&D2D capable, it can initiate a new cluster.

[0056] Phase 1, initialization phase, cluster head as cluster initializer.

[0057] During the initialization phase, the cluster head 115 (C&D2D) will start the cluster and will handle all cellular operation in the cluster until another C&D2D capable vehicle joins the cluster. No shift will be allowed until a standby node 120 is added to the cluster. The second C&D2D vehicle joining the cluster is directly assigned as standby node 120, according to the processing described in phase 2. All D2D vehicles joining the cluster are simply considered as participants.

[0058] A discovery mechanism through Multicast Domain Name Service (mDNS) / Message Queueing Telemetry Protocol (MQTT) and identification could be used to allow a vehicle to join the cluster, such as described in international patent application PCT / IB2022 / 058979, by the same inventors. It is assumed that each vehicle accessing the cluster provides to the application 105 some basic information on their destination, speed, and position.

[0059] Phase 2, C&D2D capable vehicles joining the cluster.

[0060] Each C&D2D vehicle is assigned with a set of parameters called Time to Stay in the Cluster (TTSC), and Reference Physical Connectivity (RNC).

[0061] The TTSC is a function of the speed, the destination and the position, which approximates the time the vehicle is expected to stay in the cluster. The RNC is afunction of the cellular plan characterizing the effectiveness of the communication between the mobile network and the vehicle. ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) = ^^^^( ^^^^ ^^^^ ^^^^ ^^^^ ^^^^, ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^, ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^) (1) and ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) = ^^^^( ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^) (2)

[0062] For a particular vehicle, ^^^^& ^^^^2 ^^^^^^^^, when it enters the cluster, a standby node factor (StandbyNode_factor), which is an indication of the suitability or reliability of a node to be standby node, is estimated as follows: ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) = ^^^^^^^^ ^^^^∙ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) + (1 − ^^^^^^^^ ^^^^) ∙ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) (3) where ^^^^^^^^ ^^^^is a tuning smoothing parameter between 0 and 1 that can be activated to favor time to stay in the cluster or effective connectivity of the vehicle.

[0063] Let us assume a current standby node vehicle, with label ^^^^& ^^^^2 ^^^^^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^, then, a new C&D2D vehicle entering the cluster is labelled: ^^^^& ^^^^2 ^^^^^^^^and will replace the standby node vehicle if: ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) > ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^) (4)

[0064] Anytime a new C&D2D capable vehicle joins the cluster, the test above is executed to verify whether the current standby node vehicle could be replaced or not. This operation is also activated when switching between cluster head and standby node occurs. The new standby node vehicle is compared against the best or most reliable C&D2D vehicle in the cluster.

[0065] Phase 3, switching between the cluster head and the standby node, introducing MAB algorithm.

[0066] Reinforcement Learning is qualified as a solution for the problem of switching between the cluster head and the standby node. Candidates for this RL could be, Deep Q-Learning, MAB, etc. Herein, a standard Reinforcement Learning variant is adopted to perform the switching between the cluster head and the standby node. The Multiarmed bandit is used, but any other RL model could be used instead. The reason why the MAB was adopted is for its simplicity, the limited action and the limited space.

[0067] The application 105 running the RL resides in the cloud and has full view on the cluster. Let’s consider the initial time ^^^^0, for the proposed algorithm, as the moment a new standby node is assigned. In the bandit game, the available parametersfor the forecaster are the number of arms (or action), two in the present example (cluster head and standby node), and the number of rounds n, unknown to the forecaster. The gain vector ^^^^^^^^= ( ^^^^1, ^^^^, ^^^^2, ^^^^) at each round ^^^^ is generated as follow:

[0068] For each round ^^^^ = 1, 2, … , ^^^^ - The forecaster chooses an arm ^^^^^^^^∈ (1,2), - The forecaster receives the gain ^^^^^^^^ ^^^^, ^^^^, which is extrapolated by the environement and is computed using the number of rounds and the chosen arm, - Only ^^^^^^^^ ^^^^, ^^^^is revealed to the forecaster.

[0069] The cumulative regret goal is to maximize the cumulative gains obtained. More precisely, the goal is to minimize:

[0070] Where ^^^^^^^^is the expected cumulative regret, and the expectation ^^^^ comes from both a possible stochastic generation of the gain vector and a possible randomization in the choice of ^^^^^^^^.

[0071] In a bandit game, the environment is stochastic (randomly determined). The gain vector ^^^^^^^^is sampled from an unknown product distribution (of reward distributions) × ^^^^ on22[0,1 ] that is ^^^^^^^^, ^^^^≈ ^^^^^^^^. Also, the environment is in the way that the gain vector ^^^^^^^^is chosen by an adversary (which at time ^^^^, knows all the past, but not ^^^^^^^^). There are several existing variants of bandit problem and several applications.

[0072] The MAB problems were first introduced by H. Robbins in 1952 and are used to model the tradeoff faced by an automated agent which aims to gain new knowledge by exploring its environment and to exploit its current knowledge. H. Robbing in [H. Robbins, Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society, 58(5):527–535, 1952] maps the MAB problem into the class of partial-information sequential resource allocation problems, concerned with allocating between multiple options, where the benefit of each option is not known at the time of allocation. The benefit is thus discovered as time passes and resources are reallocated.

[0073] With the stochastic bandit game introduced by Robbins, the unknown parameters to the forecaster are the reward distributions ^^^^1, ^^^^2of the arms (with respective mean ^^^^1, ^^^^2). The algorithm is deployed as follows:

[0074] For each round ^^^^ = 1, 2, … , ^^^^:- The forecaster chooses an arm ^^^^^^^^∈ (1,2), - The environment draws the gain vector ^^^^^^^^= ( ^^^^1, ^^^^, ^^^^2, ^^^^) according to ^^^^2, - The forecaster receives the gain ^^^^^^^^ ^^^^, ^^^^.

[0075] Notation:

[0076] The cumulative regret is given by:

[0077] Finally, the goal is to minimize the expected cumulative regret:

[0078] Mapping of the problem to the bandit algorithm.

[0079] In such a problem, there is a set of arms, two in the current problem (cluster head and standby node), each of which, when played or pulled by the forecaster yields some reward depending on its internal state which evolve stochastically over time.

[0080] Decomposition of the reward.

[0081] To draw the adopted reward, a switching between the cluster head and the standby node is considered in the following scenario. All parameters in the criteria below are configurable. The default values can be set based on experimental results. Here, example values are provided for illustration purposes and are normalized between 0 and 100.

[0082] The vehicle is about to exit the cluster, meaning the time to stay in the cluster is expiring. A parameter called the Exit Criteria (EC) is defined, relative to the cluster, as follows (associated with distance up to go within the cluster): ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) > 2 ^^^^ ^^^^ ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 100 ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ 1 ^^^^ ^^^^ < ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) ≤ 2 ^^^^ ^^^^ ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 75 ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ 500 ^^^^ > ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) ≤ 1 ^^^^ ^^^^ ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 50 ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) < 500 ^^^^ ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 0 (9)

[0083] If the speed of vehicle is not stationary relative to the cluster, a parameter called Speed Criteria is defined as follows: ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^−1) − ^^^^^^^^< ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) < ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^−1) + ^^^^^^^^, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 100 (10) ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) < ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^−1) − ^^^^^^^^^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) > ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^−1) + ^^^^^^^^, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 0 (11)

[0084] A Mobile Signal quality parameter, labelled as High, Medium, or low, and characterized as a function of Signal-to-Noise Ratio (SNR), of the Channel Quality Indicator (CQI), which characterizes the wireless channel quality as a scalar between [0, 15] in 5G and maps the channel quality with the modulation and coding scheme, and of the throughput, also referenced as Signal Criteria, is in a certain range and is defined as follows: ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) = ^^^^ ^^^^ ^^^^ℎ, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 100 ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) = ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 50 ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) = ^^^^ ^^^^ ^^^^, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 25 (12)

[0085] The compute resource availability of the vehicle may be dropping or may be exhausted. This is represented by a parameter called CPU Criteria. Typically, the CPU load is in the range of 50%,75%,80%. CPU Criteria is defined as follows: ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) < 50%, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 100 ^^^^ ^^^^ 50% < ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) < 75%, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 50 ^^^^ ^^^^ 75% < ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) < 80%, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 25 ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) > 80%, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 0 (13)

[0086] The network bandwidth (BW) availability of the vehicle may be dropping or may be exhausted. This is represented by a parameter called BW Criteria. The usage of network BW is typically in the range of 50%,75%,80%. BW Criteria is defined as follows: ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) < 50%, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 100 ^^^^ ^^^^ 50% < ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) < 75%, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 50 ^^^^ ^^^^ 75% < ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) < 80%, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 25 ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^& ^^^^2 ^^^^^^^^) > 80%, ^^^^ℎ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 0 (14)

[0087] Five smoothing parameters can then be suggested to aggregate the overall reward for each C&D2D (cluster head and standby node): ^^^^^^^^ ^^^^ ^^^^ ^^^^, ^^^^^^^^ ^^^^ ^^^^ ^^^^ ^^^^, ^^^^^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^, ^^^^^^^^ ^^^^ ^^^^and ^^^^^^^^ ^^^^. So that at each round or iteration, the vehicle availability is estimated as follow:^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^) + ^^^^^^^^ ^^^^ ^^^^∙ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^) + ^^^^^^^^ ^^^^∙ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^( ^^^^) (15) with the requirement that:where K is either the cluster head or the standby node. The objective is to select as cluster head the vehicle that maximizes the criteria, meaning high ^^^^(∙).

[0088] Fitting the Switching mechanism into the MAB algorithm.

[0089] Turning to figure 4, a process 400 is presented which can be used to decide which vehicle should stay as cluster head between the current cluster head and the standby node. The forecaster (described previously) is the application 105 unaware of the stochastic evolution of the two vehicles, with the objective at maximizing their ability to control the cluster. The application 105 wants to validate which vehicle (C&D2D) is a reliable candidate to play the role of cluster head and manage the cluster.

[0090] Step 410: a first C&D2D vehicle forms the cluster.

[0091] Step 415: a standby vehicle (standby node) joins the cluster.

[0092] Step 420: the application 105 sends to the MAB algorithm 405 the reference of the two vehicles ^^^^& ^^^^2 ^^^^^^^^(arms), through the cluster of vehicles 415, which also acquires this information.

[0093] Step 425: the MAB formulation is established for the arms.

[0094] Step 430: the MAB algorithm 405 sets all criteria ^^^^^^^^and the number of times a vehicle is selected ^^^^^^^^to zero.

[0095] Step 435: at iteration m, initially m = 1, the application 105 requests the identity of the reliable vehicle from the MAB algorithm. Information from the cluster of vehicles such as Exit criteria, Speed criteria, Signal criteria and CPU criteria for cluster head and standby node is provided, step 440, from the cluster of vehicles 130 to the MAB algorithm 405.

[0096] Step 445: The MAB 405 releases the vehicle with the highest criteria (maximum reward).

[0097] Steps 450, 460: the (new) cluster head and standby nodes are sent to the application through the cluster of vehicles 130. The cluster head and standby nodes, if changed, are reassigned, step 455, in the cluster of vehicles 130.

[0098] Step 470: the application 105 then relies on this selected vehicle to execute the tasks assigned to the cluster head.

[0099] Step 475: the MAB algorithm 405 consolidates the criteria ^^^^( ^^^^)^^^^evaluated for the vehicles ^^^^& ^^^^2 ^^^^^^^^and updates ^^�^^^^^^and ^^^^^^^^according to the algorithm used. The MAB algorithm then becomes ready for the next iteration.

[0100] After step 470, the application 105 constantly verifies the effectiveness of the operational status. - If the application is running properly, then the application executes step 480. The delay between the execution of the iterations can be configurable. - If the application identities that the vehicles have changed, meaning if the standby is replaced by another vehicle, for example, all resources are released, the MAB sequence ends, and the process continues from step 415.

[0101] Figure 5 and Figure 6 illustrate how the two activities described in relation to figure 4 occur, namely when new cars join the cluster and there is a switching between standby vehicles and when there is a new assignation of cluster head.

[0102] In figure 5, vehicle I is cluster head and vehicle II is the standby node. Two new vehicles join the cluster at location B.

[0103] In figure 6, at location C, the vehicle I remains cluster head, but vehicle III becomes the standby node. At location D, there is yet another change, the cluster head and standby node are switched. Vehicle III becomes cluster head and vehicle I becomes the standby node. This example clearly illustrates the dynamicity of the solution.

[0104] The solution proposed herein is proactive instead of passive. The prediction of overload of any node including cluster head and standby noise is captured by the observed state and the rewards in the MAB algorithm.

[0105] The selection of the cluster head and standby node will be done by MAB from time to time based of observed state and the rewards thus tackling any consideration of overload of cluster head or standby node.

[0106] Turning to figure 7, there is provided a computer implemented method 700, executed by an application running in a cloud computing network, for managing a cluster of vehicles. The method comprises receiving, step 701, a request from a first vehicle, through a mobile communications network, for establishing a cluster of vehicles. The first vehicle is the cluster initializer, which has cellular and D2D (C&D2D) capabilities. To achieve high availability, the minimum number of devices that have C&D2D is two, but the application may still work without enhance capabilities of high availability.

[0107] The method comprises assigning, step 702, the first vehicle as cluster head and authorizing the first vehicle to start broadcasting information about thecluster using short range communication, and to establish communication with other vehicles. A discovery mechanism that could be adopted for that purpose is either MQTT or MDNS, as described in PCT / IB2022 / 058979, by the same inventors. A baseline for using MQTT with proper authentication scheme is also described by Robert Sandor et al, “Vehicle2X communication proposal for Adaptive AUTOSAR”, IEEE ICCE, 2018. Such solutions would work in combination with a Secure Credential Management System (SCMS). The cluster head, when re-assigned, is picked by the application in the cloud, then it broadcasts the message through D2D, and waits for the D2D vehicles to join in. Then the cloud application can retrieve the D2D vehicle information via the cluster head. Later on, based on the update status of the D2D vehicle and cluster head, the application in the cloud can decide to assign or unassign a specific vehicle to the cluster (such as a moving vehicle) from one cluster to the other. This is also the case when the cluster head has been changed.

[0108] The method comprises receiving, step 703, a request from at least one second vehicle, through the cluster head, for joining the cluster. Clustering is made dynamically. Vehicles joining the cluster are expected to move from location A to location B. In some applications, locations may be restricted by the 5G coverage and the application availability. Joining / exiting is possible as the cluster is formed by a vehicle supporting 5G connectivity.

[0109] The method comprises assigning, step 704, a standby node, selected among the at least one second vehicle, the standby node having mobile communications network capacity (or capability). The cluster head is assisted by a standby node during the execution of the application. The MAB approach is adopted to manage the switch between the cluster head and the standby node using a ranking mechanism. The principle of cluster head and standby node could be easily adjusted to deal with more than one standby node. In that case, the MAB is still a valid solution, providing more flexibility.

[0110] The method comprises, periodically assigning, step 705, a new cluster head, to the cluster of vehicles, using a reinforcement learning model.

[0111] Periodically assigning may further comprise periodically assigning a new standby node.

[0112] The new cluster head and the new standby node may be selected among vehicles of the cluster having mobile communications network capacity. Insidethe cluster (network of vehicles) there are two subgroups of vehicles. One with Cellular+D2D (C&D2D) capabilities, one with D2D only capabilities.

[0113] The standby node may be assigned as the new cluster head and the cluster head may be assigned as the new standby node.

[0114] The reinforcement learning model may apply a ranking mechanism to select the new cluster head and the new standby node.

[0115] The method may further comprise receiving, step 706, a notification from the cluster head indicating that the cluster head is leaving the cluster of vehicles, assigning, step 707, the standby node as the new cluster head, and assigning, step 708, a new standby node. The participants are not restricted by the driving plan of other participants; any participant can drop from the cluster anytime. Such application is suitable for transportation between major cities, for example, with the option for participants to join or leave the cluster randomly. As soon as the cluster head is about to drop from the cluster, it may be replaced by the standby automatically based a Reinforcement Learning (RL) solution. A solution may also be provided to re-assigns a new standby task to another cellular capable vehicle.

[0116] The method may further comprise initiating broadcasting, step 709, by the cluster head, of the cluster head and the standby node identities to the cluster of vehicles.

[0117] The method may further comprise receiving, step 710, a notification from the cluster head indicating that at least one vehicle requests joining or leaving the cluster of vehicles and assigning the at least one vehicle to the cluster or unassigning the at least one vehicle from the cluster.

[0118] The cluster head may relay, to the application in the cloud, all communications from the cluster of vehicles and wherein the standby node is operative to take over the relay the any communications that the cluster head is not able to relay. Handling Dead-Ends (Random Access Link Disconnection): High availability of the application is maintained by the Reinforcement Learning model. The RL model helps the cluster to stick on more reliable access link with a standby (C&D2D) considered as guard node in case of disruption. The RL approach ensures that a connection to 5G is maintained. The cluster head (active / head C&D2D) should be always connected to the 5G network. The backup (standby C&D2D) is used to automatically handle the 5G connection if the head vehicle drops or if its link quality becomes useless.

[0119] The reinforcement learning model may be a Multiarmed Bandit (MAB) based model.

[0120] The vehicles may be selected among any of the categories comprising: drones, cars, trucks, motorcycles, bicycles, planes, trains, boats and unmanned vehicles. Unmanned vehicles may include any moving object. It is envisioned that moving objects could be virtual objects e.g., in the metaverse, for which the methods described herein could be applied.

[0121] Turning to figure 8, there is provided a computer implemented method 800, executed by a vehicle. The method comprises sending, step 801, a request to an application running in a cloud computing network, through a mobile communications network, for establishing a cluster of vehicles. The method comprises receiving, step 802, an assignment as cluster head and an authorization to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles. The method comprises forwarding, step 803, a request for joining the cluster, from at least one second vehicle, to the application. The method comprises forwarding, step 804, a standby node assignment, to a vehicle selected among the at least one second vehicle, the standby node having mobile communications network capacity. The method comprises periodically receiving, step 805, a new cluster head assignment, made using a reinforcement learning model, and initiating assignment of the cluster head.

[0122] Periodically receiving may further comprise periodically receiving a new standby node assignment.

[0123] The new cluster head and the new standby node may be selected among vehicles having mobile communications network capacity.

[0124] The the standby node may be assigned as the new cluster head and the cluster head may be assigned as the new standby node.

[0125] The reinforcement learning model may apply a ranking mechanism to select the new cluster head and the new standby node.

[0126] The method may further comprise sending, step 806, a notification, to the application running in a cloud, indicating that the cluster head is leaving the cluster of vehicles, receiving and forwarding, step 807, a cluster head assignment, to the standby node and receiving and forwarding, step 808, a standby node assignment, to another vehicle selected among the at least one second vehicle. This step may be done either by the old cluster head or by the standby node (new cluster head). It ispossible that the application in the cloud already has in the background a potential standby node (most reliable c&D2D, see EQ 3 and EQ 4) in the cluster that should become the standby node. If there is no C&D2D remaining in the cluster and the previous standby node becomes the new cluster head, it is possible that there is no standby node until a new C&D2D joins the cluster.

[0127] The method may further comprise broadcasting, step 809, the cluster head and the standby node identities to the cluster of vehicles.

[0128] The method may further comprise sending, step 810, a notification, to the application running in a cloud, indicating that at least one vehicle requests joining or leaving the cluster of vehicles and receiving an assignment of the at least one vehicle to the cluster or an unassignment of the at least one vehicle from the cluster.

[0129] The method may further comprise relaying, step 811, to the application in the cloud, all communications from the cluster of vehicles. The standby node may be operative to take over relaying the any communications that the cluster head is not able to relay.

[0130] The reinforcement learning model may be a Multiarmed Bandit (MAB) based model.

[0131] The vehicles may be selected among any of the categories comprising: drones, cars, trucks, motorcycles, bicycles, planes, trains and boats.

[0132] It should be noted that methods and steps described herein are, generally, computer implemented methods and steps. The term computer may be interpreted as having different meanings, such as explained next, for example.

[0133] Referring to figure 9, there is provided an apparatus (HW) 901, in which functions and steps described herein can be implemented.

[0134] The apparatus 901 (which may go beyond what is illustrated in figure 9), may be a server, network node, radio base station, or other computing device which may be part of a cloud computing system, edge computing system, or which may be a standalone device.

[0135] The apparatus 901 may be a vehicle, such as a drone, a car, a truck, a motorcycle, a bicycle, a plane, a train, a boat or any other equivalent means of transportation or moving vehicle or object.

[0136] C&D2D (higher capability vehicles) or D2D (vehicles with lower capability) systems use a combination of hardware and software, including vehicle controllers, inverters and communication protocols, to enable bi-directional flow ofdata between the vehicle and any other external system. The Application which resides in the cloud acts as a central controller, where it is instantiated. The application in the cloud has a powerful database and virtual machine computing capability. Each vehicle participating in the cluster is assigned an IP address used for discovery and authentication as it enters the cluster. The vehicle should contain both Transmission Control Protocol (TCP)and User Datagram Protocol (UDP)application and transport layer submodules. This is where the application written for the vehicle will be inserted.

[0137] Some of the steps executed by involved vehicles include: discovery based on the IP address, data collection (memory), data processing (cleaning collected data, data compression, processing received data to extract knowledge, display or alert to the driver), data send / receive to the cluster head, information exchange with the vehicle controller to access sensors, security. At high level, the apparatus provides capability for mobile connectivity, such as eSIM or SIM card slot, wireless connectivity, such as WiFi and Bluetooth, that are configurable. From the software point of view, the cloud application is able to manage the devices (either C&D2D or D2D) via an agent application deployed on the devices. It imposes the connection for data transfer between devices, as well as between device and the application in the cloud via the mobile network. The business logic regarding the cluster management (including the head and standby) is done at the software level in the cloud application. Looking forward, in a virtual world (i.e., metaverse, digital twins), the same techniques could be applied in relevant virtual scenarios.

[0138] The apparatus 901 comprises processing circuitry 903 and memory 905. The memory 905 can contain instructions executable by the processing circuitry 903 whereby functions and steps described herein may be executed to provide any of the relevant features and benefits disclosed herein.

[0139] The apparatus 901 may also include non-transitory, persistent, machine-readable storage media 907 having stored therein software and / or instruction 909 executable by the processing circuitry 903 to execute functions and steps described herein. The apparatus may also include network interface(s) and a power source.

[0140] The instructions 909 may include a computer program for configuring the processing circuitry 903. The computer program may be stored in a physical memory local to the device, which can be removable, or it could alternatively, or inpart, be stored in the cloud. The computer program may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0141] Referring to figure 10, there is provided a virtualization environment 1000 in which functions and steps described herein can be implemented.

[0142] The virtualization environment 1000 (which may go beyond what is illustrated in figure 10), may comprise systems, networks, servers, nodes, devices, etc., that are in communication with each other either through wire or wirelessly, e.g., through a network interface component (NIC) comprising physical network interface(s). Some or all of the functions and steps described herein may be implemented as one or more virtual components (e.g., via one or more applications, components, functions, virtual machines, containers, etc.) executing on one or more physical apparatus in one or more networks, systems, environment, etc.

[0143] A virtualization environment provides hardware 1001 comprising processing circuitry 1003 and memory 1005. The memory 1005 can contain instructions executable by the processing circuitry 1003 whereby functions and steps described herein may be executed to provide any of the relevant features and benefits disclosed herein.

[0144] The hardware 1001 may also include non-transitory, persistent, machine-readable storage media 1007 having stored therein software and / or instruction 1009 executable by the processing circuitry 1003 to execute functions and steps described herein.

[0145] The instructions 1009 may include a computer program for configuring the processing circuitry 1003. The computer program may be stored in a removable memory, such as a portable compact disc, portable digital video disc, or other removable media. The computer program may be stored in a physical memory local to the hardware 1001, which can be removable, or it could alternatively, or in part, be stored in the cloud. The computer program may also be embodied in a carrier such as an electronic signal, optical signal, radio signal, or computer readable storage medium.

[0146] Referring again to figures 9 and 10, there is provided an apparatus 901, 1001, in a cloud computing network, comprising processing circuitry 903, 1003 and a memory 905, 1005. The memory contains instructions executable by the processing circuitry whereby the apparatus is operative to receive a request from a first vehicle,through a mobile communications network, for establishing a cluster of vehicles. The apparatus is operative to assign the first vehicle as cluster head and authorize the first vehicle to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles. The apparatus is operative to receive a request from at least one second vehicle, through the cluster head, for joining the cluster. The apparatus is operative to assign a standby node, selected among the at least one second vehicle, the standby node having mobile communications network capacity. The apparatus is operative to periodically assign a new cluster head, to the cluster of vehicles, using a reinforcement learning model.

[0147] The apparatus is further operative to execute any of the steps of the methods described herein.

[0148] There is provided a vehicle 115 (fig.1) (the vehicle comprising the components of, or the whole, apparatus 901), comprising processing circuitry 903 and a memory 905. The memory 905 contains instructions executable by the processing circuitry 903 whereby the vehicle is operative to send a request to an application running in a cloud computing network, through a mobile communications network, for establishing a cluster of vehicles. The vehicle is operative to receive an assignment as cluster head and an authorization to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles. The vehicle is operative to forward a request for joining the cluster, from at least one second vehicle, to the application. The vehicle is operative to forward a standby node assignment, to a vehicle selected among the at least one second vehicle, the standby node having mobile communications network capacity. The vehicle is operative to periodically receive a new cluster head assignment, made using a reinforcement learning model, and initiating assignment of the cluster head.

[0149] The vehicle is further operative to execute any of the steps of the methods described herein.

[0150] There is also provided a non-transitory computer readable media 907, 1007, containing instructions for executing any one or more of the steps described herein.

[0151] Modifications will come to mind to one skilled in the art having the benefit of the teachings presented in the foregoing description and the associated drawings. Therefore, it is to be understood that modifications, such as specific forms other than those described above, are intended to be included within the scope of thisdisclosure. The previous description is merely illustrative and should not be considered restrictive in any way. The scope sought is given by the appended claims, rather than the preceding description, and all variations and equivalents that fall within the range of the claims are intended to be embraced therein. Although specific terms may be employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

Claims

CLAIMS 1. A computer implemented method, executed by an application running in a cloud computing network, for managing a cluster of vehicles, comprising: - receiving a request from a first vehicle, through a mobile communications network, for establishing a cluster of vehicles; - assigning the first vehicle as cluster head and authorizing the first vehicle to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles; - receiving a request from at least one second vehicle, through the cluster head, for joining the cluster; - assigning a standby node, selected among the at least one second vehicle, the standby node having mobile communications network capacity; and - periodically assigning a new cluster head, to the cluster of vehicles, using a reinforcement learning model.

2. The method of claim 1, wherein periodically assigning further comprises periodically assigning a new standby node.

3. The method of claim 2, wherein the new cluster head and the new standby node are selected among vehicles of the cluster having mobile communications network capacity.

4. The method of claim 2 or 3, wherein the standby node is assigned as the new cluster head and the cluster head is assigned as the new standby node.

5. The method of any one of claims 2 to 4, wherein the reinforcement learning model applies a ranking mechanism to select the new cluster head and the new standby node.

6. The method of any one of claims 1 to 5, further comprising: - receiving a notification from the cluster head indicating that the cluster head is leaving the cluster of vehicles; - assigning the standby node as the new cluster head; and- assigning a new standby node.

7. The method of any one of claims 1 to 6, further comprising initiating broadcasting, by the cluster head, of the cluster head and the standby node identities to the cluster of vehicles.

8. The method of any one of claims 1 to 7, further comprising receiving a notification from the cluster head indicating that at least one vehicle requests joining or leaving the cluster of vehicles and assigning the at least one vehicle to the cluster or unassigning the at least one vehicle from the cluster.

9. The method of any one of claims 1 to 8, wherein the cluster head relays, to the application in the cloud, all communications from the cluster of vehicles and wherein the standby node is operative to take over the relay the any communications that the cluster head is not able to relay.

10. The method of any one of claims 1 to 9, wherein the reinforcement learning model is a Multiarmed Bandit (MAB) based model.

11. The method of any one of claims 1 to 10, wherein the vehicles are selected among any of the categories comprising: drones, cars, trucks, motorcycles, bicycles, planes, trains, boats and unmanned vehicles.

12. A computer implemented method, executed by a vehicle, comprising: - sending a request to an application running in a cloud computing network, through a mobile communications network, for establishing a cluster of vehicles; - receiving an assignment as cluster head and an authorization to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles; - forwarding a request for joining the cluster, from at least one second vehicle, to the application; - forwarding a standby node assignment, to a vehicle selected among the at least one second vehicle, the standby node having mobile communications network capacity; and- periodically receiving a new cluster head assignment, made using a reinforcement learning model, and initiating assignment of the cluster head.

13. The method of claim 12, wherein periodically receiving further comprises periodically receiving a new standby node assignment.

14. The method of claim 13, wherein the new cluster head and the new standby node are selected among vehicles having mobile communications network capacity.

15. The method of claim 13 or 14, wherein the standby node is assigned as the new cluster head and the cluster head is assigned as the new standby node.

16. The method of any one of claims 13 to 15, wherein the reinforcement learning model applies a ranking mechanism to select the new cluster head and the new standby node.

17. The method of any one of claims 12 to 16, further comprising: - sending a notification, to the application running in a cloud, indicating that the cluster head is leaving the cluster of vehicles; - receiving and forwarding a cluster head assignment, to the standby node; and - receiving and forwarding a standby node assignment, to another vehicle selected among the at least one second vehicle.

18. The method of any one of claims 12 to 17, further comprising broadcasting the cluster head and the standby node identities to the cluster of vehicles.

19. The method of any one of claims 12 to 18, further comprising sending a notification, to the application running in a cloud, indicating that at least one vehicle requests joining or leaving the cluster of vehicles and receiving an assignment of the at least one vehicle to the cluster or an unassignment of the at least one vehicle from the cluster.

20. The method of any one of claims 12 to 19, further comprising relaying, to the application in the cloud, all communications from the cluster of vehicles and wherein the standby node is operative to take over relaying the any communications that the cluster head is not able to relay.

21. The method of any one of claims 12 to 20, wherein the reinforcement learning model is a Multiarmed Bandit (MAB) based model.

22. The method of any one of claims 12 to 21, wherein the vehicles are selected among any of the categories comprising: drones, cars, trucks, motorcycles, bicycles, planes, trains and boats.

23. An apparatus, in a cloud computing network, comprising processing circuits and a memory, the memory containing instructions executable by the processing circuits whereby the apparatus is operative to: - receive a request from a first vehicle, through a mobile communications network, for establishing a cluster of vehicles; - assign the first vehicle as cluster head and authorize the first vehicle to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles; - receive a request from at least one second vehicle, through the cluster head, for joining the cluster; - assign a standby node, selected among the at least one second vehicle, the standby node having mobile communications network capacity; and - periodically assign a new cluster head, to the cluster of vehicles, using a reinforcement learning model.

24. The apparatus of claim 23 further operative to execute the method according to any one of claims 2 to 11.

25. A vehicle, comprising processing circuits and a memory, the memory containing instructions executable by the processing circuits whereby the vehicle is operative to:- send a request to an application running in a cloud computing network, through a mobile communications network, for establishing a cluster of vehicles; - receive an assignment as cluster head and an authorization to start broadcasting information about the cluster using short range communication, and to establish communication with other vehicles; - forward a request for joining the cluster, from at least one second vehicle, to the application; - forward a standby node assignment, to a vehicle selected among the at least one second vehicle, the standby node having mobile communications network capacity; and - periodically receive a new cluster head assignment, made using a reinforcement learning model, and initiating assignment of the cluster head.

26. The vehicle of claim 25 further operative to execute the method according to any one of claims 13 to 22.

Images