Communication system and method for cluster management and coordination

Abstract

The invention relates to a system for coordinating and managing a cluster of automated entities, the cluster comprising at least a specified master entity and slave entities. Each automation entity includes a data processing mechanism, at least one processor, coupled with at least one memory and adapted to execute instructions that facilitate operation of a program module, and each automation entity has particular assets and capabilities necessary to collaborate with other entities and perform missions as a cluster. A cluster management module is instantiated at the host entity, the cluster management module including a mission planning manager adapted to receive or load a mission and provide a mission configuration. The specified cluster master entity is configured to announce itself through the communication module and communication link, and to request at least one automation entity different from and external to the master entity to join the execution of the mission as a cluster slave entity. A method of coordinating and managing automated entity clusters is also provided. The method includes receiving, by the host entity, an indication of a mission assigned to the cluster; sending, by the master entity, a master entity announcement message to at least one potential slave entity that is directly reachable or indirectly reachable over at least one communication link, the master entity announcement message comprising a mission type; receiving, by all reachable potential slave entities, the master entity announcement message to check whether at least one potential slave entity has a profile that matches the mission type, the profile including assets and capabilities that match the announced mission type; at least one cluster join message is sent by at least one of the slave entities having a profile matching the announced mission type directly or indirectly through the communication link to the master entity in response.

Description

Technical Field

[0001] The present invention relates to a communication method and system for coordinating and managing a cluster of automated entities, and relates to the field of connectivity of autonomous operating devices. Background Art

[0002] Many industries (e.g., agriculture, construction, mining, logistics) view higher levels of autonomy as a means to improve operational efficiency and potentially save resources such as energy, emissions, and personnel. This includes automated interactions between automated machines / vehicles as well as their orchestration. Connectivity is therefore seen as a key enabler for optimizing operational efficiency.

[0003] There are already existing connectivity solutions for off-highway use cases, where the corresponding solutions vary widely in terms of application areas and requirements. For example, there are already commercial solutions for cloud-controlled swarms of smart farming robots (e.g., Fendt's Xaver2), but they are mainly coordinated via a central cloud platform and therefore require cloud connectivity. Commercial solutions have also been announced by Komatsu (V2V mining), JCA (fleet management of autonomous machines), Rajant Technologies (so-called "Kinetic Mesh" technology), Kinèis satellite connectivity solutions for IoT, SATEL, and others.

[0004] 3GPP Technical Report TR 38.836 V17.0.0 (2021-03), TSG RAN WG2 specifies the results of its 3GPP Release 17 "Study on NR Sidelink Relay", whereby the remote user equipment (UE) is out of coverage or in a different cell coverage, and the relay from the user equipment to the network (UE-NW) can assume the interworking function as an inter-cell relay. For Release 18, another scenario of UE-to-UE relay will become relevant, in which any vehicle with an on-board communication unit can operate as a relay user equipment. It is assumed that the NR sidelink is on the direct interface (PC5) between (multiple) remote UEs and the UE-to-UE relay. Additionally, there is an active 5GAA work item on NR-V2X (T-210026) which specifies “System enablers and best practices for next generation cooperative use cases” which include C-ITS use cases that require some form of grouping and cooperation, such as group initiation, coordinated cooperative driving maneuvers, roadside management, interactive VRU intersections, vehicle decision assistance, cooperative traffic gaps, etc.

[0005] In the scientific literature, there are many works related to Wireless Sensor Networks (WSN) and wireless short range communications as listed below, where most of the references focus on solutions for out-of-network coverage operations:

[0006] [1] “A Comparative Study of Wireless Protocols: Bluetooth, UWB, ZigBee, and Wi-Fi”, Karunakar Pothuganti and Anusha Chitneni, Advance in Electronic and Electric Engineering, ISSN 2231-1297, Vol. 4, No. 6 (2014), pp. 655-662.

[0007] [2] “Wireless sensor networks: A survey on the state of the art and the 802.15.4 and ZigBee standards”, Paolo Baronti, Prashant Pillai, Vince W.C. Chook, Stefano Chess, Alberto Gotta, Y. Fun Hua, Computer Communications, Vol. 30, No. 7, May 26, 2007, pp. 1655-1695.

[0008] [3] “Evaluation of wireless short-range communication performance in aquarry environment”, Susanne Vernersson, Eleni Kalpaxidou, and David Rylander, 2013 International Conference on Connected Vehicles and Expo (ICCVE), DOI: 10.1109 / ICCVE.2013.6799812.

[0009] [4] “Energy savings by wireless control of speed, scheduling and traveltimes for hauling operation”, David Rylander, Jakob Axelsson, and Peter Wallin, 2014 IEEE Intelligent Vehicles Symposium (IV), DOI: 10.1109 / IVS.2014.6856451.

[0010] [5] “Artificial Intelligence Supervised Swarm UAVs for Reconnaissance” by Saatvik Awasthi, Balamurugan Balamurugan, and V. Porkodi, Data Science and Analytics, May 2020, DOI: 10.1007 / 978-981-15-5827-6_33.

[0011] [6] “Flying ad-hoc networks (FANETs): A review of communication architectures, and routing protocols”, Muhammad Asghar Khan, Alamgir Safi, Ijaz Mansoor Qureshi, Inam Ullah Khan, 2017 First International Conference on Latest trends in Electrical Engineering and Computing Technologies (INTELLECT), DOI: 10.1109 / INTELLECT.2017.8277614.

[0012] [7] ENSEMBLE project presentation “Intelligent Maneuver Automation-cooperative hazard avoidance in realtime (IMAGinE)”, author Lehmann B., 9th ETSI IST Workshop, March 6-8, 2018, Berlin.

[0013] [8] ENSEMBLE project publication “Specifications for multi-brand truck platooning”, Edoardo Mascalchia, Alessandro Coda, Dehlia Willemsen, Proceedings of 8th Transport Research Arena TRA 2020, April 27–30, 2020, Helsinki, Finland.

[0014] A large number of patent documents propose connectivity solutions based on cluster intelligence and related aspects. For example, US2013123981A1 discloses a cluster intelligent routing robot device, wherein a plurality of cluster intelligent robot devices configure a cluster. The cluster intelligent routing robot device configures and manages a wireless communication network to relay communications between cluster intelligent robot devices moving in an atypical environment in the cluster and selects their locations. US2013128726A1 discloses a system and method for packet delivery backtracking, which uses dynamic mesh networks and routing redundancy in the mesh and other technologies to improve network reliability. US2014080527A1 describes an invention that provides a modular radio frequency communication component that includes a radio frequency hub and one or more radio frequency modules. US2017093697A1 discloses a method for controlling flooding broadcasts in a wireless mesh network. In order to balance reliability and efficiency, the invention defines a threshold value of the number of neighboring nodes seen by a given node before a flooding operation to determine whether the data should be unicast or broadcast. Below the threshold, unicast is used; at or above the threshold, broadcast is used. The invention also incorporates knowledge of the nodes that neighboring nodes, and therefore nodes, see as part of this decision. US2018097647A1 describes systems, devices, and methods for eliminating echo and reducing congestion in multicast (broadcast) on a dynamic self-healing mobile mesh network by using a discrete embedded computer that synchronously tracks mesh connectivity and link quality on multiple RF connections, thereby keeping multicast efficient and effective in a highly dynamic, ever-changing mesh topology. US2020322895A1 provides a system and method for controlling dynamic transmission power in a mesh network. A distributed power transmission management method is used on a peer-to-peer basis, which implements transmission power management based on a comparison of the signal-to-noise ratio from a received beacon packet. Embodiments are used to keep all nodes accessible, dynamically adapt to constant changes in the network, maximize frequency reuse, and reduce power requirements to maximize network performance while minimizing interference. US2016381596A1 relates to an intelligent multi-beam media access control in the KU band for a mission-oriented mobile mesh network. It describes a MAC design for KU-band mobile wireless mesh networks with multi-beam smart antennas. This MAC includes coverage control to separate collision domains. It also has a lower-layer CSMA-like scheme. The disclosed design includes enhanced PCF and enhanced DCF for the following two purposes: (1) to exploit multi-beam concurrent communication capabilities, and (2) to support QoS and mission-based communications. An efficient time synchronization scheme is also disclosed to ensure that all beams can send data to the star nodes at the same time. Finally, a prediction scheme based on ARMA or HMM is disclosed to predict the future traffic profile in each beam. This helps the star nodes better prepare queue contents and scheduling information.US2018167452A1 relates to a method and system for distributed spatial control of a vehicle formation, the method and system comprising receiving driving direction and formation density information at a first formation vehicle via a peer communication interface, the information indicating the driving route of the first vehicle when it is traveling as a member of a vehicle formation. The peer formation density information indicates the distance maintained from other adjacent formation vehicles. A formation vehicle self-navigation command is generated for navigating the first vehicle when the first vehicle is traveling in a one-dimensional, two-dimensional, or three-dimensional space as a member of a vehicle formation. The self-navigation command is based on the peer driving direction and formation density information. The driving direction information is based on a locally determined spatial relationship of a portion of the vehicle formation. Intel Corporation's (Intel) patent application US2019049931A1 discloses systems, devices, and methods for robot cluster coordination. The University of Southern California's patent US6636781 describes distributed control and coordination techniques for autonomous agents in dynamic, reconfigurable systems.

[0015] However, connectivity (or more precisely, mobile network coverage) is often the decisive factor in determining which connectivity use cases can be enabled. More precisely, limited or insufficient mobile network coverage in remote areas remains a key challenge in these industry sectors. Automated vehicles or robots may operate in workplaces that are potentially of enormous size and with large physical separations between swarm members, which requires the selection of appropriate radio technologies depending on the operating environment.

[0016] Therefore, in this context, the technical problem to be solved is to find solutions that enable efficient operation of autonomous clusters even in areas without network coverage and in harsh environments. Summary of the invention

[0017] The object of the invention is to provide a communication system for coordinating and managing the operation of a cluster of automated entities (vehicles and / or robots) according to independent claim 1, a corresponding communication method according to independent claim 19. Additional independent claims claim protection for a cluster master, a cluster slave, a computer-implemented program for directing the computing system of the cluster master to function in a specified manner to manage the cluster, and a computer-implemented program for directing the computing system of the cluster slave to function in a specified manner to be managed in the cluster.

[0018] According to a first aspect of the present invention, a system for coordinating and managing a cluster of automated entities is provided, wherein the cluster includes at least a designated master entity and a slave entity. Each automated entity includes a data processing mechanism, i.e., at least one processor coupled to at least one memory and adapted to execute instructions that facilitate the operation of a program module, and each automated entity has specific assets and capabilities necessary for collaborating with other entities and performing missions as a cluster. A cluster management module is instantiated at the master entity, the cluster management module includes a mission planning manager, the mission planning manager is adapted to receive or load missions and provide mission configurations, wherein the designated cluster master entity is configured to announce itself through a communication module and a communication link, and request at least one automated entity different from the master entity and external to the master entity to join the execution of the mission as a cluster slave entity.

[0019] Therefore, the concept of a "master entity" (or "cluster master entity") is introduced to operate locally as a management and coordination entity that can orchestrate automated cluster operations without the need for cloud backend system connectivity. Further, it is assumed that such a master entity has wireless communication capabilities for coordinating the cluster's mission execution.

[0020] In one embodiment, the mission planning manager of the master entity is configured to receive or load a mission plan defined according to a predefined catalog, further perform an asset inventory based on the received or loaded mission plan, which means further requesting or rejecting assets, and create a mission configuration derived from the mission plan based on the performed asset inventory.

[0021] The high performance computer is capable of generating a mission plan based on the received or loaded mission, deriving the mission configuration from the mission plan, further dividing the complex mission plan into a set of tasks to be performed, and generating a mission schedule and dependencies between the tasks and various cluster entities. The cluster is dynamically managed, which means that in order to complete the mission, the mission execution status is continuously analyzed based on the status reports collected from the cluster entities.

[0022] In some embodiments, a cluster fleet manager is instantiated at each slave entity that responds to a request from the master entity, provided that the slave entity matches a profile derived from the mission configuration, wherein the cluster fleet manager of each slave entity is configured to receive a role in the cluster, join the cluster and receive instructions related to cluster formation. A cluster operation module is further instantiated at the master entity, the cluster operation module being configured to receive or load instructions and key data sets to coordinate the execution of operations and tasks derived from the mission configuration, and to monitor the cluster's execution of the mission.

[0023] In a similar manner, the cluster operation module is further instantiated at each cluster slave entity that responds to the request sent by the master entity, and is configured to receive or load instructions and key data sets to coordinate when performing operations and tasks derived from the mission configuration, and report the execution of tasks by each cluster slave entity. Each cluster entity may include a human-machine interface through which manual control instructions are received from an operator. The master entity may be included in a backend system, and more specifically in a cloud backend system.

[0024] In some embodiments, the cluster fleet manager of the master entity is configured to generate a first key data set, which includes: cluster identity, slave entity identity, mission type, cluster entity type, role assignment, behavioral strategy encoded as a list of rules for each event, entity role and entity type, task schedule, cluster entity trajectory.

[0025] In some embodiments, the cluster formation manager of the master entity is further configured to generate a second key data set as input data for the cluster operation module of the master entity, and the second key data set includes: mission type, behavior strategy, task schedule and cluster entity trajectory. The cluster formation manager of each slave entity is configured to receive the first key data set and provide the second key data set, and the second key data set includes: mission type, behavior strategy, task schedule and cluster entity trajectory. Further, the cluster formation manager of each cluster slave entity is configured to provide the second key data set as input data for the cluster operation module of the corresponding cluster slave entity.

[0026] In some embodiments, each cluster entity further comprises a motion module configured to process the received task and trajectory data and provide motion status data as output.

[0027] In some embodiments, each cluster entity further includes a perception module configured to process data provided by the sensory mechanism and provide object data about the presence of entities in the cluster operating environment. The motion module of each cluster entity further receives object data related to other entities from the perception module, and object data related to other cluster entities received from the communication module.

[0028] In some embodiments, the cluster operation module of the master entity is configured to regularly collect mission status data. Each cluster entity further includes an event module, which is configured to record status and data, and further regularly performs health checks on its own cluster entity and reports its own status to the cluster operation module. Each cluster entity further includes a danger module, which is configured to record danger data and status related to unexpected events, accidents and problems, and reports such danger data to the event module.

[0029] In some embodiments, the communication module of the master entity sends and receives a third key data set to and from the communication module of each slave entity, and the third key data set includes: cluster identity, slave entity identity, mission type, entity type, role assignment, behavior strategy, status report, task schedule, and entity trajectory.

[0030] According to a second aspect of the present invention, a communication method for coordinating and managing a cluster of automation entities is provided. The method comprises: receiving an indication of a mission assigned to a cluster by a master entity; sending a master entity announcement message to at least one potential slave entity that can be reached directly or indirectly through at least one communication link, the master entity announcement message including a mission type; receiving the master entity announcement message by all reachable potential slave entities to check whether at least one potential slave entity has a profile that matches the mission type, the profile including assets and capabilities that match the declared mission type; and sending at least one cluster join message to the master entity in response by at least one slave entity among the slave entities that has a profile that matches the declared mission type directly or indirectly through a communication link.

[0031] In one embodiment of the method according to the invention, as part of the cluster container, all cluster-related messages have a standardized and extended message format.

[0032] In addition, the method includes analyzing, by a data processing mechanism of the master entity, messages received from all responding slave entities to determine a mission plan or mission configuration based on the assigned mission; and distributing, by a communication mechanism of the master entity, the mission plan or mission configuration to all slave entities. The mission plan and / or mission configuration is defined according to a predefined directory, and the mission configuration specifies a set of entity types to be assigned to the cluster.

[0033] In addition, the method includes sending a set of cluster management messages by the master entity, wherein the cluster management messages include a first key data set, which includes an identifier of the cluster, a cluster entity type, a mission type, a role, a behavioral strategy encoded as a list of rules for each event and role, an entity trajectory, and a set of task schedules to be performed during execution of the mission plan.

[0034] In addition, the method includes sending by the slave entity regularly at predefined time intervals or in the event of an unexpected event and receiving by the master entity regularly at predefined time intervals or in the event of an unexpected event a set of status report messages, wherein the status report messages include an identifier of the cluster, a cluster entity type, a mission type, a role, a task, a task start time and an estimated task end time, a task status, an estimated time to complete the task, a communication status, and a fuel or energy status.

[0035] Additionally, the method includes each cluster slave entity propagating a cluster management message to thereby spread the first critical data among reachable cluster slave entities, and optionally spread the first critical data together with the second critical data to further reachable cluster entities via various communication protocols.

[0036] Each reachable entity that joins a cluster reached by another cluster slave entity considers the master entity of the reaching cluster slave entity as its own master entity.

[0037] Optionally, the method includes assigning a secondary master entity role to at least one cluster entity having a profile matching the primary entity profile and being qualified as a secondary master entity. The method further includes monitoring primary entity message delivery by the secondary master entity. The primary entity role can be transferred to the secondary master entity when the current cluster master entity is unable to act according to the primary entity role requirement, wherein the primary entity role transfer is controlled by a behavior policy.

[0038] Optionally, the method includes announcing a change or transfer of the master entity role by issuing a new master entity announce message, which notifies the cluster that a new cluster master entity is replacing the current master entity.

[0039] Further, the method includes synchronizing operations in time across the master entity and the slave entity, the synchronization including generating and maintaining an event queue that records conditions and data and performs routine health checks, the recorded conditions and data including hazardous data such as unexpected events, incidents, and problems.

[0040] In a preferred embodiment of the method according to the invention, the cluster communication is hybrid, employing client-server and / or peer-to-peer communication schemes.

[0041] According to a third aspect of the present invention, a cluster master device is provided, the cluster master device comprising an autonomous driving module, a motion module and a perception module. The cluster master device comprises: at least one processor coupled to at least one memory and adapted to execute a routine, the routine comprising instructions for managing the cluster and executing a mission assigned to the cluster; and a hardware mechanism necessary for executing the mission, the mechanism comprising a cluster management middleware. A mechanism for communicating with other cluster entities and entities outside the cluster is also provided, the mechanism comprising a communication management middleware.

[0042] In addition, a cluster slave device is provided, which includes an automatic driving module, a motion module and a perception module. Further, the cluster slave device includes: at least one processor coupled to at least one memory and adapted to execute a routine, the routine including instructions for joining a cluster at the request of a cluster master entity and executing a task scheduled based on a mission configuration; and a hardware mechanism necessary for executing the task; and a mechanism for communicating with the cluster master entity, other cluster slave entities and entities outside the cluster, such mechanism including a communication management middleware.

[0043] As such, the onboard / onboard communication modules of each cluster entity are ultimately controlled by the cluster management middleware, which selects the appropriate radio technology or communication channel for each data type or message based on the data communication requirements and the operating environment of the cluster entity, taking into account parameters such as location, vehicle speed, relative speed or orientation with respect to other cluster entities, and round-trip time estimates of radio signals for ranging. In addition, the communication costs and related trade-offs are also taken into account, especially when cellular or satellite communications are involved. Here, the energy and fuel consumption of individual cluster entities are also important parameters for the overall mission efficiency.

[0044] In a fourth aspect of the present invention, a computer-implemented program is provided, the computer-implemented program including at least one program module, the at least one program module directing the computing system of the cluster master entity to function in a specified manner to manage the cluster and perform the mission assigned to the cluster. The program module may include instructions for the following operations: receiving an indication of a mission assigned to the cluster; sending a master entity announcement message to at least one potential slave entity that can be reached directly or indirectly through at least one communication link, the master entity announcement message including a mission type; receiving a cluster join message from at least one responding slave entity having a profile matching the announced mission type directly or indirectly through a communication link; analyzing the messages received from all responding slave entities to determine a mission plan or mission configuration based on the assigned mission; assigning the mission plan or mission configuration encoded in the cluster management message to the responding slave entity; and periodically collecting reports from the slave entity on the status of mission execution.

[0045] In addition, a computer-implemented program is provided, which includes at least one program module, which guides the computing system of the cluster slave entity to function in a specified manner to join the cluster and perform tasks scheduled according to the mission assigned to the cluster, and the program module includes instructions for the following operations: receiving a master entity declaration message including a mission type; checking whether the slave entity profile matches the mission type, the slave entity profile including computing resources and capabilities matching the declared mission type; when the slave entity profile matches the declared mission type, sending at least one cluster join message to the master entity in response; joining the execution of the task; and periodically sending reports to the master entity on the execution status of the scheduled tasks.

[0046] The present invention is advantageous because it enables automated cluster coordination and management, including the selection of the most appropriate communication channel (among many supported connectivity technologies) for wireless communication with vehicles / devices located outside the mobile network coverage area through the use of relays and multi-hop communications. In addition, cluster management establishes a certain hierarchy within the cluster structure, including fallback mechanisms.

[0047] Further particular features and advantages of the invention can be gathered from the following description of advantageous embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 An exemplary cluster with one master vehicle and two slave vehicles is shown.

[0049] Figure 2 An exemplary embodiment of a swarm formation for an agricultural scenario with one master vehicle and multiple slave vehicles is presented.

[0050] Figure 3 shows an exemplary mission schedule for a cluster dedicated to the mission,

[0051] Figure 4 presents a block diagram of a cluster system according to the present invention,

[0052] Figure 5 A block diagram of one embodiment of a host entity computing system is shown,

[0053] Figure 6 shows a block diagram of one embodiment of a slave system,

[0054] Figure 7 A block diagram of an embodiment of a cluster architecture including a backend device, a master device and a slave device is shown,

[0055] Figure 8 presents a sequence diagram of a master entity announcement message using the extended CAM format,

[0056] Fig. 9 A sequence diagram showing the communication between the operator and the master entity regarding mission planning.

[0057] Fig.10 A sequence diagram showing the communication between the operator, the backend and the master entity regarding mission planning is presented.

[0058] Fig.11 A sequence diagram showing the communication between the master and slave entities during cluster formation,

[0059] Fig.12 presents a sequence diagram of the communication between the master entity and the slave entities during cluster operation,

[0060] Fig.13 A comprehensive sequence diagram showing the communication between the backend, master and slave entities regarding cluster operations,

[0061] Fig.14 A sequence diagram showing a master entity announcement message using the extended CoAP format. DETAILED DESCRIPTION

[0062] In order to better understand the principles of the present invention, embodiments of the present invention will be explained in more detail below with reference to the accompanying drawings. In the accompanying drawings, similar reference numerals are used for identical or equivalent elements, and it is not necessary to describe them again for each of the accompanying drawings. It should be understood that the present invention is not limited to the embodiments shown, and the described features may also be combined or modified without departing from the scope of the present invention defined in the appended claims.

[0063] As referred to herein, a "cluster of automated entities" or "cluster" or "automated cluster" means two or more automated entities, such as autonomous vehicles, robots, or unmanned aerial vehicles, whose movements are automatically coordinated with each other by a remote coordinator / coordinator (operator or automated entity). In this article, a "coordinating entity" is referred to as a "master entity" (or "cluster master entity"). The general term "slave entity" or "cluster slave entity" referred to herein includes an automated entity (i.e., a vehicle or robot) that is directly controlled by a master entity. Both the master entity and the slave entity as members of a cluster are referred to as "cluster entities."

[0064] Various embodiments described herein generally relate to communication techniques for managing and coordinating a cluster, and more particularly, monitoring the execution of tasks derived from a mission assigned to the cluster, which tasks are distributed among multiple cluster entities.

[0065] However, as part of achieving dynamic cluster management and coordination, communication between cluster entities must remain unimpeded until the assigned mission is completed, no matter how much the entities and hierarchies in the cluster change.

[0066] In order to solve this problem, one or more program modules that perform various cluster coordination and / or management functions can be instantiated in one or more cluster entities according to the role (such as master entity or slave entity) played in the cluster. Such cluster-related program modules can process, generate and maintain key data sets as input and output between cluster entities. Communication can use various protocols that enable information about the execution status of various tasks in the cluster entity to be saved. Therefore, when a certain event causes the execution of a task to stop, another instance of the same task can be restarted to ensure that the task that should be executed is ultimately executed by the same entity or by a replacement entity.

[0067] A first aspect of the invention relates to a system for coordinating and managing a cluster of automation entities.

[0068] Reference now Figure 1 , which shows an example cluster using one master entity and two slave entities as cluster entities. In this embodiment, the cluster entities involved are heavy machines used for mining (e.g., excavators, loaders, and dump trucks).

[0069] Figure 2 An exemplary embodiment of a cluster formation of an agricultural scenario with one master entity and multiple slave entities as cluster entities is presented. More specifically, a scenario using multiple groups of combine harvesters and loaders is presented.

[0070] In the context of the present invention, a mission plan is a set of steps, resources (e.g., assets), dependencies, and conditions organized in sequence that can be scheduled and executed by various cluster entities to complete the mission. A mission configuration is an arrangement of cluster entities that are assigned to corresponding roles and tasks and interconnected according to functional and operational requirements and constraints to enable the cluster to operate as efficiently as possible. A task is a routine, action, or function assigned to be performed by a cluster entity to complete the assigned mission.

[0071] During mission planning, taking into account the profiles of the cluster entities, it is possible to assign a subset of suitable entities to a task type. Multiple entities with the same profile may be determined to perform the same task together.

[0072] Since each operational cluster scenario may require different types of entities with certain capabilities and roles, a catalog of mission types should be defined. This catalog can be customized for specific customers. Table 1 depicts an exemplary catalog of mission types.

[0073] Table 1: Example missions Type Catalog

[0074] Mission Type #1 Mission Type #2 Mission Type #3 Vehicle Type #1 Vehicle Type #3 Vehicle Type #5 Vehicle Type #2 Vehicle Type #4 Vehicle Type #6 Vehicle Type #7

[0075] In addition, a specific schedule for possible parallel execution of tasks will be determined during mission planning. The master entity may choose to broadcast a task type identifier associated with a predefined mission type catalog. Table 2 shows an exemplary predefined task type catalog. In this particular case, two specific types of missions are considered: "construction" with operations (tasks) such as drilling, digging, loading, dumping, paving, etc., and "agriculture" with specific operations such as plowing, sowing, harvesting, dumping, fertilizing, etc. The mission type and task type can be encoded using hexadecimal identifiers, respectively.

[0076] Further, the task types are associated with predefined mappings of suitable entity profile identifiers. These implicit definitions should help reduce the amount of data to be exchanged between cluster entities. Table 3 presents such exemplary data. For example, for a mining mission, and for the first task (drilling), multiple slave entities are assigned. The same is true for the remaining tasks (digging, loading). For transport and dumping, only one slave vehicle is assigned.

[0077] Table 2: Example task type list

[0078] Mission Type #1 (Construction) Mission Type #2 (Agriculture) Task Type #1 (Drilling) Task Type #1 (Plowing) Mission Type #2 (Excavation) Mission Type #2 (Sowing) Mission Type #3 (Loading) Mission Type #3 (Harvest) Mission Type #4 (Dumping) Mission Type #4 (Dumping) Mission Type #5 (Paving the Road) Task Type #5 (Fertilization)

[0079] Each cluster entity should know these predefined relationships in advance. Therefore, each cluster entity receiving a certain mission type identifier can independently check whether its capabilities encoded as a profile (i.e., entity profile identifier) ​​matches the profile of the requested mission type and respond with its entity profile identifier and entity type identifier.

[0080] Table 3: Predefined relationships between mission type, task type and vehicle profile identifier

[0081]

[0082]

[0083] Initially, all slave entities with entity profile identifiers that match the requested mission type respond to the master entity and provide their entity identifiers. The master entity receives each response and determines how many entities with a certain profile identifier are required for the various tasks specified in the mission configuration. However, for a specific task, cluster formation may mean that only a subset of those cluster entities with matching entity profile identifiers can receive requests to join a cluster or a group dedicated to a specific task within a cluster. For example, if there are more slave entities with a certain entity profile identifier than required for the task, the master entity can only request those slave entities selected for the specific task. However, this means that the master entity can reach all slave entities (directly or indirectly). Further, the master entity should receive replies from all qualified slave entities in order to identify suitable cluster entities for the specific task. A slave entity that has not received a response from the master entity within a predefined time interval can perform another predefined task (as specified in the behavior policy). Optionally, the master entity may respond to slave entities that were not selected for a specified task or mission and may notify these slave entities of their "rejection", upon which these slave entities perform predefined tasks (as specified in the behavior policy).

[0084] The behavior policy may implement a set of rules in which a list of rules for each event and entity role is encoded. The behavior policy may provide, for example, rules on how to proceed when an emergency brake occurs from an entity, or how to trigger a transfer of a role from one cluster entity to another, or may implement one or more limits on time and / or resource consumption and / or other parameters regarding the execution of a task or communication. The behavior policy may implement a set of rules in which the execution of a task that exceeds one or more of such parameters results in a failure of the task execution, making it deemed necessary to trigger a stop to the associated mission plan.

[0085] Exemplary predefined behavior policies are described below:

[0086] Sending a "rendezvous" position on startup when a cluster entity cannot connect to any other cluster entity,

[0087] When the mission is complete, send the "rendezvous" position,

[0088] In hazardous situations: record and report event data, check vehicle health, continue operation.

[0089] Several incidents were considered hazardous and were recorded and reported as follows:

[0090] - running out of fuel or energy,

[0091] - not communicating with any cluster entity since X time units,

[0092] - Inoperable due to broken parts / repair required,

[0093] - No request to join the cluster was received.

[0094] Figure 3 An exemplary mission schedule for a cluster dedicated to a mission is shown, and more specifically, a cluster having a mission of mining from a starting location that coincides with a first mission (e.g., digging) start location. This exemplary mission may be communicated by a master entity to a slave entity. For example, each mission schedule may provide data as a start time, a start location, an estimated end time, and an end location. Optionally, an entity track consisting of different segments may be provided:

[0095] Mission #1:

[0096] - start time, start position,

[0097] - (estimated) end time, end location,

[0098] - physical track (composed of different segments),

[0099] Mission #2:

[0100] - start time, start position,

[0101] - (estimated) end time, end location,

[0102] - entity tracks,

[0103] Mission #3:

[0104] - start time, start position,

[0105] - (estimated) end time, end location,

[0106] -Entity tracks.

[0107] Figure 4 A block diagram of an exemplary embodiment of a cluster system according to the present invention is presented.

[0108] The cluster system 1 includes multiple instances of master entities and slave entities, which are interoperable as a dynamically coordinated computing system and are shown as a representative cluster master entity 11 (or "master entity") and a representative cluster slave entity 12 (or "slave entity"). More specifically, the master entity 11 is a computing system configured to instantiate the cluster management module 1001, cluster operation module 200, motion module 300, perception module 400, communication module 500 and positioning module 600 (e.g., a computing unit capable of processing and providing location data) of the master entity. The slave entity 12 is a computing system configured to instantiate the cluster management module 1002 of the slave entity and the corresponding cluster operation module 200, motion module 300, perception module 400, communication module 500 and positioning module 600. The perception module 400, i.e., a computing unit capable of processing object data provided by a sense / sensing mechanism (e.g., radar, camera, ultrasonic sensor, etc.), can perceive the presence of other entities outside the cluster (e.g., vehicles or road users 700).

[0109] The cluster master entity management module 1001 and the cluster slave entity management module 1002 each include a cluster formation manager 120, and the cluster master entity management module 1001 additionally has a mission planning manager 110. An operator can assume the role of the master entity and thus manually control the cluster via an interface module 900 (e.g., a human-machine interface HMI). If a connection to such a backend system 800 can be established, the master entity role can be supported by the backend system 800 or even assumed by the backend system. More specifically, the backend system 800 can be a cloud backend system.

[0110] The cluster master entity 11 of any appointment is configured to declare itself so through the communication module 500 and the communication link, and requests to be different from this master entity and at least one automation entity (that is, from entity) outside this master entity as the execution of the cluster slave entity 12 joining the mission. The mission planning manager 110 at the cluster master entity 11 place is configured to: once the mission planning module 110 (from the operator or from the back-end system 800) receives or loads the mission or mission plan, the mission configuration is generated and provided to the cluster formation module 120 of the cluster master system 11. In one embodiment, the mission configuration can also be enhanced with the manual control instructions to be executed during the mission execution. For example, the interface module 900 can provide the operator with the opportunity to select one or more manual control commands, such as leaving, rejoining or resetting / reconfiguring in response to the specific status report received from the cluster.

[0111] Further, the cluster formation manager 120 of the master entity is configured to provide a first key data set, which is derived from the mission configuration, to the communication module 500 of the master entity 11. As will be explained in more detail, examples of the first key data set include cluster identity, slave entity identity, entity type identifier (e.g., what type of vehicle is needed to perform the mission), mission type, role assignment, mission schedule (including mission start time and estimated mission end time, estimated time to complete the mission), mission status, communication status, fuel or energy status, behavior strategy and entity trajectory.

[0112] The cluster fleet manager 120 of the master entity 11 is configured to provide a second key data set to the cluster operation module 200 of the cluster master system 11. The second key data set is derived from the mission plan and is transmitted to each cluster entity. Examples of the second key data set include at least mission type, behavior strategy, task schedule and entity trajectory.

[0113] The cluster operation module 200 of the main entity 11 includes two managers: an operation and task execution manager 210 and a status collection and monitoring manager 2201 .

[0114] Given the aforementioned second key data set, the operation and mission execution component 210 of the host entity 11 configures the mission and entity trajectory data and provides it to the motion module 300 of the host entity 11. The host entity motion module 300 further processes the motion status and reports it back to the operation and mission execution manager 210, which is configured to further report the status to the status collection and monitoring manager 2201 (or "host entity status manager" 2201) of the host entity 11. Other data and information are periodically collected by the host entity status manager 2201 from the perception module 400 (i.e., object data about the presence of other vehicles and / or road users 700 other than the cluster entity) and the communication module 500 (about other conditions), and the host entity status manager 2201 sends periodic reports about its own status to the communication module 500. Once the mission is completed, the same host entity status manager 2201 sends this information to the mission planning manager 110 of the host entity 11.

[0115] The motion module 300 also provides entity motion data (e.g., vehicle dynamics data such as vehicle speed, acceleration, and orientation when the entity of interest is a vehicle) to the communication module 500, and receives external or V2X (i.e., received by a vehicle-to-everything communication mechanism) object data from the communication module 500 in addition to the directly sensed object data from the perception module 400 discussed above. At its conclusion, the perception module 400 provides the directly sensed object data to the communication module 500, the host entity status manager 2201, and the motion module 300. Another item of data sent to the communication module 500 is the location of the host entity, and is provided by the positioning module 600.

[0116] The cluster formation manager 120 at each slave entity 12 is configured to receive the aforementioned first key data set and the second key data set from the communication module 500 of the master entity 11. Further, the cluster formation manager 120 of each slave entity 12 is configured to process the received first key data set and provide a second key data set intended to be transmitted to the cluster entity, the second key data set at least including the mission type, the behavior strategy, the task schedule and the entity trajectory. The second data set is sent to the cluster operation module 200 of each slave entity 12.

[0117] The cluster operation module 200 of the slave entity 12 includes two managers: an operation and task execution manager 210 and a status collection and reporting manager 2202 (or "slave entity status manager" 2202).

[0118] Given the aforementioned second data set (mission type, behavior strategy, mission schedule, and entity trajectory), the operation and task execution manager 210 of each slave entity 12 configures the mission and entity trajectory data and provides it to the motion module 300 of the slave entity 12. The motion module 300 further processes the motion status and reports it back to the operation and task execution manager 210 of each slave entity 12, which is configured to further report the status to the slave entity status manager 2202 of each slave entity 12. Other reports are collected by each status manager 2202 of each slave entity 12 from the perception module 400 (including object data of other vehicles and / or road users 700 other than the cluster entity) and its own slave entity communication module 500 (reports on other conditions), and the status manager 2202 of each slave entity 12 sends a report on its own status to the communication module 500 of each slave entity 12.

[0119] The motion module 300 of each slave entity 12 also provides dynamic data to the slave entity's own communication module 500, and receives external or V2X object data from its own communication module 500 in addition to the directly sensed object data from the perception module 400 of the slave entity. At the end of it, the perception module 400 of the slave entity 12 provides the directly sensed object data to the communication module 500 of the corresponding slave entity, the motion module 300 of the slave entity, and the status manager 2202. Another data sent to each slave entity communication module 500 is the position of the own slave entity, and is provided by the slave entity positioning module 600. Finally, each communication module 500 of the slave entity 12 sends back a third key data set to the communication module 500 of the master entity 11, which includes updated information of cluster identity, slave entity identity, mission type, role assignment, behavior strategy, status report, mission schedule, and entity trajectory.

[0120] Figure 5 A block diagram of one embodiment of a host system according to the present invention is shown in order to illustrate specific functions provided by specific modules or managers of the host entity.

[0121] The master system includes specific modules and managers that are assigned to enable specific capabilities and functions associated with the role of the cluster master entity, namely:

[0122] - Mission planning, including performing an asset inventory and creating a mission configuration,

[0123] - Cluster formation via initial cluster announcement; assigning or changing roles (e.g. including fallback for hijacked / manipulated master entities); adding or removing cluster entities,

[0124] - Check cluster health,

[0125] - assigning or sending instructions or commands to cluster entities (cluster operations),

[0126] - Record all activities (messages, information flows),

[0127] - Send status reports to the master entity or backend (including "mission accomplished" status reports).

[0128] As in Figure 5As more specifically depicted in , once the master entity is designated, its operation is as follows: the regular startup module starts the wake-up routine, including health checks and reports; once the wake-up routine is completed, the mission plan is loaded or received at the mission planning manager (as discussed earlier). At this stage, it is also possible to directly intervene and reconfigure the initial mission via the manual control interface using instructions and commands such as "leave", "rejoin", "reset" or "reconfigure". The next step is an asset inventory, through which "assets" are understood any data, hardware or software means / mechanisms (e.g., program modules) that support cluster-related activities. After this inventory, additional assets can be requested or some of them can be rejected. This step is revisited at any time when the cluster is changed or reconfigured due to various reasons (one or more cluster entities leave, some modules crash, system failure, etc.). Once the asset inventory is completed, the mission planning manager is able to generate the mission configuration. Further, the cluster formation module receives the mission configuration and proceeds to the initial cluster formation through role assignment: for example, not only the role of the slave entity or the master entity is assigned to the cluster entity of the corresponding type, but also the additional role of the deputy master entity, wherein the deputy master entity role refers to a cluster entity configured to monitor the specified master entity message so as to be able to detect whether the master entity stops normal communication due to various reasons. In the case where the cluster has been formed, the cluster can be changed (some entities are to be added, revoked or suspended). Once the cluster has been formed, (multiple) task schedules are provided to each cluster entity. Further, the cluster operation manager of the master entity starts the task execution. As long as the mission is close to completion, the relevant mission status and data are recorded, and the mission status report is collected by the cluster collection and reporting manager of the master entity. If an unexpected event occurs, dangerous conditions and data are also recorded, health checks are triggered, and the mission status report is updated. Sometimes, this danger may hinder the termination of the mission, which means that the mission plan should adapt to this situation.

[0129] Figure 6 A block diagram of one embodiment of a slave system according to the present invention is shown.

[0130] The slave system includes specific modules and managers that are assigned to enable specific capabilities and functions associated with the role of a cluster slave entity, namely:

[0131] -wake,

[0132] - Perform your own asset inventories and / or health checks and report on them,

[0133] - Change role (serve as primary or secondary entity),

[0134] - join the cluster, respond to commands,

[0135] - log all activity (messages, information flows), and

[0136] - Send status reports to the master entity (including "mission accomplished" status reports).

[0137] The slave system also includes a regular startup module, which starts a wake-up routine, including health checks and reports; once the wake-up routine is completed, the role within the cluster will be received by the cluster fleet manager of the slave entity, or directly from the operator via the manual control interface. The next step is to initiate the execution of (multiple) tasks by the cluster operation module of the slave entity. Recording of execution status and data, and reporting of status regularly. Whenever a task is close to completion, the relevant task status and data are recorded, and the task status report is collected and further reported to the cluster master entity. Each slave entity also records dangerous conditions and data, triggers health checks and updates task status reports. Sometimes, such dangers may hinder the termination of a task, which means that the task schedule should be readjusted according to actual conditions.

[0138] Figure 7 A block diagram of an embodiment of a cluster architecture including backend equipment, a master device, and slave devices is shown.

[0139] According to one embodiment of the present invention, the cluster master may include an autonomous driving module (not shown), a motion module and a perception module. The perception module may include an electronic unit capable of collecting and processing sensor data and providing object data about the existence of entities in the operating environment of the cluster. The motion control module may include an electronic unit capable of controlling the movement of entities in terms of speed, acceleration, deceleration, relative speed and orientation. As shown, the cluster master may be provided with a hardware security module (including an electronic unit capable of controlling the network security of the entity system) and a positioning module (i.e., a GPS unit). Most importantly, at least one processor is provided, which is coupled to at least one memory and is adapted to execute routines, which routines include instructions for managing the cluster and executing missions assigned to the cluster. Also provided are: hardware mechanisms necessary for executing the mission, such mechanisms including cluster management middleware; and mechanisms for communicating with other cluster entities and entities outside the cluster, such mechanisms including communication management middleware.

[0140] A cluster slave device is provided, the cluster slave device includes at least one processor coupled to at least one memory and adapted to execute a routine, the routine including instructions for joining a cluster at the request of a cluster master entity and executing tasks scheduled based on a mission configuration. In addition, there are provided: a hardware mechanism necessary for executing the task; and a mechanism for communicating with the cluster master entity, other cluster slave entities, and entities outside the cluster, such a mechanism including a communication management middleware. Similarly, the cluster slave device may include an autonomous driving module, a motion module, and a perception module.

[0141] In some embodiments, the backend equipment includes a hardware module of the cluster master entity, plus an interface that provides operators with enhanced opportunities to manage the cluster through direct intervention.

[0142] The communication method for coordinating and managing a cluster of automation entities according to the second aspect of the present invention is described in more detail in the following description. More specifically, the method of the present invention includes receiving an indication of a mission assigned to the cluster by an automation entity designated as a cluster master entity. The master entity sends a master entity announcement message to at least one potential (qualified) slave entity that can be reached directly or indirectly through at least one communication link. The master entity announcement message includes at least a mission type (e.g., "construction" or "mining" encoded as a hexadecimal identifier). Once the master entity announcement message is received by all reachable potential slave entities, each slave entity checks whether it has a profile that matches the mission type, and the profile includes assets and capabilities that match the declared mission type. If so, at least one slave entity with a profile that matches the declared mission type sends a cluster join message to the master entity directly or indirectly through a communication link in response.

[0143] The method further includes: analyzing, by a cluster management module of the master entity, messages received from all responding slave entities to determine a mission plan or mission configuration based on the assigned mission; and distributing, by a communication mechanism of the master entity, the mission plan or mission configuration encoded in the cluster management message to all slave entities. The mission plan and / or mission configuration is defined according to a predefined directory; the mission configuration specifies a set of entity types and a set of task schedules to be assigned to the cluster.

[0144] The method further includes sending a cluster management message by the master entity, wherein the cluster management message includes key data, the key data including cluster identity, cluster entity type, mission type, entity role, behavioral strategy encoded as a list of rules for each event and role, entity trajectory, and a set of task schedules to be performed during the execution of the mission plan. A set of cluster control messages are sent in response by the slave entity at predefined time intervals regularly or in the event of an unexpected event. The cluster control message includes a status report of key data, the key data including an identifier of the cluster, cluster entity type, mission type, entity role, task, task start time and estimated task end time, task status, estimated time to complete the task, communication status, and fuel or energy status.

[0145] Each cluster slave entity propagates cluster management messages, thereby spreading key data among reachable cluster slave entities. Optionally, the first and second key data are spread to further reachable cluster entities via various communication protocols.

[0146] In some embodiments, each reachable entity that joins a cluster reached by another cluster slave entity considers the master entity of the reaching cluster slave entity to be its own master entity.

[0147] Optionally, assignment of a sub-master entity role to at least one cluster entity having a profile that matches the master entity profile and that qualifies as a sub-master entity is provided. The sub-master entity monitors master entity messaging. If the designated cluster master entity is no longer able to act in accordance with the master entity role requirements, the master entity role is transferred to the sub-master entity. This transfer is controlled by a behavioral policy. An option is provided to announce a change or transfer of the master entity role by issuing a new master entity announce message that notifies the cluster that a new cluster master entity is replacing the designated master entity.

[0148] In addition, the method provides time synchronization operations across master and slave entities. Synchronization means generating and maintaining an event queue that records conditions and data and performs regular health checks, including dangerous data related to unexpected events, problems or incidents.

[0149] Throughout the cluster, communication is hybrid, employing client-server and / or peer-to-peer communication schemes.

[0150] In some embodiments, as part of the cluster container, all cluster-related messages have a standardized and extended message format. Multiple message sequences are introduced into a message structure referred to in this description as a cluster management message (SMM), which can be managed by a message agent that can implement various standardized protocols (e.g., restricted application protocol CoAP, message queue telemetry transport MQTT, advanced message queue protocol AMQP, etc.). Cluster management messages are intended to be received by all cluster entities. This can allow multiple qualified cluster members to "listen" to the same message and only give them the freedom to respond to the message if they have a matching profile (e.g., possess the requested assets and capabilities). This approach can provide a degree of resilience in situations where a cluster entity is rendered inoperative (e.g., unable to execute). In some embodiments, only the relevant cluster entity responds, with the effect that cluster communication is more efficient, but an overall cluster inventory cannot be performed.

[0151] An exemplary embodiment of a cluster management message is a master entity announcement message, which will be presented in more detail in the following description.

[0152] Figure 8 A sequence diagram of a master entity announcement message using an exemplary extended Collaboration Awareness Message (CAM) format is presented.

[0153] At the beginning of a mission, cluster vehicles may be fairly co-located or very close (e.g., hundreds of meters). In this case, cluster announcement by the master entity starts using short-range communication, for example, communicating at 5.9 GHz. The cluster entities in close proximity will remain aware of each other by broadcasting their presence periodically, for example, using Collaborative Awareness Messages (CAM, defined by ETSI standard EN 302 637-2: "Collaborative Awareness Basic Service Specification") or Basic Safety Messages (BSM, defined by the Society of Automotive Engineers (SAE) J2735 standard). By sending master entity announcement messages, the master entity announces its presence and triggers cluster management related actions, such as cluster formation. A new cluster container is introduced in CAM to support cluster operations. This cluster container should always be included in the CAM.

[0154] The transmission of the master entity announcement message can be implemented using broadcast (including multi-hop) communication (e.g., DSRC, LTE-V2X, NR-V2X) or geo-networking. The remaining hop limit (RHL) in the geo-networking header should be set equal to the number of cluster entities. In the worst case, all cluster entities may be linearly separated and multi-hop communication of the message is required.

[0155] The master entity will receive messages, e.g., "Join Cluster Request", and can use ranging or arrival time difference measurements to estimate its approximate distance. Such messages can be propagated by other cluster entities in the case where the transmitting slave entity and the master entity are not in direct proximity to the currently used radio technology. The master entity can keep using this radio technology as long as the messages forwarded in a multi-hop manner still work. In the event that an expected transmission from a slave entity is not received within a predefined time window, the master entity can switch to a long-range radio communication channel (e.g., at UHF), or instruct the last slave entity that made the last contact / communication with the slave entity that failed to respond to the master entity to use the long-range radio communication channel (e.g., at UHF), and retransmit the master entity's request.

[0156] While standardized or extended structures are used for primary entity announcements, additional messages are newly defined for cluster management purposes. The following is a list of key data elements that are important for cluster management:

[0157] - cluster identifier,

[0158] - vehicle type, encoded as a vehicle identifier,

[0159] - Mission type, encoded as a mission identifier,

[0160] - behavioral policies, encoded as a list of rules for each event, entity role, and entity type,

[0161] - role assignment, encoded as a role identifier (where role update is possible, included in the message titled "Join Cluster Response"),

[0162] - Task schedule,

[0163] -Entity tracks.

[0164] An exemplary master entity declaration sequence diagram is specified below:

[0165] CamParameters::=SEQUENCE{

[0166] basicContainer BasicContainer,

[0167] highFrequencyContainer HighFrequencyContainer,

[0168] lowFrequencyContainer LowFrequencyContainer OPTIONAL,

[0169] specialVehicleContainer SpecialVehicleContainer OPTIONAL,

[0170] …

[0171] swarmContainer SwarmContainer OPTIONAL,

[0172] …

[0173] }

[0174] SwarmContainer::=SEQUENCE{

[0175] swarmIsJoinable BOOLEAN,

[0176] swarmMissionType INTEGER (takes a list of predefined mission type identifiers or e.g. an enumeration {Agriculture(0), Cargo Handling(1), Construction(2), Material Supply(3), Mining(4), Orchard(5), Paving(6)} or a fine-grained enumeration {Agriculture Cultivation(0), Agriculture Seeding(1), Agriculture Fertilization(2), Agriculture Harvesting(3)}),

[0177] swarmRoleID ENUMERATED{master entity (0), slave entity (1),

[0178] Sub-primary entity (2), not applicable (3)},

[0179] swarmID INTEGER (identifier of the cluster to be swarmed),

[0180] Alternative solution (optional):

[0181] swarmVehicleProfileIDListLength INTEGER,

[0182] swarmVehicleProfileIDList SwarmVehicleProfileIDList

[0183] OPTIONAL (list of desired vehicle profile identifier INTEGER)

[0184] }

[0185] The slave entity with a matching vehicle profile identifier responds to the master entity with a "JoinSwarmRequest":

[0186] JoinSwarmRequest::=SEQUENCE{

[0187] receiver StationID,

[0188] swarmMissionType INTEGER,

[0189] swarmRoleID ENUMERATED{master entity (0), slave entity (1),

[0190] Both (2), Not applicable (3)}

[0191] (Indicates from the entity which roles it can play)

[0192] swarmID INTEGER,

[0193] swarmVehicleProfileID INTEGER (from the entity indicating its capabilities)

[0194] }

[0195] JoinSwarmResponse::=SEQUENCE{

[0196] respondingTo StationID,

[0197] joinSwarmResponseStatus CHOICE{

[0198] notAllowedToJoin NULL,

[0199] allowedToJoin JoinSwarmResponseInfo

[0200] }

[0201] }

[0202] JoinSwarmResponseInfo::=SEQUENCE{

[0203] key Key,

[0204] frequencyChannel FrequencyChannel,

[0205] swarmID INTEGER,

[0206] swarmMissionType INTEGER,

[0207] swarmRoleID ENUMERATED{master(0), slave(1), co-master(2), not applicable(3)} (master indicates the role of slave)

[0208] }

[0209] SMM:=SEQUENCE{

[0210] header ItsPduHeader,

[0211] stationType StationType,

[0212] referencePosition ReferencePosition,

[0213] heading Heading,

[0214] generationDeltaTime GenerationDeltaTime,

[0215] message CHOICE{

[0216] joinSwarmRequest JoinSwarmRequest,

[0217] joinSwarmResponse JoinSwarmResponse,

[0218] leaveSwarmRequest LeaveSwarmRequest

[0219] In addition, cluster control messages (SCM) are transmitted between cluster entities. Each slave entity transmits its status report to the master entity, for example, regularly at predefined time intervals or in the event of an unexpected event (e.g., emergency braking). These status reports can be considered the "heartbeat" of the cluster system. The master entity should receive all status reports from each slave entity. The slave entity broadcasts a status report including a task identifier. This task identifier is used within the group to understand the progress of the work of each group entity. If the slave entity cannot reach the master entity directly, the status report transmitted by the slave entity may need to be forwarded by other cluster entities. Therefore, the status report is propagated through the cluster network in a multi-hop manner.

[0220] An exemplary status report may include the following key data:

[0221] - cluster identifier,

[0222] - entity identifier,

[0223] - Mission type, encoded as a mission identifier,

[0224] - role identifier,

[0225] - task identifier (and filter options for the associated receiver),

[0226] - Task start time, estimated task end time,

[0227] -Task start location, task end location,

[0228] -Task identifier status (e.g., progress on a specific work task),

[0229] - the estimated time to complete the task,

[0230] - Communication status: the radio access technology (RAT) identifier used, the time and status of the last communication,

[0231] - fuel or energy status,

[0232] -Leave (manual, event).

[0233] Fig. 9 A sequence diagram showing the communication between the operator and the master entity regarding mission planning.

[0234] When mission planning begins, the operator may have the opportunity to create a mission plan and provide the mission plan to the main entity via a man-machine interface, or to provide the mission plan to the cluster system via a back-end system. If the main entity cannot be connected to the back-end system, the main entity may automatically generate a mission configuration using the mission plan provided by the operator. For example, the operator may provide the mission plan at the workplace or its base via the interface of the main entity.

[0235] Through the mission configuration data and status, the operator and the host entity cooperate to ensure that a feasible mission configuration is created for each mission according to the mission plan received from the operator.

[0236] Fig.10 A sequence diagram of the communication between the operator, the backend, and the master entity regarding mission planning is presented.

[0237] At the beginning of mission planning, the operator creates a mission plan. If the host entity can connect to the backend system, the backend system can automatically pre-generate the mission configuration and provide it to the host entity.

[0238] Fig.11 A sequence diagram of the communication between the master and slave entities during cluster formation is shown, as described in detail in the above description. Cluster formation also provides a mechanism for the master to actively identify situations where a cluster entity disengages and / or stops performing its tasks, which may harm mission accomplishment, and invite another cluster entity that successfully performs a mission or other operation to replace the disengaged or underperforming cluster entity.

[0239] Fig.12 A sequence diagram of the communication between the master entity and the slave entities regarding cluster operations is presented.

[0240] When tasks are successfully executed according to the schedule, the cluster operations manager of the master entity coordinating the execution of the mission receives status reports from the slave entities indicating those successful completions via mission status reports. Upon successful completion of the last task in the mission schedule, the master entity transmits a message conveying an indication of mission completion to the backend for forwarding to the operator by the backend.

[0241] If the reports received from each slave entity cluster operations manager are consistent with the continued successful execution of the task, so that no messages indicating excessive execution time and / or task stoppage have been received, the master entity's cluster operations manager may not take any action related to the execution of these tasks. However, in response to receiving one or more reports consistent with non-compliant behavior of slave entities and / or task execution failures during task execution at the master entity cluster operations module, the master entity's cluster operations module may transmit one or more messages to trigger the cessation of the schedule for executing the task. In doing so, the master entity's cluster operations module may also trigger the cessation of the mission plan for executing the task schedule.

[0242] It is possible that, during the execution of a mission, if one of those slave entities unexpectedly collides, leaves the cluster, and / or suffers some other incident or problem, the cluster operations manager of the master entity may be notified of such an event as a result of ceasing to receive status reports from that cluster entity for a predetermined period of time. In response, the cluster operations manager of the master entity may cause the execution of the mission to be restarted within another mission by the same cluster entity or by another cluster entity of the same type.

[0243] Fig.13 A sequence diagram showing the communication between the backend, master entity, and slave entities regarding cluster operations in which mission status reports are created and exchanged.

[0244] Through master status reporting, the backend and master can cooperate to monitor the master's communications and provide master status reports; in addition, they can cooperate to monitor and record cluster and mission status. Further, the master and slave entities can cooperate to exchange status information about the execution of the task to ensure its completion regardless of problems or other incidents with the cluster entity.

[0245] More specifically, when receiving the non-instantiated report status of command task, the cluster operation manager of main entity can transmit the unsuccessful indication of the attempt of execution task timetable to the mission planning manager via cluster control message.In response, the mission planning manager can stop any further execution of any task that comprises to perform this task in the mission plan, and can transmit the indication of the execution of the mission of the back end by mistake via the mission status report.The back end can then forward this indication to the operator.As will also be explained in more detail, this indication can trigger the operator to command the mission planning manager to generate the new mission plan that replaces the initial plan.

[0246] In addition, a fallback mechanism is provided. If the initial master entity is paralyzed, the pre-designated sub-master vehicle designated by the initial master entity during the cluster formation must assume the role of the master entity, possibly adjust the mission configuration, and announce the changes in the cluster settings to the cluster entity. If no cluster entity is able to assume the role of the master entity, the mission may need to be aborted. If the master entity is able to connect to the back end, it can request the help of a remote operator, who should reconfigure the cluster and its mission configuration. Similar fallback mechanisms are provided not only for the master entity to transfer its role to another cluster entity, but also for cluster slave entities that are unable to perform their tasks for various reasons.

[0247] Since cluster entities may not necessarily be within direct communication range relative to each other during the execution of their assigned missions, if the underlying radio technology is limited to a communication range, it must be able to forward and relay messages to the corresponding recipients in a multi-hop manner. For example, if a cluster slave entity wants to report its status to the master entity, only neighboring cluster entities within the communication range of the transmitting cluster slave entity can receive the message and potentially forward this message to the final destination across multiple cluster entities and therefore across multiple communication hops. In order to facilitate cluster formation, the first key data of the cluster concept (e.g., mission type, cluster identifier, vehicle profile identifier) ​​needs to be translated into CoAP resources and terms. For example, the cluster will be treated as a "group" and each cluster entity will be treated as an "endpoint".

[0248] However, since the cluster entities are considered to act as both servers and clients, extensions of the CoAP protocol can be used to implement a publish / subscribe communication model that appears to be "agent-less" and more efficient compared to MQTT. For example, the publish-subscribe communication model enables push notifications to be sent when events occur, rather than constantly polling resources. This reduces power consumption as well as messages on the cluster. Compared to MQTT, no intermediate proxy entity is required, and communication can be performed in an asynchronous manner. For example, a cluster entity can publish its status report without receiving an explicit request.

[0249] Fig.14 A sequence diagram showing a master entity announcement message using the CoAP format.

[0250] First, the master entity can act as a proxy. The master entity can use IP multicast to publish its master entity announcement. Slave entities that are not in direct communication range of the master entity can receive this message via multi-hop communication, and the message is forwarded by the intermediate node.

[0251] The communication scheme envisioned is a hybrid scheme. Depending on the operating environment of the cluster (e.g., plains vs. open pit mines), the communication technology, mode or channel may change. Further, it is envisioned that a high-altitude platform or drone supports the positioning of cluster nodes and directly communicates (i.e., acts as a relay or integrated access and backhaul (IAB) node in 3GPP terminology) and instructs other cluster entities where to move to for cluster formation.

[0252] Another aspect of the present invention provides a computer-implemented program, which includes at least one program module that directs a master entity computing system to function in a specified manner to manage a cluster and perform a mission assigned to the cluster. The program module includes the following instructions: receiving an indication of a mission assigned to the cluster, and sending a master entity announcement message to at least one potential slave entity that can be reached directly or indirectly through at least one communication link, the master entity announcement message including a mission type. Further, the following instructions are also provided: directly or indirectly through a communication link receiving a cluster join message from at least one responding slave entity having a profile matching the declared mission type; analyzing the messages received from all responding slave entities to determine a mission plan or mission configuration based on the assigned mission; assigning the mission plan or mission configuration to the responding slave entity; and periodically receiving reports from the slave entity on the status of mission execution.

[0253] In addition, the present invention provides a computer-implemented program, which includes at least one program module, which guides the slave entity computing system to function in a specified manner to join the cluster and execute tasks scheduled according to the mission assigned to the cluster. The mentioned program module includes the following instructions: receiving a master entity declaration message including a mission type by the slave entity computing system; checking whether the slave entity profile matches the mission type, the slave entity profile including computing resources and capabilities matching the declared mission type; when the slave entity profile matches the declared mission type, sending at least one cluster joining message to the master entity in response to join the execution of the task; and regularly sending a report on the execution status of the scheduled task to the master entity.

[0254] Additional potential use cases for the present invention are listed below: agricultural machinery / robotics, whereby various vehicle platforms are used for different use cases (e.g., seeding, weeding, crop inspection), or a tractor operating in an orchard plus various implements (e.g., sprayer, mower); autonomous operation of mining trucks (from "ground" to "surface"); construction / road engineering, where paving is done by an operator driving a paving machine, but compaction is done by a cluster of rollers; cargo handling: loading and unloading of containers in a defined area with reach cranes and autonomous automated guided vehicles (aAGVs); loading and unloading of cargo and luggage in a defined area (e.g., airports, ports); supplying production facilities via autonomous automated guided vehicles.

[0255] However, while certain embodiments of the present invention have been described in detail, those familiar with the art to which this invention relates will recognize various alternative designs and embodiments for practicing the invention as defined by the following claims.

[0256] Reference numerals list

[0257] aAGV - Autonomous Automatic Guided Vehicle

[0258] MV - Main Vehicle

[0259] SV - From Vehicle

[0260] 1-Cluster system

[0261] 11-Cluster Master System

[0262] 12-Mission Planning Manager

[0263] 120-Cluster Fleet Manager

[0264] 1001-Cluster management module of the main entity

[0265] 1002-Cluster management module from the entity

[0266] 200-Cluster Operation Module

[0267] 210-Cluster Operation and Task Execution Manager

[0268] 2201-Cluster master entity status collection and monitoring manager

[0269] 2202-Cluster slave entity status collection and reporting manager

[0270] 300-Motion Module

[0271] 400-Perception Module

[0272] 500-Communication Module

[0273] 600-Positioning Module

[0274] 700 - Other vehicles / road users

[0275] 800-Backend

[0276] 900-Human Machine Interface

[0277] References

[0278] Non-patent literature

[0279] [1] “A Comparative Study of Wireless Protocols: Bluetooth, UWB, ZigBee, and Wi-Fi”, Karunakar Pothuganti and Anusha Chitneni, Advance in Electronic and Electric Engineering, ISSN 2231-1297, Vol. 4, No. 6 (2014), pp. 655-662

[0280] [2] “Wireless sensor networks: A survey on the state of the art and the 802.15.4 and ZigBee standards”, Paolo Baronti, Prashant Pillai, Vince W.C. Chook, Stefano Chess, Alberto Gotta, Y. Fun Hua, Computer Communications, Vol. 30, No. 7, May 26, 2007, pp. 1655-1695.

[0281] [3] “Evaluation of wireless short-range communication performance in aquarry environment”, Susanne Vernersson, Eleni Kalpaxidou, and David Rylander, 2013 International Conference on Connected Vehicles and Expo (ICCVE), DOI: 10.1109 / ICCVE.2013.6799812

[0282] [4] “Energy savings by wireless control of speed, scheduling and traveltimes for hauling operation”, David Rylander, Jakob Axelsson, Peter Wallin, 2014 IEEE Intelligent Vehicles Symposium (IV), DOI: 10.1109 / IVS.2014.6856451

[0283] [5] “Artificial Intelligence Supervised Swarm UAVs for Reconnaissance”, Saatvik Awasthi, Balamurugan Balamurugan, and V. Porkodi, Data Science and Analytics, May 2020, DOI: 10.1007 / 978-981-15-5827-6_33

[0284] [6] “Flying ad-hoc networks (FANETs): A review of communication architectures, and routing protocols”, Muhammad Asghar Khan, Alamgir Safi, Ijaz Mansoor Qureshi, Inam Ullah Khan, 2017 First International Conference on Latest trends in Electrical Engineering and Computing Technologies (INTELLECT), DOI: 10.1109 / INTELLECT.2017.8277614

[0285] [7] ENSEMBLE project presentation “Intelligent Maneuver Automation-cooperative hazard avoidance in realtime (IMAGinE)”, author Lehmann B., 9th ETSI IST Workshop, March 6-8, 2018, Berlin

[0286] [8] ENSEMBLE project publication “Specifications for multi-brand truck platooning”, Edoardo Mascalchia, Alessandro Coda, Dehlia Willemsen, Proceedings of 8th Transport Research Arena TRA 2020, April 27-30, 2020, Helsinki, Finland

[0287] Patent Literature

[0288] US2013123981A1

[0289] US2013128726A1

[0290] US2014080527A1

[0291] US2017093697A1

[0292] US2018097647A1

[0293] US2020322895A1

[0294] US2016381596A1

[0295] US2018167452A1

[0296] US2019049931A1

[0297] US6636781

Claims

1. A system for coordinating and managing a cluster of automated entities, wherein: The cluster includes at least the specified master entity and slave entity. each automated entity comprises a data processing mechanism, namely at least one processor coupled to at least one memory and adapted to execute instructions facilitating the operation of the program modules, and each automated entity has specific assets and capabilities necessary to collaborate with other entities and perform the mission as a cluster, The invention is characterized in that a cluster management module is instantiated at the main entity, the cluster management module includes a mission planning manager, the mission planning manager is adapted to receive or load a mission and provide a mission configuration, The designated cluster master entity is configured to announce itself through the communication module and the communication link, and request at least one automation entity different from and external to the master entity to join in the execution of the mission as a cluster slave entity.

2. The system according to claim 1, characterized in that The cluster fleet manager is instantiated at each slave entity that responds to the master entity's request, provided that the slave entity matches the profile derived from the mission configuration, The cluster formation manager of each slave entity is configured to receive a role in the cluster, join the cluster and receive instructions related to the cluster formation.

3. The system according to claim 1, characterized in that A cluster operation module is further instantiated at the master entity, the cluster operation module being configured to receive or load instructions and key data sets to coordinate the execution of operations and tasks derived from the mission configuration and to monitor the cluster's execution of the mission.

4. The system according to claim 1, characterized in that The cluster operations module is further instantiated at each cluster slave entity in response to a request sent by the master entity, and is configured to receive or load instructions and key data sets to coordinate in executing operations and tasks derived from the mission configuration, and report on the execution of tasks by each cluster slave entity.

5. The system according to claim 1, characterized in that Each cluster entity further includes a human-machine interface through which manual control instructions are received from an operator.

6. The system according to claim 1, characterized in that The master entity is included in a backend system, and more specifically in a cloud backend system.

7. The system according to claim 1, characterized in that The mission planning manager of the master entity is configured to receive or load a mission plan defined according to a predefined catalog, further perform asset counts based on received or loaded mission plans, which means further requesting or rejecting assets, and, Based on the asset inventory performed, create a mission configuration derived from the mission plan.

8. The system according to claims 1 and 2, characterized in that The cluster fleet manager of the master entity is configured to generate a first key data set, the first key data set comprising: Cluster identity, From the entity identity, Mission type, Cluster entity type, Role assignment, A behavioral policy, which is encoded as a list of rules for each event, entity role, and entity type, Task schedule, Cluster entity trajectories.

9. The system according to claims 1 and 8, characterized in that The cluster fleet manager of the master entity is further configured to generate a second key data set as input data of the cluster operation module of the master entity, the second key data set comprising: Mission type, Behavioral strategies, Task schedule, and Cluster entity trajectories.

10. The system according to claims 1, 2 and 8, characterized in that The cluster fleet manager of each slave entity is configured to receive the first key data set and provide the second key data set, the second key data set comprising: Mission type, Behavioral strategies, Task schedule, and Cluster entity trajectories.

11. The system according to claims 1 and 10, characterized in that The cluster fleet manager of each cluster slave entity is configured to provide the second key data set as input data to the cluster operation module of the corresponding cluster slave entity.

12. The system according to claim 1, characterized in that Each cluster entity further comprises a motion module configured to process the received task and trajectory data and provide motion status data as output.

13. The system according to claim 1, characterized in that Each cluster entity further includes a perception module configured to process data provided by the sensing mechanism and provide object data regarding the presence of the entity in the cluster operating environment.

14. The system according to claims 1, 12 and 13, characterized in that The motion module of each cluster entity further receives object data related to other entities from the perception module and object data related to other cluster entities received from the communication module.

15. The system according to claim 1, characterized in that The cluster operation module of the master entity is configured to collect mission status data periodically.

16. System according to the preceding claim, characterised in that Each cluster entity further includes an event module, which is configured to record status and data, and further periodically perform health checks on its own cluster entity, and report its own status to the cluster operation module.

17. System according to the preceding claim, characterized in that Each cluster entity further includes a hazard module configured to record hazard data and conditions related to incidents, accidents, and problems, and report such hazard data to the event module.

18. System according to the preceding claim, characterised in that The communication module of the master entity sends a third key data set to and receives a third key data set from the communication module of each slave entity, the third key data set comprising: Cluster identity, From the entity identity, Mission type, Entity type, Role assignment, Behavioral strategies, Status report, Task schedule, Entity tracks.

19. A method for coordinating and managing a cluster of automated entities, wherein: The cluster includes at least a designated master entity and a slave entity, and is characterized in that the method includes: receiving, by the designated master entity, an indication of a mission assigned to the cluster, The designated master entity sends a master entity announcement message to at least one potential slave entity that can be reached directly or indirectly through at least one communication link, the master entity announcement message including a mission type, receiving the master entity announcement message by all reachable potential slave entities to check whether at least one potential slave entity has a profile matching the mission type, the profile including assets and capabilities matching the announced mission type, At least one slave entity among the slave entities having a profile matching the announced mission type sends at least one cluster join message to the master entity directly or indirectly through the communication link in response.

20. The method according to claim 19, characterized in that As part of the cluster container, all cluster-related messages have a standardized and extended message format.

21. The method according to claim 19, characterized in that The method further includes: analyzing, by the cluster management module of the master entity, messages received from all responding slave entities to determine a mission plan or mission configuration based on the assigned mission; and The mission plan or the mission configuration encoded in a cluster management message is distributed to all slave entities by the communication mechanism of the master entity.

22. The method according to claims 19, 20 and 21, characterized in that The mission plan and / or the mission configuration is defined according to a predefined catalog, and the mission configuration specifies a set of entity types and a set of task schedules to be assigned to cluster entities.

23. The method according to claim 19, characterized in that The method further includes sending a set of status report messages by the slave entity regularly at predefined time intervals or in the event of an unexpected event and receiving a set of status report messages by the master entity regularly at predefined time intervals or in the event of an unexpected event, wherein the status report messages include a first set of key data, namely, an identifier of the cluster, a cluster entity type, a mission type, an entity role, a task, a task start time and an estimated task end time, a task status, an estimated time to complete the task, a communication status, and a fuel or energy status.

24. The method according to claim 19, characterized in that The method further includes sending a set of cluster management messages by the master entity, wherein the cluster management messages include a second set of key data, the second set of key data including an identifier of the cluster, a cluster entity type, a mission type, an entity role, a behavioral strategy encoded as a list of rules for each event and role, an entity trajectory, and a set of task schedules to be performed during execution of the mission plan.

25. The method according to claim 19, characterized in that The method further includes: each cluster slave entity propagating a cluster management message, thereby distributing the second key data among reachable cluster slave entities, and optionally distributing the second key data together with the first key data to more distant reachable cluster entities through various communication protocols.

26. The method according to claim 19, characterized in that The method further includes: each reachable entity that joins a cluster reached by another cluster slave entity regards the master entity of the reached cluster slave entity as its own master entity.

27. The method according to claim 19, characterized in that The method further includes an option to assign a secondary master entity role to at least one cluster entity having a profile matching the primary entity profile and qualifying as a secondary master entity.

28. The method according to claim 27, characterized in that The method further includes monitoring, by the secondary host entity, host entity messaging.

29. The method according to claims 19, 27 and 28, characterized in that The method further includes transferring the master entity role to the secondary master entity when the current cluster master entity cannot act according to the master entity role requirement, wherein the master entity role transfer is controlled by the behavior strategy.

30. The method according to claims 19 and 29, characterized in that The method includes an option to announce a change or transfer of the master entity role by issuing a new master entity announce message that notifies the cluster that a new cluster master entity is replacing the current master entity.

31. The method according to the preceding method claim, characterized in that Further comprising synchronizing operations in time across the master entity and the slave entities, the synchronization comprising generating and maintaining an event queue that records conditions and data and performs routine health checks, the recorded conditions and data including hazardous data related to unexpected events, problems or accidents.

32. The method according to claim 19, characterized in that The communication is hybrid, employing client-server and / or peer-to-peer communication schemes.

33. A cluster master device, comprising: Autonomous driving module; Movement module; Perception module; at least one processor coupled to the at least one memory and adapted to execute routines including instructions for managing the cluster and performing missions assigned to the cluster; and the hardware mechanisms necessary to perform the mission, which include cluster management middleware; and mechanisms for communicating with other cluster entities and entities outside the cluster, which include communication management middleware.

34. A cluster slave device, comprising: Autonomous driving module; Movement module; Perception module; at least one processor coupled to the at least one memory and adapted to execute a routine including instructions for joining the cluster at the request of the cluster master entity and performing tasks scheduled based on the mission configuration; and the hardware mechanisms necessary to perform the task; and mechanisms for communicating with the cluster master entity, other cluster slave entities, and entities outside the cluster, such mechanisms including communication management middleware.

35. A computer-implemented program comprising at least one program module that directs a host entity computing system to function in a specified manner to manage a cluster and perform a mission assigned to the cluster, the program module comprising instructions for: receiving an indication of a mission assigned to the cluster, sending a master entity announcement message to at least one potential slave entity that can be reached directly or indirectly through at least one communication link, the master entity announcement message including a mission type, receiving, directly or indirectly through the communication link, a cluster join message from at least one responding slave entity having a profile matching the announced mission type, Analyze the messages received from all responding slave entities to determine the mission plan or mission configuration based on the assigned mission, assigning the mission plan or the mission configuration to the responding slave entities, and Receive regular reports from slave entities on mission performance status.

36. A computer-implemented program, comprising at least one program module, the at least one program module directing a slave computing system to function in a specified manner to join a cluster and perform tasks scheduled according to a mission assigned to the cluster, the program module comprising instructions for: The slave computing system receives a master declaration message including a mission type, checking whether a slave entity profile matches the mission type, the slave entity profile including computing resources and capabilities matching the declared mission type, When the slave entity profile matches the declared mission type, sending at least one cluster join message to the master entity in response, Join the execution of this task, and Periodically send reports to the master entity on the execution status of scheduled tasks.

Images