Artificial intelligence-based system and method for managing heterogeneous robot populations that are independent of networks.

Abstract

An AI-based system and method for managing a network-independent heterogeneous robot fleet is disclosed. The method includes receiving a command set from a human-machine interface (102) associated with one or more electronic devices (104), determining one or more robot capabilities associated with the autonomous robots, and acquiring one or more position parameters using one or more sensors (112). The method includes broadcasting the one or more robot capabilities and the one or more position parameters to each of the one or more autonomous robots (110) and determining one or more situation parameters associated with the one or more autonomous robots (110). Additionally, the method includes detecting one or more targets and allocating one or more tasks and the detected one or more targets among the one or more autonomous robots (110).

Description

Technical Field

[0001] Earliest Priority Date: The present application claims priority from a provisional patent application with patent application number 202141010296 filed in India on March 11, 2021, and the title is "Artificial Intelligence Based System and Method for Managing Network-Independent Heterogeneous Unmanned Aerial Vehicle Groups".

[0002] Field of the Invention Embodiments of the present disclosure relate to artificial intelligence (AI) based systems, and more particularly, to AI-based systems and methods for managing network-independent heterogeneous robot groups.

Background Art

[0003] In general, autonomous robots such as unmanned aerial vehicles are useful in many application scenarios, especially scenarios that are unreachable or dangerous for humans, such as regional patrol, disaster rescue missions, event monitoring and surveillance, etc. Furthermore, a "swarm" refers to a collection of autonomous robots that work cooperatively to perform any task.

[0004] In conventional approaches, swarms cooperate using a role-based approach called "master-slave". In this approach, the movement of the entire swarm is orderly controlled around the leader (master) autonomous robot. Each autonomous robot in the swarm follows decisions associated with the master autonomous robot. However, this approach has a major drawback that if the master autonomous robot malfunctions or is endangered for any reason, the entire swarm will malfunction.

[0005] Another approach to controlling swarms is via a command center, where each set of autonomous robots forming the swarm receives instructions from the command center. This approach also has limitations regarding the availability of communication networks. In the absence or vulnerability of communication networks, the swarm will not function, severely limiting its geographical reach. Furthermore, current swarming solutions can manage swarms of homogeneous autonomous robots with the aforementioned limitations, but are ineffective in controlling swarms of heterogeneous autonomous robots.

[0006] Therefore, to address the aforementioned problems, improved AI-based systems and methods are needed for managing network-independent, heterogeneous robot populations. [Overview of the project]

[0007] This summary is provided to give a brief overview of some of the concepts that will be further described in the detailed description of this disclosure. This summary is not intended to identify any important or essential inventive concepts of the subject matter, nor is it intended to determine the scope of this disclosure. In one embodiment of this disclosure, an artificial intelligence (AI)-based computing system for managing a network-independent heterogeneous robot population is disclosed. The AI-based computing system includes one or more hardware processors and memory coupled to one or more hardware processors. The memory includes a plurality of modules in the form of programmable instructions executable by one or more hardware processors. The plurality of modules includes a data receiving module configured to receive a set of commands from a human-machine interface associated with one or more electronic devices for performing one or more tasks on one or more targets via one or more autonomous robots. The set of commands is derived from one or more high-level commands provided by an operator for performing one or more tasks via one or more autonomous robots. The plurality of modules also include a capability determination module configured to determine one or more robot capabilities associated with an autonomous robot based on the received set of commands and predefined robot information. One or more robot capabilities consist of payload type, sensor type, velocity, weight-holding capability, configuration, battery level, class, and weight of the autonomous robot. The plurality of modules include a parameter capture module configured to capture one or more position parameters using one or more sensors. One or more position parameters include the position of the autonomous robot, the predicted position of the autonomous robot, one or more images of one or more targets, one or more videos of one or more targets, a set of images of other nearby autonomous robots, a set of videos of other autonomous robots, and one or more sounds of the surrounding environment. Furthermore, the multiple modules include a broadcast module configured to broadcast the determined one or more robot capabilities and the captured one or more position parameters to each of the one or more autonomous robots.The multiple modules also include a parameter determination module configured to determine one or more contextual parameters associated with one or more autonomous robots by using a robot management-based artificial intelligence (AI) model based on one or more broadcasted robot capabilities and one or more broadcasted position parameters, a set of received commands, and one or more responses of one or more determined robot capabilities and one or more captured position parameters. The one or more contextual parameters include the payload type, sensor type, speed, payload capacity, class, battery level, the position of one or more autonomous robots, the expected position of one or more autonomous robots, one or more images of one or more targets, one or more videos of one or more targets, a set of videos of one or more autonomous robots, a set of videos of one or more autonomous robots, the relative position of one or more autonomous robots to each of the one or more autonomous robots, and one or more sounds of the surrounding environment. Furthermore, the multiple modules include a target detection module configured to detect one or more targets by using a robot management-based AI model based on a set of received commands, one or more determined robot capabilities, one or more captured position parameters, and one or more determined contextual parameters. The multiple modules include a task assignment module configured to assign one or more tasks and one or more detected targets among one or more autonomous robots based on a robot management-based AI model, using a set of received commands, one or more determined robot capabilities, one or more captured position parameters, one or more determined situation parameters, one or more target parameters, and predefined assignment information. The one or more target parameters include the number of one or more targets, the size of each of the one or more targets, and the sensor footprint corresponding to each of the one or more autonomous robots.Multiple modules include task execution modules configured to perform one or more tasks on one or more detected targets, based on one or more tasks and the assignment of one or more tasks between one or more autonomous robots, using a robot management-based AI model.

[0008] In accordance with another embodiment of this disclosure, an artificial intelligence (AI)-based method for managing a network-independent heterogeneous robot population is disclosed. The AI-based method includes receiving a set of commands from a human-machine interface associated with one or more electronic devices for performing one or more tasks via one or more autonomous robots on one or more targets. The set of commands is derived from one or more high-level commands provided by an operator for performing one or more tasks via one or more autonomous robots. The AI-based method further includes determining one or more robot capabilities associated with the autonomous robots based on the received set of commands and predefined robot information. One or more robot capabilities include payload type, sensor type, velocity, weight-holding capacity, configuration, battery level, class, and weight of the autonomous robots. Furthermore, the AI-based method includes capturing one or more position parameters using one or more sensors. One or more position parameters include the position of the autonomous robot, the expected position of the autonomous robot, one or more images of one or more targets, one or more videos of one or more targets, a set of images of other nearby autonomous robots, a set of videos of other autonomous robots, and one or more sounds of the surrounding environment. The AI-based method also includes broadcasting one or more determined robot capabilities and one or more captured position parameters to each of one or more autonomous robots. Furthermore, the AI-based method includes determining one or more situational parameters associated with one or more autonomous robots based on an artificial intelligence (AI) model based on robot management, using one or more broadcasted robot capabilities and one or more broadcasted position parameters, a set of commands received, and one or more responses to the determined robot capabilities and one or more captured position parameters.One or more situational parameters include payload type, sensor type, speed, payload capacity, class, battery level of one or more autonomous robots, position of one or more autonomous robots, expected position of one or more autonomous robots, one or more images of one or more targets, one or more videos of one or more targets, a set of images of one or more autonomous robots, a set of videos of one or more autonomous robots, the relative position of one or more autonomous robots to each of the one or more autonomous robots, and one or more sounds of the surrounding environment. The AI-based method also includes detecting one or more targets based on a received command set, one or more determined robot capabilities, one or more captured positional parameters, and one or more determined situational parameters by using a robot management-based AI model. Furthermore, the AI-based method includes assigning one or more tasks and one or more detected targets among one or more autonomous robots based on a received command set, one or more determined robot capabilities, one or more captured positional parameters, one or more determined situational parameters, one or more target parameters, and predefined assignment information by using a robot management-based AI model. One or more target parameters include the number of one or more targets, the size of each of the one or more targets, and the sensor footprint corresponding to each of the one or more autonomous robots. The method includes performing one or more tasks on one or more detected targets based on the assignment of one or more tasks between one or more autonomous robots and the detected one or more tasks, using a robot management-based AI model. To further clarify the merits and features of this disclosure, a more specific description of this disclosure follows by reference to its specific embodiments shown in the accompanying figures. It should be understood that these figures only illustrate typical embodiments of this disclosure and are therefore not intended to limit its scope. This disclosure is described in more specific and detail with reference to the accompanying figures. [Brief explanation of the drawing]

[0009] This disclosure will be described in more specific and detailed terms with reference to the attached figures. [Figure 1] Figure 1 is a block diagram showing an exemplary computing environment for managing a network-independent, heterogeneous group of robots according to one embodiment of the present disclosure. [Figure 2] Figure 2 is a block diagram illustrating an exemplary artificial intelligence (AI)-based computing system for managing a network-independent, heterogeneous group of robots, according to one embodiment of the present disclosure. [Figure 3] Figure 3 is an exemplary pictorial representation showing the division of a target area between one or more autonomous robots according to one embodiment of the present disclosure. [Figure 4A] Figure 4A is a block diagram showing an exemplary action execution module according to an embodiment of the present disclosure. [Figure 4B] Figure 4B is a block diagram illustrating an exemplary operation of an AI-based computing system for managing a network-independent, heterogeneous group of robots, according to one embodiment of the present disclosure. [Figure 5] Figure 5 is a pictorial depiction of exemplary operation of an AI-based computing system for managing a network-independent, heterogeneous group of robots, according to another embodiment of the present disclosure. [Figure 6] Figure 6 is a process flow diagram illustrating an exemplary AI-based method for managing a network-independent, heterogeneous group of robots according to one embodiment of the present disclosure. [Figure 7A]Figures 7A to 7C show graphical user interface screens of an AI-based computing system for managing a network-independent, heterogeneous group of robots, according to one embodiment of the present disclosure. [Figure 7B] Figures 7A to 7C show graphical user interface screens of an AI-based computing system for managing a network-independent, heterogeneous group of robots, according to one embodiment of the present disclosure. [Figure 7C] Figures 7A–7C are graphical user interface screens of an AI-based computing system for managing a network-independent, heterogeneous group of robots, according to one embodiment of the present disclosure. Furthermore, those skilled in the art will understand that elements in the figures are illustrated for simplification and may not necessarily be drawn to scale. Furthermore, with respect to the structure of the apparatus, one or more components of the apparatus may be represented in the figures by conventional symbols, and the figures may show only certain details appropriate to understanding the embodiments of the present disclosure, so as not to obscure the figures with details that would be readily apparent to those skilled in the art who have an interest in the description herein. [Modes for carrying out the invention]

[0010] For the purpose of facilitating understanding of the principles of this disclosure, embodiments shown in the figures will now be referred to and described using specific terminology. Nevertheless, it will be understood that no limitation of the scope of this disclosure is intended thereto. Such changes and further modifications in the illustrated systems, as well as such further applications of the principles of this disclosure that would ordinarily arise for those skilled in the art, will be construed as being within the scope of this disclosure. It will be understood for those skilled in the art that the above general description and the following detailed description are illustrative and descriptive of this disclosure and are not intended to limit it.

[0011] In this specification, the term “exemplary” is used to mean “serving as an example, illustration, or illustration.” Any embodiment or implementation of the subject matter described herein as “exemplary” is not necessarily construed to be preferable or advantageous to other embodiments.

[0012] "Comprise," "comprising," or other variations thereof are intended to be non-exclusive inclusions, and one or more devices, subsystems, elements, structures, or components preceding "comprises...a" do not, without further constraints, exclude the existence of other devices, subsystems, or additional submodules. Whereever "in one embodiment," "in another embodiment," and similar expressions appear throughout this specification, they may, but not necessarily, refer to the same embodiment.

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art to which this disclosure pertains. The systems, methods, and examples provided herein are illustrative and not intended to limit the scope of this disclosure.

[0014] A computer system comprised of applications (a standalone, client, or server computer system) can consist of “modules” (or “subsystems”) that are configured and operated to perform specific actions. In one embodiment, a “module” or “subsystem” can be mechanically or electronically implemented, and therefore a module may include dedicated circuitry or logic that is permanently configured (within a dedicated processor) to perform a specific action. In another embodiment, a “module” or “subsystem” may consist of programmable logic or circuitry that is temporarily configured by software (such as being contained within a general-purpose processor or other programmable processor) to perform a specific action.

[0015] Therefore, the terms “module” or “subsystem” should be understood to encompass tangible entities that are physically hardwired or temporarily programmed, configured to operate in a particular manner and / or perform a particular operation as described herein.

[0016] Refer to the drawings here, and more specifically to Figures 1 to 7C. Figures in which similar reference letters consistently correspond to features throughout the figures show preferred embodiments, and these embodiments are described in the context of the following exemplary systems and / or methods.

[0017] Figure 1 is a block diagram showing an exemplary computing environment 100 for managing a network-independent heterogeneous robot swarm according to one embodiment of the present disclosure. According to Figure 1, the computing environment 100 includes a human-machine interface 102 associated with one or more electronic devices 104 that are communicably coupled to an artificial intelligence (AI)-based computing system via a communication network 108. In one embodiment of the present disclosure, an operator uses the human-machine interface 102 to provide one or more high-level commands to enable one or more autonomous robots 110 to perform one or more tasks. The one or more autonomous robots 110 collectively form a heterogeneous swarm and collaboratively perform one or more tasks. For example, one or more tasks may include rescuing survivors, dropping medical kits, dropping food boxes, destroying targets, taking images, etc. In one embodiment of the present disclosure, the human-machine interface 102 also derives a set of commands from one or more high-level commands provided by the operator to perform one or more tasks via one or more autonomous robots 110. Furthermore, the operator also uses the human-machine interface 102 to receive one or more live parameters. Details regarding one or more live parameters will be detailed in subsequent paragraphs of this specification with reference to Figure 2. In exemplary embodiments of this disclosure, one or more high-level commands include a predefined set of commands representing one or more tasks performed by one or more autonomous robots 110, a starting point, a destination, the time to perform one or more tasks, the respective roles and speeds of one or more autonomous machines, altitude from the ground, one or more predefined conditions, a navigation route from the starting point to the destination, boundary fences, one or more intermediate positions, one or more targets, safe areas and dangerous areas. In one embodiment of this disclosure, one or more high-level commands are provided on a georeference map. For example, the predefined set of commands may include armament, cargo delivery, herd, forage, landing, movement to a location, training, takeoff, global positioning system (GPS) standby, standby, survivor detection, payload drop, etc.The communication network 108 may be the Internet, a radio signal, a 4G network, a 5G network, or any other radio network. In embodiments of this disclosure, the AI-based computing system 106 may be hosted on each of one or more autonomous robots 110. For the purposes of this description, the computing system is described in relation to a single autonomous robot. However, it should be obvious to those skilled in the art that the AI-based computing system 106 may be deployed on each of one or more autonomous robots 110. In another embodiment of this disclosure, the AI-based computing system 106 may be hosted on a central server, such as a cloud server or remote server. In exemplary embodiments of this disclosure, one or more electronic devices 104 may include laptop computers, desktop computers, tablet computers, smartphones, wearable devices, smartwatches, digital cameras, etc. For example, a wearable device may be an augmented reality headset, a virtual reality headset, etc. Augmented reality headsets and virtual reality headsets enable interfacing with heterogeneous swarms. In one embodiment of this disclosure, a human operator can replay a swarming session deployed over space and time by using one or more electronic devices 104.

[0018] Further, the computing environment 100 includes one or more autonomous robots 110 communicatively coupled to the AI-based computing system 106 via a communication network 108. In embodiments of the present disclosure, the AI-based computing system 106 is deployed on each of the one or more autonomous robots 110. In an exemplary embodiment of the present disclosure, the one or more autonomous robots 110 are unmanned vehicles. For example, the one or more autonomous robots 110 may include unmanned aerial vehicles, unmanned ground vehicles, unmanned water vehicles, and the like. In one embodiment of the present disclosure, each of the one or more autonomous robots 110 has different robot capabilities such as payload type, sensor type, speed, weight holding capacity, configuration, battery level, class and weight. Accordingly, the one or more autonomous robots 110 collectively form a heterogeneous group. For example, the heterogeneous group is a mixture of different unmanned aerial vehicles and unmanned ground vehicles. Further, each of the one or more autonomous robots 110 can operate independent of a network connection after receiving an initial set of commands. Accordingly, the one or more autonomous robots 110 are network-independent.

[0019] Further, the computing environment 100 includes one or more sensors 112 communicatively coupled to the AI-based computing system 106 via the communication network 108. In one embodiment of the present disclosure, the one or more sensors 112 are fixed to the one or more autonomous robots 110 to capture one or more position parameters. For example, the one or more sensors 112 include one or more image capture units, one or more audio capture units, optical sensors and infrared sensors, global positioning system (GPS), and the like. Details regarding the one or more position parameters are detailed in subsequent paragraphs herein with reference to FIG. 2.

[0020] Furthermore, the one or more electronic devices 104 include a local browser, a mobile application, or a combination thereof. Furthermore, an operator can use a web application via the local browser, the mobile application, or a combination thereof to communicate with the AI-based computing system 106 and control a heterogeneous group of the one or more autonomous robots 110. In an exemplary embodiment of the present disclosure, the mobile application may be compatible with any mobile operating system such as Android, iOS, etc. In one embodiment of the present disclosure, the AI-based computing system 106 includes a plurality of modules 114. Details of the plurality of modules 114 are detailed in subsequent paragraphs of the present specification with reference to Figure 2.

[0021] In one embodiment of this disclosure, the AI-based computing system 106 is deployed on each of one or more autonomous robots 110. However, for the sake of explanation, the computing system will be described in terms of a single autonomous robot. The AI-based computing system 106, i.e., the system deployed on the autonomous robot, is configured to receive a set of commands from a human-machine interface 102 associated with one or more electronic devices 104 in order to perform one or more tasks via one or more autonomous robots 110 on one or more targets. Based on the received set of commands and predefined robot information, the AI-based computing system 106 determines one or more robot capabilities associated with the autonomous robot. The AI-based computing system 106 captures one or more position parameters by using one or more sensors 112. Furthermore, the AI-based computing system 106 broadcasts the determined one or more robot capabilities and the captured one or more position parameters to each of the one or more autonomous robots 110. The AI-based computing system 106 uses a robot management-based artificial intelligence (AI) model to determine one or more situational parameters associated with one or more autonomous robots 110 based on one or more broadcasted robot capabilities and one or more broadcasted positional parameters, a received command set, and one or more responses of one or more determined robot capabilities and one or more captured positional parameters. The AI-based computing system 106 uses a robot management-based AI model to detect one or more targets based on a received command set, one or more determined robot capabilities, one or more captured positional parameters, and one or more determined situational parameters.The AI-based computing system 106, using a robot management-based AI model, assigns one or more tasks and one or more detected targets among one or more autonomous robots 110 based on the received command set, one or more determined robot capabilities, one or more captured position parameters, one or more determined situation parameters, one or more target parameters, and predefined assignment information. Furthermore, the AI-based computing system 106, using a robot management-based AI model, executes one or more tasks on one or more detected targets based on the assignment of one or more tasks and one or more detected tasks among one or more autonomous robots 110.

[0022] Figure 2 is a block diagram illustrating an exemplary artificial intelligence (AI)-based computing system 106 for managing a network-independent heterogeneous robot population according to one embodiment of the present disclosure. Furthermore, the AI-based computing system 106 104 includes one or more hardware processors 202, memory 204, and storage device 206. The one or more hardware processors 202, memory 204, and storage device 206 are communicably coupled via a system bus 208 or any similar mechanism. The memory 204 consists of a plurality of modules 114 in the form of programmable instructions executable by one or more hardware processors 202. The plurality of modules 114 are configured to perform a particular set of functions at different levels. Furthermore, the multiple modules 114 include a data reception module 210, a capability determination module 212, a parameter capture module 214, a broadcast module 216, a parameter determination module 218, a target detection module 220, a task assignment module 222, a task execution module 224, an action execution module 226, a task optimization module 228, a composite simulation module 230, and a live data detection module 232.

[0023] In one embodiment of this disclosure, the AI-based computing system 106 corresponds to one of one or more autonomous robots 110. The AI-based computing system 106 is deployed with each of the one or more autonomous robots 110. In one embodiment of this disclosure, the AI-based computing system 106 is deployed at the level of a specific individual class of one or more autonomous robots 110 for interface with a single swarming system. In exemplary embodiments of this disclosure, one or more autonomous robots 110 are unmanned aerial vehicles (UAVs), unmanned ground vehicles, unmanned water vehicles, etc. In one embodiment of this disclosure, each of the one or more autonomous robots 110 has different robotic capabilities such as payload type, sensor type, speed, weight-holding capacity, configuration, composition, battery level, class, and weight. Thus, one or more autonomous robots 110 collectively form a heterogeneous swarm. For example, a heterogeneous swarm is a mixture of different UAVs and unmanned ground vehicles. Furthermore, each of the one or more autonomous robots 110 can operate independently of network connectivity after receiving an initial set of commands. Thus, the one or more autonomous robots 110 are network-independent. For example, a swarm mission is configured based on a set of commands and deployed to a heterogeneous swarm of one or more autonomous robots 110, and the mission proceeds. Moreover, a ground control station (GCS) or man-on-the-loop is like a monitoring system, not like a command system. The heterogeneous swarm itself does not wait for or require instructions to perform the configured one or more tasks. In one embodiment of this disclosure, one or more tasks may be performed at a distance where communication is not possible. Thus, the one or more autonomous robots 110 communicate with each other for a cooperative swarm, rather than communicating with a GCS.

[0024] As used herein, one or more hardware processors 202 means, but are not limited to, any type of computing circuit, such as a microprocessor unit, a microcontroller, a complex instruction set computing microprocessor unit, a reduced instruction set computing microprocessor unit, an extra-long instruction word microprocessor unit, an explicit parallel instruction computing microprocessor unit, a graphics processing unit, a digital signal processing unit, or any other type of processing circuit. One or more hardware processors 202 may also include embedded controllers such as general-purpose or programmable logic devices or arrays, application-specific integrated circuits, or single-chip computers.

[0025] Memory 204 may be non-transient volatile memory and non-volatile memory. Memory 204 may be coupled for communication with one or more hardware processors 202, such as a computer-readable storage medium. One or more hardware processors 202 can execute machine-readable instructions and / or source code stored in memory 204. Various machine-readable instructions may be stored in and accessed from memory 204. Memory 204 may include any suitable elements for storing data and machine-readable instructions, such as read-only memory, random-access memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, and removable media drives for handling hard drives, compact disks, digital video disks, floppy disks, magnetic tape cartridges, memory cards, etc. In this embodiment, memory 204 may include a plurality of modules 114 stored in the form of machine-readable instructions in one of the above-mentioned storage media, which can communicate with and thereby execute one or more hardware processors 202. The storage device 206 may be cloud storage or local storage within each of the one or more autonomous robots 110. The storage device 206 may store one or more high-level commands, command sets, one or more robot capabilities, one or more position parameters, and one or more status parameters. The storage device 206 may also store one or more target parameters, predefined assignment information, one or more collision parameters, one or more task parameters, one or more optimization parameters, and one or more live parameters.

[0026] A data receiving module 210 is configured to receive a set of commands from a human-machine interface 102 associated with one or more electronic devices 104 and to perform one or more tasks via one or more autonomous robots 110 at one or more targets. The human-machine interface 102 is configured on one or more electronic devices 104. In one embodiment of the present disclosure, an operator uses the human-machine interface 102 to provide one or more high-level commands to enable one or more autonomous robots 110 to perform one or more tasks. In one embodiment of the present disclosure, one or more high-level commands are prepared at a ground control station (GCS) based on a georeference map. One or more high-level commands enable one or more autonomous robots 110 to perform one or more tasks as a group. In one embodiment of the present disclosure, one or more autonomous robots 110 collectively form a heterogeneous group and collaboratively perform one or more tasks. One or more high-level commands are issued to the heterogeneous group by a human so that each of the one or more autonomous robots 110 acts collaboratively to perform lower-level tasks in order to collaboratively achieve the one or more high-level commands. One or more tasks are performed simultaneously or in stages. For example, one or more tasks may include rescuing survivors, dropping medical kits, dropping food boxes, destroying targets, and capturing images. In one embodiment of the present disclosure, the human-machine interface 102 also derives a set of commands from one or more high-level commands provided by an operator to perform one or more tasks via one or more autonomous robots 110. The human-machine interface 102 is deployed at a full swarm level. The set of commands consists of machine-readable commands that enable one or more autonomous robots 110 to collaboratively perform one or more tasks. In embodiments of the present disclosure, the set of commands corresponds to a mission.In exemplary embodiments of this disclosure, one or more high-level commands include a predefined set of commands representing one or more tasks performed by one or more autonomous robots 110, a starting point, a destination, the time to perform one or more tasks, the role and speed of each of the one or more autonomous robots, altitude from the ground, one or more predefined conditions, a navigation path from the starting point to the destination, boundary fences, one or more intermediate positions, one or more targets, safe areas and dangerous areas. In one embodiment of this disclosure, one or more high-level commands are provided on a georeference map. For example, the predefined set of commands may include armament, package delivery, herd, foraging, landing, movement to location, training, takeoff, global positioning system (GPS) standby, standby, survivor detection, payload drop, etc. In one embodiment of this disclosure, an AI-based computing system 106 may be hosted on each of the one or more autonomous robots 110. For the purposes of this description, the computing system is described in relation to a single autonomous robot. However, it should be apparent to those skilled in the art that the AI-based computing system 106 may be deployed on each of the one or more autonomous robots 110. In another embodiment of the Disclosure, the AI-based computing system 106 may be hosted on a central server such as a cloud server or remote server. In one embodiment of the Disclosure, the navigation route is provided by an operator. In another embodiment of the Disclosure, the navigation route is generated by a human-machine interface 102 based on a starting point, destination, and GPS. In an exemplary embodiment of the Disclosure, one or more electronic devices 104 may include a laptop computer, a desktop computer, a tablet computer, a smartphone, a wearable device, a smartwatch, a digital camera, etc. For example, the wearable device may be an augmented reality headset, a virtual reality headset, etc. In one embodiment of the Disclosure, the human-machine interface 102 communicates with the AI-based computing system 106 via a communication network 108.

[0027] In one embodiment of the present disclosure, each of one or more autonomous robots 110 of a heterogeneous swarm knows the current state of the swarm while moving from a starting point to a destination. The heterogeneous swarm transitions to a new state, which cascades into swarming entities (groups) that move to other roles when one or more predefined conditions are met, such as a sufficient number reaching the destination. In embodiments of the present disclosure, one or more predefined conditions are parameterized and configured before the mission. In embodiments of the present disclosure, each of the one or more autonomous robots 110 periodically synchronizes its position with one another in real time so that it can behave as a heterogeneous swarm.

[0028] The capability determination module 212 is configured to determine one or more robot capabilities associated with the autonomous robot based on the received command set and predefined robot information. In exemplary embodiments of the disclosure, one or more robot capabilities include payload type, sensor type, speed, weight-holding capability, configuration, battery level, class, and weight of the autonomous robot. For example, the class of the autonomous robot may be a 20kg hexacopter with a 10kg quadcopter, Beluga (BLL), Nimbus (NMB), Mackerall (MCL), an unmanned aerial vehicle with camera fittings, a quadcopter, a hexacopter, and a ground unmanned aerial vehicle with different working speeds. In exemplary embodiments of the disclosure, payload types may be a camera payload, a robot, a medical kit, a communicable antenna payload, a weapon, and so on. In embodiments of the disclosure, each of the one or more autonomous robots 110 can accommodate other autonomous robots depending on the time and speed.

[0029] The parameter acquisition module 214 is configured to acquire one or more position parameters using one or more sensors 112. In exemplary embodiments of the Disclosure, the one or more position parameters include the position of the autonomous robot, the expected position of the autonomous robot, one or more images of one or more targets, one or more videos of one or more targets, a set of images of other nearby autonomous robots, a set of videos of other autonomous robots, one or more sounds of the surrounding environment, etc. In one embodiment of the Disclosure, one or more sensors 112 are fixed to each of one or more autonomous robots 110 to acquire one or more position parameters. For example, the one or more sensors 112 include one or more image acquisition units, one or more sound acquisition units, a Global Positioning System (GPS), LiDAR, radar, etc.

[0030] The broadcast module 216 is configured to broadcast one or more determined robot capabilities and one or more captured position parameters to each of one or more autonomous robots 110. In one embodiment of the present disclosure, the one or more determined robot capabilities and one or more captured position parameters are broadcast via a communication network 108. In one embodiment of the present disclosure, the one or more determined robot capabilities and one or more captured position parameters are compressed and encrypted before broadcast in order to maintain data privacy.

[0031] The parameter determination module 218 is configured to determine one or more contextual parameters related to one or more autonomous robots 110 based on one or more broadcasted robot capabilities and one or more broadcasted position parameters, a set of received commands, and one or more responses of the determined robot capabilities and one or more captured position parameters, using an artificial intelligence (AI) model based on robot management. In exemplary embodiments of this disclosure, one or more contextual parameters include the payload type, sensor type, speed, payload capacity, class, and battery level of each of the one or more autonomous robots 110, the position of each of the one or more autonomous robots 110, the expected position of each of the one or more autonomous robots 110, one or more images of one or more targets, one or more videos of one or more targets, a set of images of one or more autonomous robots 110, a set of videos of one or more autonomous robots 110, the relative position of the autonomous robots to each of the one or more autonomous robots 110, and one or more sounds of the surrounding environment. Since one or more autonomous robots 110 each have different payload types, sensor types, speeds, weight-holding capabilities, configurations, classes, weights, etc., one or more autonomous robots 110 form a heterogeneous group. For example, even if one or more autonomous robots 110 are of the same class but have different payload types associated with them, one or more autonomous robots 110 can still form a heterogeneous group.

[0032] In embodiments of this disclosure, heterogeneity in the form of different types of payloads may mean that one or more autonomous robots 110 may perform a specific role, such as communication payload equipment communication, and not perform search requiring a camera system. Furthermore, an autonomous robot equipped with a camera may search for targets and share them with a swarm of one or more autonomous robots 110. For example, an autonomous robot carrying a mission payload may service one or more targets. In one embodiment of this disclosure, different autonomous robots are assigned roles based on their configuration and perform those roles accordingly. For example, in a swarm of bees, there are worker bees, warrior bees, etc., each performing different roles. Furthermore, classes of autonomous robots may be similar, such as quadcopters and hexacopters, meaning that at least their speeds are similar. Thus, while their speed envelopes may be similar, their flight times may differ. In another example, a hexacopter with a large integrated circuit (IC) engine acts as a mothership carrying smaller battery-powered multicopters as payloads, with smaller quadcopters then deploying to cascade heterogeneous swarms. Furthermore, cross-class swarms, such as class mix-and-match (e.g., fixed-wing + multicopters), are evolving where certain aspects, such as collision avoidance, still occur, and faster units may try to accommodate smaller units. In exemplary embodiments of this disclosure, this can also be done with ground vehicles as a swarm entity, where the top hovering swarm entity is in relation to slower-moving ground units, such as a manned aircraft on the ground that is part of the swarm as part of a manned-unmanned team (MUMT). Alternatively, it could be a fast-moving manned aircraft in the air for a MUMT between airborne assets. In one embodiment of this disclosure, the heterogeneous swarm corresponds to a swarm of manned and unmanned autonomous aircraft (autonomous equipment) to achieve MUMT characteristics.

[0033] The target detection module 220 is configured to detect one or more targets based on a received command set, one or more determined robot capabilities, one or more captured position parameters, and one or more determined situation parameters, by using a robot management-based AI model. In one embodiment of this disclosure, one or more targets refer to an object (object), point of interest, or region of interest for performing one or more tasks. For example, the object may be an aircraft, a ship, any ground vehicle, a person, a part of the land, etc. In one embodiment of this disclosure, the robot management-based AI model may incorporate computer vision and machine learning techniques.

[0034] The task assignment module 222 is configured to assign one or more tasks and one or more detected targets among one or more autonomous robots 110 based on a received command set, one or more determined robot capabilities, one or more captured position parameters, one or more determined situation parameters, one or more target parameters, and predefined assignment information, by using a robot management-based AI model. In exemplary embodiments of this disclosure, one or more target parameters include the number of one or more targets, the size of each of the one or more targets, the sensor footprint corresponding to each of the one or more autonomous robots 110, etc. For example, when one or more autonomous robots 110 reach one or more detected targets, the one or more autonomous robots 110 collaboratively divide the one or more target area among themselves and cover the area by searching for one or more targets in order to begin foraging behavior. As an example, if the target area is 300 square feet and there are five one or more autonomous robots 110, the target area is divided into 60 square feet, and each of the one or more autonomous robots 110 acquires an area of ​​60 square feet.

[0035] The task execution module (224) is configured to perform one or more tasks on one or more detected targets based on the assignment of one or more tasks among one or more autonomous robots 110 and the detected tasks, by using a robot management-based AI model. In one embodiment of the present disclosure, one or more autonomous robots 110 perform autonomous takeoff and exhibit swarm behavior, like a flock of birds. For example, a flock of one or more autonomous robots 110 may detect one or more targets and utilize a payload, such as a camera, to address the detection in payload drops, such as humanitarian aid during an emergency scenario like an earthquake. In one embodiment of the present disclosure, the payload is dropped based on the speed and position of one or more autonomous robots 110, the position of one or more targets, wind speed, etc. For example, a heterogeneous flock of one or more autonomous robots 110 may, in sync, know how many autonomous robots are present in a certain area. Furthermore, the “area” is divided by the sensor footprint, i.e., the sensor swath, used for exploration. Furthermore, the area is divided into lengths that need to be traversed overall. Next, the “length” is divided by the number of one or more autonomous robots 110 present, thereby giving a starting point along this length line along the length segment. Furthermore, each of the one or more autonomous robots 110 of a heterogeneous group proceeds to and traverses the length segment assigned to it. In one embodiment of this disclosure, the predefined assignment information includes a set of rules for assignment. For example, the assignment is based on the concept of “seniority.” Drones with smaller Internet Protocol (IP) addresses may be assigned first.

[0036] In one embodiment of the present disclosure, the action execution module 226 is configured to determine one or more collision parameters based on one or more determined situational parameters using a robot management-based AI model. In exemplary embodiments of the present disclosure, the one or more collision parameters include the relative positions of one or more autonomous robots 110 to each other, the free space between one or more autonomous robots 110, the position, expected position and velocity of one or more autonomous robots 110, and one or more obstacles in the vicinity of one or more autonomous robots 110. Furthermore, the action execution module 226 uses a robot management-based AI model to perform one or more actions based on the determined one or more collision parameters, threshold distance, received command set and one or more determined situational parameters to prevent collisions between one or more autonomous robots 110 as they move along a navigation path. In one embodiment of the present disclosure, one or more autonomous robots 110 avoid obstacles and collisions with each other. Furthermore, obstacle information may be provided prior to the movement of the heterogeneous group or may appear during the movement of the heterogeneous group. In one embodiment of the present disclosure, one or more autonomous robots 110 cooperate and autonomously travel from a starting point to a destination along a collision-free trajectory by performing one or more actions. The collision-free trajectory simulates schooling behavior. In exemplary embodiments of the present disclosure, one or more actions include moving left, moving up, moving right, moving down, and staying still. In one embodiment of the present disclosure, an action execution module 226 further enables cohesion while providing a collision-free navigation path for each of the one or more autonomous robots 110. Collisions between the autonomous robots can be prevented through a threshold distance-based cohesive force mechanism so that each of the one or more autonomous robots 110 operates independently. In one embodiment of the present disclosure, each of the one or more autonomous robots 110 in the swarm feels an attraction to each other when the autonomous robots exceed a threshold distance, but begins to repel each other while the autonomous robots are within the threshold distance, thereby maintaining the formation of the swarm without always colliding with each other.Furthermore, the action execution module 226 enables one or more autonomous robots 110 to move cooperatively while maintaining consistency in relative position and velocity as they fly from point A to point B within a defined area. Although the heterogeneous group moves as a group, they do not collide because they hover around a single point.

[0037] The task optimization module 228 is configured to determine one or more task parameters using one or more sensors 112 and a robot management-based AI model when one or more tasks are performed on one or more detected targets. In exemplary embodiments of this disclosure, the one or more task parameters include the number of tasks performed by the autonomous robot on one or more detected targets, images, videos, and audio captured by the autonomous robot, one or more objects (targets) in the vicinity of the one or more detected targets, and the payload used. Furthermore, the task optimization module 228 broadcasts the determined one or more task parameters to each of the one or more autonomous robots 110. In embodiments of this disclosure, the determined one or more task parameters are broadcast via a communication network 108. In embodiments of this disclosure, the one or more task parameters are compressed and encrypted before broadcast to maintain data privacy. The task optimization module 228 uses robot management-based AI to determine one or more optimization parameters related to the one or more autonomous robots 110 based on one or more responses to the broadcasted one or more task parameters. In exemplary embodiments of this disclosure, one or more optimization parameters include the number of tasks performed by each of the one or more autonomous robots 110 at one or more detected targets, images, videos, and audio captured by each of the one or more autonomous robots 110, a collection of objects in the vicinity of the one or more detected targets, and payloads used by each of the one or more autonomous robots 110. Furthermore, the task optimization module 228 optimizes the received command set by using a robot management-based AI model to efficiently perform one or more tasks based on one or more determined robot capabilities, one or more captured position parameters, one or more determined situation parameters, one or more target parameters, predefined assignment information, one or more emergency commands, and one or more determined optimization parameters.In one embodiment of the present disclosure, at a specific time during a mission, one or more autonomous robots 110 within a heterogeneous group can synchronize, and autonomous robots within communication range can receive status updates from each other, and by extension, from the entire heterogeneous group.

[0038] In one embodiment of this disclosure, the synthetic simulation module 230 is configured to create a high-fidelity representation of one or more autonomous robots 110 in three-dimensional (3D) space for testing and reproducing the dynamic behavior of one or more autonomous robots or combinations thereof. In one embodiment of this disclosure, the synthetic simulation module 230 is comprised of one or more electronic devices 104 and is operably coupled to a human-machine interface via a communication network 108. In an exemplary embodiment of this disclosure, the created high-fidelity representation of one or more autonomous robots 110 in three-dimensional (3D) space corresponds to a virtual simulation environment. Furthermore, the synthetic simulation module 230 creates one or more virtual tasks and test hypotheses in the virtual simulation environment. One or more virtual tasks correspond to missions. In one embodiment of this disclosure, an operator can use the human-machine interface 102 to create one or more virtual tasks and test hypotheses in a superior simulation environment. The synthetic simulation module 230 trains a robot management-based AI model in the virtual simulation environment based on one or more simulation rules. In one embodiment of the present disclosure, an operator can simulate a swarm mission through a synthetic environment before the mission to get a feel for how the session will unfold. The operator can interface with heterogeneous swarms on a two-dimensional screen with a point-and-click interface, or through virtual reality and augmented reality interfaces for enhanced situational awareness. In embodiments of the present disclosure, such missions are prepared by the operator within the GCS. Furthermore, the synthetic simulation module 230 tests, develops, or combines human-machine interfaces (102) to train the operator. In embodiments of the present disclosure, the developed and tested human-machine interface 102 serves as a training tool for the operator.

[0039] Furthermore, the live data detection module 232 is configured to detect one or more live parameters related to the autonomous robot in real time using one or more sensors 112 and a robot management-based AI model. In exemplary embodiments of this disclosure, the one or more live parameters include health, flight mode, current routine, routine data, speed, autonomous robot battery, tasks performed by the autonomous robot, the number of autonomous robots in the vicinity of the autonomous robot, and multimedia data of the surrounding environment. The live data detection module 232 outputs the detected one or more live parameters to a human-machine interface 102 associated with one or more electronic devices 104. In one embodiment of this disclosure, the human-machine interface 102 allows an operator to command and monitor one or more autonomous robots 110 forming a heterogeneous group using high-level commands. The operator does not need to control individual autonomous robots within the heterogeneous group. In embodiments of this disclosure, top-down sensor coverage, moving map top view, and views via a ground control station are available for human-in-loop (human participation), i.e., for an operator to plan and monitor one or more tasks. In embodiments of this disclosure, a group of one or more autonomous robots 110 may communicate with the GCS via a communication network 108, but the group of one or more autonomous robots 110 may operate without having a communication network 108 with the GCS, so the communication network 108 is not essential. The operator can clearly confirm the interaction of one or more autonomous robots 110 for situational awareness. In embodiments of this disclosure, the distance between one or more autonomous robots 110 can be easily tracked.

[0040] In one embodiment of the present disclosure, an operator can provide an emergency command to one or more autonomous robots 110. For example, the emergency command may be "Abort Mission," "Return," etc. Furthermore, one or more autonomous robots 110 may perform a task corresponding to the received emergency command. In one embodiment of the present disclosure, a human operator may interface with a swarm of one or more autonomous robots 110 on a 2D screen with a point-and-click interface, or via virtual reality and augmented reality interfaces for enhanced situational awareness.

[0041] Figure 3 is an exemplary pictorial representation depicting the division of a target area among one or more autonomous robots 110 according to an embodiment of the present disclosure. In an embodiment of the present disclosure, Figure 3 depicts a top-down track of the foraging phase of a swarm mission. An operator on the loop issues one or more high-level commands, such as a target area 302 to be explored by a heterogeneous swarm of one or more autonomous robots 110. Each of the one or more autonomous robots 110 makes low-level decisions to cooperatively divide the target area 302 among themselves. For example, in Figure 3, three autonomous robots divide the target area 302 among themselves, with an equal effort division among the three autonomous robots. Furthermore, as shown in Figure 3, solid lines represent the trajectory of the first autonomous robot, dotted lines represent the trajectory of the second autonomous robot, and dashed lines represent the trajectory of the third autonomous robot.

[0042] Figure 4A is a block diagram showing an exemplary action execution module 226 according to an embodiment of the present disclosure. Furthermore, Figure 4B is a block diagram showing an exemplary operation of an AI-based computing system 106 for managing a network-independent heterogeneous robot swarm according to an embodiment of the present disclosure. For brevity, Figures 4A and 4B will be described together.

[0043] The action execution module 226 includes a detection unit 402, a positioning unit 404, and a planning unit 406, as shown in Figure 4A. In one embodiment of the present disclosure, Figure 4B depicts a workflow demonstrating network-independent management of a heterogeneous robot swarm. Furthermore, one or more sensors 112, such as one or more image acquisition units 408, radar 410, lidar 412, GPS 414, one or more sound acquisition units 416, capture the position of the autonomous robot, one or more images of one or more targets, one or more videos of one or more targets, a set of images of other nearby autonomous robots, a set of videos of other autonomous robots, one or more sounds of the surrounding environment, etc. Furthermore, one or more robot capabilities, such as payload type, sensor type, speed, weight-holding capacity, configuration, battery level, class, weight of the autonomous robot, etc., are broadcast to each of the one or more autonomous robots 110, along with one or more position parameters, in order to determine one or more situational parameters. In exemplary embodiments of this disclosure, one or more position parameters include the payload type, sensor type, speed, payload capacity, class, and battery level of each of the one or more autonomous robots 110, the position of each of the one or more autonomous robots 110, the expected position of each of the one or more autonomous robots 110, one or more images of one or more targets, one or more videos of one or more targets, a set of images of one or more autonomous robots 110, a set of videos of one or more autonomous robots 110, the relative position of each of the one or more autonomous robots 110, and one or more sounds of the surrounding environment. Furthermore, the detection unit 402 performs collision detection 418, obstacle detection 420, and free-space detection 422 to determine one or more collision parameters based on one or more determined situational parameters using a robot management-based AI model.In exemplary embodiments of this disclosure, one or more collision parameters include the relative positions of each of the one or more autonomous robots 110 to each other, the free space between each of the one or more autonomous robots 110, the position, expected position and velocity of each of the one or more autonomous robots 110, and one or more obstacles in the vicinity of each of the one or more autonomous robots 110. In one embodiment of this disclosure, a positioning unit 404 determines the relative positions of each of the one or more autonomous robots 110 to each other on a map 424. The map 424 may be a georeferenced map. Furthermore, a target detection module 220 uses a robot management-based AI model to perform a target detection 426 operation to detect one or more targets based on a received command set, one or more determined robot capabilities, one or more captured position parameters, and one or more determined situation parameters. In one embodiment of this disclosure, the detection unit 402, the target detection 426 operation, and the positioning unit 404 correspond to a perception unit 428. Furthermore, a planning unit 406 generates a routine plan 430 based on a received command set and GPS. In one embodiment of the present disclosure, a planning unit 406 performs prediction 432 and action 434, where one or more autonomous robots 110 share each other's velocity vectors, i.e., they share their current positions along with the position they are pointing to and the velocity they are moving at. Furthermore, in “non-contradiction” resolution and collision avoidance, the planning unit 406 determines the current and predicted positions of the autonomous robots in the next few steps based on the results of prediction 432 and action 434 for collision avoidance. In one embodiment of the present disclosure, one or more autonomous robots 110 collaboratively and autonomously move from a starting point to a destination via one or more collision-free trajectories 436 by performing one or more actions. One or more collision-free trajectories 436 simulate school swimming behavior. In an embodiment of the present disclosure, a control unit 438 corresponding to a task execution module 224 performs one or more tasks on one or more detected targets by using proportional-integral-derivative (PID) 440 and model predictive control (MPC) 442.In embodiments of this disclosure, PID and MPC are examples of the types of control algorithms used. Control algorithms are primarily characteristic of low-level drone control devices.

[0044] Figure 5 is a pictorial depiction of exemplary operation of an AI-based computing system 106 for managing a network-independent heterogeneous swarm of robots according to another embodiment of the present disclosure. In one embodiment of the present disclosure, Figure 5 illustrates the concept of operation using an autonomous suite. Figure 5 is a flowchart illustrating the use of a swarm mission. In step 502, the operator performs setup and surveys the target area. Furthermore, in step 504, the operator prepares the mission at a ground control station (GCS) based on a georeference map. In embodiments of the present disclosure, the mission is prepared by giving one or more high-level commands. In step 504, the mission is prepared by loading the map and adding waypoints. Furthermore, several commands are provided, such as mission briefing to equipment, takeoff commands, and transition to automatic swarm mode. In embodiments of the present disclosure, a heterogeneous swarm of one or more autonomous robots 110 transitions to a monitoring model for real-time control or command, as needed. In step 506, the emergence of stages such as swarming, training, and foraging occurs, while exhibiting behaviors such as cohesion and collision avoidance. In embodiments of the present disclosure, an autonomous flock is achieved through steering actions, and relative speed and relative position are maintained by a heterogeneous flock of one or more autonomous robots 110 while searching for a target (one or more targets) and avoiding obstacles. In embodiments of the present disclosure, the autonomous robot, such as a drone, corresponds to a COTS (Consumer Off the Shelf) flight controller equipped with custom firmware, an onboard computer (OBC), etc., as shown in Figure 5. In one embodiment of the present disclosure, the COTS is a low-level drone controller that commands the drone. In step 508, the heterogeneous flock may communicate with the GCS via a communication network 108, but the communication network 108 is not required, as the heterogeneous flock may operate without a network connection to the GCS.In one embodiment of the present disclosure, a group of one or more autonomous robots 110 may use image acquisition, computer vision, and machine learning-based classification to search for and detect one or more targets. In step 510, the heterogeneous group may utilize a payload such as a camera to detect targets by employing computer vision and machine learning techniques, and detection is addressed by payload drop. For example, humanitarian aid supplies are dropped in an emergency such as an earthquake. Once the payload is dropped, the heterogeneous group returns to its home location.

[0045] Figure 6 is a process flow diagram illustrating an exemplary AI-based method for managing a network-independent heterogeneous robot swarm according to an embodiment of the present disclosure. In an embodiment of the present disclosure, the AI-based method 600 is performed by one or more autonomous robots 110. The one or more autonomous robots 110 collectively form a heterogeneous swarm and collaboratively perform one or more tasks. In step 602, a set of commands is received from a human-machine interface 102 associated with one or more electronic devices 104 for performing one or more tasks on one or more targets via the one or more autonomous robots 110. The human-machine interface 102 is configured on one or more electronic devices 104. In one embodiment of the present disclosure, an operator uses the human-machine interface 102 to provide one or more high-level commands to enable the one or more autonomous robots 110 to perform one or more tasks. In one embodiment of the present disclosure, the one or more high-level commands are prepared at a ground control station (GCS) based on a georeference map. The one or more high-level commands enable the one or more autonomous robots 110 to perform one or more tasks as a swarm. In one embodiment of the present disclosure, one or more autonomous robots 110 collectively form a heterogeneous swarm and collaboratively perform one or more tasks. One or more high-level commands are issued to the heterogeneous swarm by a human to cause each of the one or more autonomous robots 110 to act collaboratively to perform lower-level tasks in order to collaboratively accomplish the one or more high-level commands. The one or more tasks are performed simultaneously or in stages. For example, one or more tasks include rescuing survivors, dropping medical kits, dropping food boxes, destroying targets, and capturing images. In one embodiment of the present disclosure, a human-machine interface 102 also derives a set of commands from one or more high-level commands provided by an operator to perform one or more tasks via one or more autonomous robots 110. The human-machine interface 102 is deployed at the full swarm level.A command set is a set of machine-readable commands that enable one or more autonomous robots 110 to collaboratively perform one or more tasks. In embodiments of the Disclosure, the command set corresponds to a mission. In exemplary embodiments of the Disclosure, one or more high-level commands include a set of predefined commands (a set of commands) representing one or more tasks to be performed by one or more autonomous robots 110, a starting point, a destination, the time to perform one or more tasks, the respective roles and speeds of one or more autonomous machines (autonomous devices), altitude from the ground, one or more predefined conditions, a navigation route from the starting point to the destination, boundary fences, one or more intermediate positions, one or more targets, safe areas and dangerous areas. In one embodiment of the Disclosure, one or more high-level commands are provided on a georeference map. For example, a set of predefined commands could include armament, package delivery, swarming, foraging, landing, moving to a location, training, takeoff, Global Positioning System (GPS) standby, standby, survivor detection, payload drop, etc. In one embodiment of the Disclosure, the navigation route is provided by the operator. In another embodiment of the Disclosure, the navigation route is generated by the human-machine interface 102 based on the origin, destination, and GPS. In an exemplary embodiment of the Disclosure, one or more electronic devices 104 may include a laptop computer, desktop computer, tablet computer, smartphone, wearable device, smartwatch, digital camera, etc. For example, the wearable device may be an augmented reality headset, a virtual reality headset, etc. In one embodiment of the Disclosure, the human-machine interface 102 communicates with an AI-based computing system 106 via a communication network 108.

[0046] In one embodiment of the present disclosure, each of one or more autonomous robots 110 of a heterogeneous swarm knows the current state of the swarm while moving from a starting point to a destination. The heterogeneous swarm transitions to a new state, which cascades into swarming entities (groups) that move to other roles when one or more predefined conditions are met, such as a sufficient number reaching the destination. In embodiments of the present disclosure, one or more predefined conditions are parameterized and configured before the mission. In embodiments of the present disclosure, each of the one or more autonomous robots 110 periodically synchronizes its position with one another in real time so that it can behave as a heterogeneous swarm.

[0047] In step 604, one or more robot capabilities associated with the autonomous robot are determined based on the received command set and predefined robot information. In exemplary embodiments of the disclosure, one or more robot capabilities include payload type, sensor type, speed, weight-holding capacity, configuration, battery level, class, and weight of the autonomous robot. For example, the class of the autonomous robot may be a 20kg hexacopter with a 10kg quadcopter, Beluga (BLL), Nimbus (NMB), Mackerall (MCL), drone with camera fitting, quadcopter, hexacopter, and ground drone with different working speeds. In exemplary embodiments of the disclosure, payload types may be a camera payload, robot, medical kit, communicable antenna payload, weapon, etc. In embodiments of the disclosure, each of the one or more autonomous robots 110 can accommodate other autonomous robots depending on the time and speed.

[0048] In step 606, one or more position parameters are captured using one or more sensors 112. In exemplary embodiments of the Disclosure, the one or more position parameters include the position of the autonomous robot, the expected position of the autonomous robot, one or more images of one or more targets, one or more videos of one or more targets, a set of images of other nearby autonomous robots, a set of videos of other autonomous robots, one or more sounds of the surrounding environment, etc. In one embodiment of the Disclosure, one or more sensors 112 are fixed to each of one or more autonomous robots 110 to capture one or more position parameters. For example, the one or more sensors 112 include one or more image acquisition units, one or more sound acquisition units, a Global Positioning System (GPS), LiDAR, radar, etc.

[0049] In step 608, the determined robot capabilities and the captured position parameters are broadcast to each of the one or more autonomous robots 110. In embodiments of the present disclosure, the determined robot capabilities and the captured position parameters are broadcast via a communication network 108. In one embodiment of the present disclosure, the determined robot capabilities and the captured position parameters are compressed and encrypted before broadcast in order to maintain data privacy.

[0050] In step 610, one or more situational parameters relating to one or more autonomous robots 110 are determined by using an artificial intelligence (AI) model based on robot management, based on one or more broadcasted robot capabilities and one or more broadcasted positional parameters, a set of received commands, one or more determined robot capabilities and one or more captured positional parameters, and one or more responses. In exemplary embodiments of the Disclosure, one or more situational parameters include the payload type, sensor type, speed, payload capacity, class, and battery level of each of the one or more autonomous robots 110, the position of each of the one or more autonomous robots 110, the expected position of each of the one or more autonomous robots 110, one or more images of one or more targets, one or more videos of one or more targets, a set of images of one or more autonomous robots 110, a set of videos of one or more autonomous robots 110, the relative position of the autonomous robots to each of the one or more autonomous robots 110, and one or more sounds of the surrounding environment. Since one or more autonomous robots 110 each have different payload types, sensor types, speeds, weight-holding capabilities, configurations, classes, weights, etc., one or more autonomous robots 110 form a heterogeneous group. For example, even if one or more autonomous robots 110 are of the same class but have different payload types associated with them, one or more autonomous robots 110 can still form a heterogeneous group.

[0051] In embodiments of this disclosure, heterogeneity in the form of different types of payloads may mean that one or more autonomous robots 110 may perform a specific role, such as communication payload equipment communication, and not perform search requiring a camera system. Furthermore, an autonomous robot equipped with a camera may search for targets and share them with a swarm of one or more autonomous robots 110. For example, an autonomous robot carrying a mission payload may service one or more targets. In one embodiment of this disclosure, different autonomous robots are assigned roles based on their configuration and perform those roles accordingly. For example, in a swarm of bees, there are worker bees, warrior bees, etc., each performing different roles. Furthermore, classes of autonomous robots may be similar, such as quadcopters and hexacopters, meaning that at least their speeds are similar. Thus, while their speed envelopes may be similar, their flight times may differ. In another example, a hexacopter with a large integrated circuit (IC) engine acts as a mothership carrying smaller battery-powered multicopters as payloads, with smaller quadcopters then deploying to cascade heterogeneous swarms. Furthermore, cross-class swarms, such as class mix-and-match (e.g., fixed-wing + multicopters), are evolving where certain aspects, such as collision avoidance, still occur, and faster units may try to accommodate smaller units. In exemplary embodiments of this disclosure, this can also be done with ground vehicles as swarm entities, where the top hovering swarm entity remains in relation to slower-moving ground units, such as a manned vehicle on the ground that is part of the swarm as part of a manned-unmanned team (MUMT). Alternatively, it could be a fast-moving manned aircraft in the air for a MUMT between aerial assets. In one embodiment of this disclosure, the heterogeneous swarm corresponds to a swarm of manned and unmanned autonomous aircraft to achieve MUMT characteristics.

[0052] In step 612, by using a robot management-based AI model, one or more targets are detected based on the received command set, one or more determined robot capabilities, one or more captured position parameters, and one or more determined situation parameters. In one embodiment of the disclosure, one or more targets refer to an object (object), point of interest, or region of interest for performing one or more tasks. For example, the object may be an aircraft, a ship, any ground vehicle, a person, a part of the land, etc. In one embodiment of the disclosure, the robot management-based AI model may incorporate computer vision and machine learning techniques.

[0053] In step 614, one or more tasks and one or more detected targets are assigned among one or more autonomous robots 110 by using a robot management-based AI model, based on the received command set, one or more determined robot capabilities, one or more captured position parameters, one or more determined situation parameters, one or more target parameters, and predefined assignment information. In exemplary embodiments of this disclosure, one or more target parameters include the number of one or more targets, the size of each of the one or more targets, the sensor footprint corresponding to each of the one or more autonomous robots 110, and so on. For example, when one or more autonomous robots 110 reach one or more detected targets, the one or more autonomous robots 110 collaboratively divide the one or more target area among themselves and cover the area by searching for one or more targets in order to begin foraging behavior. For example, if the target area is 300 square feet and there are five one or more autonomous robots 110, the target area is divided into 60 square feet, and one or more autonomous robots 110 each acquire an area of ​​60 square feet.

[0054] In step 616, one or more tasks on one or more detected targets are performed based on the assignment of one or more tasks between one or more autonomous robots 110 and the detected one or more tasks, using a robot management-based AI model. In one embodiment of the present disclosure, one or more autonomous robots 110 perform autonomous takeoff and exhibit swarm behavior, like a flock of birds. For example, a swarm of one or more autonomous robots 110 can detect one or more targets and utilize a payload, such as a camera, to address the detection in payload drops, such as humanitarian aid during an emergency scenario like an earthquake. In one embodiment of the present disclosure, the payload is dropped based on the speed and position of one or more autonomous robots 110, the position of one or more targets, wind speed, etc. For example, a heterogeneous swarm of one or more autonomous robots 110 know how many autonomous robots are present in a certain area, while synchronously. Furthermore, the “area” is divided by the sensor footprint, i.e., the sensor swath, used for exploration. Furthermore, the area is divided into lengths that need to be traversed overall. Next, the “length” is divided by the number of autonomous robots 110 present, thereby giving a starting point along this length line along the length segment. Furthermore, each of the heterogeneous group of autonomous robots 110 proceeds to and traverses the length segment assigned to it. In one embodiment of this disclosure, the predefined assignment information includes a set of rules for assignment. For example, the assignment is based on the concept of “seniority.” Drones with smaller Internet Protocol (IP) addresses may be assigned first.

[0055] Furthermore, Method 600 includes determining one or more collision parameters based on one or more determined situational parameters using a robot management-based AI model. In exemplary embodiments of the Disclosure, the one or more collision parameters include the relative positions of each of the one or more autonomous robots 110 to each other, the free space between each of the one or more autonomous robots 110, the position, expected position and velocity of each of the one or more autonomous robots 110, and one or more obstacles in the vicinity of each of the one or more autonomous robots 110. Furthermore, Method 600 includes performing one or more actions based on the determined one or more collision parameters, threshold distance, received command set and one or more determined situational parameters using a robot management-based AI model to prevent collisions between the one or more autonomous robots 110 as they move along a navigation path. In one embodiment of the Disclosure, the one or more autonomous robots 110 avoid obstacles and collisions with each other. Furthermore, obstacle information may be provided prior to the movement of the heterogeneous group or may appear during the movement of the heterogeneous group. In one embodiment of the present disclosure, one or more autonomous robots 110 cooperate and autonomously travel from a starting point to a destination along a collision-free trajectory by performing one or more actions. The collision-free trajectory simulates schooling behavior. In exemplary embodiments of the present disclosure, one or more actions include moving left, moving up, moving right, moving down, staying still, etc. In one embodiment of the present disclosure, method 600 enables cohesion while providing a collision-free navigation path for each of the one or more autonomous robots 110. Since each of the one or more autonomous robots 110 is operated independently, collisions between the autonomous robots can be prevented through a threshold distance-based cohesive force mechanism. In one embodiment of the present disclosure, each of the one or more autonomous robots 110 in the flock feels an attraction to each other when the autonomous robots exceed a threshold distance, but begins to repel each other while the autonomous robots are within the threshold distance, thereby maintaining the formation of the flock without ever colliding with each other. Furthermore, Method 600 includes enabling one or more autonomous robots 110 to move cooperatively while maintaining a matching of relative position and velocity as they fly from point A to point B within a defined area.The different groups move as a group, but they hover around the point and therefore do not collide.

[0056] Furthermore, Method 600 includes determining one or more task parameters using one or more sensors 112 and a robot management-based AI model when performing one or more tasks on one or more detected targets. In exemplary embodiments of the Disclosure, the one or more task parameters include the number of tasks performed by the autonomous robot on one or more detected targets, images, videos, and audio captured by the autonomous robot, one or more objects (targets) in the vicinity of the one or more detected targets, and the payload used. Furthermore, Method 600 includes broadcasting the determined one or more task parameters to each of the one or more autonomous robots 110. In one embodiment of the Disclosure, the determined one or more task parameters are broadcast via a communication network 108. In embodiments of the Disclosure, the one or more task parameters are compressed and encrypted before being broadcast to maintain data privacy. Method 600 includes determining one or more optimization parameters related to the one or more autonomous robots 110 based on one or more responses to the broadcasted one or more task parameters using robot management-based AI. In exemplary embodiments of this disclosure, one or more optimization parameters include the number of tasks performed by each of the one or more autonomous robots 110 on one or more detected targets, images, videos and audio captured by each of the one or more autonomous robots 110, a set of objects in the vicinity of the one or more detected targets, and payloads used by each of the one or more autonomous robots 110. Furthermore, Method 600 includes optimizing a set of received commands using a robot management-based AI model to efficiently perform one or more tasks based on one or more determined robot capabilities, one or more captured position parameters, one or more determined situation parameters, one or more target parameters, predefined assignment information, one or more emergency commands, and one or more determined optimization parameters.In one embodiment of the present disclosure, at a specific time during a mission, one or more autonomous robots 110 within a heterogeneous group can synchronize, and autonomous robots within communication range can receive status updates from each other, and by extension, from the entire heterogeneous group.

[0057] In one embodiment of the present disclosure, Method 600 includes creating a high-fidelity representation of one or more autonomous robots 110 in a three-dimensional (3D) space to test and reproduce the dynamic behavior of one or more autonomous robots or combinations thereof. In an exemplary embodiment of the present disclosure, the created high-fidelity representation of one or more autonomous robots 110 in a three-dimensional (3D) space corresponds to a virtual simulation environment. Furthermore, Method 600 includes creating one or more virtual tasks and test hypotheses in the virtual simulation environment. One or more virtual tasks correspond to missions. In one embodiment of the present disclosure, an operator may use a human-machine interface 102 to create one or more virtual tasks and test hypotheses in a superior simulation environment. Method 600 includes training a robot management-based AI model in the virtual simulation environment based on one or more simulation rules. In one embodiment of the present disclosure, an operator may simulate a swarm mission through a synthetic environment before the mission to get a feel for how the session might unfold. The operator can interface with heterogeneous groups on a two-dimensional screen with a point-and-click interface, or via virtual reality and augmented reality interfaces for enhanced situational awareness. In embodiments of the present disclosure, such missions are prepared by the operator within the GCS. Furthermore, the human-machine interface 102 is tested, developed, or a combination thereof for training the operator. In embodiments of the present disclosure, the developed and tested human-machine interface 102 serves as a training tool for the operator.

[0058] Furthermore, Method 600 includes detecting one or more live parameters related to the autonomous robot in real time using one or more sensors 112 and a robot management-based AI model. In exemplary embodiments of the Disclosure, the one or more live parameters include health status, flight mode, current routine, routine data, speed, autonomous robot battery, tasks performed by the autonomous robot, the number of autonomous robots in the vicinity of the autonomous robot, and multimedia data of the surrounding environment. Method 600 includes outputting the detected one or more live parameters to a human-machine interface 102 associated with one or more electronic devices 104. In one embodiment of the Disclosure, the human-machine interface 102 allows an operator to command and monitor one or more autonomous robots 110 forming a heterogeneous group using high-level commands. The operator does not need to control individual autonomous robots within the heterogeneous group. In embodiments of the Disclosure, top-down sensor coverage, moving map top view, and views via a ground control station are available for human-in-loop (human participation), i.e., for an operator to plan and monitor one or more tasks. In embodiments of this disclosure, a group of one or more autonomous robots 110 may communicate with the GCS via a communication network 108, but the group of one or more autonomous robots 110 may operate without having a communication network 108 with the GCS, so the communication network 108 is not essential. The operator can clearly confirm the interaction of one or more autonomous robots 110 for situational awareness. In embodiments of this disclosure, the distance between one or more autonomous robots 110 can be easily tracked.

[0059] In one embodiment of the present disclosure, an operator can provide an emergency command to one or more autonomous robots 110. For example, the emergency command may be "Abort Mission," "Return," etc. Furthermore, one or more autonomous robots 110 may perform a task corresponding to the received emergency command. In one embodiment of the present disclosure, a human operator may interface with a swarm of one or more autonomous robots 110 on a 2D screen with a point-and-click interface, or via virtual reality and augmented reality interfaces for enhanced situational awareness.

[0060] The AI-based method 600 can be implemented with any suitable hardware, software, firmware, or a combination thereof.

[0061] Figures 7A–7C are graphical user interface screens of an AI-based computing system 106 for managing a network-independent heterogeneous swarm of robots, according to one embodiment of the present disclosure. Figures 7A–7B display a georeference map to which one or more high-level commands are provided by the operator, one or more sample routines on the left, one or more live parameters on the right, one or more emergency controls, and a table view at the bottom showing individual drone patterns, etc. Figure 7C displays an image showing complete situational awareness by multiple autonomous robots evolving over time. In embodiments of the present disclosure, Figure 7C shows dynamic evolution as one or more autonomous robots 110 move to different locations in different time instances. In embodiments of the present disclosure, the moving swarm is heterogeneous. For example, different drones such as Belluga (BLL), Nimbus (NMB), and Mackeral (MCL). In another example, the swarm is heterogeneous if the same platform is configured for different roles by attaching different payloads.

[0062] Thus, various embodiments of the AI-based computing system 106 provide a solution for managing network-independent heterogeneous robot swarms. The AI-based computing system 106 can be used in multiple application areas such as humanitarian assistance and disaster relief, intelligence, surveillance, reconnaissance, agriculture, logistics, and security. The AI-based computing system 106 enables heterogeneous unmanned systems of different classes of unmanned aerial vehicles (UAVs) to work autonomously and collaboratively as a single swarm for one or more tasks, simultaneously or in stages. In one embodiment of the present disclosure, the AI-based computing system 106 is deployed at the level of a specific individual class of UAV for interface with a single swarm system, while a human-machine interface 102 is deployed with the entire swarm of unmanned vehicles. The AI-based computing system 106 enables a human to issue high-level commands to the heterogeneous swarm as a whole, while one or more autonomous robots 110 act to perform lower-level tasks in order to collaboratively achieve the high-level commands. Furthermore, the synthetic simulation module 230 abstracts the AI-based computing system 106 and the human-machine interface 102 from each other, verifying hypotheses and missions through simulation in a synthetic environment. The AI-based computing system 106 enables swarm behavior through one or more autonomous robots 110 without a dominant master making decisions, i.e., a human or unmanned agent. The AI-based computing system 106 allows a human in the loop (participating human) to understand the meaning of the heterogeneous swarm exhibiting these behaviors and issue high-level commands, i.e., without controlling individual autonomous robots. Moreover, the AI-based solution allows the operator to mix heterogeneous swarms of one or more autonomous robots 110, i.e., autonomous robots with different configurations and compositions, in a collaborative swarm.For example, a heterogeneous swarm may be a mixture of unmanned aerial vehicles and unmanned ground vehicles (UAVs + UGVs), or it may be different classes of UAVs (for example, a 20kg hexacopter and a 10kg quadcopter), or it may be different configurations, e.g., UAVs with different payloads (e.g., one UAV with a camera payload versus one UAV with a communicable antenna payload). In one embodiment of this disclosure, swarming is relatively easy when a network via GCS is enabled. However, it is highly dependent on the master node and the communication is robust. The AI-based computing system 106 may operate with the link to the ground control station completely disconnected (and with humans / GCS completely removed from the loop), and may operate in a communication-deny mode or degraded mode. Furthermore, the AI-based computing system 106 may be deployed in each of one or more autonomous robots 110 and drive them into biologically inspired, replicable swarm behavior that takes into account other agents in the swarm. In a heterogeneous flock, all autonomous robots may have the same configuration. Furthermore, the human-machine interface 102 allows the operator to command and monitor the flock mission using high-level commands, eliminating the need for the operator to control individual autonomous robots within the flock. In one embodiment of this disclosure, a heterogeneous flock of one or more autonomous robots 110 performs autonomous takeoff and exhibits cohesive flock behavior, similar to a flock of birds. When the heterogeneous flock of one or more autonomous robots 110 reaches a designated area, the one or more autonomous robots 110 cooperate to divide the target area among themselves and begin foraging behavior, searching for targets and covering the area.

[0063] Furthermore, the AI-based computing system 106 enables an operator to engage with a heterogeneous swarm and issue high-level commands to the swarm in a spatial and temporal context. The heterogeneous swarm mimics the behavior of biologically and robotically inspired swarms, for example, because there is power in numbers, the elements of the swarm tend to stick together. However, this leads to the problem of collisions between swarm entities, so the AI-based computing system 106 enables the characteristic of cohesive together while performing collision avoidance within the swarm. In addition, the swarming solution enables unmanned agents to move cooperatively while maintaining consistency in relative position and velocity. In one embodiment of this disclosure, the heterogeneous swarm is also capable of obstacle avoidance, where obstacle information may be provided before the swarm moves or may appear during movement. The heterogeneous swarm can explore a given region by employing biologically and robotically inspired search optimization techniques, and by leveraging search by multiple autonomous robots that share information with each other, an optimal search can be achieved. In one embodiment of the present disclosure, a human operator does not need to operate a heterogeneous swarm involved beyond the initial mission description, but the human operator can choose to be in the loop and give dynamic instructions to modify the swarm's behavior during the swarm session. Thus, a swarm of one or more autonomous robots 110 does not require a ground control station for constant monitoring. Furthermore, the human operator can simulate the swarm's mission through a synthetic environment prior to the mission and gain a feel for how the session may unfold. In one embodiment of the present disclosure, the human operator can replay the swarm session unfolded over space and time by using one or more electronic devices 104. Furthermore, the human operator can interface with the swarm on a 2D screen with a point-and-click interface, or via virtual reality and augmented reality interfaces for enhanced contextual awareness.

[0064] In one embodiment of this disclosure, each of one or more autonomous robots 110 can operate independently of network connectivity after the initial command input. An AI-based computing system 106 is configured in each of the one or more autonomous robots 110, thereby enabling a heterogeneous swarm to act as a single entity. The AI-based computing system 106 enables controlled swarm behavior of one or more autonomous robots 110 in a predefined target environment. In such embodiments, to achieve swarm behavior, the AI-based computing system 106 issues a set of commands to each of the one or more autonomous robots 110 based on high-level commands received from a command center via a communication network 108. It should be noted that the high-level commands are for the heterogeneous swarm, not for controlling individual autonomous robots. The AI-based computing system 106 can also break down a set of activities associated with the high-level commands and assign specific activities to each of the one or more autonomous robots 110 based on their functional capabilities. Thus, upon receiving high-level commands, the heterogeneous swarm of robots can function and complete assigned tasks even without the communication network 108 from the command center. Therefore, in the context of the present invention, the term "network-independent" is used. This feature greatly expands the range and breadth of operations that heterogeneous groups can perform. In embodiments of the present disclosure, the AI-based computing system 106 enables one or more different classes of autonomous robots 110 to work autonomously and collaboratively as a single group on one or more tasks simultaneously or in stages. In such embodiments, the autonomous and collaborative work enables an operator to understand the behavior of the group. In one embodiment of the present disclosure, the AI-based computing system 106 enables the autonomous takeoff of one or more unmanned aerial vehicles (20), the swarm behavior of one or more autonomous robots 110, and so on.For example, a command set enables one or more autonomous robots 110 to autonomously take off and divide a target environment among themselves without being instructed from a master machine or command center. The command set may also include path obstacle information. A human-machine interface is configured in one or more electronic devices 104 and operably coupled to an AI-based computing system 106 via a communication network 108. The human-machine interface enables an operator to actuate and monitor the swarm behavior of one or more autonomous robots 110 via one or more electronic devices 104. In one embodiment of this disclosure, an operator can control the entire assembly of one or more autonomous robots 110 through high-level commands.

[0065] This specification describes the subject matter in such a way that any person skilled in the art can manufacture and use embodiments. The scope of embodiments of the subject matter is defined by the claims and may include other modifications that would be made possible to a person skilled in the art. Such other modifications are intended to be included in the claims if they have similar elements that are not different from the language of the claims, or if they contain equivalent elements that are not substantially different from the language of the claims. Embodiments of this specification may consist of hardware and software elements. Software-implemented embodiments include, but are not limited to, firmware, resident software, and microcode. Functions performed by the various modules described herein may be implemented by other modules or combinations of modules. In this specification, computer-usable medium or computer-readable medium may be any device capable of configuring, storing, communicating, propagating, or transporting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0066] The medium can be electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device), or a propagation medium. Examples of computer-readable media include semiconductor or solid-state memory, magnetic tape, removable computer diskettes, random-access memory (RAM), read-only memory (ROM), hard magnetic disks, and optical disks. Current examples of optical disks include compact disc read-only memory (CD-ROM), compact disc read / write (CD-R / W), and DVD. Input / output (I / O) devices (including, but not limited to, keyboards, displays, and pointing devices) can be coupled to a system directly or via an intermediary I / O controller. Network adapters can also be coupled to a system to enable data processing systems to connect with other data processing systems or remote printers or storage devices via an intermediary private or public network. Modems, cable modems, and Ethernet cards are just a few of the types of network adapters currently available.

[0067] A typical hardware environment for implementing this embodiment may include a hardware configuration of an information handling / computer system according to the embodiments herein. The system herein comprises at least one processor or central processing unit (CPU). The CPU is interconnected via a system bus 208 to various devices such as random access memory (RAM), read-only memory (ROM), and input / output (I / O) adapters. The I / O adapters can be connected to peripheral devices such as disk units and tape drives, or to other program storage devices readable by the system. The system can read instructions of the present invention from the program storage device and execute the methodology of the embodiments herein in accordance with these instructions.

[0068] The system further includes a user interface adapter that connects other user interface devices such as a keyboard, mouse, speaker, microphone, and / or touchscreen device (not shown) to the bus to collect user input. Furthermore, a communication adapter connects the bus to a data processing network, and a display adapter connects the bus to a display device which can be embodied as an output device such as a monitor, printer, or transmitter.

[0069] The description of embodiments having multiple components that communicate with each other does not mean that all such components are necessary. Rather, various arbitrary components are described in order to illustrate the wide variety of possible embodiments of the present invention. Where a single device or article is described herein, it will be obvious that multiple devices / articles (whether they cooperate or not) may be used in place of the single device / article. Similarly, where two or more devices or articles (whether they cooperate or not) are described herein, it will be obvious that a single device / article may be used in place of the two or more devices or articles, or that a different number of devices / articles may be used in place of the number of devices or programs indicated. The functionality and / or features of a device may be embodied by one or more other devices not expressly described as having such functionality / features. Therefore, other embodiments of the present invention do not need to include the device itself.

[0070] The illustrated steps are set up to illustrate the illustrated exemplary embodiments, and it should be anticipated that the way in which certain functions are performed may change due to ongoing technological development. These examples are presented herein for illustrative purposes only and are not limiting. Furthermore, the boundaries of functional building blocks are arbitrarily defined herein for the convenience of explanation. Alternative boundaries can be defined as long as the specified functions and their relationships are adequately performed. Alternatives (including equivalents, extensions, variations, deviations, etc., of those described herein) will be apparent to those skilled in the art based on the teachings contained herein. Such alternatives are included in the scope and spirit of the disclosed embodiments. Also, the words “comprising,” “having,” “containing,” and “including,” as well as other similar forms, are intended to be semantically equivalent and open-ended in that the items or items following any one of these words do not mean that the items or items following any one of these words are an exhaustive list of such items or items, nor that they are limited to the items or items listed. Furthermore, it should be noted that, as used herein and in the appended claims, the singular forms "a," "an," and "the" include plural references unless the context clearly indicates otherwise.

[0071] Finally, the language used herein has been selected primarily for readability and explanatory purposes, and may not have been selected to define or encompass the inventive subject matter. Therefore, the scope of the invention is intended to be limited not by this detailed description, but rather by the claims to be issued based on the application herein. Accordingly, the embodiments of the invention are intended to illustrate, not limit, the scope of the invention as defined in the following claims.

Claims

1. An artificial intelligence (AI) based computing system (106) for managing a group of heterogeneous robots, wherein the computing system comprises: One or more hardware processors, The system comprises a memory (204) coupled to one or more hardware processors (202), wherein the memory (204) comprises a plurality of modules (114) in the form of programmable instructions executable by the one or more hardware processors (202), The aforementioned plurality of modules (114) A human-machine interface (102) associated with one or more electronic devices (104) is configured to receive a set of commands for performing one or more tasks via one or more autonomous robots (110) at one or more targets, wherein the set of commands is derived from one or more high-level commands provided by an operator for performing one or more tasks via one or more autonomous robots (110), and a data receiving module (210) is configured to receive a set of commands for performing one or more tasks via one or more autonomous robots (110). Based on the received command set and predefined robot information, the capability determination module (212) is configured to determine one or more robot capabilities related to the autonomous robot, where one or more robot capabilities include payload type, sensor type, speed, weight holding capacity, configuration, battery level, class, and weight of the autonomous robot. A parameter acquisition module (214) is configured to acquire one or more position parameters using one or more sensors (112), where the one or more position parameters include the position of the autonomous robot, the expected position of the autonomous robot, one or more images of one or more targets, one or more videos of one or more targets, a set of images of other nearby autonomous robots, a set of videos of other autonomous robots, and one or more sounds of the surrounding environment. Each of the one or more autonomous robots (110) is configured to broadcast one or more determined robot capabilities and one or more captured position parameters, A parameter determination module (218) is configured to determine one or more situational parameters related to one or more autonomous robots (110) based on one or more broadcasted robot capabilities and one or more broadcasted positional parameters, a set of received commands, one or more determined robot capabilities, and one or more responses to one or more captured positional parameters, wherein the one or more situational parameters include payload type, sensor type, speed, payload weight, class, battery level of each of the one or more autonomous robots (110), position of each of the one or more autonomous robots (110), expected position of each of the one or more autonomous robots (110), one or more images of one or more targets, one or more videos of one or more targets, a set of images of one or more autonomous robots (110), a set of videos of one or more autonomous robots (110), the relative position of each of the one or more autonomous robots (110), and one or more sounds of the surrounding environment, by using a robot management-based artificial intelligence (AI) model. A target detection module (220) is configured to detect one or more targets based on a received command set, one or more determined robot capabilities, one or more captured position parameters, and one or more determined situation parameters, using a robot management-based AI model. By using a robot management-based AI model, the system is configured to assign one or more tasks and one or more detected targets among one or more autonomous robots (110) based on a received command set, one or more determined robot capabilities, one or more captured position parameters, one or more determined situation parameters, one or more target parameters, and predefined assignment information, wherein the one or more target parameters have the number of one or more targets, the size of each of the one or more targets, and the sensor footprint corresponding to each of the one or more autonomous robots (110), and the task assignment module (222) A task execution module (224) configured to perform one or more tasks on one or more detected targets based on the assignment of one or more tasks between one or more autonomous robots (110) and the detected task assignments, using a robot management-based AI model, Equipped with, Each of the one or more autonomous robots (110) is configured to perform one or more tasks after receiving a command set without requiring a network connection to a human-machine interface (102). Computing system.

2. The AI-based computing system (106) according to claim 1, wherein one or more high-level commands have a predefined set of commands representing one or more tasks performed by one or more autonomous robots (110), a starting point, a destination, the time to perform one or more tasks, the respective roles and speeds of one or more autonomous machines, altitude from the ground, one or more predefined conditions, a navigation route from the starting point to the destination, boundary fences, one or more intermediate positions, one or more targets, safe areas and dangerous areas, wherein one or more high-level commands are provided on a georeference map.

3. By using a robot management-based AI model, one or more collision parameters are determined based on one or more determined situational parameters, where the one or more collision parameters include the relative positions of one or more autonomous robots (110) to each other, the free space between one or more autonomous robots (110), the position, expected position and velocity of one or more autonomous robots (110), and one or more obstacles in the vicinity of one or more autonomous robots (110). Based on one or more determined collision parameters, threshold distance, received command set, and one or more determined situation parameters, a robot management-based AI model is used to perform one or more actions to prevent collisions of one or more autonomous robots (110) moving along a navigation path, wherein one or more actions include moving left, moving up, moving right, moving down, and stopping. The AI-based computing system (106) according to claim 1 further comprises an action execution module (226) configured as follows.

4. When performing one or more tasks on one or more detected targets, one or more task parameters are determined by using one or more sensors (112) and a robot management-based AI model, wherein the one or more task parameters include the number of tasks performed by the autonomous robot on one or more detected targets, images, videos and audio captured by the autonomous robot, one or more objects in the vicinity of the one or more detected targets, and the payload used. The determined task parameters are broadcast to each of the one or more autonomous robots (110). By using a robot management-based AI, one or more optimization parameters associated with one or more autonomous robots (110) are determined based on one or more responses to one or more broadcasted task parameters, wherein the one or more optimization parameters include the number of tasks performed by each of the one or more autonomous robots (110) on one or more detected targets, images, videos and audio captured by each of the one or more autonomous robots (110), a set of objects in the vicinity of the one or more detected targets, and a payload used by each of the one or more autonomous robots (110). Based on one or more determined robot capabilities, one or more captured position parameters, one or more determined situation parameters, one or more target parameters, predefined assignment information, one or more emergency commands, and one or more determined optimization parameters, a robot management-based AI model is used to optimize the received command set to efficiently perform one or more tasks. The AI-based computing system (106) according to claim 1 further comprises a task optimization module (228) configured as follows.

5. For at least one of testing and reproducing the dynamic behavior of one or more autonomous robots, a high-fidelity representation of one or more autonomous robots (110) in three-dimensional (3D) space is created, wherein the created high-fidelity representation of one or more autonomous robots (110) in three-dimensional (3D) space corresponds to a virtual simulation environment. Create one or more virtual tasks and test hypotheses in a virtual simulation environment. Based on one or more simulation rules, train a robot management-based AI model in a virtual simulation environment. To perform at least one of the testing and development of a human-machine interface (102) for training operators. The AI-based computing system (106) according to claim 1 further comprises a synthetic simulation module (230) configured as follows.

6. The AI-based computing system (106) is characterized in that it corresponds to one autonomous robot (110) from one or more autonomous robots (110), the one or more autonomous robots (110) collectively form a heterogeneous group that collaboratively perform one or more tasks, and the heterogeneous group corresponds to a group of manned and unmanned autonomous machines to achieve manned-unmanned team (MUMT) characteristics.

7. By using one or more sensors (112) and a robot management-based AI model, one or more live parameters related to the autonomous robot are detected in real time, where one or more live parameters include health, flight mode, current routine, routine data, speed, autonomous robot battery, task performed by the autonomous robot, number of autonomous robots in the vicinity of the autonomous robot, and multimedia data of the surrounding environment. The detected live parameters are output to a human-machine interface (102) associated with one or more electronic devices (104). The AI-based computing system (106) according to claim 1 further comprises a live data detection module (232) configured as follows.

8. The AI-based computing system (106) according to claim 1, wherein one or more electronic devices (104) include a laptop computer, a desktop computer, a tablet computer, a smartphone, a wearable device, a smartwatch, and a digital camera, the wearable device includes an augmented reality headset and a virtual reality headset, the augmented reality headset and the virtual reality headset enable interfacing with heterogeneous groups, and the operator uses one or more electronic devices (104) to replay a group session unfolded over space and time.

9. An artificial intelligence (AI) based method for managing a group of heterogeneous robots, The steps include: receiving a set of commands from a human-machine interface (102) associated with one or more electronic devices (104) via one or more hardware processors (202) for performing one or more tasks via one or more autonomous robots (110) at one or more targets, wherein the set of commands is derived from one or more high-level commands provided by an operator for performing one or more tasks via one or more autonomous robots (110); A step of determining one or more robot capabilities associated with an autonomous robot based on a set of received commands and predefined robot information by one or more hardware processors (202), wherein the one or more robot capabilities include payload type, sensor type, speed, weight holding capacity, configuration, battery level, class, and weight of the autonomous robot. The steps included capturing one or more position parameters using one or more sensors (112) with one or more hardware processors (202), wherein the one or more position parameters include the position of the autonomous robot, the expected position of the autonomous robot, one or more images of one or more targets, one or more videos of one or more targets, a set of images of other nearby autonomous robots, a set of videos of other autonomous robots, and one or more sounds of the surrounding environment. The steps include broadcasting one or more determined robot capabilities and one or more captured position parameters to each of one or more autonomous robots (110) using one or more hardware processors (202), Steps to determine one or more situational parameters relating to one or more autonomous robots (110) based on broadcasted robot capabilities and broadcasted positional parameters, received command sets, determined robot capabilities, and one or more responses to captured positional parameters, wherein the one or more situational parameters include payload type, sensor type, speed, payload capacity, class, battery level of each of the one or more autonomous robots (110), position of each of the one or more autonomous robots (110), expected position of each of the one or more autonomous robots (110), one or more images of one or more targets, one or more videos of one or more targets, a set of images of one or more autonomous robots (110), a set of videos of one or more autonomous robots (110), the relative position of each of the one or more autonomous robots (110), and one or more sounds of the surrounding environment, The steps include: detecting one or more targets based on a received command set, one or more determined robot capabilities, one or more captured position parameters, and one or more determined situation parameters, using a robot management-based AI model with one or more hardware processors (202); The steps of assigning one or more tasks and one or more detected targets among one or more autonomous robots (110) based on a received command set, one or more determined robot capabilities, one or more captured position parameters, one or more determined situation parameters, one or more target parameters, and predefined assignment information, wherein the one or more target parameters include the number of one or more targets, the size of each of the one or more targets, and the sensor footprint corresponding to each of the one or more autonomous robots (110), The steps include: using one or more hardware processors (202) to perform one or more tasks on one or more detected targets based on one or more tasks between one or more autonomous robots (110) and the assignment of one or more detected tasks, using a robot management-based AI model; Equipped with, Each of the one or more autonomous robots (110) can perform one or more tasks after receiving a command set without requiring a network connection to a human-machine interface (102). An artificial intelligence (AI) based method.

Images