Systems and methods for parallel management of multiple unmanned aerial vehicles

Abstract

In some embodiments, a computer-implemented method for managing a fleet of unmanned aerial vehicles (UAVs) is provided. The fleet management computing system receives telemetry information from multiple UAVs. Based on the telemetry information, the fleet management computing system generates a map interface with multiple UAV icons. The fleet management computing system receives selections for an initial group of UAV icons via the map interface, wherein the initial group of UAV icons includes two or more UAV icons. The fleet management computing system receives deselections of one or more UAV icons from the initial group of UAV icons to create a final group of selected UAV icons. The fleet management computing system transmits commands to the UAVs associated with the UAV icons in the final group of selected UAV icons.

Description

[0001] Cross-reference of related applications

[0002] This application claims priority to U.S. Patent Application No. 17 / 356,240, filed June 23, 2021, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure generally relates to unmanned aerial vehicles (UAVs), and specifically, but not exclusively, to the parallel management of multiple UAVs. Background Technology

[0004] Fleets of multiple unmanned aerial vehicles (UAVs) are increasingly being used for a variety of purposes, including but not limited to data collection and package delivery. While UAVs may be capable of operating fully autonomously to navigate paths or perform other types of tasks, supervision of even fully autonomous UAV operations is typically provided by human operators. Human operators can monitor sources of information not considered by the UAV's sensors, including but not limited to weather reports and emergency reports, and can use this information to issue commands to the UAV to deviate from its planned mission.

[0005] As UAV fleets grow larger, each human operator may be responsible for overseeing multiple UAVs that cannot be managed using current technology, which only allows commands to be transmitted to a single UAV at a time. The desired technology is one that allows human operators to manage multiple UAVs simultaneously, while remaining flexible enough to allow operators to send commands to fewer UAVs than they are responsible for. Summary of the Invention

[0006] In some embodiments, a computer-readable medium having logic stored thereon is provided. The logic is responsive to execution by one or more processors of a fleet management computing system, causing the fleet management computing system to perform actions for managing a fleet of unmanned aerial vehicles (UAVs), the actions including: the fleet management computing system receiving telemetry information from a plurality of UAVs; the fleet management computing system generating a map interface with a plurality of UAV icons based on the telemetry information; the fleet management computing system receiving a selection of an initial UAV icon group via the map interface, wherein the initial UAV icon group includes two or more UAV icons; the fleet management computing system receiving a deselection of one or more UAV icons in the initial UAV icon group to create a final selected UAV icon group; and the fleet management computing system transmitting commands to UAVs associated with the UAV icons of the final selected UAV icon group.

[0007] In some embodiments, a computer-implemented method for managing a fleet of unmanned aerial vehicles (UAVs) is provided. The fleet management computing system receives telemetry information from multiple UAVs. Based on the telemetry information, the fleet management computing system generates a map interface with multiple UAV icons. The fleet management computing system receives selections for an initial group of UAV icons via the map interface, wherein the initial group of UAV icons includes two or more UAV icons. The fleet management computing system receives deselections of one or more UAV icons from the initial group of UAV icons to create a final group of selected UAV icons. The fleet management computing system transmits commands to the UAVs associated with the UAV icons in the final group of selected UAV icons.

[0008] In some embodiments, a system is provided that includes a plurality of unmanned aerial vehicles (UAVs) and a cluster management computing system. The cluster management computing system is communicatively coupled to the plurality of UAVs and includes a non-transitory computer-readable medium on which logic is stored. In response to execution by one or more processors of the cluster management computing system, the logic causes the cluster management computing system to perform actions including receiving telemetry information from the plurality of UAVs by the cluster management computing system; generating a map interface with a plurality of UAV icons based on the telemetry information by the cluster management computing system; receiving a selection of an initial group of UAV icons via the map interface, wherein the initial group of UAV icons includes two or more UAV icons; deselecting one or more UAV icons in the initial group of UAV icons by the cluster management computing system to create a final selected group of UAV icons; and transmitting commands by the cluster management computing system to the UAVs associated with the UAV icons in the final selected group of UAV icons. Attached Figure Description

[0009] Non-limiting and non-exhaustive embodiments of the invention are described with reference to the following accompanying drawings, wherein, unless otherwise stated, the same reference numerals refer to the same parts in the various views. Not all instances of elements need to be labeled to avoid confusing the drawings where appropriate. The drawings are not necessarily drawn to scale, but rather focus on illustrating the principles described. For easy identification of any particular element or action being discussed, the most significant numeral or plurality of most significant numerals in the reference numerals refer to the drawing number in which that element was first introduced.

[0010] Figure 1 This is a block diagram illustrating aspects of a non-limiting example embodiment of a cluster management computing system according to various aspects of this disclosure.

[0011] Figure 2A and Figure 2B Non-limiting example embodiments of a UAV according to various aspects of this disclosure are shown.

[0012] Figure 3 This is a block diagram illustrating further aspects of a non-limiting example embodiment of a UAV according to various aspects of this disclosure.

[0013] Figures 4A-4B This is a flowchart illustrating a non-limiting example embodiment of a method for managing a UAV fleet according to various aspects of this disclosure.

[0014] Figure 5 These are illustrations of non-limiting example embodiments of a map interface generated by a user interface engine according to various aspects of this disclosure.

[0015] Figure 6A and Figure 6B Two non-limiting example embodiments of a technique for providing selection of an initial group of UAV icons via a map interface, according to various aspects of this disclosure, are shown.

[0016] Figure 7 This is an illustration of a non-limiting example embodiment of a map interface after canceling the selection of one or more UAV icons from the initial UAV icon group, according to various aspects of this disclosure.

[0017] Figure 8 This is an illustration of a map interface representing one or more interface elements for retrying commands to transmit to one or more UAVs, based on non-limiting example embodiments of various aspects of this disclosure. Detailed Implementation

[0018] In embodiments of this disclosure, the cluster management computing system provides a map-based user interface that can be used by an operator to flexibly select UAVs to send commands to them. In the map interface generated by the cluster management computing system, the operator can select UAVs in groups and then remove individual UAVs from the selection to precisely tailor the UAVs for receiving commands. In some embodiments, the cluster management computing system also monitors acknowledgments from UAVs and automatically updates the map interface to indicate retry transmissions of commands received by UAVs.

[0019] Figure 1 This is a block diagram illustrating aspects of a non-limiting example embodiment of a cluster management computing system according to various aspects of this disclosure. The illustrated cluster management computing system 110 can be implemented by any computing device or collection of computing devices, including but not limited to desktop computing devices, laptop computing devices, mobile computing devices, server computing devices, computing devices of cloud computing systems, and / or combinations thereof. Typically, the cluster management computing system 110 is configured to receive telemetry information from multiple UAVs and generate a map interface that allows users to select UAV groups to send commands to.

[0020] As shown in the figure, the cluster management computing system 110 includes one or more processors 102, one or more communication interfaces 104, a command log data storage repository 108, and a computer-readable medium 106.

[0021] In some embodiments, processor 102 may include any suitable type of general-purpose computer processor. In some embodiments, processor 102 may include one or more dedicated computer processors or AI accelerators optimized for a specific computing task, including but not limited to graphics processing units (GPUs), vision processing units (VPTs), and tensor processing units (TPUs).

[0022] In some embodiments, the communication interface 104 includes one or more hardware and / or software interfaces suitable for providing a communication link between the cluster management computing system 110 and multiple UAVs. The communication interface 104 may support one or more wired communication technologies (including but not limited to Ethernet, FireWire, and USB), one or more wireless communication technologies (including but not limited to Wi-Fi, WiMAX, Bluetooth, 2G, 3G, 4G, 5G, and LTE), and / or combinations thereof.

[0023] As shown, the computer-readable medium 106 has logic stored thereon that, in response to execution by one or more processors 102, enables the cluster management computing system 110 to provide a telemetry collection engine 112, a user interface engine 114, and a command transfer engine 116.

[0024] As used herein, “computer-readable medium” means any removable or non-removable device that implements a technology capable of storing information in a volatile or non-volatile manner for reading by a processor of a computing device, including but not limited to: hard disk drives; flash memory; solid-state drives; random access memory (RAM); read-only memory (ROM); CD-ROM, DVD or other disc storage devices; cassette tape; magnetic tape; and disk storage devices.

[0025] In some embodiments, the telemetry collection engine 112 is configured to receive telemetry information from multiple UAVs when multiple UAVs are in flight and to provide the telemetry information to the user interface engine 114. In some embodiments, the user interface engine 114 is configured to use the telemetry information, mission specifications, restricted airspace, and / or other information to generate a map interface from which a user can select a UAV. In some embodiments, the command transmission engine 116 is configured to transmit commands to the UAV selected within the map interface. The command transmission engine 116 may also be configured to receive acknowledgments of the transmitted commands from the UAVs, compare the acknowledgments with a record of the transmitted commands stored in the command log data repository 108, and provide information to the user interface engine 114 to allow the user interface engine 114 to generate an interface through which it can retry failed commands. Further description of the configuration of each of these components is provided below.

[0026] As used in this article, "engine" refers to the logic embodied in hardware or software instructions, which can be written in one or more programming languages, including but not limited to C, C++, C#, COBOL, and JAVA. TM The engine can be written in languages ​​such as PHP, Perl, HTML, CSS, TypeScript, JavaScript, VBScript, ASPX, Go, and Python. It can be compiled into an executable program or written in an interpreted programming language. A software engine can be invoked from other engines or from itself. Typically, the engine described in this document refers to a logical module that can be combined with other engines, or it can be divided into sub-engines. An engine can be implemented by logic stored in any type of computer-readable medium or computer storage device, and can be stored and executed by one or more general-purpose computers, thus creating a dedicated computer configured to provide the engine or its functionality. An engine can be implemented by programming logic into an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or another hardware device.

[0027] As used herein, a “data store” refers to any suitable device configured to store data for access by computing devices. One example of a data store is a highly reliable, high-speed relational database management system (DBMS) that runs on one or more computing devices and is accessible via a high-speed network. Another example of a data store is a key-value store. However, any other suitable storage technology and / or device capable of providing stored data quickly and reliably in response to queries may be used, and the computing device may be locally accessible rather than network-accessible, or may be provided as a cloud-based service. A data store may also include data stored in an organized manner on computer-readable storage media such as hard disk drives, flash memory, RAM, ROM, etc., or any other type of computer-readable storage media. Those skilled in the art will recognize that the individual data stores described herein may be combined into a single data store, and / or the single data store described herein may be separated into multiple data stores without departing from the scope of this disclosure.

[0028] Figure 2A and Figure 2B An aircraft or UAV 200 according to an embodiment of this disclosure is shown. The illustrated embodiment of UAV 200 is a vertical takeoff and landing (VTOL) unmanned aerial vehicle (UAV) including separate propulsion units 212 and 208 for providing horizontal and vertical propulsion, respectively. UAV 200 is a fixed-wing aircraft, and as the name suggests, has a wing assembly 224 that can generate lift based on the wing shape and the forward airspeed of the aircraft when horizontally propelled by propulsion unit 212. Figure 2A This is a perspective top view of the UAV200, and Figure 2B This is a bottom plan view of UAV 200. Those skilled in the art will recognize that UAV 200 is a non-limiting example of a UAV that can be used with embodiments of this disclosure, and in some embodiments, in addition to... Figure 2A , Figure 2B and Figure 3 The UAV 200 shown is either outside or replaced Figure 2A , Figure 2B and Figure 3 The UAV 200 shown, managed by the cluster management computing system, may also include other types of UAVs.

[0029] The illustrated embodiment of UAV 200 includes a fuselage 220. In one embodiment, the fuselage 220 is modular and includes a battery module, an avionics module, and a mission payload module. These modules are detachable from each other and mechanically attached to each other to continuously form at least a portion of the fuselage 220 or the UAV body.

[0030] The battery module includes a cavity for housing one or more batteries for powering the UAV 200. The avionics module houses the flight control circuitry of the UAV 200 and may include a processor and memory, communication electronics and antennas (e.g., cellular transceivers, Wi-Fi transceivers, etc.), and various sensors (e.g., GPS sensors, inertial measurement units (IMUs), magnetic compasses, etc.). The mission payload module houses equipment associated with the mission of the UAV 200. For example, the mission payload module may include a payload actuator for holding and releasing externally attached payloads. In another embodiment, the mission payload module may include a camera / sensor device holder for carrying camera / sensor devices (e.g., cameras, lenses, radar, lidar, pollution monitoring sensors, weather monitoring sensors, etc.). Figure 3 The diagram shows other components that can be carried by some embodiments of the UAV 200.

[0031] The illustrated embodiment of UAV 200 also includes horizontal propulsion units 212 positioned on wing assembly 224. Each horizontal propulsion unit 212 may include a motor, shaft, motor mount, and thruster for propelling UAV 200. The illustrated embodiment of UAV 200 includes two boom assemblies 206 fixed to wing assembly 224.

[0032] Each illustrated embodiment of the boom assembly 206 includes a boom housing 218 in which the boom is disposed, a vertical propulsion unit 208, a printed circuit board 216, and a stabilizer 202. The vertical propulsion unit 208 may each include a motor, shaft, motor mount, and thruster for providing vertical propulsion. The vertical propulsion unit 208 can be used during hovering mode, wherein the UAV 200 descends (e.g., to a delivery position) or ascends (e.g., after delivery). The stabilizer 202 (or winglets) may be included in the UAV 200 to stabilize the yaw (left or right turn) of the UAV during flight. In some embodiments, the UAV 200 may be configured to function as a glider. For this purpose, the UAV 200 may shut down its propulsion unit and glide for a period of time.

[0033] During flight, UAV 200 can control its direction and / or speed of movement by controlling its pitch, roll, yaw, and / or altitude. For example, stabilizer 202 may include one or more rudders 204 for controlling the UAV's yaw, and wing assembly 224 may include elevators for controlling the UAV's pitch and / or ailerons 210 for controlling the UAV's roll. As another example, simultaneously increasing or decreasing the speed of all thrusters can respectively cause UAV 200 to increase or decrease its altitude. UAV 200 may also include components for sensing the environment around UAV 200, including but not limited to audio sensors 222 and 214. Further examples of sensor devices are... Figure 3 It is shown in the figure and described below.

[0034] Many variations of the fixed-wing aircraft shown are possible. For example, aircraft with more wings (e.g., an "X-wing" configuration with four wings) are also possible. Although Figure 2A and Figure 2B A wing assembly 224, two boom assemblies 206, two horizontal propulsion units 212, and six vertical propulsion units 208 for each boom assembly 206 are shown; however, it should be understood that other variants of the UAV 200 can be implemented with more or fewer of these components.

[0035] It should be understood that the reference to “unmanned” aircraft or UAVs herein can be equally applied to autonomous and semi-autonomous aircraft. In a fully autonomous implementation, all functions of the aircraft are automated; for example, pre-programmed or controlled via real-time computer functions in response to inputs and / or predetermined information from various sensors. In a semi-autonomous implementation, some functions of the aircraft may be controlled by a human operator, while others are performed autonomously. Furthermore, in some embodiments, a UAV may be configured to allow a remote operator to take over functions that would otherwise be autonomously controlled by the UAV. Moreover, a given type of function may be remotely controlled at one level of abstraction and autonomously performed at another. For example, a remote operator may control high-level navigation decisions of the UAV, such as specifying that the UAV should travel from one location to another (e.g., from a warehouse in a suburban area to a delivery address in a nearby city), while the UAV’s navigation system autonomously controls finer-grained navigation decisions, such as specific routes to take between two locations, specific flight controls to achieve the route and avoid obstacles while navigating the route, etc.

[0036] Figure 3This is a block diagram illustrating further aspects of a UAV according to various aspects of this disclosure. As shown, UAV 200 includes a communication interface 302, one or more sensor devices 304, a power supply 306, one or more processors 308, one or more propulsion devices 310, and a computer-readable medium 312.

[0037] In some embodiments, communication interface 302 includes hardware and software to implement any suitable communication technology for communicating with cluster management computing system 110. In some embodiments, communication interface 302 includes multiple communication interfaces, each for use as appropriate. For example, communication interface 302 may include a long-range wireless interface, such as a 4G or LTE interface, or any other type of long-range wireless interface (e.g., 2G, 3G, 5G, or WiMAX), for communicating with cluster management computing system 110 while traversing a route. Communication interface 302 may also include a mid-range wireless interface, such as a Wi-Fi interface used when UAV 200 is in an area near a starting point or endpoint where Wi-Fi coverage is available. Communication interface 302 may also include a short-range wireless interface, such as a Bluetooth interface, for use when UAV 200 is in a maintenance position or otherwise stationary and awaiting route allocation. Communication interface 302 may also include a wired interface, such as an Ethernet interface or a USB interface, which may also be used when UAV 200 is in a maintenance position or otherwise stationary and awaiting route allocation.

[0038] In some embodiments, sensor device 304 is configured to detect states associated with various components of UAV 200 and send signals representing these states to other components of UAV 200. Some non-limiting examples of sensor device 304 that can monitor components of UAV 200 itself include battery status sensors and propulsion device health sensors. Some non-limiting examples of sensor device 304 that can monitor UAV 200 relative to the environment include positioning system sensors (including but not limited to GPS sensors), attitude sensors, airspeed sensors, compasses, and altitude sensors.

[0039] In some embodiments, power source 306 may be any suitable device or system for storing and / or generating electricity. Some non-limiting examples of power source 306 include one or more batteries, one or more solar panels, fuel tanks, and combinations thereof. In some embodiments, propulsion device 310 may include any suitable device for propelling UAV 200 along a route, and may include devices such as, but not limited to, one or more motors, one or more thrusters, and one or more flight control surfaces.

[0040] In some embodiments, processor 308 may include any type of computer processor capable of receiving signals from other components of UAV 200 and executing instructions stored on computer-readable medium 312. In some embodiments, computer-readable medium 312 may include one or more devices capable of storing information accessed by processor 308. In some embodiments, computer-readable medium 312 may include one or more of hard disk drives, flash drives, EEPROMs, and combinations thereof.

[0041] As shown in the figure, a computer-readable medium 312 stores a route data repository 314, a command processing engine 316, and a route traversal engine 318. In some embodiments, the route traversal engine 318 is configured to cause the propulsion device 310 to propel the UAV 200 along a route received from a human pilot, an automated system, or from any other device and stored in the route data repository 314. The route traversal engine 318 may use signals from other devices (e.g., GPS sensor devices, vision-based navigation devices, accelerometers, LiDAR devices, and / or other devices) to assist in positioning and navigation, which is typical for the UAV 200. In some embodiments, the command processing engine 316 is configured to receive commands from the cluster management computing system 110 to cause the route traversal engine 318 to execute the commands, and to send an acknowledgment notification to the cluster management computing system 110 upon successful receipt of the commands.

[0042] Figures 4A-4B This is a flowchart illustrating a non-limiting example embodiment of a method for managing a UAV fleet according to various aspects of this disclosure. In method 400, the fleet management computing system 110 generates a map interface that allows a user to select multiple UAVs based on a map area and send commands to the selected UAVs. Because UAV fleet group management tasks are typically applied to UAVs within a geographic area, using a map interface to select an initial UAV group and then refine that selection to fewer than all initial UAV groups (described below) provides an intuitive interface that allows the operator to quickly specify UAV groups based on geography and then conveniently modify that selection.

[0043] From the start box, method 400 proceeds to box 402, where the telemetry collection engine 112 of the fleet management computing system 110 receives telemetry information from multiple UAVs. As described above, each UAV 200 includes one or more wireless communication interfaces 302 capable of communicating with one or more communication interfaces 104 on the fleet management computing system 110 during flight. The telemetry collection engine 112 can receive any suitable telemetry information that allows the fleet management computing system 110 to generate a map interface, including but not limited to location information (including but not limited to one or more of latitude / longitude, a set of GPS coordinates, what3words geocode, or Plus Code), altitude information, attitude information, and speed information.

[0044] At box 404, the user interface engine 114 of the cluster management computing system 110 generates a map interface with multiple UAV icons based on telemetry information. The map interface can be generated by the user interface engine 114 using any suitable technology. In some embodiments, the user interface engine 114 can generate a web interface retrieved and rendered by a standard web browser on a user device. In some embodiments, the user interface engine 114 can provide information from the interface program to provide portions of the map interface (e.g., multiple UAV icons) as overlays of a map provided by a system separate from the cluster management computing system 110. In some embodiments, the user interface engine 114 can generate the map interface as a video or graphic presentation, and can provide a video or graphic presentation as the map interface. Any other suitable technology for generating interactive interfaces can be used. Because such technologies are known to those skilled in the art, they will not be described in any further detail here for the sake of brevity.

[0045] Figure 5 This is an illustration of a non-limiting example embodiment of a map interface generated by user interface engine 114 according to various aspects of this disclosure. In the illustrated embodiment, map interface 504 is presented within a standard web browser 502. As shown, map interface 504 displays a map. Map interface 504 includes a plurality of UAV icons 506a-506m, which are presented at locations indicated by telemetry information. Although all UAV icons 506a-506m match, in some embodiments, different UAV icons may be used for different types of UAVs. Users can scroll the map by clicking and dragging, using scroll arrows, or by any other suitable technique. Different UAV icons may be displayed as the area depicted by map interface 504 moves. Similarly, the positions of UAV icons 506a-506m may be updated by user interface engine 114 within map interface 504 as telemetry collection engine 112 continues to receive telemetry information.

[0046] At box 406, user interface engine 114 receives a selection of an initial UAV icon group, wherein the initial UAV icon group includes two or more UAV icons. Figure 6A and Figure 6B Two non-limiting example embodiments of a technique for providing a selection of an initial group of UAV icons via a map interface 504, according to various aspects of this disclosure, are shown.

[0047] exist Figure 6A In the middle, draw a rectangular selection border 608 around the UAV icon group to create the initial UAV icon group. (See figure.) Figure 5 The UAV icons 506a, 506b, 506c, 506d, 506k, and 506l have been changed to the selected UAV icons 602a-602h to indicate that they are part of the selected group.

[0048] The map interface 504 allows the user to draw a rectangular selection boundary 608 using any suitable technique. As a non-limiting example, the map interface 504 allows the user to click on a first corner and then drag the opposite corner of the rectangular selection boundary 608 to stretch the boundary over the desired area. As another non-limiting example, the map interface 504 allows the user to click on a first corner and then click on another corner to define the rectangular selection boundary 608.

[0049] exist Figure 6B Instead of rectangular selection boundary 608, lasso selection boundary 610 is drawn around the same group of UAV icons to create an initial group of UAV icons. Lasso selection boundary 610 can be provided by the user drawing a boundary of any shape around the desired group of UAV icons. Both rectangular selection boundary 608 and lasso selection boundary 610 have various advantages. For example, rectangular selection boundary 608 is easy to represent and can be easily transferred to the cluster management computing system 110, but it is less precise. Meanwhile, lasso selection boundary 610 is more difficult to represent and transferring it to the cluster management computing system 110 can be complex, but it allows for precise selection of a geographically contiguous group of UAV icons.

[0050] In some embodiments (particularly embodiments where clicking and dragging typically scroll the map area), the map interface 504 may use modification keys to accompany clicks (e.g., shift clicks or replacement clicks) to change the mode of the map interface 504 from scrolling mode to boundary definition mode. In some embodiments, the map interface 504 may provide controls (e.g., buttons) for switching the mode from scrolling mode to boundary definition mode. Figure 6A and Figure 6B The send command interface element 604 and select command interface element 606 are also shown, which will be discussed in further detail below.

[0051] Return to Figure 4A At selectable box 408, user interface engine 114 receives a deselection of one or more UAV icons in the initial UAV icon group to create a final selected UAV icon group. Any suitable technology can be used to provide deselection of UAV icons. In some embodiments, a modified click (e.g., control click, right-click, long tap) can be used to indicate that the selected UAV icon should be deselected. In some embodiments, a normal click or tap on a UAV icon can toggle the UAV icon between a selected state and a deselected state.

[0052] Figure 7 These are illustrations of a non-limiting example embodiment of a map interface after one or more UAV icons have been deselected from an initial UAV icon group, according to various aspects of this disclosure. Figure 7 It is still shown in the middle. Figure 6B The lasso selection boundary is 610, but in some implementations, the selection boundary can disappear once the initial selection is made. As shown, the selected UAV icon 602f and the selected UAV icon 602g have been deselected, and are now the deselected UAV icons 702b and 702a, respectively. Since they have been deselected, the selected UAV icons 602a, 602b, 602d, and 602h now constitute the final selected UAV icon group.

[0053] In some embodiments, the map interface 504 may accept multiple switching between a selected and deselected state for a given UAV icon; however, for clarity, Figure 7 Only a single toggle from selection to deselection is shown. In some embodiments, the map interface 504 may also allow additional UAV icons that are not part of the initial UAV icon group to be added to the final selected UAV icon group (e.g., by a click on the additional UAV icon that modifies the selection (e.g., a control-click or shift-click), by dragging the selection boundary around the additional UAV icon, by adding an additional selection boundary, or by any other suitable technique).

[0054] Return to Figure 4AAt option 410, the user interface engine 114 receives the label to be associated with the final selected UAV icon group and stores the label and the final selected UAV icon group in the command log data repository 108 of the cluster management computing system 110. Although not shown, the label can be provided via a map interface through one or more interface controls that allow the user to specify the label to be applied. Saving the label of the final selected UAV icon group allows the user to address the same selected UAV group in the future, which can be particularly helpful when the UAVs have been moved to different geographic areas but the user still wants to command them as a group.

[0055] At box 412, the user interface engine 114 receives input identifying the command to be sent to the UAV associated with the UAV icon in the ultimately selected UAV icon group. Any suitable interface element can be used to provide input identifying the command to be sent. Figure 7 In this configuration, a simple command selection interface element 606 is provided as a drop-down list, allowing the user to select from a plurality of available commands that can be sent. Any appropriate command may be supported by the cluster management computing system 110 and listed in the command selection interface element 606, including but not limited to the “Land Now” command that enables the UAV to autonomously land at or near its current location; the “Hover” command that enables the UAV to autonomously hover at its current location; the “Return to Base” command that enables the UAV to autonomously return to base; and the “Ceiling / Floor” command that enables the UAV to autonomously navigate above a given floor elevation and / or below a given ceiling elevation.

[0056] Return to Figure 4A At box 414, the command transmission engine 116 of the cluster management computing system 110 transmits commands to the UAVs associated with the UAV icons of the finally selected UAV icon group. In some embodiments, commands are sent to the UAVs using a communication path similar to the path used to receive telemetry information. In some embodiments, commands are transmitted in response to actuation of user interface elements, such as command sending interface element 604.

[0057] At block 416, command transmission engine 116 stores a record of commands in command record data repository 108. In some embodiments, the command record at least indicates what command was sent and to which UAVs it was sent. In some embodiments, the command record may include a timestamp, a unique identifier, or other information that allows command transmission engine 116 to uniquely match the command record to subsequently received acknowledgment notifications.

[0058] Then, method 400 proceeds to the continuing terminal (“Terminal A”).

[0059] From terminal A ( Figure 4BMethod 400 proceeds to block 418, where command transmission engine 116 receives an acknowledgment message from one or more UAVs to which a command has been transmitted. Although not shown in method 400, once UAV 200 receives a command, command processing engine 316 sends an acknowledgment message back to command transmission engine 116 to indicate that the command was received before it was implemented. In some embodiments, the acknowledgment message may include a timestamp, a unique identifier, or other identifying information that allows command transmission engine 116 to associate the acknowledgment message with a record of the command stored in command record data repository 108, as well as information that allows command transmission engine 116 to identify the UAV 200 from which it sent the acknowledgment message.

[0060] At box 420, command transmission engine 116 updates the command records in command log data repository 108 based on acknowledgment messages. In this way, the command records in command log data repository 108 will track which UAVs associated with icons in the finally selected UAV icon group have successfully received commands, and which UAVs have not yet successfully received commands.

[0061] At decision box 422, it is determined whether the cluster management computing system 110 should retry sending one or more failed commands. In some embodiments, this determination may be based on whether the command log indicates any missing acknowledgment messages. In some embodiments, this determination may wait for a predetermined amount of time after the initial transmission before determining that the command should be retried, in order to allow for transmission latency and processing delay in communication between the cluster management computing system 110 and the UAV.

[0062] If it is determined that the cluster management computing system 110 should retry transmitting the command (e.g., if there is a lost acknowledgment message and a predetermined amount of time has elapsed), then the result of decision box 422 is yes, and method 400 proceeds to box 424, where the user interface engine 114 presents one or more interface elements for retrying transmitting the command to one or more UAVs.

[0063] Figure 8 This is an illustration of a non-limiting example embodiment of a map interface according to various aspects of this disclosure, which presents one or more interface elements for retrying commands transmitted to one or more UAVs. Figure 8 In the map interface 504, a failed UAV icon 802a is displayed to indicate that the transmission of the command associated with the selected UAV icon 602e failed, and a failed UAV icon 802b is displayed to indicate that the transmission of the command associated with the selected UAV icon 602c failed. The user can activate the retry command interface element 804 to send the commands listed in the select command interface element 606 to the UAV associated with the icon indicated by the failed UAV icon.

[0064] In some embodiments, the select command interface element 606 allows the user to change the command to be sent during a retrieval, while in other embodiments, the select command interface element 606 may be read-only when the retry command interface element 804 is presented. As with other illustrations, Figure 8 The map interface 504 allows the user to deselect one or more of the failed UAV icons 802a and 802b, preventing the command from being retried to the associated UAV. In some embodiments, the command transmission engine 116 can continue to check the confirmation message and can remove the failed UAV icon from any selected UAV icons to obtain the confirmation message that entered after the initial presentation of the failed UAV icon, keeping the map interface 504 up-to-date.

[0065] Return to Figure 4B After retrying the transmission at box 424, method 400 returns to box 418 to check for further command confirmation. It then returns to decision box 422. If it is determined that no retry is needed, the result of decision box 422 is negative, and method 400 proceeds to the end box, where method 400 terminates.

[0066] In the foregoing description, numerous specific details have been set forth to provide a thorough understanding of the various embodiments of this disclosure. However, those skilled in the art will recognize that the techniques described herein can be practiced without one or more of these specific details, or can be practiced using other methods, components, materials, etc. In other instances, well-known structures, materials, or operations have not been shown or described in detail to avoid obscuring certain aspects.

[0067] Throughout this specification, references to "an embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with an embodiment is included in at least one embodiment of the invention. Therefore, the phrases "in one embodiment" or "in an embodiment" appearing in various places throughout this specification do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0068] The order in which some or all boxes appear in each method flowchart should not be considered limiting. Rather, those skilled in the art who benefit from this disclosure will understand that actions associated with some boxes can be performed in various orders not shown, or even in parallel.

[0069] The above explanation describes the process based on computer software and hardware. The described techniques can be embodied in machine-executable instructions contained in a tangible or non-transitory machine-readable storage medium, which, when executed by a machine, will cause the machine to perform the described operations. Alternatively, the process can be embodied in hardware, such as an application-specific integrated circuit (“ASIC”) or others.

[0070] The above description of the embodiments illustrated in this invention, including the content described in the abstract, is not intended to be exhaustive or to limit the invention to the precise forms disclosed. While specific embodiments and examples of the invention have been described herein for illustrative purposes, various modifications within the scope of the invention will be recognized by those skilled in the art.

[0071] Based on the above detailed description, these modifications can be made to the present invention. The terminology used in the appended claims should not be construed as limiting the invention to the specific embodiments disclosed in the specification. Rather, the scope of the invention will be fully defined by the appended claims, which will be interpreted according to the established principles of claim interpretation.

Claims

1. A computer-readable medium having logic stored thereon, the logic being responsive to execution by one or more processors of a fleet management computing system to cause the fleet management computing system to perform actions for managing a fleet of unmanned aerial vehicles (UAVs), the actions comprising: The cluster management computing system receives telemetry information from multiple UAVs; The cluster management computing system generates a map interface with multiple UAV icons based on the telemetry information; The cluster management computing system receives the selection of an initial UAV icon group via the map interface, wherein the initial UAV icon group includes two or more UAV icons; The cluster management computing system receives the deselection of one or more UAV icons in the initial UAV icon group to create the final selected UAV icon group; The cluster management computing system transmits commands to the UAVs associated with the UAV icons in the finally selected UAV icon group; The cluster management computing system receives a confirmation message from the UAV that has already received the command. as well as The cluster management computing system generates interface elements to allow retries in transmitting the command to UAVs that have not yet received an acknowledgment message from them. The generation of interface elements to allow retrying the transmission of the command includes deselecting the UAV icon on the map interface associated with the UAV from which a confirmation message has been received.

2. The computer-readable medium according to claim 1, wherein, Receiving the selection of the initial UAV icon group via the map interface includes receiving input of the boundaries around two or more UAV icons to be selected.

3. The computer-readable medium according to claim 2, wherein, The input for receiving the boundaries around the two or more UAV icons to be selected includes receiving the rectangular selection boundaries.

4. The computer-readable medium according to claim 2, wherein, The input for receiving the boundaries around the two or more UAV icons to be selected includes receiving the lasso selection boundaries.

5. The computer-readable medium according to claim 1, wherein, Receiving a deselection of one or more UAV icons in the initial UAV icon group includes receiving click and modify key input, right-click input, or long-press input.

6. The computer-readable medium according to claim 1, wherein, Sending commands to the UAV associated with the UAV icon in the final selected UAV icon group includes sending a "Land Now" command or a "Hover" command.

7. The computer-readable medium according to claim 1, wherein, The action also includes: The cluster management computing system receives an instruction to apply a label to the UAV associated with the UAV icon in the final selected UAV icon group; and The cluster management computing system stores the associations between the tags and the UAVs associated with the UAV icons in the finally selected UAV icon group.

8. A computer-implemented method for managing a fleet of unmanned aerial vehicles (UAVs), the method comprising: The cluster management computing system receives telemetry information from multiple UAVs; The cluster management computing system generates a map interface with multiple UAV icons based on the telemetry information; The cluster management computing system receives the selection of an initial UAV icon group via the map interface, wherein the initial UAV icon group includes two or more UAV icons; The cluster management computing system receives the deselection of one or more UAV icons in the initial UAV icon group to create the final selected UAV icon group; The cluster management computing system transmits commands to the UAVs associated with the UAV icons in the finally selected UAV icon group; The cluster management computing system receives a confirmation message from the UAV that has already received the command. as well as The cluster management computing system generates interface elements to allow retries in transmitting the command to UAVs that have not yet received an acknowledgment message from them. The generation of interface elements to allow retrying the transmission of the command includes deselecting the UAV icon on the map interface associated with the UAV from which a confirmation message has been received.

9. The computer-implemented method according to claim 8, wherein, Receiving the selection of the initial UAV icon group via the map interface includes receiving input of the boundaries around two or more UAV icons to be selected.

10. The computer-implemented method according to claim 9, wherein, The input for receiving the boundaries around the two or more UAV icons to be selected includes receiving the rectangular selection boundaries.

11. The computer-implemented method according to claim 9, wherein, The input for receiving the boundaries around the two or more UAV icons to be selected includes receiving the lasso selection boundaries.

12. The computer-implemented method according to claim 8, wherein, Receiving a deselection of one or more UAV icons in the initial UAV icon group includes receiving click and modify key input, right-click input, or long-press input.

13. The computer-implemented method according to claim 8, wherein, Transmitting the command to the UAV associated with the UAV icon in the final selected UAV icon group includes transmitting a now-landing command or a hovering command.

14. The computer-implemented method according to claim 8, further comprising: The cluster management computing system receives an instruction to apply a label to the UAV associated with the UAV icon in the final selected UAV icon group; as well as The cluster management computing system stores the associations between the tags and the UAVs associated with the UAV icons in the finally selected UAV icon group.

15. A system for managing multiple unmanned aerial vehicles (UAVs), comprising: The cluster management computing system is communicatively coupled to the multiple UAVs; The cluster management computing system includes a non-transitory computer-readable medium storing logic thereon, the logic being responsive to execution by one or more processors of the cluster management computing system to cause the cluster management computing system to perform actions, the actions including: The cluster management computing system receives telemetry information from the multiple UAVs; The cluster management computing system generates a map interface with multiple UAV icons based on the telemetry information; The cluster management computing system receives the selection of an initial UAV icon group via the map interface, wherein the initial UAV icon group includes two or more UAV icons; The cluster management computing system receives the deselection of one or more UAV icons in the initial UAV icon group to create the final selected UAV icon group; The cluster management computing system transmits commands to the UAVs associated with the UAV icons in the finally selected UAV icon group; The cluster management computing system receives an acknowledgment message from the UAV that has already received the command; and The cluster management computing system generates interface elements to allow retries in transmitting the command to UAVs that have not yet received an acknowledgment message from them. The generation of interface elements to allow retrying the transmission of the command includes deselecting the UAV icon on the map interface associated with the UAV from which a confirmation message has been received.

16. The system according to claim 15, wherein, Receiving a selection of the initial UAV icon group via the map interface includes receiving input around the boundaries of the two or more UAV icons to be selected; The input for receiving the boundaries around the two or more UAV icons to be selected includes receiving rectangular selection boundaries or lasso selection boundaries; and The function of receiving the deselection of one or more UAV icons in the initial UAV icon group includes receiving click and modify key input, right-click input, or long-press input.

Images