Command input for remote management of multiple autonomous aircraft

Abstract

Embodiments provide an animated graphical user interface for simultaneously monitoring multiple autonomous aircraft. More specifically, the technology disclosed herein provides a GUI that enables one-to-many management, such that a remote administrator (e.g., human or artificial intelligence) can continuously monitor and / or interact with multiple autonomous aircraft. For example, the autonomous aircraft may include one or more autonomous electric vertical take-off and landing (eVTOL) aircraft. Embodiments provide a command input system and techniques that reduce human error by assigning input mechanisms and command input menus to unique flight parameters.

Description

[Technical Field]

[0001] This application claims priority under and the benefit of U.S. Provisional Patent Application No. 63 / 422,832, entitled "Command Input For Remote Supervision Of Multiple Autonomous Aircraft Command," filed November 4, 2022, the disclosure of which is incorporated herein by reference in its entirety for all purposes.

[0002] This application claims priority to U.S. Provisional Patent Application No. 63 / 408,088, filed September 19, 2022, entitled "Remote Supervision of Multiple Autonomous Aircraft," which is related to U.S. Patent Application No. 18 / 469,965, filed September 19, 2023, entitled "Remote Supervision of Multiple Autonomous Aircraft," and U.S. Design Patent Application No. 29 / 853,825, filed September 19, 2022, entitled "Multi Vehicle Supervisor Interface." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent Application No. 18 / 469,965 [Patent Document 2] U.S. Provisional Patent Application No. 63 / 408,088 [Patent Document 3] U.S. Design Patent Application No. 29 / 853,825 Summary of the Invention [Means for solving the problem]

[0004] Embodiments provide techniques for communicating with a remote computer-implemented administrator that simultaneously manages the functions of multiple autonomous aircraft. According to various embodiments, a graphical user interface (GUI) is provided for providing control inputs to a remote computer-implemented administrator to simultaneously monitor, manage, and / or communicate (e.g., interact) with multiple autonomous aircraft. The GUI may be an animated GUI that includes a graphical representation of the aircraft and related information displayed (e.g., overlaid) on a map representing the area in which the autonomous aircraft are flying. The systems, devices, and techniques described herein provide a command input system and method that allows for rapid manipulation of autonomous aircraft under its control and scales to address display clutter as the number of autonomous aircraft managed by the same administrator increases.

[0005] An embodiment provides a method for monitoring and interacting with multiple autonomous air vehicles. The method includes displaying a graphical user interface (GUI) on a display device using a server computer. The method further includes representing the multiple autonomous air vehicles with multiple icons displayed on the GUI by the server computer. The server computer receives an input selecting a first icon from the multiple icons displayed on the GUI. The server computer receives a first signal from an input device including multiple input mechanisms, each of which is assigned to a unique flight parameter. In response to receiving the first signal, the server computer displays a first menu from multiple menus on the GUI. The first menu is associated with a first flight parameter of the autonomous air vehicle represented by the first icon. The first menu includes a plurality of first preset commands. The method further includes receiving, by the server computer, a selection of a first command from the plurality of first preset commands. The method includes transmitting, by the server computer, the first command to the first autonomous air vehicle represented by the first icon on the GUI.

[0006] Some embodiments provide a system comprising a display device, one or more processors, an input device including a plurality of input mechanisms, the input device communicatively coupled to the one or more processors, and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the methods described above.

[0007] An embodiment further provides one or more non-transitory computer-readable storage media storing instructions that, when executed on a server computer for remotely monitoring and interacting with a plurality of autonomous air vehicles, cause the server computer to perform the method described above.

[0008] An embodiment provides an input device including a plurality of input mechanisms, each configured to provide a command to alter a specific parameter, including a flight parameter, associated with the autonomous air vehicle, the input device being communicatively coupled to a server computer that transmits the command to the autonomous air vehicle.

[0009] For a better understanding of the nature and advantages of the present disclosure, reference should be made to the following description and accompanying figures. It should be understood, however, that each of the figures is provided for illustrative purposes only and is not intended as a definition of the limits of the scope of the present disclosure. Furthermore, as a general rule, and unless otherwise apparent from the description, when elements in different figures use the same reference numerals, those elements are generally either identical or at least similar in function or purpose. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram of an example flight management platform for monitoring and interacting with multiple autonomous air vehicles, in accordance with various embodiments. [Figure 2A] FIG. 1 illustrates an example animated graphical user interface (GUI) of a flight management platform for managing and interacting with multiple autonomous air vehicles, according to various embodiments. [Figure 2B] FIG. 10 illustrates another view of an exemplary animated GUI of a flight management platform, in accordance with various embodiments. [Figure 3] FIG. 1 illustrates an example technique for assigning a route to an autonomous aircraft using an example animated GUI of a flight management platform, in accordance with various embodiments. [Figure 4A] FIG. 10 illustrates the addition of layers onto a displayed exemplary animated GUI of a flight management platform, in accordance with various embodiments. [Figure 4B] FIG. 10 illustrates the removal of a layer from a displayed animated GUI of a flight management platform, according to various embodiments. [Figure 5] FIG. 1 illustrates tracking an autonomous air vehicle in flight using an exemplary animated GUI of a flight management platform, in accordance with various embodiments. [Figure 6] FIG. 1 illustrates a zoomed-in view of an exemplary animated GUI of a flight management platform, in accordance with various embodiments. [Figure 7] FIG. 10 illustrates another view of an exemplary animated GUI including a flight guidance pane, in accordance with various embodiments. [Figure 8] FIG. 10 illustrates another view of the exemplary animated GUI after a mission assignment has been accepted, in accordance with various embodiments. [Figure 9] FIG. 10 illustrates an example warning displayed using an example animated GUI of a flight management platform, in accordance with various embodiments. [Figure 10] FIG. 1 illustrates an exemplary command input device, according to various embodiments. [Figure 11] FIG. 1 illustrates an exemplary command entry UI, according to various embodiments. [Figure 12] FIG. 1 illustrates an exemplary representation of multiple aircraft and associated data on a GUI, according to various embodiments. [Figure 13] 1 is a flowchart of an exemplary process performed by a flight management platform, in accordance with various embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0011] The technology disclosed herein generally relates to an animated graphical user interface for simultaneously monitoring multiple autonomous air vehicles. More specifically, the technology disclosed herein provides a one-to-many management platform (e.g., a flight management platform) that allows a remote administrator (e.g., a human or artificial intelligence) to continuously monitor and / or interact with multiple autonomous air vehicles. The interaction may include sending commands to the autonomous air vehicles using a GUI component of the platform. For example, the autonomous air vehicles may include one or more autonomous electric vertical takeoff and landing (eVTOL) aircraft. Various inventive embodiments, including methods, processes, systems, and devices, are described herein.

[0012] The present technology further provides for communicating with a remote computer-implemented manager that simultaneously manages the functions of multiple autonomous air vehicles. The platform is further configured to provide control inputs to the remote computer-implemented manager to simultaneously monitor, manage, and / or communicate (e.g., interact) with multiple autonomous air vehicles. Embodiments allow for quick, arbitrary inputs to be provided. Embodiments provide pop-up menus that optimize the use of screen real estate. Embodiments separate commands one section at a time, allowing the remote manager to maintain a single point of gaze (e.g., without having to navigate to different parts of the platform or display).

[0013] An embodiment provides an input device (e.g., a three-button mouse) for providing commands to the aircraft. The input device may include dedicated mechanisms (e.g., buttons, wheels, touchpads) associated with one or more flight parameters (e.g., heading, true airspeed, altitude). In some embodiments, the mechanisms may be coded using sensory cues (e.g., visual cues such as colors or tactile cues such as hash patterns that can be sensed by touch and seen on a screen). For example, the same color may be applied to the input mechanisms to display menus associated with the flight parameters assigned to the mechanisms. For example, a button for providing heading input may be colored orange on the input device, and a menu for providing heading commands may be displayed on the GUI using the color orange. Similarly, heading information associated with the autonomous aircraft may also be displayed on the GUI using the color orange. Activating a mechanism on the input device (e.g., clicking a button) may cause a control menu to appear on the GUI. A user may then use the control menu to provide a desired command. In some embodiments, a user may not be required to provide any alphanumeric input to provide flight commands to the autonomous air vehicle. Commands may be sent using an input device through a series of selections from among pre-defined commands displayed on a control menu.

[0014] Several illustrative embodiments will now be described with reference to the accompanying drawings, which form a part of this specification. The following description provides examples only and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of the embodiments will provide those skilled in the art with an enabling description for practicing one or more embodiments. It will be understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope of the present disclosure. In the following description, for purposes of explanation, specific details are set forth in order to provide a thorough understanding of particular inventive embodiments. It will be apparent, however, that various embodiments may be practiced without these specific details. The figures and descriptions are not intended to be limiting. As used herein, the word "example" or "exemplary" is used to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" or "example" is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0015] An embodiment provides a flight management platform including a server computer and an animated graphical user interface (GUI) for managing, monitoring, and / or communicating with one or more autonomous aircraft. For example, a human operator may be able to simultaneously manage multiple autonomous aircraft using the flight management platform including the animated GUI displayed on a display device on the ground (e.g., at a monitoring center remote from the autonomous aircraft). The operator may send commands to change one or more of the flight parameters (e.g., altitude, attitude, airspeed, heading) of the autonomous aircraft.

[0016] According to various embodiments, an autonomous aircraft is configured to implement automated flight technologies that perform autopilot functions, precision navigation, and sense-and-avoid functions, among other technologies. The flight management platform described herein provides oversight for the autonomous aircraft and the ability to intervene when needed. The flight management platform (e.g., a multi-aircraft management platform) may oversee a flight from start to finish. During the flight, if the autonomous aircraft takes steps to avoid a potential hazard outside of the assigned flight plan, the flight management platform receives a notification or alert, which is then displayed on a GUI. The flight management platform allows a manager (e.g., a human manager) to intervene at any point during the flight by sending new commands for the autonomous aircraft to execute. For example, the manager may use the flight management platform to send a command to the aircraft to divert the aircraft to an alternate landing site.

[0017] 1 shows a block diagram of a flight management platform 1000 that may be used by an administrator 1030 (e.g., a human administrator 1030) to monitor and interact with one or more autonomous air vehicles 1040 individually and / or collectively. According to various embodiments, the administrator 1030 may interact with the flight management platform 1000 using an input device 1050. For example, the administrator 1030 may select an autonomous air vehicle and / or provide commands to the selected autonomous air vehicle using the input device 1050. Details of the input device 1050 are described in more detail below in connection with FIGS. 10-13.

[0018] The flight management platform 1000 may include a server computer 1020 having one or more processors 1004, a system memory 1002 (which may include any combination of volatile and / or non-volatile memory, such as, for example, buffer memory, RAM, DRAM, ROM, flash, or any other suitable memory device), and a network interface (e.g., external communication interface) 1006. In some embodiments, one or more of the modules may be located within one or more components of the system memory 1002 or may be located externally. The software and hardware modules illustrated in FIG. 1 are provided for illustrative purposes only and are not intended to be limiting. The processor 1004, the system memory 1002, and / or the external communication interface 1006 may implement the techniques and / or methods described herein.

[0019] Network interface 1006 may be configured or programmed to receive and generate electronic messages containing information transmitted to or from multiple autonomous air vehicles 1040 through flight management platform 1000.

[0020] Flight management platform 1000 may also include at least one display device 1010 for displaying a graphical user interface (GUI) 1012. When an electronic message is received by flight management platform 1000 via external communication interface 1006 of server computer 1020, the electronic message may be processed and relevant information may be displayed on display device 1010 via GUI 1012. When input is received from administrator 1030 via GUI 1012, the input may be processed and relevant information may be sent to the corresponding autonomous air vehicle 1040. According to various embodiments, flight management platform 1000 may be further configured to receive supplemental information from third parties, such as air traffic control, weather, other aircraft (e.g., aircraft monitoring one or more autonomous air vehicles 1040 in the air), etc. The supplemental information may be processed by flight management platform 1000 and displayed on GUI 1012.

[0021] FIG. 2A illustrates an exemplary animated graphical user interface (GUI) 100 of a flight management platform 1000 for managing and interacting with multiple autonomous aircraft, according to various embodiments. In the exemplary GUI 100, a manager (e.g., manager 1030 shown in FIG. 1 ) oversees five autonomous aircraft, depicted using five aircraft icons on the GUI 100. The GUI 100 may include a first pane 102 depicting each autonomous aircraft with a corresponding aircraft icon 116, 118, superimposed on a map of terrain over which the autonomous aircraft are flying, hovering, and / or grounded in standby mode. The amount of detail to be included in the map may be determined using human factors, including, for example, an assessment of how much information can be safely processed by a human on a screen. Some embodiments may use artificial intelligence (AI) as the manager or a combination of AI and a human manager. The amount of detail to be included in the map may vary depending on the capabilities of the managing entity.

[0022] The GUI 100 may also include a second pane 104 including sections 106, 108 corresponding to each of the aircraft icons 116, 118 shown in the first pane 102. That is, each section 106, 108 provides information associated with a different autonomous aircraft under the administrator's control. In some embodiments, when the administrator selects an autonomous aircraft, for example, by clicking the desired aircraft icon 116, a visual cue may associate the selected aircraft icon 116 displayed on the first pane 102 with the corresponding information section 106 displayed on the second pane 104. For example, the selected aircraft icon 116 and the corresponding information section 106 may be displayed in a different color, tone, and / or font than the rest of the graphical elements of the GUI. The selected aircraft icon 116 and the information section 106 represent the same autonomous aircraft (e.g., the autonomous aircraft 1040 shown in FIG. 1 ). According to various embodiments, the GUI 100 may display only active missions (e.g., missions with an active status) managed by the administrator. There may be additional missions assigned to the administrator (e.g., missions that are not yet active or that are preparing to fly), and this information may be displayed in the appropriate panes 102, 104 of the GUI 100 after the administrator accepts the mission or after the mission assumes active status.

[0023] In some examples, selection of an aircraft icon 116 on the first pane 102 may result in automatic selection of a corresponding information section 106 on the second pane 104. Alternatively, selection of a corresponding information section 106 on the second pane 104 may result in automatic selection of the aircraft icon on the first pane 102. When a server computer coupled to the GUI (e.g., server computer 1020 shown in FIG. 1 ) receives a selection of an aircraft icon 116 from among multiple icons on the first pane 102 of the GUI 100 or a selection of an information section 106 from among multiple sections on the second pane 104 of the GUI 100, the server computer identifies the other of the aircraft icon 116 or the first section 106 on the GUI 100. The aircraft icon 116 and the information section 106 are then displayed using one or more visual cues to distinguish the aircraft icon 116 and the information section 106 from the remaining graphical elements.

[0024] Displaying the aircraft icon and information section in separate locations on the GUI enables accurate identification and selection of the intended autonomous aircraft represented using the aircraft icon and information section. For example, if two autonomous aircraft are flying overlapping, their representations (e.g., aircraft icons) on the map will likely be displayed superimposed on one another. Thus, it may not be possible, or easy, for an administrator to select one of the superimposed autonomous aircraft using the first pane 102. In such cases, the administrator may select the autonomous aircraft with which to interact using the corresponding section on the second pane 104. In this manner, selection of the intended aircraft is ensured.

[0025] As illustrated in the exemplary embodiment shown in the accompanying figures, the first pane 102 is allocated a larger area on the screen than the second pane 104. This area allocation allows aircraft icons to be displayed on a larger map without cluttering the map display (e.g., while also allowing associated information to be displayed in a separate section of the screen (e.g., the second pane 104)). The relative positions and sizes of the first pane 102 and the second pane 104 may vary in various embodiments.

[0026] Flight management platform 1000, through GUI 100, allows an administrator to interact with all autonomous aircraft shown on GUI 100 via corresponding aircraft icons 116, 118 or information sections 106, 108. Prior to takeoff, an autonomous aircraft may autonomously perform a pre-flight inspection, which may include checking one or more of the power systems, control surfaces, avionics, passenger equipment, cabin systems, and / or the environment surrounding the autonomous aircraft. The autonomous aircraft may continuously monitor these systems and / or items during flight as well. The autonomous aircraft may notify flight management platform 1000 (and thereby administrator 1030) (e.g., send data and / or a message) when checks are complete and the autonomous aircraft and passengers (if applicable) are ready for takeoff.

[0027] After receiving notification that the autonomous aircraft is ready for takeoff and / or that a route has been assigned to the autonomous aircraft (discussed further below in connection with FIG. 3 ), the administrator may first select the aircraft icon 116 corresponding to the autonomous aircraft on the first pane 102, send a takeoff command to the autonomous aircraft, for example, by selecting a first button (e.g., a takeoff button) 110 on the second pane 104 (e.g., as illustrated in FIG. 2A , the selected takeoff button 110 is displayed using a visual cue such as a bold font to indicate that the button has been selected), and subsequently select a second button (e.g., an execute button) 112 on the second pane 104 to confirm that the takeoff command is sent to the selected aircraft (e.g., as illustrated in FIG. 2B , the selected execute button 112 is displayed using a visual cue such as a bold font to indicate that the button has been selected). A two-step selection involving selecting two separate buttons to send a command to the autonomous aircraft may ensure that the command is sent intentionally. Thus, the server computer 1020 may receive the selection of the aircraft icon 116 on the first pane 102 of the GUI 100 and may receive a takeoff command and subsequent execute command from the second pane 104 of the GUI 100. The server computer 1020 may then send the takeoff command to the autonomous aircraft represented using the aircraft icon 116 on the GUI 100.

[0028] In the embodiment shown in Figures 2A and 2B, when an administrator selects an icon on the first pane of the GUI or selects an information section on the second pane of the GUI, no additional pop-up windows are displayed. This reduces clutter on the GUI and distractions for the administrator. In some embodiments, pop-up windows or changes to the layout of the GUI are prevented to reduce or eliminate distractions for the administrator.

[0029] In some embodiments, while an autonomous aircraft is following an assigned route, a manager may receive commands from air traffic control (ATC) that need to be relayed to an autonomous aircraft under their control. The GUI display is optimized to allow the manager to quickly identify the correct autonomous aircraft and relay the ATC communications to the correct autonomous aircraft.

[0030] In some embodiments, the administrator may identify the location of the autonomous aircraft on the map displayed on the first pane 102 and determine whether the ATC communication is applicable. For example, if the map shows the autonomous aircraft over mountainous terrain where the maneuver suggested in the ATC communication is unsafe, the administrator may confirm the maneuver with ATC before relaying it to the autonomous aircraft, or may determine that the communication is erroneous and should be discarded based on information provided to the administrator using the map displayed on GUI 100.

[0031] 2A , four of the aircraft may already be in the air and following a route, while a fifth aircraft, represented by aircraft icon 116, may be on the ground awaiting a route assignment. According to various embodiments, a manager may communicate commands to the aircraft, such as assigning a route and giving takeoff orders. Once the autonomous aircraft receives the route assignment, it executes the flight plan to follow the route.

[0032] FIG. 3 illustrates an exemplary technique for assigning a route to an autonomous aircraft using an exemplary animated GUI of a flight management platform, according to various embodiments. GUI 100 can also be used to assign a flight path to an autonomous aircraft. For example, the autonomous aircraft may be on the ground and awaiting a flight assignment. The administrator may select the autonomous aircraft by selecting either the aircraft icon 116 representing the aircraft on the ground (as shown in FIGS. 2A and 2B) or the information section 106. Selection of the autonomous aircraft on the ground can trigger GUI 100 to display a list of available routes 202 using menu 200. Those skilled in the art will appreciate that the graphical elements illustrated in the figures for providing information or receiving input from the administrator are not limited to the means illustrated in the figures, and that alternative means can be used to convey similar information or receive similar input. For example, menu 200 may include a scroll-down menu, a drop-down menu, etc. The administrator may select one of the routes by, for example, clicking the desired route in the list of available routes 202. The administrator may be prompted to confirm the selection by, for example, selecting accept button 204. An accept button 204 for confirming the selection may be displayed on the second pane 104, while the menu 200 with the route 202 may be displayed on the first pane 102. Thus, the confirmation action may require the administrator to move from the first pane 102 to the second pane 104 to intentionally select the accept button 204. This configuration may prevent accidental selections from being made.

[0033] Once a route 206 is selected for assignment to an aircraft, the selected route 206 may be displayed on the map as route 210 displayed from the departure point to the destination point using visual cues, such as using a first color. This may also allow an administrator to visually confirm that the intended route has been assigned to the autonomous aircraft. The departure and destination points 208 of the selected route may be displayed on an information section corresponding to the selected autonomous aircraft. For example, if the intended assignment of a route from Los Angeles to San Francisco was made but the displayed route was from San Diego to Los Angeles, the administrator may realize the error before executing the route assignment command. Once the administrator confirms that the displayed route 210 is the intended route, the administrator may execute the assignment by selecting the accept button 204. Execution of the command may result in the assigned route being displayed on the map using different visual cues, such as using a second color, from the departure point to the destination point to indicate that the route is now a confirmed route. That is, the GUI 100 relays the transition from the selected route to the confirmed route.

[0034] A server computer 1020 coupled to GUI 100 may continuously receive flight parameters and position positioning signals of multiple autonomous air vehicles being monitored using GUI 100. The server computer may periodically update GUI 100 using the flight data and position positioning signals of the autonomous air vehicles to display the relative position of first icon 116 with respect to displayed route 210 assigned to the autonomous air vehicle as the autonomous air vehicle progresses along displayed route 210.

[0035] According to various embodiments, an administrator can choose the amount of detail to be shown on the map. For example, GUI 100 can have layers of information 250 that can be displayed superimposed. The administrator can select one or more of the available layers (e.g., radius information around the aircraft 252, wind information 254, measurement information received from one or more aircraft 256, and optional additional information 258) to have a desired level of detail on the map.

[0036] 4A-4B illustrate various levels of detail displayed on a GUI of a flight management platform, according to various embodiments. Specifically, FIG. 4A illustrates the addition of a layer onto a displayed exemplary animated GUI, and FIG. 4B illustrates the removal of a layer from the displayed animated GUI. FIG. 4A illustrates a visual aid 302 for indicating the surroundings of an autonomous aircraft represented using an aircraft icon 116. For example, visual aid 302 may be in the form of concentric rings, with each ring corresponding to a predetermined distance from the autonomous aircraft. In this manner, a supervisor may quickly visually assess whether other aircraft or hazards are present in close proximity to the monitored autonomous aircraft. Similarly, visual aid 302 may also convey information about the distance between the monitored autonomous aircraft and terrain elements (e.g., mountains) near the monitored autonomous aircraft.

[0037] If an administrator prefers a cleaner display, the administrator can remove visual aid 302 from the GUI, as shown in FIG. 4B. For example, visual aid 302 can be displayed when radius layer 252 (shown in FIG. 3) is selected and can be removed when radius layer 252 is not selected on GUI 100. Some of the elements of the GUI may not be subject to masking (e.g., some elements are always displayed and cannot be removed by toggling layers), while other elements may be displayed according to selected options provided on the GUI (e.g., based on the selection of map layer 250 shown in FIG. 3).

[0038] 4-5, the aircraft icons 116, 118 may have additional information 400 displayed on the map in association with the aircraft icon. For example, flight parameters such as heading, altitude, and / or speed information associated with the autonomous aircraft represented by the respective aircraft icon 116, 118 may be displayed next to the aircraft icon 116, 118. In some embodiments, the information 400 may be attached to the aircraft icon and may move with the aircraft icon along the displayed path.

[0039] As illustrated in FIG. 5 , the animated GUI may display an aircraft icon 116 moving along a route 120 assigned to that aircraft. As the autonomous aircraft progresses along its assigned route (e.g., flight path), the corresponding aircraft icon 116 moves along the route 120 on the map. The displayed information 400 moves along with the aircraft icon 116. By having all the information on the same GUI, the techniques provided herein ensure that an administrator assesses all relevant information associated with an autonomous aircraft and selects the correct autonomous aircraft when activating a route or communicating a command to the autonomous aircraft. In some embodiments, the administrator may toggle between displaying and hiding the displayed information 400. That is, flight parameters associated with the autonomous aircraft may be selectively displayed on the GUI near a first icon representing the autonomous aircraft.

[0040] According to various embodiments, an autonomous aircraft being managed using the animated GUI 100 is flying autonomously using instrument flight rules (IFR). If ATC instructs an administrator to communicate new parameters (e.g., altitude, heading, speed, etc.) to the autonomous aircraft, the administrator may do so by selecting the aircraft icon 116 and sending a command with the new parameters. In some embodiments, the autonomous aircraft may acknowledge receipt of the command and / or execution of the command by a message sent back to the server computer and displayed on the GUI 100.

[0041] In some embodiments, the GUI may be displayed at various zoom levels selected by an administrator or based on a selected action. For example, when assigning a flight path to an autonomous aircraft, the GUI may be displayed at a zoom level that shows both the start and end points of the path. For example, when entering a flight parameter change command, the GUI may be displayed at a zoom level that better shows the environment surrounding the autonomous aircraft to ensure that the flight parameter modification does not pose an imminent threat to the autonomous aircraft.

[0042] FIG. 6 shows a zoomed-in view of an exemplary animated GUI of a flight management platform, according to various embodiments. As illustrated in FIG. 6 , a flight route can be formed by connecting multiple waypoints 552, 554, 556. In some embodiments, a segment 560 (e.g., a portion) between the waypoints can change color as an autonomous aircraft flies over the segment. For example, in the exemplary embodiment shown in FIG. 6 , segment 560 (e.g., a completed portion of the route) is shown in a first color (e.g., white) to indicate that segment 560 of the route is completed, while segment 562 (e.g., the remaining portion of the route) is shown in a second color (e.g., purple) to indicate that the segment is not yet completed. These visual cues can help a manager gather various types of information simultaneously by simply looking at or observing the GUI. Using the visual cues, a manager can stay aware of the status of multiple autonomous aircraft under their management.

[0043] According to various embodiments, when an aircraft icon 116 is selected on the GUI, an administrator may select a waypoint on the planned route of the aircraft represented by the aircraft icon 116 and have the aircraft fly directly through the selected waypoint. For example, if the aircraft is configured to fly from waypoint A to waypoint B to waypoint C, the administrator may select waypoint C and instruct the aircraft to fly from waypoint A directly to waypoint C, skipping waypoint B. As with the various commands described herein, one or more selections entered on the GUI are transmitted to the autonomous aircraft by a server computer coupled to the GUI.

[0044] Once the autonomous aircraft completes its flight plan (e.g., arrives at the destination point), the aircraft icon 116 may be indicated by one or more visual cues indicating that the autonomous aircraft has completed its flight and landed.

[0045] FIG. 7 shows another view of an exemplary animated GUI including a flight guidance pane, according to various embodiments. The exemplary GUI 600 illustrated in FIG. 7 includes a first pane 602 (similar to the first pane 102 of FIG. 2A ) having multiple icons 601, 603, each representing an autonomous aircraft managed using the GUI 600. The exemplary GUI 600 further includes a second pane 604 (similar to the second pane 104 of FIG. 2A ) having multiple information sections, each corresponding to one of the displayed icons on the first pane 602. The first pane 602 and the second pane 604 may show active (e.g., having an active status or executing a flight plan) missions (e.g., flights). The exemplary GUI 600 further includes a third pane 606 displaying a list 650 of missions assigned to an administrator of the GUI 600. Each mission assignment represents an additional autonomous flight assigned to the administrator for monitoring by the GUI. That is, the third pane 606 shows autonomous aircraft that have been assigned but are not currently managed using the GUI 600. For example, the list of missions 650 may include missions that are waiting for a manager (e.g., an operator) to accept responsibility for them. Once the manager accepts the assigned mission, final preparations before departure may be completed. In some embodiments, as described above, the missions on the list 650 may not include all assigned missions, but may include only assigned missions that are in a pre-preparation state. Accepting an assigned mission (e.g., autonomous flight) is described below in connection with FIG. 8.

[0046] According to various embodiments, flight management platform 1000 may display messages reporting one or more upcoming tasks that may require the administrator 1030's approval or acceptance. For example, reminders of upcoming tasks may be displayed preferentially for a predetermined period of time, along with information necessary to avoid human error and poor human performance (e.g., delayed responses to time- and / or safety-critical events). In some embodiments, the reminder message may have a countdown displayed therein indicating the time remaining for the administrator 1030 to act on the reminder.

[0047] Continuing with the description of the exemplary GUI 600, a fourth pane 608 is provided to show a zoomed map view of a selected autonomous aircraft. For example, if icon 601 is selected on first pane 602, a corresponding information section 605 is identified on second pane 604 using visual cues (as described above in connection with FIGS. 2A and 2B ), and the immediate area around the selected autonomous aircraft is illustrated in pane 608 using a zoomed-in map. The exemplary GUI 600 also includes a flight guidance pane 610. The GUI 600 may receive modifications on flight guidance pane 610 corresponding to desired changes to one or more flight parameters associated with the selected autonomous aircraft (e.g., the aircraft represented by 601 and information section 605 in the exemplary GUI of FIG. 7). The server computer sends commands to the autonomous aircraft to effect the desired changes. According to various embodiments, flight guidance pane 610 may include one or more of a control dial 612 or 614 , a drop-down menu, a text entry field, or a widget 620 .

[0048] For example, first control dial 612 may allow an administrator to input a heading setting or modification for a selected autonomous aircraft. The GUI may display possible headings in predetermined increments. Once the administrator makes a selection, the selected heading may be indicated using visual cue 618. Second control dial 614 may allow an administrator to input an altitude setting or modification for a selected autonomous aircraft. The GUI may display possible altitudes in predetermined increments. Once the administrator makes a selection, the selected heading may be indicated using visual cue 616. In some embodiments, flight parameter settings or modifications may be entered into text entry fields. When one or more of the flight parameter settings or modifications are complete, the administrator may select widget 620 (e.g., the “Execute” widget) to cause the server computer to send a command to the autonomous aircraft to implement the one or more flight parameter settings or modifications.

[0049] FIG. 8 shows another view of an example animated GUI of a flight management platform after a mission assignment has been accepted, according to various embodiments. As described above, the example GUI 600 illustrated in FIG. 7 includes a third pane 606 displaying a list 650 of missions assigned to the administrator of GUI 600. When the administrator accepts one of the missions, the mission is moved from the third pane to a second pane that displays a list of active autonomous flights managed by the administrator. As shown in FIG. 8, the administrator has accepted mission 702, which has been removed from the third pane 705 and moved to the second pane 740. The second pane 740 displays all autonomous air vehicles managed and / or operated by the administrator, whether they are in the air or on the ground (e.g., waiting to start a mission or recently landed as part of a mission).

[0050] When the administrator accepts the new mission, flight guidance pane 610 temporarily switches to configuration view 710. Configuration view 710 provides additional information associated with the newly accepted mission, such as the departure and destination points, estimated departure time, aircraft state of charge, and flight check status. Configuration view 710 may include a text area 704 for displaying additional information associated with the newly accepted mission and one or more command widgets 708. When a takeoff command is selected on the GUI, the server computer sends a takeoff command to the autonomous aircraft, and configuration view 710 switches back to flight guidance pane 610 as shown in FIG. 7 .

[0051] Autonomous flights operated by an administrator may be represented using icons on a first pane 730 of the GUI. According to various embodiments, an icon representing an autonomous aircraft in flight 732 may have a different representation than an icon 734 representing an autonomous aircraft on the ground (e.g., an autonomous aircraft preparing for flight or that recently landed). An icon representing a newly accepted mission may not yet be associated with a flight plan (e.g., a route) on the map. The icon may be associated with a flight plan or route on the GUI once an execute takeoff command is sent by the server computer to the autonomous aircraft.

[0052] 8, the zoomed-in map pane 720 may provide a close-up aerial view of the surroundings of the newly accepted mission. The map pane 720 may also show the current status of the autonomous aircraft. For example, because the selected autonomous aircraft corresponds to the newly accepted mission 702, the status of the autonomous aircraft is shown as "Transition Level 1, Transition-Climb," which means that the autonomous aircraft is transitioning to vertical flight mode for takeoff or climb.

[0053] According to various embodiments, a server computer coupled to the GUI may receive input selecting a mission assignment displayed in pane 705. For each selected mission assignment, a separate icon 734 is generated in the first pane 730 and a corresponding separate section 702 representing information associated with the selected mission assignment is generated in the second pane 740. The accepted mission is then removed from the third pane 705 of the GUI.

[0054] According to various embodiments, an entity (e.g., ATC, a third party, another aircraft) may send an alert to an administrator. In some embodiments, the alert may be received at a server computer in communication with the flight management platform and / or the GUI, and a visual alert may be displayed on the GUI. The administrator may choose to ignore the displayed alert or act accordingly. In some embodiments, the GUI may also display proposed changes or modifications to flight parameters of the monitored autonomous aircraft. The administrator may accept the proposed changes and send commands to the autonomous aircraft to change the flight parameters. In some embodiments, in response to the alert, the administrator may use a flight guidance pane of the GUI to manually enter the desired changes to the flight parameters. The server computer may then send the commands to the autonomous aircraft using the GUI.

[0055] In some embodiments, an autonomous aircraft may be configured to change course to avoid a collision with a hazard or to avoid a mission failure due to, for example, an onboard electromechanical problem. The autonomous aircraft may change course without receiving approval from a manager. In some embodiments, the autonomous aircraft may be configured to wait for approval for a predetermined amount of time before implementing a change in flight plan.

[0056] According to various embodiments, a dedicated area of ​​the GUI may be provided for displaying alert messages. For example, feedback or data received from sensors or other equipment on the autonomous aircraft indicating a problem with the aircraft may be displayed as an alert in the dedicated area. In some embodiments, given the importance of the alert, the alert may be displayed as a pop-up screen over a main pane (e.g., the first pane) of the GUI.

[0057] FIG. 9 shows an example warning displayed using an example animated GUI, according to various embodiments. In some embodiments, an autonomous aircraft may detect a hazard 800 on its path 802 and deviate from its path 802 to avoid collision with the hazard 800. For example, the autonomous aircraft may veer left or right, changing its path to an alternate path 804 that avoids collision with the hazard 800. In some embodiments, the autonomous aircraft may need to change its path due to an issue identified with the autonomous aircraft, such as a propeller (e.g., fan) failure. The alternate path 804 and a collision avoidance alert (e.g., warning) 806 are displayed on the GUI to notify an administrator. In some embodiments, upon communicating the hazard 800 to the flight management platform, the autonomous aircraft may request a flight trajectory update. The flight trajectory update request may be displayed on the GUI. The administrator may provide a response to the flight trajectory update request by sending changes to the autonomous aircraft's flight parameters or flight plan via a server computer.

[0058] When alert 806 is displayed on the GUI, the GUI may switch the selected aircraft icon to the aircraft icon associated with alert 806. For example, if the alert was received from, or is otherwise associated with, the autonomous aircraft represented by icon 805 and corresponding section 822, the GUI may display icon 805 and section 822 as the selected autonomous aircraft. In some embodiments, if the alert is displayed because of a change already performed by the autonomous aircraft, the alert may be displayed for a predetermined amount of time before switching back to the aircraft last selected by the administrator.

[0059] In some embodiments, the ATC may recommend a flight plan change, which may be displayed as an alert on the GUI. The alert may further include a widget displayed on the GUI for the administrator to accept or reject the recommended flight plan change. If the administrator accepts the recommended flight plan change, selecting the widget to accept the recommended flight plan change may cause the server computer to send a command to the autonomous aircraft to activate the recommended flight plan change (e.g., change heading as indicated in the recommended flight plan change). The autonomous aircraft executes the received command and begins following the revised flight plan.

[0060] An embodiment provides a flight management platform (e.g., flight management platform 1000 shown in FIG. 1 ) for implementing the techniques described herein. Information received from a monitored autonomous aircraft or from any other source is analyzed and / or processed by a server computer (e.g., server computer 1020) and sent to a GUI (e.g., GUI 1012) for display. Similarly, any input or selection received at the GUI is analyzed and / or processed by the server computer and sent to the intended autonomous aircraft via a secure communication channel between the server computer and the autonomous aircraft.

[0061] FIG. 10 illustrates an exemplary input device, according to various embodiments. The exemplary input device 1050 may be communicatively coupled to the server computer 1020. The input device 1050 may communicate with the server computer 1020 via a wired or wireless connection. The input device 1050 may include multiple input mechanisms 1052, 1054, 1056 associated with specific flight parameters. The input device 1050 may be controlled by the administrator 1030 and may send a signal based on the selection (e.g., activation) of one or more of the input mechanisms 1052, 1054, 1056. The exemplary input mechanisms may include, but are not limited to, any combination of buttons, switches, wheels, trackpads, and touch sensors. Each input mechanism 1052, 1054, 1056 may be assigned to a specific flight parameter (e.g., heading, true airspeed, altitude). Selecting (or activating) an input mechanism 1052 , 1054 , 1056 sends a signal from the input device 1050 to the server computer 1020 .

[0062] For example, input device 1050 may include a multi-button mouse (e.g., a three-button mouse). A first input mechanism (e.g., a first button, switch, touch pad, pointer) 1052 may be dedicated to commands associated with a first flight parameter (e.g., heading), a second input mechanism (e.g., a second button, switch, touch pad, pointer) 1054 may be dedicated to commands associated with a second flight parameter (e.g., airspeed, true airspeed), and a third input mechanism (e.g., a third button, switch, touch pad, pointer) 1056 may be dedicated to commands associated with a third flight parameter (e.g., altitude) of the aircraft. According to various embodiments, the input mechanisms may be of the same type (e.g., all input mechanisms are buttons or wheels, etc.). Alternatively, each input mechanism may be of a different type (e.g., a first input mechanism associated with a first parameter may be a button and a second input mechanism associated with a second parameter may be a wheel, etc.). In some embodiments, input device 1050 may optionally include one or more additional input mechanisms. For example, input device 1050 may include optional side buttons 1058 for providing additional input (e.g., input other than selecting or modifying flight parameters).

[0063] In some embodiments, each input mechanism may be associated with a sensory cue. For example, each input mechanism may be color coded. The same color may be used to indicate commands associated with flight parameters on GUI 100. In other embodiments, each input mechanism may have a unique tactile texture. The same texture (e.g., hashing in a first direction, a dotted texture) may be used to indicate flight parameters on GUI 100.

[0064] Those skilled in the art will appreciate that an input device including multiple input mechanisms is not limited to the structure shown in FIG. 10, but may take any shape or form, including a dedicated input mechanism for each unique parameter that the administrator may change and / or control.

[0065] According to various embodiments, the administrator 1030 may hover a cursor over an icon representing an actual aircraft (e.g., icon 116 shown in FIG. 1 ) displayed on the GUI 100. The server computer 1020 may detect the cursor hovering over the icon for a predetermined period of time. When the predetermined period of time has elapsed, the server computer 1020 may determine the hover as an input to select an icon from among the icons displayed on the GUI 100.

[0066] Using input device 1050 and the techniques described herein, administrator 1030 can send commands to the selected aircraft to change flight parameters (e.g., airspeed, heading, or altitude). For example, administrator 1030 can send commands to the aircraft to slow down, take a new heading, or establish a new altitude.

[0067] According to an exemplary embodiment, a cursor controlled via the input device 1050 may be displayed on the GUI 100. An exemplary cursor 1108 is shown in FIG. 11 and described in more detail below. In some embodiments, the administrator 1030 may first use the cursor 1108 to select a desired aircraft (e.g., hover the cursor over the desired aircraft and select the aircraft) before sending a command to the aircraft. Alternatively, the display device 1010 may be a touch screen, and the administrator 1030 may select a desired aircraft by touching an icon representing the aircraft. In other embodiments, the administrator 1030 may hover the cursor over an icon representing the autonomous aircraft on the GUI 100 (e.g., hover the cursor over the icon for a predetermined amount of time, which may be determined by the server computer 1020 to have selected the icon) and activate a first input mechanism of the input device 1050 to display a desired command associated with the flight parameters assigned to the first input mechanism. For example, a first menu may be displayed 3 to 5 seconds after the cursor is hovered over the first icon. For example, to adjust the heading of the autonomous air vehicle, the administrator 1030 may activate (e.g., click or click and hold) the first input mechanism 1052 assigned to the flight parameter "Heading."

[0068] According to some embodiments, activating (e.g., pressing or squeezing) input mechanism 1052, 1054, or 1056 while the cursor is over an aircraft icon displayed on the GUI may trigger menus (e.g., radial menus, widgets) 1102, 1104, 1106 to be displayed on the GUI, as illustrated in Figure 11. For example, activating the first input mechanism 1052 may trigger the first radial menu 1102 to be displayed on the GUI, activating the second input mechanism 1054 may trigger the second radial menu 1104 to be displayed on the GUI, and activating the third input mechanism 1056 may trigger the third radial menu 1106 to be displayed on the GUI.

[0069] The menus 1102, 1104, 1106 may enable the administrator 1030 to modify the flight of the aircraft. Each menu 1102, 1104, 1106 may be associated with one or more preset commands that modify one or more flight parameters of the selected aircraft. Each menu 1102, 1104, 1106 may include multiple preset commands 1106. According to various embodiments, the menus 1102, 1104, 1106 may be in the form of a rosette (e.g., having a circular format) that includes a central element and multiple preset commands arranged around the central element. The example menus 1102, 1104, 1106 illustrated in FIG. 11 are radial menus. However, the menus may be provided in other forms that enable the features described herein to be implemented.

[0070] While menus 1102, 1104, or 1106 are displayed on the GUI, administrator 1030 may control a cursor using input device 1050 to select a desired command. As illustrated in FIG. 11 , each menu 1102, 1104, or 1106 includes multiple preset commands. According to various embodiments, the preset commands may be displayed in a circular format around a central element that identifies a flight parameter. The placement of the preset commands may be based on at least one of the orientation or magnitude of each preset command.

[0071] Referring again to FIG. 11 , menu 1102 is associated with aircraft heading as a flight parameter. According to various embodiments, each input mechanism and displayed menu pair is associated with the same sensory cue. Here, center element 1122 identifies heading as the flight parameter. As illustrated in FIG. 11 , center element 1122 may be represented using the same sensory cue (e.g., visual cue, such as color) as input mechanism 1052 (shown in FIG. 10 ) associated with the same flight parameter (e.g., heading). Preset commands 1105 are arranged around center element 1122. Preset commands provided for specifying a right heading angle are provided on the right side of menu 1102. Preset commands provided for specifying a left heading angle are provided on the left side of menu 1102. The preset commands for a right heading are arranged in ascending clockwise order. The preset commands for a left heading are arranged in ascending counterclockwise order. Thus, the arrangement of the preset commands 1105 is based on at least one of the direction or magnitude of each preset heading command. A "continue flight plan" preset command 1124 is also represented in the menu 1102. According to various embodiments, the menu 1102 may also include preset commands that allow the administrator 1030 to input a desired heading amount based on the desired heading direction. For example, preset command 1126 is a preset command for a right heading, allowing the administrator to input the desired heading in preset command 1126. For example, the administrator 1030 may use an input mechanism of the input device 1050 to provide the desired heading. In some embodiments, the desired input may be entered using a scroll wheel or continuous taps on a selected input mechanism. Selecting one of the pre-defined commands 1105 (e.g., the "left 360" command illustrated in FIG. 11) using cursor 1108 via input element 1050 sends a "turn left and rotate 360 ​​degrees" command to the selected aircraft. The aircraft's control computer then executes the received command.

[0072] Menu 1104 is associated with aircraft true airspeed as a flight parameter. Central element 1132 identifies true airspeed as the flight parameter. As illustrated in FIG. 11 , central element 1132 may be represented using the same sensory cues (e.g., visual cues such as color) as input mechanism 1054 (shown in FIG. 10 ) associated with the same flight parameter (e.g., true airspeed). Preset commands 1110 are arranged around central element 1132. Preset commands for increasing true airspeed are provided in the upper half of menu 1104. Preset commands for decreasing true airspeed are provided in the lower half of menu 1104. The absolute values ​​of the preset commands 1110 increase from left to right. Thus, the arrangement of the preset commands 1110 is based on at least one of the magnitude and value (or absolute value) of each preset true airspeed command. Selecting one of the pre-set commands 1116 (e.g., the "+5 KTS" command illustrated in FIG. 11) using cursor 1108 via input element 1050 sends a "increase airspeed by 5 knots" command to the selected aircraft. The aircraft's control computer then executes the received command.

[0073] Menu 1106 is associated with aircraft altitude as a flight parameter. Central element 1142 identifies altitude as the flight parameter. As illustrated in FIG. 11 , central element 1142 may be represented using the same sensory cues (e.g., visual cues such as color) as input mechanism 1056 (shown in FIG. 10 ) associated with the same flight parameter (e.g., true airspeed). Preset commands 1112 are arranged around central element 1142. Preset commands for increasing altitude are provided in the top half of menu 1106. Preset commands for decreasing altitude are provided in the bottom half of menu 1106. The absolute values ​​of the preset commands 1110 increase from left to right. Thus, the arrangement of the preset commands 1110 is based on at least one of the magnitude and value (or absolute value) of each preset altitude command. Selecting one of the pre-set commands 1116 (e.g., the "+100 feet" command shown in FIG. 11) using cursor 1108 via input element 1050 sends a "Gain altitude 100 feet" command to the selected aircraft. The aircraft's control computer then executes the received command.

[0074] Those skilled in the art will understand that the arrangement of the pre-configured commands is not limited to the layout shown in FIG. 11 and that any layout (e.g., a layout that would be intuitive to the administrator 1030) can be selected for any of the menus described above.

[0075] According to various embodiments, once the administrator 1030 has completed sending the desired commands to the aircraft, the menus 1102, 1104, or 1106 disappear from the GUI, reducing clutter on the GUI 100 and allowing the administrator 1030 to focus on the aircraft under their control.

[0076] In some embodiments, the administrator 1030 may press or press and hold a key (e.g., a "Ctrl" key) on a keyboard coupled to the input device 1050 while activating the input mechanism 1052, 1054, or 1056. The key may also be located on the input device 1050 (e.g., on a surface such as a side of the input device 1050). Simultaneous activation of multiple buttons and / or keys may ensure that commands being sent to the aircraft are intended and not accidental.

[0077] In some embodiments, the most common commands may be placed in menus 1102, 1104, 1106, within easy reach of the administrator 1030. In some embodiments, a center element 1122, 1132, 1142 indicating a selected parameter (e.g., heading, airspeed, altitude) may be placed in the center of the aircraft icon selected by the administrator 1030. In this way, the administrator 1030 has increased visual feedback that a command is being sent to the selected aircraft represented by the aircraft icon without having to move the cursor 1108 away from the aircraft icon.

[0078] FIG. 12 shows another exemplary display of multiple aircraft simultaneously controlled by a controller. Each aircraft icon 1202, 1204 may be associated with an autonomous aircraft controlled by the same administrator 1030. As illustrated in FIG. 12 , the aircraft icons 1202, 1204 may be minimal representations of the aircraft without incorporating elaborate design features to further reduce GUI clutter. The aircraft's flight path (e.g., track) 1206 may be indicated using a line having a first visual cue (e.g., a first color). One or more predetermined flight parameters may be displayed in the form of a legend 1208 near the aircraft icon 1202. For example, heading, airspeed, and altitude information may be displayed using colors associated with the input mechanisms 1052, 1054, 1056 of the input device 1050 and menus 1102, 1104, 1106 associated with each specific flight parameter.

[0079] The use of consistent sensory cues (e.g., the same color to represent distinctive parameters, such as a first color for heading, a second color for airspeed, and a third color for altitude) reduces human error (e.g., sending the wrong command or sending a command to the wrong aircraft).

[0080] Commands sent to the aircraft may also be displayed on the GUI. As illustrated in FIG. 12 , legend 1210 shows the current heading (e.g., heading 315°) and target parameters (e.g., heading 045°) of the aircraft associated with icon 1204. Legend 1210 shows the previous command, or current flight parameters, and the target flight parameters. That is, the GUI shows the maneuver the aircraft will soon perform. Once the target is achieved, the previous flight parameters disappear and the achieved target (now the current parameters) are displayed instead. In the example shown in FIG. 12 , once the aircraft represented by icon 1204 is oriented to a heading of 045°, the number “315” disappears and the legend shows “HDG045” (e.g., an implemented command sent to the aircraft using flight management platform 1000 described herein).

[0081] According to various embodiments, the administrator 1030 may desire to display the terrain surrounding the aircraft. The GUI may include an option for the administrator 1030 to turn the display of the terrain on or off. For example, the GUI may include a widget 1212 that can be selected to reveal or mask the terrain. In some embodiments, the terrain may not be displayed by default to reduce GUI clutter. The GUIs described herein show multiple aircraft simultaneously. Because each aircraft may be at a different altitude than the rest of the aircraft, permanently displaying the terrain may increase the likelihood of an administrator error. However, the administrator 1030 may desire to see the terrain and may use the widget 1212 to turn the display of the terrain on or off. The widget 1212 may be controlled (activated, deactivated) using the input device 1050. For example, clicking, clicking and holding, or hovering over the widget 1212 may activate the widget and display the terrain on the GUI. The terrain may be removed from the GUI by releasing the widget with the input device, hovering away from the widget, or clicking on it again. For example, widget 1212 may be selected using an additional input mechanism 1058 of input device 1050 (e.g., an input mechanism not assigned to a specific flight parameter).

[0082] Figure 13 is a flowchart of an example process 1300. In some embodiments, one or more process blocks of Figure 13 may be implemented by a flight management platform.

[0083] In step 1302, a graphical user interface (GUI) is displayed on a display device using the server computer. An exemplary GUI 100 is shown in FIG. 2A.

[0084] In step 1304, the plurality of autonomous air vehicles are represented by a plurality of icons on the GUI. For example, referring again to FIG. 2A , each of icons 116, 118 represents a different autonomous air vehicle on GUI 100. In some embodiments, information about one or more flight parameters associated with each of the plurality of autonomous air vehicles may be displayed near the icon representing the autonomous air vehicle on the GUI using distinctive sensory cues for each flight parameter. For example, as illustrated in FIG. 12 , each flight parameter may be assigned a color and displayed near the corresponding icon using the assigned color.

[0085] In step 1306, the server computer receives input selecting a first icon from among multiple icons on the GUI. For example, the server computer may receive input (e.g., a signal sent from an input device to the server computer) to select icon 116 of FIG. 2A. In some embodiments, the server computer may detect a cursor hovering over the first icon for a predetermined period of time. The cursor is controlled by an input device communicatively coupled to the server computer. When the predetermined period of time has elapsed, the server computer may determine the icon under the cursor as selected.

[0086] In step 1308, the server computer receives a first signal from an input device including multiple input mechanisms. Each input mechanism is assigned to a unique flight parameter. The first signal is generated by selecting a first input mechanism of the input device, and a first component is associated with the first flight parameter. An exemplary input device is shown in FIG. 10. Input device 1050 includes input mechanisms 1052, 1054, and 1056, each assigned to control the aircraft's heading, airspeed, and altitude. For example, selecting input mechanism 1052 assigned to true airspeed generates a first signal and sends it to the server computer to display a menu associated with true airspeed. According to various embodiments, the input mechanism of the input device associated with the first flight parameter is identified on the input device using a unique sensory cue assigned to the first flight parameter. For example, if a first color is assigned to true airspeed, the input mechanism associated with true airspeed is marked on the input device with the first color.

[0087] In step 1310, a first menu of the plurality of menus is displayed on the GUI in response to the server computer receiving the first signal. The first menu is associated with a first flight parameter of the autonomous aircraft represented by the first icon. The first menu includes a plurality of first preset commands. For example, the first menu may include one of menus 1102, 1104, or 1106 shown in FIG. 11 . In response to an input mechanism being activated on the input device, a menu associated with the flight parameter assigned to the activated input mechanism is displayed on the GUI. According to various embodiments, the menu may be in the shape of a rosette including a central element and a plurality of preset commands arranged around the central element. In some embodiments, the first preset commands are displayed in a circular format around the central element identifying the first flight parameter. The placement of the first preset commands may be based on at least one of the orientation or magnitude of each preset command.

[0088] In step 1312, the server computer receives a selection of a first command from among a plurality of first preset commands. The administrator may use an input device to select one of the preset commands in the menu.

[0089] In step 1314, the server computer sends a first command to the first autonomous aircraft represented by a first icon on the GUI. A control computer of the first autonomous aircraft receives and implements the first command. In some embodiments, information about one or more flight parameters associated with each of the plurality of autonomous aircraft may be displayed near the icon representing the autonomous aircraft on the GUI. For example, the information may include values ​​of the one or more current flight parameters and a value of the first flight parameter specified in the first command. For example, the current value of the first flight parameter and a target value of the flight parameter in the view of the first command may be displayed near the first icon.

[0090]

[0009] Embodiments further provide a system comprising: a display screen; one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the steps described herein. The system may further comprise an input device including a plurality of input mechanisms, each input mechanism assigned to a unique flight parameter, the flight parameter including one of heading, airspeed, and altitude.

[0091] In the foregoing specification, examples of the present disclosure have been described with reference to numerous specific details that may vary from embodiment to embodiment. Therefore, the specification and drawings should be considered in an illustrative rather than a restrictive sense. The sole and exclusive indication of the scope of the present disclosure, and what is intended by the applicant to be the scope of the present disclosure, is the set of claims issuing from this application, in the specific form in which such claims issue, literally and in their equivalents, including any subsequent modifications. Specific details of particular examples may be combined in any suitable manner without departing from the spirit and scope of the examples of the present disclosure.

[0092] Additionally, spatially relative terms such as "bottom," "top," or "side" may be used to describe the relationship of an element and / or feature to another element and / or feature, for example, as shown in the figures. It will be understood that spatially relative terms are intended to encompass different orientations of the device in use and / or operation in addition to the orientation depicted in the figures. For example, if the device in the figures were inverted, an element described as being on the "bottom" side might then be oriented "above" the other element or feature. The device may be oriented in other ways (e.g., rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein would be interpreted accordingly.

[0093] The methods, systems, and devices discussed herein are examples. Various embodiments may omit, substitute, or add various procedures or components as appropriate. For example, features described with respect to particular embodiments may be combined in various other embodiments. Different aspects and elements of the embodiments may be similarly combined. Furthermore, technology evolves, and therefore, many of the elements are examples that do not limit the scope of the disclosure to those particular examples.

[0094] As used herein, the terms "and," "or," and "and / or" can include a variety of meanings that are expected to depend, at least in part, on the context in which such terms are used. Typically, "or," when used to associate a list such as A, B, or C, means A, B, and C, where it is used in an inclusive sense, and "or" when used to refer to A, B, or C, where it is used in an exclusive sense. Additionally, as used herein, the term "one or more" can be used to describe any feature, structure, or characteristic in the singular, or can be used to describe a certain combination of features, structures, or characteristics. However, it should be noted that this is merely an illustrative example, and claimed subject matter is not limited to this example. Furthermore, the term "at least one of," when used to associate a list such as A, B, or C, can be interpreted to mean any combination of A, B, and / or C, such as A, B, C, AB, AC, BC, AA, AAB, ABC, AABBCCC, etc.

[0095] References throughout this specification to "one example," "an example," "particular example," or "exemplary embodiment" mean that a particular feature, structure, or characteristic described in connection with a feature and / or example may be included in at least one feature and / or example of the claimed subject matter. Thus, the appearances of the phrases "in one example," "an example," "in a particular example," "in a particular embodiment," or other similar phrases in various places throughout this specification do not necessarily all refer to the same features, examples, and / or limitations. Furthermore, particular features, structures, or characteristics may be combined in one or more examples and / or characteristics.

[0096] In the foregoing detailed description, numerous specific details have been set forth to provide a thorough understanding of the claimed subject matter. However, it will be understood by those skilled in the art that the claimed subject matter may be practiced without these specific details. In other instances, methods and apparatuses known by those skilled in the art have not been described in detail so as not to obscure the claimed subject matter. Therefore, it is intended that the claimed subject matter not be limited to the particular examples disclosed, but that such claimed subject matter may include all embodiments falling within the scope of the appended claims and equivalents thereof.

Claims

1. 1. A computer-implemented method for monitoring and interacting with a plurality of autonomous air vehicles, comprising: displaying a graphical user interface (GUI) on a display device using a server computer; representing, by the server computer, a plurality of autonomous air vehicles with a plurality of icons displayed on the GUI; receiving, by the server computer, an input selecting a first icon from the plurality of icons displayed on the GUI; receiving, by the server computer, a first signal from an input device including a plurality of input mechanisms, each input mechanism being assigned to a unique flight parameter; displaying, by the server computer, on the GUI in response to receiving the first signal, a first menu from a plurality of menus, the first menu being associated with a first flight parameter of the autonomous air vehicle represented by the first icon, the first menu including a plurality of first preset commands; receiving, by the server computer, a selection of a first command from among the plurality of first preset commands; transmitting, by the server computer, the first command to a first autonomous air vehicle represented by the first icon on the GUI; A method comprising:

2. The method of claim 1 , wherein the first signal is generated by activating a first input mechanism of the input device, the first input mechanism being associated with the first flight parameter.

3. The method of claim 1 , wherein the first flight parameter is one of a heading, an airspeed, or an altitude.

4. displaying information associated with one or more flight parameters associated with each of the plurality of autonomous aircraft on the GUI near the icons representing the autonomous aircraft using distinctive sensory cues for each flight parameter, the one or more flight parameters including one or more of an altitude, a heading, or a speed of an autonomous aircraft represented by each of the plurality of icons. The method of claim 1 further comprising:

5. 5. The method of claim 4, wherein the input mechanism of the input device associated with the first flight parameter is identified on the input device using the distinctive sensory cue assigned to the first flight parameter.

6. The method of claim 4 , wherein the information includes values ​​of one or more current flight parameters and values ​​of the first flight parameter specified in the first command.

7. receiving an input selecting a first icon; detecting a cursor hovering over the first icon for a predetermined time, the cursor being controlled by the input device communicatively coupled to the server computer; The method of claim 1 further comprising:

8. displaying, adjacent to the first icon, a current value of the first flight parameter and a target value of the flight parameter in the first command view. The method of claim 1 further comprising:

9. 10. The method of claim 1, wherein the first preset commands are displayed in a circular format around a central element identifying the first flight parameter, and placement of the first preset commands is based on at least one of an orientation or a magnitude of each preset command.

10. A display device; one or more processors; an input device including a plurality of input mechanisms, the input device communicatively coupled to the one or more processors; a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: Displaying a graphical user interface (GUI) on a display device; representing a plurality of autonomous air vehicles by a plurality of icons displayed on the GUI; receiving an input selecting a first icon from the plurality of icons displayed on the GUI; receiving a first signal from the input device; displaying, in response to receiving the first signal, a first menu from a plurality of menus, the first menu being associated with a first flight parameter of the autonomous air vehicle represented by the first icon, the first menu including a plurality of first preset commands; receiving a selection of a first command from among the plurality of first preset commands; sending the first command to a first autonomous air vehicle represented by the first icon on the GUI; and a memory and A system comprising:

11. The system of claim 10 , wherein the first signal is generated by activating a first input mechanism of the input device, the first input mechanism being associated with the first flight parameter.

12. The instructions, when executed by the one or more processors, cause the one or more processors to: displaying information associated with one or more flight parameters associated with each of the plurality of autonomous aircraft near the icons representing the autonomous aircraft on the GUI, the one or more flight parameters including one or more of an altitude, a heading, or a speed of the autonomous aircraft represented by each of the plurality of icons, using a distinctive sensory cue for each flight parameter. The system of claim 10 , wherein the system performs steps including:

13. 13. The system of claim 12, wherein the input mechanism of the input device associated with the first flight parameter is identified on the input device using the distinctive sensory cue assigned to the first flight parameter.

14. The system of claim 12 , wherein the information includes values ​​of one or more current flight parameters and values ​​of the first flight parameter specified in the first command.

15. receiving an input selecting a first icon; Detecting a cursor controlled by the input device over the first icon for a predetermined time. The system of claim 10 further comprising:

16. 11. The system of claim 10, wherein the first preset commands are displayed in a circular format around a central element identifying the first flight parameter, and placement of the first preset commands is based on at least one of an orientation or a magnitude of each preset command.

17. 11. The system of claim 10, wherein each input mechanism and displayed menu set is associated with the same sensory cue.

18. 18. The system of claim 17, wherein the sensory cue is color, such that a first input mechanism assigned to the first flight parameter is provided in a first color and the first menu is displayed on the GUI using the first color.

19. One or more non-transitory computer-readable storage media storing instructions that, when executed on a server computer, cause the server computer to: Displaying a graphical user interface (GUI) on a display device; representing a plurality of autonomous air vehicles by a plurality of icons displayed on the GUI; receiving an input selecting a first icon from the plurality of icons displayed on the GUI; receiving a first signal from an input device including a plurality of input mechanisms, each of the input mechanisms being assigned to a unique flight parameter; in response to receiving the first signal, displaying a first menu from a plurality of menus, the first menu being associated with a first flight parameter of the autonomous air vehicle represented by the first icon and including a plurality of first preset commands; receiving a selection of a first command from among the plurality of first preset commands; causing a first autonomous air vehicle represented by the first icon on the GUI to transmit the first command. One or more non-transitory computer-readable storage media.

20. 20. The one or more non-transitory computer-readable storage media of claim 19, wherein the first preset commands are displayed in a circular format around a central element identifying the first flight parameter, and wherein placement of the first preset commands is based on at least one of an orientation or a magnitude of each preset command.

Images