Remote management of multiple autonomous aircraft

Abstract

Embodiments provide a flight management platform that includes a server computer and an animated graphical user interface for simultaneously monitoring multiple autonomous aircraft. More specifically, the technology disclosed herein provides a flight management platform that enables one-to-many management, such that a remote administrator (e.g., a human or an artificial intelligence) can continuously monitor and / or interact with multiple autonomous aircraft. For example, the autonomous aircraft may include one or more autonomous electric vertical take-off and landing (eVTOL) aircraft. The flight management platform may display information, receive input via the GUI, and send commands to the autonomous aircraft based on the received input.

Description

[Technical Field]

[0001] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 408,088, filed September 19, 2022, entitled "Remote Supervision of Multiple Autonomous Aircraft," the disclosure of which is incorporated herein by reference in its entirety for all purposes.

[0002] This application is related to a U.S. design patent application entitled "Multi Vehicle Supervisor Interface," filed on September 19, 2022, attorney docket number 105984-1346953-010100US. Summary of the Invention [Means for solving the problem]

[0003] Embodiments provide techniques for safely and efficiently monitoring and managing multiple autonomous aircraft simultaneously. According to various embodiments, a graphical user interface (GUI) is provided for simultaneously monitoring, managing, and / or communicating (e.g., interacting) with multiple autonomous aircraft. The GUI may be an animated GUI that includes a graphical representation of the aircraft and associated information displayed (e.g., overlaid) on a map representing an area in which the autonomous aircraft are flying. The GUI may further include a flight guidance pane for sending commands to modify flight parameters of the autonomous aircraft being monitored. In some embodiments, the flight guidance pane may always be displayed in association with a particular administrator using the GUI.

[0004] According to various embodiments, a computer-implemented method for monitoring and interacting with a plurality of autonomous aircraft is provided. The method includes using a computing device to display a graphical user interface (GUI) on a display device and representing the plurality of autonomous aircraft by a plurality of icons on a first pane of the GUI. Information associated with each one of the plurality of autonomous aircraft is displayed in a plurality of sections on a second pane of the GUI. The method further includes receiving a selection of one of a first icon among the plurality of icons on the first pane of the GUI or a first section among the plurality of sections on the second pane of the GUI. The method also includes identifying the other of the first icon among the plurality of icons on the first pane of the GUI or the first section among the plurality of sections on the second pane of the GUI. The first icon and the first section are displayed using one or more visual cues to distinguish the first icon and the first section from the remaining graphical elements. The method further includes sending a command or message to the autonomous aircraft represented by the first icon on the GUI.

[0005] Various embodiments provide a system (e.g., a flight management platform) for monitoring and interacting with multiple autonomous air vehicles, the system comprising: a display screen; a server computer including one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the methods described above.

[0006] Some embodiments provide one or more computer-readable storage media storing instructions that, when executed on a flight management platform having a server computer, cause the server computer to perform the methods described above.

[0007] For a better understanding of the nature and advantages of the present disclosure, reference should be made to the following description and accompanying figures. It should be understood, however, that each of the figures is provided for illustrative purposes only and is not intended as a definition of the limits of the scope of the present disclosure. Unless otherwise apparent from the description, when elements in different figures use the same reference numerals, those elements are generally either identical or at least similar in function or purpose. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 illustrates a block diagram of an exemplary flight management platform for monitoring and interacting with multiple autonomous air vehicles, in accordance with various embodiments. [Figure 2A] 1 illustrates an exemplary animated graphical user interface (GUI) of a flight management platform for managing and interacting with multiple autonomous air vehicles, according to various embodiments. [Figure 2B] 1 illustrates another view of an exemplary animated GUI of a flight management platform, in accordance with various embodiments. [Figure 3] 1 illustrates an example technique for assigning a route to an autonomous aircraft using an example animated GUI of a flight management platform, according to various embodiments. [Figure 4A] 1 illustrates the addition of layers onto a displayed exemplary animated GUI of a flight management platform, according to various embodiments. [Figure 4B] 10 illustrates the removal of a layer from a displayed animated GUI of a flight management platform, in accordance with various embodiments. [Figure 5] 1 illustrates tracking an autonomous air vehicle in flight using an exemplary animated GUI of a flight management platform, in accordance with various embodiments. [Figure 6] 1 illustrates a zoomed-in view of an exemplary animated GUI of a flight management platform, in accordance with various embodiments. [Figure 7] 1 illustrates another view of an exemplary animated GUI including a flight guidance pane, according to various embodiments. [Figure 8] 10 illustrates another view of an exemplary animated GUI after a mission assignment has been accepted, in accordance with various embodiments. [Figure 9] 1 illustrates an exemplary warning displayed using an exemplary animated GUI of a flight management platform, according to various embodiments. [Figure 10] 1 is a flowchart of an exemplary process performed by a flight management platform, according to various embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0009] The technology disclosed herein generally relates to an animated graphical user interface for simultaneously monitoring multiple autonomous air vehicles. More specifically, the technology disclosed herein provides a one-to-many management platform (e.g., a flight management platform) that allows a remote administrator (e.g., a human or artificial intelligence) to continuously monitor and / or interact with multiple autonomous air vehicles. The interaction may include sending commands to the autonomous air vehicles using a GUI component of the platform. For example, the autonomous air vehicles may include one or more autonomous electric vertical take-off and landing (eVTOL) aircraft. Various inventive embodiments, including methods, processes, systems, and devices, are described herein.

[0010] Several exemplary embodiments will now be described with reference to the accompanying drawings, which form a part of this specification. The following description provides exemplary embodiments only and is not intended to limit the scope, applicability, or configuration of the present disclosure. Rather, the following description of the embodiments will provide those skilled in the art with an enabling description for implementing one or more embodiments. It will be understood that various changes can be made in the function and arrangement of elements without departing from the spirit and scope of the present disclosure. In the following description, for purposes of explanation, specific details are set forth in order to provide a thorough understanding of particular inventive embodiments. It will be apparent, however, that various embodiments may be practiced without these specific details. The figures and descriptions are not intended to be limiting. As used herein, the word "example" or "exemplary" is used to mean "serving as an example, instance, or illustration." Any embodiment or design described herein as "exemplary" or "example" is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0011] Embodiments provide a flight management platform that includes a server computer and an animated graphical user interface (GUI) for managing, monitoring, and / or communicating with one or more autonomous aircraft. For example, a human operator may be able to simultaneously manage multiple autonomous aircraft using the flight management platform that includes the animated GUI displayed on a display device on the ground (e.g., at a monitoring center remote from the autonomous aircraft). The operator may send commands to change one or more of the flight parameters (e.g., altitude, attitude, airspeed, heading) of the autonomous aircraft.

[0012] According to various embodiments, an autonomous aircraft is configured to implement automated flight technologies that provide, among other technologies, autopilot functionality, precision navigation, and sense-and-avoid capabilities. The flight management platform described herein provides oversight for the autonomous aircraft and the ability to intervene when needed. The flight management platform (e.g., a multi-aircraft management platform) can oversee a flight from start to finish. During the flight, if the autonomous aircraft takes steps to avoid a potential hazard outside of the assigned flight plan, the flight management platform receives a notification or alert, which is then displayed on a GUI. The flight management platform allows a manager (e.g., a human manager) to intervene at any point during the flight by sending new commands for the autonomous aircraft to execute. For example, the manager can use the flight management platform to send a command to the aircraft to divert the aircraft to an alternate landing site.

[0013] FIG. 1 illustrates a block diagram of a flight management platform 1000 that may be used by an administrator 1030 (e.g., a human administrator 1030) to monitor and interact with one or more autonomous air vehicles 1040, individually and / or collectively. The flight management platform 1000 may include a server computer 1020 that includes one or more processors 1004, a system memory 1002 (which may include any combination of volatile and / or non-volatile memory, such as, for example, buffer memory, RAM, DRAM, ROM, flash, or any other suitable memory device), and a network interface (e.g., an external communication interface) 1006. In some embodiments, one or more of the modules may be located within one or more of the components of the system memory 1002 or may be located externally. The software and hardware modules illustrated in FIG. 1 are provided for illustrative purposes only and are not intended to be limiting. The processor 1004, the system memory 1002 and / or the external communication interface 1006 may implement the techniques and / or methods described herein.

[0014] Network interface 1006 may be configured or programmed to receive and generate electronic messages containing information transmitted to or from multiple autonomous air vehicles 1040 through flight management platform 1000.

[0015] Flight management platform 1000 may also include at least one display device 1010 for displaying a graphical user interface (GUI) 1012. When an electronic message is received by flight management platform 1000 via external communication interface 1006 of server computer 1020, the electronic message may be processed and relevant information may be displayed on display device 1010 via GUI 1012. When input is received from administrator 1030 via GUI 1012, the input may be processed and relevant information may be sent to the corresponding autonomous air vehicle 1040. According to various embodiments, flight management platform 1000 may further be configured to receive supplemental information from third parties, such as air traffic control, weather, other aircraft (e.g., aircraft monitoring one or more autonomous air vehicles 1040 in the air), etc. The supplemental information may be processed by flight management platform 1000 and displayed on GUI 1012.

[0016] FIG. 2A illustrates an exemplary animated graphical user interface (GUI) 100 of a flight management platform 1000 for managing and interacting with multiple autonomous air vehicles, according to various embodiments. In the exemplary GUI 100, a manager (e.g., manager 1030 shown in FIG. 1) oversees five autonomous air vehicles, depicted using five aircraft icons on the GUI 100. The GUI 100 may include a first pane 102 depicting each autonomous air vehicle with a corresponding aircraft icon 116, 118, overlaid on a map of terrain over which the autonomous air vehicles are flying, hovering, and / or grounded in standby mode. The amount of detail to be included in the map may be determined using human factors, such as an assessment of how much information can be safely processed by a human on a screen. Some embodiments may use artificial intelligence (AI) as the manager or a combination of AI and a human manager. The amount of detail to be included in the map may vary depending on the capabilities of the managing entity. In some embodiments, the manager

[0017] The GUI 100 may also include a second pane 104 that includes sections 106, 108 corresponding to each of the aircraft icons 116, 118 shown in the first pane 102. That is, each section 106, 108 provides information associated with a different autonomous aircraft under the administrator's control. In some embodiments, when the administrator selects an autonomous aircraft, for example, by clicking on the desired aircraft icon 116, a visual cue may associate the selected aircraft icon 116 displayed on the first pane 102 with the corresponding information section 106 displayed on the second pane 104. For example, the selected aircraft icon 116 and the corresponding information section 106 may be displayed in a different color, tone, and / or font than the rest of the graphical elements of the GUI. The selected aircraft icon 116 and the information section 106 represent the same autonomous aircraft (e.g., the autonomous aircraft 1040 shown in FIG. 1 ). According to various embodiments, the GUI 100 may display only active missions (e.g., missions having an active status) managed by the administrator. There may be additional missions (e.g., missions that are not yet active or that are preparing to fly) assigned to the administrator, and this information may be displayed in the appropriate panes 102, 104 of the GUI 100 after the administrator accepts the mission or after the mission assumes an active state.

[0018] In some embodiments, selection of an aircraft icon 116 on the first pane 102 may result in automatic selection of a corresponding information section 106 on the second pane 104. Alternatively, selection of a corresponding information section 106 on the second pane 104 may result in automatic selection of the aircraft icon on the first pane 102. When a server computer coupled to the GUI (e.g., server computer 1020 shown in FIG. 1 ) receives a selection of an aircraft icon 116 from among multiple icons on the first pane 102 of the GUI 100 or a selection of an information section 106 from among multiple sections on the second pane 104 of the GUI 100, the server computer identifies the other of the aircraft icon 116 or the information section 106 on the GUI 100. The aircraft icon 116 and the information section 106 are then displayed using one or more visual cues to distinguish the aircraft icon 116 and the information section 106 from the remaining graphical elements.

[0019] Displaying the aircraft icon and information section in separate locations on the GUI enables accurate identification and selection of the intended autonomous aircraft represented using the aircraft icon and information section. For example, if two autonomous aircraft are flying overlapping, their representations (e.g., aircraft icons) on the map will likely be displayed overlaid on one another. Thus, it may not be possible or easy for an administrator to select one of the overlapping autonomous aircraft using the first pane 102. In such cases, the administrator may select the autonomous aircraft with which to interact using the corresponding section on the second pane 104. In this manner, selection of the intended aircraft is ensured.

[0020] As illustrated in the exemplary embodiment shown in the accompanying figures, the first pane 102 is allocated a larger area on the screen than the second pane 104. This area allocation allows aircraft icons to be displayed on a larger map without cluttering the map display (e.g., while also allowing associated information to be displayed in a separate section of the screen (e.g., the second pane 104)). The relative positions and sizes of the first pane 102 and the second pane 104 may vary in various embodiments.

[0021] Flight management platform 1000, through GUI 100, allows an administrator to interact with all autonomous aircraft shown on GUI 100 via corresponding aircraft icons 116, 118 or information sections 106, 108. Prior to takeoff, an autonomous aircraft may autonomously perform a pre-flight inspection, which may include checking one or more of the power systems, control surfaces, avionics, passenger equipment, cabin systems, and / or the environment surrounding the autonomous aircraft. The autonomous aircraft may continuously monitor these systems and / or items during flight as well. The autonomous aircraft may notify flight management platform 1000 (and thereby administrator 1030) (e.g., send data and / or a message) when checks are complete and the autonomous aircraft and passengers (if applicable) are ready for takeoff.

[0022] After receiving notification that the autonomous aircraft is ready for takeoff and / or that a route has been assigned to the autonomous aircraft (discussed further below in connection with FIG. 3 ), the administrator first selects the aircraft icon 116 corresponding to the autonomous aircraft on the first pane 102, sends a takeoff command to the autonomous aircraft, for example, by selecting a first button (e.g., a Liftoff button) 110 on the second pane 104 (e.g., as illustrated in FIG. 2A , the selected Liftoff button 110 is displayed using a visual cue such as a bold font to indicate that the button has been selected), and then selects a second button (e.g., an Execute button) 112 on the second pane 104 to confirm that the takeoff command is sent to the selected aircraft (e.g., as illustrated in FIG. 2B , the selected Execute button 112 is displayed using a visual cue such as a bold font to indicate that the button has been selected). A two-step selection involving selecting two separate buttons to send a command to the autonomous aircraft may ensure that the command is sent intentionally. Thus, the server computer 1020 may receive the selection of the aircraft icon 116 on the first pane 102 of the GUI 100 and may receive a takeoff command and subsequent execute command from the second pane 104 of the GUI 100. The server computer 1020 may then send the takeoff command to the autonomous aircraft represented using the aircraft icon 116 on the GUI 100.

[0023] In the embodiment shown in Figures 2A and 2B, when an administrator selects an icon on the first pane of the GUI or selects an information section on the second pane of the GUI, no additional pop-up windows are displayed. This reduces clutter on the GUI and distractions for the administrator. In some embodiments, pop-up windows or changes to the layout of the GUI are prevented to reduce or eliminate distractions for the administrator.

[0024] In some embodiments, while the autonomous aircraft is following its assigned route, the administrator may receive commands from air traffic control (ATC) that need to be relayed to the autonomous aircraft under their control. The GUI display is optimized to allow the administrator to quickly identify the correct autonomous aircraft and relay the ATC communications to the correct autonomous aircraft.

[0025] In some embodiments, the administrator may identify the location of the autonomous aircraft on the map displayed on the first pane 102 and determine whether the ATC communication is applicable. For example, if the map shows the autonomous aircraft over mountainous terrain that is unsafe for the maneuver proposed in the ATC communication, the administrator may confirm the maneuver with ATC before relaying it to the autonomous aircraft, or may determine that the communication is erroneous and should be discarded based on information provided to the administrator using the map displayed on GUI 100.

[0026] 2A , four of the aircraft may already be in the air and following a route, while a fifth aircraft, represented by aircraft icon 116, may be on the ground awaiting a route assignment. According to various embodiments, a manager may communicate commands to the aircraft, such as assigning a route and giving takeoff orders. Once the autonomous aircraft receives the route assignment, it executes the flight plan to follow the route.

[0027] FIG. 3 illustrates an exemplary technique for assigning a route to an autonomous aircraft using an exemplary animated GUI of a flight management platform, according to various embodiments. GUI 100 can also be used to assign a flight path to an autonomous aircraft. For example, an autonomous aircraft may be on the ground and awaiting a flight assignment. An administrator may select an autonomous aircraft by selecting either the aircraft icon 116 representing the aircraft on the ground or the information section 106 (as shown in FIGS. 2A and 2B ). Selecting an autonomous aircraft on the ground triggers GUI 100 to display a list of available routes 202 using menu 200. Those skilled in the art will appreciate that the graphical elements shown in the figures for providing information or receiving input from an administrator are not limited to the means illustrated in the figures, and that alternative means may be used to convey similar information or receive similar input. For example, menu 200 may include a scroll-down menu, a drop-down menu, etc. An administrator may select one of the routes by, for example, clicking the desired route from the list of available routes 202. The administrator may be asked to confirm the selection, for example, by selecting an Accept button 204. The Accept button 204 for confirming the selection may be displayed on the second pane 104, while the menu 200 with the route 202 may be displayed on the first pane 102. Thus, the confirmation action may require the administrator to move from the first pane 102 to the second pane 104 to intentionally select the Accept button 204. This configuration may avoid making an inadvertent selection.

[0028] Once a route 206 is selected for assignment to an aircraft, the selected route 206 may be displayed on the map as route 210 displayed from the departure point to the destination point using visual cues, such as using a first color. This may also allow an administrator to visually confirm that the intended route has been assigned to the autonomous aircraft. The departure and destination points 208 of the selected route may be displayed on an information section corresponding to the selected autonomous aircraft. For example, if the intended assignment of a route from Los Angeles to San Francisco was made but the displayed route was from San Diego to Los Angeles, the administrator may realize the error before executing the route assignment command. Once the administrator confirms that the displayed route 210 is the intended route, the administrator may execute the assignment by selecting the accept button 204. Execution of the command may result in the assigned route being displayed on the map using different visual cues, such as using a second color, from the departure point to the destination point to indicate that the route is now a confirmed route. That is, the GUI 100 relays the transition from the selected route to the confirmed route.

[0029] A server computer 1020 coupled to GUI 100 may continuously receive flight parameters and position positioning signals of multiple autonomous air vehicles being monitored using GUI 100. The server computer may periodically update GUI 100 using the flight data and position positioning signals of the autonomous air vehicles to display the relative position of first icon 116 with respect to displayed route 210 assigned to the autonomous air vehicle as the autonomous air vehicle progresses along displayed route 210.

[0030] According to various embodiments, an administrator can choose the amount of detail to be shown on the map. For example, GUI 100 can have layers of information 250 that can be displayed overlaid. The administrator can select one or more of the available layers (e.g., radius information around the aircraft (Radius) 252, wind information (Wind) 254, measurement information received from one or more aircraft (Telem) 256, and optional additional information (Info) 258) to have a desired level of detail on the map.

[0031] 4A-4B illustrate various levels of detail displayed on a GUI of a flight management platform, according to various embodiments. Specifically, FIG. 4A illustrates the addition of a layer onto a displayed exemplary animated GUI, and FIG. 4B illustrates the removal of a layer from the displayed animated GUI. FIG. 4A illustrates a visual aid 302 for indicating the surroundings of an autonomous aircraft represented using an aircraft icon 116. For example, visual aid 302 may be in the form of concentric rings, with each ring corresponding to a predetermined distance from the autonomous aircraft. In this manner, a supervisor may quickly visually assess whether other aircraft or hazards are present in close proximity to the monitored autonomous aircraft. Similarly, visual aid 302 may also convey information about the distance between the monitored autonomous aircraft and terrain features (e.g., mountains) near the monitored autonomous aircraft.

[0032] If an administrator prefers a cleaner display, the administrator can remove visual aid 302 from the GUI, as shown in FIG. 4B. For example, visual aid 302 can be displayed when Radius layer 252 (shown in FIG. 3) is selected and can be removed when Radius layer 252 is not selected on GUI 100. Some of the elements of the GUI may not be subject to masking (e.g., some elements are always displayed and cannot be removed by toggling layers), while other elements may be displayed according to selected options provided on the GUI (e.g., based on the selection of Map Layers 250 shown in FIG. 3).

[0033] 4-5, the aircraft icons 116, 118 may have additional information 400 displayed on the map in association with the aircraft icon. For example, flight parameters such as heading, altitude, and / or speed information associated with the autonomous aircraft represented by the respective aircraft icon 116, 118 may be displayed adjacent to the aircraft icon 116, 118. In some embodiments, the information 400 may be attached to the aircraft icon and may move with the aircraft icon along the displayed path.

[0034] As illustrated in FIG. 5 , the animated GUI may display an aircraft icon 116 moving along a route 120 assigned to that aircraft. As the autonomous aircraft progresses along its assigned route (e.g., flight path), the corresponding aircraft icon 116 moves along the route 120 on the map. The displayed information 400 moves along with the aircraft icon 116. By having all the information on the same GUI, the techniques provided herein ensure that an administrator assesses all relevant information associated with an autonomous aircraft and selects the correct autonomous aircraft when activating a route or communicating a command to the autonomous aircraft. In some embodiments, the administrator may toggle between displaying and hiding the displayed information 400. That is, flight parameters associated with the autonomous aircraft may be selectively displayed on the GUI near a first icon representing the autonomous aircraft.

[0035] According to various embodiments, an autonomous aircraft being managed using the animated GUI 100 is flying autonomously using instrument flight rules (IFR). If ATC instructs an administrator to communicate new parameters (e.g., altitude, heading, speed, etc.) to the autonomous aircraft, the administrator may do so by selecting the aircraft icon 116 and sending a command with the new parameters. In some embodiments, the autonomous aircraft may acknowledge receipt of the command and / or execution of the command by a message sent back to the server computer and displayed on the GUI 100.

[0036] In some embodiments, the GUI may be displayed at various zoom levels selected by an administrator or based on a selected action. For example, when assigning a flight path to an autonomous aircraft, the GUI may be displayed at a zoom level that shows both the start and end points of the path. For example, when entering a flight parameter change command, the GUI may be displayed at a zoom level that better shows the environment surrounding the autonomous aircraft to ensure that the flight parameter modification does not pose an imminent threat to the autonomous aircraft.

[0037] FIG. 6 shows a zoomed-in view of an exemplary animated GUI of a flight management platform, according to various embodiments. As illustrated in FIG. 6 , a route can be formed by connecting multiple waypoints 552, 554, 556. In some embodiments, a segment 560 (e.g., a portion) between the waypoints can change color as an autonomous aircraft flies over the segment. For example, in the exemplary embodiment shown in FIG. 6 , segment 560 (e.g., the completed portion of the route) is shown in a first color (e.g., white) to indicate that route segment 560 is complete, while segment 562 (e.g., the remaining portion of the route) is shown in a second color (e.g., purple) to indicate that the segment is not yet complete. These visual cues can help a manager gather various types of information simultaneously by simply looking at or observing the GUI. Using the visual cues, a manager can stay aware of the status of multiple autonomous aircraft under their management.

[0038] According to various embodiments, when an aircraft icon 116 is selected on the GUI, an administrator may select a waypoint on the planned route of the aircraft represented by the aircraft icon 116 and have the aircraft fly directly through the selected waypoint. For example, if the aircraft is configured to fly from waypoint A to waypoint B to waypoint C, the administrator may select waypoint C and instruct the aircraft to fly from waypoint A directly to waypoint C, skipping waypoint B. As with the various commands described herein, one or more selections entered on the GUI are transmitted to the autonomous aircraft by a server computer coupled to the GUI.

[0039] Once the autonomous aircraft completes its flight plan (e.g., arrives at the destination point), the aircraft icon 116 may be indicated by one or more visual cues indicating that the autonomous aircraft has completed its flight and landed.

[0040] FIG. 7 shows another view of an exemplary animated GUI including a flight guidance pane, according to various embodiments. The exemplary GUI 600 illustrated in FIG. 7 includes a first pane 602 (similar to the first pane 102 of FIG. 2A ) having multiple icons 601, 603, each representing an autonomous aircraft managed using the GUI 600. The exemplary GUI 600 further includes a second pane 604 (similar to the second pane 104 of FIG. 2A ) having multiple information sections, each corresponding to one of the displayed icons on the first pane 602. The first pane 602 and the second pane 604 may show active (e.g., active or executing flight plans) missions (e.g., flights). The exemplary GUI 600 further includes a third pane 606 displaying a list 650 of missions assigned to an administrator of the GUI 600. Each mission assignment represents an additional autonomous flight assigned to the administrator for monitoring by the GUI. That is, the third pane 606 shows autonomous aircraft that have been assigned but are not currently being managed using the GUI 600. For example, the list of missions 650 may include missions that are waiting for a manager (e.g., an operator) to accept responsibility for them. Once the manager accepts an assigned mission, final preparations before departure may be completed. In some embodiments, as described above, the missions on the list 650 may not include all assigned missions, but may include only assigned missions that are in a pre-preparation state. Accepting an assigned mission (e.g., autonomous flight) is described below in connection with FIG. 8.

[0041] According to various embodiments, the flight management platform 1000 may display messages reporting one or more upcoming tasks that may require manager approval or acceptance. For example, reminders of upcoming tasks may be displayed preferentially for a predetermined period of time, along with information necessary to avoid human error and poor human performance (e.g., delayed responses to time- and / or safety-critical events). In some embodiments, the reminder message may have a countdown displayed therein indicating the time remaining for the manager 1030 to act on the reminder.

[0042] Continuing with the description of the exemplary GUI 600, a fourth pane 608 is provided to show a zoomed map view of a selected autonomous aircraft. For example, if icon 601 is selected on first pane 602, a corresponding information section 605 is identified on second pane 604 using visual cues (as described above in connection with FIGS. 2A and 2B ), and the immediate area around the selected autonomous aircraft is illustrated in pane 608 using a zoomed-in map. The exemplary GUI 600 also includes a flight guidance pane 610. The GUI 600 may receive modifications on flight guidance pane 610 corresponding to desired changes to one or more flight parameters associated with the selected autonomous aircraft (e.g., the aircraft represented by 601 and information section 605 in the exemplary GUI of FIG. 7). A server computer transmits commands to the autonomous aircraft to effect the desired changes. According to various embodiments, flight guidance pane 610 may include one or more of a control dial 612 or 614 , a drop-down menu, a text entry field, or a widget 620 .

[0043] For example, first control dial 612 may allow an administrator to input a heading setting or modification for a selected autonomous aircraft. The GUI may display possible headings in predetermined increments. Once the administrator makes a selection, the selected heading may be indicated using visual cue 618. Second control dial 614 may allow an administrator to input an altitude setting or modification for a selected autonomous aircraft. The GUI may display possible altitudes in predetermined increments. Once the administrator makes a selection, the selected heading may be indicated using visual cue 616. In some embodiments, flight parameter settings or modifications may be entered into text entry fields. When one or more of the flight parameter settings or modifications are complete, the administrator may select widget 620 (e.g., an “Execute” widget) to cause the server computer to send commands to the autonomous aircraft to effect the one or more flight parameter settings or modifications.

[0044] FIG. 8 shows another view of an exemplary animated GUI of a flight management platform after a mission assignment has been accepted, according to various embodiments. As described above, the exemplary GUI 600 illustrated in FIG. 7 includes a third pane 606 displaying a list 650 of missions assigned to the administrator of GUI 600. When the administrator accepts one of the missions, the mission is moved from the third pane to a second pane displaying a list of active autonomous flights managed by the administrator. As shown in FIG. 8, the administrator has accepted mission 702, which has been removed from third pane 705 and moved to second pane 740. Second pane 740 displays all autonomous air vehicles managed and / or operated by the administrator, whether they are in the air or on the ground (e.g., waiting to start a mission or recently landed as part of a mission).

[0045] When the administrator accepts the new mission, the flight guidance pane 610 temporarily switches to a configuration view 710. The configuration view 710 provides additional information associated with the newly accepted mission, such as the departure and destination points, the estimated departure time, the aircraft's state of charge, and the flight check status. The configuration view 710 may include a text area 704 for displaying additional information associated with the newly accepted mission, and one or more command widgets 708. When the LIFTOFF command is selected on the GUI, the server computer sends a liftoff command to the autonomous air vehicle, and the configuration view 710 switches back to the flight guidance pane 610 as shown in FIG. 7 .

[0046] Autonomous flights operated by an administrator may be represented using icons on a first pane 730 of the GUI. According to various embodiments, an icon representing an autonomous aircraft in flight 732 may have a different representation than an icon 734 representing an autonomous aircraft on the ground (e.g., an autonomous aircraft preparing for flight or that recently landed). An icon representing a newly accepted mission may not yet be associated with a flight plan (e.g., a route) on the map. The icon is associated with a flight plan or route on the GUI when an EXECUTE LIFTOFF command is sent by the server computer to the autonomous aircraft.

[0047] 8, the zoomed-in map pane 720 may provide a close-up aerial view of the surroundings of the newly accepted mission. The map pane 720 may also show the current state of the autonomous aircraft. For example, because the selected autonomous aircraft corresponds to the newly accepted mission 702, the state of the autonomous aircraft is shown as "Transition Level 1, Transition-Climb," which means that the autonomous aircraft is transitioning to vertical flight mode for takeoff or climb.

[0048] According to various embodiments, a server computer coupled to the GUI may receive input selecting a mission assignment displayed in pane 705. For each selected mission assignment, a separate icon 734 is generated in the first pane 730 and a corresponding separate section 702 representing information associated with the selected mission assignment is generated in the second pane 740. The accepted mission is then removed from the third pane 705 of the GUI.

[0049] According to various embodiments, an entity (e.g., ATC, a third party, another aircraft) may send an alert to an administrator. In some embodiments, the alert may be received at a server computer in communication with the flight management platform and / or the GUI, and a visual alert may be displayed on the GUI. The administrator may choose to ignore the displayed alert or act accordingly. In some embodiments, the GUI may also display proposed changes or modifications to flight parameters of the monitored autonomous aircraft. The administrator may accept the proposed changes and send commands to the autonomous aircraft to change the flight parameters. In some embodiments, in response to the alert, the administrator may use a flight guidance pane of the GUI to manually enter the desired changes to the flight parameters. The server computer may then send the commands to the autonomous aircraft using the GUI.

[0050] In some embodiments, the autonomous aircraft is configured to change course to avoid a collision with a hazard or to avoid a mission failure due to, for example, an onboard electro-mechanical problem. The autonomous aircraft may change course without receiving approval from an administrator. In some embodiments, the autonomous aircraft may be configured to wait for approval for a predetermined amount of time before implementing the change in the flight plan.

[0051] According to various embodiments, a dedicated area of ​​the GUI may be provided for displaying alert messages. For example, feedback or data received from sensors or other equipment on the autonomous aircraft indicating a problem with the aircraft may be displayed as an alert in the dedicated area. In some embodiments, given the importance of the alert, the alert may be displayed as a pop-up screen over a primary pane (e.g., the first pane) of the GUI.

[0052] FIG. 9 shows an example warning displayed using an example animated GUI, according to various embodiments. In some embodiments, an autonomous aircraft may detect a hazard 800 on its path 802 and may deviate from its path 802 to avoid colliding with the hazard 800. For example, the autonomous aircraft may veer left or right, changing its path to an alternate path 804 that avoids colliding with the hazard 800. In some embodiments, the autonomous aircraft may need to change its path due to an identified issue with the autonomous aircraft, such as a propeller (e.g., fan) failure. The alternate path 804 and a collision avoidance alert (e.g., warning) 806 are displayed on the GUI to notify a manager. In some embodiments, upon communicating the hazard 800 to the flight management platform, the autonomous aircraft may request a flight trajectory update. The flight trajectory update request may be displayed on the GUI. The administrator may provide a response to a request for a flight trajectory update by sending changes to the flight parameters or flight plan of the autonomous aircraft via the server computer.

[0053] When alert 806 is displayed on the GUI, the GUI may switch the selected aircraft icon to the aircraft icon associated with alert 806. For example, if the alert was received from, or is otherwise associated with, the autonomous aircraft represented by icon 805 and corresponding section 822, the GUI may display icon 805 and section 822 as the selected autonomous aircraft. In some embodiments, if the alert is displayed because of a change already performed by the autonomous aircraft, the alert may be displayed for a predetermined amount of time before switching back to the aircraft last selected by the administrator.

[0054] In some embodiments, the ATC may recommend a flight plan change, which may be displayed as an alert on the GUI. The alert may further include a widget displayed on the GUI for the administrator to accept or reject the recommended flight plan change. If the administrator accepts the recommended flight plan change, selecting the widget to accept the recommended flight plan change may cause the server computer to send a command to the autonomous aircraft to activate the recommended flight plan change (e.g., change heading as indicated in the recommended flight plan change). The autonomous aircraft executes the received command and begins following the revised flight plan.

[0055] An embodiment provides a flight management platform (e.g., flight management platform 1000 shown in FIG. 1 ) for implementing the techniques described herein. Information received from a monitored autonomous aircraft or from any other source is analyzed and / or processed by a server computer (e.g., server computer 1020) and sent to a GUI (e.g., GUI 1012) for display. Similarly, any input or selection received at the GUI is analyzed and / or processed by the server computer and sent to the intended autonomous aircraft via a secure communication channel between the server computer and the autonomous aircraft.

[0056] 10 is a flowchart of an example process 900. In some embodiments, one or more process blocks of FIG. 10 may be implemented by a flight management platform.

[0057] As illustrated in FIG. 10 , process 900 may include displaying a graphical user interface (GUI) on a display device using a server computer (block 902). A plurality of autonomous air vehicles are represented by a plurality of icons on a first pane of the GUI (block 904). As further shown in FIG. 10 , process 900 may include displaying information associated with each one of the plurality of autonomous air vehicles in a plurality of sections on a second pane of the GUI (block 906). The server computer may receive input selecting one of (1) a first icon among the plurality of icons on the first pane of the GUI or (2) a first section among the plurality of sections on the second pane of the GUI (block 908). The server computer may then identify the other of (1) the first icon among the plurality of icons on the first pane of the GUI or (2) the first section among the plurality of sections on the second pane of the GUI (block 910). The first icon and the first section are displayed on the GUI using one or more visual cues to distinguish the first icon and the first section from the remaining graphical elements (block 912). The server computer sends a command or message to the autonomous air vehicle represented by the first icon on the GUI (block 914).

[0058] In some embodiments, flight parameters associated with the autonomous aircraft are displayed on the GUI near a first icon representing the autonomous aircraft, the flight parameters including one or more of an altitude, a heading, or a speed of the autonomous aircraft. According to various embodiments, the method may include receiving flight data messages from the plurality of autonomous aircraft, each flight data message including one or more flight parameters associated with one of the plurality of autonomous aircraft. A plurality of icons representing the plurality of autonomous aircraft may be displayed on the GUI along with the corresponding one or more flight parameters.

[0059] 10 illustrates example blocks of process 900, in some embodiments, process 900 may include additional, fewer, different, or differently arranged blocks relative to those illustrated in FIG 10. Additionally or alternatively, two or more of the blocks of process 900 may be performed in parallel.

[0060] Embodiments may also provide one or more computer-readable storage media storing instructions that, when executed on a flight management platform having a server computer, cause the server computer to perform the method described above.Embodiments may also provide a system (e.g., a flight management platform) comprising: a display screen; a server computer including one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the method described above.

[0061] In the foregoing specification, embodiments of the present disclosure have been described with reference to numerous specific details that may vary from implementation to implementation. Therefore, the specification and drawings should be considered in an illustrative rather than a restrictive sense. The sole and exclusive indication of the scope of the present disclosure, and what is intended by the applicant to be the scope of the present disclosure, is the set of claims issuing from this application, in the specific form in which such claims issue, literally and in their equivalents, including any subsequent modifications. Specific details of particular embodiments may be combined in any suitable manner without departing from the spirit and scope of the embodiments of the present disclosure.

[0062] Additionally, spatially relative terms such as "bottom," "top," or "side" may be used to describe the relationship of an element and / or feature to another element and / or feature, for example, as shown in the figures. It will be understood that spatially relative terms are intended to encompass different orientations of the device in use and / or operation in addition to the orientation depicted in the figures. For example, if the device in the figures were inverted, an element described as being on the "bottom" side might then be oriented "above" the other element or feature. The device may be oriented in other ways (e.g., rotated 90 degrees or at other orientations), and the spatially relative descriptors used herein would be interpreted accordingly.

[0063] The methods, systems, and devices discussed herein are examples. Various embodiments may omit, substitute, or add various procedures or components as appropriate. For example, features described with respect to particular embodiments may be combined in various other embodiments. Different aspects and elements of the embodiments may be similarly combined. Furthermore, technology evolves, and therefore, many of the elements are examples that do not limit the scope of the disclosure to those particular examples.

[0064] As used herein, the terms "and," "or," and "and / or" can include a variety of meanings that are expected to depend, at least in part, on the context in which such terms are used. Typically, "or," when used to associate a list such as A, B, or C, means A, B, and C, where it is used in an inclusive sense, and "or" when used to refer to A, B, or C, where it is used in an exclusive sense. Additionally, as used herein, the term "one or more" can be used to describe any feature, structure, or characteristic in the singular, or can be used to describe a certain combination of features, structures, or characteristics. However, it should be noted that this is merely an illustrative example, and claimed subject matter is not limited to this example. Furthermore, the term "at least one of," when used to associate a list such as A, B, or C, can be interpreted to mean any combination of A, B, and / or C, such as A, B, C, AB, AC, BC, AA, AAB, ABC, AABBCCC, etc.

[0065] References throughout this specification to "one example," "an example," "particular example," or "exemplary embodiment" mean that a particular feature, structure, or characteristic described in connection with a feature and / or example may be included in at least one feature and / or example of the claimed subject matter. Thus, the appearances of the phrases "in one example," "an example," "in a particular example," "in a particular embodiment," or other similar phrases in various places throughout this specification do not necessarily all refer to the same features, examples, and / or limitations. Furthermore, particular features, structures, or characteristics may be combined in one or more examples and / or characteristics.

[0066] In the foregoing detailed description, numerous specific details have been set forth to provide a thorough understanding of the claimed subject matter. However, it will be understood by those skilled in the art that the claimed subject matter may be practiced without these specific details. In other instances, methods and apparatuses known by those skilled in the art have not been described in detail so as not to obscure the claimed subject matter. Therefore, it is intended that the claimed subject matter not be limited to the particular examples disclosed, but that such claimed subject matter may include all embodiments falling within the scope of the appended claims and equivalents thereof.

Claims

1. 1. A computer-implemented method for monitoring and interacting with a plurality of autonomous air vehicles, the method comprising: displaying a graphical user interface (GUI) on a display device using a server computer; representing, by the server computer, a plurality of autonomous air vehicles with a plurality of icons on a first pane of the GUI; displaying, by the server computer, information associated with each one of the plurality of autonomous air vehicles in a plurality of sections on a second pane of the GUI; receiving, by the server computer, an input selecting one of a first icon among the plurality of icons on the first pane of the GUI or a first section among the plurality of sections on the second pane of the GUI; identifying, by the server computer, the other of the first icon among the plurality of icons on the first pane of the GUI or the first section among the plurality of sections on the second pane of the GUI; displaying, by the server computer, the first icon and the first section using one or more visual cues to distinguish the first icon and the first section from the remaining graphical elements; sending, by the server computer, a command or message to the autonomous air vehicle represented by the first icon on the GUI; A method comprising:

2. displaying one or more mission assignments on a third pane of the GUI, each mission assignment representing an additional autonomous flight to be monitored by the GUI; receiving an input selecting at least one of the one or more mission assignments; For each selected mission assignment, generating a separate icon and a separate section representing information associated with the selected mission assignment; displaying the individual icon on the first pane of the GUI; displaying the individual sections on the second pane of the GUI; removing the selected mission from the third pane of the GUI; The method of claim 1 further comprising:

3. displaying a flight guidance pane on the GUI; receiving a modification on the flight guidance pane corresponding to a desired change in one or more flight parameters associated with the autonomous aircraft represented by the first icon; sending the command to the autonomous aircraft represented by the first icon to effectuate the desired change; The method of claim 1 further comprising:

4. The method of claim 3 , wherein the flight guidance pane includes one or more of a control dial, a drop-down menu, a text entry field, or a widget.

5. The method of claim 1 , wherein the plurality of icons are overlaid on a map displayed on the first pane, the first pane being larger than the second pane on the display device.

6. displaying a list of one or more routes on the GUI; receiving a route selection from the list; assigning the route to the autonomous aircraft represented by the first icon on the GUI, the autonomous aircraft executing a flight plan to follow the route upon receiving the route assignment; periodically updating the GUI to display a relative position of the first icon with respect to the route assigned to the autonomous aircraft as the autonomous aircraft progresses along the route; The method of claim 1 further comprising:

7. displaying on the GUI a portion of the route that has been completed using a first visual cue and a remaining portion of the route using a second visual cue. The method of claim 6 further comprising:

8. Selectively displaying flight parameters associated with the autonomous aircraft on the GUI adjacent to the first icon representing the autonomous aircraft, the flight parameters including one or more of an altitude, a heading, or a speed of the autonomous aircraft. The method of claim 1 further comprising:

9. receiving a message from the autonomous aircraft, the message indicating a hazard detected by the autonomous aircraft along a route and an alternative route to be followed by the autonomous aircraft; displaying a collision avoidance alert and the alternative route on the GUI; The method of claim 1 further comprising:

10. receiving positioning signals from the plurality of autonomous air vehicles; displaying the plurality of icons representing the plurality of autonomous air vehicles on a map based on the positioning signals; The method of claim 1 further comprising:

11. receiving flight data messages from the plurality of autonomous air vehicles, each flight data message including one or more flight parameters associated with one of the plurality of autonomous air vehicles; displaying, on the GUI, the plurality of icons representing the plurality of autonomous air vehicles along with one or more corresponding flight parameters; The method of claim 1 further comprising:

12. 1. A system for monitoring and interacting with a plurality of autonomous air vehicles, the system comprising: a display screen; one or more processors; a memory storing instructions that, when executed by the one or more processors, cause the one or more processors to: Displaying a graphical user interface (GUI) on a display device; representing a plurality of autonomous air vehicles by a plurality of icons on a first pane of the GUI; displaying information associated with each one of the plurality of autonomous air vehicles in a plurality of sections on a second pane of the GUI; receiving an input selecting one of a first icon among the plurality of icons on the first pane of the GUI or a first section among the plurality of sections on the second pane of the GUI; Identifying the other of the first icon among the plurality of icons on the first pane of the GUI or the first section among the plurality of sections on the second pane of the GUI; displaying the first icon and the first section using one or more visual cues to distinguish the first icon and the first section from the remaining graphical elements; sending a command or message to the autonomous air vehicle represented by the first icon on the GUI; a memory for causing the steps comprising: A system comprising:

13. The instructions, when executed by the one or more processors, cause the one or more processors to: displaying one or more mission assignments on a third pane of the GUI, each mission assignment representing an additional autonomous flight to be monitored by the GUI; receiving an input selecting at least one of the one or more mission assignments; For each selected mission assignment, generating a separate icon and a separate section representing information associated with the selected mission assignment; Displaying the individual icon on the first pane of the GUI; Displaying the individual sections on the second pane of the GUI; and removing the selected mission from the third pane of the GUI; The system of claim 12 , further comprising:

14. The instructions, when executed by the one or more processors, cause the one or more processors to: displaying a list of one or more routes on the GUI; receiving a route selection from the list; assigning the route to the autonomous aircraft represented by the first icon on the GUI; periodically updating the GUI to display a relative position of the first icon with respect to the route assigned to the autonomous aircraft as the autonomous aircraft progresses along the route; displaying on the GUI a portion of the completed route using a first visual cue and a remaining portion of the route using a second visual cue; The system of claim 12 , further comprising:

15. The instructions, when executed by the one or more processors, cause the one or more processors to: displaying a flight guidance pane on the GUI; receiving a modification on the flight guidance pane corresponding to a desired change in one or more flight parameters associated with the autonomous aircraft represented by the first icon; sending the command to the autonomous aircraft represented by the first icon to effectuate the desired change; The system of claim 12 , further comprising:

16. The instructions, when executed by the one or more processors, cause the one or more processors to: displaying flight parameters associated with the autonomous aircraft near the first icon representing the autonomous aircraft on the GUI, the flight parameters including one or more of an altitude, a heading, or a speed of the autonomous aircraft. The system of claim 12 , further comprising:

17. The instructions, when executed by the one or more processors, cause the one or more processors to: receiving a message from the autonomous aircraft, the message indicating a hazard detected by the autonomous aircraft along a route and an alternative route to be followed by the autonomous aircraft; displaying the collision avoidance alert and the alternative route on the GUI; The system of claim 12 , further comprising:

18. The instructions, when executed by the one or more processors, cause the one or more processors to: receiving positioning signals from the plurality of autonomous air vehicles; displaying the plurality of icons representing the plurality of autonomous air vehicles on a map based on the positioning signals; The system of claim 12 , further comprising:

19. The instructions, when executed by the one or more processors, cause the one or more processors to: receiving flight data messages from the plurality of autonomous air vehicles, each flight data message including one or more flight parameters associated with one of the plurality of autonomous air vehicles; displaying, on the GUI, the plurality of icons representing the plurality of autonomous air vehicles along with one or more corresponding flight parameters; The system of claim 12 , further comprising:

20. One or more computer-readable storage media storing instructions that, when executed on a flight management platform having a server computer, cause the server computer to: Displaying a graphical user interface (GUI) on a display device; representing a plurality of autonomous air vehicles by a plurality of icons on a first pane of the GUI; displaying information associated with each one of the plurality of autonomous air vehicles in a plurality of sections on a second pane of the GUI; receiving an input selecting one of a first icon among the plurality of icons on the first pane of the GUI or a first section among the plurality of sections on the second pane of the GUI; Identifying the other of the first icon among the plurality of icons on the first pane of the GUI or the first section among the plurality of sections on the second pane of the GUI; displaying the first icon and the first section using one or more visual cues to distinguish the first icon and the first section from the remaining graphical elements; sending a command or message to the autonomous air vehicle represented by the first icon on the GUI; One or more computer-readable storage media for performing steps comprising:

Images