Generate flight plans for semi-autonomous drones

Abstract

A system for generating an environment for operations with assets includes one or more processors configured to obtain data associated with one or more tasks performed by the assets, where 1) the assets include semi-autonomous drones, and 2) the data associated with the tasks includes flight plans of other drones, determine discrete representations of geographic locations, the discrete representations including discrete elements each corresponding to a volume associated with the geographic location, annotate the discrete representations with the flight plans of the other drones to create an annotated representation, determine a first flight plan for one drone, the first flight plan determined based on the annotated representation, and communicate information regarding the first flight plan to the at least one other asset.

Description

[Background technology]

[0001] Conventional unmanned aerial vehicles (UAVs), or drones, are useful for performing many tasks. Such drones may perform surveillance, deliver commercial cargo or weapons, map remote or inaccessible areas, and / or other missions. While useful, such drones suffer from many drawbacks. For example, drones are typically remotely piloted, may be unreliable, may be slower than desired, may have limited range, and / or may have other issues that adversely affect performance. Furthermore, coordination of multiple assets, including one or more drones, is complex and generally requires a user at a control center to control and coordinate the assets in connection with the performance of a task. [Brief explanation of the drawings]

[0002] Various embodiments of the present invention are disclosed in the following detailed description and the accompanying drawings.

[0003] [Figure 1] 1 illustrates a system for performing operations in accordance with various embodiments of the present application.

[0004] [Figure 2] FIG. 1 is a block diagram illustrating a device for configuring or controlling operations in accordance with various embodiments of the present application.

[0005] [Figure 3] FIG. 1 is a block diagram illustrating a device for performing at least some of the tasks in accordance with various embodiments of the present application.

[0006] [Figure 4A] 1 illustrates a system for performing at least some of the operations in accordance with various embodiments of the present application.

[0007] [Figure 4B]1 illustrates a system for performing at least some of the operations in accordance with various embodiments of the present application.

[0008] [Figure 4C] 1 illustrates a system for performing at least some of the operations in accordance with various embodiments of the present application.

[0009] [Figure 5A] 1 illustrates a system for performing at least some of the operations in accordance with various embodiments of the present application.

[0010] [Figure 5B] 1 illustrates a system for performing at least some of the operations in accordance with various embodiments of the present application.

[0011] [Figure 5C] 1 illustrates a system for performing at least some of the operations in accordance with various embodiments of the present application.

[0012] [Figure 6] 1A-1C illustrate user interfaces for configuring, monitoring, and / or controlling operations in accordance with various embodiments of the present application.

[0013] [Figure 7A] FIG. 1 illustrates a method for structuring work in accordance with various embodiments of the present application.

[0014] [Figure 7B] FIG. 1 illustrates a method for structuring work in accordance with various embodiments of the present application.

[0015] [Figure 8A] 1 illustrates a method for performing at least one task of a work in accordance with various embodiments herein.

[0016] [Figure 8B]1 illustrates a method for performing at least one task of a work in accordance with various embodiments herein.

[0017] [Figure 8C] 1 illustrates a method for performing at least one task of a work in accordance with various embodiments herein.

[0018] [Figure 9A] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0019] [Figure 9B] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0020] [Figure 9C] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0021] [Figure 9D] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0022] [Figure 9E] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0023] [Figure 10A] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0024] [Figure 10B] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0025] [Figure 11]FIG. 1 illustrates a method for structuring work in accordance with various embodiments of the present application.

[0026] [Figure 12A] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0027] [Figure 12B] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0028] [Figure 12C] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0029] [Figure 13] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0030] [Figure 14] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0031] [Figure 15] 1 illustrates a method for implementing a plan associated with an operation in accordance with various embodiments of the present application.

[0032] [Figure 16A] 1 illustrates a discrete representation in accordance with various embodiments of the present application.

[0033] [Figure 16B] 1 illustrates a discrete representation of geographic locations in accordance with various embodiments of the present application.

[0034] [Figure 16C] 1 illustrates a discrete representation of geographic locations in accordance with various embodiments of the present application.

[0035] [Figure 17A] 1 illustrates a discrete representation of geographic locations in accordance with various embodiments of the present application.

[0036] [Figure 17B] 1 illustrates a discrete representation of geographic locations in accordance with various embodiments of the present application.

[0037] [Figure 17C] 1 illustrates a discrete representation of geographic locations in accordance with various embodiments of the present application.

[0038] [Figure 17D] 1 illustrates a discrete representation of geographic locations in accordance with various embodiments of the present application.

[0039] [Figure 18] 1 illustrates a method for determining a plan for performing one or more tasks according to various embodiments of the present application.

[0040] [Figure 19A] 1 illustrates a method for determining a flight plan in accordance with various embodiments of the present application.

[0041] [Figure 19B] 1 illustrates a method for determining a flight plan in accordance with various embodiments of the present application. DETAILED DESCRIPTION OF THE INVENTION

[0042] The present invention may be embodied in various forms, including as a process, an apparatus, a system, a composition of matter, a computer program product embodied on a computer-readable storage medium, and / or a processor configured to execute instructions stored in and / or provided by a memory coupled to the processor. These embodiments, or any other form the present invention may take, may be referred to herein as technology. In general, the order of steps in a disclosed process may be varied within the scope of the present invention. Unless otherwise noted, components, such as a processor or memory, described as configured to perform a task may be implemented as general components temporarily configured to perform the task at a given time, or as specific components manufactured to perform the task. As used herein, the term “processor” refers to one or more devices, circuits, and / or processing cores configured to process data, such as computer program instructions.

[0043] The following is a detailed description of one or more embodiments of the present invention with reference to figures that illustrate the principles of the invention. While the present invention has been described in connection with such embodiments, it is not limited to any particular embodiment. The scope of the present invention is limited only by the claims, and the present invention includes many alternatives, modifications, and equivalents. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. These details are for the purpose of example, and the present invention may be practiced according to the claims without some or all of these specific details. For simplicity, technical matters that are well known in the art related to the present invention have not been described in detail so as not to unnecessarily obscure the present invention.

[0044] According to various embodiments, a system for grouping assets is disclosed. The system may include a communications interface and one or more processors coupled to the communications interface. The one or more processors may be configured to: (i) receive, via the communications interface, data associated with one or more tasks performed by assets, the assets including a plurality of drones, the drones being at least semi-autonomous; (ii) determine, from among the plurality of assets, a group of assets for performing the one or more tasks; and (iii) communicate, via the communications interface, instructions to at least one drone among the assets, the instructions directing that the one or more tasks be at least partially completed by the at least one drone. Determining the group of assets for performing the one or more tasks may include determining one or more functions associated with one or more features of the one or more tasks, and determining the plurality of drones based at least in part on (i) the one or more functions associated with one or more features of the one or more tasks and (ii) one or more drone functions associated with each of the plurality of drones.

[0045] According to various embodiments, a drone is disclosed. The drone may be a semi-autonomous drone. The drone may include a communications interface and one or more processors coupled to the communications interface. The one or more processors may be configured to (i) receive, via the communications interface, an indication that the drone is part of a constellation of assets, the constellation being tasked with performing one or more elements of one or more tasks, the constellation including multiple drones; (ii) communicate, via the communications interface, information regarding the one or more elements; and (iii) communicate, via the communications interface, information regarding a plan for performing the one or more tasks, the information regarding the plan for performing the one or more tasks being communicated with at least one other drone in the constellation of assets. The information regarding the one or more elements may be communicated with at least one other drone in the constellation of assets. The information regarding the one or more elements may be based at least in part on information acquired by one or more sensors of the constellation of assets. The information regarding the one or more elements may be used in connection with determining a plan for performing the one or more tasks.

[0046] According to various embodiments, a drone is disclosed. The drone may be a semi-autonomous drone. The drone may include a communications interface and one or more processors coupled to the communications interface. The one or more processors may be configured to (i) receive, via the communications interface, an indication that the drone is part of a constellation of assets, the constellation being tasked with performing one or more elements of one or more tasks, the constellation including multiple drones; (ii) determine that at least one drone in the constellation of assets has experienced a malfunction; (iii) update a plan to perform the one or more tasks to an updated plan in response to determining that the at least one drone has experienced a malfunction; and (iv) communicate information regarding the updated plan via the communications interface, the information regarding the updated plan being communicated to at least one remaining drone in the constellation of assets. The one or more processors may be further configured to communicate, via the communications interface, information obtained during execution of at least one element of the updated plan, the information regarding the updated plan being communicated to at least one remaining asset in the set of drones.

[0047] According to various embodiments, a system for dynamically grouping assets is disclosed. The system may include a communications interface and one or more processors coupled to the communications interface. The one or more processors may be configured to: (i) acquire data associated with one or more tasks performed by a group of assets, the group of assets including a plurality of drones, the plurality of drones being at least semi-autonomous; (ii) determine to modify the group of assets based at least in part on the data associated with the one or more tasks; and (iii) communicate, via the communications interface, instructions to at least one drone among the group of assets, the instructions directing the modification to the group of assets. Determining to modify the group of assets based at least in part on the data associated with the one or more tasks may include determining one or more functions associated with one or more remaining tasks, and determining the modification to the group of assets based at least in part on (i) the one or more functions associated with the one or more remaining tasks and (ii) one or more drone functions respectively associated with the assets in the set.

[0048] According to various embodiments, a system is disclosed. The system may include a communications interface and one or more processors coupled to the communications interface. The one or more processors may be configured to: (i) display a first user interface, the first user interface including one or more selectable elements associated with characteristics of one or more tasks to be performed; (ii) receive, via the first user interface, one or more user selections related to the characteristics of the one or more tasks to be performed; (iii) display a second user interface in response to receiving the one or more user selections entered into the first interface; (iv) receive, via the second user interface, one or more user selections related to a group of one or more assets to be deployed to perform the operations; (v) determine an operation to be performed, the operation being determined based at least in part on (a) the one or more user selections related to the characteristics of the one or more tasks to be performed and (b) the one or more user selections related to the group of one or more assets to be deployed to perform the operations; and (vi) communicate information related to the operation via the communications interface. The second user interface may be configured at least in part based on at least one of the one or more user selections entered into the user interface. The second user interface may include one or more selectable elements associated with a fleet of one or more assets performing a task, where the fleet of one or more assets includes one or more drones, and the one or more drones are semi-autonomous. Information regarding the task is communicated to at least one drone in the fleet of assets. The information regarding the task causes the fleet of assets to be deployed to perform at least a portion of the task.

[0049] According to various embodiments, a system is disclosed. The system may include a communications interface and one or more processors coupled to the communications interface. The one or more processors may be configured to (i) obtain information associated with one or more tasks performed by a fleet of assets; (ii) determine a discrete representation of a geographic area, the discrete representation including a plurality of discrete elements, each corresponding to a volume within the geographic area; (iii) annotate the discrete representation to create an annotated representation with one or more parameters related to the geographic area comprising at least a subset of the plurality of discrete elements based at least in part on determining that the one or more parameters relate to the geographic area; (iv) determine a plan for performing the one or more tasks, the plan based at least in part on the annotated representation; and (v) cause the one or more tasks to be performed at least in part based on the plan. The fleet of assets may include a plurality of drones, the plurality of drones being at least semi-autonomous. The data associated with the one or more tasks may include one or more parameters related to the geographic area in which at least one asset in the fleet of assets performs the one or more tasks.

[0050] According to various embodiments, a system is disclosed. The system may include a communications interface and one or more processors coupled to the communications interface. The one or more processors may be configured to (i) obtain data associated with one or more tasks performed by a constellation of assets, (ii) determine a discrete representation of a geographic area, where the discrete representation includes a plurality of discrete elements each corresponding to a volume within the geographic area, (iii) annotate the discrete representation with flight plans of one or more other drones to create an annotated representation, (iv) determine a first flight plan for at least one drone of the plurality of drones, where the first flight plan is determined at least in part based on the annotated representation, and (v) communicate information regarding the first flight plan to at least one other asset in the constellation of assets. The constellation of assets may include a plurality of drones, and the plurality of drones may be at least semi-autonomous. The data associated with the one or more tasks may include flight plans of the one or more other drones.

[0051] According to various embodiments, a system receives high-level instructions for performing a task using a swarm of drones. The high-level instructions may include a high-level description or definition of the task. In response to receiving the high-level instructions, the system may determine one or more lower-level instructions related to performing the task using the swarm of drones. For example, a semi-autonomous drone, such as a leader drone, may break down the high-level instructions into one or more tasks for the swarm, and the semi-autonomous drone may communicate the one or more tasks (or a plan for completing one or more tasks or task elements) to one or more drones (e.g., follower drones) in the swarm. In some embodiments, a user or a control system provides a high-level definition of the task, and one or more semi-autonomous drones autonomously determine the tasks or task elements associated with the task. The swarm of drones (e.g., including one or more semi-autonomous drones) then performs the tasks or task elements associated with the task.

[0052] As used herein, a “semi-autonomous drone” refers to a drone that, without human intervention, determines a plan for performing at least a portion of a task or task based on high-level instructions for performing the task or task, receives feedback information regarding the status of the drone and / or the plan, and updates or determines a new plan to accommodate the feedback information. The updated or new plan may provide instructions for controlling the drone in a manner consistent with the high-level instructions or task. The high-level instructions may be provided to the drone by a user interface or by a server via a network connection. In response to receiving the high-level instructions, the drone determines a plan for performing the task or portion of the task and executes the plan or commands another asset to perform the task or portion of the task and provide feedback to the drone. The term “semi-autonomous drone” may be used interchangeably with “drone” herein. For example, a task “scan a polygon and track a moving target” may involve one or more drones planning how to effectively scan the polygon; however, during the scan pattern, a new target may appear, and the semi-autonomous drone re-plans and begins tracking the new target.

[0053] High-level instructions may include one or more of an indication of the type of work, the location where the work is to be performed, the target of the work, the time of day the work is to be performed, etc. Low-level instructions may provide more specific information or definition regarding the work. In some embodiments, low-level instructions may indicate specific assets to perform tasks within the work, how the tasks are to be performed (e.g., speed, altitude, type of sensor, etc.). Low-level instructions may further be determined based on environmental or situational information within the area where the work is to be performed. An example of a high-level instruction may be an instruction to perform a work to monitor a specific road on a specific date. In contrast, a corresponding low-level instruction may include an instruction to an asset in a group of assets indicating a specific location (or range of locations) along the road from which the specific asset will capture information. A leader drone may determine the specific follower drone to execute the low-level instruction. The low-level instruction may indicate the type of information to be captured, the speed at which the asset will travel, and the altitude at which the asset will travel or perform monitoring. As another example, a high-level command may include "observe this target from several good vantage points," and the corresponding low-level command may be "fly drone 1 100 meters away from the target on a 30-degree heading, and drone 2 150 meters away on a 180-degree heading."

[0054] As used herein, an “asset” may correspond to a device, terminal, vehicle, and / or system that may receive information from a server (e.g., directly or via another asset), utilize sensors to acquire information, communicate the collected information to a leader drone / asset or server, deliver a payload, and / or execute a plan to perform a task or elements of a task. Examples of assets include, without limitation, drones, satellites, communication towers, observation stations, vehicles such as autonomous or semi-autonomous vehicles (e.g., airplanes, boats, cars, trucks, helicopters, etc.), drones (e.g., large drones, small drones, etc.), jet aircraft, and small autonomous sensors (e.g., sensors, “dust” sensors, etc.). An asset has an associated identifier and a defined set of features, functions, characteristics, etc. (collectively referred to herein as “capabilities”). The defined set of capabilities may indicate the set of functions that the asset may perform. For example, the set of capabilities may indicate a set of peripherals (cameras, sensors, payloads, payload deployment mechanisms, pre-loaded software, etc.) that are included with or otherwise operatively connected to the asset. As another example, the set of defined capabilities may indicate whether the asset includes a turret (e.g., a multi-axis pointing mechanism to which other functions are attached (e.g., a pan-tilt unit on a surveillance camera)). As another example, the set of defined capabilities may indicate whether the asset can send and receive data from other communication sources, such as disabling / enabling transmissions.

[0055] As used herein, "planning information" may correspond to information regarding a plan for performing tasks, elements, and / or work. By way of example, planning information may include: task , elements, and / or parameters for the operation. As an example, planning information may include instructions for operating assets.

[0056] As used herein, "capability information" may correspond to information regarding the capabilities of an asset (e.g., a drone). As an example, the capability information may include an indication of a capability and / or set of capabilities of the drone.

[0057] As used herein, "sensor information" may correspond to information obtained by a sensor (e.g., a sensor on an asset). The sensor information may be processed locally by the asset or device, such as to convert raw sensor input into a predetermined format.

[0058] As used herein, "status information" may correspond to information regarding the status or state of a device, such as an asset (e.g., a drone). As an example, status information may include information indicating the status of a particular function or set of functions of a drone. As an example, status information may include information indicating the status of one or more tasks (e.g., the extent to which an asset has completed a task). As an example, status information may include information regarding the results of the execution of a task, element, and / or work.

[0059] Various embodiments include a system for configuring a task to be performed by a group of assets, such as semi-autonomous drones. The system may include one or more servers, client terminals, and / or a group of one or more assets. The server, client terminal, and the group of one or more assets may communicate through the one or more assets. The client terminal may provide a user interface through which the task can be configured (e.g., defined), the assets may be deployed to perform at least a portion of the task, and feedback regarding the status of the execution of the task may be provided. In connection with configuring the task, the assets to perform at least a portion of the task may be determined. The portions of the task may be determined by a semi-autonomous asset, such as a leader drone. For example, the leader drone may analyze the task (e.g., high-level instructions for performing the task) and determine one or more tasks each corresponding to at least a portion of the task. For example, a first task may correspond to a first portion of the task, and a second task may correspond to a second portion of the task. The assets may at least semi-autonomously determine a plan for performing at least a portion of the task and implement the plan.

[0060] In some embodiments, the server may determine a set of assets to perform one or more tasks associated with the work. The set of assets may be determined based at least in part on one or more functions associated with one or more characteristics of the one or more tasks. For example, the one or more functions associated with one or more characteristics of the one or more tasks may correspond to a function to be performed (e.g., for a surveillance task, the function may correspond to video capture, image capture, etc.; for a drone delivery task, the function may correspond to a flight range, a mechanism for carrying a payload, a weight limit for the payload, etc.). The set of assets to perform the one or more tasks may be determined based at least in part on one or more drone functions associated with each of a plurality of drones. For example, in response to determining the configuration of the work, the server may determine one or more tasks corresponding to the work and determine functions associated with the tasks (e.g., based at least in part on a mapping of tasks to functions, etc.). The server may determine one or more assets that match the functions associated with the task (e.g., based at least in part on the mapping of assets to functions) and then determine a group of assets to perform one or more tasks or to add an asset to a predetermined group of assets to perform one or more tasks. In response to deploying the group of assets to perform the work or determining the group of assets, at least one asset in the group of assets (e.g., a leader drone) may provide information about the work (e.g., one or more characteristics or identifiers associated with the one or more tasks, an indication of the group of assets to perform the one or more tasks, etc.). In some embodiments, multiple assets in the group of assets, or each asset in the group of assets, are provided with an indication that the particular asset will be used in connection with performing one or more tasks. The group of assets may be determined at least in part based on the availability of one or more assets. For example, asset ownership (e.g., lease agreements) may be used in connection with determining whether to include an asset in a group of assets to perform one or more tasks.

[0061] Various embodiments include a grouping of assets that perform at least a portion of a task (e.g., one or more tasks). The asset group may include one or more semi-autonomous drones. In response to configuring the task (e.g., on a server), high-level instructions for the task (or a portion thereof (e.g., a task)) are provided to at least one drone in the asset group. The high-level instructions may include a high-level definition of the task, such as one or more characteristics of the task or task (e.g., the task to be performed, the location where the task is to be performed, the time the task is to be performed, the area in which the assets are permitted to operate, etc.). In response to receiving information about the task (e.g., an indication that the drone is included in the asset group), the semi-autonomous drone may determine one or more elements of the task or tasks to be performed by the asset group. The semi-autonomous drone may be a leader drone among multiple drones in the asset group. In some embodiments, the leader drone may determine a plan for at least one drone in the asset group to perform at least one element of the task or tasks. For example, the leader drone may execute (e.g., locally by a processor on the drone) a planning service that determines a plan for executing one or more tasks assigned to the fleet of assets. Determining the plan for executing the one or more tasks may be based at least in part on one or more of: (i) the functionality to be performed; (ii) the environment in which the task is to be performed; (iii) the functionality associated with the task to be performed; (iv) the functionality of at least one drone in the fleet of assets; etc. The plan may be updated based on feedback information provided by one or more assets in the fleet (e.g., indications of asset failure, changes in the environment, location of targets, etc.).

[0062] According to various embodiments, assets within an asset group may communicate with one another. For example, a leader drone within an asset group may provide information regarding at least one element of a task (e.g., a plan for an asset to perform an element of a task or perform a task). As another example, another drone within the asset group or another asset within the asset group may send information to the leader drone regarding the status of the execution of the plan, the status of the asset (e.g., the asset's environment (weather, etc.)), etc.). In some embodiments, one or more assets (e.g., drones) may listen to information communicated between the leader drone and follower drones or other assets within the asset group. For example, a dormant leader may obtain information communicated between the assets to enable a relatively seamless takeover of leader responsibilities / roles in response to a determination that the leader drone has failed or in response to a determination that a dormant drone will become the leader of a partition of the asset group. The leader drone may also communicate with a control center, such as a server, that manages / coordinates the execution of the work. For example, the leader drone may provide feedback information regarding the work (e.g., the real-time status of the work, such as an indication of whether one or more tasks are completed, etc.). As another example, the leader drone may send a request for additional assets or an indication that some of the assets are being relieved of work, such as when some of the assets have completed corresponding elements of one or more tasks and the functionality associated with some of the assets is no longer needed to perform one or more tasks.

[0063] According to various embodiments, an asset fleet may semi-autonomously respond to the failure of one or more assets within the asset fleet. For example, in response to a control center (e.g., a server) providing an indication that the assets will perform a task or one or more tasks associated with the task, the assets may break down the task or task to determine a plan to be implemented in connection with the execution of the task or task associated with the asset fleet. During the execution of the plan, an asset may fail due to a loss of power, a loss of communication with the asset fleet (leader drone), a crash, an interaction with a third party, or the like. In response to the failure of an asset within the asset fleet, the remaining assets within the asset fleet may continue to execute the task or task associated with the asset fleet without human intervention. The remaining assets may determine that the asset has failed and update the plan for executing the task or task, for example, to reallocate elements or functions assigned to the failed asset. In response to determining that the leader drone has failed, another drone within the asset fleet may be promoted to the role of leader drone among the remaining assets of the asset fleet. The drone promoted to leader drone may be based at least in part on a predetermined ranking / priority, etc. In some examples, a dormant leader is promoted to the role of leader drone upon failure of the current leader drone. The dormant leader may correspond to the highest-ranking / priority drone among the assets (e.g., next to the leader drone), as indicated by a predetermined ranking / priority, etc. The dormant leader may synchronize with the leader drone in real time or at predetermined intervals (e.g., short intervals to ensure that the dormant leader can seamlessly assume the role of leader drone, if necessary).

[0064] In various embodiments, the group of assets determined to perform a work (or one or more tasks associated with a work) may be updated during the performance of the work. In some examples, the group of assets may be updated based on the features or functions of tasks or elements of tasks that have not yet been performed and / or the features or functions of assets in the group of assets. For example, if an asset in the group of assets does not have the features / features of the remaining tasks to be performed, the asset may be released. As another example, if the group of assets does not have assets with features / features that match the features / features of the remaining tasks to be performed, or if there are an insufficient number of such assets in the group of assets, the group of assets may be updated to include one or more assets with features / features that match the features / features of the remaining tasks to be performed. Updates to the group of assets may be determined by a control center (e.g., a server) that manages or coordinates the work, and the updates to the group of assets may be communicated to at least a leader drone in the group of assets.

[0065] According to various embodiments, one or more user interfaces may be provided in connection with one or more of configuring the job, communicating (e.g., real-time updates) the status of the job, and / or updating the job (e.g., based on user input, etc.). In conventional systems for defining a job, a user inputs the job parameters, the specific assets to be used, and the plan to be implemented with a high degree of specificity. In contrast, various embodiments include conditional display of various user interfaces that allow a user to configure the job, where a high-level plan or definition of the job is provided to at least a leader drone, and assets (e.g., the leader drone) determine plans for executing tasks or task elements decomposed from the high-level plan or definition of the job. In this manner, various embodiments provide a user interface wizard for inputting one or more characteristics or requirements of the job. The server may cause a client terminal to display user interfaces to a user. The terminal may be caused to display a first user interface, the first user interface comprising one or more selectable elements associated with characteristics of one or more tasks to be performed, and in response to input to the user interface, the terminal may be caused to display a second interface associated with the assets performing the job. The second user interface may be configured based on input to the first user interface. In response to the input to the second user interface, an operation may be determined. The server may cause the terminal to sequentially configure and provide user interfaces based at least in part on user input to previous user interfaces, and the series of user interfaces may be used to collectively determine the operation to be performed. After the operation is determined and the assets are instructed to perform the operation, the server may cause the terminal to provide a user interface that includes the current status of the operation (e.g., a real-time feed or real-time updates to various parts of the operation), etc.In some embodiments, a user may enter one or more inputs into the user interface while a task is being performed in connection with pausing the task, updating the task (e.g., requesting faster completion of the task, such as by increasing the number of assets in the group of assets, requesting more fire capabilities, requesting faster coverage of a geographic area, etc.), canceling the task, etc.

[0066] In some embodiments, the system determines a discrete representation of a geographic location. The geographic location may correspond to a location where an operation is performed or a location where a task (or element of a task) of the operation is performed. The discrete representation of the geographic location may include multiple discrete elements, each corresponding to a volume at the geographic location. The system may use the discrete representation of the geographic location in connection with determining and / or communicating parameters associated with the operation (e.g., parameters associated with the task or element of the task), determining a plan for performing the task, and determining flight paths (e.g., trajectories) for assets (e.g., drones) of the asset group determined to perform the operation (e.g., set of one or more tasks). According to various embodiments, multiple assets in the asset group are configured to determine the discrete representation. For example, an asset performing a planning service may be configured to determine the discrete representation of locations of tasks or elements assigned (e.g., allocated) to a location or asset corresponding to the operation. As an example, the discrete representation is determined locally at an asset in the asset group performing one or more tasks. In some embodiments, multiple assets determine their own respective versions of the discrete representation of the location based at least in part on information communicated regarding one or more tasks.As an example, the leader drone and / or follower drone may each determine a discrete representation of a geographic location (e.g., a local version of the discrete representation) based at least in part on one or more of the following: (i) information about one or more tasks, such as one or more parameters or characteristics associated with one or more tasks received from a server (e.g., a server associated with the configuration of the related work) (e.g., keep-in / keep-out definitions for one or more tasks, target locations, maximum altitude, etc.); (ii) information about the geographic location from a third-party service, such as a service for map information (e.g., geographic information), a service for weather information, or a service for information obtained by another asset or sensor (e.g., a camera, satellite, etc.) at or near the geographic location; and (iii) information about the geographic location or one or more tasks from another asset in the group of assets assigned to the work (e.g., the status of execution of one or more tasks, information associated with communication line-of-sight, a flight plan for the asset, an indication that a particular location / volume / area is occupied, etc.).

[0067] According to various embodiments, determining a discrete representation of the geographic location includes determining (e.g., creating, generating, populating, etc.) a 3-D representation of the geographic location, the 3-D representation including a plurality of discrete elements each corresponding to a voxel. In some embodiments, the 3-D representation of the geographic location may be determined at least in part based on transforming the geographic location from a real-world representation into a plurality of discrete elements based at least in part on the curvature of the Earth, with the downward direction in the grid of the 3-D representation corresponding to the direction of gravity. The dimensions of the 3-D representation may be configured to include a substantially larger number of discrete elements (e.g., boxes) in the x-axis and y-axis compared to the number of discrete elements in the z-axis. The z-axis may be an axis parallel to gravity. As an example, when an asset moves downward in the z-direction across the 3-D representation, the asset moves downward toward the ground; conversely, when the asset moves upward across the 3-D representation in the z-direction, the asset moves toward the sky. The length or dimension of the discrete representation in the z-axis may be configured based at least in part on a maximum elevation configuration set in association with the configuration of the task. For example, the system may have a default maximum elevation associated with the task (or associated with various types of tasks). As another example, a user may input a maximum elevation during task configuration.

[0068] According to various embodiments, a system (e.g., a server, a leader drone, a follower drone, etc.) generates a model of a geographic location associated with one or more tasks. Generating the model of the geographic location may include determining a discrete representation of the geographic location and annotating the discrete representation to create an annotated representation. For example, the annotated representation may correspond to the model of the geographic location. In some embodiments, the system updates the annotated representation based at least in part on received information regarding one or more tasks and / or geographic locations. For example, the system may repeatedly update the annotated representation periodically, such as when updated or new information regarding one or more tasks and / or geographic locations is received. The annotated representation may be updated until one or more tasks are completed (e.g., until a determination is made that work is completed, stopped, paused, etc.). In some embodiments, annotating the discrete representation of the geographic location includes associating metadata with one or more discrete elements of the discrete representation, respectively. The metadata may include values ​​or suggestions associated with one or more tasks, geographic locations, etc.

[0069] Various embodiments for managing, coordinating, and / or executing work increase efficiency in utilizing resources (e.g., assets) assigned to perform work and improve fault tolerance of work execution by enabling dynamic grouping of assets and / or dynamic updating of plans for performing tasks / elements associated with the work. Various embodiments increase efficiency and effectiveness in defining work, matching assets to perform the work, and determining how the work will be performed. In related art methods, a human operator selects assets to utilize in performing the work, and a centralized service (e.g., running on a server) defines the work in more detail. The centralized service provides a detailed work plan for performing the work. Various embodiments include a user interface system configured to intuitively allow a user to provide a high-level description of the desired work. Various embodiments provide an efficient method for grouping assets to perform the work. Asset grouping may be based at least in part on automatic negotiation of ownership availability and functionality and asset operations associated with the work. Asset grouping is more efficient and provides better organized task execution. In the related art, systems for defining operations generally require a human operator to select assets to use in performing the operation, and the pool of assets available for selection by the human operator may be further limited because ownership availability is not readily discernible. In some embodiments, systems for executing operations are more fault-tolerant. For example, utilizing a leader drone (e.g., a semi-autonomous drone) in connection with determining one or more tasks and to coordinate / manage the execution of the tasks by one or more assets (e.g., follower drones) in the pool of assets provides a more efficient way to dynamically update plans for executing tasks or task elements associated with the operation.As another example, dynamic grouping of assets to perform work is more efficient because assets with capabilities that are no longer needed to perform the remaining set of tasks are released from the asset group (and made available for reassignment to another work). Dynamic grouping of assets is also more fault-tolerant because additional assets can be assigned to the asset group in response to asset failures or changes in the work's conditions. Dynamic management / coordination of work execution (e.g., by a leader drone) allows tasks to be quickly reassigned to other assets in the asset group in response to changes in asset capabilities and / or changes in the work's conditions (e.g., environment), etc.

[0070] 1 is a diagram illustrating a system for performing operations in accordance with various embodiments of the present application. In the example illustrated in FIG. 1, system 100 may include server 105 and one or more assets 120, 125, and 130. System 100 may further include network 110, network 115, and / or client terminal 135. Server 105 and assets 120, 125, and / or or , 130 may communicate with one another, such as via one or more networks (e.g., network 110, network 115, and / or any other suitable network), which may include wired and / or wireless networks (e.g., cellular networks, wireless local area networks (WLANs), etc.). According to various embodiments, server 105 may correspond to a single server or multiple servers. Similarly, system 100 may include various other assets.

[0071] In some embodiments, the server 105 may configure a task, such as determining a group of assets to perform the task. The group of assets may include one or more assets 120, 125, and / or 130. The server 105 may communicate an indication that an asset will perform the task and / or an indication of the group of assets (e.g., a list of assets in the group, such as a list of asset identifiers) to at least one asset in the group of assets. In some embodiments, the indication that an asset will perform the task may include a high-level definition or description of the task, such as a high-level task to be performed (e.g., monitor an area or target, deliver a payload, etc.). At least one asset in the group of assets (e.g., a leader drone) may resolve the high-level definition or description of the task and autonomously determine a plan for at least some of the assets to perform / complete the task. The high-level definition or description of the task may further include several parameters of the task, such as a location, an indication of restricted areas where the assets should not enter / operate, an indication of an area / zone where the assets should operate, etc. After receiving the high-level definition or description of the task, at least one asset (e.g., a leader asset) may automatically determine a plan for at least some of the assets to perform / complete the task, and the at least one asset may cause the assets to execute the plan and communicate information about the task (e.g., the current state of the task, etc.) to the server 105 via the network 110 and / or the network 115. For example, the leader drone may decompose the high-level definition or description of the task and determine a plan for one or more tasks corresponding to the task. In some embodiments, a discrete representation of a geographic location associated with one or more tasks is generated. As an example, the discrete representation of the geographic location is generated based at least in part on the geographic location and / or information about the one or more tasks. The information is received from the server 105 and / or another asset in the asset fleet (e.g., a follower asset, a leader drone, etc.). In some embodiments, a plan for one or more tasks is determined based at least in part on the discrete representation.For example, the discrete representation is annotated to create an annotated representation. As an example, annotating the discrete representation includes setting metadata or associating metadata with one or more discrete elements of the discrete representation. In some embodiments, an asset in the asset group (e.g., asset 120, asset 125, and / or asset 130) and / or server 105 uses the annotated representation to determine a plan, such as using information included in the metadata associated with the discrete elements (e.g., to determine a trajectory or flight path of the asset). In response to determining a plan for one or more tasks, the leader drone may transmit the plan for at least one task to another asset in the asset group (e.g., a follower asset). In response to receiving the plan for at least one task, the follower asset may execute the plan and autonomously determine any further instructions to be executed locally to perform the at least one task. The assets performing the work may be dynamically updated (e.g., while the work is being performed) to add assets (e.g., when an asset with a particular function fails and is replaced with a new asset), to remove assets (e.g., for any task requiring a particular function, an asset is included in the assets to provide that function and the task is completed), etc.

[0072] In various embodiments, the server 105 communicates with the client terminal 135 over a network (e.g., network 115). The server 105 may cause the client terminal 135 to display one or more user interfaces in connection with configuring a task, displaying the status of the task, and / or updating the task (e.g., pausing, canceling, or modifying the task). The client terminal 135 may receive one or more user inputs to the various user interfaces and communicate suggestions of the one or more user inputs to the server 105. The server 105 may use the one or more user inputs to determine the various user interfaces to be displayed on the client terminal 135, such as in connection with configuring the various user interfaces to create wizards through which a user defines or inputs task characteristics. During task execution, the server 105 may use the user inputs to change the status or parameters of the task, and in response to such changes, the server 105 may communicate the changes to at least the leader asset / drone (e.g., to have the leader drone implement the changes). For example, in response to changing the status or parameters of the operation, the server 105 may update a high-level definition or description of the operation, and the server 105 provides corresponding high-level instructions or updates to a leader drone in the fleet of assets performing the operation. In some embodiments, the server 105 receives information (e.g., feedback information) about the operation from the fleet of assets (e.g., from the leader drone). The information about the operation may indicate the status of the operation, real-time images or video of the operation, etc. In response to receiving the information about the operation, the server 105 may provide a status update, real-time operation status, to the client terminal 135. In some embodiments, the server 105 may determine one or more recommendations or options for updating the operation and provide at least one of the recommendations or options to the user via a user interface displayed on the client terminal 135.As an example, one or more recommendations or options may include a suggestion of the option to increase the number of assets assigned to the asset group to speed up the execution of the work (e.g., it may indicate that adding a certain number of assets may reduce the time to completion by a calculated estimated time).

[0073] FIG. 2 is a block diagram illustrating a device for configuring or controlling a job in accordance with various embodiments of the present application. In the example illustrated in FIG. 2, device 200 may include a communications interface 202 and / or one or more processors 205. Device 200 may correspond to server 105 of FIG. 1, control center 460 of FIGS. 4A-4C, and / or the control center of FIGS. 5A-5C. Device 200 may perform process 700 of FIG. 7A, process 720 of FIG. 7B, process 1000 of FIG. 10A, and / or process 1020 of FIG. 10B, and process 1100 of FIG. 11. According to various embodiments, one or more processors 205 may include or perform one or more of a communications module 210, an asset ownership module 220, a function-to-asset mapping module 230, a job definition module 240, a grouping module 250, a pre-planning module 260, a feedback information module 270, and a user interface module 280.

[0074] Device 200 may implement one or more modules in connection with determining a task, such as one or more characteristics of the task, determining a group of assets to perform the task, and communicating suggestions for the task (e.g., a high-level description of the task) to one or more assets (e.g., a leader drone) in the group of assets. During the execution of the task, device 200 may implement one or more modules in connection with communicating information with at least a leader drone to receive information about the current status of the task or to send instructions for changes to the task, and / or communicating information with a client terminal to provide updates about the status of the task or to receive changes to the task.

[0075] Device 200 may use communication module 210 to communicate with assets, a client terminal, another server or terminal, etc. For example, the communication module may provide communication interface 202 to be communicated. As another example, communication interface 202 may provide information received by device 200 to communication module 210.

[0076] According to various embodiments, the determination of the set of assets to perform the task is based, at least in part, on the ownership of one or more assets in the set of assets. Device 200 may determine to include in the set of assets only assets for which device 200 has ownership (e.g., the organization to which device 200 belongs owns or manages the asset) or for which device 200 may acquire at least temporary ownership. Asset ownership may include the ability to control the asset, such as providing instructions to the asset to perform a specific function (e.g., a task, an element of a task, etc.). As an example, asset ownership may include a lease agreement under which the owner has the ability to control the asset. The lease agreement may be for a fixed / predetermined period of time or may be tied to the completion of a specific task. In some embodiments, the transfer of ownership (e.g., associated with ownership negotiations to determine the set of assets) is a permanent transfer of ownership or is otherwise for an indefinite period of time or until device 200 or its organization relinquishes ownership of the asset. In some examples, an asset fleet is configured to have a leader drone have a longer ownership period (e.g., lease length, permanent ownership) than the ownership periods of other assets (e.g., follower drones) in the asset fleet. In some examples, if it is determined that an operation involves a relatively high likelihood that a follower drone may lose communication while performing the operation (e.g., based on the topography of the area in which the operation is being performed, etc.), device 200 may configure the asset fleet to have a longer ownership period for the follower drone. The longer ownership period may ensure that ownership of the follower drone persists during periods when the follower drone is not in communication with the leader drone and / or device 200, etc. In some embodiments, asset ownership includes a failover entity to which asset ownership is transferred in the event of a failure of the current entity having ownership.Information regarding asset ownership may be stored in one or more of: (i) locally on the asset; (ii) a centralized ownership service, such as a service hosted by one or more servers that provide services across multiple organizations; (iii) the device 200; and (iv) the owner of the asset. Information regarding asset ownership may include one or more of an indication of the current owner; an indication of the length of the current ownership; availability of ownership transfer (e.g., the current owner's willingness to transfer ownership), such as an indication of the ability of another organization to acquire ownership; a current failover owner; an indication of procedures for negotiating a transfer of ownership; etc. In some embodiments, asset ownership may include automatic takeover ownership, whereby a device or organization requesting ownership (e.g., with proper authorization) may automatically take over ownership of the asset, such as upon request of ownership.

[0077] In some embodiments, device 200 may utilize asset ownership module 220 in connection with determining ownership of a particular asset and / or to negotiate the acquisition of ownership of an asset. Asset ownership module 220 may obtain information regarding asset ownership, and device 200 may use such information in connection with determining a group of assets. For example, device 200 may use information regarding asset ownership to determine whether a particular asset is available for allocation / assignment to a group of assets to perform work. Asset ownership module 220 may negotiate with an entity the transfer of asset ownership for use associated with the work. For example, asset ownership module 220 may request a lease for a predetermined period of time. The predetermined period may be the expected length of the work, the expected length of time an asset is needed to perform a particular task or element of a task (e.g., the period of time the asset's functionality is required to perform the work), etc. Asset ownership module 220 may request ownership information from a particular asset, or a particular asset may communicate information regarding the ownership of such asset. In some embodiments, device 200 may transmit a request / command to take ownership of an asset in response to determining a group of assets used in connection with performing a task. For example, device 200 may transmit a request to take ownership of an asset in connection with (e.g., as part of, along with, etc.) providing an asset with an indication of the group of assets, such as an instruction indication indicating that the asset is included in the group of assets. As a further example, device 200 may transmit a request to take ownership of one or more assets in connection with providing the indication of the group of assets to a leader, who can then communicate with one or more assets in the group of assets in connection with taking applicable ownership of the one or more assets (e.g., to perform at least one task in the task).

[0078] According to various embodiments, a set of assets to be used in connection with performing a task is determined and / or updated based at least in part on the capabilities of corresponding assets included in the set of assets. In connection with configuring a task, device 200 may determine a set of capabilities associated with performing the task (e.g., capabilities required to perform various functions included in performing the task). Device 200 may use the set of capabilities associated with performing the task in connection with determining the set of assets. For example, device 200 may match assets in a superset of assets having capabilities matching the set of capabilities associated with performing the task. Device 200 may query a mapping of capabilities to assets (or asset identifiers) in connection with determining a superset of assets having capabilities matching at least one function in the set of capabilities associated with performing the task. In response to determining that a function of the task does not have a matching asset (e.g., an asset with a corresponding capability) in the superset of available assets, device 200 may determine whether the function can be collectively achieved using two or more assets or reconfigure the task. For example, device 200 may provide an indication to the client terminal / user that the work feature does not have a matching asset and / or prompt the user with instructions to reconfigure the work (e.g., based on further user input) or cancel the work.

[0079] In some embodiments, device 200 may use asset-to-capability mapping 230 in connection with determining asset capabilities. Asset-to-capability mapping 230 may store such capability mappings locally or may communicate with a remote service that stores the mappings. Asset-to-capability mapping 230 may query a mapping with a specific identifier of an asset to look up capabilities for such assets. Similarly, asset-to-capability mapping 230 may query a specific capability mapping to look up assets having such capabilities. In some examples, in response to receiving one or more parameters of a task, device 200 may determine one or more capabilities associated with the task and use asset-to-capability mapping 230 to determine assets having matching capabilities. For example, device 200 may determine a group of assets to perform the task based at least in part on determining that the assets collectively have capabilities that match capabilities associated with performing the task. Device 200 may determine a group of assets such that at least one asset in the group has at least one capability that matches capabilities associated with performing the task. The asset-to-capability mapping 230 may be stored locally on the device 200, or a module for obtaining a particular mapping of capabilities may be used to query a remote service that provides the asset-to-capability mapping 230. The asset-to-capability mapping may be updated in response to a determination that the asset's capabilities have changed. For example, an asset may advertise a new capability, a loss of capability (e.g., in response to detecting a peripheral device failure), and / or an update to a capability. In some embodiments, the device 200 may update the asset-to-capability mapping.In some embodiments, a remote service (e.g., a service hosted by a server) updates the functions mapped to particular assets, and device 200 may periodically synchronize updates to the mapping of functions to assets, or device 200 may query the mapping of functions to assets as needed.

[0080] In some embodiments, an asset stores an indication of its capabilities. For example, an asset may store a definition of its capabilities locally, and the asset may update the definition of its capabilities based on the asset's current capabilities in response to failure of a module that provided the functionality (e.g., a peripheral such as a sensor, camera, or added gimbal), addition of a new module (e.g., addition of a new peripheral, new software loaded on the asset, updates to existing software, etc.), etc. The definition of the capabilities may be a list of the asset's capabilities or an indication of the capabilities according to a predetermined format / protocol. In some examples, an asset may advertise its capabilities by transmitting a definition of the capabilities over one or more networks, to a leader drone during execution of a plan / task, etc., or the asset may provide an indication of its capabilities in response to a query from a server (e.g., a server configuring a task), a centralized capabilities server, a leader drone, etc. In some embodiments, device 200 (e.g., asset-to-capability mapping 230) or a remote service managing asset-to-capability mapping may ping one or more assets for updates regarding the current capabilities of the asset or may otherwise provide pings for asset health updates, and the corresponding asset-to-capability mapping may be updated (e.g., to reflect the current set of capabilities and / or the state of the capabilities for the asset or assets).

[0081] According to various embodiments, device 200 may define one or more tasks to be performed by one or more assets. For example, defining the task may include at least one of setting one or more objectives associated with the task, setting a location where the task is to be performed, setting a target for the task, setting one or more restrictions on the performance of the task (e.g., rules of engagement for engaging the target, restricted or permitted airspace, etc.), setting a date for the task, etc. The one or more tasks may be defined based at least in part on one or more user inputs, such as user selections entered into a user interface provided by user interface module 280. In some embodiments, device 200 defines the one or more tasks using task definition module 240. For example, task definition module 240 may receive one or more user inputs by a user from communication module 210 and / or user interface module 280, and task definition module 240 may determine one or more characteristics / parameters associated with the task in connection with defining the task. An example of a task definition may include a task to locate or track a specific target (e.g., person XYZ, truck ABC, etc.) in Los Angeles from January 1, 2021 to January 3, 2021, with the asset not venturing beyond 100 miles of Los Angeles or within two miles of a permanently restricted area or airport. In some embodiments, one or more user inputs associated with the task may include a suggestion of one or more types of assets to be used in connection with performing the task. Continuing with the previous example, the task definition may further include an indication that the task will be performed using a drone and / or a fixed-wing aircraft.

[0082] In some embodiments, a task may be decomposed into a set of one or more associated functions or features. For example, a definition of a task (e.g., a task defined using a task definition module) may be analyzed, and one or more functions associated with the task may be determined based on the analysis. The set of one or more functions associated with the task may be determined based at least in part on querying a mapping of task parameters to functions. The parameters associated with the task may be determined based on the task definition. Examples of task parameters include type of task (e.g., scanning a road network, finding, determining, tracking, loitering flight, scanning along a route, etc.), task classification (e.g., attack, defense, surveillance, etc.), location, the path to be followed when the task is performed (e.g., path for tracking / scanning), type of asset (e.g., drone, fixed wing, helicopter, boat, etc.), target (e.g., personal, airplane, land vehicle, target of unknown type, etc.), range in which to perform the task, location of the task, rule set for performing the task, indication of what can / cannot be performed (e.g., permitted actions), survivability during the task (e.g., how likely is the asset to be shot down or otherwise terminated), communication requirements (e.g., whether the asset will engage in communications silent mode or , indications of when a link may be lost, indications of the type of communication to be utilized, or other parameters or constraints regarding communication during the task, etc.), target behavior (e.g., whether the target is stationary or moving along a road, indications of expected target movement, movement patterns, etc.), potential threats (e.g., locations of sites that can destroy or disable drones, such as SAMs (surface-to-air missile launchers), RF jamming sites, etc.), unlikely target zones (e.g., areas not predicted to contain any targets of interest specified by the user, etc.), maximum altitude (e.g., above sea level) that the asset will travel to, keep-in areas (e.g., areas where the asset will remain), keep-out areas (e.g., areas where the asset is prevented / prohibited from entering), etc. In some examples, the task definition module 240 determines one or more functions associated with the task.In another embodiment, the grouping module 250 determines one or more functions associated with the task.

[0083] According to various embodiments, device 200 determines a group of assets (e.g., a team of assets) to perform a task. Device 200 may automatically determine a group of assets to perform a task in response to a task being defined (e.g., based on one or more user inputs). The group of assets may be determined based at least in part on one or more of: (i) task parameters, (ii) one or more capabilities associated with the task, (iii) capabilities of one or more assets, and (iv) ownership of one or more assets. Various other factors / variables may be used in connection with determining a group of assets to perform a task. In some embodiments, device 200 determines a group of assets based on a best fit based at least in part on one or more factors / variables associated with the assets and / or the task. For example, device 200 may determine a group of assets using a cost function. For example, the cost function may include weightings for various factors used in determining the group of assets. Device 200 may determine a group of assets based on minimizing an overall cost value determined by the cost function for the combination of assets in the asset group. In some embodiments, the assets may be determined such that their corresponding total cost values ​​using a cost function are less than a cost value threshold. The cost value threshold may be configurable (e.g., by an administrator, a user, etc.) in connection with the definition of the job, etc. In some embodiments, the cost value threshold is determined to correspond to a predetermined percentile of all possible sets of assets (e.g., such that the assets performing the job are in the top 10 percent of all possible sets of assets, or some other configurable percentile). An example of a cost function in a context involving pairing assets to road assignments may include weighting a) distance from the road and b) the probability that the target is on the road. Using the above cost function, if there are assets that are far from the road, those assets will not be selected because the distance component of the cost function would exclude them from being desirable assets.Similarly, if there are less likely roads, the less likely roads are scanned in the future, and higher priority roads take precedence.

[0084] In some embodiments, determining the asset group may include determining the number of assets to include in the asset group. The number of assets to include in the asset group may be determined based at least in part on the task (e.g., functionality associated with the task) and the functionality of the assets. For example, determining the number of assets may be based on a cost value of the asset grouping and / or a set of functionality provided by the asset grouping related to the functionality associated with the task. As discussed further below, the number of assets to include in the asset group may be based at least in part on a decision to include redundancy in the asset group with respect to one or more functionality among the assets in the asset group and / or the degree to which redundancy is included among the assets in the asset group.

[0085] The device 200 may determine a group of assets using the grouping module 250. The grouping module 250 may determine a group of assets based at least in part on a definition of the task. For example, the group of assets may be determined at least in part on a set of one or more functions associated with the task and / or one or more task parameters. The grouping module 250 may determine a group of one or more assets having functionality that matches the set of one or more functions associated with the task. As an example, the grouping module 250 may determine a group of assets such that, for each function in the set of one or more functions associated with the task, the group of assets includes at least one asset with the matching functionality.

[0086] In some embodiments, device 200 determines a pre-plan for the work in connection with determining a set of assets to perform the work. Device 200 may determine the pre-plan for the work using pre-plan module 260. A pre-plan for a work may be a high-level description or decomposition of the work (e.g., into a set of tasks or task elements to be performed) to identify what will be done during the performance of the work. Generating the pre-plan in connection with determining a set of assets to be used during the performance of the work enables grouping module 250 to evaluate the functions associated with the work (e.g., various functions required to perform the work) and / or the characteristics or features of assets required or desired to perform the work (e.g., a set of tasks or task elements to be performed during the work). In some embodiments, in response to determining the pre-plan for the work, device 200 determines assets having one or more functions that match the function or requirements of the work.

[0087] Device 200 may determine to include redundancy in the asset group for at least one of one or more functions associated with the operation. For example, device 200 may use grouping module 250 to determine to include redundancy in the asset group based at least in part on the type of operation, operation classification, operation location, etc. (e.g., if the operation type / operation classification / location exceeds a probability threshold or is otherwise associated with a corresponding probability of asset failure that is associated with an elevated risk of failure). If the operation is an offensive operation or is in a hostile environment, the likelihood of asset failure may be elevated. The degree of redundancy that device 200 incorporates into the asset group may be based on the degree of likelihood of asset failure. For example, multiple thresholds may be defined, each including a corresponding likelihood of asset failure, and the multiple thresholds may be used in connection with determining the degree of redundancy to incorporate into the asset group (e.g., the degree of redundancy may be mapped to a particular threshold corresponding to the likelihood of asset failure, and the mapping may be queried to determine the degree of redundancy to apply to the asset group). The degree of redundancy may indicate the number of assets included in an asset group that each have a particular function (e.g., a function that is mapped to an increased risk of asset failure, or a function of an asset that has been identified as having an increased risk of asset failure).

[0088] The device 200 may communicate a suggestion of the assets in response to determining the assets to be used in connection with performing the work. For example, in response to the grouping module 250 determining the assets, the device 200 communicates the suggestion of the assets using the communication module 210. In some embodiments, the device 200 transmits the suggestion to at least a leader (e.g., a leader drone) in the assets. The leader drone may then provide a suggestion to the other assets in the assets that the other assets are in the assets. For example, the leader drone may notify each asset that it is included in a team of assets (e.g., a group of assets) for the work. As another example, the leader drone provides a suggestion of the assets in the group of assets to each asset in the assets so that each member is aware of, for example, the entire group of assets in the team for performing the work. The suggestion provided to the leader drone or to one or more other assets in the assets may be communicated together with or separately from the task or plan associated with performing the work.

[0089] According to various embodiments, a pre-plan for performing a task may be determined via a remote service and then provided to a fleet of assets (e.g., at least a leader drone). The remote service for determining the pre-plan may be performed by device 200, such as using pre-planning module 260 or via one or more third-party pre-planning modules with which pre-planning module 260 or device 200 can communicate. For example, pre-planning module 260 may include an application programming interface (API) for interfacing with device 200 when one or more third-party pre-planning modules provide a pre-plan associated with the task. Pre-planning module 260 and / or the third-party pre-planning modules may obtain a task definition and / or one or more parameters associated with the task. In some embodiments, determining a pre-plan for a task is an iterative process of determining a pre-plan, providing information about the pre-plan to a user (e.g., displaying the pre-plan on a client device) via user interface module 280 or the like, receiving feedback information about the pre-plan (e.g., one or more inputs regarding the pre-plan or parameters of the associated task), and updating the pre-plan based on the feedback information. The updated pre-plan may then be provided to the user, and feedback information regarding the pre-plan may be received for further refinement. Providing the pre-plan and receiving feedback information may be performed iteratively, for example, until the user provides instructions for the operation to begin. As one example, device 200 may determine a pre-plan that indicates the assets will track a target along the path or otherwise move along a particular path, and device 200 may display the path to the user at a client terminal. The user may provide one or more inputs that modify the path of the assets, and device 200 may update the pre-plan accordingly. As another example, device 200 may determine a pre-plan that indicates how the assets will fly when scanning some area.For example, when performing or analyzing a lawn service, if the original polygon shape does not lead to a less than optimal scan, it may be useful to the task to see a back-and-forth mowing pattern on the polygon, and potentially how to fly when scanning the area. For example, if a user performs a vertical mowing but constructs a long horizontal polygon to scan / analyze the mowing, performing the scan according to the original long horizontal polygon may lead to significant overshooting and back-and-forth of the asset, so the user or drone may adapt to a horizontal scan pattern in this case.

[0090] In some embodiments, device 200 may transmit a pre-plan for the assets to follow when performing the task, and the leader asset may determine a plan for the assets to perform the task based on the pre-plan. As an example, a device may determine and transmit a pre-plan when a task is defined to ensure that the assets performing the task follow the planning framework. The pre-plan may be a high-level definition or description of what will be accomplished by the assets or a portion of them, and the leader drone may determine a lower-level plan that further specifies how the assets will perform the task (or its tasks). The leader drone may then provide the plan for performing the task to another asset (e.g., a follower drone) in the asset group, which may then determine a more specific plan for executing the plan provided by the leader drone. As an example, in the context of a surveillance task, device 200 (e.g., a server) may determine a pre-plan that indicates a route or boundary for the assets to travel to obtain surveillance information about a specific location, and device 200 may provide the pre-plan to a leader asset (e.g., a leader drone) in the asset group. In response to receiving the pre-plan, the first asset may break down the boundaries or paths within the pre-plan into smaller segments, assign such segments to various follower assets (e.g., follower drones), and correspondingly provide instructions to the various follower assets to travel to the segments assigned to the follower assets to capture surveillance information about the segments. Continuing with this example, the various follower assets may receive instructions to capture surveillance information about particular segments, and the various follower assets may determine the specific manner in which to travel to the segments and capture surveillance information (e.g., the follower drones may plan their routes to avoid obstacles (such as trees) along the route).

[0091] According to various embodiments, device 200 may obtain feedback information regarding a task definition, a pre-plan for the task, and / or a status or result of task execution. Device 200 may obtain feedback information using feedback information module 270, such as via communication module 210. In some embodiments, feedback information module 270 may obtain information regarding task definition (e.g., one or more characteristics or parameters of the task entered into a user interface displayed on the client terminal). In response to receiving information regarding the task definition, feedback information module 270 may provide such information to task definition module 240, etc. In some embodiments, feedback information module 270 may obtain information regarding a pre-plan for the task (e.g., input to change or update the pre-plan), such as via input into a user interface provided by the client terminal. In response to receiving information regarding the pre-plan, feedback information module 270 may provide information regarding the pre-plan to pre-plan module 260, which may then update the pre-plan based on such information. In some embodiments, feedback information module 270 may obtain information regarding a status or result of task execution. For example, the feedback information module 270 may receive real-time operation information (e.g., indications of asset failure, live video feeds, etc.) from the assets (e.g., the leader drone) while the operation is being performed. The feedback information module 270 may provide the real-time operation information to the user interface module 280 in connection with displaying the status of the operation. As another example, the feedback information module 270 may receive an indication that the operation is completed.In response to receiving an indication that the work is complete, the feedback information module 270 may provide a suggestion or corresponding instruction to one or more modules, such as (i) to the asset ownership module 220 to provide an indication that the work is complete, or (ii) to the asset ownership module 220 to relinquish or return ownership of one or more assets in the asset group.

[0092] According to various embodiments, device 200 may cause one or more user interfaces to be displayed on one or more client terminals. The user interfaces may be displayed in connection with configuring or defining a job, providing and updating pre-planning, and / or providing job status. Device 200 may use user interface module 280 to configure the user interfaces displayed on the client terminals. User interface module 280 may configure the user interfaces based at least in part on one or more templates. Additionally, user interface module 280 may provide a wizard or workflow for generating and providing multiple user interfaces to a user for configuring a job based on one or more inputs to the multiple user interfaces. In some embodiments, user input to one user interface when configuring a job is used in connection with generating the next user interface used to configure the job. In this manner, user interface module 280 may provide one or more user interfaces (or user interface pages) to a client terminal to guide a user through the job configuration process. In contrast, a job is currently defined within a single composite interface, which requires all job features or parameters to be entered on a single page. The use of multiple user interfaces that are logically connected through user input to previous user interfaces simplifies the organization of tasks, thereby lowering the skill level of human operators and reducing the risk of errors or inaccurate inputs.

[0093] FIG. 3 is a block diagram illustrating a device for performing at least a portion of the operations in accordance with various embodiments of the present application. In the example illustrated in FIG. 3, device 300 may include a communications interface 302 and / or one or more processors 304. Device 300 may correspond to asset 120, asset 125, and / or asset 130 of FIG. 1, asset 405, asset 410, asset 420, asset 430, asset 440 (e.g., a satellite), and / or asset 450 (e.g., a tower) of FIGS. 4A-4C, and / or asset 505, asset 510, asset 515, asset 520, asset 525, asset 530, asset 535, asset 540 (e.g., a satellite), and / or asset 545 (e.g., a tower) of FIGS. 5A-5C. 8A, process 830 of FIG. 8B, process 830 of FIG. 8C, process 900 of FIG. 9A, process 930 of FIG. 9B, process 950 of FIG. 9C, process 951-2 of FIG. 9C, process 950 of FIG. 9D, process 1000 of FIG. 10A, process 1020 of FIG. 10B, process 1200 of FIG. 12A, process 1215 of FIG. 12B, process 1215 of FIG. 12C, process 1300 of FIG. 13, process 1400 of FIG. 14, process 1500 of FIG. 15, process 1800 of FIG. 18, process 1900 of FIG. 19A, and / or process 1940 of FIG. 19B.

[0094] According to various embodiments, one or more processors 30 4 The system includes a communications module 310, a planner services module 320, a mapping of capabilities to assets 330, a dynamic grouping module 340, an ownership module 350, and a segmentation module 360. 60, feedback information module 370, and asset failure module 380. Device 300 may implement one or more modules associated with performing a task, such as determining a plan for assets to perform a task (or elements of a task), dynamically modifying the asset group, communicating with one or more other assets in the asset group, updating a plan for performing a task based on information received from another asset in the asset group, and communicating with one or more other assets in the asset group assigned to perform the task.

[0095] According to various embodiments, device 300 corresponds to a semi-autonomous drone. Device 300 may receive a high-level plan or instructions for performing a task (or element thereof), and device 300 may autonomously determine how to perform the task (or element thereof). In some embodiments, device 300 corresponds to a leader drone in a group of assets assigned to perform a task. In some embodiments, device 300 corresponds to a follower drone in a group of assets assigned to perform a task. Device 300 may include the same set of modules regardless of whether the device is a leader drone or a follower drone. For example, multiple assets in a group of assets may include the same modules as the leader drone, such as to provide redundancy in case of a leader drone failure, or when an asset group is partitioned to provide different aspects of a task (or task of a task) and multiple partitions include a leader drone for the partition. As another example, an asset group may include a dormant leader drone that synchronizes with or backs up the leader drone to provide redundancy in case of a leader drone failure. In some examples, certain types of assets have the same set of modules or leader capabilities as a leader drone, while other types of assets have some of the modules or leader capabilities. For example, a semi-autonomous drone in a fleet of assets may have the same set of modules or capabilities as a leader drone, while an autonomous tower, sensor, or satellite in the fleet may have a different set of assets and may not have sufficient capabilities to be the leader of the fleet.

[0096] The device 300 may use the communication module 310 to communicate with assets, another server, a terminal (e.g., server 105 of system 100 in FIG. 1 ), etc. For example, the communication module 310 may provide the communication interface 302 to be communicated. As another example, the communication interface 302 may provide information received by the device 300 to the communication module 310. In response to receiving information from the communication interface 302, the communication module 310 may provide such information to a corresponding module. For example, if the device 300 receives a suggestion to perform a task, the communication module 310 may provide such suggestion to the planner service module 320. As another example, if feedback information is received from one or more follower drones during the performance of a task, the communication module 310 may provide the feedback information to the feedback information module 370, the planner service module 320, etc.

[0097] According to various embodiments, planning of how a work, task, or element will be performed occurs through various entities within the system. A server configuring a work may determine work parameters and provide such parameters to a leader drone and / or determine a high-level pre-plan for a group of assets to perform the work and provide such pre-plan to the leader drone. The leader drone may then determine a lower-level plan than the parameters or pre-plan for the work received from the server. The plan determined by the leader drone may autonomously fill in any gaps in the information received from the server related to performing the work. The leader drone may provide a plan for performing the task or element of the task to one or more follower drones. In response to receiving a plan from the leader drone, one or more follower drones may each determine how to execute the plan, including autonomously filling in gaps in the plan received from the leader. The foregoing framework enables a system (such as system 100 of FIG. 1) to be a modular system supporting hierarchical planning, and such corresponding planning by the leader drone and / or follower drones may be autonomous. In some embodiments, assets within an asset fleet may advertise the level of planning or tasking they support. The leader drone may determine plans corresponding to assets within the asset fleet based on the level of planning or tasking supported by the follower assets. For example, follower drones with more robust planning services may not require as detailed a plan as assets with rudimentary planning services (or no planning services at all). In some embodiments, the leader drone may store a mapping of planning or tasking capabilities to assets and use such mapping in connection with determining respective plans for the follower assets.According to various embodiments, at least one follower drone is capable of performing a robust planning service that is the same or substantially the same as the planning service performed by the leader drone.

[0098] According to various embodiments, in response to receiving an indication that device 300 is included in a constellation of assets (e.g., from a server (e.g., a server providing an operation control service)), device 300 autonomously determines (e.g., generates) a plan for performing one or more tasks associated with the operation, and device 300 communicates at least a portion of the plan to one or more other assets. Device 300 may be a leader asset (e.g., a leader drone) in the constellation of assets for performing one or more tasks associated with the operation. Device 300 may receive information regarding the operation to be performed by the constellation of assets. The information regarding the operation may include a high-level definition or description of the operation. As an example, the high-level definition or description of the operation may include high-level tasks to be performed (e.g., surveillance of an area or target, delivery of a payload, etc.). Examples of information included in the high-level definition or description of the operation include: (i) an indication of the constellation of assets; i The information about the operation may include one or more of: (i) some parameters of the operation, such as the location, suggested restricted areas where the assets should not enter / operate, suggested areas / zones where the assets should operate; (ii) functions associated with the operation; and / or (iv) a pre-plan for performing the operation (or its tasks). In some embodiments, information about the operation is communicated only to a leader asset (or leader asset and a dormant leader) in the asset group. In response to receiving information about the operation, the leader asset may communicate the asset group's suggestions to other assets in the asset group (e.g., to follower assets).

[0099] In some embodiments, information about the operation is transmitted only to the leader asset (e.g., the leader asset) of the asset group. Information about the operation may also be transmitted to a dormant leader (e.g., which may also be referred to herein as a second leader) of the asset group, which may be a backup leader asset that synchronizes operation information (e.g., current operation status, plans for the execution of various tasks, etc.). In this manner, the dormant leader may have a complete backup of the operation information stored / managed at the leader asset. In some embodiments, the dormant leader is selected based on negotiations between multiple assets in the asset group (e.g., negotiations between semi-autonomous drones in the asset group). An asset may be selected as a dormant leader for an operation based at least in part on a determination of one or more capabilities the asset possesses and / or the availability of ownership (e.g., lease or ownership available for the duration of the operation). For example, an asset may be selected as a dormant leader based at least in part on a determination that the dormant leader has one or more predetermined capabilities (e.g., a planning service module), etc. As another example, an asset may be selected as a dormant leader based at least in part on a determination that the asset has a particular set of capabilities that match the set of capabilities of the leader asset (e.g., the dormant leader has all of the capabilities of the leader asset, the dormant leader has the planning or work management capabilities of the leader asset, etc.) As another example, an asset may be selected as a dormant leader based at least in part on a determination that the asset is the best fit as a dormant leader among the other assets in the group of assets.

[0100] In some embodiments, multiple assets in the asset group (e.g., each of the assets, each of the drones in the asset group, etc.) may receive suggestions for the asset group. The suggestions for the asset group may include information identifying the assets in the asset group. The suggestions may further include suggestions for various functions of the assets in the asset group. In some examples, each asset in the asset group (e.g., each member of a team performing a task) may have knowledge of all other assets in the asset group. A follower asset may receive the suggestions for the asset group from a leader asset for the asset group, or a follower asset may receive the suggestions for the asset group from a server (e.g., a server providing task control services). One asset may be determined as the leader asset (e.g., a leader drone) for the asset group based on predetermined leader rankings of the assets, negotiations among at least some of the assets, etc. In some examples, the predetermined leader rankings are input by a user, such as during configuration of the task. In other examples, the leader rankings are negotiated (or the predetermined leader rankings are updated), such as by at least some of the assets.

[0101] According to various embodiments, information about the work NewsIn response to receiving the signal, the leader asset (e.g., the leader drone) autonomously determines a plan for executing the task (or task element of the operation) associated with the operation. The leader asset may communicate at least a portion of the plan for executing the task or task element to at least one follower asset (e.g., a follower asset determined to execute the task or task element). The leader asset may determine a plan for executing the task or task element on a task-by-task, element-by-element, or follower asset-by-follower asset (e.g., follower drone-by-follower drone) basis. In some embodiments, the leader asset determines an individual plan for the follower drone to utilize in connection with executing the corresponding task or task element. The leader asset may determine an individualized plan for each follower drone in the group of assets, for each follower drone that has a currently assigned task or task element, etc.

[0102] In some embodiments, the device 300 uses the planner service module 320 in connection with determining a plan for executing a task or task element. If the device 300 is a leader drone, the planner service module 320 may determine a plan for assets to execute an operation (e.g., one or more tasks or task elements associated with the operation). For example, the planner service module 320 may determine a plan for one or more follower drones to implement in connection with executing the task or task element. The plan may be determined based on one or more parameters of the operation and the assets. In some embodiments, the plan may be determined based at least in part on capabilities associated with the corresponding task or task element and the capabilities of the follower drones. For example, the planner service may determine one or more follower drones that have capabilities matching the capabilities associated with the corresponding task or task element and determine at least one of such follower drones to be assigned to execute at least a portion of the task or element. In some embodiments, the plan determined by the device 300 is a lower-level plan than the pre-plan (if any) received from the server (e.g., along with instructions to execute the operation). For example, device 300 may determine more specific tasks or elements to perform than are provided in the corresponding pre-plan. As an example, in the context of road search (e.g., a task to search a specific set of roads), a high-level task may search a polygon that includes a set of roads and a set of preferred target locations (where targets were believed to be located at a given time). A leader drone may evaluate the set of roads included in the polygon, spread the probability of preferred targets across the roads, and then assign assets (e.g., follower drones) to the highest priority and closest roads (e.g., to perform a search of those roads). The low-level plan provided to the follower drones includes the roads along which the follower drones will scan and the order in which to scan those roads.

[0103] In some embodiments, the device 300 uses the planner service module 320 in connection with determining a method for executing a received plan. If the device 300 is a follower drone, in response to receiving a plan for performing a task or an element of a task, the device 300 may use the planner service module 320 to plan the implementation of the plan. For example, the planner service module 320 may determine an even lower-level plan for executing the plan received from the leader drone. The planner service module 320 may determine a specific method for controlling the device 300 to perform the assigned task or element of the task. As an example, if the device 300 receives a plan to perform monitoring of a defined area, the device 300 may obtain information about the area (such as the locations of various obstacles or objects) and determine a path for moving the device 300 within the area while avoiding the obstacles or objects. As another example, even if the plan received from the leader drone includes a defined path along which the device 300 will move, the device 300 may perform dynamic monitoring of the area to perform collision avoidance during the implementation of the plan. In some embodiments, a follower drone receives a detailed plan from a leader drone that effectively identifies different steps to be taken by the follower drone. The follower drone may implement the steps defined by the leader drone while dynamically monitoring or updating the plan to prevent asset failure or to keep the follower drone on course to implement the plan.

[0104] According to various embodiments, the device 300 uses the planner service module 320 in connection with determining a discrete representation of a geographic location associated with an operation or where at least one asset in the asset group performs one or more tasks. As one example, the discrete representation includes a plurality of discrete elements, each corresponding to a volume at a geographic location. As another example, the planner service module 320 determines the discrete representation in connection with determining a plan for the device 300 or a plan for an asset in the asset group that performs one or more tasks.

[0105] In some embodiments, after the discrete representation is determined (e.g., in response to determining the discrete representation of the geographic location), the device 300 uses the planner service module 320 to annotate the discrete representation, such as to create an annotated representation. As an example, the planner service module 320 annotates the discrete representation based at least in part on received information about one or more tasks and / or geographic locations. The planner service module 320 annotates the discrete representation based at least in part on current information about the one or more tasks and / or geographic locations. The current information corresponds to information stored by or accessible to the device 300 at the time the annotated representation is created. In some embodiments, the system updates the annotated representation based at least in part on received information about the one or more tasks and / or geographic locations. For example, the system may repeatedly update the annotated representation periodically, such as when updated or new information about one or more tasks and / or geographic locations is received. The annotated representation may be updated until one or more tasks are completed (e.g., until a decision is made to complete, stop, pause, etc.). In some embodiments, annotating the discrete representation of the geographic location includes associating metadata with one or more discrete elements of the discrete representation, respectively. The metadata may include values ​​or suggestions associated with one or more tasks, geographic locations, etc.

[0106] One example of annotating the discrete representation includes setting metadata associated with at least a portion of the plurality of discrete elements to establish keep-in areas (e.g., areas where devices 300 or assets in the group of assets are kept) or keep-out areas (e.g., areas where devices 300 or assets in the group of assets are excluded / prevented from entering). For example, each discrete element in the group of discrete elements has a metadata field corresponding to a keep-in indicator and / or a keep-out indicator. As an example, the keep-in and / or keep-out areas are configured in connection with the configuration of the work (e.g., a user inputs the keep-in and / or keep-out areas, certain predetermined keep-in and keep-out areas are set based on restricted airspace, etc., a third-party service provides input to set the keep-in and / or keep-out areas, etc.).

[0107] Another example of annotating a discrete representation includes setting metadata associated with at least some of a plurality of discrete elements to set the location of one or more targets of a task. For a task to perform surveillance of a specific target, configuring the task includes obtaining the target's current location (predicted location, last known location, etc.). As an example, a user sets the current location of a specific target in connection with configuring the task. The planner service module 320 determines a specific discrete element corresponding to the current location of the specific target and sets a metadata field associated with the specific discrete element to indicate that the target is located at / within the discrete element. As another example, for a task to perform surveillance of a specific target (such as a building or road), the planner service module 320 determines a discrete element that corresponds to (or includes) the specific target (such as a building or road) and sets a metadata field corresponding to the discrete element to indicate that the specific target is included in the discrete element.

[0108] Another example of annotating the discrete representation includes setting metadata associated with at least some of the multiple discrete elements to define a line of sight of communications between assets, between assets and a leader drone, between a leader drone and a ground control station, etc. As an example, setting the metadata to define a line of sight of communications includes indicating for a particular discrete element whether a line of sight of communications is obtainable when an asset is located at a geographic location corresponding to the particular discrete element.

[0109] Another example of annotating a discrete representation includes setting metadata associated with at least some of the discrete elements to indicate whether the discrete elements are occupied or unoccupied. As one example, a discrete element is considered occupied if an asset (e.g., an asset from the asset pool, another friendly asset not in the asset pool, etc.) is included in the discrete element. As another example, a discrete element is considered occupied if an asset cannot move into the discrete element, such as because there is insufficient space within the discrete element to support the asset without risk of collision (e.g., a risk of collision above a threshold risk) with another asset or object (e.g., a building, tree, mountain, etc.). In some embodiments, the planner services module 320 sets metadata associated with the discrete elements to indicate whether the discrete elements are occupied or unoccupied in connection with registering or storing a flight plan. For example, device 300 determines a flight plan for an asset in the asset fleet or receives information about the flight plan from an asset in the asset fleet, and planner services module 320 determines discrete elements affected by the flight plan (e.g., a set of discrete elements that intersect with the flight plan) and sets metadata for such discrete elements to indicate that they are occupied. Setting discrete elements corresponding to a flight plan as occupied may ensure that another flight plan that intersects with the flight plan is not created or registered. For example, device 300 and / or other assets to which an annotated representation or model (or information from which the annotated representation may be created) is issued utilize the annotated representation in connection with determining a new flight plan, and the annotated representation includes the discrete elements that were set as occupied for the existing flight plan (e.g., at least during the forecast period corresponding to the flight plan). In some embodiments, planner services module 320 further sets a threshold number of neighboring discrete elements adjacent to the set of discrete elements corresponding to the flight plan. Adjacent discrete elements may act as buffers to provide additional spatial distance between the existing flight plan and any subsequently created flight plans, and so on.The threshold of adjacent discrete elements may be configurable. As an example, for any particular discrete element corresponding to a flight plan, the planner services module 320 sets N discrete elements in all directions of the particular discrete element as occupied, where N is a positive integer.

[0110] In some embodiments, the device 300 uses the planner service module 320 to update the annotated representation based at least in part on received information regarding one or more tasks and / or geographic locations. For example, the planner service module 320 repeatedly updates the annotated representation, e.g., periodically (e.g., at default intervals, configurable intervals, etc.), such as after updated or new information regarding one or more tasks and / or geographic locations is received. As an example, the planner service module 320 updates the annotated representation until one or more tasks are completed (e.g., until a decision is made to complete, stop, pause, etc. the work). In some embodiments, annotating the discrete representation of the geographic location includes associating metadata with one or more discrete elements of the discrete representation, respectively. The metadata may include values ​​or suggestions associated with one or more tasks, geographic locations, etc. Repeated or continuous updates of the annotated representation refine the model of the geographic location or work. For example, as additional information is obtained regarding geographic location, such as via assets (e.g., follower drones) deployed at specific locations, the resolution of certain amounts or types of information included in the model is improved.

[0111] According to various embodiments, the planner service module 320 uses the annotated representation in connection with determining a plan for performing one or more tasks. For example, the planner service module 320 utilizes metadata associated with discrete elements in the discrete representation to determine destinations for one or more assets in the fleet of assets, determine routes for one or more assets, determine elements to be performed by particular assets, etc. In some embodiments, the resolution of the geographic location / task model improves, such as by receiving feedback information (e.g., updated information about a task or element of a task, or information about a portion of an asset's geographic location) from an asset (e.g., a follower drone). As the resolution of the geographic location / task model improves, the plan for performing one or more tasks is updated (e.g., refined).

[0112] According to various embodiments, device 300 uses the asset-to-capability mapping in connection with assigning tasks or elements of a work operation and / or to determine a plan for particular follower assets to perform the tasks or elements. If device 300 is a leader drone, device 300 may store asset-to-capability mapping 330 locally on device 300 and query the asset-to-capability mapping to assign tasks or elements or to determine a plan. At the start of a work operation (e.g., in response to receiving an instruction to perform a work operation), the leader drone may receive a current set of capabilities for various assets in the asset group. For example, the leader drone may cause the assets to provide their current capabilities to the leader drone (e.g., send a request to the follower drone to register their corresponding capabilities with the leader drone). As another example, the leader drone may receive a set of capabilities for various assets in the asset group from a server. The leader drone may update asset-to-capability mapping 330 during the performance of the work operation. For example, the leader drone may ping / request each follower drone to provide updates about its current set of capabilities, and the leader drone may update the mapping accordingly. As another example, the leader drone may determine updates to a particular follower drone's capabilities based on feedback information communicated by the follower drone to the leader drone during task execution (e.g., the follower drone's implementation of a plan). Device 300 may determine from the feedback information that a particular sensor or camera from the follower drone is not functioning properly or has failed (e.g., the feedback information may not have a clear signal associated with the normal operation of such sensor / camera).

[0113] According to various embodiments, device 300 may utilize a mapping of capabilities to assets in connection with the execution of a plan and / or to maintain the current state of the asset's capabilities. If device 300 is a follower drone, device 300 may provide the corresponding leader drone with the follower drone's current set of capabilities or the status of various capabilities in the set of capabilities associated with the follower drone (e.g., an indication that a particular capability is fully operational, partially operational, or inoperable, etc.). Device 300 may provide such information to the leader drone at predetermined intervals, upon request from the leader drone, or along with feedback information communicated to the leader drone during the execution of a plan.

[0114] According to various embodiments, the assets assigned to perform a task may be dynamically updated during the execution of the task. For example, assets within the asset group may be modified / updated in response to (i) a user request to reallocate an asset within the asset group to another task, (ii) loss of ownership of an asset within the asset group, (iii) loss of ownership of an asset within the asset group (e.g., if functionality required to complete a task is lost during the execution of the task, a new / additional asset with such functionality may be assigned to the asset group), (iv) user input that changes the rate at which the task is expected to be completed (e.g., additional assets are added to speed completion or assets are removed from the asset group if the user decides to slow completion), (v) a cost function associated with the execution of multiple tasks indicates that more optimal completion of the multiple tasks includes utilizing additional assets or reallocating assets from the asset group, (vi) a determination that an asset within the asset group has failed, or (vii) a condition defined within the task is met (e.g., under a surveillance task, the condition may include the detection of a target that another or different asset is assigned to intercept, etc.). The assets may be dynamically updated in response to various other circumstances. The assets may be dynamically updated by the server, the leader drone, or both.

[0115] In some embodiments, the dynamic grouping module 340 may be used in connection with dynamically updating a group of assets. The device 300 may use the dynamic grouping module 340 to determine whether another asset should be added to the group of assets and / or whether an asset should be removed from the group of assets. In response to the dynamic grouping module 340 determining to update a group of assets, the dynamic grouping module may provide such suggestions to various other modules (e.g., to the planner services module 320 to update or create a new plan for performing a task or element using the updated group of assets, or to the asset-to-capability mapping 330 to update the capability mapping for the updated group of assets, or to the communications module 310 to notify or request an update from a server, or to the ownership module 350 to request / update ownership of the asset being added to or removed from the group of assets).

[0116] According to various embodiments, a leader drone may manage ownership of assets within an asset fleet. In response to receiving an indication that the leader drone is among the assets performing work, the leader drone may obtain ownership authorization and / or ownership status corresponding to the assets within the asset fleet (e.g., transfer of ownership of the corresponding assets to the organization for which the work is being performed, if applicable). The ownership transfer may correspond to the transfer of asset ownership (i) for a portion of the work, such as during the performance of a specific task or element; (ii) for the duration of the work; or (iii) indefinitely (e.g., permanent transfer of ownership until the transferring organization relinquishes ownership). The ownership authorization may include automatic handover (e.g., allowing ownership to be transferred to the transferring organization / leader drone upon request), negotiation of asset handover approval / denial (e.g., conditional transfer pending an emerging need for the transferring organization, a specified time period / lease period for which ownership is transferred, a specified task / element for which ownership is transferred, etc.). The ownership state may indicate whether an asset is available for transfer to the leader drone (or the organization for which the leader drone is performing the operation), whether the asset is unavailable for transfer, or a set of conditions under which the asset may or may not be transferred. In some embodiments, the leader drone may obtain the ownership permissions and / or ownership state based on querying an ownership service (e.g., a third-party service or a centralized service where various organizations can register / transfer ownership of assets), querying a mapping of assets to ownership rights stored locally or remotely on a server managing the operation (e.g., the server that configured the operation), and / or querying various assets within the asset group (e.g., each asset may store its corresponding ownership permissions and / or ownership state and / or a protocol under which the leader drone can request an ownership transfer). The leader drone may use the ownership module 350 to manage / update ownership of assets within the asset group and / or negotiate or request transfer of assets to / from the asset group.For example, in response to an indication that an asset is being added to an asset group or that ownership of an asset within an asset is being requested or updated, the ownership module 350 may be called upon to request / arrange a transfer of ownership of the asset.

[0117] In the course of performing a task, the leader drone may determine that the task (or tasks / elements of the task) could be performed more effectively or efficiently by one or more portions of the assets, and the leader drone may divide the asset group. The leader drone may decide how to divide the asset group. For example, various partitions may be determined based on the functions of the tasks / elements performed by various partitions and / or functions of the assets within the asset group. In some embodiments, partitions are assigned corresponding leader assets. For example, each partition has a corresponding leader drone. Accordingly, a follower drone may be promoted to the position of leader drone for a particular partition. The leader drone of the asset group may be the primary leader, and various partition leaders may report feedback information (e.g., the status of assigned tasks / elements of the partition). The primary leader may then aggregate feedback information received from the partition leader and / or any assets reporting directly to the primary leader (e.g., assets in the same partition as the primary leader) and provide the feedback information to the server and / or provide instructions to the partition leader, such as modifications to tasks or elements assigned to the partition. In response to splitting a group of assets, the primary leader may provide instructions to the partitions (e.g., partition leaders) to execute tasks or elements that the primary leader assigned to the partitions. In response to a partition receiving an indication to execute a task or element, the partition leader may determine a plan for the corresponding split group of assets. As an example, the partition leader may play a role for the partition such that the role is similar to that played by the primary leader for the group of assets (e.g., before the split). Similarly, a partition may have one or more follower assets that (i) receive plans from the partition leader, (ii) implement the plans received from the partition leader, and / or (iii) communicate feedback information to the partition leader.According to various embodiments, in response to determining that a partition has completed the tasks / elements assigned to it, the divided assets may be (i) reintegrated into the asset pool (e.g., with the partition corresponding to the primary leader), (ii) reassigned to other partitions to assist in the completion of the tasks / elements assigned to such other partitions, or (iii) released from the asset pool (e.g., to perform other work for another leader or organization). In some embodiments, when a partition is created, a set of constraints within which the partition will operate may be determined. The primary leader drone may provide the set of constraints to the corresponding partition leader.

[0118] According to various embodiments, device 300 uses partitioning module 360 ​​in connection with determining whether to partition assets and how the assets are to be partitioned. Partitioning module 360 ​​may determine the assets to be included in various partitions. In response to determining that a partition has completed the tasks / work assigned to it, if an asset in a partition is to be assigned to another partition or reintegrated into the assets, partitioning module 360 ​​may determine the partition in which such asset will be included. In some embodiments, after completing the tasks / work assigned to the partition, the partition is redeployed to perform another task / work (e.g., the primary leader or a planner service module on the primary leader determines to assign another task / work).

[0119] The device 300 may send / receive feedback information regarding the execution of a task. If the device 300 is a leader drone, the device 300 may receive feedback information from one or more follower drones, such as information acquired by the sensors / cameras of the follower drones, the status of tasks / elements assigned to the follower drones, the status of tasks / elements assigned to the follower drones, the status of the drones, updates to the status or functionality of the drones' functions, etc. Additionally or alternatively, the device 300 may send feedback information to a server (such as a server managing (or configuring) the task). For example, the leader drone may send the status of the task, a live feed of information acquired by one or more sensors in the assets, a request for future instructions regarding the task, etc. If the device 300 is a follower drone, the device 300 may send feedback information to the leader drone, such as information acquired by the sensors / cameras of the follower drones, the status of tasks / elements assigned to the follower drones, the status of the drones, updates to the status or functionality of the drones' functions, etc. The device 300 may communicate the feedback information using the feedback information module 370. The feedback information module 370 may receive the feedback information, determine various information contained in the feedback information, and forward the information to a corresponding module. As an example, if the feedback information includes an indication that a sensor on a follower device has failed, the feedback information module 370 may determine that such information relates to asset functionality and provide such information to the functionality to asset mapping 330 so that the functionality to asset mapping in the asset group is updated.As another example, if a follower drone communicates sensor information (e.g., images, live feeds, weather conditions, etc.) to the leader drone, the feedback information module 370 may determine such sensor information in the feedback information and provide such information to the planner services module so that the leader drone can utilize such information in connection with determining the status of the operation and / or deciding whether to update the plan for the follower drone (or other follower drones).

[0120] According to various embodiments, the team of assets is dynamically managed throughout the execution of the task, such as from the time the task is configured to the time the task is completed. Assets may be added to or removed from the pool of assets assigned to perform the task during task execution. The decision to add / remove an asset may be performed by a server. In some examples, the decision to add / remove an asset may be performed by a leader drone for the asset pool. Dynamic team management provides an extensible framework that can adapt to changes in the pool of assets, unexpected or conditional events that occur during task execution, or changes in the configuration of the task (e.g., user-initiated changes to the task). Examples of changes in the pool of assets may include asset failure, failure / loss of asset functionality, or asset partitioning (e.g., an asset pool may have only one specific functionality required by multiple partitions, resulting in a requirement to add a specific functionality to an asset). Examples of unexpected or conditional events occurring during the execution of a task may include a new target appearing while monitoring an area, weather or environmental changes that require specific capabilities or that may enhance such capabilities (e.g., a larger drone to withstand extreme weather such as severe winds or rain), a task that drags on into the night, thereby enhancing the effectiveness of a drone with night vision capabilities, etc. Examples of changes in the configuration of a task may include a user request to complete the task faster (e.g., speeding up execution by the asset fleet), a user request or other decision to prioritize another task, a user request to pause the task, etc. In some embodiments, in response to determining that a new asset is to be added to the asset fleet, the server sends an indication to the leader that the new asset has been included in the asset fleet. The leader drone may then responsively determine a plan for executing the task and / or manage ownership of the new asset (e.g., taking appropriate ownership of the new asset). In some embodiments, in response to a follower drone being removed from the asset fleet, an indication is sent to the follower drone indicating that the follower drone is no longer part of the asset fleet.The server and / or leader drone may send such instructions to the follower drones.

[0121] According to various embodiments, in response to determining that a follower drone has failed while performing a task, the leader drone may update the plan for performing the task. For example, the leader drone may determine the status of the task / element assigned to the failed follower drone and reallocate the remaining portion of the task / element (or the entire task / element) to another asset in the asset group. In response to the follower drone's failure, the leader drone may update the capabilities of the asset group, and the leader drone may use the updated capabilities to reallocate / reallocate elements of the plan to other follower drones. In some embodiments, in response to determining that a follower drone has failed, the leader drone may add additional assets to the asset group or send a request (e.g., to a server) to add additional assets. For example, the leader drone may identify one or more capabilities of the failed follower drone and attempt to compensate for those capabilities (or capabilities for which the asset group has no redundancy or insufficient redundancy). The leader drone may determine that a follower drone has failed in response to determining that the leader drone has not received feedback information from the follower drone within a predetermined period of time and / or by an estimated time (e.g., the estimated time may be based on a schedule currently assigned to the follower drone, such as when the follower drone is predicted not to communicate for a period of time). In some examples, if the leader drone does not receive feedback information from the follower drone within a predetermined period of time or by the estimated time, the leader drone may send a ping to the follower drone for the follower drone's health status, and in response to the follower drone failing to respond to the ping (e.g., within a threshold period of time), the leader drone may determine that the follower drone has failed. In some examples, the server may determine that a follower drone has failed in response to at least determining that the server has not received any information or pings from the follower within a predetermined period of time and / or by the estimated time, and the server may notify the leader drone that the follower drone has failed.

[0122] According to various embodiments, in response to determining that a leader drone has failed during the execution of a task, a remaining asset in the asset group is promoted to a leader position, and such asset becomes the new leader drone for the execution of the task. A leader drone may be determined to have failed if the leader drone has failed or has lost communication with some or all of the assets. The remaining assets in the asset group may negotiate to determine a new leader among the assets. The negotiation to determine a new leader may be based at least in part on the leader rankings of the assets in the asset group (or a prioritized list of the assets). The leader ranking may be a predetermined ranking determined during the configuration of the task and provided to all assets in the asset group, or a predetermined ranking provided to at least one asset other than the initial leader asset, and the asset with the highest ranking among the remaining assets is promoted to the new leader drone. In some examples, the remaining assets negotiate a new leader ranking for the assets, and the asset with the highest ranking among the remaining assets is promoted to the new leader drone. In some embodiments, when a dormant drone is included in a group of assets (the dormant drone is among the remaining assets), the dormant leader is automatically promoted to the new leader drone.

[0123] According to various embodiments, partitions may be formed based on network or communication failures. For example, if a portion of the assets loses communication with a leader drone, the portion of the assets may determine (individually or collectively) that the leader drone has failed. In response to determining that the leader drone has failed, the portion of the assets may form a group of assets (e.g., a partition), and the portion of the assets may negotiate for a partition leader. Once a partition leader is determined, the partition leader may determine a plan for the partition to continue performing the work (or collective tasks / elements assigned to the portion of the assets). The partition leader may attempt to establish communication with a server and / or another partition associated with the same work. In response to establishing communication with the server, the partition leader may provide feedback information regarding the performance of the work (or portion of the work performed by the partition). In response to establishing communication with the partition leader of another partition, the partition leader may request or otherwise attempt to group the two partitions, or the partition leader may provide feedback information regarding the work to the other partition leader. In some embodiments, different partition leaders may negotiate the division of tasks / elements associated with a work, such that different partitions perform different parts of the work. Partitions may have predetermined or negotiated constraints under which they operate. Examples of constraints include geographic constraints, time restrictions, restrictions on certain actions (e.g., restrictions on engaging with targets), etc.

[0124] In some embodiments, if a partition leader fails, the remaining assets in the partition fail over to a partition or group of assets (e.g., the partition with the leader drone corresponding to the highest-ranking leader in a given leader ranking). The remaining assets in the partition may fail over to a given partition or group of assets if a new partition leader cannot be determined.

[0125] According to various embodiments, each operation has a default plan that is implemented by the assets in the asset group if a new leader cannot be determined. For example, the default plan may be a plan in which each asset returns to its home in response to a determination that a new leader cannot or will not be determined (e.g., within a predetermined period of time).

[0126] The device 300 may use the asset failure module 380 in connection with determining that an asset has failed and to determine a failover protocol to implement if an asset is determined to have failed. For example, if the failed asset was a leader drone, the asset failure module 380 may determine to negotiate with the remaining assets to determine a new leader drone. As another example, if the failed asset was a follower drone, the asset failure module 380 may determine to reallocate the portion of the plan assigned to the failed follower drone among the remaining assets.

[0127] According to various embodiments, in addition to determining a plan for executing a task or element, the leader drone determines a method for communicating the plan to the follower drones. The leader drone may determine a method for communicating the plan based on the communication link between the leader drone and the follower drone, the quality of the communication link, etc. In some embodiments, the leader drone may determine a method for communicating the plan based on an estimated quality of communication between the leader drone and the follower drone at a future point in time (e.g., a particular point in time in the execution of the task / element). For example, the leader drone may determine that the follower drone will move over a mountain in connection with executing the task, that the follower drone may lose communication with the leader drone, or that the communication link may have a relatively low quality of service. In response to such a determination, the leader drone may provide a wider range of plans for the follower drone to execute the task / task. As another example, if the leader drone determines that the current communication link is relatively weak (e.g., low speed / low bandwidth), in response to such a determination, the leader drone may decide to send only a relatively small portion of the plan for the task to the follower drone and to send a larger portion of the plan at a future point in time when the communication link is or is predicted to be good. According to various embodiments, the leader drone may determine the granularity at which the plan is divided and communicated to the follower drones based on network conditions and / or network constraints. The network conditions and / or network constraints used in such determination may be current conditions / constraints or predicted conditions / constraints at a particular future point in time. Examples of network constraints may be one or more values ​​related to one or more of network topology (e.g., areas accessible to the network by the leader drone or follower drones), communication bandwidth, link quality, etc. Determining the granularity at which the plan is divided and / or communicated to the follower drones and / or how the plan is communicated to the follower drones may be part of the planning process performed by the leader drone.For example, the leader drone may plan the operation in multiple stages and sequentially transmit portions of the current plan or portions of the plan for the multiple stages of the operation. Similarly, in some embodiments, the server managing / configuring the operation may determine how / to what extent information about the operation (e.g., operation parameters) is communicated to the leader drone based at least in part on network conditions and / or network constraints and / or the planning service capabilities of the leader drone.

[0128] FIG. 4A illustrates a system for performing at least a portion of a task in accordance with various embodiments of the present application. In the example illustrated in FIG. 4A , system 400 may include a group of assets (e.g., assets 405, 410, 420, 430, 440, and / or 450) and a control center 460 (e.g., a server providing task control services). System 400 may be implemented by system 100 of FIG. 1. Control center 460 may be implemented by device 200 of FIG. 2. One or more assets in the group of assets may be implemented by device 300 of FIG. 3. For example, asset 405, asset 410, asset 420, and / or asset 430 may be implemented by device 300 of FIG. 3. In some embodiments, asset 405, asset 410, asset 420, and / or asset 430 are semi-autonomous drones. Asset 440 may be a satellite that communicates with control center 460 and / or another asset (such as asset 405). Asset 450 may be a tower (such as a semi-autonomous tower). Asset 450 may communicate with control center 460 and / or another asset (such as asset 405).

[0129] In the example shown in FIG. 4A , a group of assets may be instructed to perform a task. For example, the task may be tracking vehicle 470 and / or monitoring a road (and tracking a particular vehicle or any vehicle traveling along the road). Control center 460 may configure the task and provide task parameters to the group of assets (e.g., asset 405). Asset 405 may be determined to be the leader asset, and the other assets (e.g., asset 410, asset 420, and / or asset 430) may be determined to be follower assets. In response to the leader asset receiving the task parameters and advance plan (if any) from control center 460, the leader asset may determine a plan for the other follower assets to perform tasks or elements of the task. Continuing with the above example, the leader asset may determine a plan for follower asset 2 (asset 410) to track vehicle 470, a plan for follower asset 4 (asset 430) to perform monitoring of a first portion of the road, and a plan for follower asset 3 (asset 420) to perform monitoring of a second portion of the road. In response to determining the plans for the follower assets, the leader asset may correspondingly transmit the plans to each follower asset. The follower assets may implement the plans in response to receiving the plan from the leader drone, and the follower assets may fill in any gaps in the plan provided by the leader drone. The leader asset may determine that asset 440 (e.g., a satellite) will provide monitoring of the road and assistance in tracking vehicles traveling along the road. The leader asset may determine a plan for asset 450 to capture information about the area / environment using one or more sensors. The captured information may be provided to the leader asset as feedback information, which may be used by the leader asset to update plans for various assets in the asset fleet.

[0130] 4B illustrates a system for performing at least a portion of the operations in accordance with various embodiments of the present application. In the example system 400 illustrated in FIG. 4B, a leader asset (asset 405) is determined to have failed. In response to determining that the leader asset has failed, a determination of a new leader for the assets is made among the remaining assets. For example, asset 410, asset 420, and asset 430 may negotiate a new leader asset.

[0131] According to various embodiments, determining a new leader asset includes determining the new leader asset based on a leader ranking. For example, leader ranking 480 may be obtained by one or more of the remaining assets. In some examples, each asset stores a predetermined leader ranking. In some examples, follower assets obtain their leader rankings by negotiating leader rankings based on the respective capabilities of the remaining assets, etc. In the example shown in FIG. 4B , leader ranking 480 is predefined, such as configured by control center 460 simultaneously with configuration of the work. Asset 1 in leader ranking 480 is the highest-ranking asset, and asset 4 is the next-highest-ranking asset. Because asset 1 has been determined to have failed, asset 4 is determined to be the highest-ranking asset among the remaining assets. Thus, asset 4 is determined to be the next leader asset for the group of assets. In some embodiments, in response to asset 4 being determined as the new leader asset, asset 4 determines the status of the work and determines a plan for executing at least the remaining portion of the work. Asset 4 may send the corresponding plans to the remaining follower assets, to which the follower assets have been assigned.

[0132] FIG. 4C illustrates a system for performing at least a portion of a task, according to various embodiments of the present application. In the example system 400 illustrated in FIG. 4C, a leader asset (asset 420) is determined to have failed. In response to determining that the leader asset has failed, the leader asset may determine the status of the task and / or determine an updated plan for the remaining assets (e.g., asset 410 and asset 430). According to various embodiments, in response to determining that a follower asset has failed, the leader asset may reallocate tasks / elements assigned to the failed follower asset (or the remaining tasks / elements) among the remaining assets. The leader asset may update the plan for the remaining assets based at least in part on the determined reallocation of tasks / elements. In some embodiments, the leader drone and / or asset 450 may determine to add one or more assets to the group of assets (e.g., to compensate for functionality lost due to the failure of asset 430, to expedite the execution of the task, etc.). For example, a list of additional assets 490 may be determined. The leader drone and / or asset 450 (e.g., a tower with a computer system) may dynamically update the group of assets, such as by adding one or more of the additional assets in list of additional assets 490. In some embodiments, list of additional assets 490 may be determined at least in part based on the capabilities of the corresponding assets, such that the assets on list of additional assets 490 have at least one capability that matches the capabilities of the remaining tasks to be completed. In some embodiments, list of additional assets 490 may be determined at least in part based on the ownership availability of the assets on the list of additional assets (e.g., the assets may be currently available for work or may be reassigned to perform work after submitting a request to take over ownership of or lease the assets).

[0133] In some embodiments, the control center (e.g., control center 460 of FIG. 4A, 4B, or 4C) is one of multiple control centers that control a system including semi-autonomous drones. In various embodiments, the control center includes a ground control center, an airborne control center, a surface control center, or any other suitable control center. In various embodiments, a single control center is in control, each control center controls a portion of the system assets, control rotates among multiple control centers for coordinated takeover during a change of control, or any other suitable assignment of control to control centers.

[0134] FIG. 5A illustrates a system for performing at least a portion of an operation in accordance with various embodiments of the present application. In the example illustrated in FIG. 5A, system 500 may include a group of assets (e.g., assets 505, 510, 515, 520, 525, 530, 535, 540, and / or 545) and control center 550 (e.g., a server providing operation control services). System 500 may be implemented by system 100 of FIG. 1. Control center 550 may be implemented by device 200 of FIG. 2. One or more assets in the group of assets may be implemented by device 300 of FIG. 3. For example, asset 505, asset 510, asset 515, asset 520, asset 525, asset 530, and / or asset 535 may be implemented by device 300 of FIG. 3. In some embodiments, asset 505, asset 510, asset 515, asset 520, asset 525, asset 530, and / or asset 535 are semi-autonomous drones. Asset 540 may be a satellite that communicates with control center 550 and / or another asset (such as asset 505). Asset 545 may be a tower (such as a semi-autonomous tower). Asset 545 may communicate with control center 550 and / or another asset (such as asset 505).

[0135] In the example shown in FIG. 5A , a group of assets may be instructed to perform a task. For example, the task may be tracking vehicle 555 and / or monitoring a road (and tracking a particular vehicle or any vehicle traveling along the road). Control center 550 may configure the task and provide task parameters to a group of assets (e.g., asset 505). Asset 505 may be determined to be a leader asset, and the other assets (e.g., asset 505, asset 510, asset 515, asset 520, asset 525, asset 530, and / or asset 535) may be determined to be follower assets. In response to the leader asset receiving the task parameters and advance plan (if any) from control center 550, the leader asset may determine a plan for the other follower assets to perform tasks or elements of the task. Continuing with the above example, the leader asset may determine a plan for follower asset 525 to track vehicle 555, a plan for follower asset 510 to perform monitoring of a first portion of the road, and a plan for follower asset 515 to perform monitoring of a second portion of the road. In response to determining the plans for the follower assets, the leader asset may correspondingly transmit the plans to each follower asset. The follower assets may implement the plans in response to receiving the plan from the leader drone, and the follower assets may fill in any gaps in the plan provided by the leader drone. The leader asset may determine that asset 540 (e.g., a satellite) will provide monitoring of the road and assistance in tracking vehicles traveling along the road. The leader asset may determine a plan for asset 545 to capture information about the area / environment using one or more sensors. The captured information may be provided to the leader asset as feedback information, which may be used by the leader asset to update plans for various assets in the group of assets.

[0136] 5B illustrates a system for performing at least a portion of an operation in accordance with various embodiments of the present application. In the example system 500 illustrated in FIG. 5B, an additional vehicle 560 appears and moves along a road monitored by the assets. One or more assets in the assets may capture images of the vehicle 560. For example, a satellite may identify the vehicle 560 moving along the road. As another example, the assets 510, 520, and / or 515 may provide feedback information to a leader asset (asset 505), which may process the feedback information to determine that the vehicle 560 is within a geographic area established by the operation parameters and / or meets the conditions for tracking the vehicle moving along the road.

[0137] In response to detecting vehicle 560, the leader asset may decide to split the asset group. For example, the leader asset may determine that an operation (e.g., including tracking vehicle 555 and vehicle 560) is likely to be performed more effectively if the asset group is split into two partitions. In response to deciding to split the asset group, the leader asset may instruct corresponding follower assets in the asset group, such as to provide the follower assets with an indication of which partition the follower asset belongs to. Assets 505, 510, 515, 520, 530, 540, 550, 560, 570, 580, 590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710, 720, 730, 740, 750, 760, 770, 780, 790, 800, 810, 820, 830, 840, 850, 860, 870, 880, 890, 900, 910, 920, 930, 940, 950, 960, 970, 980, 990, 1000, 1010, 1020, 1030, 1040, 1050, 1060, 1070, 1080, 1090, 1109, 1111, 1120, 1130, 1140, 1150, 1160, 1170, 1180, 1190, 1200, 1210, 1220, 25, 530, and 535 may be divided into two partitions (first partition 570 and second partition 580). In some embodiments, the leader asset is in first partition 570, so the leader asset is determined as the partition leader for first partition 570. In some embodiments, the second partition 580 includes assets that were originally follower assets in the group of assets, so at least one of the assets in second partition 580 is promoted to partition leader for second partition 580. The assets in second partition 580 may negotiate to determine leadership of second partition 580. For example, the partition leader may be determined by a predetermined leadership ranking, such as a leadership ranking determined concurrently with the configuration of the work. The partition leader may be determined as the asset in second partition 580 that has the highest corresponding ranking on the leadership ranking. In the illustrated example, asset 525 may be promoted to partition leader.

[0138] In the illustrated example, the task of tracking vehicle 560 may be assigned to a first partition, the task of tracking vehicle 555 may be assigned to a second partition 580, the task of providing road surveillance may be assigned to asset 540, and the task of monitoring the environment may be provided to asset 545 (e.g., a tower). In some embodiments, a leader asset in first partition 570 may determine respective plans for asset 510, asset 515, and asset 520 to perform corresponding elements of the task of tracking vehicle 560. In response to receiving the plans for performing corresponding elements, asset 510, asset 515, and asset 520 may implement the plan and fill in any gaps in the plan (e.g., provide collision avoidance, such as avoiding hills or trees, determine the flight altitude, determine when to capture images, etc.). Asset 510, asset 515, and asset 520 may provide feedback information to the leader asset, where the feedback information relates to the status of execution of elements of the task of tracking vehicle 560 and / or information captured by one or more sensors (e.g., live video feeds, images such as license plate images, etc.). The leader asset may provide feedback information to control center 550, such as information about vehicle 560.

[0139] In some embodiments, a partition leader in second partition 580 may determine respective plans for assets 530 and 535 to execute corresponding elements of the task of tracking vehicle 555. In response to receiving the plans for executing corresponding elements, assets 530 and 535 may implement the plan and fill in any gaps in the plan (e.g., provide collision avoidance, such as avoiding hills or trees, determine flight altitude, determine when to capture images, etc.). Assets 530 and 535 may provide feedback information to the partition leader, the feedback information related to the status of execution of elements of the task of tracking vehicle 555 and / or information captured by one or more sensors (e.g., live video feeds, images such as license plate images, etc.). In some embodiments, the partition leader may provide the feedback information to the leader asset (asset 505), which may then provide the feedback information (or information including such feedback information aggregated or processed / analyzed with feedback information from first partition 570). The leader asset may then provide feedback information to the control center 550, such as information regarding the vehicle 560. In some embodiments, the partition leader may provide feedback information directly to the control center 560 (e.g., provide information to the control center without providing such information to the leader asset).

[0140] 5C illustrates a system for performing at least a portion of an operation, according to various embodiments of the present application. In the example illustrated in FIG. 5C, the leader asset may determine that tracking of vehicle 555 is no longer performed based at least in part on feedback information (e.g., feedback information from asset 510, asset 515, and / or asset 520). In some embodiments, the leader asset may determine that the state of vehicle 555 no longer meets operational parameters for tracking a vehicle traveling on a monitored roadway. For example, based on the feedback information, the leader asset may determine that vehicle 555 has exited (is no longer within) a geographic area being monitored or tracked based at least in part on operational parameters and / or constraints. As another example, the leader asset may determine that vehicle 555 has entered an area that is a restricted area according to operational parameters, and that the assets will not enter such area.

[0141] In some embodiments, system 500 may decide to dynamically add assets to the asset fleet. For example, leader asset and / or control center 550 may decide to add asset 585 and / or asset 590 to the asset fleet. An asset may be added to the asset fleet in response to a determination that better coverage of a geographic area / road is desirable for road monitoring (e.g., based on user input to a user interface operatively provided by control center 550, etc.). In response to a decision to add asset 585 and / or asset 590 to the asset fleet, the leader asset may update the mapping of capabilities to the asset and update a plan for performing the work. The leader asset may determine a plan for asset 585 to perform a task based at least in part on the capabilities of asset 585, etc.

[0142] In some embodiments, the control center (e.g., control center 550 of FIG. 5A, 5B, or 5C) is one of multiple control centers that control a system including semi-autonomous drones. In various embodiments, the control center includes a ground control center, an airborne control center, a surface control center, or any other suitable control center. In various embodiments, a single control center is in control, each control center controls a portion of the system assets, control rotates among multiple control centers for coordinated takeover during a change of control, or any other suitable allocation of control to control centers.

[0143] FIG. 6 illustrates a user interface for configuring, monitoring, and / or controlling operations according to various embodiments of the present application.

[0144] According to various embodiments, the server may configure one or more user interfaces in connection with configuring a task. For example, the server may generate and configure a set of user interfaces to create a wizard that allows the configuration process to define the task efficiently and intuitively. In various embodiments, a user interface is configured based on user input to a previous user interface (e.g., the immediately preceding user interface).

[0145] As shown in FIG. 6, a set of user interfaces 610, 630, and 650 may be displayed to a user to allow the user to systematically define parameters for an operation.

[0146] In interface 610, the user interface includes elements for the user to (i) enter a name and description of the operation, (ii) select the type of operation (e.g., scan road network, scan along route, loiter flight and track, find, confirm, and track target), and (iii) select the type of target (e.g., target for surveillance or tracking) to operate on. User interface 630 may be configured in response to the inputs to interface 610.

[0147] In user interface 630, the user interface includes selectable elements for a user to select parameters associated with the type of asset and / or the manner in which the asset moves (e.g., air, ground, sea). The various selectable elements on user interface 630 may be determined in response to inputs to interface 610.

[0148] In user interface 650, the user interface includes selectable elements for a user to select one or more parameters associated with the job. For example, a user may define the terrain or boundaries within which the job is to be performed. As another example, a user may define restricted areas within which assets performing the job cannot enter, etc. As another example, a user may define the dates and times within which the job is to be performed, or the dates or times within which the job is restricted. User interface 650 may include selectable elements for causing a server to generate the job. For example, the server may generate the job based on a user-defined job configuration / definition.

[0149] In response to generating the task, the server may cause a user interface to display a live task screen. The live task screen may include selectable elements for the user to select to activate the task, deactivate the task, or abandon the task. In some embodiments, the live task screen includes current status information regarding the state of execution of the task. For example, the live task screen may indicate a percentage of the task that is complete. As another example, the live task screen may include a live stream of video captured by at least one asset in the asset group. In some embodiments, the live task screen may include selectable elements through which the user can invoke a user interface (or a wizard in the user interface) to modify the task. For example, the user may cause the server to add one or more assets to the asset group. As another example, the user interface may display options (e.g., server-generated options such as possible modifications) that allow the user to select to modify the task according to suggested modifications. The server may cause the user interface to display improvements to the task by adding a specific asset, adding an asset with specific functionality, or adding one or more assets. The user may input a selection to modify the task according to suggested modifications presented by the server. In some examples, the server may cause a user interface to display the cost associated with performing work with the asset group and / or the cost difference or cost impact when an asset is added to or removed from the asset group.

[0150] 7A illustrates a method for configuring work in accordance with various embodiments of the present application. In some embodiments, the process 700 of FIG. 7A is performed by the server 105 of the system 100 of FIG. 1 and / or the device 200 of FIG. 2.

[0151] At step 710, data associated with one or more tasks to be performed is received. In some embodiments, a server receives the data associated with the one or more tasks in connection with configuring a task. The data associated with the one or more tasks may be received based at least in part on one or more user inputs provided to a user interface presented at the client terminal. According to various embodiments, the data associated with the one or more tasks includes one or more task parameters and / or task constraints. The one or more tasks to be performed may correspond to the task to be performed.

[0152] At step 720, a group of assets is determined. In some embodiments, the server determines a group of assets to collectively perform one or more tasks or operations that include one or more tasks. The group of assets may be determined based on one or more functions associated with one or more tasks / operations and / or the capabilities of the assets. For example, the server may determine a group of assets that have capabilities that match the capabilities associated with one or more tasks / operations.

[0153] According to various embodiments, the assets may be determined based at least in part on asset availability and / or asset ownership (or availability for ownership transfer). For example, a server may determine a superset of assets (e.g., all assets across multiple organizations) and query an asset or third-party service for an indication of whether an asset is available and / or available for ownership transfer for use in performing work. The third-party service may be an ownership service that manages asset ownership across multiple organizations and / or ownership transfer or licensing according to negotiated terms (e.g., fixed-term lease, lease for a specific task, lease for a specific work, permanent transfer, etc.).

[0154] In response to determining that the assets will perform one or more tasks (or corresponding operations), process 700 proceeds to step 730, where instructions are communicated to at least one drone in the assets. In some embodiments, a server determines a leader drone in the assets, and the server communicates instructions to the leader drone. The instructions may include an indication that a leader drone is in the assets, or may be communicated in conjunction with that indication. In some embodiments, the instructions communicated to the leader drone include operation parameters. The instructions may also include a pre-plan if the server or other service generated a high-level pre-plan for the operation.

[0155] In response to communicating the command to the at least one drone, process 700 proceeds to step 740, where a determination is made as to whether the process is complete. As an example, the process may be determined to be complete based at least in part on user input (e.g., input to cancel or pause the task). As another example, the process may be determined to be complete in response to a user selecting to begin performing the task. If the process is deemed complete, process 700 ends. Otherwise, process 700 returns to step 710, where more data associated with one or more tasks is received. In some embodiments, process 700 may be deemed incomplete at step 740 in response to user input to further refine / configure the task. As an example, before starting the task, a user may input a request to edit the task into a user interface. As another example, during the performance of a task by a fleet of assets, a user may input selections to modify the task, such as dynamically adding assets to the fleet to accelerate completion of the task or removing assets based on reprioritizing the task relative to another task.

[0156] If the asset group cannot be determined at step 720, process 700 may proceed to step 740. The asset group cannot be determined if there are no assets available, if asset ownership is not available that matches the functionality of one or more tasks or jobs, if no assets are found whose functionality collectively matches the functionality of the job, etc. At step 740, a prompt may be provided to the user indicating that the asset group cannot be determined, requesting the user to modify the job configuration, or requesting whether the user requests to cancel the job.

[0157] Figure 7B illustrates a method for configuring work in accordance with various embodiments of the present application. In some embodiments, process 720 of Figure 7B is performed by server 105 of system 100 of Figure 1 and / or device 200 of Figure 2. Process 720 of Figure 7B may correspond to step 720 of Figure 7A.

[0158] One or more characteristics of one or more tasks are obtained in step 721. In some embodiments, the server may obtain one or more characteristics based on a definition of the task (e.g., task parameters).

[0159] At step 722, one or more capabilities of the asset are obtained. In some embodiments, the server may obtain one or more capabilities of the asset based on a mapping of capabilities to the asset. The mapping of capabilities of the asset may be determined based on querying the asset for an indication of its capabilities (or their current capabilities), based on the asset registering / advertising its capabilities, etc.

[0160] At step 723, a determination is made as to whether any assets match the characteristics of the task. In some embodiments, the server determines one or more functions corresponding to the characteristics of the task and determines assets that have functions that match the functions corresponding to the characteristics of the task. In response to determining that an asset or assets have functions that match the characteristics of the task, process 720 may proceed to step 727.

[0161] At step 724, an asset group is determined. In some embodiments, the server determines the asset group from among assets having functionality matching one or more task characteristics. The server may select assets to be included in the asset group based on asset availability, asset ownership, availability of asset ownership transfer, a determination that the asset group has redundancy with respect to functionality, etc. In some embodiments, the server uses a cost function in connection with selecting assets for inclusion in the asset group. For example, the server may use the cost function to optimize the cost of the asset group or select assets having costs below a threshold cost value.

[0162] In response to determining the asset group, process 720 proceeds to step 725 where a determination is made as to whether the asset group determination is complete. For example, the server may prompt the user to confirm the asset group or may provide selectable elements through which the user can choose to improve / modify one or more tasks or one or more features of the tasks. In response to determining that the asset group determination is not complete, process 720 returns to step 721. Otherwise, process 720 proceeds to step 726. If, in step 723, no assets were found that matched the characteristics of the task (or if, for each characteristic of the task, no assets with corresponding functionality were found), process 720 may proceed to step 725.

[0163] In step 726, suggestions for asset groups and / or how assets may be grouped (eg, assets may be grouped in relation to perform tasks) are provided.

[0164] FIG. 8A illustrates a method for performing at least one task of an operation, according to various embodiments herein. In some embodiments, process 800 of FIG. 8A is performed by asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1 and / or device 300 of FIG. 3. According to various embodiments, process 800 is performed by a semi-autonomous drone.

[0165] In some embodiments, process 800 is performed by a leader drone in a fleet of assets to perform a task. In some embodiments, process 800 is performed by a follower drone in a fleet of assets to perform a task.

[0166] An indication that the drone is part of a group of assets performing an element of a task is received at step 810. The drone may receive an indication that the drone is part of a group of assets associated with configuring the work.

[0167] According to various embodiments, when process 800 is performed by a leader drone, the leader drone may receive an indication from a server (e.g., a server configuring the operation) that the drone is part of a group of assets, along with operation parameters associated with the operation for which the task elements are being performed, etc.

[0168] According to various embodiments, when process 800 is performed by a follower drone, the follower drone may receive an indication from a leader drone that the drone is part of a group of assets.

[0169] Information regarding one or more elements of one or more tasks is communicated at step 820. The information regarding the one or more elements may include parameters of the elements and / or constraints for executing the elements.

[0170] According to various embodiments, when process 800 is performed by a leader drone, the leader drone may communicate a plan for executing one or more elements to one or more follower drones assigned to execute the one or more elements.

[0171] According to various embodiments, when process 800 is performed by a follower drone, the follower drone may receive a plan for executing one or more elements. The follower drone may receive the plan from the leader drone. The plan may be determined based on the capabilities of the follower drone, capabilities associated with the elements, the status of the follower drone (e.g., availability, location, etc.), etc.

[0172] At step 840, a determination is made as to whether process 800 is complete. In response to a determination that process 800 is complete, process 800 ends. In response to a determination that process 800 is not complete, process 800 may return to step 810. According to various embodiments, process 800 is determined to be complete in response to one or more elements being completed or a determination that an operation is complete. Process 800 may also be determined to be complete if an operation is paused or canceled / aborted.

[0173] FIG. 8B illustrates a method for performing at least one task of a task, according to various embodiments herein. In some embodiments, process 830 of FIG. 8B is performed by asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1 and / or device 300 of FIG. 3. According to various embodiments, process 830 is performed by a semi-autonomous drone. Process 830 of FIG. 8B may correspond to step 830 of process 800 of FIG. 8A. Process 830 of FIG. 8B may be performed by a follower drone in a group of assets assigned to perform the task.

[0174] In step 831, data from one or more sensors of the drone is received. In some embodiments, during execution of a plan (e.g., a plan provided by a leader drone), a follower drone acquires information about the element / task. For example, a follower drone may capture information such as images, live stream video, weather information, etc.

[0175] At step 832, a determination is made as to whether the data is relevant to the plan. In some embodiments, in response to receiving data from a sensor on the drone, the drone determines whether such data is relevant to the plan to be implemented. As an example, in the case of a plan to track a target, the data received from the sensor may include an image. The drone may determine whether the image contains the target that the drone is tracking. For example, the drone may perform image analysis to determine that the target is in the image (or that the likelihood that the image contains the target exceeds a predetermined threshold), etc.

[0176] In response to determining that the data is associated with the plan, process 830 may proceed to step 833 where information regarding the plan is transmitted to the leader drone. In some embodiments, the follower drone may determine that the captured information is associated with the plan (e.g., indicates the status of elements / tasks, includes targets / objectives for the plan, includes information regarding the geographic area in which the plan is to be implemented, etc.).

[0177] In response to transmitting information regarding the plan to the leader drone, process 830 proceeds to step 834, where a determination is made as to whether process 830 is complete. In some embodiments, process 830 is determined to be complete in response to a determination that the operation is completed, stopped, and / or paused. In response to a determination that process 830 is complete, process 830 ends. In response to a determination that process 830 is not complete, process 830 may return to step 831, where the drone continues to monitor data / information captured by its sensors and repeatedly transmit associated information to the leader drone. As an example, process 830 may be determined to be not complete in response to a determination that the task is not complete (e.g., if a drone is tasked with scanning a road and the drone is unable to maintain a series of sharp turns in the road and misses scanning a portion of the road, the task may be considered not complete, and the drone may turn back and rescan the missed portion of the road).

[0178] If step 832 determines that the data is not related to the plan, process 830 proceeds to step 834, which is performed as described above.

[0179] FIG. 8C illustrates a method for performing at least one task of a task, according to various embodiments herein. In some embodiments, process 830 of FIG. 8C is performed by asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1 and / or device 300 of FIG. 3. According to various embodiments, process 830 is performed by a semi-autonomous drone. Process 830 of FIG. 8C may correspond to step 830 of process 800 of FIG. 8A. Process 830 of FIG. 8C may be performed by a leader drone in a group of assets assigned to perform the task.

[0180] Feedback information is received from one or more follower drones or control information is received at step 835. According to various embodiments, the leader drone receives feedback information from the follower drones concurrently with execution of a plan for a task or task element associated with the operation.

[0181] The feedback information may include current / contemporaneous information such as current location, live video stream, current status of the operation (e.g., in the case of a surveillance operation, the status may be that no target is present within the defined area of ​​the operation).

[0182] The control information may include information provided from the server to the leader drone, such as indications that the operation has been updated, operation parameters, operation constraints, instructions / commands for modifying the assets, etc.

[0183] At step 836, a decision is made as to whether to update the plan. According to various embodiments, the leader drone determines whether to autonomously update the plan based on feedback or control information. For example, if feedback information indicates completion of an element or task of a task, the leader drone may update the plan for the corresponding follower drone so that the follower drone performs a new task / element. As another example, in the case of surveillance within a defined area, in response to determining that feedback information indicates that the target is within the defined area, the leader drone may decide to update the plan to move closer to the target in order to obtain specific details about the target, or the leader drone may decide to update the plan to instruct the follower drone to switch to a tracking task to track the target.

[0184] In response to determining at step 836 that the plan is updated, process 830 may proceed to step 837, where the updated plan is transmitted to one or more follower drones, and process 830 proceeds to step 838. Conversely, in response to determining at step 836 that the plan is not updated, process 830 proceeds to step 838.

[0185] At step 838, a determination is made as to whether process 830 is complete. In some embodiments, process 830 is determined to be complete in response to determining that the operation is completed, stopped, and / or paused. In response to determining that process 830 is complete, process 830 ends. In response to determining that process 830 is not complete, process 830 may return to step 835 where the drone continues to monitor for feedback or control information regarding the operation.

[0186] 9A illustrates a method for executing a plan associated with a task, according to various embodiments of the present application. In some embodiments, process 900 of FIG. 9A is performed by asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1 and / or device 300 of FIG. 3. According to various embodiments, process 800 is performed by a semi-autonomous drone.

[0187] An indication that the drone is part of a group of assets performing an element of a task is received at step 910. The drone may receive an indication that the drone is part of a group of assets associated with configuring the work.

[0188] According to various embodiments, when process 900 is performed by a leader drone, the leader drone may receive an indication from a server (e.g., a server configuring the operation) that the drone is part of a group of assets, along with operation parameters associated with the operation for which the task elements are being performed, etc.

[0189] According to various embodiments, when process 900 is performed by a follower drone, the follower drone may receive an indication from a leader drone that the drone is part of a group of assets.

[0190] The plan is implemented at step 920. In some embodiments, the drone obtains a plan associated with performing a task and implements the plan.

[0191] According to various embodiments, when process 900 is performed by a leader drone, the leader drone may receive control information from a server (e.g., a server that configured the task). The control information may include task parameters, constraints for performing the task, a pre-plan, any combination thereof, etc. The leader drone may determine a plan for performing the task based on high-level information for the task. For example, the leader drone may determine a plan for one or more follower drones to perform tasks or task elements of the task and command the follower drones to implement the plan for performing the tasks / elements.

[0192] According to various embodiments, when process 900 is performed by a follower drone, the follower drone may implement a plan for executing the task / element. The follower drone may receive the plan from the leader drone. In some embodiments, the follower drone may update the plan to fill gaps in the plan received from the leader drone. For example, the follower drone modifies the plan to have finer granularity than the plan received from the leader drone.

[0193] At step 930, a determination is made as to whether the asset has failed. According to various embodiments, the determination of whether the asset has failed may be based on the time since the asset last communicated with the drone or the lack of a response to a ping for the asset's health status. In response to determining that the asset has failed, process 900 proceeds to step 940.

[0194] The plan is updated at step 940. According to various embodiments, the plan is updated to take into account the determination that the asset has failed.

[0195] According to various embodiments, when process 900 is performed by a leader drone, the leader drone may determine the status of the work, such as the status of tasks / elements assigned to the asset determined to have failed. Based on the status of the tasks / elements, the leader drone may update the plan to reallocate any remaining assignments (e.g., uncompleted portions of tasks / elements assigned to the asset) to one or more remaining assets in the group of assets assigned to perform the work.

[0196] According to various embodiments, when process 900 is performed by a follower drone, the follower drone may update its plan in response to determining that the leader drone has failed. In response to determining that the leader drone has failed, the follower drone may negotiate with one or more other remaining assets in the group of assets to determine a new leader. In response to the leader drone being promoted to the new leader drone, the new leader drone may update its plan based on its position as the new leader drone. For example, the new leader drone may determine one or more plans for the remaining assets. The new leader drone may determine a plan based on the capabilities of each of the remaining assets and the functions or features associated with tasks / elements that have not been completed in the execution of the work.

[0197] Information regarding the updated plan is communicated at step 950. In some embodiments, the updated plan is communicated to the remaining assets.

[0198] At step 960, information is communicated during the implementation of the updated plan.

[0199] According to various embodiments, when process 900 is performed by a leader drone, the leader drone may receive information from a follower drone / asset or a server. As an example, the leader drone may receive control information from a server (such as a server that configured the job). The control information may include updated job parameters, constraints for performing the job, a pre-plan, any combination thereof, etc. As another example, the leader drone may receive feedback information from the follower drone regarding the job, such as feedback information obtained by the follower drone during execution of an updated plan.

[0200] According to various embodiments, when process 900 is performed by a follower drone, the follower drone sends feedback information to the leader drone.

[0201] At step 970, a determination is made as to whether process 900 is complete. In response to a determination that process 900 is complete, process 900 ends. In response to a determination that process 900 is not complete, process 900 0 may return to step 920. According to various embodiments, process 900 may be determined to be complete in response to one or more elements being completed or a determination that the work is complete. Process 900 may also be determined to be complete if the work is paused or canceled / aborted. Process 900 may proceed to step 970 in response to a determination at step 930 that no asset failure has occurred.

[0202] FIG. 9B illustrates a method for executing a plan associated with an operation in accordance with various embodiments of the present application. In some embodiments, process 930 of FIG. 9A is performed by asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1 and / or device 300 of FIG. 3. Process 930 of FIG. 9B may correspond to step 930 of process 900 of FIG. 9A.

[0203] In step 931, feedback information is received from one or more follower drones, or control information is received. In some embodiments, the leader drone may receive feedback information from the follower drones at the same time the follower drones are implementing a plan to perform a task / operation. In some embodiments, the leader drone may receive control information (e.g., updated control information) from a server. For example, the control information may include an indication that the operation has been updated, operation parameters, operation constraints, instructions / commands to modify the assets, etc.

[0204] At step 932, a determination is made as to whether feedback information has been received from an asset in the asset group. In some embodiments, the leader drone performs process 930 of FIG. 9B for each asset in the asset group. For example, the leader drone may determine whether feedback information has been received from a particular follower drone. In some embodiments, the determination of whether feedback information has been received from a particular asset may be based on the amount of time that has elapsed since the particular asset last communicated feedback information. In some embodiments, the determination of whether feedback information has been received from a particular asset within a predetermined threshold time is done In some embodiments, determining whether feedback information has been received from a particular asset by a predetermined time (e.g., an estimated time (e.g., the estimated time may be determined based at least in part on a plan being implemented by the asset, the location of the asset, a communications link with the asset, etc.)) is done .

[0205] In response to a determination at step 932 that feedback information has been received from an asset in the asset group, process 930 proceeds to step 936. At step 936, a determination is made as to whether process 930 is complete. In response to a determination that process 930 is complete, process 930 proceeds to step 937. In response to a determination that process 930 is not complete, process 930 may return to step 931. According to various embodiments, process 930 is determined to be complete in response to a determination that one or more elements have been completed or that the task has been completed. Process 930 may also be determined to be complete if the task has been paused or canceled / aborted. At step 937, an indication of whether an asset has failed is provided. In some embodiments, the module that determines whether an asset has failed provides an indication of whether an asset has failed to a planning service of the leader drone (e.g., a service that determines / updates a plan for executing one or more tasks associated with the task).

[0206] In response to determining at step 932 that feedback information has not been received from an asset in the asset group, process 930 proceeds to step 933. At step 933, a ping is sent to the asset. The leader drone may send the ping as a health check on the asset.

[0207] At step 934, a determination is made as to whether a response has been received from the asset. In some embodiments, the leader drone determines whether a response has been received based on a determination of whether the asset responded to the ping within a threshold period of time. In response to a determination that a response (to the ping) has been received from the asset, process 930 proceeds to step 936. Conversely, in response to a determination that a response (to the ping) has not been received from the asset, process 930 proceeds to step 935, where it is determined that the asset has failed. Process 930 then proceeds to step 936.

[0208] 9C illustrates a method for executing a plan associated with a task according to various embodiments of the present application. In some embodiments, process 950 of FIG. 9C is performed by asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1 and / or device 300 of FIG. 3. Process 950 of FIG. 9C may correspond to step 950 of FIG. 9A. In some embodiments, process 950 is performed by a semi-autonomous leader drone.

[0209] At step 951-1, feedback information regarding the execution of the plan is received. According to various embodiments, the leader drone receives feedback information from the follower drones contemporaneously with the execution of the plan regarding tasks or task elements associated with the operation. The feedback information may include current / contemporaneous information such as current location, live video stream, current status of the operation (e.g., for a surveillance operation, the status may be that no targets are present within the defined area of ​​the operation).

[0210] At step 951-2, a determination is made as to whether the plan is to be updated. In some embodiments, the leader drone determines whether to update the plan (e.g., a plan for a particular follower drone or plans for multiple follower drones) based at least in part on the feedback information.

[0211] In response to a determination that the plan will not be updated, process 950 proceeds to step 951-7 where a determination is made as to whether process 950 is complete. In response to a determination that process 950 is complete, process 950 ends. In response to a determination that process 950 is not complete, process 950 may return to step 951-1. According to various embodiments, process 950 is determined to be complete in response to one or more elements being completed or a determination that the work is complete. Process 950 may also be determined to be complete if the work is paused or canceled / aborted.

[0212] In response to a determination that the plan will be updated, process 950 proceeds to step 951-3 where a determination is made as to whether a plan update is possible. In some embodiments, the leader drone determines whether a plan update is possible based on parameters of the operation, the status of the operation, the capabilities of one or more assets in the asset pool, etc. For example, if it is determined that the operation will be updated, or the capabilities required to perform the operation have changed or been newly determined, the leader drone may update the plan. Do The loan may determine whether the asset group includes an asset with the required functionality.

[0213] In response to determining in step 951-3 that a plan update is not possible, process 950 indicates that a plan update is not possible. is communicated The process 950 then proceeds to step 951-5. In some embodiments, the leader drone may provide a prompt to the user at a user interface or to the server (which may provide a prompt at the client terminal) that the plan cannot be updated. The prompt may include elements that allow the user to cancel the operation, modify the operation, etc. The process 950 then proceeds to step 951-7.

[0214] In response to determining at step 951-3 that a plan update is possible, process 950 proceeds to step 951-4, where the plan is updated. In some embodiments, the leader terminal updates the plan accordingly (e.g., based at least in part on the feedback information). The plan may be updated based at least in part on the capabilities of the follower drones.

[0215] At step 951-6, the updated plan is transmitted to the follower drone. The process 950 may then proceed to step 951-7.

[0216] FIG. 9D illustrates a method for executing a plan associated with a task according to various embodiments of the present application. In some embodiments, operation 951-2 of FIG. 9D is performed by asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1 and / or device 300 of FIG. 3. Operation 951-2 of FIG. 9D may correspond to operation 951-2 of FIG. 9C. In some embodiments, operation 951-2 is performed by a semi-autonomous leader drone.

[0217] At step 951-A, a determination is made as to whether an asset failure has occurred. According to various embodiments, the determination of whether an asset has failed may be based on the time since the asset last communicated with the drone or the lack of a response to a ping for the asset's health status.

[0218] Assets When it broke down In response to the determination, process 951-2 proceeds to step 951-B where a determination is made as to whether the asset failure affected the performance of the work. The leader drone may determine whether the failed asset had any outstanding tasks or elements that affected the work.

[0219] In response to a determination that the asset failure does not affect the execution of the work, process 951-2 may proceed to step 951-G, where a decision is made not to update the plan. Process 951-2 may then proceed to step 951-H, where a determination is made as to whether process 951-2 is complete. In response to a determination that process 951-2 is complete, process 951-2 ends. In response to a determination that process 951-2 is not complete, process 951-2 may return to step 951-A. According to various embodiments, process 951-2 is determined to be complete in response to one or more elements being completed or a determination that the work is complete. Process 951-2 may also be determined to be complete if the work is paused or canceled / aborted.

[0220] In response to a determination that the asset failure will not affect the execution of the work, process 951-2 may proceed to step 951-C where a decision is made to update the plan. Process 951-2 may then proceed to step 951-H where a determination is made as to whether process 951-2 is complete. In response to a determination that process 951-2 is complete, process 951-2 ends.

[0221] In response to a determination that the asset has not failed, process 951-2 proceeds to step 951-D. At step 951-D, a determination is made as to whether the plan has failed. In some embodiments, the leader drone may determine whether the plan has failed based on a determination of whether the asset has failed. For example, if the plan had a corresponding capability (e.g., a required capability) and the failed asset had a capability for which the asset group did not have redundancy in at least one other asset (and it was determined that a new asset / capability could not be acquired), the leader drone may determine that the plan has failed.

[0222] In response to determining that the plan has failed, process 951-2 proceeds to step 951-E where a determination is made as to whether the work will be stopped (or paused). The leader drone may determine to stop the work in response to determining that the plan is a necessary component of the work (including tasks / elements required by the work). The leader drone may determine to stop the work based, at least in part, on the work having a corresponding function (e.g., a required function) and the failed asset had a function for which the asset group does not have redundancy in at least one other asset (and it was determined that a new asset / function cannot be acquired).

[0223] In response to the decision to stop work at step 951-E, process 951-2 proceeds to step 951-H.

[0224] In response to a determination at step 951-D that no plan failure has occurred or a determination at step 951-E that work will not be stopped, process 951-2 proceeds to step 951-F where a determination is made that the plan will be updated.

[0225] FIG. 9E illustrates a method for executing a plan associated with a task according to various embodiments of the present application. In some embodiments, process 950 of FIG. 9E is performed by asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1 and / or device 300 of FIG. 3. Process 950 of FIG. 9E may correspond to step 950 of FIG. 9A. In some embodiments, process 950 is performed by a semi-autonomous follower drone.

[0226] At step 952-1, data is received from one or more sensors of the drone. In some embodiments, during execution of a plan (e.g., a plan provided by a leader drone), a follower drone acquires information about the element / task. For example, the follower drone may capture information such as images, live stream video, weather information, etc.

[0227] At step 952-3, a determination is made as to whether the data is relevant to the plan. In some embodiments, in response to receiving data from a sensor on the drone, the drone determines whether such data is relevant to the plan to be implemented. As an example, in the case of a plan to track a target, the data received from the sensor may include an image. The drone may determine whether the image includes the target that the drone is tracking. For example, the drone may perform image analysis to determine that the target is in the image (or that the likelihood that the image includes the target exceeds a predetermined threshold), etc.

[0228] In response to determining that the data is related to the plan, process 950 may proceed to step 952.5, where information regarding the plan is transmitted to the leader drone. In some embodiments, the follower drone may determine that the captured information is associated with the plan (e.g., indicates the status of elements / tasks, includes targets / objectives for the plan, includes information regarding the geographic area in which the plan is to be implemented, etc.).

[0229] In response to transmitting information regarding the plan to the leader drone, process 950 proceeds to step 952-7 where a determination is made as to whether process 950 is complete. In some embodiments, process 950 is determined to be complete in response to a determination that the operation is completed, stopped, and / or paused. In response to a determination that process 950 is complete, process 950 ends. In response to a determination that process 950 is not complete, process 950 may return to step 952-1, where the drone continues to monitor data / information captured by its sensors and repeatedly transmit associated information to the leader drone.

[0230] If step 952-3 determines that the data is not related to the plan, then process 950 proceeds to step 952-1, which is performed as described above.

[0231] FIG. 10A illustrates a method for executing a plan associated with a task, according to various embodiments of the present application. In some embodiments, process 1000 of FIG. 10A is performed by server 105, asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1, device 200 of FIG. 2, and / or device 300 of FIG. 3. According to various embodiments, process 1000 is performed by a semi-autonomous drone or a server managing the task.

[0232] In operation 1010, data associated with one or more tasks being performed by the assets is obtained. According to various embodiments, the data associated with the one or more tasks may include information regarding updates to the configuration / definition of the task or feedback information obtained and communicated during the performance of the task or elements of the task. Updates to the configuration / definition of the task may be based at least in part on one or more user inputs corresponding to a user request to modify the task (e.g., to accelerate completion of the task, to change the priority of the task relative to at least one other task, etc.). The feedback information may include a change in the status of the task, such as the satisfaction of a condition that triggers one or more other tasks. As a specific example, in the context of a surveillance task in a defined geographic area, the task may be defined to trigger a tracking task if a target object / vehicle is detected within the geographic area. Thus, during a surveillance task by one or more follower drones, if a target is found within the geographic area, the follower drone may provide feedback information to the leader drone indicating its sighting. As another specific example, the feedback information may include an indication that the follower drone has malfunctioned or been destroyed. The failure or destruction of a follower drone may degrade the functionality of the asset group or the redundancy of the asset group's functionality.

[0233] According to various embodiments, when process 1000 is performed by a leader drone, the leader drone may receive data associated with one or more tasks from a server, such as via control information, or from one or more follower drones, such as via feedback information transmitted simultaneously as the follower drone executes a plan associated with performing work (e.g., performing one or more tasks).

[0234] According to various embodiments, when process 1000 is performed by a server, the server may receive data associated with one or more tasks from the server, such as through user input to a user interface. A user may enter commands or requests into the user interface that may cause an operation to be modified or parameters of an operation to be modified.

[0235] At step 1020, a determination is made as to whether the asset group is to be modified. According to various embodiments, the determination as to whether to modify the asset group may be based at least in part on data associated with one or more tasks being performed by the assets (e.g., operation data related to the current state or characteristics of the operation). The determination as to whether to modify the asset group may include a determination as to whether to add one or more assets to the asset group. Similarly, the determination as to whether to modify the asset group may include a determination as to whether to remove one or more assets from the asset group. For example, the device may determine, based at least in part on the data, that a new task is to be performed. As another example, the new task may include or require a function for which the current asset group does not include an asset with the corresponding function. As another example, the device may determine that the asset group has lost a function or redundancy in a function due to an asset failure. Redundancy in a function may allow multiple tasks requiring the function to be performed (e.g., in parallel) by multiple assets having the function. Loss of functional redundancy may reduce the number of tasks that can be performed in parallel or reduce the flexibility with which a leader drone can allocate tasks or make planning decisions across a fleet of assets.

[0236] According to various embodiments, a decision to modify a group of assets may be based on a decision to add functionality to the group of assets and / or a determination that the functionality of the group of assets is no longer needed (or is predicted to be no longer needed) for the current task. The group of assets may be determined to be modified to add one or more assets, remove one or more assets, or both add one or more assets and remove one or more assets. In some embodiments, the decision whether to modify the group of assets may be based on a cost (e.g., a cost determined by a predetermined cost function) associated with using the group of assets to perform the task or the remainder of the task. Determining whether to modify the group of assets while the task is being performed enables dynamic resource management that may ensure, for example, that appropriate functionality is assigned to the group of assets. Furthermore, dynamic resource management may enable optimization or improvement of costs associated with performing the task.

[0237] In response to a decision to modify the asset group at step 1020, process 1000 proceeds to step 1030, where instructions are communicated indicating the modification of the asset group. The instructions may be provided to the affected assets (e.g., assets being added to or removed from the group) and / or the leader asset of the corresponding asset group.

[0238] According to various embodiments, when process 1000 is performed by a leader drone, the leader drone may send instructions to modify the assets to the assets affected by the modification and / or to a server (such as a server managing or configuring the work). In some embodiments, the leader drone may send the modification instructions or suggestions to a property rights service that manages ownership of assets across multiple organizations. For example, instructions may be sent to the property rights service in connection with negotiating ownership of assets affected by the modification.

[0239] According to various embodiments, when process 1000 is performed by a server, the server communicates instructions to modify the assets to a leader drone and / or to assets affected by the modification, and the server may provide a suggestion to a user via a user interface on a client terminal, where the suggestion notifies the user that the assets will be modified.

[0240] In response to communicating instructions indicating the modification of the assets, process 1000 proceeds to step 1040, where a determination is made as to whether process 1000 is complete. In some embodiments, process 1000 is determined to be complete in response to a determination that the work is completed, stopped, and / or paused. In response to a determination that process 1000 is complete, process 1000 terminates. In response to a determination that process 1000 is not complete, process 1000 may return to step 1010, where the drone continues to monitor data / information captured by its sensors and repeatedly transmit associated information to the leader drone. Further, in response to a determination at step 1020 that the assets are not modified, process 1000 may proceed to step 1040.

[0241] FIG. 10B illustrates a method for executing a plan associated with an operation in accordance with various embodiments of the present application. In some embodiments, process 1020 of FIG. 10B is performed by server 105, asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1, device 200 of FIG. 2, and / or device 300 of FIG. 3. Process 1020 of FIG. 10B may correspond to step 1020 of FIG. 10A.

[0242] In step 1021, one or more features of one or more remaining tasks are obtained. In some embodiments, a device (e.g., a server or a drone, such as a leader drone) may determine the remaining tasks for the work and determine one or more features associated with the remaining tasks. The one or more features associated with the remaining tasks may be determined based on a mapping of tasks to features. For example, the device may use the remaining tasks to query the mapping of tasks to features.

[0243] In step 1022, one or more capabilities of the asset are obtained. In some embodiments, the device may obtain one or more capabilities of the asset based on a mapping of capabilities to the asset. The mapping of asset capabilities may be determined based on querying the asset for an indication of its capabilities (or its current capabilities), based on the asset registering / advertising its capabilities, etc.

[0244] At step 1023, a determination is made as to whether any assets match the remaining task characteristics. In some embodiments, the server determines one or more capabilities corresponding to the task characteristics and determines assets that have capabilities that match the capabilities corresponding to the task characteristics. In response to determining that an asset or assets have capabilities that match the remaining task characteristics, process 1020 may proceed to step 1024.

[0245] At step 1024, modifications to the asset group are determined. In some embodiments, the device determines the asset group from among assets having functionality that matches one or more remaining task characteristics. In some embodiments, the device determines the modifications to the asset group based on assets in the asset group having a set of functionality that does not match one or more remaining task characteristics (e.g., there is no overlap between the functionality of the excluded asset and the features / functionality of the remaining task). The device may select assets to add to the asset group based on asset availability, asset ownership, availability of asset ownership transfer, a determination that the asset group has redundancy in terms of functionality, etc. In some embodiments, the device uses a cost function in connection with selecting assets to include in or exclude from the asset group. For example, the server may use the cost function to optimize the cost of the asset group or select assets having a cost below a threshold cost value.

[0246] In response to determining the asset group, process 1020 proceeds to step 1025 where a determination is made as to whether the asset group determination is complete. For example, the device may prompt the user to confirm the asset group or may provide a selectable element through which the user can select to improve / modify one or more tasks or one or more features of the tasks. In response to determining that the asset group determination is not complete, process 1020 returns to step 1021. Otherwise, process 1020 proceeds to step 1026. If, in step 1023, no assets were found that match the remaining task characteristics (or if, for each characteristic of the task, no assets with corresponding features were found), process 1020 may proceed to step 1025. In step 1026, a suggestion of whether to modify the asset group is provided. In some embodiments, the suggestion of whether to modify the assets is provided to a planning service running on the leader drone. In some embodiments, the suggestion of whether to modify the assets is provided to a control service, such as a server.

[0247] FIG. 11 illustrates a method for configuring a task according to various embodiments of the present application. 11 Processing 1 1 00 is a diagram illustrating the server 105, the asset 120, the asset 125, and / or the asset 130 of the system 100 of FIG. Figure 2 Device 20 to 0 According to various embodiments, process 1100 is performed by a server (e.g., a server that configures and / or manages operations). 1 00 may be executed by a server in conjunction with a client terminal.

[0248] In step 1105, a first user interface is displayed. In some embodiments, the server causes the client terminal to display the first user interface. The first user interface may correspond to an interface through which a user inputs one or more features or parameters associated with the task. For example, the first user interface may be used in connection with defining the task. The user may input one or more features or parameters via one or more selectable elements provided on the user interface.

[0249] A user selection regarding a feature of the task to be performed is received at step 1110. The user selection may be made to a client terminal, which may communicate the user selection to a server over one or more networks.

[0250] At step 1115, a configuration for another user interface is determined based at least in part on the user selection. In some embodiments, the server determines one or more features, parameters, or constraints that are mapped to the user selection (e.g., mapped to the characteristics of the task). For example, if the user selection for the first user interface is a selection of a type of work (e.g., surveillance), the server may determine the another user interface to include a definition of the surveillance context, such as time, date, geographic location, target type (e.g., moving target, stationary target, etc.). As another example, if the user selection for the first user interface is a selection of a type of work and / or a context of the work (e.g., surveillance), the server may determine the another user interface to include a definition of one or more features related to assets used in connection with performing the work.

[0251] The alternative user interface is displayed at step 1120. In some embodiments, in response to determining the configuration of the alternative interface, the server causes the client terminal to display the alternative user interface.

[0252] At step 1125, a user selection regarding assets to be deployed to perform the operation is received. The user selection may be a selection of specific assets or types of assets to be deployed to perform the operation. In some embodiments, the user selection regarding assets to be deployed to perform the operation includes a user selection for one or more parameters of the operation that are used by the server in determining the assets or types of assets to deploy. The user selection may be made to a client terminal, which may communicate the user selection to the server over one or more networks.

[0253] At step 1130, a decision is made whether to provide another user interface for configuring the job. The decision whether to provide another user interface may be based at least in part on job and / or asset parameters obtained in the first user interface and the other user interfaces. In some embodiments, the decision whether to provide another user interface is based on user input, such as to cancel the job or start the job. For example, the server may fill gaps in the job definition and deploy corresponding assets. As an example, the server may determine any remaining parameters of the job based at least in part on one or more of: (i) historical information, such as information about similar / past jobs; (ii) a best fit associated with the user-defined parameters of the job; and (iii) the cost of performing the job as determined using a cost function.

[0254] In response to determining that an alternative user interface is to be provided for configuring the work, process 1100 proceeds to step 1135, which causes the alternative interface to be displayed. The alternative interface may be configured based at least in part on user selections regarding characteristics of the task to be performed and / or user selections regarding assets to be deployed.

[0255] At step 1140, a user selection for another user interface is received. The user selection may be further input related to the definition or configuration of the job. The user selection may be made to a client terminal, which may communicate the user selection to a server over one or more networks. Process 1100 may then proceed to step 1160.

[0256] At step 1160, a determination is made as to whether the configuration of the job is complete. For example, the device may prompt the user to review the assets and confirm the configuration of the job, or may provide selectable elements through which the user can select to improve / modify one or more tasks or one or more features of the tasks. In response to determining that the configuration of the job is not complete, process 1100 returns to step 1130, where the process iteratively configures the user interface based on the previous user input to the user interface or user inputs to one or more previous user interfaces. Otherwise, process 1100 ends.

[0257] In response to determining at step 1130 that a user interface associated with configuring a task is not provided, process 1100 may proceed to step 1145, where the task to be performed is determined (defined) based at least in part on the user selection. Other variables may be used in connection with determining the task, such as historical information (e.g., information for similar tasks), cost of performing the task, etc. (e.g., the task may be defined to minimize cost or reduce the cost of performing the task below a specified threshold cost).

[0258] At step 1150, information regarding the task is communicated. In some embodiments, the server communicates task suggestions to one or more assets. For example, the server may communicate task suggestions to a leader drone of the assets determined to perform the task. The task suggestions may be instructions for the leader drone or the assets performing the task. In some embodiments, the task suggestions may include one or more of task parameters, task constraints, asset suggestions, etc.

[0259] At step 1155, a user interface for performing the work is displayed. In some embodiments, the server causes the client terminal to display a user interface for displaying the work. The user interface for performing the work may provide information about the current state of the work, such as a live video stream, an indication of completed work tasks, work tasks that have not yet been performed, and an indication of currently being performed tasks. In some embodiments, the user interface for performing the work includes information about the assets. For example, the user may select to drill down to view more granular information about the assets, such as the status of the asset (e.g., offline, broken, online, operating normally, partially operating normally, etc.), a list of the capabilities of the asset or assets, the current set of capabilities of the asset or assets, etc. In some embodiments, the user interface for performing the work comprises one or more selectable elements through which the user can enter user input. Examples of user input may include pausing the work, canceling the work, modifying the work (e.g., modifying the parameters of the work), modifying the assets, etc.

[0260] According to various embodiments, the server configures and provides a set of user interfaces, at least some of which are logically connected based at least in part on user input in another user interface. For example, a set of user interfaces may be configured and provided to a user to prompt the user to input information about the job being configured. The information input by the user may include job parameters or characteristics, the type of job, the type of assets utilized, etc. In some embodiments, a user interface in the set of user interfaces used to configure a job is based at least in part on one or more user inputs to a previous user interface in the set of user interfaces.

[0261] FIG. 12A illustrates a method for executing a plan associated with a task, according to various embodiments of the present application. In some embodiments, process 1200 of FIG. 12A is performed by asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1 and / or device 300 of FIG. 3. According to various embodiments, process 1000 is performed by a semi-autonomous drone.

[0262] In step 1205, information about the task is received. In some embodiments, an asset (e.g., a leader drone) obtains information about the task from a server (such as a server configuring the task). The information about the task may include suggestions for assets to perform the task and / or one or more parameters or constraints associated with the task. In some embodiments, the information about the task may include a pre-plan, which may be a high-level plan for the leader drone to perform the task. The high-level plan may have a more granular or less specific definition of the tasks to be performed and / or how the tasks will be performed than the plan determined by the leader drone.

[0263] In step 1210, the asset determines to act as a leader drone based at least in part on the information about the operation.

[0264] In step 1215, the leader drone determines a plan to complete at least one task to perform the work. In some embodiments, the leader drone determines one or more tasks of the work and a plan for performing the one or more tasks. As an example, the leader drone may decompose the work into smaller constituent tasks, and the leader drone may determine a plan for performing the tasks.

[0265] In step 1220, the leader drone determines at least one asset to execute at least a portion of the plan. For example, the leader drone may assign the plan or task to an asset in the group of assets to perform the work.

[0266] In step 1225, the leader drone communicates at least a portion of the plan to at least one asset. For example, the leader drone may communicate at least a portion of the plan to assets assigned to the plan and / or task.

[0267] At step 1230, the leader drone determines whether to communicate the portion of the plan to another asset. In response to determining the other portion of the plan to the assets, process 1200 may proceed to step 1225, where the leader drone repeatedly sends the portion of the plan to different assets, each assigned to a portion of the plan or task. The leader drone may repeatedly send the portion of the plan to different assets until the leader drone determines at step 1230 that the portion of the plan will not be communicated to another asset.

[0268] At step 1235, the leader drone determines whether to determine a plan for another task and whether to transmit the plan to assets, etc. In response to the leader drone determining to determine and transmit a plan for another task, process 1200 proceeds to step 1215, where the leader drone repeatedly determines a plan for the task and transmits at least a portion of the plan to one or more assets until the leader drone determines that no future plans need to be determined or transmitted to assets. In response to determining at step 1235 that a plan for another task does not need to be determined and transmitted, process 1200 proceeds to step 1240.

[0269] At step 1240, the leader drone acquires information during execution of at least a portion of the plan. In some embodiments, the leader drone receives feedback information from one or more follower drones during execution of the corresponding plan. The leader drone may also receive control information related to the operation from the server. For example, the control information may relate to modifications to parameters of the operation, modifications to assets, etc.

[0270] In step 1245, the leader drone transmits information regarding the status of the operation. In some embodiments, the leader drone may provide an update about the status of the operation to the server. For example, in response to receiving feedback information from one or more follower drones, the leader drone may aggregate the feedback from the various follower drones and / or determine the status of the operation. The leader drone may transmit the status of the operation and / or information acquired by one or more drones in the asset fleet (e.g., live stream video, images, current location, current status of execution of the corresponding plan, etc.) to the server. In some embodiments, the leader drone may transmit information regarding the status of the operation to one or more follower drones. For example, the leader drone may provide an update about the status or plan of the operation.

[0271] In step 1250, the leader drone determines whether the work is complete. The leader drone may determine that the work is complete in response to determining that all tasks associated with the work have been completed by the assets. The leader drone may also determine that the work is complete in response to receiving a notification from the server that the work has been canceled or paused. As an example, the work may be canceled or paused in response to a user input to a user interface.

[0272] In response to determining at step 1250 that the task is not complete, process 1200 may proceed to step 1260, where the leader drone determines whether to update the plan. For example, the leader drone may decide to update the plan based on feedback information received from the follower drones and / or control information received from the server, etc. As another example, the leader drone may decide to update the plan based on changes in the task status (e.g., target detection, payload delivery status, changes in environmental factors, etc.). In response to determining that the plan will be updated, process 1200 may proceed to step 1215. In response to determining that the plan will not be updated, process 1200 may proceed to step 1240.

[0273] In response to determining at step 1250 that the work is complete, process 1200 may proceed to step 1255 where status is provided to a server and / or a controlling terminal (eg, a client terminal).

[0274] FIG. 12B illustrates a method for executing a plan associated with an operation in accordance with various embodiments of the present application. In some embodiments, operation 1215 of FIG. 12B is performed by server 105, asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1, and / or device 300 of FIG. 3. Operation 1215 of FIG. 12B may correspond to operation 1215 of FIG. 12A.

[0275] In step 1215a, the leader drone obtains information about the task. In step 1215b, the leader drone determines a location where the plan will be performed. In step 1215c, the leader drone determines a task or element for the follower drone to perform at the location. In step 1215d, the leader drone determines one or more characteristics about the environment of the location. In step 1215e, the leader drone determines a model of the area where at least one task will be performed. According to various embodiments, the model may include a model of the environment in which the assets will operate. For example, the model may be a 2D or 3D model of the world in which the assets are operating or performing the task. In step 1215f, at least a portion of the plan is determined based at least in part on the model.

[0276] According to various embodiments, a planner service implemented on the leader drone may iteratively refine the plan. For example, as the leader drone receives more information (e.g., more specific / detailed information about the task, such as the conditions under which the assets are operating), the leader drone may update its model of the task and the corresponding plan for executing the task based on the current model. As an example, early in the execution of the task, the leader drone may have a simple understanding of the topography of the area in which the assets are operating. As time and the task progresses, the assets can monitor and capture information using various sensors or cameras on the assets, which may be returned as feedback information to the leader drone.

[0277]

[0013] Figure 12C illustrates a method for executing a plan associated with a task according to various embodiments of the present application. In some embodiments, operation 1215 of Figure 12C is performed by server 105, asset 120, asset 125, and / or asset 130 of system 100 of Figure 1, and / or device 300 of Figure 3. Operation 1215 of Figure 12B may correspond to operation 1215 of Figure 12A.

[0278] In step 1215g, the leader drone obtains a high-level predetermined plan (e.g., a pre-plan) for the task. For example, the leader drone may receive the pre-plan from a server. In step 1215h, the leader drone obtains information received from one or more assets regarding the implementation of at least a portion of the plan. As an example, the leader drone may receive feedback information obtained by follower drones during the implementation of the corresponding plan. In step 1215i, the leader drone determines at least one element of the task to be performed by the asset. The leader drone may decompose the task into components and determine the functions required to perform the element (e.g., functions mapped to the element). The leader drone may determine an asset with functions matching the functions required to perform the element and assign the element to the asset. In step 1215j, the leader drone determines a model for the implementation of at least that element of the task. In some embodiments, the leader drone determines a model of the environment in which the asset will perform the element. As an example, if the element is to track a vehicle on a road, the leader drone may generate a model of the environment including the roads, the topology of the area, the relative positions of buildings in the area, the relative positions of trees and other objects in the area, and the position of the vehicle in the area. In step 1215k, the leader drone determines a lower-level plan for the task. The lower-level plan may include more detail or finer granularity than the pre-plan received from the server. For example, the pre-plan for the task may indicate that a vehicle will be tracked within a predetermined area, and the lower-level plan may include elements for implementing the plan, such as (i) tracking the vehicle as it travels on roads in the predetermined area, (ii) staying a predetermined distance away from the vehicle being tracked, (iii) capturing images of the vehicle, (iv) sending updates about the status of the vehicle's tracking at specified time periods, and (v) sending an alert if the vehicle stops at a certain location and providing a suggestion and timestamp for stopping.

[0279] FIG. 13 illustrates a method for executing a plan associated with a task, according to various embodiments of the present application. In some embodiments, process 1300 of FIG. 13 is performed by asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1 and / or device 300 of FIG. 3. According to various embodiments, process 1000 is performed by a semi-autonomous drone. Process 1300 may be performed by a leader drone.

[0280] At step 1305, the drone obtains information regarding a higher-level task or operation to be performed. At step 1310, the drone may obtain information regarding an environment in which at least a portion of the task or operation will be performed. At step 1315, the drone determines a group of assets that will perform at least a portion of the task. At step 1320, it determines whether to split up the group of assets (e.g., a group or team of assets) that will perform a portion of the task.

[0281] In response to the drone determining at step 1320 that the asset group will not split, process 1300 may proceed to step 1325. At step 1325, the drone sends instructions to the asset group to perform at least a portion of the task. At step 1330, the drone receives feedback information. For example, the leader drone may receive feedback information from one or more follower drones. At step 1335, the leader drone determines whether to update the plan. For example, the leader drone may determine whether to update the plan based at least in part on the feedback information.

[0282] In response to determining to update the plan, process 1300 proceeds to step 1340, where the plan is updated. Updating the plan may include updating a model of the environment in which the follower drones are operating and determining whether their ability to perform tasks or elements of tasks has changed based on the model updates or changes in the context of the operation. At step 1345, the leader drone transmits the updated plan. As an example, the leader drone may transmit the updated plan to follower drones assigned to the previous version of the plan (or the task corresponding to the plan). As another example, if the plan update includes assigning new, different, or additional assets to the task, the leader drone may transmit the updated plan to such assets. Process 1300 then returns to step 1330.

[0283] In response to a determination at step 1335 that the plan will not be updated, process 1300 may proceed to step 1350 where the leader drone determines whether the task is complete. In response to a determination that the task is not complete, process 1300 may return to step 1305, and process 1300 may be repeated again. In response to a determination that the task is complete, process 1300 may end.

[0284] In response to determining at step 1320 to partition the assets that perform the portions of the task, process 1300 may proceed to step 1335, where a leader of the partition (e.g., a partition leader) is determined. At step 1360, the leader drone may send an instruction to the partition leader to perform the corresponding portion of the task. Process 1300 may then proceed to step 1330. In response to the partition leader receiving the instruction to perform the corresponding portion of the task, the partition leader may perform process 1300 as the leader drone for the partition.

[0285] Figure 14 illustrates a method for executing a plan associated with an operation, according to various embodiments of the present application. In some embodiments, process 1400 of Figure 14 is performed by asset 120, asset 125, and / or asset 130 of system 100 of Figure 1 and / or device 300 of Figure 3. According to various embodiments, process 1000 is performed by a semi-autonomous drone.

[0286] In operation 1405, a drone receives information about the operation. In some embodiments, the drone receives a plan for performing a portion of the operation (e.g., a task or element of a task). The drone may receive instructions to perform the portion of the operation as a partition that includes a portion of the assets assigned to perform the operation. According to various embodiments, the information about the operation includes an indication that the drone is assigned to be a partition leader for the partition.

[0287] In step 1410, the drone determines to act as a leader drone for the partition (e.g., a partition leader) based at least in part on the information.

[0288] In step 1415, the partition leader determines a plan for completing at least one task of the work. In some embodiments, the partition leader determines one or more plans for completing one or more tasks assigned to the partition. The partition leader may determine the plan based at least in part on one or more of the parameters and / or constraints of the work (or tasks assigned to the partition), the capabilities or features associated with the at least one task, the capabilities of one or more assets in the partition, etc. According to various embodiments, the partition leader implements a planning service similar or the same as the planner service implemented by the leader drone of the group of assets assigned to perform the work.

[0289] In step 1420, the partition leader determines at least one asset to which at least a portion of the plan will be communicated. In some embodiments, the partition leader assigns at least a portion of the plan to an asset within the partition. The partition leader may determine an asset within the partition to which at least a portion of the plan will be assigned. For example, the asset may be determined based on the asset's functionality and / or functionality associated with a task or element corresponding to at least a portion of the plan.

[0290] In step 1425, the partition leader communicates at least a portion of the plan to at least one asset. In some embodiments, in response to determining an asset in the partition to which the portion of the plan is to be assigned, the partition leader sends the portion of the plan to the at least one asset.

[0291] At step 1430, the partition leader determines whether to communicate the portion of the plan to another asset in the partition. In response to determining that another portion of the plan is assigned to the portion of the plan, process 1400 proceeds to step 1420, where the partition leader repeatedly performs steps 1420, 1425, and 1430 until the partition leader determines not to communicate the portion of the plan to another asset.

[0292] At step 1435, the partition leader determines whether to determine a plan for another task and whether to transmit the plan to assets, etc. In response to the partition leader determining to determine and transmit a plan for another task, process 1400 proceeds to step 1415, where the leader drone repeatedly determines a plan for the task and transmits at least a portion of the plan to one or more assets until the partition leader determines that no future plans need to be determined or transmitted to assets. In response to determining at step 1435 that a plan for another task does not need to be determined and transmitted, process 1400 proceeds to step 1440.

[0293] At step 1440, the partition leader obtains information during the execution of at least a portion of the plan. In some embodiments, the partition leader receives feedback information from one or more follower drones (in the partition) during the execution of the corresponding plan. The partition leader may also receive control information regarding the operation from the server (or leaders of assets assigned to the operation). For example, the control information may relate to modifications to parameters of the operation, modifications to assets, etc.

[0294] In step 1445, the partition leader transmits information regarding the status of the work. In some embodiments, the partition leader may provide updates regarding the status of the work to the server. For example, in response to receiving feedback information from one or more follower drones, the partition leader may aggregate the feedback from the various follower drones and / or determine the status of the work. The partition leader may transmit the status of the work (or its tasks) and / or information acquired by one or more drones in the asset group (e.g., live stream video, images, current locations, current status of the implementation of the corresponding plan, etc.) to the server. In some embodiments, the partition leader may transmit information regarding the status of the work to one or more follower drones. For example, the leader drone may provide updates regarding the status or plan of the work. In some embodiments, the partition leader provides information regarding the status of the work to a leader drone (e.g., the leader of the asset group originally assigned to perform the work), which then provides information regarding the status of the work to the server.

[0295] In step 1450, the partition leader determines whether the portion of work corresponding to the partition is complete. The leader drone may determine that the portion of work is complete in response to determining that all tasks associated with the portion of work have been completed by the assets. The partition leader may also determine that the portion of work is complete in response to receiving notification from the server (or leader drone) that the work has been canceled or paused. As an example, the work may be canceled or paused in response to user input to a user interface.

[0296] In response to determining at step 1450 that the task is not complete, process 1440 may proceed to step 1460, where the leader drone determines whether to update the plan. For example, the partition leader may decide to update the plan based on feedback information received from the follower drones and / or control information received from the server, etc. As another example, the partition leader may decide to update the plan based on changes in the task status (e.g., target detection, payload delivery status, changes in environmental factors, etc.). In response to determining that the plan will be updated, process 1400 may proceed to step 1415. In response to determining that the plan will not be updated, process 140 ...16. 4 You can proceed to 40.

[0297] In response to determining at step 1450 that the work is complete, process 1400 may proceed to step 1455 where status is provided to a server and / or a controlling terminal (eg, a client terminal).

[0298] Figure 15 illustrates a method for executing a plan associated with a task, according to various embodiments of the present application. In some embodiments, process 1500 of Figure 15 is performed by asset 120, asset 125, and / or asset 130 of system 100 of Figure 1 and / or device 300 of Figure 3. According to various embodiments, process 1000 is performed by a semi-autonomous drone.

[0299] In operation 1505, the drone receives information about the operation. In some embodiments, the drone receives a plan for performing the operation (e.g., a task or an element of a task). In some embodiments, the drone receives information about the operation from a leader drone of the assets. The information about the operation may include a suggestion of the assets to perform the operation. The information about the operation may include a predetermined leader ranking.

[0300] In step 1510, the drone determines to operate as a dormant leader drone based at least in part on the information about the operation. The drone may determine to operate as a dormant leader based on a predetermined leader ranking. For example, the dormant leader may be the drone with the second highest ranking (e.g., among active assets). In some embodiments, the information about the operation is an instruction for the drone to act as a dormant leader.

[0301] In step 1515, the dormant leader drone communicates at least a portion of the plan with at least one asset. In some embodiments, the dormant leader drone receives information about the plan from the leader drone. The dormant leader drone may synchronize information about the work stored by the leader drone. For example, the dormant leader may have a corresponding shadow copy of the leader drone.

[0302] In step 1520, the dormant leader drone implements the corresponding portion of the plan. In some embodiments, the dormant leader drone performs one or more tasks or elements in connection with performing the work. In some examples, the dormant leader drone does not determine or implement the plan. For example, the dormant leader drone may function as a backup for the leader drone.

[0303] In step 1525, the dormant leader drone acquires information during execution of at least a portion of the plan. In some embodiments, the information acquired during execution of at least a portion of the plan includes feedback information provided by the follower drone, synchronization information provided by the leader drone, and / or information acquired by one or a sensor of the dormant leader drone. In addition to sending feedback information to the leader drone, the follower drone may send feedback information to the dormant leader drone.

[0304] In step 1530, the dormant leader drone determines whether to operate as a leader. In some embodiments, the dormant leader drone determines to operate as a leader in response to determining that the leader drone has failed. As an example, the leader drone may be deemed to have failed if it has lost communication with the dormant leader drone or assets for, e.g., a predetermined threshold period. As another example, the leader drone may be deemed to have failed if the leader drone has a communication link with the assets and the communication link falls below a predetermined quality of service threshold. If the leader drone has not been able to communicate with the dormant leader drone or assets for a predetermined threshold period, the dormant drone or follower drone may send a ping to check the health of the leader drone. If the leader drone fails to respond to the health check within a set period, the leader drone may be deemed to have failed.

[0305] In response to the dormant leader drone determining not to act as leader, process 1500 proceeds to step 1535 where information regarding the status of the operation is sent to the leader drone.

[0306] In step 1540, the dormant leader drone determines whether an indication that the task is complete has been received. was receivedIn response to determining that an indication that the task is complete has not been received, process 1500 ends. Conversely, in response to determining that an indication that the task is complete has not been received, process 1500 proceeds to step 1545, where the dormant leader drone determines whether an updated plan has been received. In response to determining that an updated plan has been received, process 1500 proceeds to step 1520. Conversely, in response to determining that an updated plan has not been received, process 1500 proceeds to step 1525.

[0307] If step 1530 determines that the dormant leader drone will act as the leader, process 1500 proceeds to step 1550, where the dormant leader drone loads information about the job as the leader. In some embodiments, the dormant leader drone loads a synchronized copy of the leader information about the job. For example, the dormant leader drone may load a backup copy of the information about the job that was backed up from a local copy at the leader drone.

[0308] In step 1555, the leader drone (e.g., the previously dormant leader drone) determines a plan for completing at least a portion of the task.

[0309] In step 1560, the leader drone determines which assets to complete at least a portion of the plan.

[0310] In step 1565, the leader drone communicates at least a portion of the plan to the asset.

[0311] At step 1570, the leader drone determines whether to communicate a portion of the plan to another asset. In response to determining that a portion of the plan is to be communicated to another asset, process 1500 returns to step 1560. Conversely, in response to determining that a portion of the plan is not to be communicated to another asset, process 1500 proceeds to step 1570, where the leader drone determines whether another task exists (e.g., a plan is to be determined and assigned) at step 1575. In response to determining that another task exists at step 1575, process 1500 returns to step 1555. In response to determining that another task does not exist at step 1575, process 1500 proceeds to step 1580, where the leader drone obtains information during execution of at least a portion of the plan. For example, the leader drone may receive feedback information from the follower drone.

[0312] In step 1585, the leader drone transmits information about the status of the work. The leader drone may transmit information about the status of the work to a server (e.g., a control center) and / or a client terminal. The leader drone may transmit information about the status of the work to one or more follower drones.

[0313] At step 1590, the leader drone determines whether the task is complete. In response to determining at step 1590 that the task is complete, process 1500 ends. Conversely, in response to determining at step 1590 that the task is not complete, process 1500 proceeds to step 1595, where the leader drone determines whether to update the plan. As an example, a task is determined to be not complete in response to determining that the task is not complete (e.g., if a drone is tasked with scanning a road and the drone is unable to maintain a series of sharp turns in the road and misses scanning a portion of the road, the task is deemed not complete and the drone may turn back and rescan the missed portion of the road). In response to the determination at step 1595 to update the plan, process 1500 proceeds to step 1555. Conversely, in response to determining that the plan is not updated, process 1500 proceeds to step 1580.

[0314] 16A is a diagram illustrating a discrete representation according to various embodiments of the present application. In the example shown in FIG. 16A, the discrete representation 1600 includes a plurality of discrete elements corresponding to voxels. The discrete representation 1600 may be implemented by the system 100 of FIG. 1, the device 300 of FIG. 3, etc.

[0315] As used herein, the term "voxel" corresponds to a value on a regular grid in three-dimensional space, e.g., a voxel corresponds to a discrete element of a discrete representation.

[0316] According to various embodiments, each discrete element (e.g., cell) in the 3-D representation corresponds to a different portion of the geographic location. For example, portions of the geographic location are mapped to corresponding discrete elements in the 3-D representation in connection with determining discrete representation 1600. As shown in FIG. 16A , the 3-D dimensions of discrete representation 1600 correspond to the x-axis, y-axis, and z-axis. The z-axis corresponds to a direction parallel to gravity. As an example, the number of discrete elements provided on the z-axis is determined based at least in part on a maximum altitude or above sea level associated with the configuration of the operation (e.g., parameters entered by a user during configuration of the operation, a maximum altitude capability of an asset in the group of assets selected to perform the operation, a default maximum altitude or above sea level, etc.).

[0317] 16A, sets of information 1610 are each mapped to or associated with a set of discrete elements of discrete representation 1600. In some embodiments, each discrete element in discrete representation 1600 has information mapped to it. For example, a discrete element has one or more metadata fields that each correspond to a different type of information mapped to it. Examples of the type of information that may be mapped to or associated with a discrete element include an indication of whether the discrete element corresponds to a location identified as a keep-in area, an indication of whether the discrete element corresponds to a location identified as a keep-out area, an indication of whether the discrete element is occupied, an indication of whether the discrete element is available or unoccupied, an indication of whether an asset within the discrete element has a clear line of sight (e.g., to the current location of a leader drone, tower, or ground station), an indication of whether a target is included in the discrete element, an identifier of a target included in the discrete element, an indication of whether the discrete element has or is experiencing bad weather, or the type of weather (e.g., sunny, rainy, wind, temperature, wind direction and speed, etc.).

[0318] In general, terrain discretization breaks down as distance increases and the curvature of the Earth becomes significant or important to the calculation. For example, if each discrete element in the 3-D representation corresponds to a volume of the same size, the curvature of the Earth will cause the base of the 3-D representation to correspond to a plane tangent to the Earth at a particular node, and as the 3-D representation extends further in the x or y direction, the 3-D representation will increasingly deviate from the Earth's surface. In some embodiments, the discrete representation 1600 is generated based at least in part on the curvature of the Earth. For example, the system takes the curvature of the Earth into account when generating the discrete representation 1600 in a way that transforms real space into the 3-D representation such that the z-axis corresponds to the direction of gravity (or the direction perpendicular to the Earth's surface at the location corresponding to the discrete element). As another example, the z-axis corresponds to the direction of gravity regardless of how far the 3-D representation is laterally displaced from the origin in the x or y direction.

[0319] According to various embodiments, the discrete representation is used in connection with modeling one or more tasks. For example, the system references (e.g., searches) the discrete representation and associated information (e.g., metadata including fields of geographic location and / or information about one or more tasks). As an example, the system determines a first discrete element corresponding to a first point, determines a second discrete element corresponding to a second point, and connects the first and second discrete elements to determine a path from the first point to the second point through a set of consecutive (e.g., adjacent) discrete elements that the asset can traverse (e.g., the discrete elements are unoccupied, etc.). Consecutive (e.g., adjacent) discrete elements No Se To determine a set, the system searches across the discrete representation to find / determine a set of contiguous assets. In some embodiments, discrete elements that are shown to be unoccupied are searchable in connection with determining a plan (e.g., flight path), and discrete elements that are shown to be occupied are not searchable. As an example, a plan is determined based on the occupancy of one or more discrete elements of the discrete representation.

[0320] An operation (e.g., one or more tasks performed by a fleet of one or more assets) typically extends over distances in at least the x and y directions. Thus, a model of the geographic location corresponding to an operation (e.g., one or more tasks performed by a fleet of one or more assets) includes a large discrete representation and a corresponding large number of discrete elements. According to various embodiments, to aid in rapid processing of the discrete representation 1600 in connection with determining a plan (e.g., a plan for orienting follower drones, asset flight paths, etc.), the system creates a discrete representation that limits the extent to which the discrete elements extend in the z direction. For example, because altitude / above sea level is typically limited according to configured operations (e.g., user input, asset capabilities, and / or default settings), the discrete representation 1600 may be similarly limited to limit the number of cells processed. In some embodiments, the discrete representation 1600 comprises a 400 x 400 x 100 grid segment. As an example, discrete representation 1600 comprises 400 discrete elements in the dimension along the x-direction, 400 discrete elements in the dimension along the y-direction, and 100 discrete elements in the dimension along the z-direction. Discrete representation 1600 of various other dimensions may be implemented. For example, the number of discrete elements in discrete representation 1600 (e.g., in all dimensions or a single dimension) is configurable according to user input, etc.

[0321] In some embodiments, modeling of the geographic location / environment associated with determining a plan for one or more tasks uses a discrete representation of the geographic location environment having the same dimensions, e.g., the same number of discrete elements in one or more directions, regardless of the size of the geographic location (e.g., user-defined geographic location) corresponding to the operation or one or more tasks. Thus, the larger the geographic location, the coarser the granularity of the discrete representation. Conversely, the smaller the geographic location, the finer the granularity of the discrete representation. The resolution of the modeling (e.g., discrete representation) increases as the geographic location being modeled becomes smaller. Thus, in some embodiments, when a leader drone assigns a particular portion of a task / element for the operation and a follower drone is deployed to perform a portion of the task / element, the follower drone generates a model of the geographic location corresponding to the particular portion of the task / element assigned to the follower drone. As an example, a model of geographic location for a particular subset of tasks / elements assigned to a follower drone (e.g., discrete representation 1600, such as the discrete representation generated by the follower drone) has higher resolution than a model of geographic location for the work (e.g., a superset of one or more tasks that a leader drone manages and directs follower drones to perform).

[0322] 16B is a diagram illustrating a discrete representation of a geographic location, according to various embodiments of the present application. In the example shown in FIG. 16B, the discrete representation 1650 is populated with geographic information. For example, the discrete representation 1650 includes representations of trees, roads, cars, hills, and a ground control station 1620. Similarly, the discrete representation 1650 is annotated with one or more parameters related to the geographic location that comprise at least some of the discrete elements. For example, the information set 1610 includes a set of information fields, and the information set 1610 is populated with information related to the geographic location and mapped to the discrete representation 1650. As an example, as shown in FIG. 16B, the set of information 1610 may include (i) information indicating whether one or more discrete elements correspond to a keep-in area, (ii) information indicating whether one or more discrete elements are occupied or unoccupied, (iii) information indicating whether one or more discrete elements correspond to a clear line of sight (e.g., a clear communication / radio line), (iv) information indicating whether one or more discrete elements contain a target, (v) information indicating whether one or more discrete elements are exposed to bad weather, and the like.

[0323] FIG. 16C illustrates a discrete representation of geographic locations, according to various embodiments of the present application. As shown, a group of drones (e.g., D1 1630 and D2 1640, etc.) are deployed at geographic locations corresponding to the discrete representation 1675. As an example, a task may include tracking / monitoring a target located at a discrete element (x=2, y=4, z=0). According to various embodiments, the leader drone D1 1630 provides a plan to the follower drone D2 1640 for tracking / monitoring the target 1670. As an example, the leader drone D1 1630 provides a flight plan 1680 to the follower drone D2 1640. As another example, the leader drone D2 1630 provides suggestions to the follower drone D2 1640 to perform the task of tracking / monitoring the target 1670, and the follower drone D2 1640 determines a flight plan 1680 for moving to a position where the follower drone D2 1640 has line-of-sight to the target.

[0324] In the example shown in FIG. 16C , the target 1670 is not within the line of sight of the ground control station 1620 or the leader drone D1 1630. For example, a hill / mountain is located between the ground control station 1620 and the target 1670. In some embodiments, to perform the task of monitoring the target 1670, the follower drone D2 1640 is deployed to move to a position with line of sight to the target 1670. The system determines a flight plan 1680 for the follower drone D2 1640 to travel along. As shown in FIG. 16C , the flight plan 1680 is configured to direct the follower drone D2 1640 to fly over trees and around the hill. In some embodiments, a cost function associated with moving the drone along a path indicates that moving the drone vertically to an altitude sufficient to fly over the hill is more costly than moving the drone laterally in the x and / or y directions to move around the hill. As an example, moving in the x and / or y directions is more efficient for the drone than moving in the z direction (e.g., increasing altitude). Therefore, longer flight plans that involve travel in the x and / or y directions are more efficient (e.g., less costly) than shorter flight plans that require travel in the z direction.

[0325] 17A is a diagram illustrating a discrete representation of a geographic location in accordance with various embodiments of the present application. In the example shown in FIG. 17A, the discrete representation 1700 includes a plurality of discrete elements corresponding to voxels. The discrete representation 1700 may be implemented by the system 100 of FIG. 1, the device 300 of FIG. 3, etc.

[0326] According to various embodiments, the system repeatedly updates the discrete representation 1700 during the life of the task (e.g., the time until the task is paused, completed, or terminated). As an example, the discrete representation 1700 is generated at the start of the task. When the task is initiated, such as at a ground control station 1710, information about the geographic location and / or one or more tasks associated with the task is added to the discrete representation 1700 and associated set of information 1705 (e.g., metadata associated with one or more discrete elements of the discrete representation 1700). In some embodiments, the discrete representation 1700 is annotated with the set of information 1705.

[0327] According to various embodiments, the information included in set of information 1705 is obtained via one or more pre-stored configurations (e.g., definitions) of operations on a server or local service (e.g., a map or geography service, a service that manages various deployed assets), and / or a third-party service (e.g., a weather service, a map service, etc.).

[0328] 17A, only a portion of the discrete representation 1700 is filled with corresponding information. For example, only the portion 1715 visible from ground control station 1710 is provided. A hill 1720 blocks the view to the feature / information discrete element on the other side of hill 1720.

[0329] 17B is a diagram illustrating a discrete representation of geographic locations, according to various embodiments of the present application. In the example shown in FIG. 17B, a leader drone D1 1730 and a follower drone D2 1735 are deployed. For example, the leader drone D1 1730 and the follower drone D2 1735 are deployed in connection with the commencement of a task or one or more tasks associated with the task. In some embodiments, the leader drone D1 1730 commands the follower drone D2 1740 to move to a position 1745 to observe / monitor an area opposite 1720 (e.g., to identify a target, etc.). A flight plan 1740 for the follower drone D2 1735 is determined, and the follower drone D2 1735 moves along the path of the flight plan 1740 (e.g., to (x=0, y=0, z=3)). In some embodiments, the set of information 1705 associated with the discrete representation 1725 is updated in response to (or while) follower drone D2 1735 moves along the path of the flight plan 1740. For example, follower drone D2 1735 communicates feedback information (e.g., to leader drone D1 1730) regarding one or more tasks and / or geographic location. As shown in FIG. 17B , the discrete representation 1725 is updated to include more information regarding road 1715. Additionally, the set of information 1705 is updated to include information associated with a discrete element located at (x=0, y=0, z=3). In some embodiments, the annotated representation is updated locally in each of follower drone D2 1735 and leader drone D1 1730.

[0330] 17C is a diagram illustrating a discrete representation of a geographic location, according to various embodiments of the present application. As shown in FIG. 17C, as follower drone D2 1735 moves to a discrete location located at (x=0, y=2, z=3), the discrete representation 1750 and / or the set of information 1705 are updated (e.g., the annotated representation is updated). For example, as follower drone D2 1735 moves to (x=0, y=2, z=3), follower drone D2 has line of sight to more roads 1715 and trees 1755. Follower drone D2 1735 updates the annotated representation locally at follower drone D2 1735 and / or communicates feedback information to leader drone D1 1730, which correspondingly updates its locally stored discrete representation 1750 and / or the set of information 1705 (e.g., the annotated representation stored in leader drone D1 1730).

[0331] 17D is a diagram illustrating a discrete representation of a geographic location, according to various embodiments of the present application. As shown in FIG. 17D, as follower drone D2 1735 moves to a discrete location located at (x=0, y=3, z=4), the discrete representation 1775 and / or the set of information 1705 are updated (e.g., the annotated representation is updated). For example, as follower drone D2 1735 moves to (x=0, y=3, z=4), follower drone D2 now has line of sight to more of road 1715, and follower drone D2 1735 identifies target 1780 moving on road 1715. Follower drone D2 1735 updates the annotated representation locally at follower drone D2 1735 and / or communicates feedback information to leader drone D1 1730, which correspondingly updates its locally stored discrete representation 1775 and / or set of information 1705 (e.g., the annotated representation stored in leader drone D1 1730). As an example, the set of information 1705 annotating discrete representation 1775 includes an entry identifying that a discrete element located at (x=2, y=5, z=0) includes a target (e.g., target 1780).

[0332] 18 illustrates a method for determining a plan for performing one or more tasks in accordance with various embodiments of the present application. According to various embodiments, a process 1800 may be implemented in conjunction with the system 100 of FIG. 1 and / or the system 100 of FIG. 30 The process 1800 may be performed at least in part by device 300. In an embodiment, process 1800 is performed in connection with determining or updating a plan. Process 1800 may be performed during the execution of an operation. Process 1800 may be performed by a leader drone and / or a follower drone of a group of assets assigned to perform one or more tasks.

[0333] At step 1810, data associated with the one or more tasks is obtained. According to various embodiments, obtaining the data associated with the one or more tasks includes one or more of step 710 of process 700 of Figure 7A, step 835 of process 835 of Figure 8C, step 931 of process 930 of Figure 9B, step 950 of process 900 of Figure 9A, step 952-1 of process 950 of Figure 9E, step 1010 of process 1000 of Figure 10A, step 1021 of process 1020 of Figure 10B, step 1215a of process 1215 of Figure 12B, step 1305 and / or step 1310 of process 1300 of Figure 13, step 1405 and / or step 1440 of process 1400 of Figure 14, and / or step 1505 and / or step 1525 of process 1500 of Figure 15.

[0334] In some embodiments, the data associated with one or more tasks may include the geographic location where the activity or one or more tasks is performed, and / or one or more Special features The data associated with one or more tasks may be associated with a feature or parameter. As one example, data associated with one or more tasks may be received from a server (e.g., a server providing an operation control service) along with the configuration of the operation (e.g., based on user input, etc.). As another example, data associated with one or more tasks may be received from another asset in the fleet of assets performing the operation. In the case of a leader drone, the leader drone receives data associated with one or more tasks, such as feedback information regarding updates to the status of the execution of one or more tasks, from a follower drone. In the case of a follower drone, the follower drone receives data associated with one or more tasks from the leader drone and / or another asset in the fleet of assets. As another example, data associated with one or more tasks may be received from a third-party service, such as a server providing services / information regarding the operation (e.g., weather services), information regarding other assets deployed in a geographic location, mapping or geographic services, etc.

[0335] At step 1820, a discrete representation of the geographic location is determined. According to various embodiments, the discrete representation corresponds to discrete representation 1600 of FIG. 16A and / or discrete representation 1700 of FIG. 17A. The discrete representation is generated based at least in part on at least a portion of data associated with one or more tasks. In some embodiments, determining the discrete representation includes converting the real-world representation of the geographic location into a predetermined number / dimensions of discrete elements. For example, the volume or amount of space represented by a particular discrete element is based at least in part on the size of the geographic location for which the discrete representation is generated.

[0336] At step 1830, the discrete representation is annotated. In some embodiments, the discrete representation is annotated based at least in part on (e.g., to include) one or more parameters related to the geographic location and / or information related to one or more tasks. As an example, the discrete representation is annotated in response to a determination that one or more parameters relate to the geographic location. In some embodiments, annotating the discrete representation includes setting / adding information about at least some of the discrete elements in the discrete representation (e.g., setting set of information 1610 of FIG. 16A and / or set of information 1705 of FIG. 17A, etc.).

[0337] According to various embodiments, the discrete representation is annotated in connection with creating an annotated representation (e.g., an annotated representation of a geographic location). As one example, annotating the discrete representation includes setting or associating metadata with one or more discrete elements of the discrete representation.

[0338] At step 1840, a plan for performing the one or more tasks is determined. According to various embodiments, the plan for performing the one or more tasks is determined based at least in part on the annotated representation. The determination of the plan is based on the one or more tasks to be performed (e.g., one or more features associated with the one or more tasks) as well as one or more parameters associated with the geographic location. As an example, the one or more parameters associated with the geographic location are included in the annotated representation.

[0339] At step 1850, information regarding the plan is communicated. In some embodiments, the information regarding the plan is transmitted to other assets in the asset fleet. As an example, in the case of a leader drone, the leader drone transmits information regarding the plan to a ground control station and / or one or more follower drones, and the leader drone receives information, such as feedback information, from the ground control station and / or one or more follower drones. As another example, in the case of a follower drone, the follower drone transmits information regarding the plan to the leader drone and / or one or more other assets in the asset fleet performing one or more tasks, and the follower drone receives update information from the leader drone and / or one or more other assets in the asset fleet.

[0340] A determination is made as to whether updated information has been received at step 1860. According to various embodiments, the drone may receive updated information from another asset in the fleet or from a server.

[0341] In some embodiments, the updated information may include the geographic location where the activity or one or more tasks are performed and / or one or more location names associated with one or more of the activities or tasks. Special featuresThe data associated with one or more tasks may be associated with a feature or parameter. As one example, updated information associated with one or more tasks may be received from a server (e.g., a server providing an operation control service) along with changes to the operation (e.g., based on user input, etc.). As another example, data associated with one or more tasks may be received from another asset in the fleet of assets performing the operation. In the case of a leader drone, the leader drone receives updated information from the follower drone, such as feedback information regarding updates to the status of the execution of one or more tasks or information about a portion of the geographic location detected by a sensor or camera of the follower drone. In the case of a follower drone, the follower drone receives data associated with one or more tasks from the leader drone and / or another asset in the fleet of assets. As another example, data associated with one or more tasks may be received from a third-party service, such as a server providing services / information regarding the operation (e.g., a weather service), information about other assets deployed in the geographic location, mapping or geographic services, flight plans for another asset, etc.

[0342] In response to determining at step 1860 that updated information has been received, process 1800 returns to step 1830. In some embodiments, the annotated representation is repeatedly updated with the updated information in response to determining that updated information has been received.

[0343] In response to determining at step 1860 that updated information has not been received, process 1800 proceeds to step 1870 where a determination is made as to whether the process is complete. As one example, process 1800 may be determined to be complete based at least in part on user input (e.g., input to cancel or pause the task). As another example, the process may be determined to be complete in response to a user selecting to end the task. As another example, the process may be determined to be complete in response to completion of the task (e.g., completion of one or more tasks associated with the task). If the process is deemed complete, process 1800 ends. Otherwise, process 1800 returns to step 1860 where the process checks / monitors for updated information.

[0344] 19A is a diagram illustrating a method for determining a flight plan in accordance with various embodiments of the present application. According to various embodiments, a process 1900 may be implemented using the system 100 of FIG. 1 and / or the system 100 of FIG. 30 The process 1900 may be performed at least in part by device 300. In an embodiment, process 1900 is performed in connection with determining or updating a plan (e.g., a flight plan). Process 1900 may be performed during the execution of an operation. Process 1900 may be performed by a leader drone and / or a follower drone of a group of assets assigned to perform one or more tasks.

[0345] At step 1910, data associated with the one or more tasks is obtained. According to various embodiments, obtaining the data associated with the one or more tasks includes one or more of step 710 of process 700 of Figure 7A, step 835 of process 835 of Figure 8C, step 931 of process 930 of Figure 9B, step 950 of process 900 of Figure 9A, step 952-1 of process 950 of Figure 9E, step 1010 of process 1000 of Figure 10A, step 1021 of process 1020 of Figure 10B, step 1215a of process 1215 of Figure 12B, step 1305 and / or step 1310 of process 1300 of Figure 13, step 1405 and / or step 1440 of process 1400 of Figure 14, and / or step 1505 and / or step 1525 of process 1500 of Figure 15. The step of acquiring data along with one or more tasks may be similar to step 1810 of process 1800 of FIG.

[0346] At step 1920, a discrete representation of the geographic location is determined. According to various embodiments, the discrete representation corresponds to discrete representation 1600 of Figure 16A and / or discrete representation 1700 of Figure 17A. In some embodiments, the discrete representation is determined in a manner similar to step 1820 of process 1800 of Figure 18.

[0347] At step 1930, the discrete representation is annotated. In some embodiments, the discrete representation is annotated based at least in part on (e.g., to include) one or more parameters related to the geographic location and / or information related to one or more tasks. As an example, the discrete representation is annotated in response to determining that one or more parameters relate to the geographic location. In some embodiments, annotating the discrete representation includes setting / adding information about at least some of the discrete elements in the discrete representation (e.g., setting set of information 1610 of FIG. 16A and / or set of information 1705 of FIG. 17A, etc.). In some embodiments, the discrete representation is annotated in a manner similar to step 1830 of process 1800 of FIG. 18.

[0348] According to various embodiments, annotating the discrete representation includes registering one or more flight plans associated with another asset in the fleet. For example, in response to receiving a flight plan from a drone in the fleet, the discrete representation is annotated to associate the flight plan with discrete elements of the discrete representation that the flight plan intersects / occupies. As another example, the discrete representation is annotated to set discrete elements affected by the discrete representation (e.g., discrete elements intersected / occupied by the flight plan) to indicate that such discrete elements were occupied for at least the time period associated with the flight plan or until the time the drone associated with the flight plan passed through the particular discrete element. The set of discrete elements affected by a flight plan is updated as the corresponding drone executes the flight plan (e.g., to set discrete elements passed by the drone as unoccupied, to free up discrete elements for use by another asset, etc.).

[0349] At operation 1940, a flight plan for the drone is determined. According to various embodiments, the flight plan is determined based at least in part on annotations in the discrete representation (e.g., the annotated representation). In some embodiments, the drone determines the flight plan based on a geographic location and / or parameters associated with one or more tasks. For example, the drone determines the flight plan based on a current location (e.g., a discrete element corresponding to the current location), a destination location (e.g., a discrete element corresponding to the destination location), and parameters associated with one or more tasks (e.g., a type of task assigned to the drone). As another example, the flight plan is further based on one or more parameters of a set of discrete elements in the discrete representation (e.g., a discrete element between the current location and the destination location, a series of discrete elements along the flight plan connecting the current location and the destination location, etc.).

[0350] In some embodiments, the leader drone determines multiple flight plans, such as plans for different follower drones in the asset fleet, flight plans to avoid each other (e.g., to ensure collision avoidance among the asset fleet), and so on.

[0351] At operation 1950, information associated with the flight plan is communicated. According to various embodiments, in response to determining the flight plan, the flight plan is communicated to one or other assets in the fleet of assets. As an example, the flight plan is published to the other assets in the fleet (e.g., on an information feed / channel for information regarding the flight plan or one or more tasks). The information associated with the flight plan is communicated to the other assets in the fleet (e.g., the leader drone, the follower drone, etc., if the follower drone is communicating the information) to enable the other assets to store such information locally and / or update their respective annotated representations of geographic locations to indicate that the discrete elements corresponding to the flight plan are occupied during the time the flight plan is predicted to affect the corresponding discrete elements, or until the next communication indicates updated information about the drone's flight status and provides clearing of the discrete elements corresponding to the previous portion of the flight plan.

[0352] A determination is made as to whether updated information has been received at step 1960. According to various embodiments, the drone may receive updated information from another asset in the fleet or from a server.

[0353] In some embodiments, the updated information may include the geographic location where the activity or one or more tasks are performed and / or one or more location names associated with one or more of the activities or tasks. Special featuresThe data associated with one or more tasks may be associated with a feature or parameter. As one example, updated information associated with one or more tasks may be received from a server (e.g., a server providing an operation control service) along with changes to the operation (e.g., based on user input, etc.). As another example, data associated with one or more tasks may be received from another asset in the fleet of assets performing the operation. In the case of a leader drone, the leader drone receives updated information from the follower drone, such as feedback information regarding updates to the status of the execution of one or more tasks or information about a portion of the geographic location detected by a sensor or camera of the follower drone. In the case of a follower drone, the follower drone receives data associated with one or more tasks from the leader drone and / or another asset in the fleet of assets. As another example, data associated with one or more tasks may be received from a third-party service, such as a server providing services / information regarding the operation (e.g., a weather service), information about other assets deployed in the geographic location, mapping or geographic services, flight plans for another asset, etc.

[0354] In response to determining at step 1960 that updated information has been received, process 1900 returns to step 1930. In some embodiments, the annotated representation is iteratively updated with the updated information in response to determining that updated information has been received. In response to the iterative updates of the annotated representation, the flight plan is iteratively determined / updated (e.g., a decision is made whether to keep the flight plan the same or whether to update the flight plan, etc.).

[0355] In response to determining at step 1960 that updated information has not been received, process 1900 proceeds to step 1970 where a determination is made as to whether the process is complete. As one example, process 1900 may be determined to be complete based at least in part on user input (e.g., input to cancel or pause the task). As another example, the process may be determined to be complete in response to a user selecting to end the task. As another example, the process may be determined to be complete in response to completion of the task (e.g., completion of one or more tasks associated with the task). If the process is deemed complete, process 1900 ends. Otherwise, process 1900 returns to step 1960 where the process checks / monitors for updated information.

[0356] 19B illustrates a method for determining a flight plan in accordance with various embodiments of the present application. According to various embodiments, process 1940 of FIG. 19B is performed in conjunction with step 1940 of process 1900 of FIG. 19A. According to various embodiments, process 1940 is performed in conjunction with system 100 of FIG. 1 and / or FIG. 30 Process 1940 may be performed at least in part by device 300. In an embodiment, process 1940 is performed in connection with determining or updating a plan (e.g., a flight plan). Process 1940 may be performed during the execution of an operation. Process 1940 may be performed by a leader drone and / or a follower drone of a group of assets assigned to perform one or more tasks.

[0357] In operation 1941, a current location is determined. In some embodiments, the current location is associated with the asset for which the flight plan is being determined. The current location indicates a discrete element of a discrete representation in which the asset is located. In some embodiments, a GPS location of the asset is determined, and a discrete element corresponding to the GPS location is determined. As one example, the current location is determined based at least in part on feedback information received from the asset. As another example, the current location of the asset is determined based at least in part on a locally stored annotated representation of a geographic location. For example, the annotated representation is queried for a discrete element in which the metadata indicates the asset is located.

[0358] In step 1942, a target location (or destination location) is determined. The target location indicates a discrete element of the discrete representation to which the asset will move. The target location is determined at least in part based on a plan or task associated with the item. As an example, if the leader drone determines that the task for the follower drone is to perform surveillance of a road or building, the target location is determined to be a location corresponding to the road or building. As another example, if the leader drone determines that the task for the follower drone is to intercept or monitor a target, the target location is determined at least in part based on the location of the target. For example, the location of the target is determined by querying the annotated representation for discrete elements that include the target to be intercepted / monitored.

[0359] In step 1943, a set of consecutive or adjacent discrete elements from the current location to the target location is determined. The set of consecutive or adjacent discrete elements is determined based at least in part on the annotated representation. For example, the set of consecutive or adjacent discrete elements corresponds to a path along which the asset can travel from the current location to the target location. In some embodiments, the set of consecutive or adjacent discrete elements includes only discrete elements that are not indicated as occupied or are indicated as unoccupied.

[0360] At step 1944, a value of a cost function is determined for the set of contiguous discrete elements determined at step 1943. In some embodiments, a cost associated with moving an asset from a current location to a target location via a flight plan corresponding to the set of contiguous discrete elements is determined. According to various embodiments, the cost function includes one or more variables. In some embodiments, each of the one or more variables has a corresponding weighting (e.g., between 0 and 1, etc.). The variables in the cost function may relate to the amount of effort to move the asset from the current location to the target location, the risk of loss, the time to traverse the flight path, etc. Examples of cost functions include (i) the length of the flight plan, (ii) the extent to which the flight plan includes a vertical climb (e.g., moving the asset to a higher altitude), (iii) the extent to which the flight plan maintains line of sight (e.g., radio line of sight) with the leader drone or other assets or control stations, (iv) the extent to which the flight plan includes adverse weather, (v) the extent to which the flight plan exposes the asset to risk of loss (e.g., risk of loss exceeding a threshold probability of loss), (vi) the time required for the asset to move from a current location to a target location along a flight path of a continuous set of discrete elements, etc. Various other variables may be implemented in conjunction with the cost function.

[0361] In step 1945, a decision is made as to whether to determine another set of contiguous discrete elements.

[0362] In some embodiments, the system iteratively determines a set of consecutive discrete elements until the consecutive discrete elements meet a cost threshold (e.g., a predetermined threshold, a configurable threshold, etc.) For example, the system may utilize a "good enough" determination method to determine a flight plan such that if a flight plan is determined that has a cost less than the cost threshold, process 1940 proceeds to step 1946; otherwise, a further set of consecutive discrete elements is determined.

[0363] In some embodiments, the system determines a predetermined number of sets of contiguous discrete elements from which a select set of contiguous discrete elements is selected to correspond to the flight plan. By way of example, the predetermined number of sets of contiguous discrete elements determined may be configurable, such as by a user or administrator.

[0364] In some embodiments, the system determines the set of consecutive discrete elements over a predetermined period of time, for example, the system may allocate a specific time to determining the set of consecutive discrete elements, and the system may determine the set of consecutive discrete elements until such predetermined period has elapsed.

[0365] In response to a determination at step 1945 that another set of contiguous discrete elements is determined, process 1940 returns to step 1943. Conversely, in response to a determination at step 1945 that another set of contiguous elements is not determined, process 1940 proceeds to step 1946.

[0366] A set of contiguous discrete elements is selected at step 1946. According to various embodiments, the set of contiguous discrete elements is selected from among a set (e.g., multiple sets), such as a set determined based on repeatedly performing steps 1943-1945 of process 1940. In some embodiments, the set of contiguous discrete elements is selected as the set of contiguous discrete elements corresponding to a flight plan for a particular asset (e.g., a current location, a target location, and a selected set of contiguous discrete elements corresponding to the flight plan).

[0367] According to various embodiments, the selected set of contiguous discrete elements is selected based at least in part on the value of a cost function determined for the selected set of contiguous discrete elements. For example, the system determines an optimal set of contiguous discrete elements from among the set of contiguous discrete elements (e.g., determined based on repeatedly performing steps 1943-1945 of process 1940) based at least in part on the cost function. For example, the system selects the set of contiguous discrete elements having the lowest cost. As another example, the system selects multiple flight plans for multiple assets based on an optimal aggregate cost associated with moving the multiple assets from their respective current locations to their respective target locations.

[0368] In step 1947, information about the flight plan is provided. For example, the information about the flight plan includes a selected set of continuous discrete elements.

[0369] At step 1948, a determination is made as to whether the process is complete. As one example, process 1940 may be determined to be complete based at least in part on user input (e.g., input to cancel or pause the operation). As another example, process 1940 may be determined to be complete in response to a user selecting to end the operation. As another example, process 1900 may be determined to be complete in response to a determination that there are no further flight plans (e.g., one or more tasks associated with completing the operation) to be determined. If process 1900 is deemed complete, process 1940 ends. Otherwise, process 1900 returns to step 1941 where another flight plan is determined by repeatedly performing steps 1941-1947.

[0370] Various example embodiments described herein are described with reference to flowcharts. While the examples may include particular steps performed in a particular order, various steps may be performed in different orders according to various embodiments. In some embodiments, some steps may be combined or excluded from the examples discussed herein.

[0371] Although the above embodiments have been described in some detail for ease of understanding, the invention is not limited to the details provided. There are many alternative ways of implementing the invention. The disclosed embodiments are illustrative and are not intended to be limiting. [Application Example 1] A system, a communication interface; one or more processors connected to the communication interface; Equipped with the one or more processors: Retrieve data associated with one or more tasks performed by the assets; the assets include a plurality of drones, the plurality of drones being at least semi-autonomous; the data associated with the one or more tasks includes flight plans for one or more other drones; determining a discrete representation of the geographic location, the discrete representation including a plurality of discrete elements each corresponding to a volume associated with the geographic location; annotating the discrete representation with flight plans of the one or more other drones to create an annotated representation; determining a first flight plan for at least one drone of the plurality of drones, the first flight plan determined based at least in part on the annotated representation; A system configured to communicate information regarding the first flight plan to at least one other asset in the group of assets. [Application Example 2] A system according to Application Example 1, wherein determining the discrete representation of the geographic location includes determining a 3-D representation associated with the geographic location. [Application Example 3] The system according to Application Example 2, determining the 3-D representation associated with the geographic location includes transforming the geographic location from a real-world representation to a 3-D representation based at least in part on a curvature of the Earth; The downward direction in the grid of the 3-D representation corresponds to the direction of gravity in the system. [Application Example 4] The system according to Application Example 1, annotating the discrete representation with flight plans of the one or more other drones to create an annotated representation; determining a set of one or more discrete elements of the flight plans of the one or more other drones that intersect with a second flight plan; setting one or more metadata fields associated with the set of one or more discrete elements to indicate that the corresponding discrete elements are occupied. [Application Example 5] A system as described in Application Example 4, wherein the one or more processors are further configured to update the one or more metadata fields associated with the set of one or more discrete elements to be set as unoccupied based at least in part on (i) the current time and (ii) the time at which a first drone corresponding to the second flight is predicted to intersect with the set of one or more discrete elements. [Application Example 6] A system as described in Application Example 5, wherein the second flight plan indicates the time at which the first drone is predicted to be located at a particular point in time, and the time at which the first drone is predicted to intersect with the set of one or more discrete elements is based at least in part on the second flight plan. [Application Example 7] In the system according to Application Example 4, the one or more processors further receiving an updated position or a second flight plan associated with the first drone; determining whether to update the one or more metadata fields associated with the set of one or more discrete elements based at least in part on the updated position or the second flight plan; in response to a determination to update the one or more metadata fields associated with the one or more sets of discrete elements, updating at least a portion of the one or more metadata fields associated with the one or more sets of discrete elements. [Application Example 8] A system as described in Application Example 1, wherein determining the first flight plan among the at least one flight plan comprises determining a set of continuous discrete elements between the position of the first drone and the target destination of the first drone. [Application Example 9] A system as described in Application Example 1, wherein determining the first flight plan of the at least one flight plan comprises determining a set of continuous discrete elements between the position of the first drone at a first time and the target destination of the first drone at a second time. [Application Example 10] In the system according to Application Example 1, determining the first flight plan among the at least one flight plan includes: determining a plurality of sets of continuous discrete elements between the position of the first drone at a first time and a target destination; determining, from among the plurality of sets of continuous discrete elements, a particular set of continuous discrete elements that is optimal with respect to a cost function. [Application Example 11] A system as described in Application Example 10, wherein the cost function is an index of the cost associated with the at least one drone moving from the position of the first drone at the first time to the target destination. [Application Example 12] The system according to Application Example 10, the cost function is also based at least in part on one or more variables; The one or more variables include one or more of: (i) a change in altitude along a flight path corresponding to a set of continuous discrete elements; (ii) a change in position in a plane perpendicular to the change in altitude along the flight path corresponding to the set of continuous discrete elements; and (iii). [Application Example 13] A system as described in Application Example 12, wherein the one or more variables include the number or percentage of consecutive discrete elements in the set of consecutive discrete elements that have a clear line of sight of communication with a leader drone or ground control service. [Application Example 14] In the system described in Application Example 1, determining the first flight plan of the at least one flight plan comprises determining at least one set of continuous discrete elements between the position of the first drone at a first time and a target destination; the first flight plan corresponds to one set of the at least one set of contiguous discrete elements between the position of the first drone at the first time and the target destination; The first flight plan satisfies a predetermined threshold associated with a line of sight of communication. [Application Example 15] A system as described in Application Example 14, wherein the predetermined threshold associated with communication line of sight includes a requirement that a predetermined number or a predetermined percentage of elements in the set of continuous discrete elements have associated metadata indicating that the corresponding discrete element has clear communication line of sight with a leader drone or ground control service. [Application Example 16] A system as described in Application Example 15, wherein the number or percentage of elements in the set of continuous discrete elements having associated metadata indicating that the corresponding discrete element has a clear line of sight is such that all discrete elements in the set of continuous discrete elements of the first flight plan have the clear line of sight. [Application Example 17] In the system according to Application Example 1, the one or more processors further determining to update said first flight plan; updating the first flight plan to generate an updated first flight plan; In response to determining that the first flight plan has been updated, the system is configured to communicate information regarding the updated first flight plan. [Application Example 18] In the system according to Application Example 1, the one or more processors monitoring communications regarding the one or more tasks from at least one other drone or ground control service in the fleet of assets; determining that the communication regarding the one or more tasks has been received from the at least one other drone ...

Claims

1. 1. A system comprising: a semi-autonomous leader drone equipped with a communications interface; one or more processors connected to the communication interface; Equipped with the one or more processors: receiving an indication via the communications interface that the leader drone is part of a group of assets; obtaining data associated with one or more tasks performed by the assets; the assets include a plurality of drones, the plurality of drones being at least semi-autonomous; the data associated with the one or more tasks includes one or more drone flight plans; determining a discrete representation of the geographic location, the discrete representation including a plurality of discrete elements each corresponding to a volume associated with the geographic location; annotating the discrete representation with the one or more drone flight plans to create an annotated representation; determining a first flight plan for at least one drone of the plurality of drones, the first flight plan being determined based at least in part on the annotated representation; It is structured as follows: determining the first flight plan comprises: determining a plurality of sets of continuous discrete elements between the position of the at least one drone at a first time and a target destination; determining, from among the plurality of sets of the continuous discrete elements, a particular set of continuous discrete elements that is optimal with respect to a cost function based at least in part on the annotated representation and one or more variables, the one or more variables including a degree to which the at least one drone in the first flight plan maintains a clear line of sight with one or more of the leader drone, the target destination, a control station, and other assets in the group of assets, the clear line of sight being a visual line of sight or a communication line of sight; controlling the at least one drone to execute the first flight plan, the at least one drone communicating the first flight plan; Including, the system.

2. 10. The system of claim 1, wherein determining the discrete representation of the geographic location includes determining a 3-D representation associated with the geographic location.

3. 3. The system of claim 2, determining the 3-D representation associated with the geographic location includes transforming the geographic location from a real-world representation to a 3-D representation based at least in part on the curvature of the Earth; The system wherein the downward direction in the grid of the 3-D representation corresponds to the direction of gravity.

4. 10. The system of claim 1, annotating the discrete representation with the one or more drone flight plans to create an annotated representation includes: determining a set of one or more discrete elements that intersect with a second one of the one or more drone flight plans; setting one or more metadata fields associated with the set of one or more discrete elements to indicate that the corresponding discrete elements are occupied.

5. 5. The system of claim 4, wherein the one or more processors are further configured to update the one or more metadata fields associated with the set of one or more discrete elements to be set as unoccupied based at least in part on (i) a current time and (ii) a time when a first drone corresponding to the second flight plan is predicted to intersect with the set of one or more discrete elements.

6. 1. A system comprising: a communication interface; one or more processors connected to the communication interface; Equipped with the one or more processors: Obtaining data associated with one or more tasks performed by the assets; the assets include a plurality of drones, the plurality of drones being at least semi-autonomous; the data associated with the one or more tasks includes flight plans for one or more other drones; determining a discrete representation of the geographic location, the discrete representation including a plurality of discrete elements each corresponding to a volume associated with the geographic location; annotating the discrete representation with flight plans of the one or more other drones to create an annotated representation; determining a first flight plan for at least one drone of the plurality of drones, the first flight plan determined based at least in part on the annotated representation; configured to communicate information regarding the first flight plan to at least one other asset in the fleet; annotating the discrete representation with flight plans of the one or more other drones to create an annotated representation includes: determining a set of one or more discrete elements of the flight plans of the one or more other drones that intersect with a second flight plan; setting one or more metadata fields associated with the set of one or more discrete elements to indicate that the corresponding discrete elements are occupied; The one or more processors are further configured to update the one or more metadata fields associated with the set of one or more discrete elements to be set as unoccupied based at least in part on (i) a current time and (ii) a time when a first drone corresponding to the second flight plan is predicted to intersect with the set of one or more discrete elements; The second flight plan indicates a time when the first drone is predicted to be located at a particular point in time, and the time when the first drone is predicted to intersect with the set of one or more discrete elements is based at least in part on the second flight plan.

7. 1. A system comprising: a communication interface; one or more processors connected to the communication interface; Equipped with the one or more processors: Obtaining data associated with one or more tasks performed by the assets; the assets include a plurality of drones, the plurality of drones being at least semi-autonomous; the data associated with the one or more tasks includes flight plans for one or more other drones; determining a discrete representation of the geographic location, the discrete representation including a plurality of discrete elements each corresponding to a volume associated with the geographic location; annotating the discrete representation with flight plans of the one or more other drones to create an annotated representation; determining a first flight plan for at least one drone of the plurality of drones, the first flight plan determined based at least in part on the annotated representation; configured to communicate information regarding the first flight plan to at least one other asset in the fleet; annotating the discrete representation with flight plans of the one or more other drones to create an annotated representation includes: determining a set of one or more discrete elements of the flight plans of the one or more other drones that intersect with a second flight plan; setting one or more metadata fields associated with the set of one or more discrete elements to indicate that the corresponding discrete elements are occupied; The one or more processors further receiving an updated position or a second flight plan associated with the first drone; determining whether to update the one or more metadata fields associated with the set of one or more discrete elements based at least in part on the updated position or the second flight plan; in response to a determination to update the one or more metadata fields associated with the set of one or more discrete elements, updating at least a portion of the one or more metadata fields associated with the set of one or more discrete elements.

8. 2. The system of claim 1, wherein determining the first flight plan of the at least one flight plan comprises determining a set of contiguous discrete elements between a position of a first drone at a first time and a target destination of the first drone at a second time.

9. 2. The system of claim 1, wherein determining the first flight plan of the at least one flight plan comprises: determining a plurality of sets of continuous discrete elements between the position of the first drone at a first time and the target destination; determining, from among the plurality of sets of continuous discrete elements, a particular set of continuous discrete elements that is optimal with respect to a cost function.

10. 10. The system of claim 9, wherein the cost function is an indication of a cost associated with the at least one drone traveling from the position of the first drone at the first time to the target destination.

11. 10. The system of claim 9, the cost function is also based at least in part on one or more variables; The one or more variables include one or more of: (i) a change in altitude along a flight path corresponding to a set of continuous discrete elements; (ii) a change in position in a plane perpendicular to the change in altitude along the flight path corresponding to the set of continuous discrete elements; (iii) the extent to which the flight plan includes adverse weather; (iv) the extent to which the flight plan exposes an asset to a risk of loss that exceeds a threshold probability of loss; and (v) the time required for an asset to move from a current location to a target location along a flight path of a set of continuous discrete elements.

12. 1. A system comprising: a communication interface; one or more processors connected to the communication interface; Equipped with the one or more processors: Obtaining data associated with one or more tasks performed by the assets; the assets include a plurality of drones, the plurality of drones being at least semi-autonomous; the data associated with the one or more tasks includes flight plans for one or more other drones; determining a discrete representation of the geographic location, the discrete representation including a plurality of discrete elements each corresponding to a volume associated with the geographic location; annotating the discrete representation with flight plans of the one or more other drones to create an annotated representation; determining a first flight plan for at least one drone of the plurality of drones, the first flight plan determined based at least in part on the annotated representation; configured to communicate information regarding the first flight plan to at least one other asset in the fleet; Determining the first flight plan of the at least one flight plan includes: determining a plurality of sets of continuous discrete elements between the position of the first drone at a first time and the target destination; determining, from among the plurality of sets of continuous discrete elements, a particular set of continuous discrete elements that is optimal with respect to a cost function; the cost function is also based at least in part on one or more variables; The one or more variables include one or more of: (i) a change in altitude along a flight path corresponding to a set of continuous discrete elements; (ii) a change in position in a plane perpendicular to the change in altitude along the flight path corresponding to the set of continuous discrete elements; (iii) the extent to which the flight plan includes adverse weather; (iv) the extent to which the flight plan exposes the asset to a risk of loss that exceeds a threshold probability of loss; and (v) the time required for the asset to move from a current location to a target location along a flight path of a set of continuous discrete elements. wherein the one or more variables include a number or percentage of the contiguous discrete elements in the set of contiguous discrete elements that have a clear line of sight of communication with a leader drone or ground control service.

13. 2. The system of claim 1, wherein determining the first flight plan of the at least one flight plan comprises determining at least one set of continuous discrete elements between a position of a first drone at a first time and a target destination; the first flight plan corresponds to one set of the at least one set of contiguous discrete elements between the position of the first drone at the first time and the target destination; The system, wherein the first flight plan satisfies a predetermined threshold associated with communication line of sight.

14. 1. A system comprising: a communication interface; one or more processors connected to the communication interface; Equipped with the one or more processors: Obtaining data associated with one or more tasks performed by the assets; the assets include a plurality of drones, the plurality of drones being at least semi-autonomous; the data associated with the one or more tasks includes flight plans for one or more other drones; determining a discrete representation of the geographic location, the discrete representation including a plurality of discrete elements each corresponding to a volume associated with the geographic location; annotating the discrete representation with flight plans of the one or more other drones to create an annotated representation; determining a first flight plan for at least one drone of the plurality of drones, the first flight plan determined based at least in part on the annotated representation; configured to communicate information regarding the first flight plan to at least one other asset in the fleet; Determining the first flight plan of the at least one flight plan comprises determining at least one set of continuous discrete elements between a position of the first drone at a first time and a target destination; the first flight plan corresponds to one set of the at least one set of contiguous discrete elements between the position of the first drone at the first time and the target destination; the first flight plan satisfies a predetermined threshold associated with a line of sight of communication; the predetermined threshold associated with communication line of sight includes a requirement that a predetermined number or a predetermined percentage of elements in the set of contiguous discrete elements have associated metadata indicating that the corresponding discrete element has a clear communication line of sight with a leader drone or ground control service.

15. 15. The system of claim 14, wherein the number or percentage of elements in the set of contiguous discrete elements having the associated metadata indicating the corresponding discrete element has a clear line of sight of communications is that all discrete elements in the set of contiguous discrete elements of the first flight plan have the clear line of sight of communications.

16. 10. The system of claim 1, wherein the one or more processors further comprise: determining to update the first flight plan; updating the first flight plan to generate an updated first flight plan; In response to determining that the first flight plan has been updated, the system is configured to communicate information regarding the updated first flight plan.

17. 1. A system comprising: a communication interface; one or more processors connected to the communication interface; Equipped with the one or more processors: Obtaining data associated with one or more tasks performed by the assets; the assets include a plurality of drones, the plurality of drones being at least semi-autonomous; the data associated with the one or more tasks includes flight plans for one or more other drones; determining a discrete representation of the geographic location, the discrete representation including a plurality of discrete elements each corresponding to a volume associated with the geographic location; annotating the discrete representation with flight plans of the one or more other drones to create an annotated representation; determining a first flight plan for at least one drone of the plurality of drones, the first flight plan determined based at least in part on the annotated representation; configured to communicate information regarding the first flight plan to at least one other asset in the fleet; the one or more processors: monitoring communications regarding the one or more tasks from at least one other drone or ground control service in the fleet of assets; determining that the communication regarding the one or more tasks has been received from the at least one other drone in a fleet of assets or the ground control service; determining whether to update a plan for executing at least one element of the one or more tasks in response to receiving a communication regarding the one or more tasks from the at least one other drone in the fleet of assets or the ground control service; responsive to determining to update the plan for performing at least one element of the one or more tasks, updating the plan to generate an updated plan; A system configured to cause the updated plan to be implemented in connection with performing the elements of the one or more tasks.

18. 20. The system of claim 17, wherein the communication regarding the one or more tasks from the at least one other drone in the fleet of assets or the ground control service includes information regarding one or more of terrain, weather, targets, and characteristics of discrete elements of the discrete representation of the geographic location.

19. 1. A method comprising: Obtaining data associated with one or more tasks performed by the assets; the assets include a plurality of drones, the plurality of drones being at least semi-autonomous, the plurality of drones including a leader drone; the data associated with the one or more tasks includes one or more drone flight plans; determining a discrete representation of the geographic location, the discrete representation including a plurality of discrete elements each corresponding to a volume associated with the geographic location; annotating the discrete representation with the one or more drone flight plans to create an annotated representation; determining a first flight plan for at least one drone of the plurality of drones, the first flight plan determined based at least in part on the annotated representation; determining the first flight plan comprises: determining a plurality of sets of continuous discrete elements between the position of the at least one drone at a first time and a target destination; determining, from among the plurality of sets of the continuous discrete elements, a particular set of continuous discrete elements that is optimal with respect to a cost function based at least in part on the annotated representation and one or more variables, the one or more variables including a degree to which the at least one drone in the first flight plan maintains a clear line of sight with one or more of the leader drone, the target destination, a control station, and other assets in the group of assets, the clear line of sight being a visual line of sight or a communication line of sight; controlling the at least one drone to execute the first flight plan, the at least one drone communicating the first flight plan; A method comprising:

20. A computer program product embodied in a non-transitory computer-readable medium, computer instructions for receiving, via a communication interface of a semi-autonomous leader drone, an indication that the leader drone is part of a fleet of assets; computer instructions for obtaining data associated with one or more tasks performed by the assets; the assets include a plurality of drones, the plurality of drones being at least semi-autonomous; the data associated with the one or more tasks includes one or more drone flight plans; computer instructions for determining a discrete representation of a geographic location, the discrete representation including a plurality of discrete elements each corresponding to a volume associated with the geographic location; computer instructions for annotating the discrete representation with the one or more drone flight plans to create an annotated representation; computer instructions for determining a first flight plan for at least one drone of the plurality of drones; Equipped with the first flight plan is determined based at least in part on the annotated representation; determining the first flight plan comprises: determining a plurality of sets of continuous discrete elements between the position of the at least one drone at a first time and a target destination; determining, from among the plurality of sets of the continuous discrete elements, a particular set of continuous discrete elements that is optimal with respect to a cost function based at least in part on the annotated representation and one or more variables, the one or more variables including a degree to which the at least one drone in the first flight plan maintains a clear line of sight with one or more of the leader drone, the target destination, a control station, and other assets in the group of assets, the clear line of sight being a visual line of sight or a communication line of sight; controlling the at least one drone to execute the first flight plan, the at least one drone communicating the first flight plan; a computer program product,

Images