Planning work for asset groups

Abstract

Coordinate multiple assets, including one or more drones, in relation to the execution of a task. [Solution] The system for planning work receives a suggestion that a drone is part of an asset group, the asset group is tasked with performing one or more elements of one or more tasks, the asset group includes multiple drones, communicates information about one or more elements, 1) the information about one or more elements is communicated to one or more other drones, 2) the information about one or more elements is at least partially based on information acquired by one or more sensors of the asset group, and 3) the information about one or more elements is used in connection with determining a plan for performing one or more tasks, communicates information about the plan for performing one or more tasks, and communicates information about the plan for performing one or more tasks to at least one other drone.

Description

Background Art

[0006] , , , , , , , , [Figure 3] , [Figure 4B] ,

[0007]

[0001] Conventional unmanned aerial vehicles (UAVs) or drones are useful for performing many tasks. Such drones can perform surveillance, commercial cargo or weapon delivery, mapping of remote or inaccessible areas, and / or other missions. Although useful, such drones have many drawbacks. For example, drones are typically remotely controlled, may lack reliability, may be slower than desired, may have a limited range, and / or may have other problems that negatively affect performance. Furthermore, the coordination of multiple assets including one or more drones is complex and generally requires a user at a control center to control and coordinate the assets in relation to the execution of the task.

Brief Description of the Drawings

[0002] In the following detailed description and the accompanying drawings, various embodiments of the present invention are disclosed.

[0003] [Figure 1] A diagram showing a system for performing a task according to various embodiments of the present application. <​​​​​​​​​​​​​​​​​​​​A diagram showing a system for performing at least a portion of the work according to various embodiments of the present application.

[0008] [Figure 4C] A diagram showing a system for performing at least a portion of the work according to various embodiments of the present application.

[0009] [Figure 5A] A diagram showing a system for performing at least a portion of the work according to various embodiments of the present application.

[0010] [Figure 5B] A diagram showing a system for performing at least a portion of the work according to various embodiments of the present application.

[0011] [Figure 5C] A diagram showing a system for performing at least a portion of the work according to various embodiments of the present application.

[0012] [Figure 6] A diagram showing a user interface for configuring, monitoring, and / or controlling operations according to various embodiments of the present application.

[0013] [Figure 7A] A diagram illustrating a method for configuring the work according to various embodiments of the present application.

[0014] [Figure 7B] A diagram illustrating a method for configuring the work according to various embodiments of the present application.

[0015] [Figure 8A] A diagram illustrating a method for performing at least one task of the work according to various embodiments of this specification.

[0016] [Figure 8B]A diagram showing a method for performing at least one task of an operation according to various embodiments of this specification.

[0017] [Figure 8C] A diagram showing a method for performing at least one task of an operation according to various embodiments of this specification.

[0018] [Figure 9A] A diagram showing a method for implementing a plan associated with an operation according to various embodiments of this application.

[0019] [Figure 9B] A diagram showing a method for implementing a plan associated with an operation according to various embodiments of this application.

[0020] [Figure 9C] A diagram showing a method for implementing a plan associated with an operation according to various embodiments of this application.

[0021] [Figure 9D] A diagram showing a method for implementing a plan associated with an operation according to various embodiments of this application.

[0022] [Figure 9E] A diagram showing a method for implementing a plan associated with an operation according to various embodiments of this application.

[0023] [Figure 10A] A diagram showing a method for implementing a plan associated with an operation according to various embodiments of this application.

[0024] [Figure 10B] A diagram showing a method for implementing a plan associated with an operation according to various embodiments of this application.

[0025] [Figure 11]A diagram illustrating a method for configuring the work according to various embodiments of the present application.

[0026] [Figure 12A] A diagram illustrating a method for carrying out a plan associated with work, according to various embodiments of the present application.

[0027] [Figure 12B] A diagram illustrating a method for carrying out a plan associated with work, according to various embodiments of the present application.

[0028] [Figure 12C] A diagram illustrating a method for carrying out a plan associated with work, according to various embodiments of the present application.

[0029] [Figure 13] A diagram illustrating a method for carrying out a plan associated with work, according to various embodiments of the present application.

[0030] [Figure 14] A diagram illustrating a method for carrying out a plan associated with work, according to various embodiments of the present application.

[0031] [Figure 15] A diagram illustrating a method for carrying out a plan associated with work, according to various embodiments of the present application.

[0032] [Figure 16A] A diagram showing discrete representations according to various embodiments of the present application.

[0033] [Figure 16B] A diagram showing discrete representations of geographical locations according to various embodiments of the present application.

[0034] [Figure 16C] A diagram showing discrete representations of geographical locations according to various embodiments of the present application.

[0035] [Figure 17A] A diagram showing discrete representations of geographical locations according to various embodiments of the present application.

[0036] [Figure 17B] A diagram showing discrete representations of geographical locations according to various embodiments of the present application.

[0037] [Figure 17C] A diagram showing discrete representations of geographical locations according to various embodiments of the present application.

[0038] [Figure 17D] A diagram showing discrete representations of geographical locations according to various embodiments of the present application.

[0039] [Figure 18] A diagram illustrating a method for determining a plan for performing one or more tasks according to various embodiments of the present application.

[0040] [Figure 19A] A diagram illustrating a method for determining a flight plan according to various embodiments of the present invention.

[0041] [Figure 19B] A diagram illustrating a method for determining a flight plan according to various embodiments of the present invention. [Modes for carrying out the invention]

[0042] The present invention can be implemented in various forms, including processes, apparatus, systems, compositions of materials, computer program products embodied on computer-readable storage media, and / or processors (processors configured to execute instructions stored and / or provided by memory connected to the processor). In this specification, these embodiments or any other forms the invention may take may be referred to as "technologies." Generally, the order of the processes of the disclosed processes may be modified within the scope of the invention. Unless otherwise specified, components such as processors or memory described as configured to perform a task may be implemented as general components temporarily configured to perform a task at a given time, or as specific components manufactured to perform a task. As used herein, the term "processor" refers to one or more devices, circuits, and / or processing cores configured to process data such as computer program instructions.

[0043] The following provides a detailed description of one or more embodiments of the present invention, with reference to drawings illustrating the principles of the present invention. While the present invention is described in relation to such embodiments, it is not limited to any of these embodiments. The scope of the present invention is limited only by the claims, and the present invention includes many substitutes, variations, and equivalents. The following description includes many specific details to provide a complete understanding of the present invention. These details are illustrative, and the present invention can be implemented in accordance with the claims without some or all of these specific details. For simplicity, technical matters well known in the art related to the present invention are not described in detail, so as not to complicate the present invention unnecessarily.

[0044] Systems for grouping assets are disclosed according to various embodiments. The system may comprise a communication interface and one or more processors connected to the communication interface. The one or more processors may be configured to (i) receive data via the communication interface associated with one or more tasks performed by a group of assets, the group of assets comprising a plurality of drones, the drones being at least semi-autonomous, (ii) determine from the plurality of assets which group of assets will perform one or more tasks, and (iii) communicate a command via the communication interface to at least one drone in the group of assets, the command instructing that one or more tasks be at least partially completed by at least one drone. Determining a group of assets to perform one or more tasks may include determining one or more functions associated with one or more features of the one or more tasks, and determining a plurality of drones at least in part on (i) one or more functions associated with one or more features of the one or more tasks, and (ii) one or more drone functions associated with each of the plurality of drones.

[0045] Drones are disclosed according to various embodiments. A drone may be a semi-autonomous drone. A drone may comprise a communication interface and one or more processors connected to the communication interface. One or more processors may be configured to (i) receive via the communication interface an indication that a drone is part of an asset group, the asset group being tasked with performing one or more elements of one or more tasks, the asset group comprising multiple drones, (ii) communicate information about one or more elements via the communication interface, and (iii) communicate information about a plan for performing one or more tasks via the communication interface, the information about the plan for performing one or more tasks being communicated to at least one other drone in the asset group. Information about one or more elements may be communicated to at least one other drone in the asset group. Information about one or more elements may be at least in part based on information obtained by one or more sensors of the asset group. Information about one or more elements may be used in connection with determining a plan for performing one or more tasks.

[0046] Drones are disclosed according to various embodiments. A drone may be a semi-autonomous drone. A drone may comprise a communication interface and one or more processors connected to the communication interface. One or more processors may be configured to (i) receive via the communication interface an indication that a drone is part of an asset group, the asset group being tasked with performing one or more elements of one or more tasks, the asset group comprising multiple drones, (ii) determine that at least one drone in the asset group has experienced a failure, (iii) update the plan to perform one or more tasks in response to the determination that at least one drone has experienced a failure, and (iv) communicate information about the updated plan via the communication interface, the information about the updated plan being communicated to at least one remaining drone in the asset group. One or more processors may further be configured to communicate via the communication interface information acquired during the execution of at least one element of the updated plan, the information about the updated plan being communicated to at least one remaining asset of a set of drones.

[0047] A system for dynamically grouping assets is disclosed according to various embodiments. The system may comprise a communication interface and one or more processors connected to the communication interface. The one or more processors may be configured to (i) acquire data associated with one or more tasks performed by a group of assets, the group of assets comprising a plurality of drones, the plurality of drones being at least semi-autonomous, (ii) decide to modify the group of assets based at least in part on the data associated with one or more tasks, and (iii) communicate a command to at least one of the drones in the group of assets via the communication interface, the command instructing the group of assets to modify. Deciding to modify the group of assets based at least in part on the data associated with one or more tasks may include determining one or more functions associated with the remaining one or more tasks, and deciding to modify the group of assets based at least in part on (i) one or more functions associated with one or more remaining tasks, and (ii) one or more drone functions associated with each of the assets in the set.

[0048] Systems are disclosed according to various embodiments. A system may comprise a communication interface and one or more processors connected to the communication interface. One or more processors may be configured to (i) display a first user interface, the first user interface comprising one or more selectable elements associated with features of one or more tasks to be performed; (ii) receive one or more user selections regarding features of one or more tasks to be performed via the first user interface; (iii) display a second user interface in response to receiving one or more user selections entered into the first interface; (iv) receive one or more user selections regarding a set of one or more assets to be deployed to perform work via the second user interface; (v) determine a work to be performed, the work being determined at least in part on (a) one or more user selections regarding features of one or more tasks to be performed and (b) one or more user selections regarding a set of one or more assets to be deployed to perform work; and (vi) communicate information about the work via the communication interface. A second user interface may be configured at least in part based on at least one of the one or more user selections entered into the user interface. The second user interface may include one or more selectable elements associated with a group of one or more assets that perform a task, where the group of one or more assets includes one or more drones, and the one or more drones are semi-autonomous. Information about the task is communicated to at least one of the drones in the asset group. The information about the task causes the asset group to deploy to perform at least part of the task.

[0049] The system is disclosed according to various embodiments. The system may comprise a communication interface and one or more processors connected to the communication interface. The one or more processors may be configured to (i) acquire information associated with one or more tasks performed by a group of assets, (ii) determine a discrete representation of a geographic area, the discrete representation comprising a plurality of discrete elements, each corresponding to a volume within the geographic area, (iii) annotate the discrete representation to create an annotated representation of the geographic area comprising one or more parameters, comprising at least a subset of the plurality of discrete elements, at least based on a determination that one or more parameters relate to the geographic area, and (iv) determine a plan for performing one or more tasks, the plan being at least partially based on the annotated representation, and (v) causing one or more tasks to be performed at least partially based on the plan. The group of assets may comprise a plurality of drones, the plurality of drones being at least semi-autonomous. The data associated with one or more tasks may comprise one or more parameters relating to a geographic area in which at least one asset in the group of assets performs one or more tasks.

[0050] The system is disclosed according to various embodiments. The system may comprise a communication interface and one or more processors connected to the communication interface. The one or more processors may be configured to (i) retrieve data associated with one or more tasks performed by a group of assets, (ii) determine a discrete representation of a geographic area, where the discrete representation comprises a plurality of discrete elements, each corresponding to a volume within the geographic area, (iii) annotate the discrete representation with flight plans of one or more other drones to create an annotated representation, (iv) determine a first flight plan for at least one of the drones, where the first flight plan is determined at least in part on the annotated representation, and (v) communicate information about the first flight plan to at least one other asset in the group of assets. The group of assets comprises a plurality of drones, which may be at least semi-autonomous. The data associated with one or more tasks may include flight plans of one or more other drones.

[0051] According to various embodiments, a system receives high-level instructions for performing a task using a swarm of drones. These high-level instructions may include a high-level description or definition of the task. Upon receiving the high-level instructions, the system may determine one or more low-level instructions related to performing the task using the drone swarm. For example, a semi-autonomous drone, such as a leader drone, may break down the high-level instructions into one or more tasks for the drone swarm, and the semi-autonomous drone may communicate one or more tasks (or a plan for completing one or more tasks or elements of tasks) to one or more drones in the swarm (e.g., follower drones). In some embodiments, a user or control system provides a high-level definition of the task, and one or more semi-autonomous drones autonomously determine the tasks or elements of tasks associated with the task. The drone swarm (e.g., including one or more semi-autonomous drones) then performs the tasks or elements of tasks associated with the task.

[0052] As used herein, “semi-autonomous drone” means a drone that, without human intervention, determines a plan to perform at least part of a task or operation based on high-level instructions for doing so, receives feedback information about the status of the drone and / or the plan, and updates or determines a new plan in response to the feedback information. The updated or new plan may provide instructions for controlling the drone in a manner consistent with the high-level instructions or operation. High-level instructions may be provided to the drone by a user interface or by a server via a network connection. In response to receiving high-level instructions, the drone determines a plan to perform part of the task or operation, executes the plan, or instructs another asset to provide feedback to the drone by delivering part of the task or operation. The term “semi-autonomous drone” may be used interchangeably with “drone” herein. For example, the task “scan polygons and track moving targets” may involve one or more drones planning how to effectively scan polygons, but a new target may appear in the middle of the scan pattern, and the semi-autonomous drone replans and begins tracking the new target.

[0053] High-level instructions may include one or more of the following: an indication of the type of work, the location where the work will be performed, the target of the work, the time of day when the work will be performed, etc. Low-level instructions may provide more specific information or definitions about the work. In some embodiments, low-level instructions may indicate the specific asset that will perform the task within the work, and how the task will be performed (e.g., speed, altitude, type of sensor, etc.). Low-level instructions may further be determined based on environmental or situational information within the area where the work will be performed. An example of a high-level instruction might be an instruction to perform the task of monitoring a specific road on a specific date. In contrast, the corresponding low-level instruction may include an instruction to an asset in a group of assets indicating a specific location (or range of locations) along the road where the specific asset will capture information. A leader drone may determine which follower drones will perform the low-level instructions. Low-level instructions may indicate the type of information to be captured, the speed at which the asset will move, and the altitude at which the asset will move or monitor. As another example, a high-level instruction might include "observe this target from several good vantage points," and the corresponding low-level instruction might be "fly drone 1 at a 30-degree angle from a distance of 100 meters from the target, and fly drone 2 at a 180-degree angle from a distance of 150 meters."

[0054] As used herein, “Asset” can correspond to a device, terminal, vehicle, and / or system capable of carrying out a plan to perform a task or an element of a task, such as receiving information from a server (e.g., directly or via another asset), using sensors to acquire information, communicating the collected information to a leader drone / asset or server, and transporting a payload. Examples of assets include, but are not limited to, drones, satellites, communication towers, observation stations, vehicles such as autonomous or semi-autonomous vehicles (e.g., airplanes, boats, cars, trucks, helicopters, etc.), drones (e.g., large drones, small drones, etc.), jet aircraft, and small autonomous sensors (e.g., sensors, “dust” sensors, etc.). An asset has an associated identifier and a set of defined features, functions, characteristics, etc. (collectively referred to herein as “functions”). The set of defined functions may represent a set of functions that the asset can perform. For example, a set of functions may indicate a set of peripherals (such as cameras, sensors, payloads, payload deployment mechanisms, and pre-installed software) that are provided with or otherwise operablely connected to the asset. As another example, a defined set of functions may indicate whether the asset has a turret (e.g., a multi-axis pointing mechanism to which other functions can be attached, e.g., a pan-tilt unit for a surveillance camera). As yet another example, a defined set of functions may indicate whether the asset can send and receive data from other communication sources, such as enabling / disabling transmissions.

[0055] As used herein, “planning information” may correspond to information about a plan for performing a task, element, and / or work. For example, planning information may include parameters for a task, element, and / or work. For example, planning information may include instructions for operating an asset.

[0056] As used herein, “Functional Information” may refer to information about the functions of an asset (such as a drone). For example, functional information may include suggestions of the functions and / or set of functions of a drone.

[0057] As used herein, “sensor information” may correspond to information acquired by a sensor (such as an asset’s sensor). Sensor information may be processed locally by the asset or device, for example, by converting raw sensor input into a predetermined format.

[0058] As used herein, “state information” may correspond to information about the status or state of a device, such as an asset (e.g., a drone). For example, state information may include information indicating the state of a particular function or set of functions of a drone. For example, state information may include information indicating the state of one or more tasks (e.g., the extent to which an asset has completed a task). For example, state information may include information about the results of tasks, elements, and / or work execution.

[0059] Various embodiments include a system for configuring work performed by a group of assets, such as semi-autonomous drones. The system may include one or more servers, client terminals, and / or a group of one or more assets. The servers, client terminals, and a group of one or more assets may communicate through one or more assets. The client terminals may provide a user interface through which the work can be configured (e.g., defined), the group of assets may be deployed to perform at least part of the work, and feedback on the status of the work execution may be provided. In connection with configuring the work, the group of assets that will perform at least part of the work may be determined. Part of the work may be determined by a semi-autonomous asset, such as a leader drone. For example, the leader drone may analyze the work (e.g., high-level instructions for performing the work) and determine one or more tasks, each corresponding to at least part of the work. For example, a first task may correspond to the first part of the work, and a second task may correspond to the second part of the work. The group of assets may determine and execute a plan for performing at least part of the work, at least semi-autonomously.

[0060] In some embodiments, the server may determine a set of assets that perform one or more tasks associated with the work. The set of assets may be determined at least in part on one or more functions associated with one or more features of the one or more tasks. For example, one or more functions associated with one or more features of the one or more tasks may correspond to functions to be performed (for example, in the case of a surveillance task, functions may correspond to video capture, image capture, etc., and in the case of drone transport, functions may correspond to range, payload carrying mechanism, payload weight limit, etc.). The set of assets that perform one or more tasks may be determined at least in part on one or more drone functions associated with each of the drones. For example, depending on the configuration of the work, the server may determine one or more tasks that correspond to the work and determine the functions associated with the tasks (for example, at least in part on the mapping of tasks to functions, etc.). The server may determine one or more assets that match the functionality associated with a task (for example, based at least in part on the mapping of assets to functionality), and may determine a group of assets that perform one or more tasks, or decide to add an asset to a predetermined group of assets that perform one or more tasks. Depending on whether a group of assets that perform work has been deployed or an asset group has been determined, at least one asset in the asset group (e.g., a leader drone) may provide information about the work (e.g., one or more features or identifiers associated with one or more tasks, an indication of a group of assets that perform one or more tasks, etc.). In some embodiments, multiple assets in an asset group, or each asset in an asset group, may be provided with instructions indicating that a particular asset is used in connection with performing one or more tasks. The asset group may be determined at least in part on the availability of one or more assets. For example, ownership of an asset (e.g., a lease agreement) may be used in connection with deciding whether or not to include an asset in an asset group that performs one or more tasks.

[0061] Various embodiments include grouping of assets to perform at least a portion of a task (e.g., one or more tasks). The asset group may include one or more semi-autonomous drones. Depending on how the task is configured (e.g., on a server), a high-level instruction for the task (or a portion thereof, such as a task) is provided to at least one drone in the asset group. The high-level instruction may include a high-level definition of the task, such as one or more characteristics of the task (e.g., the task to be performed, where the task will be performed, the time of day the task will be performed, the area in which the asset group is permitted to operate, etc.). Depending on whether the semi-autonomous drone has received information about the task (e.g., an indication that a drone is included in the asset group), it may determine one or more elements of the one or more tasks to be performed by the asset group, or one or more tasks. The semi-autonomous drone may be a leader drone among several drones in the asset group. In some embodiments, the leader drone may determine a plan for at least one drone in the asset group to perform at least one element of the one or more tasks. For example, a leader drone may perform a planning service (e.g., locally by a processor on the drone) that determines a plan for performing one or more tasks assigned to a set of assets. Determining a plan for performing one or more tasks may be at least partially based on one or more of the following: (i) the functions to be performed, (ii) the environment in which the tasks are performed, (iii) the functions associated with the tasks to be performed, and (iv) the functions of at least one drone in the set of assets. The plan may be updated based on feedback information provided by one or more assets in the set of assets (e.g., indications of asset failure, changes in the environment, target location, etc.).

[0062] In various embodiments, assets within an asset group may communicate with one another. For example, a leader drone within an asset group may provide information about at least one element of a task (such as an asset executing an element of the task or a plan for executing the task). As another example, another drone or another asset within an asset group may transmit information to the leader drone about the status of the plan's execution, the asset's status (e.g., the asset's environment (weather, etc.)), etc. In some embodiments, one or more assets (e.g., drones) may listen to information communicated between the leader drone and follower drones or other assets within the asset group. For example, a dormant leader may retrieve information communicated between asset groups to enable a relatively seamless transition of leadership responsibilities / roles in response to a determination that the leader drone has failed or a decision that the dormant drone will become the leader of a partition in the asset group. The leader drone may also communicate with a control center, such as a server, that manages / coordinates the execution of work. For example, the leader drone may provide feedback information about the work (e.g., real-time status of the work, such as an indication of whether one or more tasks have been completed). As another example, the leader drone may send a request for additional assets, or an indication that some of the assets should be released from work, for example, when some of the assets have completed corresponding elements of one or more tasks and the functionality associated with some of the assets is no longer required to perform one or more tasks.

[0063] According to various embodiments, an asset group may respond semi-autonomously to the failure of one or more assets within the asset group. For example, in response to a suggestion from a control center (e.g., a server) that the asset group will perform a work or one or more tasks associated with a work, the asset group may break down the work or tasks to determine a plan to be implemented in connection with the execution of the work or tasks associated with them. During the execution of the plan, an asset may fail due to power loss, loss of communication with the asset group (leader drone), a crash, or interaction with a third party. In response to the failure of an asset in the asset group, the remaining assets in the asset group may continue to perform the work or tasks associated with them without human intervention. The remaining assets may determine that an asset has failed and update the plan to perform the work or tasks, for example, to reassign elements or functions assigned to the failed asset. In response to the determination that the leader drone has failed, another drone in the asset group may be promoted to the role of leader drone among the remaining assets in the asset group. The drone promoted to leader drone may be based at least in part on a predetermined ranking / priority, etc. In some embodiments, a dormant leader is promoted to the role of the leader drone in response to a failure of the current leader drone. The dormant leader may correspond to the highest-ranking / priority drone in the asset group (e.g., second only to the leader drone), as indicated by a predetermined ranking / priority. The dormant leader may synchronize with the leader drone in real time or at predetermined intervals (e.g., short intervals to ensure that the dormant leader can seamlessly assume the role of the leader drone as needed).

[0064] In various embodiments, the set of assets determined to perform a task (or one or more tasks associated with a task) may be updated during the execution of the task. In some embodiments, the set of assets may be updated based on the characteristics or functions of tasks or elements of tasks that have not yet been performed, and / or the characteristics or functions of assets within the set of assets. For example, if an asset in the set of assets does not have the functions / characteristics of the remaining tasks to be performed, that asset may be released. As another example, if there are no assets in the set of assets with functions / characteristics that match the functions / characteristics of the remaining tasks to be performed, or if there are insufficient assets in the set of assets that have such functions / characteristics, the set of assets may be updated to include one or more assets with functions / characteristics that match the functions / characteristics of the remaining tasks to be performed. Updates to the set of assets may be determined by a control center (e.g., a server) that manages or coordinates the task, and updates to the set of assets may be communicated to at least one leader drone within the set of assets.

[0065] According to various embodiments, one or more user interfaces may be provided in relation to one or more of the following: configuring the work, communicating the status of the work (e.g., real-time updates), and / or updating the work (e.g., based on user input). In conventional systems for defining work, the user inputs the parameters of the work, the specific assets to be used, and the plan to be carried out with a high degree of specificity. In contrast, various embodiments include conditional display of various user interfaces that allow the user to configure the work, where a high-level plan or definition of the work is provided to at least a leader drone, and the asset group (e.g., the leader drone) determines a plan for executing tasks or elements of tasks decomposed from the high-level plan or definition of the work. Thus, various embodiments provide a user interface wizard for inputting one or more features or requirements of the work. The server may cause a client terminal to display a user interface to the user. The terminal may be shown a first user interface, which comprises one or more selectable elements associated with the features of one or more tasks to be performed, and depending on the input to the user interface, the terminal may be shown a second interface associated with the asset group on which the work will be performed. A second user interface may be configured based on input to the first user interface. Depending on the input to the second user interface, a task may be determined. The server may cause the terminal to sequentially configure and provide user interfaces based at least in part on user input to previous user interfaces, and a series of user interfaces may be used to collectively determine the task to be performed. After the task has been determined and the asset group has been instructed to perform the task, the server may cause the terminal to provide a user interface that includes the current state of the task (e.g., a real-time feed, or real-time updates to various parts of the task).In some embodiments, the user may input one or more inputs into the user interface while the task is running, such as in connection with pausing the task, updating the task (e.g., requesting faster completion of the task by increasing the number of assets in the asset group, requesting further firing capability, requesting faster coverage of a geographic area, etc.), or canceling the task.

[0066] In some embodiments, the system determines a discrete representation of a geographical location. The geographical location may correspond to a location where work is performed, or a location where a task (or element of a task) of work is performed. The discrete representation of a geographical location may include a plurality of discrete elements, each corresponding to a volume at the geographical location. The system may use the discrete representation of a geographical location in connection with determining and / or communicating parameters associated with work (e.g., parameters associated with a task or element of a task), determining a plan for performing the task, and determining the flight path (e.g., trajectory) of an asset (e.g., a drone) of an asset set determined to perform work (e.g., a set of one or more tasks). According to various embodiments, a plurality of assets in an asset set are configured to determine the discrete representation. For example, an asset performing a planning service may be configured to determine a discrete representation of a location corresponding to work or the location of a task or element assigned (e.g., allocated) to the asset. As an example, the discrete representation is determined locally by an asset in an asset set performing one or more tasks. In some embodiments, multiple assets determine their own respective versions of the discrete representation of the location, at least partially based on information communicated with respect to one or more tasks.As an example, a leader drone and / or follower drone each determine a discrete representation of the geographic location (e.g., a local version of the discrete representation) based at least in part on one or more of the following: (i) information about one or more tasks, such as one or more parameters or features associated with one or more tasks, received from a server (such as a server associated with the configuration of the relevant work) (e.g., keep-in / keep-out definitions for one or more tasks, target location, maximum altitude, etc.); (ii) information about the geographic location from third-party services such as services for map information (e.g., geographic information), services for weather information, services for information acquired by another asset or sensor located at or near the geographic location (e.g., camera, satellite, etc.); and (iii) information about the geographic location or one or more tasks from another asset within the asset group assigned to the work (e.g., the execution status of one or more tasks, information associated with the line of sight, the asset's flight plan, indication that a particular location / volume / area is occupied, etc.).

[0067] According to various embodiments, determining a discrete representation of a geographic location involves determining (e.g., creating, generating, inputting data into, etc.) a 3-D representation of the geographic location, where the 3-D representation includes a plurality of discrete elements, each corresponding to a voxel. In some embodiments, the 3-D representation of a geographic location may be determined at least in part based on transforming the geographic location from a real-world representation to a plurality of discrete elements, at least in part based on the curvature of the Earth, where the downward direction in the grid of the 3-D representation corresponds to the direction of gravity. The dimensions of the 3-D representation may be configured to include a substantially larger number of discrete elements (e.g., boxes) on the x and y axes compared to the number of discrete elements on the z axis. The z axis may be an axis parallel to gravity. As an example, if an asset moves the 3-D representation downward in the z direction, the asset moves downward towards the ground, and conversely, if an asset moves the 3-D representation upward in the z direction, the asset moves towards the sky. The length or dimension of the discrete representation on the z-axis may be configured at least in part based on the maximum altitude configuration set in relation to the work configuration. For example, the system may have a default maximum altitude associated with the work (or associated with different types of work). As another example, the user may enter the maximum altitude during the work configuration.

[0068] In various embodiments, a system (e.g., a server, a leader drone, a follower drone, etc.) generates a model of geographical locations associated with one or more tasks. Generating a model of geographical locations may include determining a discrete representation of the geographical locations and annotating the discrete representation to create an annotated representation. For example, the annotated representation may correspond to the geographical location model. In some embodiments, the system updates the annotated representation based at least in part on received information about one or more tasks and / or geographical locations. For example, the system may periodically update the annotated representation repeatedly, such as when updated or new information about one or more tasks and / or geographical locations is received. The annotated representation may be updated until one or more tasks are completed (e.g., until a decision is made to complete, abort, or pause the work). In some embodiments, annotating a discrete representation of geographical locations includes associating metadata with one or more discrete elements of the discrete representation, respectively. The metadata may include values ​​or suggestions associated with one or more tasks, geographical locations, etc.

[0069] Various embodiments for managing, coordinating, and / or executing work improve the efficiency of resource utilization (e.g., assets) assigned to perform work and enhance the fault tolerance of work execution by enabling dynamic grouping of assets and / or dynamic updates of plans for performing tasks / elements associated with the work. Various embodiments improve efficiency and effectiveness in defining work, matching assets for performing work, and determining how work is performed. In methods of the related technology, a human operator selects the assets to be used to perform work, and a centralized service (e.g., running on a server) defines the work in more detail. The centralized service provides a detailed work plan for performing the work. Various embodiments include a system of user interfaces configured to allow users to intuitively provide a high-level description of the desired work. Various embodiments provide an efficient method for grouping the assets that will perform the work. Asset grouping may be based at least in part on the automatic negotiation of ownership availability, as well as on the functions and assets associated with the work. Asset grouping is more efficient and provides better organized task execution. In related technologies, systems for defining work generally require a human operator to select the assets to be used to perform the work, and the set of assets available for selection by the human operator may be further limited because the availability of ownership is not readily apparent. In some embodiments, the system for performing work is more fault-tolerant. For example, utilizing a leader drone (e.g., a semi-autonomous drone) in relation to determining one or more tasks, and to coordinate / manage the execution of tasks by one or more assets (e.g., follower drones) within a set of assets, provides a more efficient way to dynamically update the plan for performing tasks or elements of tasks associated with the work.As another example, dynamic grouping of assets for performing work is more efficient because assets with functions no longer needed to perform the rest of the task are released from the asset group (and made available for reassignment to other work). Dynamic grouping of assets is also more fault-tolerant because additional assets can be assigned to the asset group in response to asset failure or changes in work conditions. Dynamic management / coordination of work execution (e.g., by a leader drone) allows tasks to be quickly reassigned to other assets in the asset group in response to changes in asset functionality and / or changes in work conditions (environment, etc.).

[0070] Figure 1 shows a system for performing work according to various embodiments of the present application. In the example shown in Figure 1, system 100 may include a server 105 and one or more assets 120, 125, and 130. System 100 may further include a network 110, a network 115, and / or a client terminal 135. Server 105, and assets 120, 125, and / or 130 may communicate with each other, for example, via one or more networks (e.g., network 110, network 115, and / or any other suitable network) (which may include wired networks and / or wireless networks (cellular networks, wireless local area networks (WLAN0), etc.)). According to various embodiments, server 105 may correspond to a single server or multiple servers. Similarly, system 100 may include various other assets.

[0071] In some embodiments, the server 105 may configure the work, such as determining a group of assets to perform the work. The group of assets may include one or more assets 120, 125, and / or 130. The server 105 may communicate to at least one asset in the group of assets an indication that an asset will perform the work, and / or an indication of a group of assets (e.g., a list of assets in the group, such as a list of asset identifiers). In some embodiments, the indication that an asset will perform the work may include a high-level definition or description of the work, such as a high-level task to be performed (e.g., monitoring an area or target, transporting a payload, etc.). At least one asset in the group of assets (e.g., a leader drone) may decompose the high-level definition or description of the work and autonomously determine a plan for at least some of the assets in the group to perform / complete the work. The high-level definition or description of the work may further include several parameters of the work, such as location, indications of restricted areas that the asset will not enter / operate in, and indications of areas / regions in which the asset will operate. After receiving a high-level definition or description of a task, at least one asset (e.g., a leader asset) may automatically determine a plan for at least some of the asset group to perform / complete the task, and at least one asset may cause the asset group to execute the plan and communicate information about the task (e.g., the current state of the task) to the server 105 via network 110 and / or network 115. For example, a leader drone may decompose the high-level definition or description of the task and determine a plan for one or more tasks corresponding to the task. In some embodiments, a discrete representation of geographical locations associated with one or more tasks is generated. As an example, the discrete representation of geographical locations is generated based at least in part on information about the geographical locations and / or one or more tasks. The information is received from the server 105 and / or another asset in the asset group (e.g., a follower asset, a leader drone, etc.). In some embodiments, the plan for one or more tasks is determined based at least in part on the discrete representation.For example, a discrete representation is annotated to create an annotated representation. Annotating a discrete representation, for example, includes setting metadata or associating metadata with one or more discrete elements of the discrete representation. In some embodiments, assets in the asset group (e.g., asset 120, asset 125, and / or asset 130), and / or server 105, use the annotated representation to determine a plan, such as using information contained in the metadata associated with the discrete elements (e.g., to determine the trajectory or flight path of an asset). Having determined a plan for one or more tasks, the leader drone may transmit the plan for at least one task to another asset in the asset group (e.g., a follower asset). Upon receiving the plan for at least one task, the follower asset may execute the plan and autonomously determine any further instructions to be executed locally to perform at least one task. The asset group performing the work may be dynamically updated (for example, during the execution of the work) for purposes such as adding assets (for example, when an asset with a particular function fails and needs to be replaced with a new one), or removing assets (for example, when an asset is included in the asset group to provide a specific function for any task that requires that function, and that task is completed).

[0072] In various embodiments, server 105 communicates with client terminal 135 via a network (such as network 115). Server 105 may display one or more user interfaces to client terminal 135 in connection with configuring a task, displaying the status of a task, and / or updating a task (e.g., pausing, canceling, or modifying a task). Client terminal 135 may receive one or more user inputs to the various user interfaces and communicate suggestions of one or more user inputs to server 105. Server 105 may use one or more user inputs to determine the various user interfaces displayed to client terminal 135, for example, in connection with configuring the various user interfaces to create a wizard in which the user defines or inputs the characteristics of the task. While a task is running, server 105 may use user inputs to change the status or parameters of the task, and in response to such changes, server 105 may communicate the changes to at least a leader asset / drone (e.g., to have a leader drone implement the changes). For example, in response to a change in the status or parameters of a task, the server 105 may update the high-level definition or description of the task and provide the corresponding high-level instruction or update to the leader drone among the asset group performing the task. In some embodiments, the server 105 receives information about the task (e.g., feedback information) from the asset group (e.g., from the leader drone). This information may include the status of the task, real-time images or videos of the task, etc. Upon receiving information about the task, the server 105 may provide status updates, real-time task status, etc., to the client terminal 135. In some embodiments, the server 105 may determine one or more recommendations or options for updating the task and provide at least one of the recommendations or options to the user via a user interface displayed on the client terminal 135.For example, one or more recommendations or options may include a suggestion of increasing the number of assets assigned to a set of assets in order to speed up the execution of the work (for example, it may show that adding a certain number of assets could reduce the time to completion by a calculated estimated time).

[0073] Figure 2 is a block diagram showing a device for configuring or controlling work according to various embodiments of the present application. In the example shown in Figure 2, device 200 may comprise a communication interface 202 and / or one or more processors 205. Device 200 may correspond to the server 105 in Figure 1, the control center 460 in Figures 4A-4C, and / or the control center in Figures 5A-5C. Device 200 may perform the process 700 in Figure 7A, the process 720 in Figure 7B, the process 1000 in Figure 10A, and / or the process 1020 in Figure 10B and the process 1100 in Figure 11. According to various embodiments, one or more processors 205 may comprise or perform one or more of the following: a communication module 210, an asset ownership module 220, a function mapping to assets 230, a work definition module 240, a grouping module 250, a pre-planning module 260, a feedback information module 270, and a user interface module 280.

[0074] Device 200 may implement one or more modules related to determining a task, determining the set of assets on which to perform the task, and communicating suggestions of the task (e.g., a high-level description of the task) to one or more assets in the set of assets (e.g., a leader drone), including one or more characteristics of the task. During the execution of the task, Device 200 may implement one or more modules related to communicating information with at least the leader drone to receive information about the current state of the task or to send instructions for changes to the task, and / or communicating information with a client terminal to provide updates on the state of the task or to receive changes to the task.

[0075] Device 200 may communicate with asset groups, client terminals, other servers or terminals, etc., using the communication module 210. For example, the communication module may be provided to the communication interface 202 to which communication is being performed. As another example, the communication interface 202 may provide information received by device 200 to the communication module 210.

[0076] According to various embodiments, the determination of the asset group on which to perform work is based at least in part on the ownership of one or more assets within the asset group. Device 200 may decide to include only assets in the asset group that Device 200 owns (e.g., assets owned or controlled by the organization to which Device 200 belongs), or assets for which Device 200 may acquire at least temporary ownership. Asset ownership may include the ability to control the asset, such as providing instructions to the asset to perform a specific function (e.g., a task, an element of a task, etc.). As an example, asset ownership may include a lease agreement that gives the owner the ability to control the asset. The lease agreement may be for a fixed / predetermined period or may be associated with the completion of a specific task. In some embodiments, the transfer of ownership (e.g., related to the negotiation of ownership for determining the asset group) may be a permanent transfer of ownership, or otherwise for an unclear period, or until Device 200 or its organization relinquishes ownership of the asset. In some embodiments, the asset group is configured to give the leader drone a longer ownership period (e.g., lease length, permanent ownership) than the other assets in the asset group (e.g., follower drones). In some embodiments, if it is determined that the work is one in which the follower drone is relatively likely to lose communication during the execution of the work (e.g., based on the terrain of the area in which the work is performed), device 200 may configure the asset group to give the follower drone a longer ownership period. A longer ownership period can ensure that ownership of the follower drone persists for periods when the follower drone is not communicating with the leader drone and / or device 200, etc. In some embodiments, asset ownership includes a failover entity to which ownership of the asset is transferred if the current entity that owns it fails.Information regarding asset ownership may be stored in one or more of the following: (i) locally with the asset, (ii) a centralized ownership service such as a service hosted by one or more servers providing services across multiple organizations, (iii) device 200, and (iv) the asset owner. Information regarding asset ownership may include one or more of the following: indication of the current owner, indication of the length of current ownership, indication of the availability of ownership transfer (e.g., the current owner's willingness to transfer ownership), current failover owner, and indication of procedures for negotiating ownership transfer. In some embodiments, asset ownership may include automatic transfer ownership, where a device or organization requesting ownership (e.g., with appropriate authorization) can automatically assume ownership of the asset when ownership is requested.

[0077] In some embodiments, device 200 may utilize the asset ownership module 220 in connection with determining ownership of a particular asset and / or negotiating the acquisition of ownership of an asset. The asset ownership module 220 may acquire information regarding asset ownership, and device 200 may use such information in connection with determining a group of assets. For example, device 200 may use the asset ownership information to determine whether a particular asset is available for allocation / assignment to a group of assets to perform work. The asset ownership module 220 may negotiate with an entity for the transfer of ownership of an asset for use associated with work. For example, the asset ownership module 220 may request a lease for a predetermined period, which may be the expected length of work, the expected length of time the asset is needed to perform a particular task or element of a task (e.g., the period during which the asset's functionality is required to perform the work), and so on. The asset ownership module 220 may request ownership information from a particular asset, or a particular asset may notify it of ownership information for such asset. In some embodiments, device 200 may send requests / commands to acquire ownership of assets, depending on the determination of the asset group to be used in connection with performing the work. For example, device 200 may send a request to acquire ownership of an asset in connection with providing an asset group suggestion to the asset, such as a command suggestion indicating that the asset is included in the asset group (e.g., as part of or together with it). As a further example, device 200 may send a request to acquire ownership of one or more assets in connection with providing an asset group suggestion to a reader, the reader can then communicate with one or more assets in the asset group in connection with acquiring applicable ownership of one or more assets (e.g., to perform at least one task in the work).

[0078] According to various embodiments, the set of assets used in connection with performing a task is determined and / or updated at least in part on the functionality of the corresponding assets included in the set of assets. In connection with configuring a task, device 200 may determine a set of functions associated with performing the task (e.g., functions required to perform the various functions included in the execution of the task). In connection with determining the set of assets, device 200 may use a set of functions associated with performing the task. For example, device 200 may match assets in a superset of assets having functions that match the set of functions associated with performing the task. In connection with determining a superset of assets having functions that match at least one function in the set of functions associated with performing the task, device 200 may query a mapping of functions to an asset (or asset identifier). In response to determining that there are no matching assets (e.g., assets with the corresponding functions) in the superset of available assets for the functions of the task, device 200 may determine whether the functions can be achieved collectively using two or more assets, or whether to reconfigure the task. For example, device 200 may provide the client terminal / user with a suggestion that it does not have an asset that matches the functionality of the task, and / or may prompt the user to reconfigure the task or cancel the task (for example, based on further user input).

[0079] In some embodiments, device 200 may use a mapping of functions to assets 230 in relation to determining the functions of an asset. The mapping of functions to assets 230 may be stored locally or may communicate with a remote service that stores the mappings. The mapping of functions to assets 230 may query the mapping with a specific identifier of the asset to look up functions for such asset. Similarly, the mapping of functions to assets 230 may query a mapping of a specific function to look up an asset having such function. In some embodiments, in response to receiving one or more parameters of a task, device 200 may determine one or more functions associated with the task and use the mapping of functions to assets 230 to determine assets having matching functions. For example, device 200 may determine a group of assets to perform a task based at least in part on the determination that the group of assets collectively has functions that match the functions associated with performing the task. Device 200 may determine a group of assets such that at least one asset in the group has at least one function that matches the functions associated with performing the task. The function mapping 230 to the asset may be stored locally on the device 200, or a module for retrieving a specific function mapping may be used to query a remote service that provides the function mapping 230 to the asset. The function mapping to the asset may be updated in response to a determination that the asset's functionality has changed. For example, the asset may notify of new functionality, loss of functionality (e.g., in response to detection of peripheral device failure), and / or updates to functionality. In some embodiments, the device 200 may update the function mapping to the asset.In some embodiments, a remote service (e.g., a service hosted by a server) updates a function mapped to a specific asset, and device 200 may periodically synchronize the update to the function mapping to the asset, or device 200 may query the function mapping to the asset as needed.

[0080] In some embodiments, an asset stores suggestions of its functionality. For example, an asset may locally store definitions of its functionality, and may update these definitions based on the asset's current functionality in response to failures of modules that provided the functionality (e.g., sensors, cameras, or peripherals such as added gimbals), additions of new modules (e.g., addition of new peripherals, new software loaded onto the asset, updates to existing software, etc.). The definition of functionality may be a list of the asset's functions or suggestions of functionality in accordance with a predetermined format / protocol. In some embodiments, an asset may notify its functionality by transmitting definitions of functionality across one or more networks, or by means of a leader drone during the execution of a plan / task, or the asset may provide suggestions of its functionality in response to queries from servers (e.g., servers that configure the task), a centralized functionality server, a leader drone, etc. In some embodiments, device 200 (e.g., a mapping of functions to assets 230), or a remote service managing the mapping of functions to assets, may ping one or more assets for updates on the current functions of one or more assets, or otherwise, may ping for updates on the health of the assets, and the corresponding mapping of functions to assets may be updated (e.g., to reflect the current set of functions for one or more assets, and / or the state of the functions).

[0081] According to various embodiments, device 200 may define one or more tasks performed by one or more assets. For example, a task definition may include at least one of the following: setting one or more objectives associated with the task, setting the location where the task will be performed, setting the target of the task, setting one or more constraints on the execution of the task (e.g., rules of combat, restricted or permitted airspace for engaging the target, etc.), setting the date of the task, etc. One or more tasks may be defined at least in part on one or more user inputs, such as user selections entered into a user interface provided by user interface module 280. In some embodiments, device 200 uses a task definition module 240 to define one or more tasks. For example, the task definition module 240 may receive one or more user inputs from the communication module 210 and / or user interface module 280 to user inputs, and the task definition module 240 may determine one or more features / parameters associated with the task in connection with defining the task. An example of a work definition may include the task of locating or tracking specific targets (e.g., person XYZ, track ABC, etc.) in Los Angeles from January 1 to January 3, 2021, with assets not exceeding 100 miles from Los Angeles or entering within 2 miles of a permanent no-go zone or airport. In some embodiments, one or more user inputs associated with the work may include suggestions of one or more types of assets to be used in connection with the performance of the work. Continuing in the above example, the work definition may further include instructions that the work will be performed using drones and / or fixed-wing aircraft.

[0082] In some embodiments, a task may be broken down into one or more associated sets of functions or features. For example, the definition of a task (e.g., a task defined using a task definition module) may be parsed, and one or more functions associated with the task may be determined based on the parsing. The set of one or more functions associated with the task may be determined at least in part based on querying a mapping of task parameters to the functions. The parameters associated with the task may be determined based on the definition of the task. Examples of work parameters include: type of work (e.g., road network scanning, discovery, confirmation, tracking, Reuters flight, scanning along a path, etc.), work classification (e.g., attack, defense, surveillance, etc.), location, path taken when the task is performed (e.g., path for tracking / scanning), asset type (e.g., drone, fixed-wing, helicopter, boat, etc.), target (e.g., individual, airplane, land vehicle, unknown type of target, etc.), scope of work, location of work, rule set for work execution, indication of what can / cannot be done (e.g., permitted actions), survivability during work (e.g., how likely is the asset to be shot down or otherwise terminated), communication requirements (e.g., whether the asset is engaged in silent communication mode, or This includes, for example, suggestions about when the link might be lost, suggestions about the type of communication to be used, or other parameters or constraints regarding communication during the operation, etc.; target behavior (e.g., suggestions about the prediction of the target's movement, such as whether the target is stationary or moving on a road, patterns of movement, etc.); potential threats (e.g., locations of sites that can destroy or disable the drone, such as SAMs (surface-to-air missile launchers), RF jamming sites, etc.); low-probability target zones (e.g., areas not expected to contain any target of interest specified by the user); maximum altitude the asset will travel at (e.g., above sea level); keep-in areas (e.g., areas where the asset will remain); keep-out areas (e.g., areas where the asset is prevented / prohibited from entering), etc. In some embodiments, the work definition module 240 determines one or more functions associated with the operation.In other embodiments, the grouping module 250 determines one or more functions associated with the task.

[0083] According to various embodiments, device 200 determines a group of assets (e.g., a team of assets) to perform a task. Device 200 may automatically determine a group of assets to perform a task depending on how the task is defined (e.g., based on one or more user inputs). The group of assets may be determined based at least in part on one or more of the following: (i) task parameters, (ii) one or more functions associated with the task, (iii) functions of one or more assets, and (iv) ownership of one or more assets. Various other factors / variables may be used in connection with determining a group of assets to perform a task. In some embodiments, device 200 determines a group of assets based on a best fit, based at least in part on one or more factors / variables associated with the assets and / or the task. For example, device 200 may determine a group of assets using a cost function, for example, which may include weights for various factors used in determining the group of assets. Device 200 may determine a group of assets based on minimizing the overall cost value determined by the cost function for combinations of assets in the group of assets. In some embodiments, asset sets may be determined such that the corresponding total cost value using a cost function is less than a cost threshold. The cost threshold may be configurable (e.g., by an administrator, user, etc.) in relation to the definition of the work. In some embodiments, the cost threshold is determined to correspond to a predetermined percentile of all possible sets of assets (e.g., so that the set of assets performing the work is in the top 10 percent of all possible sets of assets, or to any other configurable percentile). An example of a cost function in a context involving pairing assets to a road assignment may include weighting of a) distance from the road and b) the probability that the target is on the road. Using the cost function described above, if there are assets that are far from the road, those assets are not selected because the distance component of the cost function excludes those assets from being desirable assets.Similarly, if there are less likely roads, these will be scanned in the future, and higher-priority roads will take precedence.

[0084] In some embodiments, determining an asset group may include determining the number of assets to include in the asset group. The number of assets to include in an asset group may be determined at least in part on the work (e.g., the functions associated with the work) and the functions of the assets. For example, determining the number of assets may be based on the cost value of the asset grouping and / or the set of functions provided by the asset grouping with respect to the functions associated with the work. As will be discussed further later, the number of assets to include in an asset group may be determined at least in part on the decision to include redundancy in the asset group with respect to one or more functions among the assets in the asset group and / or the extent to which redundancy is included among the assets in the asset group.

[0085] Device 200 may determine asset groups using a grouping module 250. The grouping module 250 may determine asset groups based at least partially on the definition of the work. For example, asset groups may be determined at least partially on a set of functions associated with the work, and / or a set of work parameters. The grouping module 250 may determine one or more asset groups having functions that match the set of functions associated with the work. As an example, the grouping module 250 may determine asset groups such that for each function in the set of functions associated with the work, the asset group contains at least one asset with a matching function.

[0086] In some embodiments, device 200 determines a work preplanning in relation to determining the set of assets on which the work will be performed. Device 200 may use a preplanning module 260 to determine the work preplanning. The preplanning for the work may be a high-level description or decomposition of the work (e.g., a set of tasks or elements of tasks to be performed) to determine what will be done during the execution of the work. Generating a preplanning in relation to determining the set of assets to be used during the execution of the work allows the grouping module 250 to evaluate the functions associated with the work (e.g., various functions required to perform the work) and / or the properties or characteristics of the assets required or desired for the execution of the work (e.g., a set of tasks or elements of tasks to be performed during the work). In some embodiments, depending on having determined the work preplanning, device 200 determines one or more assets having functions that match the functions or requirements of the work.

[0087] Device 200 may decide to include redundancy in the asset group for at least one of the one or more functions associated with the work. For example, Device 200 may use the grouping module 250 to decide to include redundancy in the asset group based at least partially on the type of work, work classification, work location, etc., when the type of work / work classification / location is related to the corresponding probability of asset failure that exceeds a probability threshold or otherwise increases the risk of failure. If the work is an offensive operation or in a hostile environment, the probability of asset failure may be higher. The degree of redundancy that Device 200 incorporates into the asset group may be based on the degree of probability of asset failure. For example, multiple thresholds may be defined, each containing a corresponding probability of asset failure, and multiple thresholds may be used in relation to determining the degree of redundancy to be incorporated into the asset group (for example, the degree of redundancy may be mapped to a specific threshold corresponding to the probability of asset failure, and the mapping may be queried to determine the degree of redundancy to apply to the asset group). The degree of redundancy can indicate the number of assets that can be included in a group of assets each possessing a specific function (such as a function mapped to an increased risk of asset failure, or a function of an asset identified as having an increased risk of asset failure).

[0088] Device 200 may communicate an asset group suggestion in response to determining the asset group to be used in connection with performing the work. For example, in response to the grouping module 250 determining the asset group, device 200 communicates the asset group suggestion using the communication module 210. In some embodiments, device 200 transmits to at least one leader in the asset group (e.g., a leader drone). The leader drone may then provide suggestions to other assets in the asset group that other assets are in the asset group. For example, the leader drone may notify each asset that it is included in a team of assets for the work (e.g., an asset group). As another example, the leader drone provides each asset in the asset group with an asset group suggestion so that each member knows, for example, the entire asset group in the team for performing the work. Suggestions provided to the leader drone, or to one or more other assets in the asset group, may be communicated together with or separately from the tasks or plans associated with performing the work.

[0089] In various embodiments, a pre-plan for performing a task may be determined via a remote service and subsequently provided to a group of assets (at least a leader drone, for example). The remote service for determining the pre-plan may be performed by device 200, such as using the pre-plan module 260, or via one or more third-party pre-plan modules that the pre-plan module 260 or device 200 can communicate with. For example, the pre-plan module 260 may have an application programming interface (API) for interface with device 200 when one or more third-party pre-plan modules provide pre-plans associated with a task. The pre-plan module 260 and / or third-party pre-plan modules may obtain the definition of the task and / or one or more parameters associated with the task. In some embodiments, determining a pre-plan for a task is a process of repeatedly determining a pre-plan, providing information about the pre-plan to a user via a user interface module 280, etc. (e.g., displaying the pre-plan on a client terminal), receiving feedback information about the pre-plan (e.g., one or more inputs about the pre-plan or parameters of the associated task), and updating the pre-plan based on the feedback information. The updated pre-plan may then be provided to the user, and feedback information regarding the pre-plan may be received for further improvement. The provision of the pre-plan and the receipt of feedback information may be repeated, for example, until the user provides instructions for the work to begin. As an example, device 200 may determine a pre-plan showing that a group of assets will track targets along a path, or otherwise move along a specific path, and device 200 may display the path to the user on a client terminal. The user may provide one or more inputs to modify the path of the group of assets, and device 200 may update the pre-plan accordingly. As another example, device 200 may determine a pre-plan showing how assets will fly when scanning several areas.For example, when performing or analyzing a lawn service, if the original polygon shape does not lead to an optimal scan, it may be useful to see the back-and-forth mowing pattern on the polygon, and potentially the flight path when scanning that area. For instance, if a user performs vertical mowing but constructs a long horizontal polygon to scan / analyze the mowing, performing the scan according to the original long horizontal polygon may lead to significant overshoot and back-and-forth of the asset, so the user or drone may adapt to a horizontal scanning pattern in this case.

[0090] In some embodiments, device 200 transmits a pre-plan that the asset group will follow when performing work, and the leader asset may determine a plan for the asset group to perform the work based on the pre-plan. For example, the device may determine and transmit a pre-plan when the work is defined in order to ensure that the asset group performing the work follows the framework of the plan. The pre-plan may be a high-level definition or description of what will be achieved by the asset group or a part of it, and the leader drone may determine a lower-level plan that further specifies how the asset group will perform the work (or its task). The leader drone may then provide the plan for performing the work to another asset in the asset group (e.g., a follower drone), and the other asset may then determine a more specific plan for executing the plan provided by the leader drone. For example, in the context of a monitoring task, device 200 (e.g., a server) may determine a pre-plan that shows the path or boundary lines that the asset group will travel to in order to acquire monitoring information about a particular location, and device 200 may provide the pre-plan to the leader asset in the asset group (e.g., a leader drone). Upon receiving the pre-plan, the first asset may break down the boundaries or paths within the pre-plan into smaller segments, assign such segments to various follower assets (e.g., follower drones), and accordingly provide the various follower assets with commands to move to the assigned segments to capture monitoring information about those segments. Continuing in this example, the various follower assets may receive commands to capture monitoring information about specific segments, and the various follower assets may determine the specific methods for moving to the segments and capturing the monitoring information (e.g., a follower drone may plan its route to avoid obstacles (such as trees) along the route).

[0091] In various embodiments, device 200 may acquire feedback information regarding the definition of a task, a pre-plan for the task, and / or the status or result of the task execution. Device 200 may acquire feedback information using a feedback information module 270 via a communication module 210 or the like. In some embodiments, the feedback information module 270 may acquire information regarding the definition of a task (such as one or more features or parameters of the task entered into a user interface displayed on a client terminal). Upon receiving information regarding the definition of a task, the feedback information module 270 may provide such information to a task definition module 240 or the like. In some embodiments, the feedback information module 270 may acquire information regarding the pre-plan for a task (e.g., input for changing or updating the pre-plan) via input to a user interface provided by a client terminal. Upon receiving information regarding the pre-plan, the feedback information module 270 may provide the information regarding the pre-plan to a pre-plan module 260, which may then update the pre-plan based on such information. In some embodiments, the feedback information module 270 may acquire information regarding the status or result of the task execution. For example, the feedback information module 270 may receive real-time work information (e.g., an indication of asset failure, a live video feed, etc.) from the asset group (e.g., a leader drone) while the work is being performed. The feedback information module 270 may provide real-time work information to the user interface module 280 in connection with displaying the status of the work. As another example, the feedback information module 270 may receive an indication that the work has been completed.Upon receiving an indication that the work is complete, the feedback information module 270 may provide an indication or corresponding instruction to one or more modules, such as (i) the asset ownership module 220 to provide an indication that the work is complete, or (ii) the asset ownership module 220 to relinquish or return ownership of one or more assets in the asset group.

[0092] In various embodiments, device 200 may display one or more user interfaces on one or more client terminals. The user interfaces may be displayed in connection with configuring or defining work, providing and updating pre-plans, and / or providing the status of work. Device 200 may use a user interface module 280 to configure the user interfaces displayed on client terminals. The user interface module 280 may configure user interfaces at least partially based on one or more templates. Furthermore, the user interface module 280 may provide a wizard or workflow for generating and providing multiple user interfaces to the user for the purpose of configuring work based on one or more inputs to multiple user interfaces. In some embodiments, user input to a certain user interface when configuring work is used in connection with generating the next user interface used to configure the work. Thus, the user interface module 280 may provide one or more user interfaces (or pages of user interfaces) to client terminals to guide the user through the work configuration process. In contrast, work is currently defined within a single composite interface that requires all work features or parameters to be entered on a single page. The use of multiple user interfaces logically connected via user input to previous user interfaces simplifies the structure of the work, thereby reducing the skill level of human operators and lowering the risk of errors or inaccurate input.

[0093] Figure 3 is a block diagram showing a device for performing at least part of the work according to various embodiments of the present application. In the example shown in Figure 3, device 300 may include a communication interface 302 and / or one or more processors 304. Device 300 may correspond to assets 120, 125, and / or asset 130 in Figure 1, assets 405, 410, 420, 430, 440 (e.g., satellite), and / or asset 450 (e.g., tower) in Figures 4A to 4C, and / or assets 505, 510, 515, 520, 525, 530, 535, 540 (e.g., satellite), and / or asset 545 (e.g., tower) in Figures 5A to 5C. Device 200 may perform the processes 800 in Figure 8A, 830 in Figure 8B, 830 in Figure 8C, 900 in Figure 9A, 930 in Figure 9B, 950 in Figure 9C, 951-2 in Figure 9C, 950 in Figure 9D, 1000 in Figure 10A, 1020 in Figure 10B, 1200 in Figure 12A, 1215 in Figure 12B, 1215 in Figure 12C, 1300 in Figure 13, 1400 in Figure 14, 1500 in Figure 15, 1800 in Figure 18, 1900 in Figure 19A, and / or 1940 in Figure 19B.

[0094] According to various embodiments, one or more processors 305 may include or perform one or more of the following: a communication module 310, a planner service module 320, a function mapping module 330 for assets, a dynamic grouping module 340, an ownership module 350, a partitioning module 350, a feedback information module 370, and an asset failure module 380. Device 300 may implement one or more modules associated with performing work, such as determining a plan for the asset group to perform a task (or elements of a task) of work, dynamically changing the asset group, communicating with one or more other assets in the asset group, updating the plan for performing the task based on information received from another asset in the asset group, and communicating with one or more other assets in the asset group assigned to perform work.

[0095] According to various embodiments, device 300 corresponds to a semi-autonomous drone. Device 300 may receive high-level plans or instructions for performing work (or tasks or elements thereof), and device 300 may autonomously determine how to perform the work (or tasks or elements thereof). In some embodiments, device 300 corresponds to a leader drone in a group of assets assigned to perform work. In some embodiments, device 300 corresponds to a follower drone in a group of assets assigned to perform work. Device 300 may have the same set of modules regardless of whether the device is a leader drone or a follower drone. For example, multiple assets in a group of assets may have the same modules as the leader drone to provide redundancy in case of leader drone failure, or when the group of assets is divided to provide different aspects of work (or tasks of work), and multiple partitions include a leader drone for each partition. As another example, the group of assets may include a dormant leader drone that synchronizes with or backs up the leader drone to provide redundancy in case of leader drone failure. In some embodiments, certain types of assets may have the same set of modules or leadership functions as the leader drone, while other types of assets may have only a portion of the modules or leadership functions. For example, a semi-autonomous drone in a group of assets may have the same set of modules or functions as the leader drone, while an autonomous tower, sensor, or satellite in a group of assets may have a different set of assets and may not have enough functionality to be the leader of the group of assets.

[0096] Device 300 may communicate with asset groups, other servers or terminals (for example, server 105 in system 100 in Figure 1) using the communication module 310. For example, the communication module 310 may provide information to the communication interface 302 with which it is communicating. As another example, the communication interface 302 may provide information received by device 300 to the communication module 310. In response to receiving information from the communication interface 302, the communication module 310 may provide such information to the corresponding module. For example, if device 300 receives a suggestion to perform a task, the communication module 310 may provide such suggestion to the planner service module 320. As another example, if feedback information is received from one or more follower drones while the task is being performed, the communication module 310 may provide the feedback information to the feedback information module 370 and / or the planner service module 320, etc.

[0097] In various embodiments, planning of how work, tasks, or elements are performed is carried out through various entities within the system. A server constituting the work may determine work parameters and provide such parameters to a leader drone, and / or determine a high-level pre-plan for the asset group to perform the work and provide such pre-plan to the leader drone. The leader drone may then determine a lower-level plan than the parameters or pre-plan for the work received from the server. The plan determined by the leader drone may autonomously fill in some gaps in the information received from the server in relation to performing the work. The leader drone may provide a plan for performing a task or elements of a task to one or more follower drones. In response to receiving the plan from the leader drone, one or more follower drones may each determine how to perform the plan, such as autonomously filling in gaps in the plan received from the leader. The framework described above allows the system (such as system 100 in Figure 1) to be a modular system that supports hierarchical planning, and such corresponding planning by the leader drone and / or follower drones can be autonomous. In some embodiments, each asset within an asset group may indicate the level of planning or task it supports. The leader drone may determine the plan corresponding to the asset within the asset group based on the level of planning or task supported by the follower asset. For example, a follower drone with more robust planning services may not require as detailed planning as an asset with basic (or no) planning services. In some embodiments, the leader drone may store a mapping of planning or task capabilities to an asset and use such mapping in relation to determining the plan for each follower asset.According to various embodiments, at least one follower drone has the capability to perform the same or substantially the same robust planning services as those performed by the leader drone.

[0098] In various embodiments, upon receiving a suggestion (e.g., from a server, such as a server providing work control services) that device 300 is included in a group of assets, device 300 autonomously determines (e.g., generates) a plan for performing one or more tasks associated with the work, and communicates at least a portion of the plan to one or more other assets. Device 300 may be a leader asset (e.g., a leader drone) in a group of assets for performing one or more tasks associated with the work. Device 300 may receive information about the work that the group of assets will perform. The information about the work may include a high-level definition or description of the work. For example, the high-level definition or description of the work may include high-level tasks to be performed (e.g., monitoring an area or target, transporting a payload, etc.). Examples of information included in a high-level definition or description of an operation include one or more of the following: (i) suggestions for the asset group; (iii) several parameters of the operation, such as location, suggestions for restricted areas where the asset will not enter / be active, and suggestions for areas / regions where the asset will be active; (iii) functions associated with the operation; and / or (iv) a prior plan for performing the operation (or its tasks). In some embodiments, information about the operation is communicated only to the leader asset in the asset group (or the leader asset and dormant leaders). Upon receiving information about the operation, the leader asset may communicate suggestions for the asset group to other assets in the asset group (e.g., to follower assets).

[0099] In some embodiments, work information is transmitted only to the leader asset of the asset group (e.g., the leader asset). Work information may also be transmitted to a dormant leader of the asset group (e.g., which may also be referred to herein as a second leader), which may be a backup of the leader asset that synchronizes work information (e.g., current work status, plans for performing various tasks, etc.). Thus, the dormant leader may have a complete backup of the work information stored / managed in the leader asset. In some embodiments, the dormant leader is selected based on negotiations among multiple assets within the asset group (e.g., negotiations among semi-autonomous drones within the asset group). An asset may be selected as a dormant leader for work based at least in part on a determination of one or more functions the asset possesses and / or the availability of ownership (e.g., whether a lease or ownership is available during the work period). For example, an asset may be selected as a dormant leader based at least in part on a determination that the dormant leader has one or more predetermined functions (e.g., a planning service module, etc.). As another example, an asset may be selected as a dormant leader based at least in part on the determination that the asset has a specific set of functions that match the set of functions of a leader asset (for example, a dormant leader has all the functions of a leader asset, or a dormant leader has the planning or work management functions of a leader asset). As yet another example, an asset may be selected as a dormant leader based at least in part on the determination that the asset is the best fit as a dormant leader among the other assets in the asset group.

[0100] In some embodiments, multiple assets within an asset group (e.g., each asset, each drone within the asset group, etc.) may receive suggestions for the asset group. These suggestions may include information identifying the assets within the asset group. The suggestions may further include suggestions for various functions of the assets within the asset group. In some embodiments, each asset within the asset group (e.g., each member of a team performing a task) may have knowledge of all other assets within the asset group. Follower assets may receive suggestions for the asset group from a leader asset for the asset group, or follower assets may receive suggestions for the asset group from a server (e.g., a server providing work control services). One asset may be determined as the leader asset for the asset group (e.g., leader drone) based on a predetermined leader ranking of assets, negotiation among at least some of the asset groups, etc. In some embodiments, the predetermined leader ranking is entered by a user, such as during task configuration. In other embodiments, the leader ranking is negotiated (or updated) by at least some of the asset groups, etc.

[0101] In various embodiments, upon receiving information about a task, a leader asset (e.g., a leader drone) autonomously determines a plan for performing the tasks (or elements of the tasks) associated with the task. The leader asset may communicate at least a portion of the plan for performing the tasks or elements of the tasks to at least one follower asset (such as a follower asset that has been determined to perform the tasks or elements of the tasks). The leader asset may determine a plan for performing tasks or elements of tasks on a task-by-task, element-by-element, or follower asset-by-follower asset (e.g., each follower drone). In some embodiments, the leader asset determines individual plans for follower drones that follower drones will use in connection with performing the corresponding tasks or elements of tasks. The leader asset may determine individualized plans for each follower drone in a group of assets, each follower drone currently assigned a task or element of a task, and so on.

[0102] In some embodiments, device 300 uses a planner service module 320 in relation to determining a plan for executing a task or element of a task. If device 300 is a leader drone, the planner service module 320 may determine a plan for a group of assets to execute work (e.g., one or more tasks or elements of a task associated with the work). For example, the planner service module 320 may determine a plan that one or more follower drones will implement in relation to executing a task or element of a task. The plan may be determined based on one or more parameters of the work and a group of assets. In some embodiments, the plan may be determined at least in part based on the capabilities associated with the corresponding task or element of a task and the capabilities of the follower drones. For example, the planner service may determine one or more follower drones having capabilities that match the capabilities associated with the corresponding task or element of a task, and determine at least one of such follower drones to be assigned to execute at least a portion of the task or element. In some embodiments, the plan determined by device 300 is a lower-level plan than any prior plan (if any) received from a server (e.g., along with instructions to execute the work). For example, device 300 may decide to perform a more specific task or element than those provided in the corresponding pre-plan. As an example, in the context of road search (e.g., the task of searching for a particular set of roads), the high-level task may search a polygon containing a set of roads and a set of preferred target locations (places where the targets were thought to be located at a given time). The leader drone may evaluate the set of roads contained in the polygon, spread the probabilities of preferred targets across the roads, and then assign assets (e.g., follower drones) to the highest priority and closest roads (e.g., to perform the search for those roads). The low-level plan provided to the follower drone includes the roads along which the follower drone will scan and the order in which these roads will be scanned.

[0103] In some embodiments, device 300 uses a planner service module 320 in relation to determining how to execute a received plan. If device 300 is a follower drone, and has received a plan to execute a task or element of a task, device 300 may use the planner service module 320 to plan the execution of the plan. For example, the planner service module 320 may determine an even lower-level plan to execute the plan received from the leader drone. The planner service module 320 may determine specific ways to control device 300 to execute an assigned task or element of a task. As an example, if device 300 has received a plan to perform monitoring of a defined area, device 300 may acquire information about the area (such as the locations of various obstacles or objects) and determine a path for device 300 to move within the area while avoiding obstacles or objects. As another example, even if the plan received from the leader drone includes a defined path along which device 300 will move, device 300 may perform dynamic monitoring of the area to perform collision avoidance during the execution of the plan. In some embodiments, the follower drone receives a detailed plan from the leader drone that effectively identifies the different steps to be taken by the follower drone. The follower drone may perform the steps defined by the leader drone while dynamically monitoring or updating the plan to prevent asset failure or to keep the follower drone on course for carrying out the plan.

[0104] According to various embodiments, device 300 uses planner service module 320 in relation to determining a discrete representation of geographical locations related to work or in relation to at least one asset in a group of assets performing one or more tasks. As an example, the discrete representation includes a plurality of discrete elements, each corresponding to a volume at the geographical location. As another example, planner service module 320 determines the discrete representation in relation to determining a plan for device 300 or a plan for an asset in a group of assets performing one or more tasks.

[0105] In some embodiments, after a discrete representation is determined (for example, in response to the determination of a discrete representation of a geographical location), device 300 annotates the discrete representation using the planner service module 320, for example, to create an annotated representation. As an example, the planner service module 320 annotates the discrete representation based at least in part on received information about one or more tasks and / or geographical locations. The planner service module 320 annotates the discrete representation based at least in part on current information about one or more tasks and / or geographical locations. Current information corresponds to information stored by or accessible to device 300 at the time the annotated representation is created. In some embodiments, the system updates the annotated representation based at least in part on received information about one or more tasks and / or geographical locations. For example, the system may periodically update the annotated representation repeatedly, such as when updated or new information about one or more tasks and / or geographical locations is received. Annotated representations may be updated until one or more tasks are completed (e.g., until a decision is made to complete, abort, or pause the work). In some embodiments, annotating a discrete representation of a geographic location involves associating metadata with one or more discrete elements of the discrete representation. The metadata may include values ​​or implications associated with one or more tasks, geographic locations, etc.

[0106] One example of annotating discrete representations involves setting metadata associated with at least some of multiple discrete elements to define keep-in areas (e.g., areas where assets within a device 300 or asset group are held) or keep-out areas (e.g., areas where assets within a device 300 or asset group are excluded / prevented from entering). For example, each discrete element within some discrete elements has metadata fields corresponding to keep-in indicators and / or keep-out indicators. For example, keep-in areas and / or keep-out areas are configured in relation to the configuration of the work (e.g., a user inputs keep-in areas and / or keep-out areas, specific predetermined keep-in and keep-out areas are set based on flight restriction airspace, etc., and a third-party service provides inputs to configure keep-in and / or keep-out areas).

[0107] Another example of annotating discrete representations involves setting metadata associated with at least some of multiple discrete elements to set the location of one or more targets for an operation. For an operation to perform monitoring of a specific target, configuring the operation involves obtaining the target's current location (predicted location, most recently known location, etc.). As an example, the user sets the current location of a specific target in connection with configuring the operation. The planner service module 320 determines the specific discrete element corresponding to the current location of the specific target and sets metadata fields associated with the specific discrete element to indicate that the target is located in / within the discrete element. As another example, for an operation to perform monitoring of a specific target (such as a building or a road), the planner service module 320 determines the discrete element corresponding to (or containing) the specific target (such as a building or a road) and sets metadata fields corresponding to the discrete element to indicate that the specific target is contained within the discrete element.

[0108] Another example of annotating discrete representations involves setting metadata associated with at least some of multiple discrete elements to define lines of sight, such as between assets, between assets and leader drones, and between leader drones and ground control stations. For example, setting metadata to define lines of sight includes indicating for a particular discrete element whether a line of sight is attainable when an asset is located at a geographical location corresponding to that discrete element.

[0109] Another example of annotating discrete representations involves setting metadata associated with at least some of the discrete elements to indicate whether the discrete elements are occupied or not. For example, a discrete element is considered occupied if it contains assets (e.g., assets in an asset group, other friendly assets not in an asset group, etc.). For another example, a discrete element is considered occupied if an asset cannot move into the discrete element for reasons such as insufficient space within the discrete element to maintain the asset without the risk of collision with another asset or object (e.g., a building, tree, mountain, etc.) (e.g., a collision risk exceeding a threshold risk). In some embodiments, the planner service module 320 sets metadata associated with discrete elements to indicate whether the discrete elements are occupied or not in connection with registering or storing flight plans. For example, device 300 determines a flight plan for an asset in an asset group, or receives information about a flight plan from an asset in an asset group, and planner service module 320 determines discrete elements affected by the flight plan (e.g., a set of discrete elements that intersect with the flight plan) and sets metadata for such discrete elements to indicate that they are occupied. Setting discrete elements corresponding to a flight plan as occupied can ensure that no other flight plan intersects with the existing one is created or registered. For example, device 300 and / or other assets to which an annotated representation or model (or the source information from which the annotated representation may be created) is issued utilize the annotated representation in connection with determining a new flight plan, and the annotated representation includes discrete elements that are set as occupied for an existing flight plan (e.g., at least for the forecast period corresponding to the flight plan). In some embodiments, planner service module 320 further sets a threshold number of adjacent discrete elements adjacent to the set of discrete elements corresponding to the flight plan. Adjacent discrete elements can function as buffers to provide additional spatial distance between existing flight plans and any subsequently created flight plans.The threshold for adjacent discrete elements may be configurable. For example, for any particular discrete element corresponding to a flight plan, the planner service module 320 sets N discrete elements as occupied in all directions of that particular discrete element (where N is a positive integer).

[0110] In some embodiments, device 300 updates the annotated representation using planner service module 320, at least in part, based on received information regarding one or more tasks and / or geographic locations. For example, planner service module 320 repeatedly updates the annotated representation, for example, periodically (e.g., at default intervals, configurable intervals, etc.), after updated or new information regarding one or more tasks and / or geographic locations has been received. As an example, planner service module 320 updates the annotated representation until one or more tasks are completed (e.g., until a decision is made to complete, abort, or pause the work). In some embodiments, annotating a discrete representation of geographic locations involves associating metadata with one or more discrete elements of the discrete representation, respectively. The metadata may include values ​​or suggestions associated with one or more tasks, geographic locations, etc. Iterative or continuous updates of the annotated representation improve the model of geographic locations or work. For example, additional information regarding geographical location can be obtained through assets deployed at specific locations (e.g., follower drones), thus improving the resolution of certain quantities or types of information included in the model.

[0111] According to various embodiments, the planner service module 320 uses annotated representations in relation to determining a plan for performing one or more tasks. For example, the planner service module 320 utilizes metadata associated with discrete elements in discrete representations to determine destinations for one or more assets in a set of assets, determine routes for one or more assets, determine elements performed by a particular asset, etc. In some embodiments, the resolution of the geographic location / work model is improved by receiving feedback information (e.g., updated information about tasks or elements of tasks, or information about part of the geographic location of an asset) from assets (e.g., follower drones). As the resolution of the geographic location / work model improves, the plan for performing one or more tasks is updated (e.g., improved).

[0112] In various embodiments, device 300 uses a mapping of functions to assets in connection with assigning tasks or elements of work, and / or determining a plan for a particular follower asset to perform a task or element. If device 300 is a leader drone, device 300 may store the mapping of functions to assets 330 locally and query the mapping of functions to assets to assign tasks or elements, or to determine a plan. At the start of work (for example, in response to receiving an instruction to perform work), the leader drone may receive a current set of functions for various assets in the asset group. For example, the leader drone may have the asset group provide current functions to the leader drone (for example, it may send a request to follower drones to register the corresponding functions with the leader drone). In another example, the leader drone may receive a set of functions for various assets in the asset group from a server. The leader drone may update the mapping of functions to assets 330 during the execution of work. For example, the leader drone may ping / request the follower drones to provide updated information about the current set of functions of each follower drone, and the leader drone may update the mapping accordingly. As another example, the leader drone may decide to update the functions of a particular follower drone based on feedback information communicated to the leader drone by the follower drones during the execution of the task (e.g., the execution of a plan by the follower drones). Device 300 may determine from the feedback information that a particular sensor or camera from a follower drone is not functioning properly or has failed (e.g., the feedback information may not have a clear signal associated with the normal operation of such sensor / camera).

[0113] According to various embodiments, device 300 may utilize a mapping of functions to an asset in connection with the execution of a plan, and / or utilize the mapping to maintain the current state of the asset's functions. If device 300 is a follower drone, device 300 may provide the corresponding leader drone with the current set of functions of the follower drone, or the state of various functions within the set of functions associated with the follower drone (e.g., an indication that a particular function is fully operational, partially operational, or not operational). Device 300 may provide such information to the leader drone at predetermined intervals, upon request from the leader drone, or together with feedback information communicated to the leader drone during the execution of a plan.

[0114] According to various embodiments, the set of assets assigned to perform a task may be dynamically updated during the execution of the task. For example, assets within an asset set may be modified / updated in response to (i) a user request to reallocate an asset within the asset set to another task, (ii) loss of ownership of an asset within the asset set, (iii) loss of ownership of an asset within the asset set (e.g., if a function necessary to complete the task is lost during the execution of the task, a new / additional asset with such function may be assigned to the asset set), (iv) user input that changes the expected speed at which the task is expected to be completed (e.g., additional assets may be added to expedite completion, or assets may be removed from the asset set if the user decides to slow down completion), (v) a cost function associated with the execution of multiple tasks indicates that a more optimal completion of multiple tasks involves the use of additional assets or the reallocation of assets from the asset set, (vi) a determination that an asset within the asset set has failed, or (vii) the fulfillment of a condition defined within the task (e.g., under a monitoring task, the condition may include the detection of a target to which another or different asset is assigned to intercept). The asset set may be dynamically updated in response to various other circumstances. The asset set may be dynamically updated by a server, a leader drone, or both.

[0115] In some embodiments, the dynamic grouping module 340 may be used in connection with dynamically updating the asset group. Device 300 may use the dynamic grouping module 340 to determine whether another asset is added to the asset group and / or whether an asset is removed from the asset group. Depending on whether the dynamic grouping module 340 has decided to update the asset group, the dynamic grouping module may provide such indication to various other modules (for example, to the planner service module 320 to update or create a new plan for the execution of a task or element using the updated asset group, or to the function mapping to assets 330 to update the function mapping for the updated asset group, or to the communication module 310 to notify the server of the update or to request such update from the server, or to the ownership module 350 to request / update ownership of the asset being added to or removed from the asset group).

[0116] In various embodiments, a leader drone may manage ownership of assets within an asset group. Upon receiving an indication that a leader drone is in an asset group on which work is being performed, the leader drone may obtain permission for ownership and / or ownership status corresponding to the assets within the asset group (e.g., transfer of ownership of the corresponding asset to the organization on which the work is being performed, where applicable). Transfer of ownership may correspond to a transfer of ownership of an asset for (i) part of the work, such as during the performance of a particular task or element; (ii) the duration of the work; or (iii) indefinite (e.g., permanent transfer of ownership until the acquiring organization relinquishes ownership). Permission for ownership may include automatic transfer (e.g., allowing ownership to be transferred to the acquiring organization / leader drone upon request), or negotiation of approval / denial of asset transfer (e.g., conditional transfer while holding aside needs arising for the transferring organization, a specified period / lease term on which ownership is transferred, a specified task / element on which ownership is transferred, etc.). The ownership status may indicate whether an asset is available for transfer to a leader drone (or the organization on which the leader drone is performing work), whether an asset is unavailable for transfer, and a set of conditions under which an asset may or may not be transferable. In some embodiments, the leader drone may obtain ownership permissions and / or ownership status based on queries to ownership services (e.g., third-party or centralized services on which various organizations can register / transfer ownership of assets), queries to mappings of assets to ownership stored locally or remotely on a server managing the work (e.g., the server on which the work was configured), and / or queries to various assets within an asset group (e.g., each asset may store its corresponding ownership permissions and / or ownership status and / or protocols on which the leader drone can request ownership transfer). The leader drone may use the ownership module 350 to manage / update ownership of assets within an asset group, and / or negotiate or request the transfer of assets to / from an asset group.For example, in response to an indication that an asset is being added to an asset group or that ownership of an asset within an asset group is being requested or updated, the ownership module 350 may be prompted to request / arrange for the transfer of ownership of the asset.

[0117] During the execution of a task, the leader drone may determine that the task (or tasks / elements of the task) could be performed more effectively or efficiently by one or more parts of the asset group, and the leader drone may divide the asset group. The leader drone may decide how to divide the asset group. For example, different partitions may be determined based on the functions of tasks / elements performed by different partitions and / or functions of the assets within the asset group. In some embodiments, partitions are assigned corresponding leader assets. For example, each partition has a corresponding leader drone. Thus, follower drones may be promoted to the position of leader drone for a particular partition. The leader drone of the asset group may be the principal leader, and various partition leaders may report feedback information (such as the status of tasks / elements assigned to partitions). The principal leader then aggregates the feedback information received from partition leaders and / or any assets that report directly to the principal leader (e.g., assets in the same partition as the principal leader) and provides the feedback information to the server and / or provides instructions to partition leaders, such as modifications to tasks or elements assigned to partitions. In response to the division of the asset group, the primary leader may provide instructions to the partition (e.g., the partition leader) to perform tasks or elements that the primary leader has assigned to the partition. In response to the partition receiving suggestions to perform tasks or elements, the partition leader may determine a plan for the corresponding divided asset group. For example, the partition leader may play a role for the partition such that the role is similar to that played by the primary leader for the asset group (e.g., before the division). Similarly, a partition may have one or more follower assets that (i) receive plans from the partition leader, (ii) implement the plans received from the partition leader, and / or (iii) communicate feedback information to the partition leader.In various embodiments, upon determination that a partition has completed the tasks / elements assigned to it, the divided assets may be (i) reintegrated into an asset group (e.g., with the partition corresponding to the primary leader), (ii) reassigned to other partitions to assist in the completion of tasks / elements assigned to those other partitions, or (iii) released from the asset group (e.g., released to perform other work for another leader or organization). In some embodiments, when a partition is created, a set of constraints on which the partition operates may be determined. The primary leader drone may provide a set of constraints to the corresponding partition leader.

[0118] In various embodiments, device 300 uses a partition module 360 ​​in relation to determining whether to partition an asset group and how the asset group is partitioned. The partition module 360 ​​may determine which assets are to be included in different partitions. When an asset in a partition is to be assigned to another partition or reintegrated into an asset group in response to a determination that the partition has completed the task / element assigned to the partition, the partition 360 may determine which partition such assets are to be included in. In some embodiments, a partition is redeployed to perform another task / work after the completion of the task / work assigned to the partition (for example, the primary leader, or a planner service module on the primary leader, decides to assign another task / work).

[0119] Device 300 may send / receive feedback information regarding the execution of a task. If Device 300 is a leader drone, Device 300 may receive feedback information from one or more follower drones, such as information acquired by the sensors / cameras of the follower drones, the status of tasks / elements assigned to the follower drones, the status of the drones, and the status or updates to the drones' functions. Additionally or alternatively, Device 300 may send feedback information to a server (such as a server that manages (or configured) the task). For example, a leader drone may send the status of the task, a live feed of information acquired by one or more sensors in the asset group, and requests for future commands regarding the task. If Device 300 is a follower drone, Device 300 may send feedback information to the leader drone, such as information acquired by the sensors / cameras of the follower drones, the status of tasks / elements assigned to the follower drones, the status of the drones, and the status or updates to the drones' functions. Device 300 may communicate feedback information using the feedback information module 370. The feedback information module 370 may receive feedback information, determine various pieces of information contained in the feedback information, and transfer that information to the corresponding module. For example, if the feedback information includes a suggestion that a sensor on a follower device has malfunctioned, the feedback information module 370 may determine that such information relates to the function of an asset and provide such information to the function mapping 330 for assets so that the mapping of functions to assets in the asset group is updated.As another example, if a follower drone communicates sensor information (e.g., images, live feed, weather conditions, etc.) to the leader drone, the feedback information module 370 may determine such sensor information in the feedback information and provide such information to the planner service module so that the leader drone can use such information in relation to determining the status of the operation and / or whether to update the plan for the follower drone (or other follower drones).

[0120] In various embodiments, asset teams are dynamically managed throughout the execution of a task, from the time the task is configured until it is completed. Assets may be added to or removed from a group of assets assigned to perform a task during the task's execution. Decisions to add or remove assets may be made on a server. In some examples, decisions to add or remove assets may be made by a leader drone of the asset group. Dynamic management of teams provides an extensible framework that can adapt to changes within the asset group, unexpected or conditional events occurring during the execution of a task, or changes in the configuration of the task (such as user-initiated changes to the task). Examples of changes within the asset group may include asset failure, failure / loss of asset functionality, or partitioning of the asset group (for example, a request to add a specific functionality to an asset may arise because the asset group only has one of the specific functionality required by multiple partitions). Examples of unexpected or conditional events occurring during the execution of an operation may include new targets appearing during area monitoring, weather or environmental changes requiring specific capabilities or where such capabilities could enhance effectiveness (e.g., larger drones to withstand extreme weather such as strong winds or rain), and operations that extend into the night, thereby increasing the value of night vision capabilities. Examples of changes in the configuration of an operation may include user requests to complete an operation faster (e.g., speeding up execution by the asset group), user requests or other decisions to prioritize another operation, and user requests to pause an operation. In some embodiments, upon deciding that a new asset is to be added to the asset group, the server sends a suggestion to the leader that the new asset has been included in the asset group. The leader drone may then, in response, determine a plan for performing the operation and / or manage ownership of the new asset (e.g., acquire appropriate ownership of the new asset). In some embodiments, upon a follower drone being removed from the asset group, an instruction is sent to the follower drone indicating that the follower drone is no longer part of the asset group.The server and / or leader drone may transmit such instructions to the follower drones.

[0121] In various embodiments, upon determination that a follower drone has failed during the execution of a task, the leader drone may update the plan for performing the task. For example, the leader drone may determine the status of the task / element assigned to the failed follower drone and reallocate the remainder of the task / element (or the entire task / element) to another asset in the asset group. The leader drone may update the functionality of the asset group in response to the failure of the follower drone, and the leader drone may utilize the updated functionality to reallocate / redistribute elements of the plan to other follower drones. In some embodiments, upon determination that a follower drone has failed, the leader drone may add additional assets to the asset group or send a request (e.g., to a server) to add additional assets. For example, the leader drone may identify one or more functions of the failed follower drone and attempt to compensate for those functions (or functions that are not redundant or are not sufficiently redundant in the asset group). The leader drone may determine that the follower drone has failed if it has determined that it has not received feedback information from the follower drone within a predetermined period and / or by an estimated time (for example, the estimated time may be based on a plan currently assigned to the follower drone, such as when it is predicted that the follower drone will not communicate for a certain period of time). In some embodiments, if the leader drone has not received feedback information from the follower drone within a predetermined period or by an estimated time, the leader drone may send a ping to the follower drone for the health status of the follower drone, and the leader drone may determine that the follower drone has failed if the follower drone has not responded to the ping (for example, within a threshold period). In some embodiments, the server may determine that the follower drone has failed if it has determined that it has not received any information or pings from the follower at all within a predetermined period and / or by an estimated time, and the server may notify the leader drone that the follower drone has failed.

[0122] In various embodiments, upon determination that the leader drone has failed during the execution of a task, the remaining assets in the asset group are promoted to leader status, and such assets become the new leader drone for the execution of the task. A leader drone may be determined to have failed if it has failed or has lost communication with some or all of the assets. The remaining assets in the asset group may negotiate to determine a new leader within the asset group. Negotiations to determine a new leader may be based at least in part on a leader ranking (or priority list of assets) of the assets in the asset group. The leader ranking may be a predetermined ranking determined during the configuration of the task and provided to all assets in the asset group, or a predetermined ranking provided to at least one asset other than the initial leader asset, and the highest-ranked asset among the remaining assets is promoted to the new leader drone. In some embodiments, the remaining assets negotiate a new leader ranking for the assets, and the highest-ranked asset among the remaining assets is promoted to the new leader drone. In some embodiments, if a dormant drone is included in the asset group (the dormant drone is among the remaining assets), the dormant leader is automatically promoted to the new leader drone.

[0123] In various embodiments, partitions may be formed based on network or communication failures. For example, if a portion of an asset loses communication with a leader drone, the portion of the asset may determine (individually or collectively) that the leader drone has failed. In response to the determination that the leader drone has failed, the portion of the asset may form a group of assets (e.g., a partition), and the portion of the asset may negotiate for a partition leader. Once a partition leader has been determined, the partition leader may determine a plan for the partition to continue performing its work (or collective task / element assigned to the portion of the asset). The partition leader may attempt to establish communication with a server and / or another partition associated with the same work. In response to establishing communication with the server, the partition leader may provide feedback information regarding the performance of the work (or portion of the work performed by the partition). In response to establishing communication with the partition leader of another partition, the partition leader may request or attempt to group the two partitions together, or the partition leader may provide feedback information regarding the work to the other partition leader. In some embodiments, different partition leaders may negotiate the division of tasks / elements associated with the work so that different partitions perform different parts of the work. Partitions may have predetermined or negotiated constraints on how they operate. Examples of constraints include geographical constraints, time limits, and restrictions on specific actions (e.g., restrictions on engagement with a target).

[0124] In some embodiments, if a partition leader fails, the remaining assets in the partition fail over to a partition or group of assets (for example, a partition having a leader drone corresponding to the highest-ranking leader in a given leader ranking). The remaining assets in the partition may fail over to a partition or group of assets if a new partition leader cannot be determined.

[0125] According to various embodiments, each operation has a default plan that is carried out by the assets within the asset group if a new leader cannot be determined. For example, the default plan may be a plan in which each asset returns to its home in response to a determination that a new leader cannot or will not be determined (e.g., within a given period).

[0126] Device 300 may use the asset failure module 380 in relation to determining that an asset has failed and to determine the failover protocol to be implemented when an asset has failed. For example, if the failed asset is a leader drone, the asset failure module 380 may decide to negotiate with the remaining assets to determine a new leader drone. As another example, if the failed asset is a follower drone, the asset failure module 380 may decide to reallocate part of the plan assigned to the failed follower drone among the remaining assets.

[0127] In various embodiments, in addition to determining a plan for performing a task or element, the leader drone determines a method for communicating the plan to the follower drone. The leader drone may determine how to communicate the plan based on the communication link between the leader drone and the follower drone, the quality of the communication link, etc. In some embodiments, the leader drone may determine how to communicate the plan based on the estimated quality of communication between the leader drone and the follower drone at a future point in time (e.g., a specific point in the execution of the task / element). For example, the leader drone may determine that the follower drone will move over a mountain in connection with performing the task, that the follower drone may lose communication with the leader drone, or that the communication link may have relatively poor quality service. Depending on such determination, the leader drone may provide the follower drone with a broader scope of the plan for performing the task / work. As another example, if the leader drone determines that the current communication link is relatively weak (e.g., slow / low bandwidth), depending on such determination, the leader drone may decide to send only a relatively small portion of the plan for the work to the follower drone and send the larger portion of the plan at a future point in time when the communication link is good or is expected to be good. In various embodiments, the leader drone may determine the granularity at which the plan is divided and communicated to the follower drones, based on network conditions and / or network constraints. The network conditions and / or network constraints used in such determination may be current conditions / constraints or conditions / constraints expected at a specific future point in time. Examples of network constraints may be one or more values ​​relating to one or more of the network topology (e.g., areas accessible to the leader drone or follower drones), communication bandwidth, link quality, etc. The determination of the granularity at which the plan is divided and / or communicated to the follower drones, and / or how the plan is communicated to the follower drones, may be part of the planning process performed by the leader drone.For example, a leader drone may plan a task in multiple stages and sequentially transmit parts of the current plan or parts of the plan for multiple stages of the task. Similarly, in some embodiments, a server managing / configuring a task may determine how / to what extent information about the task (e.g., task parameters) is communicated to the leader drone, based at least in part on network conditions and / or network constraints and / or the planning service capabilities of the leader drone.

[0128] Figure 4A shows a system for performing at least part of a task according to various embodiments of the present application. In the example shown in Figure 4A, system 400 may include a group of assets (e.g., assets 405, 410, 420, 430, 440, and / or 450) and a control center 460 (such as a server providing task control services). System 400 may be implemented by system 100 in Figure 1. The control center 460 may be implemented by device 200 in Figure 2. One or more assets in the group of assets may be implemented by device 300 in Figure 3. For example, assets 405, 410, 420, and / or 430 may be implemented by device 300 in Figure 3. In some embodiments, assets 405, 410, 420, and / or 430 are semi-autonomous drones. Asset 440 may be a satellite that communicates with control center 460 and / or another asset (such as asset 405). Asset 450 may be a tower (such as a semi-autonomous tower). Asset 450 may communicate with control center 460 and / or another asset (such as asset 405).

[0129] In the example shown in Figure 4A, a group of assets may be commanded to perform a task. For example, the task may be tracking vehicle 470 and / or monitoring the road (as well as tracking a specific vehicle or any vehicle moving along the road). The control center 460 may configure the task and provide task parameters to a group of assets (such as asset 405). Asset 405 may be determined to be the leader asset, and the other assets (e.g., asset 410, asset 420, and / or asset 430) may be determined to be follower assets. In response to the leader asset receiving the task parameters and any prior plans from the control center 460, the leader asset may determine a plan for other follower assets to perform tasks or elements of the task. Continuing in the above example, the leader asset may determine a plan for follower asset 2 (asset 410) to track vehicle 470, a plan for follower asset 4 (asset 430) to perform monitoring of the first part of the road, and a plan for follower asset 3 (asset 420) to perform monitoring of the second part of the road. In accordance with the determination of the plans for the follower assets, the leader asset may transmit the plans to each follower asset accordingly. The follower assets may implement the plans upon receiving them from the leader drone, and may fill in any gaps in the plans provided by the leader drone. The leader asset decides that asset 440 (e.g., a satellite) will provide road monitoring and assistance in tracking vehicles traveling along the road. The leader asset may determine a plan for asset 450 to capture area / environmental information using one or more sensors. The captured information may be provided to the leader asset as feedback information, which the leader asset may use to update the plans for various assets in the asset group.

[0130] Figure 4B shows a system for performing at least part of the work according to various embodiments of the present application. In the example of system 400 shown in Figure 4B, the leader asset (asset 405) is determined to have failed. In response to the determination that the leader asset has failed, a new leader for the asset group is determined among the remaining assets. For example, assets 410, 420, and 430 may negotiate a new leader asset.

[0131] According to various embodiments, determining a new leader asset includes determining a new leader asset based on a leader ranking. For example, the leader ranking 480 may be obtained by one or more of the remaining assets. In some embodiments, each asset stores a predetermined leader ranking. In some embodiments, a follower asset obtains a leader ranking by negotiating a leader ranking based on the respective functions of the remaining assets. In the example shown in Figure 4B, the leader ranking 480 is predetermined, such as being configured by the control center 460 at the same time as the work configuration. Asset 1 in the leader ranking 480 is the highest-ranking asset, and asset 4 is the next highest-ranking asset. Since asset 1 is determined to have failed, asset 4 is determined to be the highest-ranking asset among the remaining assets. Therefore, asset 4 is determined to be the next leader asset for the asset group. In some embodiments, in response to asset 4 being determined as the new leader asset, asset 4 determines the status of the work and determines a plan for executing at least the remainder of the work. Asset 4 may transmit the corresponding plan assigned to the follower asset to the remaining follower assets.

[0132] Figure 4C shows a system for performing at least part of a task according to various embodiments of the present application. In the example of system 400 shown in Figure 4C, the leader asset (asset 420) is determined to be faulty. In response to the determination that the leader asset is faulty, the leader asset may determine the status of the task and / or determine an updated plan for the remaining assets (e.g., assets 410 and 430). According to various embodiments, in response to the determination that a follower asset is faulty, the leader asset may reassign the tasks / elements (or remaining tasks / elements) assigned to the faulty follower asset among the remaining assets. The leader asset may update the plan for the remaining assets based at least in part on the determined reassignment of tasks / elements. In some embodiments, the leader drone and / or asset 450 may decide to add one or more assets to the asset group (e.g., to compensate for the functionality lost due to the failure of asset 430, to facilitate the execution of the task, etc.). For example, a list 490 of additional assets may be determined. The leader drone and / or asset 450 (e.g., a tower with a computer system) may dynamically update the asset set, for example, by adding one or more additional assets from the list of additional assets 490. In some embodiments, the list of additional assets 490 may be determined at least partially on the functionality of the corresponding assets, such that the assets on the list of additional assets 490 have at least one functionality that matches the functionality of the remaining tasks to be completed. In some embodiments, the list of additional assets 490 may be determined at least partially on the availability of ownership of the assets on the list of additional assets (e.g., an asset may be currently available for work or may be reassigned to perform work after a request is submitted to take ownership of the asset or enter into a lease agreement).

[0133] In some embodiments, the control center (e.g., control center 460 in Figures 4A, 4B, or 4C) is one of several control centers that control a system including a semi-autonomous drone. In various embodiments, the control center includes a ground control center, an airborne control center, a waterborne control center, or any other suitable control center. In various embodiments, a single control center performs control, each control center controls a portion of the system assets, and the control center performing the control is rotated among several control centers so that it takes over cooperatively when control changes occur, or any other suitable control is assigned to a control center.

[0134] Figure 5A shows a system for performing at least part of a task according to various embodiments of the present application. In the example shown in Figure 5A, system 500 may include a group of assets (e.g., assets 505, 510, 515, 520, 525, 530, 535, 540, and / or 545) and a control center 550 (such as a server providing task control services). System 500 may be implemented by system 100 in Figure 1. The control center 550 may be implemented by device 200 in Figure 2. One or more assets in the group of assets may be implemented by device 300 in Figure 3. For example, assets 505, 510, 515, 520, 525, 530, and / or 535 may be implemented by device 300 in Figure 3. In some embodiments, assets 505, 510, 515, 520, 525, 530, and / or 535 are semi-autonomous drones. Asset 540 may be a satellite communicating with a control center 550 and / or another asset (such as asset 505). Asset 545 may be a tower (such as a semi-autonomous tower). Asset 545 may communicate with a control center 550 and / or another asset (such as asset 505).

[0135] In the example shown in Figure 5A, a group of assets may be commanded to perform a task. For example, the task may be tracking vehicle 555 and / or monitoring the road (and tracking a specific vehicle or any vehicle moving along the road). The control center 550 may configure the task and provide task parameters to a group of assets (such as asset 505). Asset 505 may be determined to be the leader asset, and the other assets (e.g., asset 505, asset 510, asset 515, asset 520, asset 525, asset 530, and / or asset 535) may be determined to be follower assets. In response to the leader asset receiving the task parameters and any prior plans from the control center 550, the leader asset may determine a plan for other follower assets to perform tasks or elements of the task. Continuing in the above example, the leader asset may determine a plan for follower asset 525 to track vehicle 555, a plan for follower asset 510 to perform monitoring of the first part of the road, and a plan for follower asset 515 to perform monitoring of the second part of the road. In accordance with the determination of the plans for the follower assets, the leader asset may transmit the plans to each follower asset accordingly. The follower assets may implement the plans upon receiving them from the leader drone, and may fill in any gaps in the plans provided by the leader drone. The leader asset decides that asset 540 (e.g., a satellite) will provide road monitoring and assistance in tracking vehicles traveling along the road. The leader asset may determine a plan for asset 545 to capture area / environmental information using one or more sensors. The captured information may be provided to the leader asset as feedback information, which the leader asset may use to update the plans for various assets in the asset group.

[0136] Figure 5B shows a system for performing at least part of the work according to various embodiments of the present application. In the example of system 500 shown in Figure 5B, an additional vehicle 560 appears and moves on a road being monitored by the asset group. One or more assets in the asset group may capture images of the vehicle 560. For example, a satellite may identify the vehicle 560 moving on the road. In another example, assets 510, 520, and / or 515 may provide feedback information to a leader asset (asset 505), which may process the feedback information to determine that the vehicle 560 is within a geographic area defined by the work parameters and / or meets the conditions for tracking a vehicle moving on the road.

[0137] Upon detecting vehicle 560, the leader asset may decide to split the asset group. For example, the leader asset may determine that the work (including tracking vehicles 555 and 560) is more likely to be performed more effectively if the asset group is split into two partitions. Upon deciding to split the asset group, the leader asset may instruct the corresponding follower assets in the asset group, for example, to provide the follower assets with an indication of which partition each follower asset belongs to. Assets 505, 510, 515, 520, 55, 530, and 535 may be split into two partitions (first partition 570 and second partition 580). In some embodiments, the leader asset is in the first partition 570, so the leader asset is determined to be the partition leader of the first partition 570. In some embodiments, the second partition 580 contains assets that were originally follower assets in the asset group, so at least one of the assets in the second partition 580 is promoted to partition leader of the second partition 580. The assets within the second partition 580 may negotiate to determine the leadership of the second partition 580. For example, the partition leader may be determined by a predetermined leadership ranking, such as a leadership ranking determined at the same time as the work configuration. The partition leader may be determined by the asset within the second partition 580 that has the highest corresponding ranking in the leadership ranking. In the example shown in the figure, asset 525 may be promoted to partition leader.

[0138] In the example shown in the figure, the task of tracking vehicle 560 may be assigned to a first partition, the task of tracking vehicle 555 may be assigned to a second partition 580, the task of providing road monitoring may be assigned to asset 540, and the task of monitoring the environment may be provided to asset 545 (e.g., a tower). In some embodiments, a leader asset in the first partition 570 may determine the respective plans for assets 510, 515, and 520 to perform the corresponding elements of the task of tracking vehicle 560. Upon receiving the plans for performing the corresponding elements, assets 510, 515, and 520 may implement the plans and fill in any gaps in the plans (e.g., providing collision avoidance such as avoiding hills or trees, determining the altitude of the flight, determining when to capture images, etc.). Assets 510, 515, and 520 may provide feedback information to the leader asset, which relates to the execution status of elements of the task of tracking the vehicle 560, and / or information captured by one or more sensors (e.g., live video feeds, images such as license plate images, etc.). The leader asset may provide feedback information to the control center 550, such as information about the vehicle 560.

[0139] In some embodiments, a partition reader in the second partition 580 may determine the respective plans for assets 530 and 535 to perform corresponding elements of the task of tracking the vehicle 555. Upon receiving the plans for performing the corresponding elements, assets 530 and 535 may implement the plans and fill in any gaps in the plans (e.g., providing collision avoidance such as avoiding hills or trees, determining the altitude of the flight, determining when to capture images, etc.). Assets 530 and 535 may provide feedback information to the partition reader, which relates to the status of the execution of elements of the task of tracking the vehicle 555, and / or information captured by one or more sensors (e.g., live video feeds, images such as license plate images, etc.). In some embodiments, the partition reader may provide feedback information to a leader asset (asset 505), which may then provide feedback information (or information including such feedback information aggregated or processed / analyzed together with feedback information from the first partition 570). The leader asset may then provide feedback information to the control center 550, such as information about the vehicle 560. In some embodiments, the partition leader may provide feedback information directly to the control center 560 (for example, providing information to the control center without providing such information to the leader asset).

[0140] Figure 5C shows a system for performing at least part of the work according to various embodiments of the present application. In the example shown in Figure 5C, the leader asset may determine, at least in part, based on feedback information (such as feedback information from asset 510, asset 515, and / or asset 520), that tracking of vehicle 555 is no longer performed. In some embodiments, the leader asset may determine that the state of vehicle 555 no longer meets the work parameters for tracking a vehicle moving on a monitored road. For example, based on feedback information, the leader asset may determine, at least in part, based on the work parameters and / or constraints, that vehicle 555 has left (is no longer in) the geographic area being monitored or tracked. In another example, the leader asset may determine that vehicle 555 has entered an area that is a restricted area according to the work parameters, and that the group of assets does not enter such an area.

[0141] In some embodiments, the system 500 may decide to dynamically add assets to the asset group. For example, the leader asset and / or control center 550 may decide to add asset 585 and / or asset 590 to the asset group. Assets may be added to the asset group in response to a determination that better coverage of a geographic area / road is desirable for road monitoring (for example, based on user input to a user interface operationally provided by the control center 550). In response to the decision to add asset 585 and / or asset 590 to the asset group, the leader asset may update the function mapping to the assets and update the plan for performing the work. The leader asset may determine a plan for asset 585 to perform the task, at least in part, based on the function of asset 585, etc.

[0142] In some embodiments, the control center (e.g., control center 550 in Figures 5A, 5B, or 5C) is one of several control centers that control a system including a semi-autonomous drone. In various embodiments, the control center includes a ground control center, an airborne control center, a waterborne control center, or any other suitable control center. In various embodiments, a single control center performs control, each control center controls a portion of the system assets, and the control center performing the control is rotated among several control centers so that it takes over cooperatively when control changes occur, or any other suitable control is assigned to a control center.

[0143] Figure 6 shows a user interface for configuring, monitoring, and / or controlling operations according to various embodiments of the present application.

[0144] According to various embodiments, the server may configure one or more user interfaces in connection with configuring a task. For example, the server may generate and configure a set of user interfaces to create a wizard that allows the user to define a task effectively and intuitively in the configuration process. In various embodiments, the user interface is configured based on user input to a preceding user interface (e.g., the previous user interface).

[0145] As shown in Figure 6, a set of user interfaces 610, 630, and 650 may be displayed to the user to allow the user to systematically define parameters for their work.

[0146] In interface 610, the user interface includes elements for the user to (i) input the name and description of the task, (ii) select the type of task (e.g., road network scanning, route scanning, Reuters flight and tracking, target discovery, confirmation, and tracking), and (iii) select the type of target to be targeted by the task (e.g., a target for monitoring or tracking). Depending on the input to interface 610, user interface 630 may be configured.

[0147] In user interface 630, the user interface includes selectable elements for the user to select parameters associated with the type of asset and / or how the asset moves (e.g., by air, on land, or at sea). The various selectable elements on user interface 630 may be determined in response to input to interface 610.

[0148] In user interface 650, the user interface includes selectable elements for the user to select one or more parameters associated with the task. For example, the user may define the terrain or boundary on which the task will be performed. Another example is that the user may define restricted areas that the asset performing the task cannot enter. Another example is that the user may define the date and time on which the task will be performed, or the date or time on which the task will be restricted. User interface 650 may also include selectable elements for causing the server to generate the task. For example, the server may generate the task based on the user-defined task configuration / definition.

[0149] Depending on the task generated, the server may display a user interface for a live work screen. The live work screen may include selectable elements for the user to choose to activate, deactivate, or abandon the task. In some embodiments, the live work screen includes current status information regarding the status of the task's execution. For example, the live work screen may show the percentage of the task completed. As another example, the live work screen may include a live stream of video captured by at least one asset in the asset group. In some embodiments, the live work screen may include selectable elements that allow the user to invoke a user interface (or wizard for the user interface) to modify the task. For example, the user may have the server add one or more assets to the asset group. As another example, the user interface may display options (e.g., server-generated options such as possible modifications) that allow the user to choose to modify the task according to suggested modifications. The server may display in the user interface the addition of a specific asset, the addition of an asset with specific functionality, or improvements to the task by adding one or more assets. The user may input a selection to modify the task according to the suggested modifications presented by the server. In some embodiments, the server may display in the user interface the costs associated with performing work using an asset group, and / or the cost difference or cost impact, when an asset is added to or removed from the asset group.

[0150] Figure 7A shows a method for configuring the operation according to various embodiments of the present application. In some embodiments, the operation 700 in Figure 7A is performed by the server 105 of the system 100 in Figure 1 and / or the device 200 in Figure 2.

[0151] In step 710, data associated with one or more tasks to be performed is received. In some embodiments, the server receives data associated with one or more tasks in connection with configuring the work. The data associated with one or more tasks may be received at least in part based on one or more user inputs to the user interface provided to the user interface presented to the client terminal. According to various embodiments, the data associated with one or more tasks includes one or more work parameters and / or work constraints. The one or more tasks to be performed may correspond to the work to be performed.

[0152] In step 720, the asset group is determined. In some embodiments, the server determines the asset group to collectively execute one or more tasks, or an operation containing one or more tasks. The asset group may be determined based on one or more functions associated with one or more tasks / operations, and / or the functions of the assets. For example, the server may determine an asset group having functions that match the functions associated with one or more tasks / operations.

[0153] In various embodiments, asset groups may be determined at least in part on the availability of assets and / or the availability of ownership (or transfer of ownership) of assets. For example, a server may determine a superset of assets (e.g., all asset groups across multiple organizations) and query assets or third-party services for indications of whether assets are available and / or their availability for transfer of ownership for use in performing work. The third-party service may be an ownership service that manages ownership of assets across multiple organizations and / or transfers or licenses of ownership according to negotiated terms (e.g., fixed-term lease, lease for a specific task, lease for a specific job, perpetual transfer, etc.).

[0154] Depending on whether it is determined that the asset group will perform one or more tasks (or corresponding work), process 700 proceeds to step 730 in which an instruction is communicated to at least one drone in the asset group. In some embodiments, the server determines a leader drone in the asset group and communicates an instruction to the leader drone. The instruction may include, or be communicated in connection with, an indication that a leader drone is in the asset group. In some embodiments, the instruction communicated to the leader drone includes work parameters. The instruction may also include a high-level preplanning of the work if the server or other service has generated such preplanning.

[0155] In response to having communicated a command to at least one drone, process 700 proceeds to process 740, where a determination is made as to whether the process is complete. For example, the process may be determined to be complete based at least in part on user input (such as input to cancel or pause the work). For another example, the process may be determined to be complete in response to the user choosing to start execution of the work. If the process is deemed complete, process 700 terminates. Otherwise, process 700 returns to process 710, where further data associated with one or more tasks is received. In some embodiments, process 700 may be deemed incomplete in process 740 in response to user input for further refining / configuring the work. For example, before starting the work, the user may input a request to edit the work into the user interface. For another example, while the asset group is performing the work, the user may input choices to modify the work, such as dynamically adding assets to the asset group to accelerate the completion of the work, or removing assets based on reprioritizing the work for another work.

[0156] If the asset set is not determined in step 720, process 700 may proceed to step 740. The asset set cannot be determined if, for example, there are no available assets, ownership of assets matching the functionality of one or more tasks or work is not available, or no asset set of functionality collectively matching the functionality of the work can be found. In step 740, the user may be given a prompt indicating that the asset set cannot be determined, and may be asked to modify the work configuration or to ask whether the user wants to cancel the work.

[0157] Figure 7B shows a method for configuring the work according to various embodiments of the present application. In some embodiments, the process 720 in Figure 7B is performed by the server 105 of the system 100 in Figure 1 and / or the device 200 in Figure 2. The process 720 in Figure 7B may correspond to the process 720 in Figure 7A.

[0158] In step 721, one or more features of one or more tasks are obtained. In some embodiments, the server may obtain one or more features based on the definition of the work (e.g., work parameters).

[0159] In step 722, one or more functions of the asset are retrieved. In some embodiments, the server may retrieve one or more functions of the asset based on a mapping of functions to the asset. The mapping of asset functions may be determined by querying the asset for suggestions of its functions (or their current functions), by the asset registering / notifying of those functions, etc.

[0160] In step 723, a determination is made as to whether any asset matches the characteristics of the task. In some embodiments, the server determines one or more functions corresponding to the characteristics of the task and determines an asset that has functions that match the functions corresponding to the characteristics of the task. Depending on whether it is determined that an asset or group of assets has functions that match the characteristics of the task, process 720 may proceed to step 727.

[0161] In step 724, the asset group is determined. In some embodiments, the server determines the asset group from assets having functions that match the characteristics of one or more tasks. The server may select assets to be included in the asset group based on the availability of the assets, the ownership of the assets, the availability of transferring ownership of the assets, and whether the asset group has redundancy in terms of functionality. In some embodiments, the server uses a cost function in connection with selecting assets to be included in the asset group. For example, the server may use a cost function to optimize the cost of the asset group or select an asset group with a cost lower than a threshold cost value.

[0162] Depending on the determination of the asset group, process 720 proceeds to process 725, in which a determination is made as to whether the determination of the asset group is complete. For example, the server may prompt the user to review the asset group, or it may provide selectable elements that allow the user to choose to improve / modify one or more tasks or one or more functions of a task. Depending on whether it is determined that the determination of the asset group is not complete, process 720 returns to process 721. Otherwise, process 720 proceeds to process 726. If no assets matching the characteristics of the task are found in process 723 (or if no assets with corresponding functions are found for each characteristic of the task), process 720 may proceed to process 725.

[0163] Step 726 provides an indication of an asset group and / or an indication that an asset group is possible (for example, that the asset group may be grouped in relation to performing work).

[0164] Figure 8A shows a method for performing at least one task of work according to various embodiments of this specification. In some embodiments, the process 800 in Figure 8A is performed by assets 120, 125, and / or 130 of the system 100 in Figure 1, and / or the device 300 in Figure 3. According to various embodiments, the process 800 is performed by a semi-autonomous drone.

[0165] In some embodiments, the process 800 is performed by a leader drone in the asset group to carry out the task. In some embodiments, the process 800 is performed by a follower drone in the asset group to carry out the task.

[0166] In step 810, an indication is received that the drone is part of a set of assets that perform elements of the task. The indication may also be received that the drone is part of a set of assets associated with the work.

[0167] According to various embodiments, when the process 800 is performed by a leader drone, the leader drone may receive an indication from a server (e.g., a server that constitutes the work) that the drone is part of an asset group, along with work parameters associated with the work on which the elements of the task are performed.

[0168] According to various embodiments, when processing 800 is performed by a follower drone, the follower drone may receive an indication from the leader drone that the drone is part of an asset group.

[0169] In step 820, information about one or more elements of one or more tasks is communicated. The information about one or more elements may include the parameters of the elements and / or constraints for performing the elements.

[0170] According to various embodiments, when the process 800 is performed by a leader drone, the leader drone may communicate a plan for performing one or more elements. The leader drone may communicate the plan for performing one or more elements to one or more follower drones assigned to perform one or more elements.

[0171] According to various embodiments, when the process 800 is performed by a follower drone, the follower drone may receive a plan for performing one or more elements. The follower drone may receive the plan from the leader drone. The plan may be determined based on the capabilities of the follower drone, the capabilities associated with the elements, the status of the follower drone (e.g., availability, location, etc.).

[0172] In step 840, a determination is made as to whether or not process 800 is complete. If it is determined that process 800 is complete, process 800 terminates. If it is determined that process 800 is not complete, process 800 may return to step 810. According to various embodiments, process 800 is determined to be complete if it is determined that one or more elements have been completed, or that the work has been completed. Process 800 may also be determined to be complete if the work has been temporarily suspended or canceled / stopped.

[0173] Figure 8B shows a method for performing at least one task of work according to various embodiments of this specification. In some embodiments, process 830 in Figure 8B is performed by assets 120, 125, and / or 130 of system 100 in Figure 1, and / or device 300 in Figure 3. According to various embodiments, process 830 is performed by a semi-autonomous drone. Process 830 in Figure 8B may correspond to step 830 of process 800 in Figure 8A. Process 830 in Figure 8B may be performed by a follower drone in a group of assets assigned to perform work.

[0174] In step 831, data is received from one or more sensors on the drone. In some embodiments, during the execution of a plan (such as a plan provided by the leader drone), the follower drone acquires information about elements / tasks. For example, the follower drone may capture information such as images, live stream video, and weather information.

[0175] In step 832, a determination is made as to whether the data is relevant to the plan. In some embodiments, upon receiving data from sensors on the drone, the drone determines whether such data is relevant to the plan to be implemented. For example, in a plan to track a target, the data received from the sensors may include images. The drone may determine whether the images include the target being tracked by the drone. For example, the drone may perform image analysis to determine whether the target is in the image (or whether the likelihood of the image containing the target exceeds a predetermined threshold).

[0176] Depending on whether the data is determined to be related to the plan, process 830 may proceed to step 833, in which information relating to the plan is transmitted to the leader drone. In some embodiments, the follower drone may determine that the captured information is related to the plan (e.g., including information indicating the status of elements / work, including the target / object of the plan, including information relating to the geographical area in which the plan will be implemented, etc.).

[0177] In response to transmitting information about the plan to the leader drone, process 830 proceeds to step 834, in which a determination is made as to whether or not process 830 is complete. In some embodiments, process 830 is determined to be complete in response to a determination that the work is completed, aborted, and / or paused. In response to a determination that process 830 is complete, process 830 terminates. In response to a determination that process 830 is not complete, process 830 may return to step 831, in which the drone monitors the data / information captured by its sensors and continues to repeatedly transmit the associated information to the leader drone. For example, process 830 is determined to be incomplete in response to a determination that the task is not completed (for example, if a drone is tasked with scanning a road and the drone is unable to maintain a series of sharp curves on the road and misses scanning part of the road, the task is considered incomplete and the drone may turn back and rescan the missed portion of the road).

[0178] If process 832 determines that the data is not relevant to the plan, process 830 proceeds to process 834, which is executed as described above.

[0179] Figure 8C shows a method for performing at least one task of work according to various embodiments of this specification. In some embodiments, process 830 in Figure 8C is performed by assets 120, 125, and / or 130 of system 100 in Figure 1, and / or device 300 in Figure 3. According to various embodiments, process 830 is performed by a semi-autonomous drone. Process 830 in Figure 8C may correspond to step 830 of process 800 in Figure 8A. Process 830 in Figure 8C may be performed by a leader drone in a group of assets assigned to perform work.

[0180] In step 835, feedback information is received from one or more follower drones, or control information is received. According to various embodiments, the leader drone receives feedback information from the follower drones simultaneously with the execution of a plan regarding the task or elements of the task associated with the work.

[0181] Feedback information may include current / contemporaneous information such as current location, live video stream, and current status of the work (for example, in the case of a monitoring task, the status could be that the target is not present within the defined area of ​​the work).

[0182] Control information may include information provided from the server to the leader drone. For example, control information may include indications that work has been updated, work parameters, work constraints, instructions / commands to modify asset groups, etc.

[0183] In step 836, a decision is made as to whether or not to update the plan. According to various embodiments, the leader drone autonomously decides whether or not to update the plan based on feedback information or control information. For example, if the feedback information indicates the completion of an element of work or task, the leader drone may update the plan for the corresponding follower drone so that the follower drone performs the new task / element. As another example, in the case of monitoring within a defined area, in response to a determination that the feedback information indicates that the target is within the defined area, the leader drone may decide to update the plan to move closer to the target in order to obtain specific details about the target, or the leader drone may decide to update the plan to instruct the follower drone to switch to a tracking task to track the target.

[0184] Depending on the decision in step 836 that the plan is to be updated, process 830 may proceed to step 837 in which the updated plan is sent to one or more follower drones, and then process 830 proceeds to step 838. Conversely, depending on the decision in step 836 that the plan is not to be updated, process 830 proceeds to step 838.

[0185] In step 838, a determination is made as to whether or not process 830 is complete. In some embodiments, process 830 is determined to be complete if it is determined that the work is completed, stopped, and / or paused. If it is determined that process 830 is complete, process 830 terminates. If it is determined that process 830 is not complete, process 830 may return to step 835, in which the drone continues to monitor for feedback or control information regarding the work.

[0186] Figure 9A shows a method for carrying out a plan associated with a task according to various embodiments of the present application. In some embodiments, the process 900 in Figure 9A is performed by assets 120, 125, and / or 130 of the system 100 in Figure 1, and / or the device 300 in Figure 3. According to various embodiments, the process 800 is performed by a semi-autonomous drone.

[0187] In step 910, an indication is received that the drone is part of a set of assets that perform elements of the task. The indication may also be received that the drone is part of a set of assets associated with the work that the drone constitutes.

[0188] According to various embodiments, if the process 900 is performed by a leader drone, the leader drone may receive an indication from a server (e.g., a server that constitutes the work) that the drone is part of an asset group, along with work parameters associated with the work on which the elements of the task are performed.

[0189] According to various embodiments, when processing 900 is performed by a follower drone, the follower drone may receive an indication from the leader drone that the drone is part of an asset group.

[0190] In step 920, the plan is implemented. In some embodiments, the drone acquires a plan associated with performing the work and then implements the plan.

[0191] In various embodiments, when processing 900 is performed by a leader drone, the leader drone may receive control information from a server (such as the server that configured the work). The control information may include work parameters, constraints for performing the work, a pre-plan, any combination thereof, etc. Based on high-level information for the work, the leader drone may determine a plan for performing the work. For example, the leader drone may determine a plan for one or more follower drones to perform a task or element of the work and instruct the follower drones to implement the plan for performing the task / element.

[0192] According to various embodiments, when the process 900 is performed by a follower drone, the follower drone may implement a plan for performing the task / element. The follower drone may receive the plan from the leader drone. In some embodiments, the follower drone may update the plan to fill in any gaps in the plan received from the leader drone. For example, the follower drone may modify the plan to have a finer granularity than the plan received from the leader drone.

[0193] In step 930, a determination is made as to whether the asset has failed. According to various embodiments, the determination of whether the asset has failed may be based on the time elapsed since the asset last communicated with the drone, or on the absence of a response to a ping for the asset's health status. If it is determined that the asset has failed, process 900 proceeds to step 940.

[0194] In step 940, the plan is updated. According to various embodiments, the plan is updated to take into account the determination that an asset has failed.

[0195] According to various embodiments, if the process 900 is performed by a leader drone, the leader drone may determine the status of the work, such as the status of tasks / elements assigned to the asset that has been determined to be faulty. Based on the status of the tasks / elements, the leader drone may update the plan to reallocate any remaining assignments (e.g., incomplete portions of tasks / elements assigned to an asset) to one or more remaining assets in the group of assets assigned to perform the work.

[0196] In various embodiments, if process 900 is performed by a follower drone, the follower drone may update the plan in response to a determination that the leader drone has failed. In response to a determination that the leader drone has failed, the follower drone may negotiate with one or more other remaining assets in the asset group to determine a new leader. In response to the leader drone being promoted to a new leader drone, the new leader drone may update the plan based on its position as the new leader drone. For example, the new leader drone may determine one or more plans for the remaining assets. The new leader drone may determine the plans based on the functionality of each of the remaining assets and the functionality or features associated with any tasks / elements that have not been completed in the execution of the work.

[0197] In step 950, information regarding the updated plan is communicated. In some embodiments, the updated plan is communicated to the remaining assets.

[0198] In process 960, information is communicated during the implementation of the updated plan.

[0199] In various embodiments, when process 900 is performed by a leader drone, the leader drone may receive information from follower drones / assets or a server. For example, the leader drone may receive control information from a server (such as the server that configured the work). This control information may include updated work parameters, constraints for performing the work, a pre-plan, and any combination thereof. In another example, the leader drone may receive feedback information about the work from the follower drones, such as feedback information acquired by the follower drones during the execution of the updated plan.

[0200] According to various embodiments, when processing 900 is performed by a follower drone, the follower drone transmits feedback information to the leader drone.

[0201] In step 970, a determination is made as to whether or not process 900 is complete. If it is determined that process 900 is complete, process 900 terminates. If it is determined that process 900 is not complete, process 90 may return to step 920. According to various embodiments, process 900 is determined to be complete if it is determined that one or more elements have been completed, or that the work has been completed. Process 900 may also be determined to be complete if the work has been temporarily suspended or canceled / stopped. If it is determined in step 930 that no asset failure has occurred, process 900 may proceed to step 970.

[0202] Figure 9B shows a method for carrying out a plan associated with a task according to various embodiments of the present application. In some embodiments, process 930 in Figure 9A is performed by assets 120, 125, and / or 130 of system 100 in Figure 1, and / or device 300 in Figure 3. Process 930 in Figure 9B may correspond to step 930 of process 900 in Figure 9A.

[0203] In step 931, feedback information is received from one or more follower drones, or control information is received. In some embodiments, the leader drone may receive feedback information from the follower drones while simultaneously implementing a plan for the follower drones to perform a task / work. In some embodiments, the leader drone may receive control information (e.g., updated control information) from a server. For example, the control information may include indications that work has been updated, work parameters, work constraints, instructions / commands to modify asset sets, etc.

[0204] In step 932, a determination is made as to whether or not feedback has been received from an asset in the asset group. In some embodiments, the leader drone performs the process 930 in Figure 9B for each asset in the asset group. For example, the leader drone may determine whether or not feedback information has been received from a particular follower drone. In some embodiments, the determination as to whether or not feedback information has been received from a particular asset may be based on the time elapsed since the particular asset last communicated feedback information. In some embodiments, a determination is made as to whether or not feedback information has been received from a particular asset within a predetermined threshold time. In some embodiments, a determination is made as to whether or not feedback information has been received from a particular asset by a predetermined time (e.g., an estimated time (which may be determined at least in part on plans being implemented by the asset, the asset's location, communication links with the asset, etc.)).

[0205] In response to the determination in step 932 that feedback information has been received from an asset in the asset group, process 930 proceeds to step 936. In step 936, a determination is made as to whether or not process 930 is complete. In response to the determination that process 930 is complete, process 930 proceeds to step 937. In response to the determination that process 930 is not complete, process 930 may return to step 931. According to various embodiments, process 930 is determined to be complete in response to the determination that one or more elements have been completed, or that the work has been completed. Process 930 may also be determined to be complete if the work has been paused or canceled / stopped. In step 937, an indication is provided as to whether or not the asset has failed. In some embodiments, a module that determines whether or not the asset has failed provides an indication as to whether or not the asset has failed to a planning service of the leader drone (e.g., a service that determines / updates a plan for performing one or more tasks associated with the work).

[0206] If the process in step 932 determines that no feedback information has been received from an asset in the asset group, process 930 proceeds to step 933. In step 933, a ping is sent to the asset. The leader drone may send a ping as a health check of the asset.

[0207] In step 934, a determination is made as to whether a response has been received from the asset. In some embodiments, the leader drone determines whether a response has been received based on a determination of whether the asset responded to the ping within a threshold period. If it is determined that a response (to the ping) has been received from the asset, process 930 proceeds to step 936. Conversely, if it is determined that a response (to the ping) has not been received from the asset, process 930 proceeds to step 935, in which it is determined that the asset has failed. Then, process 930 proceeds to step 936.

[0208] Figure 9C shows a method for carrying out a plan associated with a task according to various embodiments of the present application. In some embodiments, process 950 in Figure 9C is performed by assets 120, 125, and / or 130 of system 100 in Figure 1, and / or device 300 in Figure 3. Process 950 in Figure 9C may correspond to process 950 in Figure 9A. In some embodiments, process 950 is performed by a semi-autonomous leader drone.

[0209] In step 951-1, feedback information regarding the execution of the plan is received. According to various embodiments, the leader drone receives feedback information from the follower drone simultaneously with the execution of the plan regarding the tasks or elements of the tasks associated with the work. The feedback information may include current / contemporaneous information such as the current location, live video stream, and the current status of the work (for example, in the case of a monitoring task, the status may be that the target is not present within the defined area of ​​the work).

[0210] In step 951-2, a decision is made as to whether or not to update the plan. In some embodiments, the leader drone decides whether or not to update the plan (e.g., a plan for a specific follower drone or a plan for multiple follower drones) based at least in part on feedback information.

[0211] In response to the decision that the plan will not be updated, process 950 proceeds to step 951-7, in which a determination is made as to whether or not process 950 is complete. In response to the determination that process 950 is complete, process 950 terminates. In response to the determination that process 950 is not complete, process 950 may return to step 951-1. According to various embodiments, process 950 is determined to be complete in response to the determination that one or more elements have been completed, or that the work has been completed. Process 950 may also be determined to be complete if the work has been paused or canceled / stopped.

[0212] In response to the decision to update the plan, process 950 proceeds to step 951-3, in which a determination is made as to whether the plan can be updated. In some embodiments, the leader drone determines whether the plan can be updated based on the parameters of the work, the status of the work, the functionality of one or more assets in the asset group, etc. For example, if it is decided that the work will be updated, or if the functionality required to perform the work has changed or been newly determined, the leader drone may determine whether the asset group includes assets that have the required functionality.

[0213] In response to the determination in step 951-3 that the plan cannot be updated, process 950 proceeds to step 951-5, which indicates that the plan cannot be updated. In some embodiments, the leader drone may provide a prompt to the user via the user interface or to a server (which may provide the prompt on a client terminal) indicating that the plan cannot be updated. The prompt may include elements that allow the user to cancel the work, modify the work, etc. Process 950 then proceeds to step 951-7.

[0214] In response to the determination in step 951-3 that the plan can be updated, process 950 proceeds to step 951-4 in which the plan is updated. In some embodiments, the leader terminal updates the plan accordingly (for example, at least in part on feedback information). The plan may be updated at least in part on the capabilities of the follower drone.

[0215] In step 951-6, the updated plan is transmitted to the follower drone. Process 950 may then proceed to step 951-7.

[0216] Figure 9D shows a method for carrying out a plan associated with a task according to various embodiments of the present application. In some embodiments, process 951-2 in Figure 9D is performed by assets 120, 125, and / or 130 of system 100 in Figure 1, and / or device 300 in Figure 3. Process 951-2 in Figure 9D may correspond to process 951-2 in Figure 9C. In some embodiments, process 951-2 is performed by a semi-autonomous leader drone.

[0217] In step 951-A, a determination is made as to whether or not an asset failure has occurred. According to various embodiments, the determination of whether or not an asset has failed may be based on the time elapsed since the asset last communicated with the drone, or on the absence of a response to a ping for the asset's health status.

[0218] Depending on the asset's condition, process 951-2 proceeds to process 951-B, in which a determination is made as to whether the asset's failure affects the execution of the work. The leader drone may determine whether the failed asset had any outstanding tasks or elements that affected the work.

[0219] If it is determined that the asset failure does not affect the execution of the work, process 951-2 may proceed to process 951-G, in which a decision is made not to update the plan. Then, process 951-2 may proceed to process 951-H, in which a determination is made as to whether or not process 951-2 is complete. If it is determined that process 951-2 is complete, process 951-2 terminates. If it is determined that process 951-2 is not complete, process 951-2 may return to process 951-A. According to various embodiments, process 951-2 is determined to be complete if it is determined that one or more elements have been completed, or that the work has been completed. Process 951-2 may also be determined to be complete if the work has been paused or canceled / stopped.

[0220] If it is determined that the asset failure will not affect the execution of the work, process 951-2 may proceed to process 951-C, in which a decision is made to update the plan. Then, process 951-2 may proceed to process 951-H, in which it is determined whether or not process 951-2 is complete. If it is determined that process 951-2 is complete, process 951-2 terminates.

[0221] If the determination is made that the asset is not faulty, process 951-2 proceeds to process 951-D. In process 951-D, a determination is made as to whether the plan has failed. In some embodiments, the leader drone may determine whether the plan has failed based on the determination as to whether the asset is faulty. For example, if the plan had a corresponding function (e.g., a required function), and the faulty asset had a function for which the asset group did not have redundancy in at least one other asset (and it is determined that a new asset / function cannot be acquired), the leader drone may determine that the plan has failed.

[0222] If the plan is determined to have failed, process 951-2 proceeds to process 951-E, in which a determination is made as to whether the work should be aborted (or suspended). The leader drone may decide to abort the work if it is determined that the plan is a necessary component of the work (including tasks / elements required by the work). The leader drone may decide to abort the work at least in part on the fact that the work has a corresponding function (e.g., a required function) and the failed asset had a function that the asset group does not have redundancy in at least one other asset (and it is determined that a new asset / function cannot be acquired).

[0223] If the operation is stopped, process 951-2 proceeds to process 951-H, according to the decision made in process 951-E.

[0224] Depending on the determination in step 951-D that no plan failure occurred, or the decision in step 951-E that the work will not be stopped, process 951-2 proceeds to step 951-F, where it is determined that the plan will be updated.

[0225] Figure 9E shows a method for carrying out a plan associated with a task according to various embodiments of the present application. In some embodiments, process 950 in Figure 9E is performed by assets 120, 125, and / or 130 of system 100 in Figure 1, and / or device 300 in Figure 3. Process 950 in Figure 9E may correspond to process 950 in Figure 9A. In some embodiments, process 950 is performed by a semi-autonomous follower drone.

[0226] In step 952-1, data is received from one or more sensors on the drone. In some embodiments, during the execution of a plan (such as a plan provided by the leader drone), the follower drone acquires information about elements / tasks. For example, the follower drone may capture information such as images, live stream video, and weather information.

[0227] In step 952-3, a determination is made as to whether the data is relevant to the plan. In some embodiments, upon receiving data from sensors on the drone, the drone determines whether such data is relevant to the plan to be implemented. For example, in a plan to track a target, the data received from the sensors may include images. The drone may determine whether the images include the target being tracked by the drone. For example, the drone may perform image analysis and determine whether the target is in the image (or whether the likelihood of the image containing the target exceeds a predetermined threshold).

[0228] Depending on whether the data is determined to be related to the plan, process 950 may proceed to step 952.5, in which information relating to the plan is transmitted to the leader drone. In some embodiments, the follower drone may determine that the captured information is related to the plan (e.g., including information indicating the status of elements / work, including the target / object of the plan, including information relating to the geographical area in which the plan will be implemented, etc.).

[0229] In response to transmitting information about the plan to the leader drone, process 950 proceeds to step 952-7, in which a determination is made as to whether or not process 950 is complete. In some embodiments, process 950 is determined to be complete in response to a determination that the work is completed, aborted, and / or paused. In response to a determination that process 950 is complete, process 950 terminates. In response to a determination that process 950 is not complete, process 950 may return to step 952-1, in which the drone monitors the data / information captured by its sensors and continues to repeatedly transmit the associated information to the leader drone.

[0230] If process 952-3 determines that the data is not relevant to the plan, process 950 proceeds to process 952-1, which is executed as described above.

[0231] Figure 10A shows a method for carrying out a plan associated with a task according to various embodiments of the present application. In some embodiments, the process 1000 in Figure 10A is performed by the server 105, asset 120, asset 125, and / or asset 130 of the system 100 in Figure 1, the device 200 in Figure 2, and / or the device 300 in Figure 3. According to various embodiments, the process 1000 is performed by a semi-autonomous drone or a server that manages the task.

[0232] In step 1010, data associated with one or more tasks being performed by the asset group is acquired. According to various embodiments, the data associated with one or more tasks may include information regarding updates to the configuration / definition of the work, or feedback information acquired and communicated during the execution of the task or elements of the task. Updates to the configuration / definition of the work may be at least in part based on one or more user inputs in response to user requests to modify the work (for example, to accelerate the completion of the work, to change the priority of the work, etc., for at least one other work). Feedback information may include changes in the status of the work, such as the fulfillment of conditions that trigger one or more other tasks. For example, in the context of a monitoring work in a defined geographical area, the work may be defined to trigger a tracking task when a target object / vehicle is detected within the geographical area. Thus, during a monitoring task by one or more follower drones, if a target is found within the geographical area, the follower drones may provide feedback information to the leader drone to indicate the sighting. In another example, the feedback information may include an indication that a follower drone has malfunctioned or been destroyed. Failure or destruction of a follower drone can reduce the functionality of the asset group or the redundancy of its functionality.

[0233] According to various embodiments, when the process 1000 is performed by a leader drone, the leader drone may receive data associated with one or more tasks from a server via control information, etc., or from one or more follower drones via feedback information, etc., transmitted simultaneously with the follower drones carrying out the plan associated with the execution of work (e.g., the execution of one or more tasks).

[0234] According to various embodiments, when the process 1000 is performed by a server, the server may receive data associated with one or more tasks from the server, such as through user input to a user interface. The user may input commands or requests to the user interface that may modify the work or modify the parameters of the work.

[0235] In step 1020, a decision is made as to whether or not to modify the asset group. According to various embodiments, the decision to modify the asset group may be based at least in part on data associated with one or more tasks being performed by the asset group (e.g., work data relating to the current state or characteristics of the work). The decision to modify the asset group may include a decision to add one or more assets to the asset group. Similarly, the decision to modify the asset group may include a decision to remove one or more assets from the asset group. For example, the device may decide, at least in part, that a new task should be performed. As another example, the new task may include or require a function that the current asset group does not include an asset with the corresponding function. As yet another example, the device may determine that the asset group has lost a function or redundancy in a function due to an asset failure. Redundancy in a function may allow multiple tasks that require that function to be performed (e.g., in parallel) by multiple assets having that function. Loss of functional redundancy can reduce the number of tasks that can run concurrently, or reduce the flexibility with which the leader drone can assign or plan tasks across asset groups.

[0236] According to various embodiments, the decision to modify an asset group may be based on a decision to add functionality to the asset group, and / or a determination that the functionality of the asset group is no longer required (or is expected not to be required) for the current work. An asset group may be decided to be modified to add one or more assets, remove one or more assets, or both add one or more assets and remove one or more assets. In some embodiments, the decision to modify an asset group may be based on the cost associated with using the asset group to perform the work or the remainder of the work (e.g., the cost determined by a given cost function). Deciding whether to modify an asset group during the execution of work enables dynamic resource management, which can, for example, ensure that the appropriate functionality is assigned to the asset group. Furthermore, dynamic resource management can enable optimization or improvement of the cost associated with performing the work.

[0237] In response to the decision in step 1020 to modify the asset group, process 1000 proceeds to step 1030, where instructions indicating the modification of the asset group are communicated. The instructions may be provided to the affected assets (e.g., assets to be added to or removed from the group) and / or to the leader asset of the corresponding asset group.

[0238] In various embodiments, when process 1000 is performed by a leader drone, the leader drone may send instructions to modify the assets affected by the modification, and / or to a server (such as a server that manages or configures the work). In some embodiments, the leader drone may send instructions or suggestions for modification to an ownership service that manages ownership of assets across multiple organizations. For example, instructions may be sent to the ownership service in connection with the negotiation of ownership of the assets affected by the modification.

[0239] In various embodiments, when process 1000 is performed by a server, the server communicates instructions to the leader drone and / or to the assets affected by the modification to modify the asset group. The server may also provide a suggestion to the user via a user interface on a client terminal, where the suggestion notifies the user that the asset group will be modified.

[0240] In response to the communication of instructions indicating the modification of the asset group, process 1000 proceeds to step 1040, in which a determination is made as to whether or not process 1000 is complete. In some embodiments, process 1000 is determined to be complete in response to a determination that the work is completed, aborted, and / or paused. In response to the determination that process 1000 is complete, process 1000 terminates. In response to a determination that process 1000 is not complete, process 1000 may return to step 1010, in which the drone monitors the data / information captured by its sensors and continues to repeatedly transmit the associated information to the leader drone. Furthermore, in response to the determination in step 1020 that the asset group is not modified, process 1000 may proceed to step 1040.

[0241] Figure 10B shows a method for carrying out a plan associated with a task according to various embodiments of the present application. In some embodiments, process 1020 in Figure 10B is performed by server 105, asset 120, asset 125, and / or asset 130 of system 100 in Figure 1, device 200 in Figure 2, and / or device 300 in Figure 3. Process 1020 in Figure 10B may correspond to step 1020 in Figure 10A.

[0242] In step 1021, one or more features of one or more remaining tasks are obtained. In some embodiments, a device (e.g., a server, or a drone such as a leader drone) may determine the remaining tasks for work and determine one or more features associated with those remaining tasks. The one or more features associated with the remaining tasks may be determined based on a mapping of tasks to features. For example, the device may use the remaining tasks to query the mapping of tasks to features.

[0243] In step 1022, one or more functions of the asset are acquired. In some embodiments, the device may acquire one or more functions of the asset based on a mapping of functions to the asset. The mapping of asset functions may be determined by querying the asset for suggestions of its functions (or their current functions), by the asset registering / notifying of those functions, etc.

[0244] In step 1023, a determination is made as to whether any asset matches the characteristics of the remaining tasks. In some embodiments, the server determines one or more functions corresponding to the characteristics of the tasks and determines assets that have functions that match the functions corresponding to the characteristics of the tasks. Depending on whether it is determined that an asset or group of assets has functions that match the characteristics of the remaining tasks, process 1020 may proceed to step 1024.

[0245] In step 1024, modifications to the asset group are determined. In some embodiments, the device determines an asset group from assets having functions that match the characteristics of one or more remaining tasks. In some embodiments, the device determines modifications to the asset group based on the fact that the assets in the asset group have a set of functions that do not match the characteristics of one or more remaining tasks (e.g., there is no overlap between the functions of the excluded assets and the characteristics / functions of the remaining tasks). The device may select assets to be added to the asset group based on the availability of the assets, the ownership of the assets, the availability of transferring ownership of the assets, a determination that the asset group has redundancy in terms of functions, etc. In some embodiments, the device uses a cost function in relation to selecting assets to include in or exclude from the asset group. For example, the server may use a cost function to optimize the cost of the asset group or select an asset group with a cost lower than a threshold cost value.

[0246] Depending on the determination of the asset group, process 1020 proceeds to step 1025, in which a determination is made as to whether the determination of the asset group is complete. For example, the device may prompt the user to review the asset group, or it may provide selectable elements that allow the user to choose to improve / modify one or more tasks or one or more functions of a task. Depending on whether it is determined that the determination of the asset group is not complete, process 1020 returns to step 1021. Otherwise, process 1020 proceeds to step 1026. If, in step 1023, no assets matching the characteristics of the remaining tasks are found (or, for each characteristic of the tasks, no assets with corresponding functions are found), process 1020 may proceed to step 1025. In step 1026, an indication is provided as to whether or not to modify the asset group. In some embodiments, the indication as to whether or not to modify the assets is provided to a planning service running on the leader drone. In some embodiments, the indication as to whether or not to modify the assets is provided to a control service located on a server or similar.

[0247] Figure 11 shows a method for configuring work according to various embodiments of the present application. In some embodiments, process 1000 in Figure 10A is performed by the server 105, asset 120, asset 125, and / or asset 130 of system 100 in Figure 1, and / or device 200 or Figure 2. According to various embodiments, process 1100 is performed by a server (such as a server that configures and / or manages work). Process 1000 may be performed by a server together with a client terminal.

[0248] In step 1105, a first user interface is displayed. In some embodiments, the server causes the client terminal to display the first user interface. The first user interface may correspond to an interface used when one or more features or parameters associated with a task are entered by the user. For example, the first user interface may be used in connection with defining a task. The user may enter one or more features or parameters through one or more selectable elements provided on the user interface.

[0249] In step 1110, user selections regarding the characteristics of the task to be performed are received. User selections may be made to a client terminal, which may communicate the user selections to the server via one or more networks.

[0250] In step 1115, a configuration for another user interface is determined, at least in part, based on the user selection. In some embodiments, the server determines one or more features, parameters, or constraints that are mapped to the user selection (e.g., mapped to the characteristics of a task). For example, if the user selection to the first user interface is a selection of the type of work (e.g., monitoring), the server may determine another user interface to include definitions of the monitoring conditions, such as time, date, geographical location, and target type (e.g., moving target, stationary target, etc.). As another example, if the user selection to the first user interface is a selection of the type of work and / or the conditions of the work (e.g., monitoring), the server may determine another user interface to include definitions of one or more features relating to the assets used in connection with performing the work.

[0251] In step 1120, a different user interface is displayed. In some embodiments, depending on the configuration of the different interface determined, the server causes the client terminal to display the different user interface.

[0252] In step 1125, a user selection is received regarding the set of assets to be deployed to perform the task. The user selection may be a selection of specific assets or types of assets to be deployed to perform the task. In some embodiments, the user selection regarding the set of assets to be deployed to perform the task includes a user selection for one or more parameters of the task used by the server when determining the set of assets or types of assets to deploy. The user selection may be made to a client terminal, which may communicate the user selection to the server over one or more networks.

[0253] In step 1130, a decision is made whether or not to provide an alternative user interface for configuring the task. The decision to provide an alternative user interface may be based at least in part on the parameters of the task and / or asset set obtained in the first user interface and the alternative user interface described above. In some embodiments, the decision to provide an alternative user interface may be based on user input such as canceling or starting the task. For example, the server may fill in gaps in the task definition and deploy the corresponding asset set. As an example, the server may determine any remaining parameters of the task based at least in part on one or more of the following: (i) historical information such as information on similar / past tasks, (ii) best fit associated with the user-defined parameters of the task, and (iii) the cost of executing the task determined using a cost function.

[0254] Depending on whether it is determined that a different user interface is to be provided to configure the work, process 1100 proceeds to step 1135 which displays the different interface. The different interface may be configured at least in part on user selections regarding the characteristics of the task to be performed and / or user selections regarding the set of assets to be deployed.

[0255] In step 1140, a user selection to another user interface is received. The user selection may be further input regarding the definition or configuration of a task. The user selection may be made to a client terminal, which may communicate the user selection to the server via one or more networks. The process 1100 may then proceed to step 1160.

[0256] In step 1160, a determination is made as to whether the configuration of the work has been completed. For example, the device may prompt the user to review the asset group and confirm the configuration of the work, or it may provide selectable elements that allow the user to choose to improve / modify one or more tasks or one or more functions of a task. Depending on whether it is determined that the configuration of the work has not been completed, process 1100 returns to step 1130, in which the process iteratively configures the user interface based on user input to previous user interfaces, or one or more previous user inputs to user interfaces. Otherwise, process 1100 terminates.

[0257] Depending on the decision in step 1130 that no user interface associated with configuring the work is provided, process 1100 may proceed to step 1145 in which the work to be performed is determined (defined) on at least partly based on user selection. Other variables, such as historical information (e.g., information on similar work) and the cost of performing the work, may be used in connection with determining the work (e.g., the work may be defined to minimize costs or to reduce the cost of performing the work to below a specified threshold cost).

[0258] In step 1150, information regarding the work is communicated. In some embodiments, the server communicates work suggestions to one or more assets. For example, the server may communicate work suggestions to a leader drone of a group of assets that have been determined to perform the work. The work suggestions may be instructions for the leader drone or the group of assets to perform the work. In some embodiments, the work suggestions may include one or more of the following: work parameters, work constraints, asset group suggestions, etc.

[0259] In step 1155, a user interface for performing the work is displayed. In some embodiments, the server causes the client terminal to display a user interface for displaying the work in progress. The user interface for performing the work may provide information about the current status of the work, such as a live video stream, suggestions for completed tasks, suggestions for tasks not yet performed, and suggestions for tasks currently being performed. In some embodiments, the user interface for performing the work includes information about asset groups. For example, the user may choose to drill down to see more granular information about asset groups, such as the status of the asset (e.g., offline, failed, online, functioning normally, partially functioning, etc.), a list of functions for the asset or asset group, and the current set of functions for the asset or asset group. In some embodiments, the user interface for performing the work includes one or more selectable elements into which the user can enter user input. Examples of user input may include pausing the work, canceling the work, modifying the work (e.g., modifying the parameters of the work), modifying the asset group, etc.

[0260] According to various embodiments, the server configures and provides a set of user interfaces, at least a portion of which are logically connected based at least partially on user input in another user interface. For example, a set of user interfaces may be configured and provided to a user to prompt the user to input information about a configured task. The information entered by the user may include parameters or characteristics of the task, the type of task, the type of assets used, and so on. In some embodiments, a user interface in a set of user interfaces used to configure a task is based at least partially on one or more user inputs to the preceding user interface in the set of user interfaces.

[0261] Figure 12A shows a method for carrying out a plan associated with a task according to various embodiments of the present application. In some embodiments, the process 1200 in Figure 12A is performed by assets 120, 125, and / or 130 of the system 100 in Figure 1, and / or the device 300 in Figure 3. According to various embodiments, the process 1000 is performed by a semi-autonomous drone.

[0262] In step 1205, information about the work is received. In some embodiments, an asset (e.g., a leader drone) obtains information about the work from a server (such as a server that constitutes the work). The information about the work may include suggestions for the set of assets that will perform the work, and / or one or more parameters or constraints associated with the work. In some embodiments, the information about the work may include a pre-plan, which may be a high-level plan for the leader drone to perform the work. The high-level plan may have a coarser or less specific definition of the tasks to be performed and / or how the tasks will be performed than the plan determined by the leader drone.

[0263] In step 1210, the asset decides to act as a leader drone, based at least partially on information regarding the work.

[0264] In step 1215, the leader drone determines a plan to complete at least one task in order to perform the work. In some embodiments, the leader drone determines one or more tasks of the work and a plan for performing one or more tasks. For example, the leader drone may break down the work into smaller constituent tasks and determine a plan for performing the tasks.

[0265] In step 1220, the leader drone determines at least one asset to perform at least a portion of the plan. For example, the leader drone may assign a plan or task to an asset within a group of assets that will perform the work.

[0266] In step 1225, the leader drone communicates at least a portion of the plan to at least one asset. For example, the leader drone may communicate at least a portion of the plan to an asset assigned to the plan and / or task.

[0267] In step 1230, the leader drone decides whether to communicate part of the plan to another asset. Depending on the decision to communicate another part of the plan to a group of assets, process 1200 may proceed to step 1225, in which the plan is repeatedly sent to different assets to which parts of the plan or task are assigned. The leader drone may repeatedly send parts of the plan to different assets until the leader drone decides in step 1230 that part of the plan is not to be communicated to another asset.

[0268] In step 1235, the leader drone decides whether to determine a plan for another task and whether to send the plan to an asset or the like. Depending on whether the leader drone decides to determine and send a plan for another task, process 1200 proceeds to step 1215, in which the leader drone repeatedly determines a plan for a task and sends at least a portion of the plan to one or more assets until the leader drone determines that a future plan does not need to be determined or sent to an asset. Depending on whether step 1235 determines that a plan for another task does not need to be determined and sent, process 1200 proceeds to step 1240.

[0269] In step 1240, the leader drone acquires information while executing at least a portion of the plan. In some embodiments, the leader drone receives feedback information from one or more follower drones while executing the corresponding plan. The leader drone may also receive control information about the work from a server. For example, the control information may relate to modifications to the work parameters, modifications to asset groups, etc.

[0270] In step 1245, the leader drone transmits information regarding the status of the work. In some embodiments, the leader drone may provide the server with updated information regarding the status of the work. For example, in response to receiving feedback information from one or more follower drones, the leader drone may aggregate the feedback from various follower drones and / or determine the status of the work. The leader drone may transmit the status of the work, as well as information acquired by one or more drones in the asset group (e.g., live stream video, images, current location, current status of implementation of the corresponding plan, etc.), to the server. In some embodiments, the leader drone may transmit information regarding the status of the work to one or more follower drones. For example, the leader drone may provide updated information regarding the status of the work or the plan.

[0271] In step 1250, the leader drone determines whether the task is complete. The leader drone may determine that the task is complete if it determines that all tasks associated with the task have been completed by the asset group. Alternatively, the leader drone may determine that the task is complete if it receives notification from the server that the task has been canceled or paused. For example, the task may be canceled or paused in response to user input to the user interface.

[0272] In response to the determination in step 1250 that the operation has not been completed, process 1200 may proceed to step 1260 to determine whether the leader drone updates the plan. For example, the leader drone may determine to update the plan based on feedback information received from the follower drones and / or control information received from the server. As another example, the leader drone may determine to update the plan based on changes in the situation of the operation (e.g., detection of a target, state of payload transportation, changes in environmental factors, etc.). In response to determining that the plan is updated, process 1200 may proceed to step 1215. In response to determining that the plan is not updated, process 1200 may proceed to step 1240.

[0273] In response to the determination in step 1250 that the operation has been completed, process 1200 may proceed to step 1255 where the status is provided to the server and / or the control terminal (e.g., the client terminal).

[0274] FIG. 12B is a diagram showing a method for implementing a plan associated with an operation according to various embodiments of the present application. In some embodiments, process 1215 of FIG. 12B is executed by server 105, asset 120, asset 125, and / or asset 130 of system 100 of FIG. 1, and / or device 300 of FIG. 3. Process 1215 of FIG. 12B may correspond to step 1215 of FIG. 12A.

[0275] In step 1215a, the leader drone acquires information about the work. In step 1215b, the leader drone determines the location where the plan will be implemented. In step 1215c, the leader drone determines the tasks or elements that the follower drones will perform at that location. In step 1215d, the leader drone determines one or more features of the environment at that location. In step 1215e, the leader drone determines a model of the area where at least one task will be performed. According to various embodiments, the model may include a model of the environment in which the asset group operates. For example, the model may be a 2D or 3D model of the world in which the asset group operates or performs work. In step 1215f, at least a portion of the plan is determined based at least in part on the model.

[0276] In various embodiments, the planner service implemented in the leader drone may iteratively refine the plan. For example, when the leader drone receives further information (e.g., more specific / detailed information about the work, such as the conditions in which the assets are operating), the leader drone may update the model of the work and the corresponding plan for performing the task based on the current model. As an example, early in the execution of the work, the leader drone may have a basic understanding of the topography of the area in which the asset group is operating. As time and the work progresses, the asset group may monitor and capture information using various sensors or cameras on the group of assets, and this information may be returned to the leader drone as feedback information.

[0277] Figure 12C shows a method for carrying out a plan associated with a task according to various embodiments of the present application. In some embodiments, process 1215 in Figure 12C is performed by the server 105, asset 120, asset 125, and / or asset 130 of system 100 in Figure 1, and / or device 300 in Figure 3. Process 1215 in Figure 12B may correspond to process 1215 in Figure 12A.

[0278] In step 1215g, the leader drone obtains a high-level predetermined plan (e.g., a pre-plan) for the task. For example, the leader drone may receive a pre-plan from a server. In step 1215h, the leader drone obtains information received from one or more assets regarding the implementation of at least a portion of the plan. For example, the leader drone may receive feedback information obtained by follower drones during the implementation of the corresponding plan. In step 1215i, the leader drone determines at least one element of a task to be performed by an asset. The leader drone may decompose the task into components and determine the functions required to perform the elements (e.g., functions mapped to the elements). The leader drone may determine an asset with functions that match the functions required to perform the element and assign the element to that asset. In step 1215j, the leader drone determines a model of the implementation of at least that element of the task. In some embodiments, the leader drone determines a model of the environment in which the asset performs the element. For example, if the element is to track a car driving on a road, the leader drone may generate a model of the environment including the road, the topology of the area, the relative positions of buildings within the area, the relative positions of trees and other objects within the area, and the position of the car within the area. In step 1215k, the leader drone determines a lower-level plan for the task. The lower-level plan may contain more detail or finer granularity than the pre-plan received from the server. For example, the pre-plan for the task may indicate that a car will be tracked within a given area, and the lower-level plan may include elements for implementing the plan, such as (i) tracking the car as it drives on a road within the given area, (ii) moving a predetermined distance away from the vehicle being tracked, (iii) capturing an image of the car, (iv) sending updates on the status of the car's tracking at specific intervals, and (v) sending a warning if the car stops at a certain location, providing an indication of the stop and a timestamp.

[0279] Figure 13 shows a method for carrying out a plan associated with a task according to various embodiments of the present application. In some embodiments, process 1300 in Figure 13 is performed by assets 120, 125, and / or 130 of system 100 in Figure 1, and / or device 300 in Figure 3. According to various embodiments, process 1000 is performed by a semi-autonomous drone. Process 1300 may be performed by a leader drone.

[0280] In step 1305, the drone obtains information about a higher-level task or work to be performed. In step 1310, the drone may obtain information about the environment in which at least part of the task or work will be performed. In step 1315, the drone determines the set of assets that will perform at least part of the task. In step 1320, it determines whether to split the set of assets (e.g., a group or team of assets) that will perform part of the task.

[0281] If the drone determines in step 1320 that the asset group will not be split, process 1300 may proceed to step 1325. In step 1325, the drone sends a command to the asset group to perform at least part of the task. In step 1330, the drone receives feedback information. For example, a leader drone may receive feedback information from one or more follower drones. In step 1335, the leader drone decides whether to update the plan. For example, the leader drone may decide whether to update the plan based at least in part on the feedback information.

[0282] Depending on the decision to update the plan, process 1300 proceeds to process 1340, where the plan is updated. Updating the plan may include updating the model of the environment in which the follower drones are operating and determining whether the ability to perform the task or elements of the task has changed based on the model update or changes in the work status. In process 1345, the leader drone sends the updated plan. For example, the leader drone may send the updated plan to follower drones that were assigned to the previous version of the plan (or the task corresponding to the plan). For another example, if the plan update involves assigning a new, different, or further asset to the task, the leader drone may send the updated plan to such asset. Process 1300 then returns to process 1330.

[0283] If the decision in step 1335 is that the plan is not updated, process 1300 may proceed to step 1350, in which the leader drone determines whether the work is complete. If the determination is that the work is not complete, process 1300 may return to step 1305, and process 1300 may be repeated again. If the determination is that the work is complete, process 1300 may terminate.

[0284] In response to the decision in step 1320 to divide the assets into groups that will perform part of the task, process 1300 may proceed to step 1335 in which a partition leader (e.g., a partition leader) is determined. In step 1360, the leader drone may send an instruction to the partition leader to perform the corresponding part of the task. Process 1300 may then proceed to step 1330. In response to the partition leader receiving the instruction to perform the corresponding part of the task, the partition leader may perform process 1300 as a leader drone for the partition.

[0285] Figure 14 shows a method for carrying out a plan associated with a task according to various embodiments of the present application. In some embodiments, the process 1400 in Figure 14 is performed by assets 120, 125, and / or 130 of the system 100 in Figure 1, and / or the device 300 in Figure 3. According to various embodiments, the process 1000 is performed by a semi-autonomous drone.

[0286] In step 1405, the drone receives information about the work. In some embodiments, the drone receives a plan to perform a portion of the work (e.g., a task or an element of a task). The drone may receive an instruction to perform a portion of the work as a partition, which includes a portion of the assets assigned to perform the work. According to various embodiments, the information about the work includes a suggestion that the drone is assigned as the partition leader of the partition.

[0287] In step 1410, the drone decides, at least partially, to act as the partition leader drone (e.g., partition leader) based on the information.

[0288] In step 1415, the partition leader determines a plan to complete at least one task of the work. In some embodiments, the partition leader determines one or more plans to complete one or more tasks assigned to the partition. The partition leader may determine the plan at least in part on one or more of the following: parameters and / or constraints of the work (or tasks assigned to the partition), functions or features associated with at least one task, functions of one or more assets in the partition, etc. According to various embodiments, the partition leader implements a planning service similar to or the same as the planner service implemented by the leader drone of the asset group assigned to perform the work.

[0289] In step 1420, the partition leader determines at least one asset as a communication destination for at least a portion of the plan. In some embodiments, the partition leader assigns at least a portion of the plan to an asset in a partition. The partition leader may determine an asset in a partition as a destination for at least a portion of the plan. For example, the asset may be determined based on the asset's functionality, and / or the functionality associated with a task or element corresponding to at least a portion of the plan.

[0290] In step 1425, the partition leader communicates at least a portion of the plan to at least one asset. In some embodiments, depending on which asset in the partition is determined to be allocated to a portion of the plan, the partition leader transmits the portion of the plan to at least one asset.

[0291] In step 1430, the partition leader decides whether to communicate part of the plan to another asset within the partition. Depending on whether the other part of the plan is allocated part of the plan, process 1400 proceeds to step 1420, and the partition leader repeats steps 1420, 1425, and 1430 until the partition leader decides not to communicate part of the plan to another asset.

[0292] In step 1435, the partition leader decides whether to determine a plan for another task and whether to send the plan to an asset or the like. Depending on whether the partition leader decides to determine and send a plan for another task, process 1400 proceeds to step 1415, in which the leader drone repeatedly determines a plan for the task and sends at least a portion of the plan to one or more assets until the partition leader determines that a future plan does not need to be determined or sent to an asset. Depending on whether step 1435 determines that a plan for another task does not need to be determined and sent, process 1400 proceeds to step 1440.

[0293] In step 1440, the partition leader acquires information during the execution of at least a portion of the plan. In some embodiments, the partition leader receives feedback information from one or more follower drones (within the partition) during the execution of the corresponding plan. The partition leader may also receive control information about the work from a server (or the leader of the asset group assigned to the work). For example, the control information may relate to modifications to the work parameters, modifications to the asset group, etc.

[0294] In step 1445, the partition leader transmits information regarding the status of the work. In some embodiments, the partition leader may provide the server with updated information regarding the status of the work. For example, in response to receiving feedback information from one or more follower drones, the partition leader may aggregate the feedback from various follower drones and / or determine the status of the work. The partition leader may transmit to the server the status of the work (or its task), as well as information acquired by one or more drones in the asset group (e.g., live stream video, images, current location, current status of the implementation of the corresponding plan, etc.). In some embodiments, the partition leader may transmit information regarding the status of the work to one or more follower drones. For example, the leader drone may provide updated information regarding the status of the work or the plan. In some embodiments, the partition leader provides information regarding the status of the work to the leader drone (e.g., the leader of the asset group initially assigned to perform the work), and the leader drone then provides information regarding the status of the work to the server.

[0295] In process 1450, the partition leader determines whether a part of the work corresponding to the partition has been completed. The leader drone may determine that a part of the work has been completed in response to determining that all tasks associated with the part of the work have been completed by the asset group. Also, the partition leader may determine that a part of the work has been completed in response to receiving a notification from the server (or the leader drone) that the work has been canceled or paused. As an example, the work may be canceled or paused in response to user input to the user interface.

[0296] In response to the determination in process 1450 that the work has not been completed, process 1440 may proceed to process 1460 to determine whether the leader drone updates the plan. For example, the partition leader may determine to update the plan based on feedback information received from the follower drones and / or control information received from the server. As another example, the partition leader may determine to update the plan based on a change in the work situation (e.g., detection of a target, state of payload transportation, change in environmental factors, etc.). In response to determining that the plan is updated, process 1400 may proceed to step 1415. In response to determining that the plan is not updated, process 1400 may proceed to step 1240.

[0297] In response to the determination in process 1450 that the work has been completed, process 1400 may proceed to process 1455 where the status is provided to the server and / or the control terminal (e.g., the client terminal).

[0298] FIG. 15 is a diagram showing a method for implementing a plan associated with work according to various embodiments of the present application. In some embodiments, process 1500 in FIG. 15 is executed by asset 120, asset 125, and / or asset 130 of system 100 in FIG. 1, and / or device 300 in FIG. 3. According to various embodiments, process 1000 is executed by a semi-autonomous drone.

[0299] In step 1505, the drone receives information about the work. In some embodiments, the drone receives a plan for performing the work (e.g., a task or an element of a task). In some embodiments, the drone receives information about the work from the leader drone of the asset group. The information about the work may include suggestions for the asset group on which to perform the work. The information about the work may include a predetermined leader ranking.

[0300] In step 1510, the drone decides to act as a dormant leader drone, at least in part, based on information about the task. The drone may decide to act as a dormant leader based on a predetermined leader ranking. For example, a dormant leader may be a drone with the second highest ranking (e.g., among active assets). In some embodiments, the information about the task is an instruction for the drone to act as a dormant leader.

[0301] In step 1515, the dormant leader drone communicates with at least one asset and at least a portion of the plan. In some embodiments, the dormant leader drone receives information about the plan from the leader drone. The dormant leader drone may synchronize with information about the work stored by the leader drone. For example, the dormant leader may have a corresponding shadow copy of the leader drone.

[0302] In step 1520, the dormant leader drone executes the corresponding portion of the plan. In some embodiments, the dormant leader drone executes one or more tasks or elements in connection with the execution of the work. In some embodiments, the dormant leader drone neither determines nor executes the plan. For example, the dormant leader drone may function as a backup for the leader drone.

[0303] In step 1525, the dormant leader drone acquires information during the execution of at least part of the plan. In some embodiments, the information acquired during the execution of at least part of the plan includes feedback information provided by the follower drone, synchronization information provided by the leader drone, and / or information acquired by one or a sensor of the dormant leader drone. In addition to sending feedback information to the leader drone, the follower drone may also send feedback information to the dormant leader drone.

[0304] In step 1530, the dormant leader drone determines whether or not to act as a leader. In some embodiments, the dormant leader drone decides to act as a leader in response to a determination that the leader drone has failed. For example, a leader drone may be considered to have failed if it has lost communication with the dormant leader drone or asset group over a predetermined threshold period. For example, a leader drone may be considered to have failed if it has a communication link with the asset group and the communication link falls below a predetermined quality of service threshold. If the leader drone has been unable to communicate with the dormant leader drone or asset group over a predetermined threshold period, the dormant drone or follower drone may send a ping to check the health of the leader drone. If the leader drone fails to respond to the health check within a set period, the leader drone may be considered to have failed.

[0305] In response to the decision that the dormant leader drone will not operate as a leader, process 1500 proceeds to process 1535, in which information regarding the status of the operation is transmitted to the leader drone.

[0306] In step 1540, the dormant leader drone determines whether or not it has received an indication that the task is complete. If it determines that it has received an indication that the task is complete, process 1500 ends. Conversely, if it determines that it has not received an indication that the task is complete, process 1500 proceeds to step 1545, in which the dormant leader drone determines whether or not it has received an updated plan. If it determines that it has received an updated plan, process 1500 proceeds to step 1520. Conversely, if it determines that it has not received an updated plan, process 1500 proceeds to step 1525.

[0307] If, in step 1530, it is determined that the dormant leader drone will act as a leader, process 1500 proceeds to step 1550, in which the dormant leader drone loads information about the work as a leader. In some embodiments, the dormant leader drone loads a synchronized copy of the leader information about the work. For example, the dormant leader drone may load a backup copy of the information about the work that has been backed up from a local copy on the leader drone.

[0308] In process 1555, the leader drone (for example, the previous dormant leader drone) determines a plan to complete at least part of the task.

[0309] In process 1560, the leader drone determines which assets to complete in at least part of the plan.

[0310] In process 1565, the leader drone communicates at least part of the plan to the assets.

[0311] In step 1570, the leader drone decides whether to communicate part of the plan to another asset. Depending on whether it is decided that part of the plan will be communicated to another asset, process 1500 returns to step 1560. Conversely, depending on whether it is decided that part of the plan will not be communicated to another asset, process 1500 proceeds to step 1570, in which the leader drone determines in step 1575 whether there is another task (for example, one to which the plan is determined and assigned). Depending on whether it is determined in step 1575 that there is another task, process 1500 returns to step 1555. Depending on whether it is determined in step 1575 that there is no other task, process 1500 proceeds to step 1580, in which the leader drone obtains information while at least part of the plan is being implemented. For example, the leader drone may receive feedback information from the follower drone.

[0312] In step 1585, the leader drone transmits information regarding the status of the operation. The leader drone may transmit information regarding the status of the operation to a server (e.g., a control center) and / or a client terminal. The leader drone may also transmit information regarding the status of the operation to one or more follower drones.

[0313] In step 1590, the leader drone determines whether the task is complete. If the task is determined to be complete in step 1590, process 1500 ends. Conversely, if the task is determined to be incomplete in step 1590, process 1500 proceeds to step 1595, where the leader drone decides whether to update the plan. For example, a task is determined to be incomplete if the task is determined to be incomplete (for example, if a drone is tasked with scanning a road, and the drone is unable to maintain a series of sharp curves on the road and misses scanning part of the road, the task is considered incomplete, and the drone may turn back and rescan the missed portion of the road). If the decision in step 1595 is to update the plan, process 1500 proceeds to step 1555. Conversely, if the decision is not to update the plan, process 1500 proceeds to step 1580.

[0314] Figure 16A shows a discrete representation according to various embodiments of the present application. In the example shown in Figure 16A, the discrete representation 1600 includes a plurality of discrete elements corresponding to voxels. The discrete representation 1600 may be implemented by the system 100 in Figure 1, the device 300 in Figure 3, and so on.

[0315] As used herein, the term "voxel" corresponds to a value on a regular grid in three-dimensional space. For example, a voxel corresponds to a discrete element of a discrete representation.

[0316] According to various embodiments, each discrete element (e.g., cell) in the 3-D representation corresponds to a different portion of a geographical location. For example, a portion of a geographical location is mapped to a corresponding discrete element in the 3-D representation in relation to determining the discrete representation 1600. As shown in Figure 16A, the 3-D dimensions of the discrete representation 1600 correspond to the x, y, and z axes. The z axis corresponds to the direction parallel to gravity. As an example, the number of discrete elements provided for the z axis is determined at least in part based on the maximum altitude or elevation associated with the configuration of the work (e.g., a parameter entered by the user during the configuration of the work, the maximum altitude capability of an asset among the asset group selected to perform the work, the default maximum altitude or elevation, etc.).

[0317] As shown in Figure 16A, each set of information 1610 is mapped to or associated with a set of discrete elements in a discrete representation 1600. In some embodiments, each discrete element in the discrete representation 1600 has information mapped to it. For example, a discrete element has one or more metadata fields, each corresponding to a different type of information mapped to it. Examples of the types of information mapped to or associated with discrete elements include indications whether the discrete element corresponds to a location identified as a keep-in area, indications whether the discrete element corresponds to a location identified as a keep-out area, indications whether the discrete element is occupied, indications whether the discrete element is available or not, indications whether assets within the discrete element have a clear line of sight (to the current location of, for example, a leader drone, tower, or ground station), indications whether a target is included in the discrete element, identifiers of targets included in the discrete element, indications whether the discrete element has or is experiencing adverse weather, or the type of weather (e.g., clear, rainy, windy, temperature, wind direction and speed, etc.).

[0318] Generally, the discretization of terrain breaks down as distance increases and the curvature of the Earth becomes significant or important to the calculation. For example, if each discrete element in the 3-D representation corresponds to a volume of the same size, the curvature of the Earth causes the base of the 3-D representation to correspond to a plane tangent to the Earth at a particular intersection, and as the 3-D representation extends further in the x or y direction, the 3-D representation will deviate more and more from the Earth's surface. In some embodiments, the discrete representation 1600 is generated based at least partially on the curvature of the Earth. For example, the system takes the curvature of the Earth into account when generating the discrete representation 1600 in a way that transforms real space into a 3-D representation such that the z-axis corresponds to the direction of gravity (or the direction perpendicular to the Earth's surface at the location corresponding to the discrete element). As another example, the z-axis corresponds to the direction of gravity, regardless of how far laterally the 3-D representation is moved from the origin in the x or y direction.

[0319] In various embodiments, discrete representations are used in connection with modeling one or more tasks. For example, the system references (e.g., searches) discrete representations and associated information (e.g., metadata including fields for geographical location and / or information about one or more tasks). As an example, the system determines a first discrete element corresponding to a first point, a second discrete element corresponding to a second point, and then connects the first and second discrete elements to determine a path from the first to the second point by a continuous set of discrete elements (e.g., discrete elements are not occupied, etc.) that the asset can traverse. To determine a set of continuous sets of discrete elements, the system searches across discrete representations to find / determine a set of continuous assets. In some embodiments, discrete elements shown to be unoccupied are searchable in connection with determining a plan (e.g., a flight path), while discrete elements shown to be occupied are not searchable. For example, the plan is determined based on the occupation of one or more discrete elements of the discrete representation.

[0320] A task (e.g., one or more tasks performed by a group of one or more assets) generally extends over distances in at least the x and y directions. Therefore, a model of the geographical location corresponding to a task (e.g., one or more tasks performed by a group of one or more assets) includes a large discrete representation and a corresponding number of discrete elements. According to various embodiments, to assist in the rapid processing of the discrete representation 1600 in relation to determining a plan (e.g., a plan for orienting follower drones, flight paths of assets, etc.), the system creates a discrete representation that limits the extent to which the discrete elements extend in the z direction. For example, altitude / elevation is typically limited according to the configured task (e.g., user input, asset functions, and / or default settings), so the discrete representation 1600 may similarly be limited to limit the number of cells processed. In some embodiments, the discrete representation 1600 comprises a grid segment of 400 × 400 × 100. As an example, discrete representation 1600 has 400 discrete elements in the dimension along the x-direction, 400 discrete elements in the dimension along the y-direction, and 100 discrete elements in the dimension along the z-direction. Discrete representations 1600 of various other dimensions may be implemented. For example, the number of discrete elements in discrete representation 1600 (e.g., in all dimensions or in a single dimension) can be configured according to user input, etc.

[0321] In some embodiments, the modeling of geographic location / environment associated with determining a plan for one or more tasks uses a discrete representation of the geographic location environment having the same dimensions, such as the same number of discrete elements in one or more directions, regardless of the size of the geographic location corresponding to the work or one or more tasks (e.g., a user-defined geographic location). Thus, the larger the geographic location, the coarser the granularity of the discrete representation. Conversely, the smaller the geographic location, the finer the granularity of the discrete representation. The resolution of the model (e.g., the discrete representation) increases as the geographic location being modeled becomes smaller. Thus, in some embodiments, when a leader drone is assigned a particular portion of the tasks / elements of the work and a follower drone is deployed to perform that portion of the tasks / elements, the follower drone generates a model of the geographic location corresponding to the particular portion of the tasks / elements assigned to the follower drone. For example, a model of the geographical location of a specific portion of the tasks / elements assigned to a follower drone (e.g., a discrete representation such as a discrete representation generated by the follower drone) has a higher resolution than a model of the geographical location of the work (e.g., a superset of one or more tasks that a leader drone manages and directs the follower drones to perform).

[0322] Figure 16B shows discrete representations of geographic locations according to various embodiments of the present application. In the example shown in Figure 16B, the discrete representation 1650 is filled with geographic information. For example, the discrete representation 1650 includes representations of trees, roads, cars, hills, and ground control stations 1620. Similarly, the discrete representation 1650 is annotated with one or more parameters relating to a geographic location, comprising at least some of a plurality of discrete elements. For example, a set of information 1610 includes a set of information fields, and the set of information 1610 is filled with information relating to a geographic location and mapped to the discrete representation 1650. As an example, as shown in Figure 16B, the set of information 1610 includes (i) information indicating whether one or more discrete elements correspond to a keep-in area, (ii) information indicating whether one or more discrete elements are occupied or not, (iii) information indicating whether one or more discrete elements correspond to a clear line of sight (e.g., a clear communication / radio line), (iv) information indicating whether one or more discrete elements include a target, and (v) information indicating whether one or more discrete elements are exposed to adverse weather conditions, and so on.

[0323] Figure 16C shows discrete representations of geographical locations according to various embodiments of the present application. As shown in the figure, a group of drones (e.g., D1 1630 and D2 1640, etc.) are deployed at geographical locations corresponding to discrete representations 1675. As an example, the work involves tracking / monitoring a target located at, for example, discrete elements (x=2, y=4, z=0). According to various embodiments, the leader drone D1 1630 provides the follower drone D2 1640 with a plan for tracking / monitoring the target 1670. As an example, the leader drone D1 1630 provides the follower drone D2 1640 with a flight plan 1680. As another example, the leader drone D2 1630 provides a suggestion for the follower drone D2 1640 to perform the task of tracking / monitoring the target 1670, and the follower drone D2 1640 determines a flight plan 1680 for the follower drone D2 1640 to move to a position where it has a line of sight to the target.

[0324] In the example shown in Figure 16C, target 1670 is not in the line of sight of the ground control station 1620 or the leader drone D1 1630. For example, a hill / mountain is located between the ground control station 1620 and target 1670. In some embodiments, to perform the task of monitoring target 1670, follower drone D2 1640 is deployed to move to a position that has a line of sight to target 1670. The system determines a flight plan 1680 as follower drone D2 1640 moves. As shown in Figure 16C, the flight plan 1680 is configured to orient follower drone D2 1640 over trees and around a hill. In some embodiments, the cost function associated with moving the drone along the path shows that moving the drone vertically to an altitude sufficient to fly over the hill is more costly than moving the drone laterally in the x and / or y directions to bypass the hill. For example, movement in the x and / or y directions is more efficient for the drone than movement in the z direction (e.g., gaining altitude). Therefore, a longer flight plan that involves movement in the x and / or y directions is more efficient (e.g., less expensive) than a shorter flight plan that requires movement in the z direction.

[0325] Figure 17A shows discrete representations of geographic locations according to various embodiments of the present application. In the example shown in Figure 17A, the discrete representation 1700 includes a plurality of discrete elements corresponding to voxels. The discrete representation 1700 may be implemented by the system 100 in Figure 1, the device 300 in Figure 3, and so on.

[0326] According to various embodiments, the system repeatedly updates the discrete representation 1700 during the lifespan of the work (e.g., the time until the work is paused, completed, or terminated). For example, the discrete representation 1700 is generated at the start of the work. If the work is started at a ground control station 1710 or the like, information about the geographical location and / or one or more tasks associated with the work is added to the discrete representation 1700 and the associated set of information 1705 (e.g., metadata associated with one or more discrete elements of the discrete representation 1700). In some embodiments, the discrete representation 1700 is annotated with the set of information 1705.

[0327] According to various embodiments, the information contained in the set of information 1705 is retrieved via one or more configurations (e.g., definitions) of work that are pre-stored in a server or local service (e.g., a map or geographic service, a service that manages various deployment asset sets), and / or third-party services (e.g., a weather service, a map service, etc.).

[0328] As shown in the example in Figure 17A, only a portion of the discrete representation 1700 is filled with corresponding information. For example, only the portion of 1715 visible from the ground control station 1710 is provided. Hill 1720 obstructs the view to the feature / information discrete element on the opposite side of hill 1720.

[0329] Figure 17B shows discrete representations of geographical locations according to various embodiments of the present application. In the example shown in Figure 17B, a leader drone D1 1730 and a follower drone D2 1735 are deployed. For example, the leader drone D1 1730 and the follower drone D2 1735 are deployed in connection with the commencement of one or more tasks associated with a work or work. In some embodiments, the leader drone D1 1730 commands the follower drone D2 1740 to move to position 1745 to observe / monitor an area opposite to 1720 (for example, to identify a target). A flight plan 1740 for the follower drone D2 1735 is determined, and the follower drone D2 1735 moves along the path of the flight plan 1740 (for example, to (x=0, y=0, z=3)). In some embodiments, the set of information 1705 associated with the discrete representation 1725 is updated in response to (or during) the follower drone D2 1735 moving along the path of the flight plan 1740. For example, the follower drone D2 1735 communicates feedback information (e.g., to the leader drone D1 1730) regarding one or more tasks and / or geographical location. As shown in Figure 17B, the discrete representation 1725 is updated to include further information about the road 1715. Furthermore, the set of information 1705 is updated to include information associated with the discrete element located at (x=0, y=0, z=3). In some embodiments, the annotated representation is updated locally in each of the follower drone D2 1735 and the leader drone D1 1730.

[0330] Figure 17C shows discrete representations of geographical locations according to various embodiments of the present application. As shown in Figure 17C, when the follower drone D2 1735 moves to a discrete location located at (x=0, y=2, z=3), the discrete representation 1750 and / or set of information 1705 are updated (e.g., the annotated representation is updated). For example, when the follower drone D2 1735 moves to (x=0, y=2, z=3), the follower drone D2 will have a view of more roads 1715 and trees 1755. The follower drone D2 1735 updates the annotated representation locally on the follower drone D2 1735 and / or communicates feedback information to the leader drone D1 1730, so that the leader drone D1 1730 updates the locally stored discrete representation 1750 and / or set of information 1705 (e.g., the annotated representation stored on the leader drone D1 1730) accordingly.

[0331] Figure 17D shows discrete representations of geographic locations according to various embodiments of the present application. As shown in Figure 17D, when the follower drone D2 1735 moves to a discrete location located at (x=0, y=3, z=4), the discrete representation 1775 and / or set of information 1705 are updated (e.g., the annotated representation is updated). For example, when the follower drone D2 1735 moves to (x=0, y=3, z=4), the follower drone D2 will have a better view of road 1715, and the follower drone D2 1735 will identify a target 1780 moving along road 1715. The follower drone D2 1735 updates the annotated representation locally on the follower drone D2 1735 and / or communicates feedback information to the leader drone D1 1730, which in turn updates the locally stored discrete representation 1775 and / or set of information 1705 (e.g., the annotated representation stored on the leader drone D1 1730). For example, the set of information 1705 to annotate the discrete representation 1775 includes an entry that identifies the discrete element located at (x=2, y=5, z=0) as containing a target (e.g., target 1780).

[0332] Figure 18 shows a method for determining a plan to perform one or more tasks according to various embodiments of the present application. According to various embodiments, process 1800 is performed at least partially by system 100 in Figure 1 and / or device 300 in Figure 300. In embodiments, process 1800 is performed in connection with determining or updating the plan. Process 1800 may be performed during the execution of the work. Process 1800 is performed by leader drones and / or follower drones of a group of assets assigned to perform one or more tasks.

[0333] In step 1810, data associated with one or more tasks is acquired. According to various embodiments, the acquisition of data associated with one or more tasks includes one or more of the following: step 710 of process 700 in Figure 7A, step 835 of process 835 in Figure 8C, step 931 of process 930 in Figure 9B, step 950 of process 900 in Figure 9A, step 952-1 of process 950 in Figure 9E, step 1010 of process 1000 in Figure 10A, step 1021 of process 1020 in Figure 10B, step 1215a of process 1215 in Figure 12B, step 1305 and / or step 1310 of process 1300 in Figure 13, step 1405 and / or step 1440 of process 1400 in Figure 14, and / or step 1505 and / or step 1525 of process 1500 in Figure 15.

[0334] In some embodiments, data associated with one or more tasks is associated with the geographical location where the work or one or more tasks are performed, and / or with one or more features or parameters associated with the work. For example, data associated with one or more tasks is received from a server (e.g., a server providing work control services), along with the configuration of the work (e.g., based on user input). For another example, data associated with one or more tasks is received from another asset in a group of assets where the work is performed. In the case of a leader drone, the leader drone receives data associated with one or more tasks from follower drones, such as feedback information regarding updates to the execution status of one or more tasks. In the case of follower drones, the follower drones receive data associated with one or more tasks from the leader drone and / or other assets in the group of assets. For yet another example, data associated with one or more tasks is received from third-party services, such as servers providing services / information about the work (e.g., weather services), information about other assets deployed at geographical locations, mapping or geographic services, etc.

[0335] In step 1820, a discrete representation of the geographic location is determined. According to various embodiments, the discrete representation corresponds to discrete representation 1600 in Figure 16A and / or discrete representation 1700 in Figure 17A. The discrete representation is generated based at least partially on at least a portion of the data associated with one or more tasks. In some embodiments, determining the discrete representation involves converting a real-world representation of the geographic location into discrete elements of a predetermined number / dimension. For example, the volume or quantity of space represented by a particular discrete element is based at least partially on the size of the geographic location from which the discrete representation is generated.

[0336] In step 1830, the discrete representation is annotated. In some embodiments, the discrete representation is annotated at least partially based on (e.g., to include) one or more parameters relating to geographic location and / or information relating to one or more tasks. For example, the discrete representation is annotated in response to a determination that one or more parameters relate to geographic location. In some embodiments, annotating the discrete representation involves setting / adding information relating to at least some of the discrete elements within the discrete representation (e.g., setting the set of information 1610 in Figure 16A and / or the set of information 1705 in Figure 17A).

[0337] According to various embodiments, discrete representations are annotated in connection with creating annotated representations (e.g., annotated representations of geographical locations). As an example, annotating a discrete representation involves setting metadata or associating metadata with one or more discrete elements of a discrete representation.

[0338] In step 1840, a plan for performing one or more tasks is determined. According to various embodiments, the plan for performing one or more tasks is determined at least in part on an annotated expression. The plan determination is based on one or more tasks to be performed (e.g., one or more features associated with one or more tasks) and one or more parameters associated with the geographical location. As an example, one or more parameters associated with the geographical location are included in the annotated expression.

[0339] In step 1850, information regarding the plan is communicated. In some embodiments, the information regarding the plan is transmitted to other assets in the asset group. For example, in the case of a leader drone, the leader drone transmits information regarding the plan to a ground control station and / or one or more follower drones, and the leader drone receives information such as feedback information from the ground control station and / or one or more follower drones. In another example, in the case of a follower drone, the follower drone transmits information regarding the plan to the leader drone and / or one or more other assets in the asset group that perform one or more tasks, and the follower drone receives update information from the leader drone and / or one or more other assets in the asset group.

[0340] In step 1860, a determination is made as to whether or not updated information has been received. According to various embodiments, the drone may receive updated information from another asset in the asset group or from a server.

[0341] In some embodiments, the updated information is associated with the geographical location where the work or one or more tasks are performed, and / or with one or more features or parameters associated with one or more tasks within the work or tasks. For example, the updated information associated with one or more tasks is received from a server (e.g., a server providing work control services), along with changes to the work (e.g., based on user input). For another example, the data associated with one or more tasks is received from another asset in the asset group on which the work is performed. In the case of a leader drone, the leader drone receives updated information from the follower drone, such as feedback information regarding updates to the execution status of one or more tasks, or some information about the geographical location detected by the follower drone's sensors or cameras. In the case of a follower drone, the follower drone receives data associated with one or more tasks from the leader drone and / or other assets in the asset group. For another example, the data associated with one or more tasks is received from a third-party service, such as a server providing services / information related to the work (e.g., weather services), information about other assets deployed at the geographical location, mapping or geographic services, flight plans for other assets, etc.

[0342] If process 1800 determines that updated information has been received, process 1800 returns to process 1830. In some embodiments, the annotated expression is repeatedly updated with the updated information as it is determined that updated information has been received.

[0343] If process 1860 determines that no updated information has been received, process 1800 proceeds to process 1870, where a determination is made as to whether the process is complete. For example, process 1800 may be determined to be complete based at least in part on user input (such as input to cancel or pause the work). For example, the process may be determined to be complete if the user chooses to end the work. For example, the process may be determined to be complete if the work is completed (e.g., completion of one or more tasks associated with the work). If the process is deemed complete, process 1800 terminates. Otherwise, process 1800 returns to process 1860, where the process investigates / monitors for updated information.

[0344] Figure 19A shows a method for determining a flight plan according to various embodiments of the present application. According to various embodiments, process 1900 is performed at least in part by system 100 in Figure 1 and / or device 300 in Figure 300. In embodiments, process 1900 is performed in connection with determining or updating a plan (such as a flight plan). Process 1900 may be performed in the course of performing a task. Process 1900 is performed by a leader drone and / or follower drones of an asset group assigned to perform one or more tasks.

[0345] In step 1910, data associated with one or more tasks is acquired. According to various embodiments, the acquisition of data associated with one or more tasks includes one or more of the following: step 710 of process 700 in Figure 7A, step 835 of process 835 in Figure 8C, step 931 of process 930 in Figure 9B, step 950 of process 900 in Figure 9A, step 952-1 of process 950 in Figure 9E, step 1010 of process 1000 in Figure 10A, step 1021 of process 1020 in Figure 10B, step 1215a of process 1215 in Figure 12B, step 1305 and / or step 1310 of process 1300 in Figure 13, step 1405 and / or step 1440 of process 1400 in Figure 14, and / or step 1505 and / or step 1525 of process 1500 in Figure 15. The process of acquiring data along with one or more tasks may be similar to process 1810 of process 1800 in Figure 18.

[0346] In step 1920, a discrete representation of the geographical location is determined. According to various embodiments, the discrete representation corresponds to discrete representation 1600 in Figure 16A and / or discrete representation 1700 in Figure 17A. In some embodiments, the discrete representation is determined in a manner similar to step 1820 of process 1800 in Figure 18.

[0347] In step 1930, the discrete representation is annotated. In some embodiments, the discrete representation is annotated at least partially based on (e.g., to include) information about one or more parameters relating to geographic location and / or one or more tasks. For example, the discrete representation is annotated in response to a determination that one or more parameters relate to geographic location. In some embodiments, annotating the discrete representation involves setting / adding information about at least some of the discrete elements within the discrete representation (e.g., setting up set 1610 of information in Figure 16A and / or set 1705 of information in Figure 17A). In some embodiments, the discrete representation is annotated in a manner similar to step 1830 of process 1800 in Figure 18.

[0348] According to various embodiments, the process of annotating a discrete representation includes registering one or more flight plans associated with another asset in the asset set. For example, upon receiving a flight plan from a drone in the asset set, the discrete representation is annotated to associate the flight plan with the discrete elements of the discrete representation that the flight plan intersects / occupies. In another example, the discrete representation is annotated to set an indication that such discrete elements were occupied for at least the duration associated with the flight plan, or until the time the drone associated with the flight plan passed over a particular discrete element, by setting the discrete elements affected by the discrete representation (e.g., discrete elements that the flight plan intersects / occupies). The set of discrete elements affected by the flight plan is updated when the corresponding drone executes the flight plan (for example, to set the discrete elements that the drone passed over as unoccupied, and to release the discrete elements for use by another asset).

[0349] In step 1940, a flight plan for the drone is determined. According to various embodiments, the flight plan is determined at least in part on annotations in a discrete representation (e.g., an annotated representation). In some embodiments, the drone determines the flight plan based on its geographical location and / or parameters associated with one or more tasks. For example, the drone determines the flight plan based on its current location (e.g., discrete elements corresponding to the current location), its destination location (e.g., discrete elements corresponding to the destination location), and parameters associated with one or more tasks (e.g., the type of task assigned to the drone). In another example, the flight plan is further based on one or more parameters of a set of discrete elements in the discrete representation (e.g., discrete elements between the current location and the destination location, a set of discrete elements along the flight plan connecting the current location and the destination location, etc.).

[0350] In some embodiments, the leader drone determines multiple flight plans, such as each plan for different follower drones in the asset group. The leader drone plans multiple flight plans, such as avoiding each other (for example, to ensure collision avoidance within the asset group).

[0351] In step 1950, information associated with the flight plan is communicated. According to various embodiments, depending on the determination of the flight plan, the flight plan is communicated to one or other assets in the asset group. As an example, the flight plan is published to other assets in the asset group (e.g., on an information feed / channel for information about the flight plan or one or more tasks). Information associated with the flight plan is communicated to other assets in the asset group (e.g., leader drone, follower drones, etc., when a follower drone communicates information) to enable the other asset to store such information locally and / or update each annotated representation of the geographical location to indicate that the discrete element corresponding to the flight plan is occupied until the next communication indicates updated information about the drone's flight status and provides clearing of the discrete element corresponding to the previous part of the flight plan.

[0352] In step 1960, a determination is made as to whether or not updated information has been received. According to various embodiments, the drone may receive updated information from another asset in the asset group or from a server.

[0353] In some embodiments, the updated information is associated with the geographical location where the work or one or more tasks are performed, and / or with one or more features or parameters associated with one or more tasks within the work or tasks. For example, the updated information associated with one or more tasks is received from a server (e.g., a server providing work control services), along with changes to the work (e.g., based on user input). For another example, the data associated with one or more tasks is received from another asset in the asset group on which the work is performed. In the case of a leader drone, the leader drone receives updated information from the follower drone, such as feedback information regarding updates to the execution status of one or more tasks, or some information about the geographical location detected by the follower drone's sensors or cameras. In the case of a follower drone, the follower drone receives data associated with one or more tasks from the leader drone and / or other assets in the asset group. For another example, the data associated with one or more tasks is received from a third-party service, such as a server providing services / information related to the work (e.g., weather services), information about other assets deployed at the geographical location, mapping or geographic services, flight plans for other assets, etc.

[0354] If process 1900 determines that updated information has been received, process 1900 returns to process 1930. In some embodiments, the annotated expression is repeatedly updated with the updated information as it is determined that updated information has been received. As the annotated expression is repeatedly updated, the flight plan is repeatedly determined / updated (for example, a decision is made on whether to keep the flight plan the same or whether to update the flight plan).

[0355] If process 1960 determines that no updated information has been received, process 1900 proceeds to process 1970, where a determination is made as to whether the process is complete. For example, process 1900 may be determined to be complete based at least in part on user input (such as input to cancel or pause the work). For example, the process may be determined to be complete if the user chooses to end the work. For example, the process may be determined to be complete if the work is completed (e.g., completion of one or more tasks associated with the work). If the process is deemed complete, process 1900 terminates. Otherwise, process 1900 returns to process 1960, where the process investigates / monitors for updated information.

[0356] Figure 19B shows a method for determining a flight plan according to various embodiments of the present application. According to various embodiments, process 1940 in Figure 19B is performed in relation to step 1940 of process 1900 in Figure 19A. According to various embodiments, process 1940 is performed at least in part by system 100 in Figure 1 and / or device 300 in Figure 300. In embodiments, process 1940 is performed in relation to determining or updating a plan (such as a flight plan). Process 1940 may be performed in the course of performing an operation. Process 1940 is performed by a leader drone and / or follower drones of an asset group assigned to perform one or more tasks.

[0357] In step 1941, the current location is determined. In some embodiments, the current location is associated with an asset on which the flight plan is determined. The current location represents a discrete element of a discrete representation on which the asset is located. In some embodiments, the GPS location of the asset is determined, and a discrete element corresponding to the GPS location is determined. As an example, the current location is determined at least in part on feedback information received from the asset. As another example, the current location of the asset is determined at least in part on a locally stored annotated representation of its geographic location. For example, the annotated representation is queried for a discrete element on which metadata indicates the asset is located.

[0358] In step 1942, the target location (or destination location) is determined. The target location represents a discrete element of the discrete representation to which the asset will move. The target location is determined at least in part on the plan or task associated with the item. For example, if the leader drone determines that the task for the follower drone is to perform surveillance of a road or building, the target location is determined on the location corresponding to the road or building. As another example, if the leader drone determines that the task for the follower drone is to intercept or monitor a target, the target location is determined at least in part on the location of the target. For example, the location of the target is determined by querying the annotated representation for the discrete element containing the target to be intercepted / monitored.

[0359] In step 1943, a set of consecutive or adjacent discrete elements from the current location to the target location is determined. This set of consecutive or adjacent discrete elements is determined at least partially based on the annotated representation. For example, the set of consecutive or adjacent discrete elements corresponds to the path the asset can take from the current location to the target location. In some embodiments, the set of consecutive or adjacent discrete elements includes only discrete elements that are not indicated as occupied or are indicated as unoccupied.

[0360] In step 1944, the value of the cost function is determined for the set of continuous discrete elements determined in step 1943. In some embodiments, the cost associated with moving the asset from its current position to its target position by the flight plan corresponding to the set of continuous discrete elements is determined. According to various embodiments, the cost function includes one or more variables. In some embodiments, each of the one or more variables has a corresponding weight (e.g., between 0 and 1). The variables in the cost function may relate to the amount of effort required to move the asset from its current position to its target position, the risk of loss, the time it takes to travel the flight path, and so on. Examples of cost functions include: (i) the length of the flight plan; (ii) the extent to which the flight plan involves vertical ascent (e.g., moving the asset to a higher altitude); (iii) the extent to which the flight plan maintains a line of sight to the leader drone or other assets or control stations (e.g., line of radio line); (iv) the extent to which the flight plan involves adverse weather conditions; (v) the extent to which the flight plan exposes the asset to the risk of loss (e.g., the risk of loss exceeding the threshold probability); and (vi) the time required for the asset to travel from its current position to its target position along a flight path of a continuous set of discrete elements. Various other variables may be implemented in relation to the cost function.

[0361] In step 1945, a decision is made as to whether or not to determine another set of continuous discrete elements.

[0362] In some embodiments, the system iteratively determines a set of continuous discrete elements until the continuous discrete elements satisfy a cost threshold (e.g., a predetermined threshold, a configurable threshold, etc.). For example, the system may employ a "sufficiently good" determination method to determine a flight plan such that if a flight plan with a cost below the cost threshold is determined, process 1940 proceeds to process 1946; otherwise, a further set of continuous discrete elements is determined.

[0363] In some embodiments, the system determines a predetermined number of sets of continuous discrete elements from which a selection set of continuous discrete elements is chosen to correspond to a flight plan. For example, the predetermined number of sets of continuous discrete elements to be determined can be configured by a user or administrator, for instance.

[0364] In some embodiments, the system determines a continuous set of discrete elements over a predetermined period of time. For example, the system may allocate a specific time to determining a continuous set of discrete elements, and the system may continue determining the continuous set of discrete elements until such predetermined period has elapsed.

[0365] Depending on the determination in step 1945 that another set of continuous discrete elements is determined, process 1940 returns to step 1943. Conversely, depending on the determination in step 1945 that no other set of continuous elements is determined, process 1940 proceeds to step 1946.

[0366] In step 1946, a set of continuous discrete elements is selected. According to various embodiments, the set of continuous discrete elements is selected from a set (e.g., multiple sets) such as a set determined based on repeatedly performing steps 1943-1945 of process 1940. In some embodiments, the set of continuous discrete elements is selected as a set of continuous discrete elements corresponding to a flight plan for a particular asset (e.g., a selected set of continuous discrete elements corresponding to the current position, target position, and flight plan).

[0367] According to various embodiments, a selected set of continuous discrete elements is selected based at least in part on the value of a cost function determined for the selected set of continuous discrete elements. For example, the system determines an optimal set of continuous discrete elements from a set of continuous discrete elements (determined, for example, based on repeatedly performing steps 1943-1945 of process 1940) based at least in part on the cost function. For example, the system selects a set of continuous discrete elements with the lowest cost. As another example, the system selects multiple flight plans for multiple assets based on the optimal aggregate cost associated with multiple assets moving from their respective current positions to their respective target positions.

[0368] In step 1947, information regarding the flight plan is provided. For example, the flight plan information includes a selected set of continuous discrete elements.

[0369] In step 1948, a determination is made as to whether the process is complete or not. For example, process 1940 may be determined to be complete based at least in part on user input (such as input to cancel or pause the work). For another example, the process may be determined to be complete in response to the user choosing to end the work. For yet another example, the process may be determined to be complete in response to the determination that there are no further flight plans to be determined (e.g., one or more tasks associated with the completion of the work). If the process is deemed to be complete, process 1940 terminates. Otherwise, process 1900 returns to step 1941, in which another flight plan is determined by repeatedly executing steps 1941-1947.

[0370] Various examples of embodiments described herein are described in relation to flowcharts. These examples may include specific steps performed in a particular order, but depending on the various embodiments, the various steps may be performed in a different order. In some embodiments, some steps may be combined or excluded from the examples discussed herein.

[0371] Although the embodiments described above have been explained in some detail for the sake of clarity, the present invention is not limited to the details provided. Many alternative methods exist for carrying out the present invention. The disclosed embodiments are illustrative and not intended to be limiting.

Claims

1. It's a drone, Communication interface, One or more processors connected to the aforementioned communication interface, Equipped with, The one or more processors described above are: The communication interface receives an indication that the drone is part of an asset group, the asset group is tasked with performing one or more elements of one or more tasks, and the asset group includes multiple drones. Information regarding the one or more elements is communicated via the aforementioned communication interface. The information relating to the one or more elements is communicated to at least one other drone in the asset group. The information relating to the one or more elements is at least partially based on information acquired by one or more sensors of the asset group. The information relating to the one or more elements is used in connection with determining a plan for performing the one or more tasks. A drone configured to communicate information regarding the plan for performing the one or more tasks via the communication interface, and the information regarding the plan for performing the one or more tasks is communicated to at least one other drone in the asset group.

2. A drone according to claim 1, The aforementioned drone is the leader drone among at least a portion of the asset group, The drone further comprises one or more processors configured to receive information from a work control system regarding the one or more elements performed by the asset group.

3. A drone according to claim 1, wherein the information relating to the plan for performing the one or more tasks is communicated. A drone comprising receiving the information relating to the plan for performing the one or more tasks from at least one other asset in the asset group.

4. A drone according to claim 1, Communicating the information relating to the plan for performing the one or more tasks is, The system includes receiving the information relating to the plan for performing the one or more tasks from at least one other asset in the asset group, The information relating to the plan for performing the one or more tasks includes the completion status of at least one of the elements of the drone.

5. A drone according to claim 1, Communicating the information relating to the plan for performing the one or more tasks is, The system includes receiving the information relating to the plan for performing the one or more tasks from at least one other asset in the asset group, The information relating to the plan for performing the one or more tasks includes the result of at least one of the one or more elements of the drone, the result being at least in part based on sensor information obtained by one or more sensors of the at least one other asset.

6. A drone according to claim 1, wherein the information relating to one or more elements is communicated, A drone comprising communicating instructions to one or more other assets in the asset group for performing at least one or more of the aforementioned elements.

7. A drone according to claim 1, wherein the information relating to the one or more elements includes one or more constraint parameters relating to the performance of at least one of the one or more tasks.

8. The drone according to claim 1, wherein the one or more processors further At least some of the plurality of drones are configured to determine the plan for performing at least some of the one or more tasks, The drone communicates at least a portion of the plan to at least one other drone among the plurality of drones.

9. A drone according to claim 1, The one or more processors described above further: At least some of the plurality of drones are configured to determine the plan for performing at least some of the one or more tasks, The drone communicates at least a portion of the plan to at least one other drone among the plurality of drones. The plan is determined at least in part on the basis of performing a planning service carried out by the one or more processors of the drone.

10. A drone according to claim 1, The one or more processors described above further: At least some of the plurality of drones are configured to determine the plan for performing at least some of the one or more tasks, Determining the aforementioned plan means This includes determining, with respect to a group of drones among the plurality of drones, the corresponding portion of the plan to be performed by each of the drones in the group of drones, The drone communicates at least the portion of the plan to the corresponding drone in the group of drones.

11. A drone according to claim 1, The one or more processors described above further: At least some of the drones determine the plan for performing at least some of the one or more tasks, Determining the plan comprises determining, for a group of drones among the plurality of drones, the corresponding portion of the plan to be performed by each of the drones in the group of drones. The system is configured to receive functional information from the group of drones indicating the functions supported by each of the drones in the group. The drone communicates at least the portion of the plan to the corresponding drone in the group of drones. A system in which the corresponding portion of the plan performed by each of the drones in the group of drones is determined at least in part on the function information indicating the function supported by each of the drones in the group.

12. A drone according to claim 1, The one or more processors described above further: At least some of the plurality of drones are configured to determine the plan for performing at least some of the one or more tasks, Determining the aforementioned plan means From a group of drones among the aforementioned multiple drones, functional information indicating the functions supported by each of the drones in the group is received. The work control system receives suggestions for the one or more tasks to be performed by the asset group, The planning service performed on the drone breaks down the one or more tasks into one or more elements that are performed by at least a portion of the asset group. Assign the one or more elements to one or more assets of the asset group, Determine the plan for carrying out one or more of the above elements, With respect to the group of drones, the system includes determining the corresponding portion of the plan to be performed by each of the drones in the group. The drone communicates at least the portion of the plan to the corresponding drone in the group of drones. The corresponding portion of the plan performed by each of the drones in the group of drones is determined at least in part on the function information indicating the functions supported by each of the drones in the group, Communicating the information relating to the one or more elements includes transmitting at least a portion of the plan for performing the one or more elements to the one or more assets to which the one or more elements are assigned.

13. A drone according to claim 1, The one or more processors described above further: At least some of the drones determine the plan for performing at least some of the one or more tasks, Determining the aforementioned plan means From a group of drones among the aforementioned multiple drones, functional information indicating the functions supported by each of the drones in the group is received. The work control system receives suggestions for the one or more tasks to be performed by the asset group, The planning service performed on the drone breaks down the one or more tasks into one or more elements that are performed by at least a portion of the asset group. Assign the one or more elements to one or more assets of the asset group, Determine the plan for carrying out one or more of the above elements, With respect to the group of drones, the system includes determining the corresponding portion of the plan to be performed by each of the drones in the group. The information relating to the one or more elements is configured to determine how it is communicated to the one or more assets to which the one or more elements are assigned. The drone communicates at least the portion of the plan to the corresponding drone in the group of drones. The corresponding portion of the plan performed by each of the drones in the group of drones is determined at least in part on the function information indicating the functions supported by each of the drones in the group, Communicating the information relating to the one or more elements includes transmitting at least a portion of the plan for executing the one or more elements to the one or more assets to which the one or more elements are assigned. The method by which the information relating to one or more elements is communicated is determined at least in part on the network constraints of the drone or at least one asset within the asset group.

14. A drone according to claim 1, Communicating planning information relating to the plan for performing the one or more tasks comprises receiving feedback information from at least one other drone. The feedback information includes state information relating to one or more of the following: the state of the at least one other drone, the functional state of the at least one other drone, the state of the execution of the one or more tasks, and the results relating to the execution of the one or more tasks. The drone is further configured to update the plan based at least in part on the feedback information, one or more of the processors.

15. A drone according to claim 1, Communicating planning information relating to the plan for performing the one or more tasks comprises receiving feedback information from at least one other drone. The feedback information includes state information relating to one or more of the following: the state of the at least one other drone, the functional state of the at least one other drone, the state of the execution of the one or more tasks, and the results relating to the execution of the one or more tasks. The one or more processors described above further: Based at least partially on the aforementioned feedback information, the plan is updated, A drone configured to transmit information about the updated plan to at least one other drone in response to the update of the aforementioned plan.

16. A drone according to claim 15, Communicating planning information relating to the plan for performing the one or more tasks comprises receiving feedback information from at least one other drone. The feedback information includes state information relating to one or more of the following: the state of the at least one other drone, the functional state of the at least one other drone, the state of the execution of the one or more tasks, and the results relating to the execution of the one or more tasks. The one or more processors described above further: Based at least partially on the aforementioned feedback information, the plan is updated, In response to updating the aforementioned plan, it is configured to transmit information regarding the updated plan to at least one other drone. The updated information is at least partially based on feedback information transmitted to the drone by multiple other assets within the asset group.

17. A drone according to claim 1, The aforementioned drone is the leader drone of at least some of the asset group, Communicating the information relating to the plan for performing the one or more tasks is, The drone is configured to transmit feedback information regarding the execution status of the one or more tasks, The aforementioned feedback information is transmitted to the drone's work control system.

18. A drone according to claim 1, The aforementioned drone follows the leader drone selected from among the multiple drones, A drone that communicates the information relating to one or more of the elements, comprising transmitting information acquired by one or more sensors of the drone to the leader drone.

19. A drone according to claim 1, The aforementioned drone follows the leader drone selected from among the multiple drones, A drone that communicates the information relating to the plan for performing one or more of the aforementioned tasks, comprising receiving commands from the leader drone to perform at least a portion of the plan.

20. A drone according to claim 1, The aforementioned drone follows the leader drone selected from among the multiple drones, Communicating the information relating to the plan for performing one or more of the aforementioned tasks comprises receiving an instruction from the leader drone to perform at least a portion of the plan. The instructions for performing at least a portion of the plan are provided to the drone in relation to one or more constraint parameters relating to the performance of at least one of the one or more tasks.

21. A drone according to claim 1, The aforementioned drone follows the leader drone selected from among the multiple drones, Communicating the information relating to the plan for performing one or more of the aforementioned tasks comprises receiving an instruction from the leader drone to perform at least a portion of the plan. A drone that communicates the information relating to one or more of the elements, and includes transmitting functional information relating to the functions of the drone to the leader drone.

22. A drone according to claim 21, The aforementioned drone follows the leader drone selected from among the multiple drones, Communicating the information relating to the plan for performing one or more of the aforementioned tasks comprises receiving an instruction from the leader drone to perform at least a portion of the plan. Communicating the information relating to one or more of the elements comprises transmitting functional information relating to the functions of the drone to the leader drone. The function information relating to the functions of the drone includes an updated state of the functions of the drone.

23. A method for carrying out a plan using drones, The system receives an indication via a communication interface that the drone is part of an asset group, the asset group is tasked with performing one or more elements of one or more tasks, and the asset group includes multiple drones. Information regarding the one or more elements is communicated via the aforementioned communication interface. The information relating to the one or more elements is communicated to at least one other drone in the asset group. The information relating to the one or more elements is at least partially based on information acquired by one or more sensors of the asset group. The information relating to the one or more elements is used in connection with determining a plan for performing the one or more tasks. A method comprising communicating information relating to the plan for performing the one or more tasks via the communication interface, wherein the information relating to the plan for performing the one or more tasks is communicated to at least one other drone in the asset group.

24. A computer program product for implementing a plan using a drone, embodied in a persistent computer-readable medium, A computer instruction to receive an indication via a communication interface that the drone is part of an asset group, the asset group being tasked with performing one or more elements of one or more tasks, and the asset group including multiple drones, A computer instruction for communicating information about the one or more elements via the aforementioned communication interface, The information relating to the one or more elements is communicated to at least one other drone in the asset group. The information relating to the one or more elements is at least partially based on information acquired by one or more sensors of the asset group. The information relating to the one or more elements is used in connection with determining a plan for performing the one or more tasks. A computer program product comprising: computer instructions for communicating information regarding the plan for performing the one or more tasks via the communication interface, wherein the information regarding the plan for performing the one or more tasks is communicated to at least one other drone in the asset group.

Images