Method, device and system for managing a robotic system

Abstract

A method for managing a robotic system includes determining a connection performance metric indicative of a current performance characteristic of a connection to a remote control center; determining safety envelope descriptions from a set based on the connection performance metric, wherein each description includes a mapping of the connection performance metric to an operational parameter (the operational parameter defining an operational parameter for the system), and the determined safety envelope description maps the connection performance metric to a first operational parameter; determining that the first operational parameter is different from a second operational parameter currently being used by the system; and applying the first operational parameter to govern operation of the system.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of managing autonomous robotic systems; and more particularly, to managing the operation of an autonomous robotic system based on the status of a communication channel with a remote control center as indicated by an adaptive virtual deadman's switch. Background Art

[0002] There is growing interest in using autonomous robotic systems (ARS), including unmanned aerial vehicles (UAVs) and automated guided vehicles (AGVs), for a wide variety of applications throughout society. Examples include delivery services, aerial photography and filmmaking, remote sensing tasks for agriculture, urban planning, civil engineering, and support for public safety and rescue services. To serve these applications, ARS is developing at an increasingly rapid pace.

[0003] ARS developed and tested in controlled environments and closed test tracks have since been deployed in field environments. For example, over the past few years, ARS has been deployed in real-world / field environments. This includes the deployment of self-driving shuttles, autonomous freight trucks, construction and agricultural vehicles, surveillance and delivery UAVs / drones, and warehouse inventory tracking robots. With all of these advances, ARS is entering a new phase of development where the primary focus of effective operation and scaling is on building prototypes with the goal of demonstrating the technical feasibility of autonomous operation. As effective operation and scaling become the primary focus of ARS deployments, the integration of global wide-area connectivity and services will take center stage when evaluating connectivity options. Summary of the Invention

[0004] A method for managing the operation of an autonomous robotic system based on a connection status between the autonomous robotic system and a remote control center is described, the method managing the operation of the autonomous robotic system. The method includes determining, by the autonomous robotic system, a set of connection performance metrics indicating current performance characteristics of a connection between the autonomous robotic system and a remote control center during a mission; determining, by the autonomous robotic system, a first safety envelope description from a set of safety envelope descriptions for the autonomous robotic system based on the set of connection performance metrics, wherein each safety envelope description in the set of safety envelope descriptions includes a mapping of the connection performance metrics to operational parameters that define operating parameters for the autonomous robotic system, and the first safety envelope description maps the set of connection performance metrics to a first set of operational parameters; determining, by the autonomous robotic system, that the first set of operational parameters is different from a second set of operational parameters currently being used by the autonomous robotic system; and in response to determining that the first set of operational parameters is different from the second set of operational parameters, applying, by the autonomous robotic system, the first set of operational parameters such that the first set of operational parameters governs operation of the autonomous robotic system during the mission.

[0005] A non-transitory computer-readable storage medium is described that stores instructions that, when executed by a set of one or more processors of an autonomous robotic system in communication with a remote control center via a connection, cause a computing device to: determine a set of connection performance metrics indicative of current performance characteristics of the connection between the autonomous robotic system and the remote control center during a mission; determine a first safety envelope description from a set of safety envelope descriptions for the autonomous robotic system based on the set of connection performance metrics, wherein each safety envelope description in the set of safety envelope descriptions includes a mapping of the connection performance metrics to operational parameters that define operational parameters for the autonomous robotic system, and the first safety envelope description maps the set of connection performance metrics to a first set of operational parameters; determine that the first set of operational parameters is different from a second set of operational parameters currently being used by the autonomous robotic system; and in response to determining that the first set of operational parameters is different from the second set of operational parameters, apply the first set of operational parameters such that the first set of operational parameters governs operation of the autonomous robotic system during the mission.

[0006] The embodiments described herein describe the implementation of an Adaptive Virtual Non-Live Switch (AVDMS), which provides a practical approach for optimizing the operation of autonomous robotic systems. In particular, the operation of the AVDMS assists in complying with specific safety requirements under different connectivity conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The present invention may best be understood by referring to the following description and accompanying drawings which illustrate embodiments of the invention. In the drawings:

[0008] Figure 1 Illustrated is an unmanned transportation system for managing the operation of a collection of autonomous robotic systems (ARS) with support from a remote control center (RCC) according to one example embodiment.

[0009] Figure 2 A block diagram of an ARS is illustrated according to an example embodiment.

[0010] Figure 3 A component diagram illustrating an Adaptive Virtual Non-Live Switch (AVDMS) ARS device according to an example embodiment.

[0011] Figure 4A A set of Security Envelope Definitions (SEDs) mapping connectivity performance metrics / parameters to a set of operational parameters is shown according to an example embodiment.

[0012] Figure 4B A set of security envelope definitions (SEDs) that map connectivity performance metrics / parameters and environmental conditions to a set of operational parameters is shown according to an example embodiment.

[0013] Figure 5 A component diagram illustrating an AVDMS RCC apparatus according to an example embodiment.

[0014] Figure 6 Illustrated is an unmanned transportation system for managing the operation of a collection of autonomous robotic systems (ARS) with support from a remote control center (RCC) and an intermediate control entity (ICE), according to one example embodiment.

[0015] Figure 7 A method for managing operations of an autonomous robotic system based on a connection status between the autonomous robotic system and a remote control center according to one embodiment is illustrated. The method manages operations of the autonomous robotic system.

[0016] Figure 8 A computing / networking device is illustrated according to an example embodiment. DETAILED DESCRIPTION

[0017] In the following description, numerous specific details are set forth. However, it is understood that embodiments of the present invention can be practiced without these specific details. In other instances, well-known circuits, structures, and techniques are not shown in detail in order to avoid obscuring the understanding of this description. Using the included description, one of ordinary skill in the art will be able to implement appropriate functionality without undue experimentation.

[0018] Bracketed text and boxes with dashed borders (e.g., long dashed lines, short dashed lines, dot-dashed lines, and dots) are used herein to illustrate optional operations that add additional features to embodiments of the present invention. However, such notation should not be taken to mean that in certain embodiments of the present invention, these are the only options or optional operations and / or that boxes with solid borders are not optional.

[0019] References in this specification to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it is considered within the knowledge of those skilled in the art to achieve (affect) such feature, structure, or characteristic in conjunction with other embodiments, whether or not explicitly described.

[0020] In the following description and claims, the terms "coupled" and "connected," along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. "Coupled" is used to indicate that two or more elements, which may or may not be in direct physical or electrical contact with each other, cooperate or interact with each other. "Connected" is used to indicate the establishment of communication between two or more elements that are coupled to each other.

[0021] Electronic devices use machine-readable media (also called computer-readable media) to store and transmit code (which consists of software instructions and is sometimes called computer program code or computer program) and / or data (internally and / or with other electronic devices over a network), such as machine-readable storage media (e.g., magnetic disks, optical disks, read-only memory (ROM), flash memory devices, phase-change memory) and machine-readable transmission media (also called carriers) (e.g., electrical, optical, radio, acoustic or other form of propagated signals - such as carrier waves, infrared signals). Thus, an electronic device (e.g., a computer) includes hardware and software, such as a set of one or more processors coupled to one or more machine-readable storage media to store code for execution on the set of processors and / or to store data. For example, an electronic device may include non-volatile memory containing code, because the non-volatile memory is able to persist code even when the electronic device is turned off, and when the electronic device is turned on, the code portions to be executed by the electronic device's processor(s) are typically copied from the electronic device's slower non-volatile memory to a volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)). Typical electronic devices also include a collection or one or more physical network interfaces for establishing network connections with other electronic devices (to transmit and / or receive code and / or data using propagated signals). One or more portions of embodiments of the present invention may be implemented using different combinations of software, firmware, and / or hardware.

[0022] Figure 1 An unmanned transportation system 100 for managing the operation of a collection of autonomous robotic systems (ARS) 104 is shown according to an example embodiment. As used herein, an ARS 104 may be an unmanned aerial vehicle (UAV) (sometimes referred to as a drone or unmanned aircraft system (UAS)) and / or an automated guided vehicle (AGV) (i.e., a vehicle that operates on the ground or in water). The unmanned transportation system 100 may be used to manage the operation, including paths and operational parameters (e.g., maximum speed and / or maximum altitude), of the ARS 104 that are supervised and / or owned by corresponding ARS operators 106 (e.g., human operators) via a remote control center (RCC) 108. For example, Figure 1 As shown in the ARS operators 1061-106 P Coupled to the RCC 108 to control and / or monitor one or more ARS 1041-104 NFor example, one ARS operator 106 can manage one or more ARSs 104 . In some embodiments, the ARS operator 106 can utilize an electronic device (e.g., a smartphone, tablet, laptop, etc.) for connecting and communicating with the RCC 108 to manage one or more ARSs 104 .

[0023] Although described with respect to a set of ARS operators 106 managing the ARS 104, the ARS 104 may be autonomous such that the ARS operator 106 provides general guidance to the ARS 104 via the RCC 108. For example, the ARS operator 106 may provide the ARS 104 with tasks or goals to be accomplished, and the ARS 104 autonomously performs the tasks and / or achieves the goals without further interaction with the ARS operator 106. As will be described in greater detail below, the ARS 104 may achieve the set of tasks or operations based on a varying set of operational parameters determined by the state of the communication channel 110 between the ARS 104 and the RCC 108 and / or the environment in which the ARS 104 operates.

[0024] like Figure 1 As shown in ARS 1041-104 NThe ARS 104 is coupled to the RCC 108 via a collection of networks 102. The collection of networks 102 (sometimes referred to as networks 102) can be a combination of any networks, including a combination of cellular networks. In some embodiments, the collection of networks 102 can include a 3rd Generation Partnership Project (3GPP) network system. For example, the collection of networks 102 can include an Evolved Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access Network (E-UTRAN), a Universal Terrestrial Radio Access Network (UTRAN), and / or a Global System for Mobile Communications (GSM) Enhanced Data Rates for GSM Evolution (EDGE) Radio Access Network (GERAN). The collection of networks 102 can be managed by a network operator (e.g., a cellular network operator), and the ARS 104 can each be one or more subscribers to these networks 102. The collection of networks 102 can include various network devices. In some embodiments, each network device can be an electronic device capable of being communicatively connected to other electronic devices (e.g., other network devices, user equipment devices (such as ARS 104), radio base stations, etc.). In some embodiments, a network device may include radio access features that provide wireless radio network access to other electronic devices, such as user equipment devices (UEs) (e.g., "radio access network devices" may refer to such network devices). For example, a network device may be a base station, such as an eNodeB in Long Term Evolution (LTE), a NodeB in Wideband Code Division Multiple Access (WCDMA), or other types of base stations, as well as a radio network controller (RNC), a base station controller (BSC), or other types of control nodes. Each of these network devices (which includes radio access features to provide wireless radio network access to other electronic devices) may be referred to as a cell, a tower, a cellular tower, or the like. As will be described in more detail below, the collection of networks 102 communicates via corresponding channels 1101-110 X (sometimes referred to as connection 1101-110 X ) Promote ARS 1041-104 N and transmission of messages between RCC 108.

[0025] As mentioned above, ARS 1041-104 N Any type of robotic vehicle or system, including those that are fully autonomous (e.g., completely without human operators (e.g., ARS operators 1061-106 P ) or partially autonomous (e.g., an ARS 104 that performs a task or achieves a goal only partially with the assistance of a human operator).

[0026] Figure 2A block diagram of the ARS 104 according to one example embodiment is shown. Each element of the ARS 104 will be described below by way of example, with the understanding that each ARS 104 may include more or fewer components than those shown and described herein.

[0027] like Figure 2 As shown in FIG, the ARS 104 may include engines 2021-202 controlled by one or more engine controllers 204. N The one or more engine controllers 204 control the engines 2021-202 N The speed of rotation (e.g., revolutions per minute). As used herein, the term engine may be used synonymously with the term motor and shall designate a machine that converts one form of energy into mechanical energy. For example, engine 2021-202 N The engines 202 may be electric engines that convert the electricity stored in the batteries 206 into mechanical energy. The ARS 104 may include any number of engines 202 placed in any configuration relative to the body of the ARS 104 and / or the intended heading of the ARS 104. For example, the engines 202 may be configured so that the ARS 104 is a multi-rotor helicopter (e.g., a quadcopter). In other embodiments, the engines 202 may be configured so that the ARS 104 is a fixed-wing aircraft (e.g., a single-engine or twin-engine airplane). In these embodiments, the engines 202 are used in conjunction with other elements of the ARS 104 to keep the ARS 104 flying and / or propel the ARS 104 in the desired direction. In some embodiments, the ARS 104 may not include an engine 202 for propelling the ARS 104 forward. In this embodiment, the ARS 104 may be a glider or a lighter-than-air aircraft (e.g., a weather balloon). Although described with respect to an aircraft, Figure 2 The ARS 104 shown in FIG. 1 may be a ground vehicle, a water vessel, or any other type of vehicle / system. Therefore, aircraft or components commonly employed in aircraft are used in the description for illustrative purposes.

[0028] As described above, the engines 202 are controlled by one or more engine controllers 204, which govern the rotational speed of each engine 202. In one embodiment, the engine controllers 204 may work in conjunction with actuators 210 and actuator controllers 208 that control the pitch, angle, and / or rotation of propellers, flaps, slats, slots, rotors, rotor blades / wings, shafts, and other control systems. The engine controllers 204 and actuator controllers 208 may be managed / controlled by one or more processors 212A communicatively coupled to a memory 212B and one or more interfaces 212C.

[0029] In some embodiments, the memory 212B may store instructions that, when executed by the processor 212A, cause the ARS 104 to move (vertically or horizontally) in a particular direction, maintain a particular flight path (e.g., hover at a particular altitude), and / or comply with a set of updateable / modifiable operational parameters / constraints (e.g., maximum speed and / or maximum altitude) via adjustments to settings / parameters of the engine controller 204 and the actuator controller 208.

[0030] The ARS 104 may communicate with one or more other devices (e.g., the RCC 108) using one or more interfaces 212C. In one embodiment, one of the interfaces 212C in the ARS 104 may comply with a 3GPP protocol, such as one used by the set of networks 102, so that the ARS 104 can be associated with or otherwise operate in the set of networks 102. For example, the interface 212C may comply with one or more of Global System for Mobile Communications (GSM) (including General Packet Radio Service (GPRS) and Enhanced Data Rates for GSM Evolution (EDGE)), UMTS (including High Speed ​​Packet Access (HSPA)), and Long Term Evolution (LTE). In some embodiments, the one or more interfaces 212C in the ARS 104 may allow the ARS operator 106 and / or the RCC 108 to control, monitor, or otherwise communicate with the ARS 104.

[0031] As described above, the RCC 108 can maintain a set of connections 110 with the corresponding ARS 104. For example, each ARS 104 can establish a set of connections 110 with the RCC 108 to communicate a set of messages, organized in the form of data streams, via the set of networks 102. The set of connections 110 can be established through one or more interfaces 212C and can form a wireless command and control (C2) connection to allow the RCC 108 to control and / or monitor the ARS 104 and establish the connectivity and / or operational status of the ARS 104. In some embodiments, the set of connections 110 can additionally allow the RCC 108 and the ARS operator 106 to receive data from the ARS 104. For example, the data can include images, video streams, telemetry data, and system status (e.g., battery level / status).

[0032] The ARS 104 may be required to comply with minimum operational performance standards, metrics, and / or regulations. Some of these performance metrics may be specified by industry-specific standardization organizations and regulatory bodies. For example, the National Highway Traffic Safety Administration (NHTSA) sets standards and rules for automobiles or other ground-based devices operating within the United States, while the Radio Technical Commission for Aeronautics (RTCA) provides guidance for airborne systems operating within the United States. In addition to these baseline performance metrics, the entity responsible for the operation of the ARS 104 may define additional performance metrics. These performance metrics may vary based on the actual usage of the ARS 104 (e.g., environmental conditions, wind speed, or other environmental conditions that can indicate whether the ARS 104 is deployed in a rural or urban environment).

[0033] Regardless of the source of the performance metrics, the purpose of these performance metrics is to ensure that the ARS 104 operates safely in its environment (i.e., each ARS 104 is within a safety envelope for the specific mission that the corresponding ARS 104 is performing). Under normal operation, the state of the ARS 104 should be within the constraints / parameters defined by the performance metrics (e.g., within a safety envelope defined by the performance metrics). In particular, operational parameters (e.g., the maximum rate and / or altitude of the ARS 104) ensure that a set of performance metrics (e.g., latency, packet loss rate, and throughput metrics relative to the connection 110) are met. Violation of the performance metrics may place the ARS 104 in a non-nominal operating mode or state and require immediate corrective action to return to a safe mode / state. These corrective actions may, for example, include returning the ARS 104 to a base or immediately halting the forward movement of the ARS 104 (e.g., maintaining the ARS 104 in a waiting pattern) to await guidance from the ARS operator 106.

[0034] With respect to the network connectivity or communication status (status / states) between the ARS 104 and the RCC 108, two states or modes may be considered: (1) a normal or nominal state, in which the network connectivity or communication status between the ARS 104 and the RCC 108 satisfies a set of performance metrics (e.g., the connection 110 between the ARS 104 and the RCC 108 satisfies the set of performance metrics) and (2) a non-nominal or lost communication state, in which the network connectivity or communication status between the ARS 104 and the RCC 108 fails to satisfy the set of performance metrics (e.g., no reply is received from the ARS 104 or the RCC 108 via the connection 110 during one or more consecutive timeout intervals). For example, these network connectivity or communication states may describe the signal quality through the connection 110 between the ARS 104 and one of the nodes in the set of networks 102. In some cases, the two network connectivity or communication states described above may be supplemented by a third state: a degraded network connectivity state. This degraded network connectivity state can serve as an early warning that network connectivity may be lost with respect to the ARS 104 in the near future. Moving to the degraded network connectivity state can trigger preventative action before a critical lost communications state is reached. For example, upon detecting that the radio signal strength on the ARS 104 has dropped below a threshold, the ARS 104 can enter a degraded network connectivity state, which indicates that the ARS 104 is approaching the maximum transmission range of the radio communication link between the ARS 104 and the network 102, which connects the ARS 104 to the RCC 108. Entering the degraded network connectivity state can trigger corrective action to change the trajectory of the ARS 104, thereby potentially avoiding the ARS 104 from entering a non-nominal or lost communications state (e.g., preventing further deterioration of network connectivity).

[0035] However, wireless communications are inherently more nuanced, particularly technologies that support multiple concurrent users and rely on networked communication nodes (e.g., Long Term Evolution (LTE) / 5th Generation 3GPP networks). Communication performance may change frequently due to dynamic radio network conditions (such as propagation loss and interference), as well as varying degrees of congestion due to varying utilization of radio channels. For example, under favorable conditions, the latency of the network 102 may be on the order of tens of milliseconds. However, under unfavorable conditions, a particular ARS 104 may experience latency on the order of hundreds of milliseconds. In order to operate safely and efficiently, the architecture of the unmanned transportation system 100 needs to be resilient to changes in communication quality. In essence, a minimum level of autonomous operation capability needs to be retained for the ARS 104 even when connectivity between the ARS 104 and the RCC 108 is lost or degraded.

[0036] In some embodiments, the unmanned transportation system 100 selects operational parameters of the ARS 104 from a set definition of a safe operating envelope to adapt the operation of the ARS 104 to the dynamically changing performance of the communication channel 110 so that the performance of the ARS 104 is maximized (e.g., the rate of the ARS 104 in the route is maximized) while ensuring a safe performance metric relative to the connection 110 (e.g., latency, packet loss rate, and / or throughput still meet the set of performance metrics). For example, an adaptive virtual non-live staff switch (AVDMS) can be used to exchange messages between the ARS 104 and the RCC 108 to (1) monitor the connectivity performance of the keep-alive message flow and the associated command, control, and telemetry communication channel 110 (i.e., establish current performance metrics for the communication channel 110) and (2) coordinate changes in the operational parameters of the ARS 104 accordingly (e.g., adjust the maximum rate and / or altitude of the ARS 104 while navigating the route).

[0037] An example of an adaptive virtual inactive person switch includes an autonomous UAV-type ARS 104 flying in a densely populated area. In this example embodiment, the safe operation of the fully autonomous ARS 104 requires telemetry updates corresponding to the ARS for the RCC 108 to track the normal operation of the ARS 104. In particular, the ARS 104 will need to receive keep-alive messages from the RCC 108 while transmitting telemetry data on the keep-alive message stream. If these keep-alive messages are not received within a predetermined period of time, the ARS 104 will enter a non-nominal or lost communication state, triggering corrective action (e.g., immediate stop / hover / wandering of the ARS 104).

[0038] In particular, the drone-type ARS 104 in this example transmits telemetry data to the RCC 108 at prescribed intervals. The RCC 108 requires periodic telemetry data updates from the ARS 104 to check system health, including performance metrics associated with the connection 110, and general progress of the mission. In response to the telemetry data or independently of the telemetry data (e.g., at a separate rate or interval), the RCC 108 transmits keep-alive messages to the ARS 104 via the connection 110. As described above, if the ARS 104 does not receive these keep-alive messages at the predetermined rate or interval, an error state will occur. Assuming a current maximum speed of 100 miles per hour (mph) (as defined by Federal Aviation Administration (FAA) regulations), the ARS 104 can travel approximately 45 meters per second (mps) when traveling at this maximum speed. If the ARS operator 106 wants to limit the outage period or distance (i.e., the time or distance traveled by the ARS 104 between the last telemetry data transmitted by the ARS 104, the receipt of the telemetry data by the RCC 108, the transmission of the subsequent action instruction by the RCC 108 to the ARS 104, and the receipt of the action instruction by the ARS 104 (i.e., the complete round-trip time (RTT) delay)) to the physical distance traveled by the ARS 104 of 4.5 meters, then the keep-alive messages need to be exchanged within 100 milliseconds, which requires an RTT delay of at most 100 ms. To accommodate delay variations and potential packet loss, the keep-alive messages may need to be exchanged more frequently to reduce the required RTT. In this scenario, if the RTT increases above 100 ms, a non-nominal state will be triggered, affecting the mission of the ARS 104. Another approach is to relax the delay requirement to a higher amount (e.g., 200 ms), which increases the outage period to a travel distance of nine meters.

[0039] In order to improve operational efficiency while maintaining safety standards for the mission of the ARS 104, the embodiments described herein adapt the operating parameters of the ARS 104 to the current actual latency performance of the connection 110. In particular, the ARS 104 and the RCC 108 can continuously measure the RTT along with other network performance metrics on the connection 110 (e.g., jitter, packet loss, throughput, etc.), and based on a predefined safety distance (e.g., 4.5 meters) corresponding to the outage period, can adjust the operational parameters of the ARS 104. In this example, it will reduce or increase the maximum allowed rate of the ARS 104 based on the network performance metrics. For example, when the RTT on the connection 110 between the ARS 104 and the RCC 108 is 200ms, the ARS 104 and / or the RCC 108 will reduce the maximum allowed rate of the ARS 104 to 50mph because updates are not being received frequently enough to guarantee a higher maximum rate. When network conditions improve, the maximum allowed rate of ARS 104 can be increased while still maintaining a low RTT and the resulting high data update frequency. This adaptive operation optimizes mission execution without compromising safety.

[0040] Another example involves remote control of the ARS 104. The ARS operator 106 can provide assistance to help the ARS 104 get out of a difficult situation. Safe remote control requires real-time exchange of sensor data (e.g., telemetry data and video camera data) and control commands (e.g., joystick movement) between the ARS 104 and the ARS operator 106. The higher the resolution of the data from the ARS 104, the better the ARS operator 106 can assess the situation and perceive the movement of the ARS 104. On the control side, the lower the round-trip delay (i.e., RTT) between the ARS 104 and the ARS operator 106 via the C2 connection 110, the better the control experience of the ARS operator 106. Similar to the previous example, the operational envelope with corresponding operational parameters can be adjusted to ensure optimal and safe remote operation under different communication performance conditions. In this case, the operational envelope and parameters can specify at what rate the ARS movement is performed to ensure a specific video frame rate and RTT delay. If latency is high, the ARS 104 may move slower to reduce interruption periods for the ARS operator 106, while under favorable connectivity conditions, operations may be more real-time with faster robot movements.

[0041] In one embodiment, an adaptive virtual non-live-person (AVDMS) switch consists of processes running on the ARS 104 and the RCC 108, and a communication protocol operating between the ARS 104 and the RCC 108. For example, there can be two deployment modes of the adaptive virtual non-live-person switch: (1) direct mode, in which the ARS 104 communicates directly with the RCC 108; and (2) cascade mode, in which one or more intermediate control entities (ICEs) bridge AVDMS sessions / flows and act as asynchronous relays between the ARS 104 and the RCC 108 (e.g., the communication frequency between the ARS 104 and the ICE is higher than the communication frequency between the ICE and the RCC 108).

[0042] Figure 3 FIG. 1 shows a component diagram of an AVDMS ARS device 300 according to an example embodiment. The AVDMS ARS device 300 may be configured to be configured to be used in one or more ARSs 1041-104 N 108 ). For example, the AVDMS ARS device 300 may reside in the memory 212B of the ARS 104 and may be processed by the processor 212A of the ARS 104 to implement an AVDMS flow on the AVDMS communication channel. As used below, the AVDMS communication channel may be one of the channels 110 and may facilitate an AVDMS flow, which is the flow of AVDMS messages in both the upstream (i.e., from the ARS 104 toward the RCC 108) and downstream (i.e., from the RCC 108 toward the ARS 104) directions. For purposes of illustration, in the following description, the AVDMS communication channel will be referred to as the AVDMS communication channel 1101.

[0043] like Figure 3 As shown in , the AVDMS ARS device 300 may include a set of security envelope definitions (SEDs) 302 that describe operational parameters 312 (e.g., rate and altitude) of the ARS 104 that map to connectivity performance metrics / parameters (e.g., latency, jitter, packet loss, and throughput). For example, Figure 4A 3021-3024, which maps connectivity performance metrics / parameters 404 to a set of operational parameters 312. In particular, Figure 4AAs shown in FIG, the set of SEDs 3021-3024 includes SED index 402, connectivity performance metrics / parameters 404 (e.g., latency 404A, packet loss rate 404B, and throughput 404C), and operational parameters 312 (e.g., maximum rate 312A and maximum height 312B). Thus, based on the set of performance metrics / parameters 404, a set of operational parameters 312 may be selected. In some embodiments, the set of SEDs 3021-3024 may additionally include a set of local conditions. For example, Figure 4B As shown in , the set of SEDs 3021 - 3024 may include a set of local / environmental conditions 406 (eg, wind speed 406A and location 406B), which may be used along with the set of performance metrics / parameters 404 to identify the set of operational parameters 312 .

[0044] In some embodiments, the set of performance metrics / parameters 404 may span multiple SEDs 302. For example, the determined / measured packet loss rate 404B may be appropriate for SED 3022 (i.e., the determined / measured packet loss rate 404B is 0.1% or within a predetermined deviation of 0.1%), but the determined / measured throughput 404C may be appropriate for SED 3023 (i.e., the determined / measured throughput 404C is 5 MB / s or within a predetermined deviation of 5 MB / s). Alternatively, or in addition to the above examples, the determined / measured performance metrics / parameters 404 may be appropriate for SED 3021, but the environmental conditions 406 may be appropriate for SED 3022. In this case, the AVDMS ARS device 300 may select a SED 302 and a corresponding set of operational parameters 312 based on one or more factors. For example, the AVDMS ARS device 300 may take a conservative approach and select the SED 302 with the most conservative set of operational parameters 312 (e.g., the lowest maximum rate 312A and the lowest maximum altitude 312B). Alternatively, the AVDMS ARS device 300 may average the sets of operational parameters 312 of multiple potential SEDs 302 to arrive at the selected set of operational parameters 312.

[0045] If still Figure 3As shown in FIG, the AVDMS ARS device 300 may include an ARS communication performance measurement unit (CPMU) 304 that continuously monitors various communication characteristics / metrics, including one or more of latency, jitter, throughput, and packet loss characteristics experienced by AVDMS messages 314 on the AVDMS communication channel 1101. The AVDMS communication channel 1101 may be a communication channel that operates at low bandwidth and high frequency to keep the RCC 108 updated with the status of the ARS 104 and to oversee the operation of the ARS 104 by off-board systems (e.g., the RCC 108 and / or the ARS operator 106). Although the ARS CPMU 304 only monitors the performance of this particular communication channel 1101, the ARS 104 may be simultaneously using other communication channels 110 and corresponding streams for payload data (e.g., sensor data, including video data). Communication performance of other communication channels 110 (eg, non-AVDMS communication channels) may be considered independent of AVDMS communication channel 1101 measurements and operations. In one embodiment, ARS CPMU 304 is used to generate performance metrics / parameters 404 that may be used to select operational parameters 312 based on corresponding SED 302 .

[0046] like Figure 3As shown in FIG, the AVDMS ARS device 300 may include an AVDMS protocol endpoint 306 that handles AVDMS messages 314 between the ARS 104 and the RCC 108. For example, the AVDMS protocol endpoint 306 receives telemetry and task status information from the autonomous agent (AA) 308 for relay to the RCC 108 on the AVDMS communication channel 1101. For example, the AA 308 may generate telemetry and status information and transfer the information to the AVDMS protocol endpoint 306 for transmission to the RCC 108. In this configuration, the AVDMS protocol endpoint 306 checks the healthy operation of the AA 308 so that critical telemetry and status information provided by the AA 308 can be consistently provided to the RCC 108 in AVDMS messages 314. This critical information can be signaled to the RCC 108 implicitly or explicitly. Using implicit signaling, AVDMS messages 314 including telemetry and status information are transmitted to the RCC 108 at a prescribed interval / frequency. However, unless the AA 308 delivers updated telemetry and status information to the AVDMS protocol endpoint 306, no AVDMS messages 314 are sent to the RCC 108 (i.e., the AVDMS messages 314 are not transmitted at the prescribed interval / frequency). This ensures that no AVDMS messages 314 are sent in the event of headless operation of the ARS 104 (e.g., the AA 308 process has crashed or is otherwise unable to operate normally). In this scenario, other subsystems and processes of the ARS 104 may still be functioning normally (e.g., sensor data is still available), but the ARS 104 is in a safety-critical state, and the RCC 108 should be notified of this state by failing to receive AVDMS messages 314 at the prescribed interval / frequency. Since the AVDMS messages 314 will not be sent, the RCC 108 will declare a critical fault and initiate appropriate action. Additionally, using explicit signaling, the AVDMS protocol endpoint 306 may still send an AVDMS message 314 in which the endpoint 306 explicitly notifies the RCC 108 of the non-responsive AA 308 .

[0047] like Figure 3 As shown in FIG, the AVDMS ARS device 300 may include selection logic (SL) 310 that operates based on information received from the ARSCPMU 304 (e.g., performance metrics / parameters 404) and / or the AVDMS protocol endpoint 306. In particular, the SL 310 may determine the performance metrics / parameters 404 of the AVDMS communication channel 110 from the CPMU 304 and determine the highest ranking matching SED 302 having the corresponding operational parameters 312. In particular, Figure 4A and 4BThe SEDs 3021-3024 of FIG302 can be arranged in order such that SED 3021 is the highest ranked SED 302 and SED 3024 is the lowest ranked SED 302. In this example, the SL 310 compares the performance metrics / parameters 404 with SED 3021 to potentially determine a match. If a match is not determined with SED 3021, the SL 310 performs a comparison with the performance metrics / parameters 404 of the next highest ranked SED 302 (e.g., SED 3022). After finding the first match between the performance metrics / parameters 404 and the SED 302, the corresponding operational parameters 312 are selected. The determined / selected operational parameters 312 are used to configure the AA 308 under the current conditions. The ranking of the SEDs 302 is handled by the operator of the unmanned transportation system 100 and provided as a priority ranking attached to each SED 302. Although the SL 310 will attempt to locate the highest priority matching SED 302, if no SED 302 matches the current connectivity performance metric, the SL 310 will declare a critical lost communications condition, and the AA 308 will trigger associated actions (e.g., optionally default actionable parameters 312) on the ARS 104. This may occur if communications between the ARS 104 and the RCC 108 on the AVDMS communication channel 110 are lost for longer than a predetermined period of time.

[0048] Turn to RCC 108, Figure 5 Component diagram of an AVDMS RCC apparatus 500 is shown according to an example embodiment. The AVDMS RCC apparatus 500 may operate in the RCC 108.

[0049] like Figure 5 As shown in FIG, the AVDMS RCC device 500 may include an AVDMS protocol endpoint 506 that processes AVDMS protocol messages 314 from the ARS 104 and relays corresponding telemetry and mission status information to an autonomy supervisor (AS) 502, which verifies that the ARS 104 is operating correctly (e.g., according to the mission plan and / or according to operational parameters 312). If the AS 502 detects a problem, corrective action can be triggered. For example, an explicit mission abort command can be immediately sent to the ARS 104 in a subsequent AVDMS message 114.

[0050] If still Figure 5As shown in FIG, the AVDMS RCC apparatus 500 may include a communication performance measurement unit (CPMU) 504. The CPMU 504 continuously monitors various communication characteristics, including one or more of latency, jitter, throughput, and packet loss characteristics experienced by AVDMS protocol messages 314 on the AVDMS communication channel 110.

[0051] If still Figure 5 As shown in FIG, the AVDMS RCC device 500 may include a policy verification entity (PVE) 508. The PVE 508, in conjunction with the SED 302 and the corresponding operational parameters 312 selected at the AS 502, checks local performance metrics / parameters (e.g., the performance metrics / parameters 404 generated by the RCC CPMU 504) and remote measurements (e.g., the performance metrics / parameters 404 generated by the ARS CPMU 304) to verify that the correct operational parameters 312 have been selected that match the communication conditions on the AVDMS communication channel 110. If the AA 308 of the AVDMS ARS device 300 selects an inappropriate set of operational parameters 312, the AS 502 may select a new set of operational parameters 312 for use by the AA 308 and the corresponding ARS 104.

[0052] As described above, the ARS 104 and the RCC 108 may exchange AVDMS messages 314 over the AVDMS communication channel 110 . In some embodiments, the AVDMS message 314 may include one or more of the following: (1) an ARS identifier that identifies or conveys the ARS 104 or the destination ARS 104; (2) an RCC identifier that identifies or conveys the RCC 108 or the destination RCC 108; (3) a sequence number of the AVDMS message 314; (4) CPMU measurement data (e.g., performance metrics / parameters 404) that may include radio signal quality, radio signal strength, latency, jitter, throughput, and / or packet loss characteristics experienced by the AVDMS protocol message 314 on the AVDMS communication channel 110; (5) ARS telemetry data that includes one or more of the speed, heading, altitude, and / or GPS coordinates of the ARS 104 and the battery level and system status of the ARS 104; (6) ARS mission status information that may include "OK" (i.e., the ARS 104 complied with the planned mission and operational parameters 312), "LOCAL ...LOCAL" (i.e., the ARS 104 complied with the planned mission and operational parameters 312), "LOCAL" (i.e., the ARS 104 complied with the planned mission and operational parameters 312), "LOCAL" (i.e., the ARS (i.e., the ARS 104 has encountered a critical / unrecoverable condition), or "CRITICAL" (i.e., the ARS 104 mission has been aborted and the critical / unrecoverable condition will immediately cause the ARS 104 to be taken offline); (7) an ARS-selected SED 302 indicating the SED index 402 corresponding to the selected SED 302 with the corresponding operational parameters 312; and (8) an RCC command including "OK" (i.e., the ARS 104 continues autonomous operation), "PAUSE" (i.e., the ARS 104 is safely paused and awaiting intervention from the ARS (i.e., human) operator 106), "ABORT" (i.e., the ARS 104 mission has been aborted and the critical / unrecoverable condition will immediately cause the ARS 104 to be taken offline). 104 aborts the current task and executes the associated non-nominal process) and "OVERRIDE," which also indicates the SED index 402 (ie, the ARS 104 is to use the SED 302 specified by the RCC 108).

[0053] The purpose of the AVDMS keep-alive message flow is to ensure that the ARS 104 is always supervised by a remote off-board agent that can quickly suspend autonomous operations and record and trigger alternative actions. This supervision function can be delegated to another entity called an intermediate control entity (ICE). The ICE can be placed closer to the device of the ARS 104 than the RCC 108 (for example, at the edge of the set of networks 102 close to the ARS 104). In particular, the communication quality between the ARS 104 and RCC 108 with direct communication deployment may be suboptimal (for example, because of the distance separating the ARS 104 and RCC 108 and the different network domains crossed, the latency and jitter in the AVDMS communication channel 1101 may be high). To alleviate this problem, the ICE can be placed on an edge node of the network 102 and near the operating area of ​​the ARS 104. For example, Figure 6 An ICE 602 is shown at the edge of a collection of networks 102, according to one embodiment. As shown, an AVDMS communication channel 1101 connecting the ARS 104 and the RCC 108 flows through the ICE 602. In this configuration, the RCC 108 can be located based on the business needs of the fleet operator (e.g., in the city where the operating company is headquartered), regardless of the location of the deployed ARS 104.

[0054] exist Figure 6 In the cascaded deployment scenario shown in FIG, the ARS 104 communicates with the ICE 602, which in turn has a decoupled communication loop with the RCC 108 or another ICE 602 (in the case where more than one ICE 602 is cascaded in the set of networks 102). Since the ICE 602 is responsible for the delay-sensitive, high-frequency AVDMS messages 314 with the ARS 104, this AVDMS loop 6041, which can represent the AVDMS communication channel 1101, will determine the communication quality with the ARS 104, which in turn guides the SED 302 and the corresponding operational parameters 312 invoked on the ARS 104. For example, if the ICE 602 is deployed at the edge of a serving base station in an LTE / 5G network, the AVDMS loop 6041 may be as fast as 10 ms, essentially not posing any practical communication-related limitations on the operation of the ARS 104. At the same time, the communication between the ICE 602 and the RCC 108 can be more relaxed via the AVDMS loop 6042, because this loop 6042 will be mainly used to keep the RCC 108 updated, while the AVDMS loop 6041 will involve higher-frequency communication. Therefore, the AVDMS loop 6042 can have a latency of the order of hundreds of milliseconds without affecting the operational performance of the ARS 104.

[0055] Now turn Figure 7According to an example embodiment, a method 700 for managing the operation of an autonomous robotic system 104 based on the state of a connection 110 between the autonomous robotic system 104 and a remote control center 108 will be described. The method manages the operation of the autonomous robotic system 104. The method 700 will be described in conjunction with one or more other supporting figures. However, the use of these supporting figures, including the elements presented therein, is for illustrative purposes, and the method 700 may be performed in conjunction with other systems and components. Furthermore, in some embodiments, the operations of the method 700 can be performed in a different order. For example, two or more operations can be performed during at least partially overlapping time periods.

[0056] like Figure 7 As shown in , method 700 may begin at operation 702 , where the autonomous robotic system 104 transmits a first set of messages 314 to the remote control center 108 over the connection 110 .

[0057] At operation 704 , the autonomous robotic system 104 receives a second set of messages 314 from the remote control center 108 over the connection 110 .

[0058] At operation 706, the autonomous robotic system 104 determines a set of connection performance metrics 404 that indicate current performance characteristics of the connection 110 between the autonomous robotic system 104 and the remote control center 108 during the mission. In one embodiment, the set of connection performance metrics 404 indicates one or more of jitter, latency, throughput, and packet loss on the connection 110. In one embodiment, determining the set of connection performance metrics 404 is based on one or more of: (1) the transmission of the first set of messages 314 and (2) the receipt of the second set of messages 314. In one embodiment, the message 314 in the first set of messages 314 includes a set of fields, wherein the set of fields includes one or more of the following: (1) an autonomous robotic system identifier that identifies the autonomous robotic system 104; (2) a remote control center identifier that identifies the remote control center 108; (3) a sequence number of the message 314; (4) a connection performance metric 404 of the connection 110; (5) telemetry data that includes one or more of the speed, heading, altitude, coordinates, battery level, and system status of the autonomous robotic system 104; (6) task status information indicating the status of the task; and (7) a safety envelope description identifier that identifies the current safety envelope description 302 used by the autonomous robotic system 104. In one embodiment, the messages 314 in the second set of messages 314 include a set of fields, wherein the set of fields includes one or more of the following: (1) an autonomous robotic system identifier identifying the autonomous robotic system 104; (2) a remote control center identifier identifying the remote control center 108; (3) a sequence number of the message 314; (4) a connection performance metric 404 of the connection 110 that indicates the performance metric 404 of the connection 110 based on the previous set of messages 314; and (5) a remote control center command that includes one or more of the following commands: a confirmation command to confirm that the autonomous robotic system 104 is to continue to utilize the currently selected safety envelope description 302 and corresponding operational parameters 312; a pause command to cause the autonomous robotic system 104 to pause execution of a task; an abort command to cause the autonomous robotic system 104 to abort a task; and an override command to cause the autonomous robotic system 104 to use a new safety envelope description 302 and corresponding operational parameters 312. Thus, a message 314 (eg, a message 314 from a remote control center 108 ) can include the performance metric 404 for a previous time period, and characteristics of the message 314 can be used to determine the performance metric 404 for the current time period.

[0059] In one embodiment, connection 110 includes a first loop 6041 connecting autonomous robotic system 104 to intermediate control entity 602 and a second loop 6042 connecting intermediate control entity 602 to remote control center 108. In this embodiment, the set of connection performance metrics 404 describes first loop 6041. In some embodiments, autonomous robotic system 104 and intermediate control entity 602 exchange messages 314 at a first frequency on first loop 6041, and intermediate control entity 602 and remote control center 108 exchange messages 314 at a second frequency on second loop 6042. In some embodiments, the first frequency is greater than the second frequency.

[0060] At operation 708, the autonomous robotic system 104 determines a first safety envelope description 3021 from the set of safety envelope descriptions 3021-3024 for the autonomous robotic system 104 based on the set of connection performance metrics 404. In some embodiments, each safety envelope description 302 in the set of safety envelope descriptions 3021-3024 includes a mapping of the connection performance metrics 404 to operational parameters 312 that define operational parameters for the autonomous robotic system 104, and the first safety envelope description 3021 maps the set of connection performance metrics 404 to a first set of operational parameters 312. In one embodiment, the safety envelope descriptions 3021-3024 of the set are ranked from highest to lowest, such that determining the first safety envelope description 302 includes comparing the set of connection performance metrics 404 to the highest ranked safety envelope description 302 in the set of safety envelope descriptions 3021-3024 (i.e., Figure 4A and 4B 3021 in the set of security envelope descriptions 3021-3024) to determine a potential match, and then when a match with the highest-ranked security envelope description 302 is not achieved, the set of connection performance metrics 404 is compared with lower-ranked security envelope descriptions (i.e., 3022-3024) in the set of security envelope descriptions 3021-3024.

[0061] At operation 710, the autonomous robotic system 104 determines that the first set of operational parameters 312 is different from the second set of operational parameters 312 currently being used by the autonomous robotic system 104. In one embodiment, the first set of operational parameters 312 includes one or more of a maximum velocity 312A at which the autonomous robotic system 104 is permitted to operate and a maximum height 312B at which the autonomous robotic system 104 is permitted to operate.

[0062] At operation 712 , autonomous robotic system 104 applies the first set of operational parameters 312 in response to determining that the first set of operational parameters 312 is different than the second set of operational parameters 312 , such that the first set of operational parameters 312 governs operation of autonomous robotic system 104 during the mission.

[0063] At operation 714 , the autonomous robotic system 104 determines a connection status of the connection 110 based on the set of connection performance metrics 404 .

[0064] At operation 716, the autonomous robotic system 104 performs a first corrective action in response to the first value of the connection status. In one embodiment, the first value of the connection status indicates that the connection 110 is inoperable, and the first corrective action is one of: (1) the autonomous robotic system 104 is to return to the designated location and (2) the autonomous robotic system 104 is to pause and await further instructions.

[0065] At operation 718, the autonomous robotic system 104 performs a second corrective action in response to the second value of the connection status. In one embodiment, the second value of the connection status indicates that the connection 110 is operating normally but may soon become inoperable (e.g., the connection performance metric 404 is approaching an inoperable threshold). In this case, the second corrective action includes selecting a more conservative SED 302 (e.g., an SED 302 with lower operational parameters 312 (e.g., a lower maximum speed 312A and / or a lower maximum altitude 312B)). Alternatively, the second value of the connection status indicates that the connection 110 is operating normally and well within the range of the connection performance metric 404 corresponding to the currently selected SED 302. In this case, the second corrective action includes taking no further action other than continuing the current task.

[0066] Each element of the unmanned transportation system 100 may be composed of a collection of computing / networking devices or implemented in other ways. For example, Figure 8 The diagram illustrates a computing / networking device 800 according to one embodiment. As shown, the computing / networking device 800 may include a processor 802 communicatively coupled to a memory 804 and an interface 806. The processor 802 may be a microprocessor, a controller, a microcontroller, a central processing unit, a digital signal processor, an application specific integrated circuit, a field programmable gate array, any other type of electronic circuit, or any combination of one or more of the foregoing. The processor 802 may include one or more processor cores. In certain embodiments, some or all of the functionality described herein as being provided by components of the unmanned transportation system 100 may be implemented by one or more processors 802 of one or more computing / networking devices 800 executing software instructions, alone or in combination with other computing / networking device 800 components such as a memory 804.

[0067] The memory 804 may store code (which consists of software instructions and is sometimes referred to as computer program code or computer program) and / or data using non-transitory machine-readable (e.g., computer-readable) media, such as non-transitory computer-readable storage media (e.g., magnetic disks, optical disks, solid-state drives, read-only memory (ROM), flash memory devices, phase-change memory) and machine-readable transmission media (e.g., electrical, optical, radio, acoustic, or other forms of propagated signals, such as carrier waves, infrared signals). For example, the memory 804 may include non-volatile memory (e.g., non-transitory computer-readable storage media 810) containing code to be executed by the processor 802. If the memory 804 is non-volatile, the code and / or data stored therein can be retained even when the computing / networking device 800 is turned off (when power is removed). In some examples, when computing / networking device 800 is turned on, the portion of the code to be executed by processor(s) 802 can be copied from non-volatile memory to volatile memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM)) of computing / networking device 800.

[0068] The interface 806 can be used in wired and / or wireless communication of signaling and / or data to or from the computing / networking device 800. For example, the interface 806 can perform any formatting, encoding, or conversion to allow the computing / networking device 800 to send and receive data via a wired connection and / or a wireless connection. In some embodiments, the interface 806 may include a radio circuit capable of receiving data from other devices in the network via a wireless connection and / or sending data to other devices via a wireless connection. The radio circuit may include (one or more) transmitters, (one or more) receivers, and / or (one or more) transceivers suitable for radio frequency communication. The radio circuit can convert digital data into a radio signal with appropriate parameters (e.g., frequency, timing, channel, bandwidth, etc.). The radio signal can then be transmitted to (one or more) appropriate recipients via the antenna 808. In some embodiments, the interface 806 may include (one or more) network interface controllers (NICs), which are also called network interface cards, network adapters, local area network (LAN) adapters, or physical network interfaces. The NIC(s) may facilitate connecting the computing / networking device 800 to other devices, allowing them to communicate over the wire by plugging cables into physical ports connected to the NICs. In particular embodiments, the processor 802 may represent a portion of the interface 806, and some or all of the functionality described as being provided by the interface 806 may be provided in part or in whole by the processor 802.

[0069] Although the flowcharts in the accompanying drawings illustrate a particular order of operations performed by certain embodiments of the present invention, it should be understood that such order is exemplary (e.g., alternative embodiments may perform operations in a different order, combine certain operations, overlap certain operations, etc.).

[0070] Furthermore, although the present invention has been described in terms of several embodiments, those skilled in the art will recognize that the present invention is not limited to the embodiments described and can be practiced with modification and alteration within the spirit and scope of the appended claims. Therefore, this description is to be considered illustrative rather than restrictive.

Claims

1. A method (700) for managing the operation of an autonomous robotic system (104) based on a status of a connection (110) between the autonomous robotic system (104) and a remote control center (108), the method managing the operation of the autonomous robotic system comprising: determining (706) by the autonomous robotic system a set of connection performance metrics (404), the connection performance metrics indicating current performance characteristics of the connection between the autonomous robotic system and the remote control center during a mission; determining (708) by the autonomous robotic system a first safety envelope description (3021) from a set of safety envelope descriptions (3021-3024) for the autonomous robotic system based on the set of connection performance metrics, wherein each safety envelope description in the set of safety envelope descriptions includes a mapping of connection performance metrics to operational parameters (312) defining operational parameters for the autonomous robotic system, and wherein the first safety envelope description maps the set of connection performance metrics to a first set of operational parameters; determining ( 710 ) by the autonomous robotic system that the first set of operational parameters is different from a second set of operational parameters currently being used by the autonomous robotic system; as well as In response to determining that the first set of operational parameters is different from the second set of operational parameters, applying (712) the first set of operational parameters by the autonomous robotic system so that the first set of operational parameters governs operation of the autonomous robotic system during the mission, wherein the set of security envelope descriptions are arranged from highest to lowest according to a connection performance metric, such that determining the first security envelope description comprises comparing the set of connection performance metrics with the highest-ranked security envelope description in the set of security envelope descriptions to determine a potential match, and then comparing the set of connection performance metrics with lower-ranked security envelope descriptions in the set of security envelope descriptions when a match with the highest-ranked security envelope description is not achieved.

2. The method of claim 1, further comprising: transmitting (702) by the autonomous robotic system a first set of messages (314) over the connection to the remote control center; as well as receiving (704) by the autonomous robotic system from the remote control center over the connection a second set of messages (314); Wherein determining the set of connection performance metrics is based on one or more of: transmitting the first set of messages and receiving the second set of messages.

3. The method according to claim 2, wherein: The messages in the first set of messages include a set of fields, wherein the set of fields includes one or more of the following: an autonomous robotic system identifier that identifies the autonomous robotic system; a remote control center identifier that identifies the remote control center; a sequence number of the message; a connection performance metric of the connection; telemetry data, the telemetry data including one or more of the speed, heading, altitude, coordinates, battery level and system status of the autonomous robotic system; task state information indicating the status of the task; and a safety envelope description identifier that identifies a current safety envelope description used by the autonomous robotic system.

4. The method according to claim 2, wherein: The messages in the second set of messages include a set of fields, wherein the set of fields includes one or more of the following: an autonomous robotic system identifier that identifies the autonomous robotic system; a remote control center identifier that identifies the remote control center; a sequence number of the message; a connection performance metric of the connection, the connection performance metric indicating a performance metric of the connection based on a previous set of messages; and a remote control center command, the remote control center command including one or more of the following commands: a confirmation command that confirms that the autonomous robotic system is to continue to utilize the currently selected safety envelope description and corresponding operational parameters, a pause command that causes the autonomous robotic system to pause execution of the task, an abort command that causes the autonomous robotic system to abort the task, and an override command that causes the autonomous robotic system to use a new safety envelope description and corresponding operational parameters.

5. The method according to claim 1, wherein The set of connection performance metrics indicates one or more of jitter, latency, throughput and packet loss on the connection.

6. The method of claim 1, wherein: The first set of operational parameters includes one or more of a maximum velocity (312A) at which the autonomous robotic system is permitted to operate and a maximum altitude (312B) at which the autonomous robotic system is permitted to operate.

7. The method of claim 1, wherein: The connection comprises a first loop (6041) connecting the autonomous robotic system to an intermediate control entity (602) and a second loop (6042) connecting the intermediate control entity to the remote control center, and Therein, the set of connection performance metrics describes the first loop.

8. The method of claim 7, wherein: The autonomous robotic system and the intermediate control entity exchange messages on the first loop at a first frequency, and the intermediate control entity and the remote control center exchange messages on the second loop at a second frequency, and The first frequency is greater than the second frequency.

9. The method of claim 1, further comprising: determining (714), by the autonomous robotic system, a connection state of the connection based on the set of connection performance metrics; performing (716), by the autonomous robotic system, a first corrective action in response to a first value of the connection state; and A second corrective action is performed (718) by the autonomous robotic system in response to the second value of the connection status.

10. The method of claim 9, wherein: The first value of the connection status indicates that the connection is inoperable, and the first corrective action is one of: the autonomous robotic system is to return to a designated location and the autonomous robotic system is to pause and await further instructions.

11. A non-transitory computer-readable storage medium (810) storing instructions that, when executed by a collection of one or more processors (802) of an autonomous robotic system (104 / 800) in communication with a remote control center (108) via a connection (110), cause a computing device to: determining (706) a set of connection performance metrics (404) indicating current performance characteristics of the connection between the autonomous robotic system and the remote control center during a mission; Based on the set of connection performance metrics, a first safety envelope description (3021) is determined (708) from a set of safety envelope descriptions (3021-3024) for the autonomous robotic system, wherein each safety envelope description in the set of safety envelope descriptions comprises a mapping of connection performance metrics to operational parameters (312), the operational parameters defining operational parameters for the autonomous robotic system, and the first safety envelope description maps the set of connection performance metrics to a first set of operational parameters; determining ( 710 ) that the first set of operational parameters is different than a second set of operational parameters currently being used by the autonomous robotic system; as well as In response to determining that the first set of operational parameters is different from the second set of operational parameters, applying (712) the first set of operational parameters such that the first set of operational parameters governs operation of the autonomous robotic system during the mission, wherein the set of security envelope descriptions are arranged from highest to lowest according to a connection performance metric, such that determining the first security envelope description comprises comparing the set of connection performance metrics with the highest-ranked security envelope description in the set of security envelope descriptions to determine a potential match, and then comparing the set of connection performance metrics with lower-ranked security envelope descriptions in the set of security envelope descriptions when a match with the highest-ranked security envelope description is not achieved.

12. The non-transitory computer-readable storage medium of claim 11, wherein: The instructions further cause the computing device to: transmitting a first set of messages (314) to the remote control center over the connection; and receiving (704) a second set of messages (314) from the remote control center over the connection; Wherein determining the set of connection performance metrics is based on one or more of: transmitting the first set of messages and receiving the second set of messages.

13. The non-transitory computer-readable storage medium of claim 12, wherein: The messages in the first set of messages include a set of fields, wherein the set of fields includes one or more of the following: an autonomous robotic system identifier that identifies the autonomous robotic system; a remote control center identifier that identifies the remote control center; a sequence number of the message; a connection performance metric of the connection; telemetry data, the telemetry data including one or more of the speed, heading, altitude, coordinates, battery level and system status of the autonomous robotic system; task state information indicating the status of the task; and a safety envelope description identifier that identifies a current safety envelope description used by the autonomous robotic system.

14. The non-transitory computer-readable storage medium of claim 12, wherein: The messages in the second set of messages include a set of fields, wherein the set of fields includes one or more of the following: an autonomous robotic system identifier that identifies the autonomous robotic system; a remote control center identifier that identifies the remote control center; a sequence number of the message; a connection performance metric of the connection, the connection performance metric indicating a performance metric of the connection based on a previous set of messages; and a remote control center command, the remote control center command including one or more of the following commands: a confirmation command that confirms that the autonomous robotic system is to continue to utilize the currently selected safety envelope description and corresponding operational parameters, a pause command that causes the autonomous robotic system to pause execution of the task, an abort command that causes the autonomous robotic system to abort the task, and an override command that causes the autonomous robotic system to use a new safety envelope description and corresponding operational parameters.

15. The non-transitory computer-readable storage medium of claim 11, wherein: The set of connection performance metrics indicates one or more of jitter, latency, throughput and packet loss on the connection.

16. The non-transitory computer-readable storage medium of claim 11, wherein: The first set of operational parameters includes one or more of a maximum velocity (312A) at which the autonomous robotic system is permitted to operate and a maximum altitude (312B) at which the autonomous robotic system is permitted to operate.

17. The non-transitory computer-readable storage medium of claim 11, wherein: The connection comprises a first loop (6041) connecting the autonomous robotic system to an intermediate control entity (602) and a second loop (6042) connecting the intermediate control entity to the remote control center, and wherein the set of connection performance metrics describes the first loop.

18. The non-transitory computer-readable storage medium of claim 17, wherein: The autonomous robotic system and the intermediate control entity exchange messages on the first loop at a first frequency, and the intermediate control entity and the remote control center exchange messages on the second loop at a second frequency, and The first frequency is greater than the second frequency.

19. An autonomous robotic system (104 / 800), comprising: Processor (212A / 802); A memory (212B / 804) coupled to the processor, wherein the memory comprises one or more instructions that, when executed by the processor, cause the autonomous robotic system to perform the operations of any one of claims 1-10.

Images