Method and system for operating an autonomous agent by a remote operator - Patents.com

Abstract

A system for operating an autonomous agent by a remote operator includes a remote subsystem, a local subsystem, and a communication subsystem. A method for operating an autonomous agent by a remote operator includes receiving and processing a set of inputs, presenting a set of outputs to the remote subsystem, receiving inputs from the remote operator, and operating the autonomous agent. Additionally or alternatively, the method may include any or all of detecting a fault and / or triggering a fault response, detecting a particular scenario associated with the autonomous agent, triggering a remote operator request, processing remote operator input, and / or any other process.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 286,448, filed December 6, 2021, which is incorporated by reference in its entirety.

[0002] The present invention relates generally to the field of autonomous vehicles, and more particularly to a novel and useful system and method for operating autonomous agents by a remote operator in the field of autonomous vehicles. [Background technology]

[0003] In current autonomous vehicle platforms, receiving human input can be beneficial in a variety of ways, such as to help the autonomous agent better replicate human driving in the event of an emergency or unknown event, to increase the overall safety associated with the vehicle, or for many other use cases.

[0004] Traditional autonomous vehicles typically utilize human input in the form of a safety driver in the vehicle. Remote operation is a possible option to eliminate this safety driver, but leveraging the vehicle's autonomous capabilities while integrating the autonomous driving capabilities with input from a remote operator is not easy to implement. Summary of the Invention [Problem to be solved by the invention]

[0005] Therefore, there is a need in the field of autonomous vehicles to create improved and useful systems and methods for operating autonomous agents by remote operators. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 is a schematic diagram of a system for operating autonomous agents by a remote operator. [Figure 2] FIG. 2 is a schematic diagram of a method for operating an autonomous agent by a remote operator. [Figure 3] FIG. 3 is a schematic diagram illustrating a variation of a system for operating an autonomous agent by a remote operator. [Figure 4] FIG. 4 is a schematic diagram illustrating a variation of a system for operating an autonomous agent by a remote operator and an associated method of use. [Figure 5] FIG. 5 is a schematic diagram illustrating a variation of a system for operating an autonomous agent by a remote operator and an associated method of use. [Figure 6] FIG. 6 illustrates a use case of a system and / or method implemented in delivery and an overview of the associated fleet. [Figure 7] FIG. 7 is a diagram showing a fleet of vehicles traveling on a fixed route network. [Figure 8] FIG. 8 illustrates a variation of the implementation of the system and method for detecting and responding to a fault. [Figure 9] FIG. 9 is a schematic diagram showing a modified system. DETAILED DESCRIPTION OF THE INVENTION

[0007] The following description of preferred embodiments of the invention is not intended to limit the invention to those preferred embodiments, but rather to enable any person skilled in the art to make and use the invention.

[0008] 1. Overview 1 , system 100 for operating an autonomous agent by a remote operator includes remote subsystem 120, local subsystem 110, and communication subsystem 130. Additionally or alternatively, the system may include any other components. Additionally or alternatively, the system may include and / or interface with any or all of the components described in U.S. patent application Ser. No. 17 / 116,810, filed December 9, 2020, U.S. patent application Ser. No. 17 / 125,668, filed December 17, 2020, U.S. patent application Ser. No. 17 / 127,599, filed December 18, 2020, and U.S. patent application Ser. No. 17 / 962,459, filed October 7, 2022, which are incorporated by reference in their entireties.

[0009] 2, method 200 for operating an autonomous agent by a remote operator includes step S210 of receiving and processing a set of inputs, step S230 of presenting a set of outputs to a remote subsystem, step S240 of receiving input from the remote operator, and step S260 of operating the autonomous agent. Additionally or alternatively, method 200 may include any or all of step S212 of detecting a fault and / or triggering a fault response, step S214 of detecting a particular scenario associated with the autonomous agent, step S220 of triggering a remote operator request, step S250 of processing the input, and / or any other process. Additionally or alternatively, method 200 may include and / or cooperate with any or all of the methods, processes, embodiments, and / or examples described in U.S. patent application Ser. No. 17 / 116,810, filed December 9, 2020, U.S. patent application Ser. No. 17 / 125,668, filed December 17, 2020, U.S. patent application Ser. No. 17 / 127,599, filed December 18, 2020, and U.S. patent application Ser. No. 17 / 962,459, filed October 7, 2022, which are incorporated by reference in their entireties, or any other suitable processes performed in any suitable order. Method 200 may be performed by the systems described below and / or any other suitable system.

[0010] 2. Advantages A system and method for operating autonomous agents by a remote operator can provide several advantages over current systems and methods.

[0011] In a first variant, the present technology provides the advantage of strategically and reliably integrating human-in-the-loop decision-making with autonomous agent decision-making, thereby eliminating the need for a human operator to be on-board the autonomous agent while it is operating. The system and / or method further preferably provides the advantage of utilizing a remote operator only in a specific subset of scenarios, with the autonomous agent leveraging any or all of the following characteristics to operate reliably on its own in the majority of scenarios: fixed routes, limited operational design domains, redundant and robust hardware and software, and / or any other characteristics. This can enable a highly scalable remote operator platform, where a single remote operator can simultaneously oversee multiple autonomous agents.

[0012] In a first set of illustrative examples, a remote operator (also referred to herein, equivalently, as a human operator and / or teleoperator and / or remote supervisor) is invoked (e.g., to notify, alert, contact, etc.) in certain contexts where human input is found to be beneficial and / or optimal and / or necessary to operate the autonomous vehicle.

[0013] In a second set of examples, additional or alternative to the first set of examples, a remote operator is called in response to detecting that a system failure has occurred and / or that a warning has occurred that may lead to a system failure.

[0014] In a second variation, additional or alternative to the first variation, the technique provides the advantage of validating input from a remote operator before implementation, which serves to maintain safety standards and implement satisfactory actions based on multiple sources of information.

[0015] In a first example, the technology further provides the advantage of validating (e.g., in a batch manner) a portion of a remote operator input, such as an initial waypoint of a series of waypoints entered by the remote operator, so that if the initial waypoint does not meet a set of safety constraints and / or other success criteria, the remote operator can be notified and / or additional waypoints can be prevented from being received and / or processed by the autonomous vehicle (AV) logic (e.g., to protect the AV's computational resources, to prevent the remote operator from wasting time and / or attention on performing low-value tasks, etc.).

[0016] Additionally or alternatively, the technology may be configured to provide the remote operator with only a set of options that have already been validated and / or pre-validated (e.g., based on the vehicle's current environmental awareness), thereby preventing the remote operator from providing feedback that is infeasible (and / or unsafe for the AV to perform).

[0017] In an example, for example, route options or any other input options (e.g., high-level behavior options, sets of waypoint options, etc.) provided to a remote operator are processed to check for safety verification and then transmitted to the remote operator for review and / or selection.

[0018] In a third variation, additional or alternative to the variations described above, the system and / or method advantageously utilizes input from a remote operator in multiple categories of use cases, such as detecting that a minimum risk condition has been triggered and / or is expected to be triggered for a vehicle, detecting a particular context related to the vehicle, detecting that the vehicle may approach a scenario associated with high uncertainty in the future (e.g., so that the remote operator can provide input that prevents the need to trigger MRC), and / or any other use case in which the remote operator may be notified and / or requested to be input by the remote operator.

[0019] In a fourth variation, additional or alternative to the variations described above, the technique provides the advantage of preventing the vehicle from implementing an overly conservative and / or overly severe fault response (e.g., immediately stopping, pulling over, etc.) when not needed (e.g., not immediately needed).

[0020] In a first set of examples, for example, the present technology provides the advantage of avoiding false positives or overly conservative minimum risk operations (also referred to herein, equivalently, as minimum risk conditions) or other failure responses by its ability to detect potential failures in a tiered manner (e.g., along a spectrum of severity), thereby allowing increasingly conservative minimum risk operations to be implemented as the likelihood and / or severity of failures increases.

[0021] Additionally or alternatively, a remote operator input request may be required before a minimum risk operation is triggered and / or before a minimum risk operation is escalated, so that the remote operator has the opportunity to provide feedback tailored to the particular situation and / or assessed severity of the potential failure (e.g., warning vs. error), and if the remote operator does not or cannot provide feedback within a specified decision window, the vehicle may autonomously select a more drastic failure response and proceed.

[0022] In a fifth variation, additional or alternative to the variations described above, the present technology provides the advantage of leveraging fixed-route use cases, which may enable high accuracy (e.g., high prediction accuracy, etc.) to be associated with failure predictions (e.g., environmental / object uncertainties) made by the AV system (e.g., diagnostic subsystem), which in turn may enable the selection of the most appropriate least-risk operation depending on the particular uncertainty level and / or uncertainty type associated with the vehicle's understanding of the environment.

[0023] Additionally or alternatively, the systems and methods may provide any other advantages.

[0024] 3. System 1 , system 100 for operating an autonomous agent by a remote operator includes remote subsystem 120, local subsystem 110, and communication subsystem 130. Additionally or alternatively, the system may include any other components. Additionally or alternatively, the system may include and / or interface with any or all of the components described in U.S. patent application Ser. No. 17 / 116,810, filed December 9, 2020, U.S. patent application Ser. No. 17 / 125,668, filed December 17, 2020, U.S. patent application Ser. No. 17 / 127,599, filed December 18, 2020, and U.S. patent application Ser. No. 17 / 962,459, filed October 7, 2022, which are incorporated by reference in their entireties.

[0025] System 100 preferably interfaces with a set of one or more autonomous vehicles (also equivalently referred to herein as autonomous agents and / or autonomous vehicles), which may be used for any or all of transporting goods (e.g., for delivery), transporting passengers, and / or any other functions. Additionally or alternatively, system 100 may interface with any other vehicle, robotic device, or any other device. The vehicles are preferably autonomous vehicles (e.g., configured for fully autonomous use, configured for level 4 autonomy, configured for level 5 autonomy), but may additionally or alternatively include semi-autonomous vehicles (e.g., level 3 autonomy, level 2 autonomy, level 1 autonomy, etc.), manually driven and / or manually drivable vehicles, and / or any combination of vehicles. The vehicles preferably include delivery vehicles (e.g., trucks) configured for delivering goods, but may additionally or alternatively include passenger cars, public transportation vehicles, and / or any other vehicles.

[0026] In a preferred set of variations (e.g., variations such as that shown in FIG. 6 , variations such as that shown in FIG. 7 ), the set of autonomous vehicles is configured for use in connection with short-distance logistics applications, such as delivering goods between destinations using the autonomous vehicles. The autonomous vehicles preferably perform these deliveries according to a set of fixed routes, but may additionally or alternatively follow dynamic routes and / or operate in other manners.

[0027] 3.1 System - Local Subsystem 110 System 100 preferably includes and / or interfaces with a local subsystem 110 that functions to operate (e.g., perform decision-making, control, manipulation, actuation, etc.) the autonomous agent. Additionally or alternatively, local subsystem 110 may function to detect one or more faults associated with the autonomous agent and / or any of the software associated with the autonomous agent, trigger a minimum-risk condition operating mode, perform a minimum-risk operation, alert a remote operator, determine a set of options for providing input to the remote operator, receive, validate, and / or implement input from the remote operator, and / or perform any other function.

[0028] The local subsystem is preferably located entirely on-board the autonomous agent. Additionally or alternatively, the local subsystem may be located partially off-board the autonomous agent, located entirely off-board the autonomous agent, communicate with computing subsystems and / or other components located off-board the autonomous agent, communicate with other autonomous agents (e.g., other vehicles in a fleet of AVs), and / or located at any other location and / or any combination of locations.

[0029] Local subsystem 110 preferably includes a local computing subsystem (also referred to herein equivalently as an autonomous computing subsystem, an autonomous computer, an autonomous vehicle [AV] computing subsystem, an autonomous vehicle [AV] computer, etc.), which functions to receive and process information used to operate an autonomous agent (also referred to herein equivalently as an autonomous vehicle and / or an autonomous agent).

[0030] In a preferred variant, for example, the local computing subsystem implements AV logic that functions to operate an autonomous agent. Additionally or alternatively, the local computing subsystem may execute any other logic.

[0031] In one set of examples, the AV logic implements a set of trained models, such as, but not limited to, the trained micromodels described in any or all of U.S. patent application Ser. No. 17 / 116,810, filed December 9, 2020, U.S. patent application Ser. No. 17 / 125,668, filed December 17, 2020, and U.S. patent application Ser. No. 17 / 127,599, filed December 18, 2020, which are incorporated herein by reference in their entireties.

[0032] The local computing subsystem preferably includes and / or cooperates (e.g., communicates with) a diagnostic subsystem (also equivalently referred to herein as a fault detection subsystem and / or a health assessment subsystem, such as shown in FIG. 9 ), which includes and / or cooperates with a set of one or more processors configured to monitor the overall health of the autonomous vehicle. The diagnostic subsystem further preferably functions to monitor communication (e.g., information exchange) between different modules / layers of the local computing subsystem, e.g., between the perception layer and the prediction layer, between the prediction layer and the planning layer, and / or between any other layers. Additionally or alternatively, the diagnostic subsystem may function to monitor connectivity between the local subsystem and a remote subsystem. For example, if communication with a remote subsystem is lost, the diagnostic subsystem may be used to perform such detection (e.g., a fault response [e.g., a minimum-risk operation] may be selected and / or implemented).

[0033] The diagnostic subsystem may further function to monitor the health of one or more hardware components (e.g., actuation subsystem, sensor subsystem, etc.) and / or any other aspect of the autonomous agent. If the diagnostic subsystem detects that a problem (also equivalently referred to herein as a failure) has occurred (e.g., communication is lost, a layer output is not verified / trusted, a hardware component has failed, etc.), a minimal risk condition (MRC) operating mode is triggered and / or considered for the autonomous agent (e.g., as shown in FIG. 4).

[0034] In some variations (e.g., variations described below, variations shown in FIG. 8, etc.), a series of minimum-risk operations of different severity (e.g., ordered in a priority list of increasing severity, organized in a hierarchical structure) may be individually selected and / or implemented depending on the type of fault or potential fault detected, whether or not a remote operator responds to an initial alert (e.g., within a predetermined time threshold), how the potential fault progresses or does not progress, the level of the detected fault, and / or any other information.

[0035] In additional or alternative variations, the remote operator may be alerted before triggering one or more minimum risk maneuvers (e.g., an initial minimum risk maneuver, a less conservative minimum risk maneuver in a prioritized list, etc.) Additionally or alternatively, the remote operator may be alerted after the vehicle initiates and / or completes a minimum risk maneuver.

[0036] In certain implementations, the MRC operational mode preferably includes performing an action that stops the agent, but may additionally or alternatively include any other behavior and / or combination of behaviors, such as, but not limited to, flashing hazard lights, pulling over (e.g., pulling off the road, pulling onto the shoulder, etc.), and / or any other behavior.

[0037] In response to detecting a failure and triggering a failure response (e.g., MRC), the local subsystem preferably generates a decision request (also equivalently referred to herein as a human-in-the-loop decision request) that is sent to the remote subsystem 120 for review by a remote operator. The local computing subsystem further preferably, in response to the failure mode being triggered, overrides any actions, behaviors, and / or trajectories that the local subsystem had planned to implement for the autonomous agent, so that the autonomous agent does not proceed as originally planned (because the detected problem may no longer be safe and / or optimal). Additionally or alternatively, any other actions may be triggered and / or processes may be implemented in other ways.

[0038] Additionally or alternatively, a human-in-the-loop decision request may be triggered in response to detecting a particular scenario (e.g., context) associated with the autonomous agent, such as a particular scenario (e.g., from a predetermined set of scenarios) in which leveraging human input has been found to be beneficial. These scenarios may include scenarios in which the agent needs or wants to perform a behavior that an autonomous computer alone cannot allow (e.g., venturing into oncoming traffic), scenarios in which it has been determined that a human would be better able to perform (e.g., moving into dynamically changing loading and / or unloading areas), and / or any other scenario.

[0039] The local subsystem may optionally include and / or interface with an actuation subsystem that functions to drive the autonomous agent. The actuation subsystem is preferably part of the drive-by-wire system of the autonomous agent and includes a set of drive-by-wire actuators (such as chassis actuators as shown in FIG. 3 ), such as a brake-by-wire subsystem, a steer-by-wire subsystem, and a throttle-by-wire subsystem, but may additionally or alternatively include a subset of these, any or all of these components in non-drive-by-wire form, and / or any other actuation components.

[0040] The local subsystem may optionally include and / or interface with a sensor subsystem that functions to receive information related to the agent's environment for use in operating the autonomous agent. Additionally or alternatively, the sensor subsystem may function to receive information (e.g., a video stream) for provision to a remote operator, detect faults (e.g., hardware faults) associated with the autonomous agent, and / or perform any other function. The sensor subsystem preferably includes a set of cameras and optionally one or both of a LiDAR sensor and a RADAR sensor, but may additionally or alternatively include a set of sensors configured to monitor the health of any or all components of the agent (e.g., operating components, the sensors described above, etc.), and / or any other sensors.

[0041] Additionally or alternatively, the local subsystem may include and / or interface with any other components, such as, but not limited to, a control subsystem, storage / memory, and / or any other components configured to implement control commands for operation of the autonomous agent.

[0042] 3.2 System - Remote Subsystem 120 The system 100 includes a remote subsystem 120 that functions to provide an interface with a remote operator, whereby output can be provided to and input can be received from the remote operator. Additionally or alternatively, the remote subsystem can perform any other function.

[0043] The remote subsystems are preferably located in a set of remote monitoring stations (RMS) where remote operators (also referred to herein equally as remote operators, operators, and / or supervisors) are located, preferably along with a set of remote interfaces, remote computing subsystems, and optionally any other components. Additionally or alternatively, any or all of the remote subsystems may be located in multiple locations, located partially onboard an autonomous agent, and / or otherwise located.

[0044] The remote subsystem preferably communicates with the local subsystem 110 via the communications subsystem 130, but may additionally or alternatively communicate with any other components.

[0045] The remote subsystem preferably includes a set of remote interfaces (also equivalently referred to herein as remote operator interfaces) that function to provide output to and receive input from a remote operator. The remote interfaces may include any or all of output devices (e.g., displays, speakers, etc.), input devices (e.g., touchscreen displays, joysticks, simulator systems, actuation components with associated sensors, etc.), and / or any other devices. Outputs provided to the remote operator preferably include sensor information (e.g., camera streams) collected on the autonomous agent, which function to provide the remote operator with awareness and understanding of the autonomous agent's surroundings. Inputs received from the operator may include decision selections (e.g., as described below), control commands (e.g., via a joystick, via a braking simulator, etc.), and / or any other input.

[0046] The remote subsystem preferably further includes a remote computing subsystem, which implements the remote subsystem logic and functions to process any or all of the input received from the remote operator. Additionally or alternatively, the remote computing subsystem may perform any other function, any or all of the remote subsystem logic may be implemented in the local computing subsystem, and / or computing may be configured in other ways.

[0047] Additionally or alternatively, the remote subsystem 120 may include any other components.

[0048] 3.3 System - Communication Subsystem 130 System 100 includes a communications subsystem 130 that functions to establish communications and information exchange between local subsystem 110 and remote subsystem 120. In a preferred variation, communications subsystem 130 includes a relay server (e.g., using multi-channel bonding), but may additionally or alternatively include any other servers, communications components and / or communications protocols (e.g., wireless communications, wired communications, WiFi, Bluetooth, radio, etc.), and / or components.

[0049] 4. Method 2, method 200 includes step S210 of receiving and processing a set of inputs, step S230 of presenting a set of outputs to a remote subsystem, step S240 of receiving input from a remote operator, and step S260 of operating an autonomous agent. Additionally or alternatively, method 200 may include step S212 of triggering a failure response mode and / or a failure response, step S214 of detecting a particular scenario associated with the autonomous agent, step S220 of triggering a remote operator request, step S250 of processing the input, and / or any other process. Additionally or alternatively, method 200 may include and / or cooperate with any or all of the methods, processes, embodiments, and / or examples described in U.S. patent application Ser. No. 17 / 116,810, filed December 9, 2020, U.S. patent application Ser. No. 17 / 125,668, filed December 17, 2020, U.S. patent application Ser. No. 17 / 127,599, filed December 18, 2020, and U.S. patent application Ser. No. 17 / 962,459, filed October 7, 2022, which are incorporated by reference in their entireties.

[0050] 4.1 Method—Step S210 of Receiving and Processing a Set of Inputs Method 200 preferably includes step S210 of receiving and processing a set of inputs, which functions to receive information for use in operating the autonomous agent, which may be autonomous, with human (e.g., remote operator, on-board human operator, etc.) input, semi-autonomous, in any other mode of operation, and / or in any combination of modes of operation. S210 may additionally or alternatively function to determine in which of these modes to operate, detect triggers and / or transitions between these modes (e.g., based on detection of a fault), and / or perform any other suitable function.

[0051] Additionally or alternatively, method 200 may be performed without S210 and / or with a modified version of S210 (e.g., any or all of the inputs are stored, any or all of the inputs are pre-processed, etc.).

[0052] S210 is preferably executed initially during method 200, and more preferably executed multiple times (e.g., continuously, at a predetermined frequency, etc.) during operation of the autonomous agent, such as continuously and / or nearly continuously (e.g., at a predetermined frequency) as data is collected by one or more sensors onboard the AV (e.g., throughout the AV's route travel). In some variations, for example, S210 is executed and / or triggered according to a predetermined frequency associated with a perception subsystem (e.g., implemented in a computing subsystem) of the vehicle. Additionally or alternatively, S210 may be executed at any other time, triggered based on any suitable input, and / or executed in any other manner as suitable.

[0053] The input is preferably processed in the local computing subsystem (eg, using AV logic), but may additionally or alternatively be processed in any other computing subsystem.

[0054] The set of inputs received at S210 preferably relate at least in part to the vehicle and / or its environmental surroundings (e.g., the environment perceivable by the AV's sensors), but may additionally or alternatively be received from and / or related to other vehicles (e.g., other AVs in the fleet, other vehicles in the AV's environment, etc.), general environmental conditions that may apply to the vehicle (e.g., weather conditions, traffic conditions, etc.), remote operator input and / or information (e.g., status, availability, etc.), and / or any other inputs received, determined, and / or obtained at S210.

[0055] The input received at S210 preferably includes sensor data (e.g., sensor streams) received from a set of sensors (e.g., cameras, LiDAR, RADAR, etc.) onboard the autonomous agent, which sensor data is used by the autonomous agent (e.g., through processing by perception and / or prediction and / or planning modules / protocols / processes of the computing subsystem) to decide how to drive the AV (e.g., decide which actions to perform, decide which trajectory to take, etc.). The sensor data also preferably serves to provide information for use in the decision-making (e.g., option selection, input determination, etc.) of a remote operator and / or on-board operator, thereby making the operator aware of the vehicle's environmental surroundings (e.g., as represented by the sensor data) and / or potential failures associated with the vehicle and / or its understanding of the environment (e.g., based on an obstructed camera view, a sensor shutting down due to a sensor failure, missing sensor data due to a loss of power, etc.). In some variations, for example, at least a portion of the sensor streams are transmitted (e.g., via communications subsystem 130) to remote subsystem 120 (e.g., a user interface such as a display) to provide a remote operator with awareness of the autonomous agent's surroundings for monitoring the agent and / or for use in decision-making performed by the remote operator (e.g., as described below). Additionally or alternatively, processed and / or modified sensor data (e.g., annotated sensor streams, aggregated sensor streams (e.g., camera data with overlay from lidar and / or radar data), prioritized sensor streams and / or sensor stream subsets (e.g., camera data only), etc.) can be provided to the operator, alerts (e.g., audio alerts at a remote monitoring station, visual alerts (e.g., annotations), tactile alerts, etc.) can be provided to the operator, and / or any other information can be provided to the operator.Additionally or alternatively, the sensor data may be received from sensors remote from the AV (e.g., non-on-board), such as, but not limited to, sensors mounted on another vehicle (e.g., fleet vehicle, non-fleet vehicle, etc.), sensors located in the vehicle's environment (e.g., sensors fixed to environmental infrastructure, sensors fixed to stationary objects in the vehicle's environment, sensors fixed to moving objects in the vehicle's environment, etc.), and / or any other sensors.

[0056] The inputs may additionally or alternatively include any other information, such as, but not limited to, a set of maps (e.g., maps utilized by the AV and / or operator for decision making), environmental information (e.g., weather information, traffic information), AV site information (e.g., loading dock information, AV base station information, etc.), fleet management information, information from other fleet vehicles (e.g., enabled via vehicle-to-vehicle communication), database information and / or information stored in memory, and / or any other information.

[0057] In some variations, for example, the set of inputs may include fleet information related to a fleet of AVs (e.g., a set of vehicles collectively used in a particular use case and / or within a particular fixed route network), such as any or all of the information utilized to manage a fleet described in U.S. Patent Application No. 17 / 962,459, filed October 7, 2022, which is incorporated herein by reference in its entirety. The fleet information may be received from any or all of other vehicles in the fleet (e.g., via vehicle-to-vehicle communication), a fleet management subsystem (e.g., fleet management software, a fleet management operator, etc.), a customer located at a site associated with the AV (e.g., inputs received from a field worker at a customer interface, inputs related to a loading dock, etc.), and / or any other source.

[0058] Additionally or alternatively, the set of inputs may include any other information.

[0059] Processing the set of inputs preferably functions to determine whether a request to a remote operator should be triggered in S220 (e.g., in response to detecting a fault associated with the AV, in response to characteristics of the detected fault, in response to detecting and / or classifying a particular environment and / or scenario associated with the AV). Additionally or alternatively, processing the set of inputs may function to determine what type of request to trigger, when to trigger the request, what options and / or types of options to provide to the remote operator (e.g., high-level behaviors vs. paths vs. waypoints, etc.), and / or processing the inputs may be suitably used in other ways.

[0060] Additionally or alternatively, the inputs may be processed and used in the autonomous decision-making and operation of the AV (e.g., to autonomously determine the behavior and / or actions and / or trajectory and / or control commands of an autonomous agent under normal autonomous operation) and / or to perform any other function.

[0061] S210 preferably includes a step of detecting (e.g., checking for) a fault and / or triggering a fault response step S212, which may function to determine whether performance of the vehicle and / or any associated components (e.g., sensors, actuators, etc.) or processes (e.g., operation of a computing subsystem) is impaired and / or is predicted to be impaired (or will be impaired) so that optimal and safe action may be triggered in response. Additionally or alternatively, S212 may function to evaluate and / or characterize the fault, such as the type of fault (e.g., which components are affected), the source / cause of the fault, the severity and / or urgency associated with the fault, and / or any other characteristics of the fault. Still additionally or alternatively, S212 may function in response to the detection of a potential fault to prevent an unnecessary and / or overly conservative response from being triggered (e.g., in response to a false-positive fault detection).

[0062] S212 may also optionally function to trigger and / or signal the execution of any or all other processes of method 200.

[0063] S212 is preferably performed by a diagnostic subsystem (e.g., a diagnostic subsystem module of a processing and / or computing subsystem, a set of processors, etc.), such as, but not limited to, a diagnostic subsystem as described above. Additionally or alternatively, any other diagnostic subsystem and / or processing subsystem may be used.

[0064] The diagnostic subsystem preferably functions to determine (e.g., check) the status (e.g., health) of the entire vehicle (e.g., a collective combination of components, a collective combination of components prone to failure, etc.), but may additionally or alternatively function to determine the status of one or more components of the vehicle, multiple vehicles, and / or any other systems, subsystems, and / or components.

[0065] In a preferred set of variations, the diagnostic subsystem functions to aggregate status and / or health and / or performance information from multiple (e.g., all, all being analyzed, etc.) components of the vehicle (e.g., sensors, processors, computers, power supplies, communication subsystems, etc.) or components associated with the vehicle (e.g., components communicating with the vehicle, controls of a remote operator station, etc.), and is also referred to herein equivalently as a diagnostic aggregator and / or diagnostic aggregator subsystem (e.g., including a processor configured for diagnostic evaluation and aggregation).

[0066] The diagnostic subsystem is preferably configured to receive at least data from sensors associated with the vehicle (e.g., on-board sensors, off-board sensors in communication with the vehicle, etc.), but may additionally or alternatively be configured to receive any or all of: control commands provided to the vehicle and / or executed by the vehicle (e.g., by the vehicle's actuators); outputs and / or intermediate outputs of the vehicle's processing and / or computing subsystems (e.g., outputs of perception and / or prediction and / or planning modules / subsystems, outputs of AV logic, outputs of a set of trained machine learning models used for vehicle action selection and / or trajectory generation, etc.); uncertainties related to the outputs and / or intermediate outputs of the vehicle's computing and / or processing subsystems (e.g., as described in U.S. Patent Application No. 17 / 127,599, filed December 18, 2020, which is incorporated herein by reference in its entirety); and / or any other information.

[0067] In some variations, for example, the diagnostic subsystem expects data from sensors and / or other information determined and / or communicated in the vehicle (e.g., control commands communicated to actuators) to be transmitted according to predetermined time parameters (e.g., at a predetermined speed or speed range, at a predetermined frequency or frequency range, at a predetermined time or set of times, within a predetermined delay or delay range relative to data generation and / or data reception at another component, etc.), and if the transmission and / or reception of data falls outside the expected time parameters, an alert regarding a potential fault may be triggered (e.g., as described below).

[0068] Additionally or alternatively, the diagnostic subsystem may consider (e.g., process, analyze, etc.) any or all information (e.g., inputs, outputs, intermediate outputs, etc.) related to the AV's software stack (e.g., AV logic, models, and / or algorithms executed in the AV's computing and / or processing subsystem, etc.), such as uncertainty measures (e.g., determined using anomaly detection and / or out-of-distribution processes) related to the AV software stack's inputs and / or outputs, such as, but not limited to, any or all of those described in U.S. Patent Application No. 17 / 127,599, filed December 18, 2020, which is incorporated herein by reference in its entirety.

[0069] In one set of examples, the diagnostic subsystem is able to detect and characterize faults with high accuracy and confidence due, at least in part, to a fixed route use case for a set of vehicles, which enables, for example, an out-of-distribution detector and / or anomaly detector to determine with high accuracy and confidence whether a detected object and / or the vehicle's environment has been seen before (e.g., if the vehicle has traveled the same route multiple times).

[0070] Additionally or alternatively, the diagnostic subsystem may process any other information and / or may be used in any suitable use case.

[0071] The diagnostic subsystem may process any or all of this data individually, collectively, and / or in any combination.

[0072] The diagnostic subsystem preferably produces as output a set of one or more scores related to the vehicle (e.g., health scores, fault scores, performance scores, status scores (e.g., on, off, communicating, not communicating, etc.), etc.), which may be any or all of scores calculated using a set of models and / or algorithms (e.g., rule-based models and / or algorithms, learned and / or trained models and / or algorithms (e.g., machine learning models, deep learning models, neural networks, etc.), equations, etc.), scores received directly from components of the system and / or scores aggregated (e.g., according to a predetermined equation used to sum, average, find a median value) based on inputs received from components of the system (e.g., sensor data, time of information receipt, signal strength, etc.), scores determined using lookup tables and / or decision trees, scores determined in any combination of methods, and / or scores determined in any other suitable manner.

[0073] The output of the diagnostic subsystem (e.g., an aggregated health score for the vehicle) and / or intermediate outputs (e.g., health scores for individual components) are preferably compared to a set of thresholds and / or success criteria (e.g., in the diagnostic subsystem, a processing and / or computing subsystem in communication with the diagnostic subsystem, etc.), which may function to determine whether a fault is detected and / or whether a fault is suspected to be present, characterize a fault type and / or characteristics (e.g., severity, confidence, etc.) associated with the fault, trigger a fault response to the vehicle, trigger a remote operator request, determine conditions (e.g., timing) associated with triggering the fault response and / or remote operator request, inform a remote operator of a decision on input options to provide (e.g., along with the remote operator request), and / or perform any other function.

[0074] In a preferred variation, for example, one or more health scores produced by the diagnostic subsystem are compared to a set of predetermined thresholds, thereby enabling potential failures to be detected and / or characterized (e.g., based on likelihood / confidence of occurrence (e.g., likelihood that the potential failure is a false positive), expected severity, etc.). In some examples, for example, aggregated health scores produced by the diagnostic subsystem are compared to a set of one or more thresholds, enabling potential failures to be distinguished from one another (e.g., more finely evaluated) based on the comparison to the thresholds. In a specific example, for example, potential failures are categorized into one of at least two categories, where a first category indicates that the potential failure should be treated as a warning (e.g., potential error / failure) and a second category indicates that the potential failure should be treated as an error / failure. In a particular implementation, the error category requires that the system's aggregated health score fall below a threshold that is lower than the score in the warning category. Additionally or alternatively, individual health scores can be compared to thresholds, the number of potential failures (e.g., within a predetermined time frame) can be used to categorize the failures, and / or failures can be suitably detected and / or categorized in other ways. Conversely, the predictive failure score may indicate a warning if the score exceeds a first predetermined threshold, and the predictive failure score may indicate an error / failure if the score exceeds a second predetermined threshold (e.g., higher than the first predetermined threshold).

[0075] Additionally or alternatively, any other potential categories can be implemented. In another set of examples, a set of categories can be selected, e.g., from multiple categories, indicating, for example, that the vehicle is operating normally, that the vehicle is at risk of failure (e.g., warning of a failure), that the vehicle is experiencing a failure, and / or any other indication. The categories can further optionally indicate and / or suggest the extent to which the vehicle handles the failure (e.g., autonomously responding to the failure) and the extent to which the remote operator has an opportunity to respond to the failure. In certain examples, for example, a first category indicates that a failure is not detected, a second category indicates that a warning threshold has been exceeded and that a remote operator should be alerted (e.g., so that the remote operator has an opportunity to intervene before the warning progresses to an error, so that the remote operator can confirm that a failure exists, so that the remote operator can confirm that the warning is a false positive and no indication of a failure exists, etc.), and a third category indicates that a failure has been determined to have occurred (e.g., with a confidence level above a predetermined threshold) and that a minimum-risk operation should be / will be performed by the vehicle (e.g., without remote operator input).

[0076] Additionally or alternatively, any or all of the categories may indicate the type and / or severity of the failure, such as, but not limited to, an indication of which components have failed, an indication of the number of components that have failed (e.g., failed simultaneously), an indication of the importance and / or priority associated with the failed components (e.g., failure of the entire computing system versus failure of a redundant sensor), and / or any other characteristics associated with the failure may be determined.

[0077] Additionally or alternatively, any or all of the characterization / classification / categorization of the fault may be performed based on time parameters associated with fault detection, number of detections, and / or other parameters.

[0078] For example, the creation and analysis of the output of the diagnostic subsystem (e.g., diagnostic aggregator) may optionally be performed according to a moving-window (e.g., sliding-window) analysis, which processes input received at the diagnostic subsystem (e.g., diagnostic aggregator) and / or creates output by the diagnostic aggregator according to a frequency (e.g., a predetermined frequency) using a moving time window process or the like, where data from a first time window is processed to create a first set of health scores, data from a second time window is processed to create a second set of health scores, and so on. This can serve to address inevitable uncertainties regarding the data received at the diagnostic subsystem and / or noise in sensors or other information sources without triggering false positive reactions, since the evolution of the health scores over time can be determined and utilized. For example, if there is a sudden spike (or drop) in the values ​​of the health scores, an alert (e.g., as described above) can be issued, but drastic minimum-risk actions need not necessarily be immediately triggered; rather, the evolution of the scores and / or the pattern of the number or occurrence of scores can be monitored for further decision-making. For example, if a fault or potential fault is detected less than a predetermined number of times within a predetermined time threshold, the potential fault may be flagged as a warning, while if the potential fault is detected continuously and / or more than a predetermined number of times within the predetermined time threshold (e.g., in each of a set of n consecutive time windows, in more than m windows within a set of n windows, etc.), it may be determined that the vehicle has an error (e.g., escalating from a warning).

[0079] In variations that implement moving time windows, the windows may be independent of one another (e.g., a next window begins where the previous window ends), overlapping (e.g., partially overlapping, fully overlapping, etc.), or any combination. The windows may be of the same time duration, different time durations, or any combination of windows.

[0080] In one set of examples, multiple characteristics of a detected potential fault may be aggregated together and used to evaluate next steps in triggering a fault response and / or a remote operator request (e.g., as described below). In a specific example, for example, the number of times a warning occurs may be used to trigger a first response, and the number of times an error occurs may be used to trigger a second response (e.g., different from the first response).

[0081] Triggering a fault response may include any or all of step S220 of triggering a remote operator request, operating the vehicle (e.g., operating autonomously) in accordance with a minimum risk condition (MRC) and / or in accordance with performance of a minimum risk maneuver (e.g., stopping, pulling over, braking, etc.), implementing the first response of a set of multiple fault responses (e.g., layered sequential fault responses, etc.), and / or any combination.

[0082] The type of fault response to be triggered (e.g., minimum risk operation vs. remote operator request) is preferably determined based on the health score or other output produced by the diagnostic subsystem, but may additionally or alternatively be based on characteristics of the health score (e.g., warning vs. error), time parameters associated with the health score, other characteristics of the health score, and / or any other information.

[0083] In one variation, for example, once the diagnostic aggregator determines the level of uncertainty and / or warnings and / or errors associated with the vehicle, there are multiple tiers of minimum-risk actions the vehicle can take depending on its health score and / or characteristics, and further, any or all of the actions can be coordinated with remote operator feedback. For example, in some examples, the actions can be organized in a tiered manner of increasing intensity (e.g., as shown in FIG. 8 , in order of increasing braking strength required of the vehicle, requiring more effort to subsequently resume normal vehicle operation; in order of increasing deceleration rates associated with the vehicle; in order of increasing speed reductions performed by the vehicle; from “slow down” behavior to “immediate stop” behavior, etc.). In this case, remote operator feedback can be requested and / or received before initiating any or all of the minimum-risk actions (e.g., between the first and second actions), which can function, for example, to prevent the vehicle from relying on receiving remote operator input to safely respond to a fault while also preventing severe minimum-risk actions from being performed when not necessary.

[0084] Examples of minimum risk maneuvers may include, but are not limited to, slowing the vehicle (e.g., applying the brakes), stopping the vehicle, pulling the vehicle onto the shoulder or far lane (e.g., curb) or designated stopping area, and / or any other maneuver.

[0085] In an example involving tiered minimum-risk operations, for example, in response to detecting a warning or fault, the vehicle may initiate (e.g., autonomously initiate) an initial minimum-risk maneuver (e.g., slowing down) and provide a remote operator request for input (e.g., within a predetermined time frame) before initiating a more intensive minimum-risk maneuver (e.g., stopping) that would later make it more difficult for the vehicle to resume normal operation (e.g., requiring direct control from a human in the vehicle or a remote operator, requiring a human to travel to the vehicle to resume operation, requiring maintenance, etc.).

[0086] S210 may optionally include a step S214 of determining (e.g., detecting, characterizing, identifying, etc.) a scenario associated with the agent, which serves to enable a particular scenario found to be optimally navigated by the remote operator (or a minimum risk operation) to trigger a remote operator request and / or any type of operation in S220. A scenario preferably refers to a type of environment surrounding the vehicle, which may be characterized by any or all of static characteristics (e.g., static infrastructure, road geometry, etc.), dynamic characteristics (e.g., behavior of other vehicles, presence of known or unknown objects, etc.), or any combination of characteristics.

[0087] Additionally or alternatively, S214 may include the vehicle being unable to identify a particular scenario (e.g., based on a sensor being obstructed, based on the output of an anomaly detection subsystem and / or an out-of-distribution detection subsystem, etc.).

[0088] S214 may be performed in parallel with S212, before S212, after S212, and / or at any other time.

[0089] The scenarios may be any or all of the following: predetermined scenarios (e.g., based on a map assignment of the scenario), dynamically determined scenarios (e.g., based on object detection, based on passing a geofence trigger, etc.), or any combination of predetermined and dynamically determined scenarios.

[0090] Examples of scenarios include, but are not limited to, scenarios related to (e.g., defined by) lane and / or road geometry (e.g., one-way roads, two-way roads, two-lane roads, four-lane roads, roads with shoulders, roads with curbs, parking lots, driveways, etc.), scenarios related to road and / or area types (e.g., residential roads, highways, school zones, commercial zones, construction zones, etc.), scenarios related to vehicle use cases (e.g., loading zones, customer sites, package pickup locations, package drop-off locations, etc.), scenarios defined by specific conditions (e.g., dynamic conditions, predetermined trends, etc.) (e.g., heavy traffic scenarios, inclement weather scenarios, etc.), scenarios defined based on the presence and / or behavior of specific objects (e.g., scenarios involving the presence of children, scenarios involving unknown objects and / or object behavior in unnatural or unexpected conditions, etc.), any combination of scenarios, and / or any other suitable scenario.

[0091] In a particular set of examples, the scenario is determined (e.g., partially, entirely, etc.) based on context related to the vehicle's location (e.g., as described in U.S. patent application Ser. No. 17 / 116,810, filed December 9, 2020, which is incorporated herein by reference in its entirety).

[0092] Detecting the scenario may optionally be part of the agent's autonomous operation (e.g., as described in U.S. Patent Application No. 17 / 116,810, filed December 9, 2020, which is incorporated herein by reference in its entirety), part of selecting and / or narrowing (e.g., filtering) a set of decision options to send to a remote operator (e.g., as described below), and / or may be used in any other process. In a preferred variation, S214 includes detecting the context based on the autonomous agent's location and map, as described in U.S. Patent Application No. 17 / 116,810, filed December 9, 2020, which is incorporated herein by reference in its entirety. Additionally or alternatively, the context associated with the autonomous agent may be suitably determined in other manners.

[0093] In some variations, such as those implemented in AV delivery use cases (e.g., fixed route delivery of goods, delivery of goods from a warehouse site to a delivery site, etc.), a scenario corresponding to a loading and / or unloading site for the vehicle may be detected and used to trigger a remote operator request in response to arrival at the site (e.g., based on the vehicle's location, based on referencing a map based on the vehicle's location, based on detecting a breach of a geofence trigger associated with the site, etc.). In specific examples, for example, a remote operator request can be used (e.g., as described below) to alert the remote operator to assign a specified loading dock to the vehicle, to alert the remote operator to provide waypoints for navigating the vehicle to an available loading dock, to alert the remote operator to select high-level behavior for the AV upon arrival at the scene (e.g., waiting in a specific waiting area because all loading docks are in use, unlocking the vehicle for unloading at a specific loading dock, etc.), to alert the remote operator to take over direct control of the vehicle (e.g., backing the vehicle into a loading dock), and / or any other input from the remote operator may be required.

[0094] Additionally or alternatively, any other scenario may trigger a specific remote operator request.

[0095] Additionally or alternatively, S210 may include any other process.

[0096] 4.2 Method - Step S220 of Triggering a Remote Operator Request Method 200 may include a step S220 of triggering a remote operator request, which functions to alert a remote operator of an event where remote operator input is required and / or desired. Additionally or alternatively, S220 may function to determine what type of remote operator request to trigger, when to trigger the remote operator request, what conditions (e.g., time restrictions) to attribute to the request, and / or to determine any other information. Additionally or alternatively, S220 may perform any other suitable function.

[0097] S220 is preferably executed in response to and based on S210 (e.g., S212, S214, etc.), but may additionally or alternatively be executed in response to any other process. S220 may additionally or alternatively be executed in response to and / or after a minimum-risk operation (e.g., the first minimum-risk operation in a tiered set) is initiated (in which case it may function, for example, to perform an initial minimum-risk operation without relying on a remote operator (e.g., if the remote operator is unavailable)), before a minimum-risk operation (e.g., any minimum-risk operation, a more aggressive minimum-risk operation, etc.) is initiated, if a minimum-risk operation has not been triggered, and / or at any other time.

[0098] In a first series of variations, a remote operator request is triggered in response to S212, and the remote operator request functions to trigger a set of decision options (e.g., as described in S230) that are sent to the remote operator in response to detecting a fault and / or executing a fault response (e.g., executing a minimum risk condition and / or minimum risk operation, contacting the remote operator, etc.), thereby enabling the remote operator to select a behavior that the agent will perform to resume operation from the MRC. In a specific example, for example, the decision options include behaviors that the autonomous agent can perform to resume operation after being stopped as part of the MRC.

[0099] In a second series of variations, additionally or alternatively to the first series of variations, a remote operator request is triggered in response to S214, the remote operator request serving to trigger a set of decision options to be provided to the remote operator. Additionally or alternatively, the remote operator request can trigger an option for direct control of the autonomous agent to the remote operator.

[0100] In a second set of variations, the remote operator request may optionally be triggered in response to detecting a particular scenario (e.g., a context from a predetermined set of contexts) associated with the autonomous agent. This may include, for example, detecting a context in which the agent finds it difficult to navigate autonomously and / or a context associated with a predetermined trigger for a remote operator request. Additionally or alternatively, the remote operator request may be triggered based on any other information and / or scenario, such as detecting that the agent is stationary for more than a threshold period of time, that an obstacle is blocking the agent's progress and / or that the obstacle is stationary, that there are no other available routes that the autonomous agent can self-select from, or that new information has been received from another fleet vehicle and / or a fleet command center.

[0101] In a first set of specific examples where the autonomous agent is used as a delivery vehicle and needs to load and unload cargo, the loading and unloading areas may be in dynamically changing locations within the pickup and delivery sites, which may be difficult for the autonomous agent to autonomously adapt to. When a context is identified indicating that the autonomous agent has arrived at a loading and / or unloading site, a request may be sent to a remote operator for input (e.g., an indication of the site's location, direct control commands for the vehicle, providing a set of waypoints for navigating the vehicle to a particular location within the site, selecting a set of paths for navigating the vehicle within the site, etc.).

[0102] In a second set of illustrative examples, the local computing subsystem detects that an obstacle is impeding the autonomous agent's movement and that the autonomous agent has been stationary for at least a predetermined threshold time (e.g., at least 10 seconds, between at least 10 seconds and 2 minutes, etc.). In response, a request is sent to the remote operator, which may include a set of behavior decision options, a set of route change options (e.g., reroute the agent to an alternate and / or backup fixed route, reroute the agent to a different locale, etc.), a direct control option, and / or any other option. Additionally or alternatively, the agent's scenario (e.g., context) can optionally be determined and / or considered, which can function to refine the decision options presented to the remote operator, determine whether to suggest a route change for the agent to the remote operator, determine whether to suggest a directional control option, and / or perform any other function.

[0103] The remote operator request preferably includes an alert provided on an interface and / or set of output devices (e.g., visual display, audio output device, tactile output device, etc.) associated with the remote operator (e.g., at the remote operator station). In an example, for example, the alert may include any or all of a visual alert (e.g., a message, notification, and / or annotation provided on a display of the remote operator workstation), an audio alert (e.g., an alert provided on an audio output device of the remote operator workstation, an alert provided on the remote operator's mobile device, etc.), a tactile alert (e.g., a vibration provided on the remote operator's handheld device (e.g., joystick, gear shift, etc.)), and / or any other alert or combination of alerts.

[0104] S220 may additionally or alternatively determine the types of outputs (also equivalently referred to herein as options) that are provided to the remote operator in S230 and / or function to trigger any or all of the providing of the outputs in S230.

[0105] Additionally or alternatively, S220 may include any other suitable processes.

[0106] 4.3 Method—Step S230 of Presenting a Set of Outputs to a Remote Subsystem Method 200 may include step S230 of presenting a set of outputs to a remote subsystem, which functions to prompt input from a remote operator, alert the remote operator of particular scenarios (e.g., contexts) and / or situations (e.g., triggering of MRCs) related to the vehicle, and / or provide the remote operator with information for decision making. Additionally or alternatively, S230 may perform any other function.

[0107] S230 is preferably performed in response to S220 (e.g., when S220 is performed). Additionally or alternatively, S230 may be performed in response to another process of method 200 (e.g., in response to S210), multiple times (e.g., continuously) during method 200, in the absence of S220, and / or at any other time.

[0108] The set of outputs is preferably provided on a set of output devices (e.g., displays, touchscreen interfaces, etc.) of a set of one or more interfaces (e.g., as described above) associated with a remote operator (e.g., as described above), but may additionally or alternatively be provided on any other device and / or in any other manner. The remote operator is preferably located remotely from the autonomous agent, such as at a remote center (e.g., a set of multiple remote monitor workstations) that hosts the set of remote interfaces. Alternatively, the remote operator may be located onboard the autonomous vehicle (e.g., as an onboard human operator, safety driver, etc.) and / or located in other manners.

[0109] In a preferred set of variations, a remote operator monitors multiple autonomous vehicles. In alternative variations, a remote operator can monitor a single autonomous vehicle in a 1:1 manner, multiple remote operators can monitor a single vehicle, and / or any combination of monitoring configurations can be implemented.

[0110] The output provided to the remote subsystem in S230 preferably includes sensor information (e.g., sensor streams, sensor data, etc.) collected in any or all of the sensor subsystems, allowing the remote operator to view the sensor information in a remote interface. The sensor information is preferably provided continuously as sensor streams (e.g., video streams from one or more cameras, sensor streams from fused sensor data, etc.), but may additionally or alternatively be provided intermittently in response to a trigger (e.g., S220) and / or at any other time.

[0111] In a preferred set of variations, a data stream from one or more optical sensors (e.g., cameras) mounted on the vehicle is provided to the remote operator, which functions to allow the remote operator to see what the vehicle is perceiving in its environment. This data stream may include any or all of individual camera streams (e.g., video streams), aggregated camera streams (e.g., from multiple cameras), processed (e.g., edited, cropped, modified, etc.) camera streams, and / or any other optical sensor data.

[0112] Additionally or alternatively, data (e.g., raw data, processed data, etc.) from any other sensors (e.g., lidar, radar, etc.) on or associated with the vehicle may be provided to the remote operator. This additional sensor data may be combined (e.g., overlaid, fused, etc.) with the optical sensor data, provided in an additional manner to the optical sensor data, provided on demand, and / or provided in any other manner as suitable.

[0113] In one set of examples, for example, decisions made by the vehicle regarding its environment (e.g., perception decisions, object detection, object characterization, etc.) are indicated to a remote operator (e.g., through annotated image data, through outlined objects in the image data, etc.).

[0114] In certain specific examples, objects identified by the vehicle's perception subsystem are shown (e.g., highlighted, annotated, etc.) to the remote operator, and the remote operator can optionally provide feedback (e.g., as described below) related to any inaccuracies in the vehicle's perception and / or understanding of the environment (e.g., feedback may be provided to the vehicle [e.g., the vehicle's processing and / or computing subsystem] for correction of the vehicle's perception and / or downstream decision-making).

[0115] S230 preferably further optionally includes providing a set of decision options to the remote operator (e.g., in response to / as part of the decision request of S220). S230 may additionally or alternatively include determining (e.g., selecting) which outputs and / or types (e.g., categories) of outputs are to be presented to the remote operator in the remote subsystem.

[0116] The set of decision outputs is preferably determined at least in part based on any or all of the detected and / or characterized faults (e.g., determined in S212), the detected and / or characterized scenarios (e.g., determined in S214), the type of minimum-risk operation associated with the vehicle's fault response (e.g., one that has already been implemented, one that is part of an ordered list of minimum-risk operation options, etc.), and / or any other information. Additionally or alternatively, the set of decision options may be determined and / or narrowed based on any or all of the set of safety analyses (e.g., described in S250), previous instances of S240 (e.g., previous remote operator input), any other process instances and / or previous instances of the method, and / or any other information.

[0117] Determining the decision options may optionally further include determining (e.g., specifying) a set of conditions (e.g., parameters) associated with the decision options presented to the remote operator, such as, but not limited to, a time limit within which the remote operator must respond to consider the remote operator's input (e.g., the time until the next minimum risk maneuver is triggered by the vehicle), the number of decision options the remote operator can select, and / or any other conditions.

[0118] In a preferred set of variations, input from the remote operator is always and / or at least sometimes optional, such that the AV will operate (e.g., enter minimum risk operation) with or without input from the remote operator. In such cases, a time limit may be provided to the remote operator indicating how quickly the optional input must be provided in order for it to be considered by the vehicle (and / or be more likely to be considered if the fault level does not progress).

[0119] The conditions associated with the decision options may be determined based on any or all of the characteristics described above for determining the decision options (e.g., fault detection and / or characterization, scenario characterization, etc.) and / or based on any other information.

[0120] In some variations, for example, the severity and / or frequency of the detected fault may be used to determine the amount of time a remote operator is given to make a decision before the vehicle autonomously selects the next action (e.g., the next minimum risk maneuver, a more severe minimum risk maneuver, etc.).

[0121] The decision options and / or associated conditions may be determined based on any or all of a set of lookup tables, a set of databases, a set of models and / or algorithms (e.g., learned and / or trained models and / or algorithms, rule-based models and / or algorithms, etc.), a set of decision trees, historical information (e.g., past success rates associated with previous vehicle actions, an aggregated corpus of past vehicle data, etc.), and / or using any combination of tools.

[0122] The set of decision options optionally includes a range of behaviors and / or actions for the autonomous agent. The behavior and / or action options presented to the remote operator may optionally be narrowed (e.g., filtered, prioritized, etc.) based on any or all of the agent's scenario and / or context (e.g., based on the agent's location relative to a map), the vehicle's use case (e.g., a delivery use case) and / or operational design domain (e.g., a fixed route), weather, traffic and / or any other conditions, detected faults and / or fault characteristics of the vehicle, and / or any other information. Alternatively, the option types provided to the remote operator may be predetermined and / or static (e.g., the same for all scenarios).

[0123] The behaviors / actions (also equivalently referred to herein as interventions) in the set of decision options can optionally include behaviors (e.g., high-level behaviors and / or goals) and / or actions that the agent cannot choose for itself and / or behaviors / actions that the agent cannot choose for itself in the current context (e.g., the agent cannot choose for itself to drive in a lane designated for oncoming traffic on a two-way road, but can choose for itself to drive in a lane designated for oncoming traffic in a parking lot). Additionally or alternatively, the behaviors / actions can include those that the agent can choose for itself, any other behaviors / actions, and / or any combination of behaviors / actions.

[0124] In one example sequence (e.g., as shown in FIG. 4), the sequence of behavior and / or action options may include borrowing a nearby lane, ending a current trip and / or starting a new trip, initiating a return of the vehicle to a base location, rerouting the vehicle to a safe stopping point (e.g., the nearest safe stopping point from a predetermined set of verified, dedicated locations labeled on a map [e.g., parking lots, gas stations, etc.], covered locations indicated on a map in response to advance warning of a hailstorm or other weather conditions [e.g., snow, rain, thunder, etc.]), rerouting the vehicle to an alternate route (e.g., having the same destination as the original route), etc. changing the speed limit, pulling the vehicle (e.g., to the curb and / or shoulder), stopping the vehicle in its lane (e.g., at its current position), stopping the vehicle gently (e.g., gradually, at the shortest distance, at maximum deceleration, etc.), stopping at a single waypoint (e.g., waypoint provided by a remote operator), docking at a loading dock, traveling at a set speed or series of speeds (e.g., in response to a speed limit change in a construction zone not yet in view, in response to the presence of a child detected near the road, etc.), and / or any other behavior / action.

[0125] Additionally or alternatively, the series of interventions may include any or all of the actions described in any or all of U.S. patent application Ser. No. 17 / 116,810, filed December 9, 2020, U.S. patent application Ser. No. 17 / 125,668, filed December 17, 2020, and U.S. patent application Ser. No. 17 / 127,599, filed December 18, 2020, which are incorporated herein by reference in their entireties.

[0126] In a particular set of illustrative examples, the set of behavior and / or action options (e.g., return to base, reroute to a safe stopping point, reroute to an alternate fixed route, approach to a loading dock, etc.) is determined at least in part based on the vehicle's fixed route, delivery use case.

[0127] In another particular example, if an object is detected behaving unexpectedly as indicated by an uncertainty level associated with the object (e.g., a bicycle turning, an object repeatedly crossing a lane, etc.), a set of behavioral options is provided to the remote operator on what the vehicle should do in response, such as, but not limited to, staying behind the object, going around the object, slowing down, waiting until a predetermined space is available in front of the vehicle, proceeding as planned, pulling over to the side of the road, stopping, and / or any other behavior.

[0128] Additionally or alternatively, the set of decision options may include an option for the remote operator to directly control the vehicle (e.g., as shown in FIG. 4), in which case the remote operator provides control commands for the vehicle directly from the remote interface. Accordingly, the remote interface preferably includes one or more input devices, such as a joystick, a brake pedal, an accelerator pedal, a steering wheel, and / or any other input device that can be operated / controlled by the remote operator.

[0129] The direct control option is preferably provided selectively in response to the detection of a particular subset of scenarios (e.g., in response to a particular context, such as the context of a loading / unloading area), but may additionally or alternatively be provided in response to a particular minimum risk operation being selected and / or implemented, be provided consistently across all scenarios and / or circumstances, be provided in response to the severity of a particular failure, and / or be otherwise suitably provided.

[0130] In one set of examples, for example, the remote operator may be provided with the option to provide a direct control command to back the vehicle into a loading dock (e.g., upon detecting that the vehicle has arrived at the loading / unloading site). In certain specific examples, the remote operator may provide the direct control command in response to receiving a site-related input or alert (e.g., notification) that the site is ready and available to dock the vehicle (e.g., at a particular loading dock) (e.g., from a field worker, another fleet vehicle at the site, a site management subsystem, a fleet command center, etc.). Additionally or alternatively, the remote operator may provide high-level docking instructions (e.g., where to dock, when to dock, where to wait, etc.).

[0131] Additionally or alternatively, the set of decision options may include options for the vehicle to take an alternative fixed route (and / or sequence of last locations and / or destinations) relative to a currently planned fixed route and / or a fixed route the vehicle is traveling on and / or detouring to that route. In one series of variations, for example, the vehicle operates along a fixed route network, which enables the vehicle to travel between various customer sites (e.g., loading / unloading sites, loading docks, warehouses, distribution centers, etc.), base stations (e.g., home site), maintenance sites, and / or any other locations. The decision options for taking an alternate route may be any or all of the following: continuously provided to the remote operator; triggered based on sensor data and / or analysis thereof; triggered based on the vehicle's stationary waiting time exceeding a predetermined threshold; triggered based on input from a fleet command center associated with the vehicle; triggered based on traffic conditions; triggered based on information from another vehicle in the fleet (e.g., a fleet vehicle positioned at the vehicle's current planned destination indicating that the destination is congested); triggered based on detected failures and / or scenarios; and / or may be otherwise suitably provided to the remote operator.

[0132] In one set of variations, for example, an option for the vehicle to take an alternate route may be provided (e.g., automatically based on sensor data, automatically based on third-party traffic information, manually by a remote operator, etc.) upon determining that the current route is congested, crowded, and / or otherwise suboptimal for the vehicle. In one set of examples, the remote operator elects to select an alternate route for the vehicle upon determining, based on viewing camera data from the vehicle, that there is an obstacle (and / or a high probability that there is an obstacle in front of the vehicle) causing significant traffic congestion.

[0133] Additionally or alternatively, the set of decision options may include commands to indicators and / or auxiliary subsystems of the vehicle, such as, but not limited to, lighting indicators (e.g., turn signals), heating and / or cooling units (e.g., A / C units), windshield wipers, gas tank covers, and / or any other subsystems.

[0134] Additionally or alternatively, the set of decision options can provide the remote operator with the ability to inform and / or modify the vehicle's current understanding of the vehicle's environment, such as object information determined using the vehicle's perception subsystem. For example, the vehicle may recognize an object (e.g., object type, object intent, etc.) incorrectly and / or with a high level of uncertainty (e.g., which may trigger a warning or error), in which case the remote operator can be notified that the object was incorrectly identified. In such a case, the remote operator can provide information that modifies this perception information (e.g., identifying which object type the object is from the set of options), and this additional information can be considered by the vehicle and optionally used (e.g., to adjust its perception understanding).

[0135] In some examples, for example, a remote operator request for clarification (e.g., selection of an object type from a set of multiple objects) may be triggered in response to the vehicle's perception system detecting an object with a high level of uncertainty (e.g., uncertainty above a predetermined threshold). Additionally or alternatively, the remote operator may provide feedback without being specifically prompted. In a specific example, the remote operator may indicate that a potential child detected by the perception system is actually a stationary object (e.g., a mailbox).

[0136] Additionally or alternatively, a remote operator can provide a set of waypoints to the vehicle (e.g., a vehicle planner), which can be used to determine a path (e.g., a trajectory) for the vehicle to follow (e.g., using a conventional motion planner). Waypoints can be provided in response to, for example, any or all of detecting that the vehicle is stranded (e.g., has not changed position for at least a predetermined time), detecting that the vehicle's planner has failed, detecting a particular type of scenario (e.g., a construction zone, a loading dock site, etc.), and / or any other time or situation.

[0137] Additionally or alternatively, the remote operator may provide a route for the vehicle (e.g., a set of waypoints, a continuous route, a route selected from a set of route options for the vehicle, etc.).

[0138] Additionally or alternatively, any other options may be provided to the remote operator in the remote subsystem.

[0139] 4.4 Method - Step S240 of Receiving Input from a Remote Operator Method 200 may include step S240 of receiving input from a remote operator, which serves to receive information that the vehicle can optionally use in vehicle operation (e.g., navigating outside of MRC, navigating in a difficult context, navigating in a context that is optimal for a human driver, etc.), decision-making, and / or any other use case.

[0140] S240 is preferably performed in response to and based on S230, but may additionally or alternatively be performed in response to any other process.

[0141] Additionally or alternatively, S240 may be executed without providing a set of outputs and / or options to the remote operator, such as when the vehicle has not yet detected a fault and / or when the vehicle has not detected a need / desire for assistance from the remote operator.

[0142] As described above, the inputs may include any or all of behavior and / or action selections, control commands (e.g., throttle and / or steering and / or braking control commands in a direct control operating mode), waypoints, paths, sensory adjustments, any other information, or any combination of information.

[0143] Additionally or alternatively, S240 may include any other process.

[0144] 4.5 Method - Step S250 for Processing Remote Operator Input Method 200 may optionally include a step S250 of processing the input, which preferably functions to verify that the input is sufficient to execute the autonomous agent (e.g., with respect to safety, with respect to the autonomous agent's goals, with respect to thresholds associated with actuation subsystems, etc.). Additionally or alternatively, S250 may function to modify / adjust the input (e.g., to make it sufficient to execute the agent), reject the input (e.g., trigger a minimum risk condition instead), and / or perform any other function.

[0145] S250 is preferably performed in response to and based on S240, but may additionally or alternatively be performed in response to any other process of method 200, before and / or during any other process of the method, based on any other information (e.g., input received in S210), and / or S250 may be suitably performed in other manners.

[0146] In some variations, S250 is performed before and / or during S240, and only verified options are presented to the remote operator.

[0147] In additional or alternative variations, S250 is executed while the remote operator is providing feedback (e.g., an initial subset of waypoints) so that the remote operator can be alerted if their input is not validated (e.g., so that the remote operator does not continue to waste time providing additional information that is not implemented).

[0148] Input from a remote operator is preferably processed in local subsystem 120, and more preferably in a local computing subsystem (e.g., an AV computer) located onboard the autonomous agent, which serves to pass the input to the AV logic of the local computing subsystem, thereby authorizing the autonomous agent to validate and implement the input and, optionally, providing an additional layer of safety for implementing human input. Additionally or alternatively, input may be processed in remote subsystem 110, another computing / processing subsystem, and / or any combination of locations.

[0149] Alternatively, input may be performed without validation, processing, and / or receipt at a local computing subsystem.

[0150] Processing the input preferably includes evaluating the input (or a derivative of the input, such as a trajectory determined based on the input) against a set of safety constraints and / or other success criteria. This preferably applies to at least control command inputs received during a remote operator's direct control mode of operation, but may additionally or alternatively apply to selected behaviors / actions, control commands generated by AV logic, and / or any other input.

[0151] The safety constraints may include and / or be determined based on (e.g., as shown in FIG. 5 ) any or all of information from an AV perception module associated with the agent (e.g., whether the vehicle is in the expected situation, whether there is enough space to execute the control command, etc.), sanity checks (e.g., rule-based constraints) and / or speed limit logic (e.g., whether a control command will cause the vehicle to exceed a speed limit or other speed threshold, whether a control command will cause the vehicle to fall below a minimum speed threshold, etc.), feedback from the vehicle's actuation subsystem (e.g., whether the control command can be implemented based on the current state of the vehicle), and / or any other information.

[0152] In a specific example, for example, a remote operator makes a decision selection to return to base, and the decision selection is communicated to AV logic in a local computing subsystem onboard the agent. However, if bad weather is detected by the sensor subsystem, the AV logic may reject this decision and then trigger a repetition of any or all of the previous processes of the method (e.g., S210, S220, etc.), which may optionally prevent this decision option from being presented to the remote operator in the next iteration of S230. Additionally or alternatively, the decision option may be re-included, may be automatically filtered by weather conditions, and / or the method may be suitably performed in other ways.

[0153] In an additional or alternative set of variations, the remote operator may be provided with preemptive warnings and / or rejections of inputs provided by the remote operator. In one example, for example, if the remote operator is entering waypoints, one or more of which require passing through predetermined / stationary objects in the map, the remote operator may be alerted that the waypoints they are providing are not feasible for the vehicle to implement. This may serve to prevent mistakes and / or dangers (e.g., collisions) by the remote operator and / or limit the time the remote operator spends on tasks that are not feasible (e.g., when multiple vehicles are being monitored by a single remote operator).

[0154] In an additional or alternative set of variations, the remote operator selects a vehicle behavior that involves the vehicle pulling up to a curb. In an example, S250 includes verifying (e.g., based on time, speed, objects near the vehicle, etc.) that a curb is present and that the vehicle can reach the curb without risk of collision, determining a trajectory for the vehicle (e.g., using traditional planning techniques rather than a learned / trained model), and operating the vehicle according to the trajectory (e.g., in S260). If the vehicle cannot reach the curb, a minimum-risk maneuver (e.g., slowing down, stopping, etc.) may additionally or alternatively be performed.

[0155] 4.6 Method - Step S260 of Operating an Autonomous Agent The method 200 may include a step S260 of operating the autonomous agent, which functions to control the autonomous agent.

[0156] S260 is preferably executed in response to S250 and / or S240, but may additionally or alternatively be executed in response to any other process, in the absence of any other process (e.g., if a remote operator is not triggered), multiple times (e.g., continuously), and / or at any other time.

[0157] The autonomous agent may act based on any or all of the input received from the remote operator, the processed input from the remote operator (e.g., validated in S250), the control commands generated by the autonomous agent (e.g., if the remote operator is not triggered, if the input from the remote operator does not satisfy the safety constraints, etc.), and / or any other information. Additionally or alternatively, if the remote operator is triggered and the input from the remote operator is not implemented (e.g., because validation failed), the autonomous agent may remain in the MRC (e.g., stopped state) until any or all processes of the method are repeated and / or until the autonomous agent is able to autonomously navigate its environment.

[0158] Additionally or alternatively, S260 may include any other process.

[0159] Although omitted for the sake of brevity, preferred embodiments include any combination and permutation of the various system components and various method processes, which may be performed in any suitable order, sequentially, or simultaneously.

[0160] System and / or method embodiments may include any combination and permutation of the various system components and various method processes, and one or more instances of the methods and / or processes described herein may be performed asynchronously (e.g., sequentially), contemporaneously (e.g., simultaneously, in parallel, etc.), or in any other suitable order by and / or using one or more instances of the systems, elements, and / or entities described herein. The following system and / or method components and / or processes may be used in conjunction with, in addition to, instead of, or otherwise integrated with all or a portion of the systems and / or methods disclosed in the above-referenced applications, which are incorporated by this reference in their entireties.

[0161] Additional or alternative embodiments implement the above-described methods and / or processing modules on a non-transitory computer-readable medium storing computer-readable instructions. The instructions may be executed by computer-executable components integrated into the computer-readable medium and / or processing system. The computer-readable medium may include any suitable computer-readable medium, such as RAM, ROM, flash memory, EEPROM, optical devices (CD or DVD), hard drives, floppy drives, non-transitory computer-readable media, or any suitable device. The computer-executable components may include a computing system and / or processing system (e.g., including one or more co-located or distributed remote or local processors) connected to a non-transitory computer-readable medium, such as a CPU, GPU, TPUS, microprocessor, or ASIC, although the instructions may alternatively or additionally be executed by any suitable dedicated hardware device.

[0162] As will be apparent from the foregoing detailed description, drawings, and claims, those skilled in the art can make modifications and variations to the preferred embodiments of the present invention without departing from the scope of the invention, which is defined in the following claims.

Claims

1. 1. A method for utilizing remote operator input in operation of an autonomous vehicle, comprising: receiving a set of sensor inputs at a sensor subsystem of the autonomous vehicle; - said set of sensor inputs a set of trained models for characterizing scenarios relevant to the autonomous vehicle in a computing subsystem of the autonomous vehicle; and processing with a diagnostic aggregator of the autonomous vehicle to determine an aggregated predicted fault score for the autonomous vehicle; comparing the aggregated predicted fault scores to a set of multiple predetermined thresholds; a first fault response is initiated if the aggregated predicted fault score exceeds a first threshold of the set of multiple predetermined thresholds; if the aggregated predicted failure score exceeds a second threshold of the set of multiple predetermined thresholds, a second failure response is initiated, the second failure response comprising autonomously performing a minimal risk maneuver for the autonomous vehicle, the minimal risk maneuver being performed in the absence of the input from the remote operator; presenting information to a remote operator, said information including said set of sensor inputs; the information further includes a set of remote operator input options; (i) the characterized scenario is within a first subset of predetermined scenarios, and the set of input options includes a set of behavior options for the vehicle; (ii) the characterized scenario is within a second subset of predetermined scenarios, and the set of input options allows the remote operator to provide at least one of a set of control commands to actuators of the autonomous vehicle or a set of locations through which the autonomous vehicle should traverse; and (iii) initiating at least one of the first fault response and the second fault response. presenting information, wherein the set of remote operator input options is determined based on at least one of the characterized scenario and the aggregated predicted failure score if any or all of the conditions in - monitoring input from said remote operator; - when the input is received from the remote operator, comparing the input with a set of constraints; - validating the input based on comparing the input to the set of constraints; operating the autonomous vehicle based on the verified input; A method comprising:

2. 2. The method of claim 1 , wherein providing the set of locations through which the autonomous vehicle should traverse includes providing at least one of a set of waypoints for the autonomous vehicle and a path for the autonomous vehicle.

3. The method of claim 2 , wherein the set of locations includes the set of waypoints, and the autonomous vehicle determines a trajectory based on the set of waypoints using a conventional motion planner.

4. The method of claim 1 , wherein the second subset of predetermined scenarios includes the autonomous vehicle's arrival at a loading dock site.

5. 5. The method of claim 4, wherein arrival of the autonomous vehicle at the loading dock site is detected based on a predetermined map and a geofence associated with the loading dock site.

6. 5. The method of claim 4, wherein the information presented to the remote operator further includes a notification from at least one of a fleet command center, a second autonomous vehicle positioned at the loading dock site, and a user interface associated with a field worker located at the loading dock site, the notification including an identifier of a particular loading dock at the loading dock site.

7. The method of claim 1, wherein the second threshold has a value higher than the first threshold.

8. The method of claim 1 , wherein the first fault response comprises sending an alert to the remote operator, and the information comprises the alert.

9. The method of claim 8 , wherein the first failure response does not initiate a minimum risk maneuver for the autonomous vehicle.

10. 2. The method of claim 1, wherein the least-risk operation is the first least-risk operation in an ordered list of least-risk operations, the ordered list of least-risk operations being ordered in order of increasing strength, and the first least-risk operation having the lowest strength.

11. The method of claim 10 , wherein the increase in intensity comprises an increase in value in braking applied by a set of brakes of the autonomous vehicle.

12. operating the autonomous vehicle based on the verified input; a first fault response escalating to a second fault response; and the second least-risk operation in the ordered list of least-risk operations is executed; The method of claim 10, wherein at least one of

13. The method of claim 1 , wherein at least a portion of the set of input options provided to the remote operator are each associated with a time limit for monitoring input from the remote operator.

14. 14. The method of claim 13, wherein a first time limit is associated with an input option provided in response to initiation of the first fault response and a second time limit is associated with an input option provided in response to initiation of the second fault response, the first time limit being greater than the second time limit.

15. 15. The method of claim 14, wherein if at least one of the first time limit and the second time limit is exceeded, the method further comprises implementing a minimum risk maneuver for the autonomous vehicle.

16. The method of claim 1 , wherein the input from the remote operator comprises a selection of an alternative fixed route for the autonomous vehicle relative to a current fixed route being traversed by the autonomous vehicle.

17. 1. A system for utilizing remote operator input in the operation of an autonomous vehicle, comprising: a sensor subsystem configured to collect sensor data; a remote operator interface associated with a remote operator, the remote operator interface being configured to provide the sensor data to the remote operator; a diagnostic aggregation subsystem including a set of processors, receiving the sensor data from the sensor subsystem; Analyzing the sensor data to determine an aggregated predicted failure score for the autonomous vehicle; comparing the aggregated predicted fault scores to a set of magnitude thresholds; a diagnostic aggregation subsystem configured to compare the aggregated predicted failure scores and a set of previously determined aggregated predicted failure scores to a set of time thresholds; a computing subsystem onboard the autonomous vehicle, the computing subsystem comprising: - processing the sensor data to characterize a scenario related to the autonomous vehicle's environment; alarms communicated to the remote operator at the remote operator interface, - exceeding the set of magnitude thresholds; - the set of time thresholds has been exceeded; and The characterized scenario is within a predetermined subset of scenarios. triggering in response to at least one of configured to determine, in response to triggering an alert, a set of response options for the remote operator based on at least one of the aggregated predicted failure score and the characterized scenario, at least some of the set of response options being each associated with a time limit for monitoring input from the remote operator, and the set of response options comprising: a set of behavior options for the autonomous vehicle; a set of locations through which the autonomous vehicle should pass; and a set of direct control commands for actuators of the autonomous vehicle; wherein the computing subsystem comprises at least one of: - providing the set of response options in the remote operator interface; - monitoring input from said remote operator; - when the input is received from the remote operator, comparing the input to a set of constraints; - validating the input based on comparing the input to the set of constraints; operating the autonomous vehicle based on the verified input; The system is configured as follows:

18. 20. The system of claim 17, wherein the computing subsystem is further configured to trigger an alert to the remote operator interface in response to the autonomous vehicle performing a minimum risk maneuver.

19. 20. The system of claim 18, wherein the minimum risk operation is triggered in response to exceeding at least one of the set of magnitude thresholds or the set of time thresholds.

20. The system described in claim 18, wherein the diagnostic aggregation subsystem communicates with a set of trained models executable by the computing subsystem, and the minimum risk operation is triggered in response to an uncertainty measure determined by an out-of-distribution detector communicating with the set of trained models exceeding a predetermined threshold.

21. comparing the aggregated predicted failure scores to the set of time thresholds; determining the number of times that the previous aggregated predicted failure scores exceeded one or more of the set of magnitude thresholds within a predetermined previous time period; and - comparing said number of times with a predetermined threshold value; 20. The system of claim 17, comprising:

22. 20. The system of claim 17, wherein the computing subsystem further provides a set of object detection information determined using a perception subsystem of the computing subsystem to the remote operator interface, and wherein monitoring the input from the remote operator includes monitoring adjustments of the object detection information.

Images