Method and system for feasibility-based operation of autonomous agents - Patents.com
The feasibility-based operation system for autonomous vehicles addresses the unpredictability of surrounding agents by estimating intent through policy scoring, enhancing navigation efficiency and safety.
Patent Information
- Application Number
- JP2024532182
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-12-02
- Filing Date
- 2022-12-01
- Publication Date
- 2024-11-26
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Autonomous vehicles struggle with unpredictability in the behavior of nearby vehicles, leading to cautious driving, inefficiencies, and potential accidents due to unanticipated actions, as current systems fail to accurately predict the actions of surrounding agents.
A feasibility-based operation system and method for autonomous agents that utilize a computing system to estimate the intent of surrounding agents by scoring policies based on their feasibility, incorporating historical data and forward simulations to determine the most likely actions, thereby improving decision-making.
Enhances the autonomous vehicle's ability to navigate more efficiently and safely by accurately predicting the behavior of surrounding agents, reducing computational load, and handling uncertainty, while simplifying intent estimation and improving decision-making processes.
Smart Images

Figure 2024543943000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates generally to the field of autonomous vehicles, and more particularly to novel and useful systems and methods for feasibility-based operation of autonomous agents in the field of autonomous vehicles.
[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 63 / 285,238, filed December 2, 2021, which is incorporated herein by reference in its entirety. [Background technology]
[0003] In current systems and methods related to the field of autonomous vehicles, a major unknown (one that can dramatically affect the outcome of an autonomous vehicle taking a particular action) is how other vehicles are driving and what actions they are about to take. The history and driving history of those nearby vehicles can provide some information about how the vehicle is currently behaving, but does not reveal how the vehicle is likely to behave in the future. This can manifest itself in the autonomous vehicle being forced to be overly cautious or driving in a manner that is significantly different from a human driver on the road, resulting in frustration for other drivers, inefficiencies in the autonomous vehicle reaching its destination, and the possibility of accidents occurring due to the autonomous vehicle's unexpected behavior.
[0004] Thus, there is a need in the field of autonomous vehicles to create improved and useful systems and methods for feasibility-based operation of autonomous agents. The present inventors have discovered such useful systems and methods. [Brief description of the drawings]
[0005] [Figure 1] FIG. 1 is a schematic diagram of a system for feasibility-based operation of autonomous agents. [Diagram 2] FIG. 2 is a schematic diagram of a method for feasibility-based operation of an autonomous agent. [Diagram 3] FIG. 3 is a schematic diagram of one embodiment of a simulation in a method for feasibility-based operation of an autonomous agent. [Figure 4] 4A-4E are schematic diagrams of one embodiment of a method showing policy scoring of agents in an environment. [Diagram 5] FIG. 5 is an example flow chart of one embodiment of a method. [Figure 6] FIG. 6 is a schematic diagram of one embodiment of the method. [Figure 7] FIG. 7 illustrates an example of policy scoring according to an embodiment of the method. [Figure 8] FIG. 8 is an example flow chart of one embodiment of a method. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0006] The following description of preferred embodiments of the invention is not intended to limit the invention to those preferred embodiments, but rather to enable any person skilled in the art to make and use the invention.
[0007] 1. Overview A system 100 for autonomous agent realizability-based operations (an example of which is shown in FIG. 1) can include a computing system 120 (equivalently, also referred to herein as a computer). System 100 further preferably includes and / or cooperates with an autonomous agent 102 (equivalently, also referred to herein as an ego agent), a sensor suite 110 (e.g., onboard the ego agent), and / or any other components. The system optionally includes or can be used with a communications interface 130, a set of infrastructure devices 140, a teleoperator platform 150, and / or any other suitable set of components. Additionally or alternatively, the system may include any or all of the components described in U.S. Application No. 16 / 514,624, filed July 17, 2019, U.S. Application No. 17 / 365,538, filed July 1, 2021, U.S. Application No. 17,550 / 461, filed December 14, 2021, U.S. Application No. 17 / 554,619, filed December 17, 2021, U.S. Application No. 17 / 712,757, filed April 4, 2022, and / or U.S. Application No. 17 / 826,655, filed May 27, 2022 (each of which applications is incorporated by reference in its entirety). The system 100 functions to facilitate autonomous operation of an autonomous vehicle based on intent estimation (based on feasibility) of an agent in an environment and / or a feasible policy of an agent in an environment.
[0008] Method 200 (an example of which is shown in FIG. 2) may include receiving a set of inputs S100, determining a set of policies based on the set of inputs S200, determining a set of scores associated with the set of environmental policies S300, and evaluating the set of policies S400. Additionally or alternatively, method 200 may include operating the ego agent according to the selected policies and / or any other process. Further, additionally or alternatively, method 200 may include and / or may cooperate with any or all of the methods, processes, embodiments and / or examples described in U.S. Application No. 16 / 514,624, filed July 17, 2019, U.S. Application No. 17 / 365,538, filed July 1, 2021, U.S. Application No. 17,550 / 461, filed December 14, 2021, U.S. Application No. 17 / 554,619, filed December 17, 2021, U.S. Application No. 17 / 712,757, filed April 4, 2022, and / or U.S. Application No. 17 / 826,655, filed May 27, 2022 (each of which applications is incorporated by reference in its entirety). Method 200 functions to facilitate scoring of policies assigned to other agents in the ego agent's environment based on how feasible it is for those other agents to implement the policies. Additionally or alternatively, method 200 may function to facilitate autonomous operation of a vehicle (e.g., based on the feasibility of the agents' policies in the environment). Additionally or alternatively, method 200 may function to facilitate intent estimation of agents in the environment.
[0009] In a first example, system 100 and / or method 200 can incorporate the feasibility of an environmental agent's policies into the ego-agent's multi-policy decision-making (MPDM) process, thereby facilitating autonomous self-decision-making (e.g., based on feasibility) under various environmental policy scenarios.
[0010] The method 200 may be performed by the system 100 as described above, and / or by any other suitable system.
[0011] 1.1 Example In a first set of aspects, a method for agents (e.g., other vehicles, pedestrians, etc.) in an environment of an autonomous vehicle (equivalently referred to herein as environmental agents) includes: determining a set of inputs using a sensor suite of the autonomous vehicle, the set of inputs including an environmental agent instance identifier and a state history associated with the environmental agent instance identifier; determining a set of environmental policies for the environmental agent instance identifier based on the set of inputs; for each environmental policy of the set of environmental policies, determining a past score by comparing a reference trajectory (i.e., a fragment of an intended future route) associated with the environmental policy with the state history of the environmental agent, and determining a feasibility score by forward simulation of the environmental policies; determining an ego policy by evaluating a set of vehicle policies of the autonomous vehicle against the set of environmental policies based on the feasibility score and the past score; and operating the autonomous vehicle based on the ego policy. In a particular example, the ego vehicle policy can be determined by a multi-policy decision-making (MPDM) system that performs a (complex) simulation of the agent's interactions based on the respective sets of scores of each policy candidate. In a particular example, the method can be a method of agent intention estimation based on the feasibility of the agent in the environment. Additionally or alternatively, the ego vehicle policy may be determined according to any other system, module, procedure and / or protocol.
[0012] In a second set of aspects, a method of operating an autonomous vehicle with respect to agents in an environment of the autonomous vehicle includes tracking a set of agents in the environment based on vehicle sensor data, the set of agents including identifying a state history for each agent of the set of agents; determining a set of policy candidates for each agent; determining a respective set of scores for each policy candidate, the respective set of scores including determining a first score based on a comparison of the policy candidates to the state history of the agents and determining a second score by forward simulation of the policy candidates; and operating the autonomous vehicle based (together) on the respective set of scores for each policy candidate. In a particular example, the method may further include determining an ego policy for the autonomous vehicle based on the respective set of scores for each policy candidate using a multi-policy decision-making (MPDM) system, and operating the autonomous vehicle may include executing the ego policy.
[0013] 2. Advantages Systems and methods for operation of autonomous agents can provide several advantages over current systems and methods.
[0014] In a first aspect, the system and / or method provides the advantage of being able to take into account the feasibility associated with each of a set of policies that may be assigned to objects (e.g., other agents, pedestrians, animals, obstacles, etc.) in the ego agent's environment (also equivalently referred to herein as environmental objects or neighboring objects or proximate objects or environmental agents), which may be used in a simulation or used to select a policy for the ego agent to execute. In a particular example, for example, a feasibility score is determined for each of the set of policies associated with the ego agent's neighboring agents / objects, thereby taking into account the feasibility of the neighboring agents / objects implementing a particular policy in the future when selecting a policy (also equivalently referred to herein as an ego policy) for the ego agent to implement as it navigates its environment.
[0015] In a particular example, the policies, along with their feasibility scores, are used to perform intention estimation of the environment agent in a simulation, and the simulation is performed according to the ego agent's multi-policy decision-making (MPDM) module.
[0016] In a second aspect, in addition to or instead of the first aspect, the system and / or method provides the advantage of performing a forward simulation that examines the predicted future behavior of the environmental agent, which helps inform which policies are likely to be executed by the agent and / or which policies will best help the agent achieve its goal (e.g., progress toward the goal, a smooth trajectory, etc.). The inventors have found that by considering future time steps and how the agent will behave when various policy options are selected, rather than only looking at the historical data that has been tracked, the most feasible policy for the environmental agent can be more reliably and accurately selected. For example, a policy that was feasible based on historical data may be difficult to continue in the future, and instead, a policy that the vehicle may take in the future may not be directly obvious from what the vehicle has done in the past. The system and / or method can optionally take historical information into account as well (e.g., when determining the distinct scores, when performing the forward simulation, etc.).
[0017] In a third aspect, in addition to or instead of those described above, the system and / or method prioritizes the most feasible policies evaluated in each selection cycle, thereby providing the advantage that the most feasible policies can be simulated before a decision is made by the ego agent. Additionally or alternatively, the system and / or method can provide the advantage of reducing the number of policy combinations tested in each selection cycle of the ego agent's operation, thereby reducing the computational load required to evaluate those policies, by eliminating policies that are difficult to realize (e.g., having a feasibility score below a pre-set threshold).
[0018] In a fourth aspect, in addition to or instead of those described above, the system and / or method may facilitate robust handling of uncertainty in the intent (estimation) of a vehicle / agent in an environment.
[0019] In a fifth aspect, in addition to or instead of those described above, a system and / or method may simplify and / or streamline intent estimation in an (uncertain) environment by individually analyzing the feasibility of each agent in the scene. For example, feasibility simulation may be considered "feed-forward" or "open-loop" since it may ignore interactions between agents (and / or ego vehicles) in the environment. Furthermore, by individually analyzing / estimating the intent of agents, feasibility simulation may be parallelized and / or made deterministic (e.g., even with different degrees of environmental uncertainty), allowing downstream simulations, such as by MPDM algorithms / modules, to handle complex interactions and planning more efficiently.
[0020] Additionally or alternatively, the systems and methods may provide any other advantages.
[0021] 3. System 100 A system 100 for feasibility-based operation of an autonomous agent (an example of which is shown in FIG. 1 ) can include a computing system 120 (equivalently, also referred to herein as a computer). The system 100 further preferably includes and / or cooperates with an autonomous agent 102 (equivalently, also referred to herein as an ego agent), a sensor suite 110 (e.g., onboard the ego agent), and / or any other components. The system 100 can optionally include or be used with a communication interface 130, a set of infrastructure devices 140, a teleoperator platform 150, and / or any other suitable set of components. The system 100 functions to facilitate autonomous operation of an autonomous vehicle based on (feasibility-based) intent estimation of an agent in an environment and / or a feasible policy of the agent in the environment.
[0022] System 100 preferably includes and / or cooperates with (e.g., is integrated within) an autonomous vehicle (equivalently referred to herein as an autonomous agent, agent, and / or ego agent). The autonomous agent is preferably an autonomous vehicle, and more preferably a fully autonomous vehicle and / or a vehicle capable of operating as a fully autonomous vehicle, but may additionally or alternatively be a semi-autonomous vehicle and / or any other vehicle.
[0023] In a preferred embodiment, the autonomous vehicle is a motor vehicle (e.g., a car, a driverless vehicle, a bus, a shuttle, a taxi, a ride-share vehicle, a truck, a semi-truck, etc.). Additionally or alternatively, the autonomous vehicle can include any or all of a watercraft (e.g., a boat, a water taxi, etc.), an air vehicle (e.g., an airplane, a helicopter, a drone, etc.), a ground vehicle (e.g., a two-wheeler, a motorcycle, a motor cycle, a scooter, etc.), and / or any other suitable vehicle and / or transportation device, an autonomous machine, an autonomous device, an autonomous robot, and / or any other suitable device.
[0024] The autonomous agent 102 preferably includes and / or interfaces with a computing system 120 operable to process information (e.g., sensor inputs) to determine a trajectory to be performed by the vehicle. Additionally or alternatively, the computing system may be operable to perform any or all of the processes involved in any or all of intent estimation, perception, prediction, localization, planning, and / or any other processes involved in the operation of the autonomous agent. Additionally or alternatively, the computing system 120 may be operable to perform any or all portions of method 200.
[0025] The computing system 120 preferably includes an on-board computing system located onboard (e.g., integrated within) the ego agent. Additionally or alternatively, the computing system may include any or all of a remote computing system (e.g., a cloud computing system in lieu of an on-board computing system, a remote computing system in communication with an on-board computing system, etc.), a computing system integrated into an auxiliary device (e.g., a mobile device, a user device, etc.), an edge device including a mobile computing device, and / or any other suitable computing system and device. In an aspect, for example, the ego agent is operable to communicate with a remote or distributed computing system, which may include a user device (e.g., a mobile phone, a laptop, etc.), a remote server, a cloud server, or any other suitable local and / or distributed computing system off the vehicle. The remote computing system may be connected to one or more systems of the autonomous agent through one or more data connections (e.g., channels), but may alternatively communicate with the vehicle systems in any suitable manner.
[0026] The computing system 120 may include and / or cooperate with a processing system (e.g., a processor or set of processors, a graphics processing unit or GPU, a central processing unit or CPU, or any suitable processing circuitry) and memory, but may additionally or alternatively include any other suitable components. The memory may be short-term (e.g., volatile, non-volatile, random access memory or RAM, etc.) and / or long-term (e.g., flash memory, hard disk, etc.) memory. In an aspect, for example, the on-board computing system functions to interact with and / or operably control one or more of the identified components or modules described herein. In a preferred aspect, for example, the on-board computing system executes computer instructions to implement a multi-policy decision-making module. In a particular example, the processing system and memory collectively function to dynamically manage a set of policies available to an autonomous agent within the framework of a multi-policy decision-making framework, such as those described in U.S. Application No. 16 / 514,624, filed July 17, 2019, which is incorporated herein by reference in its entirety. Additionally or alternatively, the processing system and memory and / or any other suitable components may be used for any other suitable functionality.
[0027] In a particular example, the multi-policy decision-making (MPDM) module includes a simulator module or similar machine or system that functions to estimate future (i.e., steps ahead in time) action policies (operations or movements) for each of the environmental agents (e.g., other vehicles in the ego agent's environment) and / or objects (e.g., pedestrians) identified in the autonomous agent's operating environment (real or virtual), including potential action policies that may be executed by the ego agent. The simulation may be based on each agent's current state (e.g., current assumptions) and / or each agent's past movements or past behaviors derived from a historical data buffer (preferably including data up to the present time). The simulation may provide data related to the mutual relationships (e.g., relative positions, relative velocities, relative accelerations, etc.) between each agent's predicted action policy and one or more potential action policies that may be executed by the autonomous agent.
[0028] System 100 optionally includes a communications interface 130 in communication with computing system 120, the communications interface functioning to allow for receiving information (e.g., from an infrastructure device, from a remote computing system and / or remote server, from teleoperator platform 150, another autonomous agent or other vehicle, etc.) and transmitting information from the computing system (e.g., to a remote computing system and / or remote server, to teleoperator platform 150, to infrastructure device 140, another autonomous agent or other vehicle, etc.). Communications interface 130 preferably includes a wireless communication system (e.g., Wi-Fi, Bluetooth, Cellular 3G, Cellular 4G, Cellular 5G, Multiple Input Multiple Output or MIMO, one or more radios, or any other suitable wireless communication system or protocol), but may additionally or alternatively include any or all of a wired communication system (e.g., modulated power line data transmission, Ethernet, or any other suitable wired data communication system or protocol), a data transmission bus (e.g., CAN, FlexRay), and / or any other suitable components.
[0029] System 100 may optionally include and / or cooperate with a set of infrastructure devices 140, also referred to herein equivalently as roadside units (e.g., FIG. 1 ), that function individually and / or collectively to observe one or more aspects and / or characteristics of the environment and collect observational data regarding one or more aspects and / or characteristics of the environment. The set of infrastructure devices preferably communicate with the autonomous agent's on-board computing system, but may additionally or alternatively communicate with the teleassist platform, any other components, and / or any combination.
[0030] The infrastructure devices preferably include devices in the immediate vicinity and / or within close or short-range communication proximity of the autonomous agent's operating location and may function to collect data regarding the environment surrounding the autonomous agent and within areas proximate to the autonomous agent's operating zone. In an embodiment, the roadside units include one or more of off-board sensing devices including flash LIDAR, thermal imaging devices (thermal cameras), still or video capture devices (e.g., imaging cameras and / or video cameras, etc.), global positioning systems, radar systems, microwave systems, inertial measurement units (IMUs), and / or any other suitable sensing device or combination of sensing devices.
[0031] The system preferably includes and / or interfaces with a sensor suite 110 (e.g., computer vision system, LIDAR, RADAR, wheel speed sensors, GPS, cameras, etc.) that communicates with the on-board computing system and functions to gather information for determining one or more trajectories of the autonomous agent. Additionally or alternatively, the sensor suite may function to enable operation of the autonomous agent (e.g., autonomous driving), acquisition of data regarding the environment surrounding the autonomous agent, acquisition of data regarding the operation of the autonomous agent, detection of the need for maintenance of the autonomous agent (e.g., via engine diagnostic sensors, external pressure sensor strips, sensor health sensors, etc.), detection of cleanliness standards within the autonomous agent (e.g., internal cameras, ammonia sensors, methane sensors, alcohol vapor sensors), and / or any other suitable functionality.
[0032] The sensor suite may include any or all of vehicle sensors onboard the autonomous agent, such as inertial sensors (e.g., accelerometers, gyroscopes, magnetometers, IMUs, INS, etc.), external antennas (e.g., GPS, cellular, Bluetooth, Wi-Fi, near field communication, etc.), diagnostic sensors (e.g., engine load, tire pressure, temperature sensors, etc.), vehicle motion sensors (e.g., inertial sensors, wheel speed sensors, encoders, resolvers, etc.), environmental sensors (e.g., cameras, time-of-flight sensors, temperature sensors, wind speed / direction sensors, barometers, etc.), inductive sensors (e.g., LIDAR, RADAR, sonar, cameras, etc.), computer vision (CV) sensors, cameras (e.g., stereo cameras, hyperspectral, multispectral, video cameras, wide angle, CMOS, CCD, etc.), time-of-flight sensors (e.g., RADAR, LIDAR, sonar, etc.), and / or any other suitable sensors. The sensor suite preferably includes sensors onboard the autonomous vehicle (e.g., RADAR and / or LIDAR sensors and / or cameras coupled to an exterior surface of the agent, IMUs and / or encoders coupled to and / or disposed within the agent, etc.), but may additionally or alternatively include sensors remote from the agent (e.g., part of one or more infrastructure devices, sensors in communication with the agent, etc.), and / or any suitable sensors in any suitable location. However, the sensor suite may include any other suitable set of sensors and / or be otherwise configured as appropriate.
[0033] The system may optionally include and / or be in communication with a vehicle control system including one or more controllers and / or control systems including any suitable software and / or hardware components (e.g., processors and computer-readable storage devices) utilized to generate routing goals for the autonomous agents and control signals for controlling the autonomous agents in accordance with a selected behavioral policy and / or selected trajectory of the autonomous agents.
[0034] In a preferred embodiment, the vehicle control system includes, interfaces with, and / or implements a drive-by-wire system of the vehicle. Additionally or alternatively, the vehicle may operate according to the actuation of one or more mechanical components and / or may be otherwise implemented.
[0035] Additionally or alternatively, system 100 may include and / or cooperate with any other suitable components.
[0036] 4. Method Method 200 (an example of which is shown in FIG. 2) may include steps S100 of receiving a set of inputs, S200 of determining a set of policies based on the set of inputs, S300 of determining a set of scores associated with the set of environmental policies, and S400 of evaluating the set of policies. Additionally or alternatively, method 200 may include operating an ego agent according to the selected policies and / or any other process. Further, additionally or alternatively, method 200 may include and / or may cooperate with any or all of the methods, processes, embodiments and / or examples described in U.S. Application No. 16 / 514,624, filed July 17, 2019, and U.S. Application No. 17 / 365,538, filed July 1, 2021, each of which is incorporated by reference in its entirety.
[0037] The method 200 functions to evaluate and / or score the feasibility associated with a set of potential / candidate policies of tracked agents in the ego agent's environment, such that the feasibility represents which policies are most likely to be implemented by those tracked agents. As such, the method preferably further functions to downweight (also equivalently referred to herein as downscale or scale down) policies that are impossible for the tracked agents to implement (e.g., those that would cause the agent to run off the road, those that would cause the agent to collide with other agents or objects, those that would cause the agent to approach a turn at an unsafe speed, those that would prevent the agent from reaching a predicted goal, etc., based on the feasibility of the "control effort" to achieve the policy's actions / trajectories) so that more feasible policies are considered first or are otherwise prioritized. As a first example, a policy related to the vehicle's behavior in cornering may become less "feasible" and / or may require greater control effort (e.g., greater acceleration values, greater steering angles and / or steering forces, greater braking values, etc.) at higher speeds and / or when larger steering angles are required, to the point where it may be impractical or impossible for the vehicle and / or driver to execute the maneuver. For example, a hairpin turn at a speed greater than 50 miles per hour may exceed the vehicle / driver's physical limits. As a second example, a policy related to stopping at a stop sign (e.g., in accordance with road rules) may similarly become less feasible (eventually unattainable) as the distance between the vehicle and the intersection approaches (or becomes smaller than) the vehicle's minimum braking distance for the vehicle's instantaneous vehicle conditions, especially taking into account the driver's reaction time (e.g., typically about 0.5 seconds to 1 second). Additionally or alternatively, the method 200 may function to determine any other score and / or perform any other suitable function.
[0038] Method 200 is preferably performed according to an MPDM framework (e.g., as described above) in which a policy for an ego agent is selected from a set of multiple policies based on simulating each of the set of multiple policies during operation of the ego agent (e.g., at each selection cycle during operation of the ego agent) and selecting an optimal policy based on the results of those simulations. In those simulations, policy predictions are also assigned to other agents in the ego agent's environment (e.g., as determined in an intent inference process) so that the behavior of those other agents and how they interact and / or react with the ego agent can be taken into account (e.g., through an intent inference process) in the selection of an ego policy. Agents in the ego agent's environment may include any or all of other vehicles (e.g., human-driven vehicles, autonomous vehicles, semi-autonomous vehicles, bicycles, motorcycles, scooters, etc.), pedestrians, animals, objects (e.g., traffic cones, construction equipment, baby strollers, shopping carts, trees, etc.), and / or any combination. Examples of MPDM frameworks are described in U.S. Application No. 16 / 514,624, filed July 17, 2019, and U.S. Application No. 17 / 365,538, filed July 1, 2021, each of which is incorporated by reference in its entirety. Additionally or alternatively, any other MPDM framework may be implemented and / or the MPDM framework may include any other process.
[0039] In a preferred embodiment, for example, the method 200 functions to assign feasibility scores to potential policies assigned to an agent within an ego agent's environment so that an optimal ego policy can be selected for the ego agent.
[0040] Additionally or alternatively, the method 200 may be performed without and / or independent of the MPDM framework and / or may be performed in one or more other frameworks.
[0041] Method 200 is preferably performed on system 100 as described above, but may additionally or alternatively be performed on any other system or systems.
[0042] 4.1 Method - Receiving a Set of Inputs S100 The method 200 includes a step S100 of receiving a set of inputs, which serves to receive information for evaluating the ego agent and / or any agents / objects within the ego agent's environment.
[0043] S100 is preferably executed first in method 200, and more preferably according to a selection cycle during operation (e.g., movement) of the ego agent. The selection cycle is preferably associated with a preset frequency (e.g., once every 1-10 seconds, more than once per second, less than once per 10 seconds, etc.), but may additionally or alternatively be associated with a variable frequency and / or random interval, executed in response to a trigger, and / or implemented in other manners. For example, the selection cycle frequency may be less than 0.1 Hz, 0.1 Hz, 1 Hz, 3 Hz, 5 Hz, 10 Hz, 15 Hz, 20 Hz, 25 Hz, 30 Hz, more than 30 Hz, any open or closed range bounded by those values, and / or any other suitable frequency. Additionally or alternatively, S100 may be executed when there is no selection cycle and / or at any other time during method 200.
[0044] The inputs preferably include information characterizing the location, classification (e.g., vehicle, pedestrian, etc.) and / or movement of environmental agents tracked by the system. Additionally or alternatively, the set of inputs may include information characterizing the ego agent's location and movement, road or other landmark / infrastructure features (e.g., where the lanes are, where the edges of the road are, where the traffic signals are and what type they are, where the agent is relative to those landmarks, etc.), and / or any other information.
[0045] The inputs preferably include sensor inputs received from a sensor suite onboard the ego agent, but additionally or alternatively may include historical information associated with the ego agent (e.g., past state estimates of the ego agent, historical information associated with an agent instance identifier) and / or historical information associated with the environmental agent (e.g., past state estimates of the environmental agent, tracking data, etc.), past feasibility / scores (e.g., from previous iterations of S300), sensor inputs, information and / or any other inputs from sensor systems external to the ego agent (e.g., those onboard other ego agents or environmental agents, those onboard a set of infrastructure devices, etc.).
[0046] S100 preferably includes pre-processing any or all of the set of inputs, which serves to prepare the set of inputs for analysis in subsequent processes of method 200.
[0047] Pre-processing the set of inputs may optionally include calculating a state estimate of the environmental agent (and / or the ego agent) based on the set of inputs. The state estimate preferably includes position data, attitude data (e.g., attitude relative to Earth / self, coordinate position, azimuth, etc.), motion data (e.g., linear motion data such as linear velocity, acceleration; angular motion parameters such as angular velocity, angular acceleration; derivatives of the aforementioned parameters such as rate of change of heading, rate of change of acceleration), historical path, and / or any other data / parameters.
[0048] Pre-processing the set of inputs may optionally additionally or alternatively include identifying (e.g., by a computer vision module of the computing system) one or more geometric properties / features associated with the environmental agent / object, such as defining a 2D geometry associated with the environmental agent (e.g., a 2D geometric hull, one or more 2D profiles, an outline of the agent, etc.), a 3D geometry associated with the environmental agent, and / or any other geometry. For example, this may be used to identify (e.g., along with an associated probability / confidence value) in which lane the environmental agent is in (e.g., along with a location or other portion of a state estimate, etc., during S100 and / or in other processes of method 200).
[0049] Pre-processing the set of inputs may optionally additionally or alternatively include determining one or more classification labels associated with any or all of the set of environmental objects / agents, and further optionally determining probabilities and / or confidences (expressed as probabilities) associated with the one or more classification labels. The classification labels preferably correspond to types of agents, including but not limited to vehicles (e.g., binary classification of vehicles) and / or types of vehicles (e.g., sedans, trucks, shuttles, buses, emergency vehicles, etc.), pedestrians, animals, inanimate objects (e.g., road hazards, construction equipment, traffic cones, etc.), and / or any other type of agent. For example, the input may be classified by a set of one or more classifiers (e.g., dynamic object classifiers, static object classifiers, etc.; binary classifiers, multi-class classifiers, etc.), but may additionally or alternatively be performed by any or all of computer vision techniques, machine learning models, object segmentation techniques, point cloud clustering, neural networks (e.g., pre-trained to identify a particular set of objects, such as cars and pedestrians, based on sensor data inputs, etc.; Convolutional Neural Networks [CNN], Fully Convolutional Networks [FCN], etc.), object detectors / classifiers (e.g., the You Only Look Once [YOLO] algorithm; non-neural net approaches such as Histogram of Oriented Gradients [HOG] and / or Scale-Invariant Feature Transform [SIFT] feature detectors, etc.), object trackers, and / or any other suitable processes.
[0050] The classification label is preferably determined at least in part based on any or all of the agent's geometric characteristics (e.g., size, profile, 2D hull, etc.) and state estimates (e.g., velocity, position, etc.), but may additionally or alternatively be determined in other manners.
[0051] In some aspects, objects / agents are preferably uniquely identified / tracked in association with an object instance identifier (equivalently referred to herein as an object / agent ID) such that the object / agent can be individually identified and distinguished from others in the environment at the current timestep and state history. However, agents may also be suitably tracked / referenced in other ways across timesteps, data frames and / or selection cycles.
[0052] In some aspects, objects may be identified, labeled (and / or re-labeled), and tracked, and / or the input may include an environmental representation (and / or tracking history) as described in U.S. Application No. 17 / 554,619, filed December 17, 2021, entitled "METHOD AND SYSTEM FOR DYNAMICALLY UPDATING AN ENVIRONMENTAL REPRESENTATION OF AN AUTONOMOUS AGENT," which is incorporated by reference in its entirety.
[0053] The set of inputs (e.g., the set of preprocessed inputs) is preferably presented as a set of vectors (e.g., a vector for each agent at each time step) for further processing in method 200, but may additionally or alternatively be presented in matrices and / or other data structures.
[0054] S100 may additionally or alternatively include any other processes.
[0055] 4.2 Method - Determining a Set of Policies Based on an Input Set S200 Method 200 includes a step S200 of determining a set of policies based on a set of inputs, which serves to determine which policies are available for the environmental objects / agents, and optionally for the ego agent, so that an optimal (e.g., most feasible) policy can be selected for any or all of those objects and agents. Additionally or alternatively, S200 may serve to eliminate policies from consideration, prioritize policies, apply various weights to any or all of the set of policies, and / or perform any other function.
[0056] A policy may preferably refer to a driving behavior (e.g., keep in lane, turn right, turn left, change lane, stop, accelerate, decelerate, etc.) and / or optionally prescribe (or serve as a reference to) a control command configured to implement or simulate the driving behavior (e.g., along a particular path or reference trajectory for the agent). For example, a policy of an agent in an environment may include or be associated with a deterministic controller that targets the policy's reference trajectory (e.g., to facilitate forward simulation based on the agent's state). As a second example, a policy of an agent in an environment may be used to model and / or simulate a particular driving intent / behavior (e.g., keep in lane, turn right, turn left, change lane, stop, accelerate, decelerate, etc.). Additionally or alternatively, a policy may include and / or reference any other information.
[0057] S200 is preferably executed in response to and based on S100, but may additionally or alternatively be executed multiple times during method 200, before S100 (e.g., in use cases where the policies considered for each agent are predetermined and / or fixed), in response to other processes of method 200 (e.g., in response to determining the set of scores in S300), in response to a trigger, and / or at any other time.
[0058] S200 preferably includes a step S210 of determining a set of environmental policies for the environmental agent based on the set of inputs, which serves to determine a set of environmental policies that the system considers for intent estimation of the environmental agent in a simulation (e.g., a forward simulation as described below). Additionally or alternatively, S200 may include a step S220 of determining a set of ego policies for the ego agent based on the set of inputs (e.g., to execute in the simulation, and / or to include determining any other policies; for use in evaluating the policies of S400).
[0059] The policies (e.g., environmental policies, ego policies, etc.) are preferably determined based on any or all of a set of inputs, such as the set of pre-processed inputs described above (e.g., state estimates, position, velocity, azimuth, classification labels, geometry, etc.). The policies are preferably determined for each individual agent in the environment (e.g., in association with a particular environmental agent instance ID), but may additionally or alternatively be determined based on classification (e.g., for each dynamic / moving object) and / or in other ways. Additionally or alternatively, the policies may be determined based on any other information, such as a map (e.g., defining a set of available policies based on the agent's position in a map), a database, and / or any other tool. As an example, the policies may include (or be associated with) a reference trajectory extracted from a pre-established (previous) vehicle route network (e.g., using a map stored in a local memory onboard the autonomous vehicle, etc.). The policy may be selected / extracted from a set of pre-configured policies, may be dynamically generated using the vehicle route network, may be dynamically curated (e.g., based on the agent's classification labels, based on instantaneous states or state history, based on maps, etc.), and / or may be determined in other ways.
[0060] In some aspects, for example, S200 includes filtering (e.g., removing, eliminating from further consideration, lowering weighting, etc.) policies from the full set of all available policies for a particular agent based on a classification label option attributed to the agent. For example, if it is determined that a particular agent is unlikely to be a pedestrian, a pedestrian-only policy (e.g., walk on the sidewalk) may be eliminated from further consideration. In additional or alternative aspects, S200 includes filtering policies from the full set of all available policies for a particular agent based on the location of the agent. For example, if the agent is located in a right turn lane, a policy that prescribes a left turn may be eliminated from further consideration. However, additionally or alternatively, the same policies may be considered for all agents and / or the policies may be determined in other suitable manners.
[0061] As an example, policies governing right turns or lane changes can be considered for vehicles in the left turn lane. As a second example, unconventional, extralegal and / or illegal driving maneuvers can be considered as available policies for agents in the environment, which can improve the robustness of intent estimation in edge and corner case environmental scenarios.
[0062] S200 may optionally include determining a set of possible routes associated with each environmental agent, and each policy may be associated (e.g., paired, included) with one or more possible routes. The possible routes preferably represent potential trajectories of the environmental agent and may be determined based on any or all of the environmental agent's potential policies, current and / or past policies determined for the environmental agent, a set of road networks and / or maps (e.g., indicating the location of lanes, traffic signals, infrastructure, etc., or indicating traffic direction, etc.), and / or any other information.
[0063] The possible (candidate) routes may be any or all of the following: distance-constrained routes to a range of a pre-set distance (e.g., 100-500 meters, 100 meters, 200 meters, 300 meters, 400 meters, 500 meters, 500-1000 meters, over 1000 meters, etc.); time-constrained routes to a range that the environmental agent can travel within a pre-set threshold time (e.g., 5-30 seconds, less than 5 seconds, more than 30 seconds, etc.); routes constrained in other ways; unconstrained routes; and / or any combination.
[0064] In a preferred set of embodiments, each potential policy (also referred to as a candidate policy) of the environmental agent is paired with all possible routes that the environmental agent can take according to that policy, and each of these policy-route pairs is preferably considered in subsequent processes of method 200.
[0065] S200 may optionally include a step S220 of determining a set of potential ego policies (also referred to as candidate ego policies) for the ego agent based on a set of inputs, which serves to determine which ego policies the ego agent should consider executing. Each of them (and / or a prioritized subset) is preferably evaluated (e.g., in the MPDM module) as described below. The potential ego policies may be determined by any or all of the following: in the same or similar manner as described above for the environmental policies, in a different manner than described above for the environmental policies, based on a goal associated with the ego agent (e.g., progress toward a destination), and / or in any other manner. The ego policies may optionally be associated with a set of potential routes (e.g., to form a policy-route pair as described above). Similarly, the policies for the ego agent and the environmental agent may be the same (e.g., a set of high-level vehicle behaviors common to all agents in the environment), substantially similar, and / or different. For example, additional policies may be available to agents in the environment (e.g., different policies may be available to pedestrians than to autonomous agents; autonomous agents may not consider some policies (e.g., speeding through a stop sign) that may be considered in estimating intent and / or evaluating the feasibility of policies relative to other vehicles in the environment). However, any other suitable policies may be determined.
[0066] In a first aspect, S200 includes determining a set of policy and route pairs for each of the environmental agents tracked by the ego agent, the set of policy and route pairs including each of a set of policies available to the environmental agent (e.g., as determined based on a classification label associated with the environmental agent, as determined based on geometric information associated with the environmental agent, as determined based on a state estimate associated with the environmental agent, as determined based on a road network associated with the environmental agent, as determined by a pre-set list, etc.), and optionally determining a set of ego policies (and / or ego policy and route pairs) available for consideration.
[0067] In a first set of particular examples, the set of policies available to the environmental agent is filtered based on any or all of a set of inputs (and / or preprocessed inputs) determined in S100, such as, but not limited to, one or more classification labels available to the agent, the agent's position (e.g., compared to a map that prescribes a set of policies based on position), the agent's geometry (e.g., which lane the agent may be in), other state information of the agent (e.g., which direction the agent is heading based on azimuth), and / or any other information.
[0068] In a second set of specific examples, all available policies are considered for each agent.
[0069] S200 may additionally or alternatively include any other suitable processes.
[0070] 4.3 Methodology - Determining a Set of Scores Associated with a Set of Environmental Policies S300 Method 200 includes determining a set of scores associated with the set of environmental policies, which serves to enable evaluation of the set of environmental policies with respect to at least the feasibility associated with the environmental agent executing a particular policy (and / or policy-route pair), and optionally with respect to other criteria. This serves preferably to ultimately enable selection and execution of an optimal policy for the ego agent (e.g., in a subsequent process of method 200), whereby the scores associated with the environmental policies are taken into account and / or otherwise used to influence this decision-making process. Additionally or alternatively, S300 may perform any other suitable functions.
[0071] The set of scores is preferably determined at least in part by a set of forward simulations, which collectively function to simulate predicted outcomes when one or more agents execute various policy options (e.g., policy and route pairs). Additionally or alternatively, S300 may include performing backward simulations, combinations of simulations, any other simulations, and / or no simulations. In certain examples, S300 simulates the policies of each environment agent individually (e.g., in isolation from other agents in the environment). However, the scores may be determined in other manners.
[0072] The score associated with an environmental policy preferably corresponds to the probability that the (candidate) policy will be implemented by the environmental agent, which is effectively integrated into the weight associated with the policy and / or policy-route pair. Alternatively, the score can correspond to the error associated with the policy being implemented, the likelihood that the candidate policy represents the agent's intent, and / or include one or more other types of metrics.
[0073] In some aspects, S300 may include a step S310 of determining a feasibility score associated with each policy; optionally, a step S320 of determining a past score associated with each policy; optionally, a step S330 of determining one or more previous scores associated with each policy; and / or any other suitable substeps / elements.
[0074] Step S310 of determining a feasibility score (also equivalently referred to herein as a future feasibility score) associated with each environmental policy serves to determine the feasibility of the environmental agent executing that environmental policy. The feasibility score is preferably determined based on the trajectory of the environmental agent obtained by simulating a particular environmental policy and / or environmental policy-route pair (e.g., as described above). This trajectory can then be analyzed to see whether and / or to what extent it matches a reasonable / feasible trajectory for the agent taking that route. For example, if simulating a policy-route pair results in the simulated environmental agent going off the road (e.g., driving off the road to make a sharp turn in accordance with a turn policy) or driving in an unexpected and / or infeasible manner (e.g., colliding with another agent or infrastructure, accelerating / decelerating in an extreme manner, driving in the wrong direction, and / or driving in an otherwise unexpected direction) in order to conform to the policy and / or the paired route, then a low feasibility score can be determined for that particular agent and policy (and / or policy-route pair). As a second example, the feasibility score may be a characterization of one or more (predicted) control effort parameters, such as throttle / acceleration, braking / deceleration, and / or steering adjustments, to achieve a simulated trajectory over a future forecast period (e.g., planning period, simulated distance, etc.; an example is shown in FIG. 7). As an example, policies that result in control inputs that exceed the physical and / or dynamic limits of the environmental agent (e.g., exceeding levels of braking considered feasible for an average agent of that classification type, exceeding levels of steering considered feasible for an average agent of that classification type) are considered infeasible.
[0075] The feasibility score is preferably determined based on one or more feasibility parameters, which serve to quantify the feasibility associated with the execution of a particular policy (and / or policy-route pair). The feasibility parameters may include control effort parameters (e.g., lateral control parameters, longitudinal control parameters, etc.), one or more geographic parameters, one or more error accumulation parameters, one or more reference trajectory deviation parameters (and / or their derivatives / rates), lateral parameters, longitudinal parameters, spatio-temporal parameters, ride comfort parameters, risk estimation parameters, vehicle wear parameters, position parameters (e.g., distance to other agents, distance to infrastructure objects, etc.), agent motion parameters (e.g., maximum speed, minimum speed, change in speed, maximum acceleration, change in acceleration, etc.), agent orientation parameters (e.g., change in azimuth, reversal of agent heading, etc.), and / or any other suitable parameters. In a preferred set of embodiments, in the case of a vehicle agent, the one or more feasibility parameters include one or more geographic parameters indicative of how much the agent is predicted to deviate from the reference trajectory associated with the policy. In a preferred particular example, for example, the feasibility parameters include a lateral error relative to the centerline of the lane the agent is traveling in (and / or a threshold lateral range therefrom, such as lane width / boundary line), with the larger the lateral error (indicative of the agent's drift in executing the policy and / or policy-route pair), the lower the feasibility (score) of the policy selection. Further, additionally or alternatively, any or all of the feasibility parameters may take into account the goals (e.g., predicted goals) of the environmental agents, including, for example, but not limited to, proceeding towards a known or predicted destination, safety, compliance with traffic rules, and / or other goals. One or more goals may be specific to an individual environmental agent, shared among any number of agents (e.g., among all vehicle agents, among all pedestrian agents, etc.), and / or in any combination.
[0076] The feasibility parameters are preferably determined based on a forward simulation of the environmental agents, where the simulation is performed based on at least a current set of inputs associated with the environmental agents (e.g., for each agent ID, the current state estimates and / or current classification labels, etc.), but may additionally or alternatively be performed using historical information associated with the environmental agents (e.g., past state estimates and classification labels up to a pre-set time prior to the current time, all past state estimates and classification labels for one or more agents, etc.) and / or using any other simulation (e.g., backward simulation).
[0077] A simulation preferably generates a single (future) feasibility score for an environmental policy, although each simulation may additionally or alternatively generate multiple feasibility scores for each environmental agent and associated policy, with each of the multiple feasibility scores associated with a particular time step in the forward simulation (e.g., associated with one second increments in a four second simulation, thereby allowing feasibility to be assessed at a fine grain and / or over multiple future time periods).
[0078] In a first set of aspects, a feasibility score may be determined in S310 based on predictive control effort (e.g., total, maximum instantaneous effort, etc.) for a forward simulation of the environmental policy. The predictive control effort may be advantageously utilized to avoid overestimating the integral impact of small actuation errors over time (e.g., where small deviations from a reference trajectory may not meaningfully affect the vehicle's net behavior; e.g., small orientation and turning radius differences during a cornering maneuver may result in relatively large lateral and / or longitudinal compound error accumulations, but the resulting impact on the vehicle behavior and control inputs may be minimal). For example, if the environmental policy may include or be associated with a particular controller (e.g., one that models a vehicle behavior attempting to track a fragment of an intended route; e.g., PID controllers for throttle, braking, and steering, or conversely, PID controllers for heading and net acceleration, etc.), the control effort may be the output of the controller when attempting to track the trajectory given the current vehicle state. The control effort parameters may include lateral effort parameters (e.g., heading / steering adjustments), longitudinal control parameters (e.g., throttle / brakes; net acceleration or speed change), and / or aggregate control parameters (e.g., heuristic functions that combine lateral and longitudinal parameters, etc.). As an example, a deterministic controller (e.g., a feedback controller) may generate a deterministic output of control effort parameters for a forward simulation (e.g., when the forward simulation is performed independently of other agents).Thus, it will be appreciated that while feasibility scoring may be considered a feedforward feasibility estimation and / or a feedforward simulation (e.g., since it may not depend on other external factors), it may additionally or alternatively be considered a feedback simulation because the simulation may depend on a feedback controller based on the simulated performance (and / or may utilize feedback terms in the simulation), and / or may be otherwise appropriately referred to (e.g., as a forward simulation). In certain examples, the feasibility score may be a feedforward estimate of predicted control effort relative to a reference trajectory (e.g., a preset distance and / or time, etc.).
[0079] In a second set of aspects, the feasibility score may be determined in S310 based on lateral error accumulation (e.g., relative to the centerline of the lane) over the predicted rollout (e.g., vehicle movement in a forward simulation), but may additionally or alternatively be determined based on the time derivative of the lateral error accumulation (e.g., the rate of lateral error accumulation with respect to time), the time derivative of the total error accumulation (e.g., longitudinal and / or lateral errors), and / or any other suitable error / deviation accumulation. For example, the feasibility score and / or likelihood of the policy intent may be estimated using the Mahalanobis distance between the predicted trajectory and the policy's reference trajectory. As a second example, the feasibility score may be estimated based on the Euclidean distance between the trajectories (e.g., the time derivative of the Euclidean distance as a function of time, etc.).
[0080] In a third set of aspects, S310 includes calculating, for each simulation, a feasibility score at each time step for each environmental policy (and / or policy-route pair) selected for each environmental agent. In a first set of particular examples, the simulation is performed based only on a set of current inputs associated with the environmental agents, such as each environmental agent's current (and / or most recent) state estimate, each environmental agent's current classification label, and / or any other information. In a second set of particular examples, the simulation is performed based on the set of current inputs and on past information associated with the environmental agents (e.g., information from a pre-defined time period prior to the current time period).
[0081] The feasibility score may be determined for a future time period for which the agent's movement is predicted / simulated, which may be a pre-set future prediction period (e.g., 3 seconds, 5 seconds, etc.), a period associated with a pre-set distance traveled (e.g., 100 m), a dynamically sized window (e.g., as a function of speed, context, etc.), and / or any other suitable time period or prediction window.
[0082] However, any other suitable (future) feasibility score may be derived.
[0083] S300 may optionally further include a step S320 of determining a historical score associated with each policy, which serves to determine, for each tracked environmental agent, how closely a potential policy matches the environmental agent's previous driving behavior. This may indicate, for example, how likely it is that the environmental agent will execute a particular policy based on how it has driven before. For example, if the environmental agent is accelerating significantly from its normal speed according to its past history, the agent may be more likely to be driving straight or attempting to change lanes rather than preparing to turn. In a preferred embodiment, S320 is performed when S310 is calculated based only on a current set of inputs (not historical information), and the historical and future scores are calculated separately and then aggregated (e.g., as described below). Additionally or alternatively, S320 may be performed in any embodiment of S310, S300 may be performed in the absence of S320, and / or method 200 may be performed in other suitable manners.
[0084] In a preferred set of embodiments, S320 is executed for a predetermined period of time in the past, e.g., a preset number of seconds (e.g., 1-10 seconds, 1-5 seconds, 2-6 seconds, 5-10 seconds, more than 10 seconds, etc.), which can serve to reduce the computational burden associated with processing each policy, focus only on the most relevant recent driving behaviors, and / or provide any other advantage.
[0085] The historical score may be derived from a direct comparison of the observed past vehicle states (e.g., past vehicle trajectories / motions) to a reference trajectory associated with the policy (e.g., an example is shown in FIG. 5). The historical score may be determined by a set of heuristics, a scoring function, a statistical comparison, a set of vector comparisons (e.g., spatial, spatio-temporal; Mahalanobis distance), and / or any other suitable matching / comparison technique based on the observed past vehicle states. Alternatively, the historical score may be evaluated as the difference between the (estimated) past control effort through the observed / past state history and the control effort associated with the reference trajectory. For example, if a policy has a past reference trajectory that is linear and does not include changes in vehicle actuation / control (i.e., going straight while maintaining speed / throttle), observed angles / linear accelerations associated with steering and / or braking / throttle changes may indicate that the policy does not closely match the past observations and / or state history, resulting in a low historical score / feasibility.
[0086] The historical scores are preferably determined separately from the (future) feasibility scores, but may alternatively be aggregated and / or combined with the future feasibility scores, integrated into the determination of the feasibility scores, and / or otherwise determined. Additionally or alternatively, any or all of this historical information may be taken into account in S310, not in S300, and / or any combination thereof.
[0087] In one set of embodiments (one example of which is shown in FIG. 8 ), determining a respective set of scores for each policy candidate may include determining a first (backward-looking) score (e.g., historical score) based on a comparison between the policy candidate and the state history of the agent, and determining a second score (e.g., feasibility score) through a future simulation of the policy candidate.
[0088] S300 may optionally additionally or alternatively include determining one or more prior scores, which function to take into account one or more other probabilities (e.g., confidence values) associated with the simulation and derived at least in part based on the set of inputs (and / or the set of preprocessed inputs). For example, in determining a classification label for an agent (e.g., pedestrian, motorized vehicle, bicycle, etc.), the classification label may include multiple classification labels with associated probabilities (e.g., 95% as the probability that the agent is a motorized vehicle and 5% as the probability that the agent is a pedestrian). Thus, if one of the possible policies is a policy dedicated to motorized vehicles, the probability that the agent is a motorized vehicle may be taken into account in determining the overall score in S300. The one or more prior scores may additionally or alternatively take into account how likely the agent is to be located in a particular lane and / or any other confidence values associated with the set of inputs.
[0089] In some aspects, one or more previous scores can be used to weight / prioritize policies based on previously established relative probabilities, as may be predetermined for an agent classification or road context. For example, one or more previous scores can be used to encode probabilities / likelihoods associated with various road maneuvers (e.g., running a stop sign, etc.). As an example, in observing road regulations, the probability that a road agent will stop at a stop sign is generally very high (e.g., over 99%) and may be higher than the probability that the agent will run through a stop sign (e.g., less than 1%). Thus, even if both policies have high past scores and high future feasibility (e.g., essentially as before a vehicle normally brakes), it may be advantageous to prioritize the more likely of two policies depending on one or more previous scores, thereby determining that the agent is more likely to intend to stop at a stop sign depending on previous information / context. As a second example, the probability that an agent classified as a car will execute a policy to stop at a stop sign may be higher than the probability that an agent classified as a bicycle will execute the same policy. Thus, one or more previous scores / weights may be used to score policies at S300 based on previous information / context.
[0090] The one or more prior scores may be dynamically determined for each agent based on any suitable information / input (e.g., classification labels / probabilities) received during S100, may be predetermined for various policies, and / or may be determined in other ways. Alternatively, prior scores may be implemented in other ways or may not be relied upon.
[0091] S300 may optionally additionally or alternatively include calculating any other scores or combinations of scores for use in evaluating the set of policies. S300 may optionally include aggregating (e.g., multiplying, averaging, calculating a median, etc.) any or all of the scores calculated in S300, for example aggregating any or all of the future feasibility scores, past scores, and previous scores (e.g., an example is shown in FIG. 6). Alternatively, a single score may be calculated in S300, the scores may be evaluated independently in subsequent processes of method 200, and / or one or more scores may be determined in other ways.
[0092] In a first set of examples, S200 may determine a set of candidate policies for an agent approaching an intersection with a stop sign, including stopping at the stop sign, turning right without stopping, turning left without stopping, and going straight without stopping, and S300 may generate a past score, a feasibility score, and optionally one or more previous scores for each policy. In some cases, each of the policies may correspond to a high feasibility score, since the vehicle may be controlled to execute each of the policies (an example is shown in FIG. 4A). The one or more previous scores may indicate that stopping at the stop sign is the most likely policy, and / or all potential policies may be considered during ego planning based on their respective scores. If the environmental agent decelerates rapidly as it approaches the intersection (e.g., an example is shown in FIG. 4B), the feasibility score associated with the policy of stopping at the stop sign may indeed indicate that this policy is the most feasible. If instead the agent maintains course at high speed (an example is shown in FIG. 4C), stopping becomes infeasible and another policy (e.g., indicating that the vehicle intends to pass through the intersection without stopping) may have the highest probability / score. If the environmental agent moves off-center as it approaches the intersection (an example is shown in FIG. 4D), feasibility and / or historical scores may indicate that the policy with the highest probability (e.g., based on a comparison of simulated trajectories and / or state histories) is for the vehicle to turn (an example is shown in FIG. 4E). In this set of examples, each policy and / or a subset of policies may be evaluated in S400 and / or S500 for ego-vehicle planning and control during each time step based on their respective scores.
[0093] 4.3 Method - Evaluating a Set of Policies Based on a Set of Scores S400 Method 200 may optionally include a step S400 of evaluating a set of policies based on the set of scores, which serves to enable selection of an optimal policy for the ego agent to implement. Additionally or alternatively, S400 may perform any other suitable function and / or the scores may be evaluated in other suitable manners.
[0094] S400 preferably includes selecting an optimal ego policy for the ego agent to implement, but may additionally or alternatively include assigning a most feasible policy to any or all of the set of environmental agents (e.g., based on the aggregate score calculated above), eliminating a policy from consideration for the environmental agent (e.g., based on the score calculated in S300 being below a threshold), and / or performing any other action.
[0095] S400 is preferably performed based on a set of simulations performed in S300, which allows an overall ego score, such as a cost function and / or loss function, to be calculated for each potential ego policy associated with the ego agent (i.e., for each simulation), and the ego policy associated with the lowest cost function can be selected. Alternatively, S400 can be performed with additional sets of simulations (e.g., performed after the simulations of S300; by MPDM) and / or any combination of simulations.
[0096] In a preferred set of embodiments, S400 executes according to an MPDM module, such as any or all of those described in U.S. Application No. 16 / 514,624, filed July 17, 2019, and U.S. Application No. 17 / 365,538, filed July 1, 2021, each of which is incorporated by reference in its entirety. As an example, S300 can include a set of parallelized independent feasibility simulations (e.g., ones that independently analyze the feasibility of each agent's policy; they can be independent and / or separate from the ego vehicle's policy), and S400 can include multiple joint scenario simulations (e.g., ones that examine interactions between agents in an environment under joint policy scenarios; this includes evaluation of the ego vehicle policy).
[0097] In one aspect, an ego policy can be selected based on multiple simulations for each of the ego policies in the set of ego policies, where each simulation evaluates the ego policy against an environmental scenario including a respective policy for each agent in the environment sampled from the respective set of environmental policies based on a score.
[0098] In a first set of aspects, S400 may include evaluating the ego policy against a set of environmental policies, the evaluation being weighted based on an aggregate score of each environmental policy in the set of environmental policies.
[0099] In a second set of aspects, not exclusive to the first set, S400 may include evaluating the set of ego policies by performing multiple simulations of each ego policy against the set of environmental policies based on the feasibility score and the historical score. In one example, evaluating the set of ego policies may include simulating vehicle policies based on environmental policy scenarios sampled from the set of environmental policies based on the aggregate score.
[0100] Evaluating the set of policies based on the set of scores may include statistical evaluation, heuristic scoring, energy maximization (or cost minimization) techniques, decision trees, Bayesian optimization, ML-based evaluation, and / or any other suitable evaluation / selection. For example, an optimal policy may be selected using the MPDM system based on the set of scores for each policy generated during S300.
[0101] However, the set of policies may be evaluated in other manners and / or the ego vehicle policies may be appropriately selected / determined in other manners.
[0102] 4.4 Other Processes Method 200 may optionally include any other process, such as step S500 of operating (e.g., controlling) the ego agent according to the selected ego policy, determining a trajectory for the ego agent based on the selected ego policy, and / or operating the ego agent according to the trajectory, and / or any other process. For example, the selected ego vehicle policy may be selected / elected and implemented by an autonomous vehicle controller of the computing system.
[0103] Although omitted for simplicity, preferred embodiments include any combinations and permutations of the various system components and various method processes, which may be performed sequentially or simultaneously, in any suitable order.
[0104] Alternative embodiments implement the above methods and / or processing modules in a non-transitory computer-readable medium that stores computer-readable instructions. The instructions may be executed by computer-executable components integrated with the computer-readable medium and / or processing system. The computer-readable medium may include any suitable computer-readable medium, such as RAM, ROM, flash memory, EEPROM, optical devices (CD or DVD), hard drives, floppy drives, non-transitory computer-readable media, or any suitable device. The computer-executable components may include a computing system and / or processing system (e.g., including one or more co-located or distributed, remote or local processors) connected to the non-transitory computer-readable medium, such as a CPU, GPU, TPUS, microprocessor, or ASIC, although the instructions may alternatively or additionally be executed by any suitable dedicated hardware device.
[0105] Embodiments of the systems and / or methods may include any combination and permutation of the various system components and various method processes, and one or more instances of the methods and / or processes described herein may be performed asynchronously (e.g., serially), contemporaneously (e.g., simultaneously, in parallel, etc.), or in any other suitable order and / or by and / or using one or more instances of the systems, elements and / or entities described herein. The following system and / or method components and / or processes may be used with, in addition to, instead of, or otherwise integrated with all or a portion of the systems and / or methods disclosed in the applications listed above, each of which is incorporated by reference in its entirety.
[0106] Those skilled in the art will recognize from the foregoing detailed description and drawings and claims that modifications and variations can be made to the preferred embodiments of the invention without departing from the scope of the invention as defined in the following claims.
Claims
1. 1. A method of operation of an autonomous vehicle within an environment, comprising: - determining a set of inputs using a sensor suite of an autonomous vehicle, the set of inputs including an environmental agent instance identifier and a state history associated with the environmental agent instance identifier; determining a set of environmental policies for said environmental agent instance identifier based on said set of inputs; For each environmental policy of the set of environmental policies: - determining a historical score by comparing the state history of the environmental agent with a reference trajectory associated with the environmental policy; - determining a feasibility score by forward simulation of the environmental policy; determining an ego policy by evaluating a set of ego policies for the autonomous vehicle against the set of environmental policies based on the feasibility score and the historical scores; and operating the autonomous vehicle based on the ego policy.
2. 10. The method of claim 1 , The method further comprising aggregating the historical scores and the feasibility scores into an aggregate score, wherein an evaluation of the set of vehicle policies against the set of environmental policies is weighted based on the aggregate score of each environmental policy in the set of environmental policies.
3. 3. The method of claim 2, The method, wherein evaluating the set of vehicle policies includes simulating vehicle policies based on environmental policy scenarios sampled from the set of environmental policies based on the aggregate score.
4. 10. The method of claim 1 , The method, wherein evaluating the set of ego policies comprises running multiple simulations of each ego policy against the set of environmental policies based on the feasibility scores and the historical scores.
5. 10. The method of claim 1 , The method, wherein the feasibility score characterizes a predictive control effort associated with a forward simulation of the environmental policy.
6. 6. The method of claim 5, The method, wherein the predictive control effort includes one or more of a lateral control parameter and a longitudinal control parameter.
7. 10. The method of claim 1 , The method of claim 1, wherein the set of inputs further includes a classification associated with the environmental agent instance identifier, and the set of environmental policies is based on the classification.
8. 8. The method of claim 7, The method further comprising determining a supplemental score for each environmental policy based on the classification and the state history, wherein the ego policy is determined further based on the supplemental score.
9. 10. The method of claim 1 , The method, wherein determining a set of environmental policies for the environmental agent instance identifier comprises extracting a reference trajectory from a previous route network.
10. 10. The method of claim 1 , The method, wherein the forward simulation of each environmental policy includes a closed-loop simulation of a deterministic controller associated with the environmental policy.
11. 1. A method of operation of an autonomous vehicle relative to an agent in an environment of the autonomous vehicle, comprising: - tracking a set of agents within an environment based on vehicle sensor data, the step including determining a state history of each agent of the set of agents; determining a set of candidate policies for each agent; Deriving a respective set of scores for each policy candidate, determining a first score based on a comparison of the agent's candidate policies to the state history; and - determining a second score by forward simulation of the candidate policy; and operating the autonomous vehicle based on the respective set of scores for each candidate policy.
12. 12. The method of claim 11, The method, further comprising determining a classification for each agent in the set of agents, wherein the set of candidate policies is determined based on the classification.
13. 12. The method of claim 11, The method, wherein operating the autonomous vehicle based on a respective set of scores for each candidate policy includes operating the vehicle based on a plurality of ego-vehicle simulations.
14. 14. The method of claim 13, the method, wherein each ego vehicle simulation of the plurality of ego vehicle simulations is based on an environment scenario including a selected policy for each agent in the environment, and a policy for each agent is selected from among the set of candidate policies based on the set of scores.
15. 14. The method of claim 13, The method, wherein the forward simulation of each candidate policy is independent, and each ego vehicle simulation of the plurality of ego vehicle simulations includes a coupled simulation of the ego vehicle and each agent in the environment.
16. 12. The method of claim 11, The method, wherein each candidate policy comprises a closed-loop deterministic controller.
17. 12. The method of claim 11, A method characterized in that each set of scores for each candidate policy collectively characterizes the likelihood of the policy over both the past observation period and the future prediction period.
18. 18. The method of claim 17, The method of claim 1, wherein the first score characterizes a historical comparison over the past observation period and the second score characterizes a forecast feasibility over the future forecast period.
19. 20. The method of claim 18, The method, wherein the future forecast period is associated with a preset distance traveled or a preset time period.
20. 12. The method of claim 11, 1. A method comprising: using a multi-policy decision-making (MPDM) system to determine an ego policy for an autonomous vehicle based on a respective set of scores for each candidate policy, and operating the autonomous vehicle comprises executing the ego policy.
Citation Information
Patent Citations
Collision damage reduction method and collision damage reduction system
JP2016005294A
How to make driving and autonomous driving easier
JP2019512824A
Method and apparatus for constructing informative outcomes to guide multi-policy decision making
JP2020511721A
Self-learning vehicle performance optimization
US20210300412A1
