Vehicle trajectory tree search for off-route driving operations

Abstract

Techniques are discussed herein for generating trajectories for controlling motion and / or other behavior of a vehicle in a driving environment. In particular, techniques are described herein for determining a trajectory for a vehicle to join a travel route from an initial vehicle state that deviates from a travel route structure using a tree search. The vehicle computing system may determine various candidate trajectories, including trajectories based on off-route inertial reference frames, additional trajectories based on route structures, disturbed trajectories, and the like. The set of candidate trajectories may be optimized and / or filtered based on objects in the environment, and corresponding candidate actions may be used to generate a search tree between a vehicle state deviating from the route and a target state on the route. Costs associated with the candidate actions may be iteratively evaluated to determine a minimum cost traversal for a tree representing a control trajectory that allows the vehicle to join the travel route structure.

Description

[0001] Cross - Reference to Related Applications

[0002] This application claims the priority of U.S. Patent Application No. 18 / 072,015, titled "Vehicle Trajectory Tree Search for Off - Route Driving Operations", filed on November 30, 2022, the entire content of which is incorporated herein by reference for all purposes. Background Art

[0003] Autonomous driving can benefit from a computing system capable of determining a driving path and navigating along a route from an initial position towards a destination. For example, autonomous and semi - autonomous vehicles can utilize systems and components to traverse a driving environment that includes other objects such as moving or stationary vehicles (autonomous or otherwise), pedestrians, buildings, etc. When traversing such an environment, a vehicle can determine a trajectory based on sensor data from the vehicle's perception system and map data of the environment. A planning system within an autonomous vehicle can include a computing system configured to navigate from an initial position to a destination along a specified route according to a route - based reference frame, such as the Frenet reference frame. The route - based reference frame can reduce the computational complexity associated with the control of an autonomous vehicle in the environment. However, a route - based reference system may limit the operation of an autonomous vehicle to a specified route associated with the route - based reference frame. Such limitations may restrict an autonomous vehicle from performing certain operations, such as parking, pulling over, reversing, etc. Brief Description of the Drawings

[0004] The detailed description is described with reference to the accompanying drawings. In the drawings, the left - most digit of a reference numeral identifies the figure in which the reference numeral first appears. The same reference numerals are used in different drawings to indicate similar or identical components or features.

[0005] Figure 1 An autonomous vehicle and an example scenario according to one or more examples of the present disclosure are shown, where a trajectory is determined to control the autonomous vehicle from an initial off - route state to a target state on the route in a driving environment.

[0006] Figure 2 It is a diagrammatic flow chart showing an example method for determining candidate trajectories and using a decision search tree to control a vehicle to join a driving route according to one or more examples of the present disclosure.

[0007] Figure 3 It is a block diagram showing an example system for determining a trajectory for a vehicle to join a driving route from an initial off - route vehicle state using a planning component according to one or more examples of the present disclosure.

[0008] Figures 4A - 4FIllustrates an example of determining and selecting candidate trajectories for controlling a vehicle from an off-route vehicle state to a target vehicle state on a driving route according to one or more examples of the present disclosure.

[0009] Figure 5A Illustrates an example representation of a set of candidate actions generated for a node in a tree search according to one or more examples of the present disclosure.

[0010] Figure 5B Illustrates an example representation of multiple sets of candidate actions associated with different nodes in a tree search to determine a trajectory according to one or more examples of the present disclosure.

[0011] Figures 6A - 6C Illustrates a diagrammatic flow chart of an example method for generating a trajectory for controlling a vehicle from an off-route vehicle state to a target vehicle state on a driving route, including evaluating various candidate actions according to an inertia-based reference system and / or a route-based reference system.

[0012] Figure 7 Is a block diagram showing an example system for implementing the various techniques described herein, the system including an autonomous vehicle and a separate computing device.

[0013] Figure 8 Is a flow chart showing an example technique for determining and following a trajectory to control a vehicle from an off-route vehicle state to a target vehicle state on a driving route according to one or more examples of the present disclosure. Detailed Description

[0014] This application relates to determining a driving trajectory of a vehicle between different states (which may include, for example, position, speed, acceleration, steering angle, yaw rate, etc.), and / or switching between control mechanisms at least partially based on different coordinate systems in a driving environment. In particular, techniques are described herein for determining a trajectory for a vehicle to join a driving route from an initial off-route state (e.g., a state where the vehicle movement is independent of map elements (e.g., lanes), reference lines, or other references for constraining the vehicle movement). For example, when parking or exiting a garage, driving off or onto a road, reversing, or performing other uncommon driving operations, or when driving in an unstructured environment such as a parking lot, driveway, or construction area, the vehicle may be in an off-route state.

[0015] As described in more detail below, a planning component of a vehicle can determine multiple candidate trajectories to move the vehicle from a current off-route vehicle state (e.g., a vehicle state without lane constraints) to a target state (e.g., a vehicle state with lane constraints) in a driving lane of a route associated with the vehicle. The various candidate trajectories can include, but are not limited to, inertial trajectories generated based on an inertial reference system (e.g., using mathematical functions), route trajectories determined based on the structure of the driving route, previous control trajectories of the vehicle, and / or perturbed trajectories of various other candidate trajectories that may include lateral movement and / or speed scaling. The set of trajectories can be optimized and / or filtered based on objects in the driving environment, and a decision search tree can be generated between the current off-route vehicle state and the target state on the driving route using the resulting modified candidate trajectories. The cost associated with candidate actions can be iteratively evaluated to determine the minimum-cost traversal of the tree, which represents the control trajectory that allows the vehicle to join the driving route.

[0016] While some examples in this document describe determining and evaluating candidate trajectories to determine a control trajectory of a vehicle, in other examples, similar or the same techniques can be used to determine and evaluate candidate driving paths and to determine a control driving path of the vehicle. As described in more detail below, the trajectories used in this document can refer to a series of spatio-temporal vehicle states. In contrast, a path can refer to a series of vehicle states without associated time information. While the various examples described in this document may involve trajectories (e.g., candidate trajectories, optimal trajectories, etc.), these examples can include determining a trajectory and / or a path for the vehicle to join a driving route from an initial off-route vehicle state.

[0017] The various examples and techniques described in this document can be implemented in a variety of ways to improve the operation of autonomous vehicles and the operation of computing systems. These techniques can improve the functionality, safety, and efficiency of autonomous and semi-autonomous vehicles operating in a real driving environment by determining improved driving trajectories (and / or driving paths) in the environment while considering passenger and vehicle safety, driving efficiency, the kinematic feasibility of the vehicle's movement, and various other cost-based metrics.

[0018] When an autonomous vehicle is driving on a route, the vehicle can navigate along a specified driving route (e.g., a route corresponding to a series of roads, driving lanes, etc.), can navigate between waypoints related to the specified route, or can otherwise plan a trajectory based on certain constraints such as lane markings, reference lines, or other references. The planning component can control the vehicle to drive along the specified route at least partially according to a route-based reference system (e.g., Frenet coordinate system, curvilinear coordinate system, etc.). For example, the planning component can receive a request to navigate the vehicle to a specific location and can determine one or more driving routes based on, for example, environmental map data, the current position of the vehicle, the destination position, etc. The planning component can drive the vehicle along the determined route to the destination position according to the route-based reference system. For example, each road and / or driving lane along the route can include a series of target states (e.g., lane reference states) that define the desired trajectory along the road or driving lane. When driving on the driving route, the planning component can use a controller to track the current state of the vehicle relative to the lane reference state and adjust the vehicle navigation (e.g., speed, steering angle, etc.) based on the lane reference state.

[0019] However, in some cases, an autonomous vehicle may need to operate off-route, i.e., deviate from the specified driving route associated with the vehicle. For example, when the vehicle is parking or pulling out of a station, has pulled over to the side of the road, or is located in a lane or off-road position, the vehicle may no longer be driving on the specified driving route. For example, in the case of a parking lot, there may be no clear driving lanes in the area, and thus, there is no lane reference to track the vehicle movement. In some cases, even when the vehicle is in the driving lane of its specified driving route, the vehicle may be in an off-route state. For example, if the vehicle is within the route driving lane but is reversing, turning around, and / or the vehicle heading is significantly deviated from the driving direction of the driving lane, the vehicle may be considered off-route relative to the lane reference trajectory of the driving lane. When the vehicle is in an off-route state, the planning component can use an alternative inertia-based reference system (instead of the route-based reference system) to control the vehicle. In some examples, the inertia-based reference system can include a body-centered reference system, such as a reference system based on the vehicle X-Y-Z reference. For example, the planning component can detect an object in the environment and can determine the position associated with the object based on the lateral distance (X) and the distance in front of and behind the vehicle (Y). In other examples, (X) can represent the longitudinal distance and (Y) can represent the lateral distance. Of course, other dimensions can also be considered. When using the inertia-based reference system, the planning component can control the vehicle by determining the off-route trajectory between various off-route positions (e.g., parking position, curbside pick-up position, etc.).

[0020] For these reasons, autonomous vehicles may face many additional technical challenges when performing off-route driving operations that may not arise when performing on-route operations. For example, a driving lane on a designated driving route may have a reference (or baseline) trajectory that is pre-determined and stored in the map data used by the vehicle for that driving lane. The reference trajectory of a driving lane may include, for example, a series of vehicle states that represent the position of the center of the driving lane / road and represent the heading and speed corresponding to the lane direction and the current speed limit of the road. For route navigation, the reference trajectory can be used to determine an optimal trajectory, determine a driving route around an object, and measure progress along the route. In contrast, for off-route driving operations, there is no lane reference trajectory available for the autonomous vehicle to use. Therefore, when performing off-route driving operations, it may be difficult for the autonomous vehicle to determine an optimal control trajectory and / or measure progress towards a goal or destination. Additionally, some autonomous vehicles may maintain two separate reference systems, each capable of computing a trajectory, such as a route-based reference system and a separate inertia-based reference system. In such cases, it may be difficult for the autonomous vehicle to determine when to switch between reference systems (e.g., at or near a road boundary) and also to know which reference system can provide the optimal trajectory for performing various driving operations.

[0021] To address these technical challenges and improve vehicle safety and efficiency in such situations, the techniques described herein involve determining a trajectory and controlling an autonomous vehicle to move from an off-route vehicle state to a goal state within the route structure of a designated driving route associated with the vehicle. To perform such operations, a planning component can first determine a goal state within the route structure of the designated driving route associated with the vehicle. In some examples, the planning component can determine a specific goal state in the driving lane reference trajectory based on any combination of the lateral and / or longitudinal distance between the vehicle and various goal states in the lane reference trajectory, the vehicle's current heading relative to the driving direction of the driving lane, the vehicle's current speed, and / or various characteristics or capabilities of the vehicle (e.g., the vehicle's turning radius and maximum steering angle). In some examples, the planning component 112 can determine multiple goal states associated with different driving lanes where the vehicle may join the route. For example, examples of various techniques for determining the goal state of an off-route vehicle in a lane reference trajectory can be found in U.S. Patent Application No. 16 / 732,122, filed on December 31, 2019, entitled "Vehicle Control to Join a Route", the entire content of which is incorporated herein by reference for all purposes.

[0022] After determining which route target state an autonomous vehicle will navigate to during a driving operation, a planning component can determine multiple candidate trajectories to control the vehicle from its current off-route state to the target state. In these examples, a "state" or "vehicle state" can include geometric state data (e.g., position, pose, heading, yaw, steering angle, etc.) and motion data of the vehicle (e.g., speed, acceleration, yaw rate, steering angle rate, etc.). A "trajectory" can refer to a series of states that a vehicle experiences from a starting state to an ending state. For example, a trajectory can be defined as a series of spatio-temporal states, where each state is specified by any combination of an x-position, a y-position, yaw, yaw rate, steering angle, steering angle rate, speed, and / or acceleration, etc. The vehicle can take any number of possible trajectories from the starting state to the ending state (or target state), including different positions, steering angles, speeds, and / or accelerations at different intermediate states in the trajectory.

[0023] In some examples, the planning component can determine candidate trajectories according to an inertia-based reference system. For example, in an inertia-based reference system, mathematical functions (e.g., cubic splines, Bezier curves, a series of clothoid curves, and / or any other closed-boundary condition solution) can be used to determine a reference (or baseline) trajectory between the current vehicle state and the target state. In some examples, a closed-boundary condition solution can include multiple free variables, the number of which is the same as or close to the number of constraints associated therewith. In such examples, a closed-boundary condition solution can produce a single solution (e.g., a single reference trajectory).

[0024] Additionally or alternatively, the planning component can determine separate candidate trajectories according to a route-based reference system. For example, the planning component can determine a route-based reference trajectory according to the lane reference trajectory state at the position on the driving lane that is closest to the current state of the vehicle. Then, the planning component can modify the lane reference trajectory so that the vehicle "tracks" (e.g., attempts to converge to) the lane reference trajectory. For example, the planning component can modify the lateral offset, speed, steering angle, etc. of the lane reference trajectory state to a route-based reference trajectory that can cause the vehicle to converge to the target state.

[0025] In some examples, the planning component can determine a set of candidate trajectories, including one or more trajectories generated using an inertia-based reference system and one or more additional trajectories generated using a route-based reference system. In other examples, the planning component can generate trajectories based on only one of these reference systems and exclude the other. For example, at a certain moment during a driving operation, the planning component can determine that the vehicle is in a driving lane but has not reached the target state. For example, when all four corners of the vehicle are on the road surface of the driving lane / road and the heading of the vehicle is within 90 degrees of the direction of the driving lane, the planning component can use heuristics to determine that the vehicle is in the driving lane. In this example, at this moment during the driving operation, the planning component can determine to generate one or more trajectories using the route-based reference system, but may not generate any trajectories using the inertia-based reference system.

[0026] As another example, the planning component can use another heuristic to determine that the distance of the vehicle from the driving lane is greater than a threshold (e.g., 5m, 10m, 20m, …, etc.) and / or the deviation from the direction of the driving lane is greater than a threshold (e.g., greater than 90 degrees). In this example, the planning component can determine that the current vehicle state is too far from the lane reference trajectory and / or the angle is too large, so any trajectory generated using the route-based reference system may result in a large tracking error. Therefore, at this time during the driving operation, the planning component can determine to generate one or more trajectories using the inertia-based reference system, but may not generate any trajectories using the route-based reference system.

[0027] In other examples, the planning component can generate at least one candidate trajectory using the inertia-based reference system and at least one candidate trajectory using the route-based reference system in each processing cycle it executes. As described below, the vehicle can be configured to determine a new control trajectory in each processing cycle of the planning component when performing a driving maneuver until the vehicle reaches the target state on the driving route. Therefore, in each processing cycle, the planning component can generate a set of additional candidate trajectories based on the current state and the target state of the vehicle in that processing cycle. The number and type of candidate trajectories generated by the planning component can vary between different processing cycles of performing a driving maneuver from deviating from the route to entering the route. Then, in each processing cycle during the maneuver, the planning component can generate a decision search tree (or search tree) based on the candidate trajectories to determine the control trajectory, as described in more detail below.

[0028] In addition to (or instead of) candidate trajectories generated using an inertia-based reference system and / or a route-based reference system, the planning component can generate additional candidate trajectories to move the vehicle from its current off-route state to a target state on the route. In some cases, the planning component can determine the control trajectory determined and executed in the previous processing cycle during the driving maneuver of the vehicle as an additional candidate trajectory.

[0029] The planning component can also determine additional candidate trajectories for performing the driving maneuver by perturbing any other candidate trajectories generated using any of the techniques described herein. For example, one or more perturbation parameters can be used to perturb (e.g., modify) one or both of the above-described route-based reference trajectory and / or inertia-based reference trajectory. A single candidate trajectory can be perturbed into any number of additional candidate trajectories by laterally shifting the trajectory using one or more lateral offset parameters and / or rescaling the speed of the trajectory using one or more speed scaling parameters.

[0030] After generating a set of candidate trajectories from the current off-route vehicle state to a target state on the route, the planning component can perform trajectory optimization and / or filtering on the set of candidate trajectories. During trajectory optimization, the planning component can modify each candidate trajectory based on one or more objects detected in the driving environment. For example, the trajectory optimization performed on each candidate trajectory can modify (and / or exclude) the trajectory such that the modified trajectory takes into account the current state of the driving environment, including the current vehicle state, the vehicle's state relative to the map data, and / or the static and dynamic objects sensed by the vehicle sensors. For example, in some cases, any candidate trajectory that intersects the position of a static object in the environment can be filtered out and excluded from the decision search tree. In contrast, a candidate trajectory that intersects a dynamic object can be retained by modification to avoid the possibility of intersection with the dynamic object. After trajectory optimization and / or filtering, each remaining candidate trajectory can be a dynamically feasible and statically aware (e.g., does not intersect a static object) trajectory that starts from the current off-route vehicle state and ends at the target state on the route.

[0031] In some examples, the planning component can use tree search techniques to evaluate a set of candidate trajectories determined from a current off-route vehicle state to a target state on the route. The tree search techniques can iteratively determine a set of candidate actions based on the candidate trajectories and predict potential future states of the vehicle in response to the candidate actions. Based on determining the candidate actions and predicting future states based on the candidate actions, various technical examples of using tree search to control autonomous vehicle motion can be found, for example, in U.S. Patent Application No. 17 / 394,334, titled "Vehicle Trajectory Control Using Tree Search," filed on August 4, 2021, and U.S. Patent Application No. 17 / 900,332, titled "Vehicle Trajectory Tree Search," filed on August 31, 2022, each of which is incorporated herein by reference in its entirety for all purposes.

[0032] In some examples, the planning component can perform a tree search by initially determining candidate actions associated with each candidate trajectory. For example, during the tree search, the planning component can determine a first set of potential candidate actions for the vehicle based on the vehicle's current off-route state in the environment and determine a set of predicted future states of the vehicle (and / or the environment) at a first future time step based on the candidate actions. Based on the predicted vehicle and / or environment states, the planning component can evaluate the potential candidate actions and select one or more candidate actions for further exploration while discarding other candidate actions. For the selected candidate action(s), the planning component can determine a set of additional candidate actions based on the (one or more) predicted states of the vehicle and / or the environment at the first future time step and a set of predicted future states of the vehicle and / or the environment at a second future time step associated with the candidate action, and so on until the tree search reaches the target vehicle state on the route.

[0033] Candidate actions can indicate, for example, state data within a candidate trajectory for controlling vehicle movement, including potential actions and / or control commands of the vehicle (e.g., speed commands, acceleration commands, steering angle commands, yaw commands, etc.). In some cases, candidate actions can also include commands for activating vehicle emitters (e.g., turn signals, headlights, speakers), etc. In some examples, each candidate action determined by the planning component can be associated with a different action node of a tree structure, and each predicted vehicle and / or environmental state can be associated with a related prediction node of the tree structure. As an initial operation, the tree search can determine a current off-route state associated with the vehicle, at least in part, based on sensor data, which can include the current state of the vehicle itself, the current state of the driving environment, and any number of static and / or dynamic objects in the environment. The current off-route vehicle state can be associated with a root node in the tree structure generated by the planning component. In some examples, the state of the vehicle and / or the environment can be indicated by a data structure (e.g., a scene embedding or encoding) associated with the root node.

[0034] Using the current off-route vehicle state and environment associated with the root node, the planning component can begin the tree search by iteratively determining and evaluating a set of potential candidate actions for the vehicle based on a group of candidate trajectories. Examples of candidate actions can include fine-grained instructions such as speed, speed change, steering angle, steering angle change, yaw rate, yaw rate change, etc., which can be tracked by the vehicle controller. Using various candidate actions at each node in the tree search can provide many technical advantages for determining a trajectory for the vehicle to perform an operation from off-route to on-route driving, including providing a relatively low-cost way to robustly explore the search space and quickly determine a low-cost (e.g., minimum cost) traversal representing an improved (or optimal) trajectory for performing the operation.

[0035] After determining a group of candidate actions for a node in the tree search, the planning component can use a cost function to evaluate the candidate actions and select the candidate action(s) for continuing the tree traversal. When generating the tree structure, the planning component can select one or more different candidate actions associated with the node for further exploration. As described in more detail below, the planning component can select a candidate action for further exploration in the tree structure based on the cost associated with each candidate action. In some examples, determining the cost of a candidate action can include projecting the candidate action into a trajectory that covers the length and / or time of a driving route (or a portion thereof), and then determining any number of constituent costs (described below) based on the projection of the candidate action.

[0036] For example, for each candidate action, the planning component can calculate the associated costs based on a variety of different cost types, including safety costs (e.g., based on determining potential intersections and / or other potential interactions between the vehicle and other objects in the environment, the proximity of the vehicle to non-drivable road surfaces, etc.), comfort costs (e.g., speed, acceleration, and / or acceleration change rate metrics associated with the candidate action, etc.), route progress costs (e.g., based on displacement or progress metrics of the driving route, etc.), energy efficiency costs (e.g., based on the vehicle model and / or drive system for performing the candidate action and / or trajectory, etc.), compliance costs, and so on. In some examples, the cost for each candidate action can be determined, and this cost can be associated with subsequent nodes (or candidate action nodes) in the tree that represents the vehicle state and the predicted environmental state updated based on the selection of the candidate action. To evaluate the cost of a candidate action node, the planning component can sum (or otherwise aggregate) the costs associated with the nodes that make up the branch of the tree structure that includes the candidate action (e.g., the cost of the candidate action node and the costs of all the parent nodes on which the candidate action node depends, traced back to the root node).

[0037] As described above, when performing a route driving operation, the progress cost can be measured relative to the lane reference state on the vehicle's driving route. In contrast, in the examples described herein, the progress cost can be determined relative to the reference state of the trajectory determined for performing an operation of deviating from the route to driving on the route, rather than relative to the lane reference state. For example, when performing an operation of deviating from the route to driving on the route using the techniques described herein, the initial portion of the "optimal trajectory" of the operation can cause the vehicle to move perpendicular (or backward) to the desired road. In this example, the initial portion of the "optimal trajectory" may not have any "progress" relative to the lane reference state on the driving route, but may have "progress" relative to the reference state determined for the operation of deviating from the route to driving on the route. Thus, these techniques provide the additional advantage of allowing trajectory optimization (e.g., using tree search) based on the progress cost and the additional costs described herein.

[0038] To generate a tree search, the planning component can iteratively determine a set of candidate actions (e.g., including inertia-based, route-based, and / or perturbed candidate actions) at each node in the search, evaluate the associated candidate action nodes using one or more cost functions, and traverse the tree based on determining one (or more) lowest-cost action nodes. The tree traversal can continue until a target state on the driving route is reached, at which point the planning component can identify one or more potential trajectories for controlling the vehicle as the set of lowest-cost nodes that connect the current off-route vehicle state to the target state on the driving route.

[0039] As these and other examples illustrate, the techniques described herein can improve the functionality, safety, and driving efficiency of autonomous and semi-autonomous vehicles. Specifically, these techniques can improve the vehicle's trajectory determination, enabling it to perform complex driving operations between off-route vehicle states and on-route vehicle states, which can be determined faster and with fewer computational resources compared to other trajectory determination / optimization techniques. Using these techniques, a planning component can determine an optimal trajectory and / or measure progress along a route, even without a predefined reference trajectory for an off-route driving path. Additionally, as described above, the planning component can use various techniques (e.g., inertia-based, route-based, and / or perturbed candidate trajectories, etc.) to generate a set of dense candidate trajectories, and then the candidate trajectories can be processed and evaluated in a similar manner and using the same tree search, regardless of the various techniques used to generate the trajectories. Further, these techniques avoid using inaccurate heuristics when performing driving operations from off-route to on-route, as well as the requirement to switch between different trajectory generation modes. For example, the techniques described herein do not need to determine an "artificial boundary" (or other heuristics) to switch between an inertia-based reference system and a route-based reference system to generate a trajectory, but can instead continuously use and evaluate multiple techniques throughout the driving operation.

[0040] As described in more detail below, certain examples herein can be implemented by a trajectory planner (e.g., a planning component) of an autonomous vehicle, which can include components for generating and traversing a tree search to determine a trajectory for the vehicle to follow. However, in other examples, techniques similar or identical to those described herein can be used with other types of data structures and / or other search algorithms. In various cases, the tree search can use a binary tree structure that includes a single route-based candidate action and a single inertia-based candidate action for each node, or in other examples, a non-binary search tree (e.g., including any number of perturbed candidate actions) can be used. The various tree search algorithms that can be used can include, but are not limited to, depth-first search and breadth-first search, as well as tree traversal techniques using sequential and / or parallel computing, etc. In examples where other types of data structures (e.g., non-tree structures) are used, other types of graph-based traversal algorithms can also be used to generate, evaluate, and traverse the nodes of the data structure to determine possible vehicle trajectories.

[0041] The techniques described herein can be implemented in a variety of ways. Example embodiments are provided below with reference to the accompanying drawings. Although discussed in the context of autonomous vehicles, the methods, apparatuses, and systems described herein can be applied to a variety of systems (e.g., sensor systems or robotic platforms) and are not limited to autonomous vehicles. In one example, similar techniques can be utilized in a driver-controlled vehicle, where such a system can provide an indication of whether it is safe to perform various maneuvers. In various other examples, these techniques can be used in the aviation or maritime domains and can be incorporated into any ground, air, or water vehicle that uses route planning techniques, including vehicles that require constant manual control by a driver to those that are partially or fully autonomously controlled.

[0042] Figure 1 An example scenario 100 is shown that includes an autonomous vehicle 102 configured to determine a plurality of possible trajectories and then select a control trajectory for performing a driving operation from off-route to on-route in an environment. In some instances, autonomous vehicle 102 can be an autonomous vehicle configured to operate according to a level 5 classification issued by the National Highway Traffic Safety Administration, which describes a vehicle capable of performing all safety-critical functions throughout a trip, where the driver (or occupant) is not expected to control the vehicle at any time. However, in other examples, autonomous vehicle 102 can be a fully autonomous vehicle or a partially autonomous vehicle with any other level or classification. It is envisioned that the techniques discussed herein can be applied to fields other than robotic control, such as for autonomous vehicles. For example, the techniques discussed herein can be applied to trajectory finding in video games, manufacturing, augmented reality, etc.

[0043] According to the techniques discussed herein, vehicle 102 can receive sensor data from one or more sensors 106 of vehicle 102. For example, sensors 106 can include position sensors (e.g., Global Positioning System (GPS) sensors), inertial sensors (e.g., accelerometer sensors, gyroscope sensors, etc.), magnetic field sensors (e.g., compasses), position / velocity / acceleration sensors (e.g., speedometers, drive system sensors), depth position sensors (e.g., lidar sensors, radar sensors, sonar sensors, time-of-flight (ToF) cameras, depth cameras, and / or other depth-sensing sensors), image sensors (e.g., cameras), audio sensors (e.g., microphones), and / or environmental sensors (e.g., barometers, hygrometers, etc.).

[0044] (One or more) sensors 106 may generate sensor data that may be received by (one or more) computing devices 104 associated with vehicle 102. However, in other examples, some or all of (one or more) sensors 106 and / or (one or more) computing devices 104 may be separated from and / or disposed remote from vehicle 102, and data capture, processing, commands, and / or control may be transmitted to / from vehicle 102 via wired and / or wireless networks by one or more remote computing devices.

[0045] Autonomous vehicle 102 may include (one or more) computing devices 104, which may include one or more ML (machine learning) models and / or the navigation systems discussed herein. For example, (one or more) computing devices 104 include a memory 108 for storing a perception component 110 and / or a planning component 112. As described below, the planning component 112 may include various components configured to perform tree search techniques, including combinations of inertia-based candidate actions, route-based candidate actions, perturbed candidate actions, etc. Vehicle 102 may also use adaptive learning to predict the future trajectory / state of vehicle 102 within scenario 100. The sensors 106, perception component 110, and / or planning component 112 may include hardware and / or software for performing the operations related to trajectory determination and navigation of autonomous vehicle 102 described herein. The various navigation systems described herein may include more or fewer components, but for ease of understanding, only the perception component 110 and / or the planning component 112 are given as non-limiting examples.

[0046] In some examples, the various vehicle navigation systems and functions described herein may include processor-executable instructions, hardware, and / or some combination thereof (e.g., field programmable gate array (FPGA), application specific integrated circuit (ASIC)) stored in and / or accessible to the memory of (one or more) computing devices 104.

[0047] In example scenario 100, autonomous vehicle 102 may be in a parked state or may be performing a parking or departure operation. Autonomous vehicle 102 is shown in a vehicle state 114 that is currently off route. Initially, autonomous vehicle 102 may receive and / or determine a travel route 116 based on the vehicle's intended destination. Travel route 116 may include a route start state (e.g., target state 118) and an end state 120 representing, for example, the position, speed, and / or attitude that autonomous vehicle 102 wants to achieve. Planning component 112 may determine route 116 at least in part based on sensor data, map data, and / or a task-based intended destination (e.g., received from a passenger, a command center, etc.). As described above, "state" or "vehicle state" may include geometric state data, such as the position (or orientation) and / or attitude of the vehicle (e.g., position and / or direction / course including yaw angle and steering angle). Additionally, in some examples, the vehicle state may include any combination of vehicle geometric state data and vehicle time state data (e.g., speed, acceleration, yaw angle, yaw rate, steering angle, steering angle rate, etc.), and / or may include any other state data related to the vehicle (e.g., current vehicle state data, state of vehicle signals and operation controls, etc.).

[0048] As autonomous vehicle 102 travels in the environment, it may receive map data of the environment (e.g., from a local or remote map system) and perception data (e.g., sensor data) from perception component 110. The map data may include, for example, road data determined based on a map of the travel environment and / or road data determined based on positioning autonomous vehicle 102 in the environment. For example, the map data may include data related to any number of segments (e.g., lane segments) in the travel environment, such as the position (e.g., boundaries), size (e.g., length and width), and shape (e.g., curvature) of the segments, as well as other attributes of the segments, such as directionality, speed limit, grade, road surface, etc.

[0049] Autonomous vehicle 102 may also receive sensor data from one or more sensors 106 of autonomous vehicle 102. Perception component 110 may include one or more ML models and / or other computer-executable instructions for detecting, identifying, segmenting, classifying, and / or tracking objects from sensor data collected from the environment of autonomous vehicle 102. For example, autonomous vehicle 102 may use the data generated by perception component 110 to locate its position in the travel environment relative to the map data. In some cases, perception component 110 may also generate a drivable surface map and / or an occupancy map indicating which areas in the environment are drivable surfaces and non-drivable surfaces, and which positions in the environment are occupied by objects or which are unoccupied and free space positions where autonomous vehicle 102 can travel.

[0050] Before the autonomous vehicle 102 can navigate along the driving route 116 to reach the end state 120, it can first determine and execute a driving maneuver from the off-route state 114 to the on-route target state 118. As described in the examples herein, the planning component 112 can use map data and / or perception data, determine candidate trajectories and / or candidate actions, and perform a tree search technique to determine the trajectory 122 that the autonomous vehicle 102 should follow when executing the driving maneuver. The trajectory 122 can continuously and feasibly connect the current off-route vehicle state 114 to the expected target state 118 on the driving route 116. As discussed in more detail below, the planning component 112 can determine the trajectory 122 as an improved or lowest-cost (e.g., optimal) trajectory by performing a tree search that includes determining a combination of inertia-based candidate trajectories and / or route-based candidate trajectories (among others), and considering the future predicted (one or more) driving scenarios of the environment to evaluate candidate actions, including the predicted trajectory of the autonomous vehicle 102 and the predicted trajectories / states of other agents or objects in the environment. In some cases, the trajectory 122 can represent the best and / or lowest-cost trajectory determined by the planning component 112 based on safety costs (e.g., potential interactions with objects / agents), passenger comfort costs, route progress costs, etc., after evaluating multiple kinematic and dynamic feasible trajectories (e.g., trajectories considering various kinematic and dynamic constraints of the vehicle) determined by the tree search technique.

[0051] In this example, the planning component 112 has determined a single trajectory 122 as the selected control trajectory for the autonomous vehicle 102 to perform the driving operation from off-route to on-route. In other examples, the planning component 112 can use a search tree and / or various other techniques described herein to determine any number of alternative low-cost trajectories. To implement a selected trajectory such as trajectory 122 (e.g., a potential control trajectory), the planning component 112 can generate multiple potential vehicle control actions for controlling the movement of the autonomous vehicle 102 based at least in part on the trajectory 122, according to a model predictive control technique (e.g., 1 microsecond, half a second, multiple seconds, etc.), substantially simultaneously. The planning component 112 can select one or more of the potential vehicle control actions and generate a drive control signal therefrom, which can be transmitted to the drive component of the autonomous vehicle 102 to control the vehicle to travel along the trajectory 122.

[0052] Figure 2 An example method 200 for an autonomous vehicle 102 to perform a driving operation from off-route to on-route in a driving environment is depicted. As described below, the planning component 112 of the autonomous vehicle can determine multiple candidate trajectories of the vehicle, including route-based candidate trajectories and inertia-based candidate trajectories, and can use a search tree to determine a control trajectory for controlling the vehicle to perform the operation of joining the driving route.

[0053] At operation 202, the planning component 112 may determine the target vehicle state of a lane (or road) in the driving route designated by the autonomous vehicle 102. For example, the planning component 112 may preliminarily determine the driving route of the autonomous vehicle based on the current position and expected destination of the vehicle, as well as based on road network data, map data, traffic data, etc. After determining the driving route, the planning component 112 may identify the part of the driving route (e.g., road and / or driving lane) that is closest to the current off-route vehicle state and accessible from the current off-route vehicle state. Within the closest accessible part of the closest accessible road, the planning component 112 may determine a specific state in the lane reference trajectory of the road as the target state for the autonomous vehicle 102 to join the driving route. In various examples, the planning component 112 may determine the target state of the autonomous vehicle 102 on the driving route based on the lateral and / or longitudinal positions of multiple possible target states in the lane reference trajectory, as well as the current heading of the vehicle's driving direction relative to the driving lane, the current speed of the vehicle, and / or various characteristics or functions of the vehicle (such as turning radius and maximum steering angle). For example, at operation 202, the planning component 112 may use a heuristic method to determine the target state based on the distance between the current position of the autonomous vehicle 102 and the desired driving lane and / or the offset angle between the current direction of the autonomous vehicle 102 and the direction of the desired driving lane. In some cases, the planning component 112 may determine a single target state, while in other examples, the planning component 112 may determine multiple possible target states of the autonomous vehicle 102 within the desired lane or road.

[0054] Box 204 shows an example driving environment, which includes the autonomous vehicle 102 that is currently parked (or may be performing a parking or departure operation) in the current vehicle state 206. As shown in this example, the current vehicle state 206 is in an off-route state relative to the driving route designated for the vehicle, and the driving route includes the lane reference trajectory 208 of the driving lane adjacent to the vehicle. Based on the relative position and direction of the current vehicle state 206 and the state of the lane reference trajectory 208, the planning component 112 may determine the target state 210 as the target state for the autonomous vehicle 102 to perform a driving operation from off-route to on-route. In this example, the driving environment also includes the state vehicle 212, whose position and direction may affect the selection of the target state 210 and / or the selection of the possible trajectory available for the autonomous vehicle 102 to navigate to the target state 210.

[0055] At operation 214, the planning component 112 can determine a first candidate trajectory between the current off-route vehicle state and the on-route target state according to an inertia-based reference system. As described above, the planning component 112 can generate a candidate trajectory (or multiple candidate trajectories) according to the inertia-based reference system, using a cubic spline curve or other mathematical functions, without referring to the lane reference trajectory of the driving lane. In various examples, the planning component 112 can use a cubic spline curve, a Bezier curve, a series of clothoid curves, and / or any other closed boundary condition solution to calculate an inertia-based reference trajectory based on the current off-route vehicle state (e.g., current vehicle position, attitude, heading, and speed) and the target state 210 (e.g., target vehicle position, attitude, heading, and speed). In some examples, the planning component 112 can use one or more mathematical functions to determine a curve (e.g., a cubic spline curve, a Bezier curve, etc.), and then apply a speed profile to the curve (e.g., based on the speed offset and angle offset between the current vehicle state and the target state) to determine the spatio-temporal vehicle state of the candidate trajectory. Box 216 shows an example driving environment, which includes an inertia-based reference trajectory 218 generated by the planning component 112.

[0056] At operation 220, the planning component 112 can determine a second candidate trajectory between the current off-route vehicle state and the on-route target state using a route-based reference system. As described above, the planning component 112 can generate a candidate trajectory (or multiple candidate trajectories) according to the route-based reference system by retrieving and tracking the lane reference trajectory 218 relative to the current off-route vehicle state. For example, the planning component 112 can determine the spatial offsets (e.g., lateral and longitudinal) and / or speed offsets between the current vehicle state and one or more nearby states of the lane reference trajectory 208. Based on these differences, the planning component 112 can determine a modification of the lane reference trajectory 218 that may cause the vehicle to converge to the target state 210. Box 222 shows an example driving environment, including a route-based reference trajectory 224 generated by the planning component 112, which is different from the inertia-based trajectory 218.

[0057] As described above, although the example includes only two candidate trajectory examples (e.g., an inertia-based trajectory 218 and a route-based trajectory 224), in other examples, the planning component 112 can generate any number of additional candidate trajectories for connecting the current off-route vehicle state 206 and the on-route target state 210. For example, the planning component 112 can determine candidate trajectories based on one or more previous control trajectories used by the autonomous vehicle 102 during the off-route to on-route driving operation. Additionally or alternatively, the planning component 112 can generate any number of perturbed trajectories by laterally shifting, speed scaling, and / or steering angle scaling any other candidate trajectory using perturbation parameters.

[0058] At operation 226, the planning component 112 can generate a search tree and determine a traversal of the tree structure representing the control trajectory of the autonomous vehicle 102 to perform an off-route to on-route driving operation between the current vehicle state and the target state. As described in more detail below, the planning component 112 can use a tree search component and / or other similar search-based components to determine and evaluate various candidate actions associated with the candidate trajectories. The tree search component can use trajectory optimization techniques, environmental data, and / or active prediction techniques, as well as a cost function, to determine the lowest cost and / or optimal path (or paths) through the tree structure. For example, box 228 shows an example tree structure 230 that represents the physical driving environment between the current off-route state 206 and the on-route target state 210.

[0059] Figure 3 An example system 300 is shown that includes a planning component 112 of an autonomous vehicle, the planning component 112 being configured to determine a trajectory of the vehicle using tree search (and / or other search algorithms), including using a combination of candidate actions determined according to an inertia-based reference system, a route-based reference system, and perturbed candidate actions, including using cost evaluation based on predicted future states of the vehicle and the environment. As described below, at least some components of the system 300 can be implemented within the planning component 112, such as a tree search component 302, an inertia-based candidate action generator 304, a route-based candidate action generator 306, a perturbed candidate action generator 308, a trajectory optimizer 310, and a cost evaluator 320. However, as described below, one or more of these components can be implemented within separate components within the computing device 104 (e.g., within a prediction component) and / or within separate computing devices / systems (e.g., within a GPU-based computing system).

[0060] System 300 may be implemented to perform on-vehicle trajectory determination for autonomous vehicle 102 in a real-time driving environment. In some examples, tree search component 302 may be configured to generate and traverse a search tree (or other data structure) to determine potential vehicle trajectories as described herein. Other examples of techniques for generating and traversing a search tree to determine a trajectory for controlling the movement of an autonomous vehicle based on determining candidate actions and predicting future states based on the candidate actions, for example, may be found in U.S. Patent Application No. 17 / 394,334, filed on August 4, 2021, entitled "Vehicle Trajectory Control Using Tree Search", the entire disclosure of which is incorporated herein by reference for all purposes. Based on the tree search techniques described herein, tree search component 302 may determine one or more paths (or traversals) through the nodes of a tree structure representing a driving route. In some cases, tree search component 302 may evaluate multiple different paths through the tree to identify one or more low-cost paths representing possible trajectories for the vehicle to follow.

[0061] Inertia-based candidate action generator 304 may determine candidate trajectories, including candidate actions at various states in the trajectory. As described above, inertia-based candidate action generator 304 may determine one or more trajectories between a current vehicle state (e.g., off-route state) and a target state on the route specified for the vehicle. Inertia-based candidate action generator 304 may generate inertia-based candidate trajectories and actions to move the vehicle from the current off-route state to the target state in a kinematically feasible manner (without reference to the structure of the driving route). To generate candidate trajectories and / or actions, inertia-based candidate action generator 304 may use mathematical functions, including but not limited to cubic splines, Bezier curves, a series of clothoid curves, and / or any other closed-form boundary condition solutions. Inputs to the mathematical functions may include the lateral and longitudinal distances of the target state relative to the current vehicle state, the lane directionality of the target state relative to the heading of the current vehicle state, and / or vehicle capabilities, such as turning radius and maximum steering angle.

[0062] The route-based candidate action generator 306 may also determine one or more candidate trajectories and / or candidate actions between the current vehicle state (e.g., off-route state) and the target state on the route specified for the vehicle. Different from the inertia-based candidate trajectories and actions, the route-based candidate trajectories and actions may be based on the reference trajectory of the driving lane where the target state is located. To determine the route-based candidate trajectories / actions, the route-based candidate action generator 306 may retrieve and "track" the lane reference trajectory relative to the current off-route vehicle state. For example, the route-based candidate action generator 306 may determine the lateral offset and / or speed difference between the current vehicle state and the corresponding state (e.g., the spatially closest state) in the driving lane reference trajectory, and may modify the speed and steering angle of the lane reference trajectory state to generate a route-based reference trajectory that can converge the vehicle to the target state.

[0063] The perturbed candidate action generator 308 may also determine one or more additional candidate trajectories and / or candidate actions between the current vehicle state (e.g., off-route state) and the target state on the route specified for the vehicle. In this example, the perturbed candidate action generator 308 may generate additional candidate trajectories by modifying other candidate trajectories based on perturbation parameters. The perturbation parameters may include, for example, a lateral offset parameter configured to laterally move the candidate trajectory a certain distance (e.g., 0.5 m, 1 m, 2 m, etc.) in either lateral direction. A speed scaling parameter may be configured to change the speed of the candidate trajectory by a certain speed amount (e.g., + / -1 MPH, 2 MPH, 5 MPH, etc.) or percentage (e.g., + / -1%, 5%, 10%, etc.). A steering angle parameter is configured to change the steering angle of the candidate trajectory by an angular distance amount (e.g., + / -1 degree, 2 degrees, 5 degrees, etc.). The perturbed candidate action generator 308 may generate any number of candidate trajectories by perturbing (or modifying) different candidate trajectories (e.g., inertia-based candidate trajectories, route-based candidate trajectories, and / or the previous control trajectory of the vehicle, etc.) using any combination of perturbation parameters (e.g., speed, steering angle, lateral offset, etc.) and / or any number of different perturbation amplitudes of the perturbation parameters.

[0064] As shown in this example, each of the inertia-based candidate action generator 304, the route-based candidate action generator 306, and the perturbed candidate action generator 308 can provide candidate trajectories to the trajectory optimizer 310. The trajectory optimizer 310 can be configured to optimize and / or filter candidate trajectories based on the current environment. For example, the trajectory optimizer 310 can modify each candidate trajectory based on map data and / or various agents and objects detected in the driving environment. In some cases, the trajectory optimizer 310 can filter out candidate trajectories that intersect (e.g., overlap) with the positions of static objects in the environment. Other candidate trajectories can be optimized based on the current state of the driving environment, which includes the current vehicle state, the vehicle's state relative to the map data, and / or static and dynamic objects sensed by the vehicle sensors. For each candidate trajectory, the trajectory optimizer 310 can modify the trajectory to output a dynamically feasible and statically aware (e.g., non-intersecting with static objects) trajectory that starts from the current vehicle state and ends at a target state on the route.

[0065] In some examples, one or more of the candidate action generators 304-308 and / or the trajectory optimizer 310 can use an active prediction model to determine one or more ML-prediction-based candidate actions. Such an active prediction model can be implemented at least in part based on the various techniques and systems described in U.S. Patent Application No. 17 / 351,641, filed on June 18, 2021, titled "Active Prediction Based on Object Trajectories", the entire content of which is incorporated herein by reference for all purposes.

[0066] The trajectory optimizer 310 can receive as input the current vehicle state of the autonomous vehicle 102 (e.g., including the current trajectory) and a representation of the driving environment in which the vehicle is located. In this example, the trajectory optimizer 310 can receive a scene encoding 312 representing the current environment, at least in part based on current vehicle state data 314 (which can include the designed driving route, intended destination, and / or driving preferences of the autonomous vehicle 102), agent data 316 based on the perception component 110 and / or data captured by vehicle sensors of static and / or dynamic agents in the environment, and map data 318 from the vehicle map component. For example, based on data captured by the vehicle sensors 106, using the perception component 110, a driving environment scene encoder can generate a scene encoding 312 (e.g., a scene embedding), which can be a vector specific to a particular driving scene and scenario for representing the driving environment at a particular time. In some cases, the driving environment scene encoder can use a trained neural network architecture to output a scene encoding based on inputs including a combination of map data and data sensed by the vehicle in the environment. Various technical examples for determining the scene encoding and / or other environment representations, for example, can be found in U.S. Patent Application No. 17,855,088, titled "Machine Learning Components for Vehicle Trajectory Generation," filed on June 30, 2022, the entire content of which is incorporated herein by reference for all purposes.

[0067] In some examples, the trajectory optimizer 310 can determine predicted future trajectories (and / or other predicted vehicle state data) for the autonomous vehicle 102 and any other agents / objects in the environment, and / or a predicted future state for the entire environment. In some examples, the trajectory optimizer 310 can output a predicted future scene encoding having an encoding format similar or identical to the current scene encoding 312. As described above, the trajectory optimization can also include iteratively determining a sequence of predicted vehicle and agent trajectories, and / or scene encodings.

[0068] The cost estimator 320 can receive the determined costs associated with the respective candidate actions determined by the respective candidate action generators 304-308. In some cases, the cost estimator 320 can use an active prediction model and / or other machine learning (ML)-based prediction models to predict the future trajectory / state of the autonomous vehicle 102 based on the candidate actions, and / or to predict the future environmental state based on the candidate actions. Then, the cost estimator 320 can evaluate the predicted trajectory and / or environment to determine a set of costs associated with the candidate trajectory. The costs can be determined by evaluating the respective predicted trajectories / states of the autonomous vehicle 102 based on the candidate actions and / or the predicted trajectories / states of other agents in the environment. In some examples, to execute faster and save computing resources, the cost estimator 320 does not need to (but can) re-execute the full active prediction model for each candidate action. For example, to evaluate the cost of a candidate action, the cost estimator 320 can make the assumption that the autonomous vehicle 102 can continue to execute the same candidate action throughout the driving operation (e.g., until a goal state is reached and / or an end time step is reached).

[0069] In addition to or as an alternative to using the active prediction model to calculate the costs associated with candidate actions, the cost estimator 320 can also include various heuristics and / or machine learning-based components that are configured to detect and calculate the costs associated with potentially unsafe, illegal, or risky driving operations. These costs (which can be referred to as safety costs) may include speeding, leaving the lane or crossing the double yellow line, stopping, braking, accelerating, or turning too sharply based on the road / lane configuration and the current driving conditions. Other costs determined by the cost estimator 320 can include passenger comfort costs (e.g., based on sharp turns, unnecessary turns, bumps, jerks, or inconsistent trajectories) and route progress costs (e.g., based on the longitudinal distance obtained, the vehicle speed, and / or the travel time cost between the current vehicle position and the end state of the route). For these costs and the various other costs described herein, the cost estimator 320 can be configured to evaluate the trajectories output by the active prediction model, including the predicted trajectories of the autonomous vehicle 102 and / or other agents in the environment, and to calculate the cost values associated with the predicted trajectories either individually or in combination.

[0070] In addition to evaluating the cost of a single trajectory, the cost estimator 320 can also determine the cost by analyzing multiple predicted trajectories (and / or the entire predicted environment) to identify potential interactions between the autonomous vehicle 102 and one or more additional agents or other objects in the environment. For example, the cost estimator 320 can calculate a cost value based on determining potential intersections between the trajectory of the autonomous vehicle 102 (e.g., assuming the autonomous vehicle 102 continues to execute the candidate action) and the trajectories of agents (or multiple agents) at any future time in the predicted driving scenario. Such interaction costs can include costs based on detecting potential collisions or near misses, failure to yield, and / or aggressive driving of the autonomous vehicle 102 relative to other vehicles, pedestrians, bicycles, etc. In some examples, the cost estimator 320 can determine the interaction cost based on potential intersections between multiple agents (possibly excluding the autonomous vehicle 102).

[0071] In various examples, the cost estimator 320 can evaluate candidate actions (e.g., based on costs associated with individual trajectories and / or groups of trajectories with potential interactions) and / or the overall driving scenario / environment over multiple predicted future time steps. As described above, the cost estimator 320 can use an active prediction model to determine predicted trajectories and / or predicted driving scenario encodings over a period of time (e.g., 2 seconds, 5 seconds, 10 seconds, etc.), which can include any number of discrete time steps between the current state and the target state. The cost estimator 320 can evaluate the trajectory / driving scenario and determine the costs associated with each discrete time step, and can sum various costs (e.g., including individual trajectory costs and vehicle-agent interaction costs) over the discrete time steps of the entire trajectory of the autonomous vehicle 102. In some examples, the cost estimator 320 can increase the weight of costs determined for earlier time steps in the trajectory (which are more likely to occur) and decrease the weight of costs determined for later time steps in the trajectory (which are less likely to occur).

[0072] Additionally, in some cases, the cost estimator 320 can calculate costs associated with multiple alternative future predictions. For example, the cost estimator 320 can separately calculate a set of costs for different possible trajectories output by the active prediction model, where the different possible trajectories are associated with the same candidate action. To determine the total cost associated with the candidate action, the cost estimator 320 can use the corresponding confidence values and / or likelihoods of the set of trajectories to aggregate and / or weight the costs from the set of alternative predictions of the trajectories / states to scale / weight the total cost calculation.

[0073] After calculating a cost value or set of cost values associated with a particular candidate action, the cost estimator 320 can return the cost to the tree search component 302, which can select one or more candidate actions from the nodes in the tree for further exploration. As described above, the tree search component 302 can iteratively perform these techniques, including determining new nodes representing the vehicle state, determining candidate actions for the vehicle state, evaluating the candidate actions using a cost function by the cost estimator 320, and generating the tree by creating one or more additional nodes based on the selected candidate actions. After reaching the expected end state of the driving route, the tree search component 302 can determine one or more node paths through the tree structure that represent one or more possible trajectories followed by the autonomous vehicle 102 in the driving environment. The tree search component 302 can determine one or more lowest-cost and / or optimal paths based on the extent to which the search space of possible trajectories has been explored using the determined candidate actions, and / or based on the cost metrics / thresholds used by the tree search component 302 to determine the optimal or lowest-cost trajectory. When the tree search component 302 has determined a possible solution trajectory based on the node costs (e.g., the optimal or lowest-cost trajectory through the tree structure), that trajectory can be selected as the control trajectory 322, and the autonomous vehicle 102 can follow the control trajectory 322 to reach the goal state on the route.

[0074] Figures 4A - 4F Illustrates several examples of determining and selecting candidate trajectories for controlling a vehicle from a current off-route vehicle state to a goal vehicle state on a driving route. As shown in these examples, the planning component 112 can use a variety of different techniques to generate a set of dense candidate trajectories to transition the vehicle from the current off-route state to the goal state on the route. The trajectory optimizer and decision search tree techniques can be used to optimize and evaluate the various candidate trajectories in a similar manner to determine the control trajectory that the autonomous vehicle 102 is to follow when performing driving operations from off-route to on-route.

[0075] Figure 4A Depicts the driving environment 402, which includes the autonomous vehicle 102 in a current off-route vehicle state 404. For example, the autonomous vehicle 102 may be in a stopped state, performing a parking or egress operation, leaving or entering the road, reversing, or performing other uncommon driving operations, etc. The autonomous vehicle 102 may also be driving in an unstructured environment such as a parking lot, lane, or construction area (e.g., an environment without a predefined route structure including lane reference trajectories). In this example, based on the driving route specified for the autonomous vehicle 102, the planning component 112 has determined a goal vehicle state 406 on the lane reference trajectory 408 of the driving lane.

[0076] Based on the current off-route vehicle state 404 and the on-route target vehicle state 406, the planning component 112 can determine various candidate trajectories to transition the autonomous vehicle 102 onto the driving route of the target state 406. In Figure 4A , the planning component 112 uses an inertia-based candidate action generator 304 to generate inertia-based candidate trajectories 410. As described above, the inertia-based candidate action generator 304 can use cubic spline functions or other mathematical functions to generate candidate trajectories 410 without referring to the lane reference trajectory 408 of the driving lane. In Figure 4B , in the driving environment 412, the planning component 112 uses a route-based candidate action generator 306 to generate separate route-based candidate trajectories 414. As described above, the route-based candidate action generator 306 can generate candidate trajectories 414 by using a controller to track the current vehicle state 404 to the lane reference trajectory 408. In Figure 4C , in the driving environment 416, the planning component 112 retrieves and uses the previous control trajectory 418 of the autonomous vehicle 102 (e.g., from a previous planner processing cycle) as a candidate trajectory in the current planner processing cycle. In Figure 4D , in the driving environment 420, the planning component 112 uses a perturbed candidate action generator 308 to generate a set of multiple perturbed candidate trajectories 422. Each perturbed candidate trajectory 422 can be based on a previously generated candidate trajectory (e.g., trajectory 410, 414, or 418, etc.) and can be laterally shifted and / or speed scaled based on one or more perturbation parameters to generate a perturbed trajectory.

[0077] In Figure 4E , in the driving environment 424, the planning component 112 can use a trajectory optimizer 310 to modify and / or filter Figures 4A - 4D the various candidate trajectories generated in

[0078] In Figure 4FIn the driving environment 432, the planning component 112 can use the tree search component 302 to evaluate a second subset 430 of the retained candidate trajectories and determine an output control trajectory 322 to control the vehicle from the current off-route state 404 to the on-route target state 408. As described above, the output trajectory determined by the tree search component 302 can represent an optimal and / or lowest-cost traversal of the tree structure using the decision tree search techniques described herein.

[0079] Figure 5A An example representation 500 of a set of candidate actions for the autonomous vehicle 102 is shown. In this example, the autonomous vehicle 102 is depicted as being in a current vehicle state 502, which can represent a node in a tree structure for determining a trajectory between an off-route start state and an on-route end state. The vehicle state 502 can include a set of parameters related to the state of the autonomous vehicle 102, including the vehicle's position, attitude, speed, steering angle, and / or yaw rate. In some examples, the vehicle state 502 can represent the current state of the autonomous vehicle 102 (e.g., the current state 114) and can correspond to the root node of the tree search. However, in other examples, the vehicle state 502 need not be the current state of the autonomous vehicle 102 but can represent a predicted future state of the vehicle, corresponding to a non-root node further down in the tree structure toward the end state on the driving route.

[0080] In this example, the candidate actions 504-512 are shown as representing different actions (e.g., trajectories) that the vehicle can execute from the vehicle state 502. Each of the candidate actions 504-512 can represent a dynamically kinematically feasible trajectory that the autonomous vehicle 102 is capable of executing. For example, the candidate actions 504-512 can be associated with vehicle speed, rate, steering angle, and / or other vehicle trajectory parameters. In some cases, the candidate actions can also include commands for activating vehicle emitters (e.g., turn signals, headlights, horns, speakers, etc.) and / or any other vehicle control commands. In this example, the candidate actions 504-512 are graphically depicted, with each candidate action representing a trajectory that the autonomous vehicle 102 might follow from the vehicle state 502. In some examples, the candidate actions 504-512 can be generated and stored as absolute parameter values (e.g., speed, steering angle, etc.), while in other cases, the generated and stored values are relative to the vehicle state 502 (e.g., speed difference, steering angle difference, etc.).

[0081] As described above, the planning component 112 can use each of the candidate actions 504-512 to determine a future state of the autonomous vehicle 102 that might be caused by the vehicle performing the candidate action. In other examples, the planning component 112 can use similar or the same techniques to generate multiple candidate vehicle states (e.g., instead of candidate actions). Each of the candidate actions 504-512 can include general driving commands and / or fine instructions for performing driving operations, such as speed or speed change, steering angle or steering angle change, and the like.

[0082] In this example, the candidate action 504 is a candidate action determined by the planning component 112 based on a route-based reference trajectory. As described above, the candidate action 504 can represent an action determined by the controller of the planning component for tracking a lane reference trajectory including a target state. The controller can determine the candidate action 504 as an action that attempts to converge the vehicle to the lane reference trajectory at the target state (e.g., speed, steering angle, etc.). In this example, the candidate action 506 is a previous control trajectory determined by the planning component 112 for controlling the vehicle at a previous time (e.g., previous processing cycle) during a maneuver. The candidate action 508 is a candidate action based on an inertia-based reference trajectory determined by the planning component 112. As described above, the candidate action 508 can be determined using a cubic spline function or other mathematical function to move the vehicle in a dynamically kinematically feasible manner (e.g., without referring to the structure of the driving route) between the current off-route state and the target state on the route. The candidate actions 510 and 512 represent perturbed (or modified) candidate actions that are generated by applying one or more perturbation parameters (e.g., lateral offset parameter and / or speed scaling parameter) to the inertia-based candidate action 508 and the route-based candidate action 504, respectively.

[0083] As described above, although this example depicts a set of candidate actions 504-512 that includes five individual candidate actions based on five different candidate trajectory generation techniques, in other examples, the planning component 112 can determine any number of candidate actions and any number of kinematic candidate trajectories determined by the planning component.

[0084] Figure 5B A representation 500 of four different sets of candidate actions (e.g., trajectories in the illustrated example) generated at four different nodes is shown, where the four nodes represent vehicle states at four different action levels of a tree search. Each vehicle state and each set of candidate actions in this example can be similar or identical to the vehicle state 502 and the set of candidate actions 504-512 described above, respectively. In this example, Figure 5BDepicts vehicle state 502, which can be the current off-route vehicle state of the vehicle and corresponds to the root node in the tree search. The space occupied by the autonomous vehicle 102 in vehicle state 502 is represented by the dashed line 514.

[0085] The planning component 112 can determine a first set of candidate actions 520 based at least in part on the vehicle state 502 of the autonomous vehicle 102 (including vehicle position, attitude, speed, acceleration, steering rate, etc.) and the environmental state data associated with the root node. As in the previous example, the length of the candidate actions can indicate the speed and / or acceleration associated with the candidate actions.

[0086] A second set of candidate actions 524 can be generated based at least in part on a first candidate action selected from the first set of candidate actions 520 for exploration and at least in part on the vehicle state 522 (e.g., position, attitude, speed, steering rate, etc.), where the selected first candidate action will cause the vehicle to execute after the execution of the first candidate action in the current environmental state. Although this example shows selecting a single vehicle state 522 corresponding to a single candidate action from the candidate trajectories, in other examples, the tree search technique can include selecting multiple candidate actions from multiple candidate trajectories at any or all branch points (e.g., vehicle state 522, vehicle state 526, vehicle state 530, etc.). Additionally, in some examples, the tree search technique can include tracking and expanding multiple branches that correspond not only to different candidate trajectories / actions but also to different possible future states of the driving environment itself (e.g., different possible object positions, agent trajectories, etc.).

[0087] In some cases, the vehicle state 522 can represent a subsequent vehicle state based on following one of these inertia-based candidate trajectories, while in other cases, the vehicle state 522 can represent a subsequent vehicle state based on following one of these route-based candidate trajectories, perturbed candidate trajectories, etc. When evaluating the first set of candidate actions 520 and / or selecting the vehicle state 522 for further exploration in the tree search, the planning component 112 can generate a search tree node representing the selected vehicle state 522. In some examples, the second set of candidate actions 524 can include the actions (or multiple actions) associated with each candidate trajectory determined (as described above) and provided as input to the tree search (e.g., inertia-based reference trajectories, route-based reference trajectories, one or more perturbed (or modified) candidate trajectories, etc.). For example, the second set of candidate actions 524 can include the actions within each candidate trajectory for the corresponding trajectory segments (e.g., time steps) in the respective trajectories.

[0088] A third set of candidate actions 528 can be similarly based at least in part on selecting a second candidate action from the second set of candidate actions 524, based at least in part on the vehicle state 526, where the selected second candidate action will cause the vehicle to execute after completion of the execution of the second candidate action in a subsequent state of the environment associated with the vehicle state 526 (e.g., a future predicted environmental state, vehicle state, agent position, etc. at the vehicle state 526). When evaluating the second set of candidate actions 524 and / or when selecting the vehicle state 526 for further exploration in the tree search, the planning component 112 can generate a search tree node representing the selected vehicle state 526. Similarly, a fourth set of candidate actions 532 can be based at least in part on selecting a third candidate action from the third set of candidate actions 528, based at least in part on the vehicle state 530, where the selected third candidate action will cause the vehicle to execute after completion of the execution of the third candidate action in an environmental state associated with the vehicle state 530. When evaluating the third set of candidate actions 528 and / or selecting the vehicle state 530 for further exploration in the tree search, the planning component 112 can generate a search tree node representing the selected vehicle state 530, and so on.

[0089] In some examples, the representation 500 can be a visual description of a Determinized Sparse Partially Observable Tree (DESPOT) determined according to a Partially Observable Markov Decision Process (POMDP).

[0090] Figures 6A - 6C A diagrammatic flowchart of an example method 600 is shown, which uses a tree search to generate a trajectory for controlling a vehicle (e.g., the autonomous vehicle 102), the tree search including iteratively determining candidate actions and evaluating corresponding vehicle / environment states to traverse the tree. The example method 600 can be executed by the planning component 112 of the autonomous vehicle 102, but in at least some cases, the example method 600 can additionally or alternatively be executed by the simulation component, the perception component, and / or the prediction component of the autonomous vehicle 102. In some examples, the tree search performed by the guidance component can include performing Monte Carlo Tree Search (MCTS); Partially Observable Monte Carlo Planning (POMCP); Markov Decision Process (MDP), such as Partially Observable MDP (POMDP); or similar techniques improved using the techniques discussed herein, including agent filtering, upper / lower cost estimation, and / or default policies.

[0091] At operation 602, the example method 600 can include receiving route data and sensor data associated with the environment and determining a target vehicle state on a designated route of the vehicle. Figure 6ADepicts an example driving environment where the autonomous vehicle 102 is in an off-route state (e.g., a parking lot, a lane, a roadside, or a roadside turnout, etc.). Based on the current off-route state of the autonomous vehicle 102 and the driving route the vehicle intends to travel, the planning component 112 can determine a target state 604 on the route. As described above, the planning component 112 can determine the target state 604 as a state in the reference trajectory of the driving lane on the vehicle's route. In some examples, the planning component 112 can determine multiple target states associated with different driving lanes where the vehicle may join the route. The target state can be determined based on any combination of the lateral and / or longitudinal distances between the vehicle and the respective target states in the lane reference trajectory, the vehicle's current heading relative to the driving direction of the driving lane, the vehicle's current speed, and / or various characteristics or capabilities of the vehicle (e.g., the vehicle's turning radius and maximum steering angle).

[0092] According to any of the techniques discussed herein, the sensor data received at operation 602 can include any sensor data associated with one or more sensors. The (one or more) sensors can be associated with the autonomous vehicle 102 and / or other computing devices. Operation 602 can also include determining environmental state data at least in part based on the sensor data. In some examples, the perception component 110 can determine environmental state data for any static and / or dynamic objects detected by the perception component. For example, the environmental state data received at operation 602 can include the position, pose, trajectory, and / or other characteristics of the autonomous vehicle 102 and / or any other objects in the environment (e.g., other vehicles, pedestrians, bicycles, buildings, traffic signals, road debris, etc.).

[0093] At operation 606, according to any of the techniques described herein, the planning component 112 can determine a root node 608 of a tree search at least in part based on the sensor data. In some examples, determining the root node can include determining a data structure 610 for the tree search, which can include setting and storing a directed acyclic graph (DAG), upper confidence bounds applied to trees (UCT), deterministic sparse partially observable trees (DESPOT), etc., or similar means for modeling control states and environmental states. The root node 608 can be associated with the current state of the autonomous vehicle 102 at the current time point and / or the latest sensor data or a batch of sensor data. Thus, the root node 608 can be associated with perception data, which may or may not include prediction data, and / or can identify environmental state data that includes the current position, orientation, speed, acceleration, classification, etc. of static and / or dynamic objects in the environment (including similar information about the vehicle, which may be generated by the vehicle's positioning component), and can additionally or alternatively include the same historical data.

[0094] In Figure 6BIn the illustrated operation 612, the planning component 112 can determine a first action node 614 based at least in part on the root node 608. In this example, the graph can depict state nodes (e.g., representing vehicle state and / or environmental state, also referred to as prediction nodes) as squares and candidate action nodes as circles. Dashed lines and / or can represent relationships between nodes that have not yet been discovered / undetermined and / or can represent nodes that have been discarded and will not be further explored during the tree search. According to any of the techniques discussed herein, an action node (e.g., the first action node 614) can represent a candidate action for controlling the movement of the autonomous vehicle 102 (e.g., based at least in part on a previous state node). In this example, the first action node 614 can represent a candidate action determined according to an inertia-based candidate trajectory. Determining the first candidate action can include using a cubic spline function or other mathematical function to determine an action for moving the vehicle in a kinematically feasible manner (e.g., without reference to the structure of the driving route) between the current off-route state and the target state 604.

[0095] In operation 616, according to any of the techniques described herein, the planning component 112 can determine a second action node 618 and a third action node 620 based on the root node, where the two action nodes represent other possible candidate actions for controlling the movement of the autonomous vehicle 102. In this example, the second action node 618 and the third action node 620 can represent route-based candidate actions, candidate actions based on a previous control trajectory of the autonomous vehicle 102, and / or candidate actions generated by perturbing other candidate actions using perturbation parameters.

[0096] At operation 622, the planning component 112 can use one or more cost functions to select a candidate action (or candidate actions) for further exploration in the tree search and / or to determine a subsequent vehicle state node 624 based on the selected candidate action. According to any of the techniques discussed herein, the cost evaluator 320 can be used to perform the cost determination and / or the selection of candidate actions. For example, a separate cost can be calculated for each candidate action (e.g., represented by the first action node 614, the second action node 618, and the third action node 620) at least in part based on various sub-costs including proximity cost, safety cost, comfort cost, and / or progress cost. These sub-costs can be at least in part based on the environmental state data indicated by the last vehicle state node (whether the last vehicle state node is the root node 608 or another vehicle state node). For example, the proximity cost can be at least in part based on the minimum distance, average distance, or other distance between the candidate action and static and / or dynamic objects. The safety cost can include a score that indicates compliance with road rules, proximity to other objects, and / or speed associated with the candidate action (e.g., the safety cost may penalize candidate actions that are close (e.g., within a threshold distance) to an object and moving at high speed, while not penalizing or imposing only a small penalty on candidate actions that are close to an object but moving at a low speed, and high-speed candidate actions that are far from other objects may not be penalized by this cost), and / or proximity to non-drivable surfaces (e.g., sidewalks, buildings, closed lanes). In an example where the safety cost includes a variable cost based on speed and lateral distance to an object, the cost can be determined at least in part based on a hinge function (e.g., L1 or L2 hinge function). In some examples, the hinge point at which the penalty starts to be applied in the hinge function can be based on the distance to the object, the speed associated with the candidate action, the object trajectory, and / or the object type. For example, the penalty may start to be applied from a place where the distance to a cyclist is greater than the distance to a vehicle, and / or the penalty for a cyclist may be higher / more severe than the penalty for a vehicle. Additionally, once the candidate action is within the threshold distance (e.g., the hinge point of the hinge function) of the vehicle, the faster the speed associated with the candidate action, the more severe the penalty may be. In at least one example, the threshold distance at which the penalty is applied as specified by the L1 or L2 hinge function can be at least in part based on the speed associated with the candidate action. In other words, a fast candidate action will have a penalty applied at a greater distance from the object than a slow candidate action, and compared to the same distance of the slow candidate action to the object, the closer the fast candidate action is to the object, the more severe the L1 or L2 penalty may become (e.g., in the case of L1, the slope is steeper, the coefficient, and / or the squared value is larger).

[0097] In some examples, the comfort cost(s) can be at least partially based on the speed, jerk, and / or acceleration associated with a candidate action, and / or whether the candidate action would violate a threshold jerk and / or acceleration. The progress cost(s) can be at least partially based on the completion of a task or sub-goal, and / or the displacement of the autonomous vehicle 102 along a reference trajectory (e.g., a route-based reference trajectory and / or an inertia-based reference trajectory). For example, if a candidate action is executed, the progress cost(s) can reward the autonomous vehicle 102 for traveling further along the trajectory. The cost that is calculated as a reward can have an opposite sign to other sub-costs. For example, if the cost for a candidate action violating a comfort metric (e.g., the candidate action will exceed a threshold jerk) is positive, the reward can be a negative sub-cost. For more details on how to determine the costs, see U.S. Patent Application No. 16 / 872,284, filed on May 11, 2020, the entire content of which is incorporated herein by reference.

[0098] In at least one example, the cost associated with a particular action node can include a reach cost (e.g., for any action node deeper than the first layer, the sum of the costs of all action nodes leading to that action node), the cost of performing the action (e.g., which can include the costs discussed above, such as comfort costs, progress costs, etc.), and the cost of further advancing after that action node, which can also be characterized as the cost of transitioning to a different state in the future. Modeling this future cost (also referred to as the cost-to-go) can be complex and require significant computational power when considering the number of action nodes explored in a tree search. In reinforcement learning, this cost-to-go is also referred to as the "value" of being in a particular state.

[0099] In some examples, the first action node 614, the second action node 618, and the third action node 620 can each be associated with controlling the vehicle during a first time period. As described below, candidate actions in a layer deeper than the layer associated with the first action node 614 (e.g., including action nodes 618 and 620) can be associated with controlling the vehicle during a second time period. In some examples, the time periods associated with subsequent layer action nodes can be equal, or, in additional or alternative examples, the length of the time period can increase (e.g., exponentially or logarithmically). For example, a first set of candidate actions (including the first action node 614) may be associated with controlling the vehicle for 1 second, a second set of candidate actions (e.g., including action nodes 628, 632, and 634), which is one layer deeper than the first layer, may control the vehicle for 1.1 seconds, and the third layer may control the vehicle for 1.25 seconds, and so on. This increasing time period can ensure that upcoming actions are given higher precision and / or accuracy, while also ensuring that actions further away do not control the vehicle in a way that results in higher costs / negative outcomes.

[0100] In Figure 6C operation 626 shown, the planning component 112 can determine another first action node 628 based at least in part on a previously selected vehicle state node 624. Consistent with any of the techniques described herein, the first action node 628, like the previous action node layer, can represent a candidate action for controlling the movement of the autonomous vehicle 102 based at least in part on the vehicle state node 624. In this example, the first action node 628 can represent an inertia-based candidate action determined using a cubic spline function based on vehicle state data and / or environmental state data associated with the vehicle state node 624 for controlling the autonomous vehicle 102 during a time period representative of a second set of candidate actions.

[0101] In operation 630, the planning component 112 can determine additional second action nodes 632 and third action nodes 634 based on subsequent vehicle state nodes 624, consistent with any of the techniques described herein, to represent additional possible candidate actions for controlling the movement of the autonomous vehicle 102. In this example, the second action nodes 632 and third action nodes 634 can represent route-based candidate actions, perturbed candidate actions, etc. determined by the autonomous vehicle 102 based on the subsequent vehicle state nodes 624.

[0102] In operation 636, the planning component 112 can use one or more cost functions to select candidate actions for further exploration in the tree search and / or determine additional subsequent vehicle state nodes 638 based on the selected candidate actions (e.g., third action node 634). As in the previous example, a cost evaluator 320 can be used to determine costs and / or select candidate actions based on various sub-costs (including proximity cost, safety cost, comfort cost, and / or progress cost, etc.) consistent with any of the techniques discussed herein. Although only two layers of candidate actions (and representative action nodes) and the selected vehicle state nodes are shown in this example, it will be understood that the planning component 112 can iteratively perform similar or identical functions for any number of layers of candidate action nodes until an expected final state of the autonomous vehicle 102 is reached.

[0103] Figure 7 is a block diagram of an example system 700 for implementing the techniques described herein. In at least one example, system 700 can include a vehicle, such as vehicle 702. Vehicle 702 can include one or more vehicle computing devices 704, one or more sensor systems 706, one or more transmitters 708, one or more network interfaces 710, at least one direct connection 712, and one or more drive systems 714.

[0104] The vehicle computing device 704 can include one or more processors 716 and a memory 718 communicatively coupled to the processor(s) 716. In the illustrated example, vehicle 702 is an autonomous vehicle; however, vehicle 702 can be any other type of vehicle (such as a semi-autonomous vehicle), or any other system having a travel trajectory planning / navigation function. For example, vehicle 702 can be similar to or the same as autonomous vehicle 102 described above. In some instances, autonomous vehicle 702 can be an autonomous vehicle configured to operate according to a level 5 classification issued by the National Highway Traffic Safety Administration, which describes a vehicle capable of performing all safety-critical functions throughout a trip, where the driver (or occupant) is not expected to control the vehicle at any time. However, in other examples, autonomous vehicle 702 can be a fully autonomous or partially autonomous vehicle having any other level or classification.

[0105] In the illustrated example, the memory 718 of the vehicle computing device 704 stores a positioning component 720, a perception component 722, one or more map components 724 (or map data), one or more system controllers 726, a prediction component 728, and a planning component 730, and the planning component 730 includes a tree search component 302, an inertia-based trajectory generator 304, a route-based trajectory generator 306, and a trajectory optimizer 310. Although depicted as residing in the memory 718 for illustrative purposes in Figure 7 it can be contemplated that the positioning component 720, the perception component 722, the map component 724, the system controller 726, the prediction component 728, the planning component 730, the tree search component 302, the inertia-based trajectory generator 304, the route-based trajectory generator 306, and the trajectory optimizer 310 can be additionally or alternatively accessed by vehicle 702 (e.g., stored on a memory 738 remote from vehicle 702 or otherwise accessible thereto, e.g., the memory 738 of one or more computing devices 734). In some examples, the memory 738 can include one or more prediction models 740, one or more search strategies 742, and / or one or more cost models / functions 744.

[0106] In at least one example, the localization component 720 can include the functionality of receiving sensor data from the sensor system 706 to determine the position and / or orientation of the vehicle 702 (e.g., one or more of x-, y-, z- position, roll, pitch, or yaw). For example, the localization component 720 can include and / or request / receive a map of the environment, such as a map from one or more map components 724, and can continuously determine the position and / or orientation of the vehicle 702 within the environment. In some cases, the localization component 720 can utilize SLAM (Simultaneous Localization and Mapping), CLAMS (Concurrent Calibration, Localization, and Mapping), relative SLAM, bundle adjustment, non-linear least squares optimization, etc., to receive image data, lidar data, radar data, inertial measurement unit (IMU) data, GPS data, wheel encoder data, etc., to accurately determine the position of the vehicle 702. In some cases, as discussed herein, the localization component 720 can provide data to various components of the vehicle 702 to determine the initial position of the vehicle 702 for determining the relevance of an object to the vehicle 702.

[0107] In some cases, the perception component 722 can include the functionality of performing object detection, segmentation, and / or classification. In some examples, the perception component 722 can provide processed sensor data that indicates the presence and / or classification of an object (e.g., an entity) approaching the vehicle 702 as an object type (e.g., car, pedestrian, cyclist, animal, building, tree, road surface, curb, sidewalk, unknown, etc.). In some examples, the perception component 722 can provide processed sensor data that indicates the presence and / or classification of a stationary entity approaching the vehicle 702 as a type (e.g., building, tree, road surface, curb, sidewalk, unknown, etc.). In additional or alternative examples, the perception component 722 can provide processed sensor data that indicates one or more features associated with a detected object (e.g., a tracked object) and / or the environment in which the object is located. In some examples, features associated with an object can include, but are not limited to: x-position (global position and / or local position), y-position (global position and / or local position), z-position (global position and / or local position), orientation (e.g., roll, pitch, yaw), object type (e.g., classification), speed of the object, acceleration of the object, size (dimensions) of the object, etc. Features associated with the environment can include, but are not limited to: the presence of another object in the environment, the state of another object in the environment, time of day, day of the week, season, weather conditions, indication of darkness / light, etc.

[0108] The memory 718 may further include one or more map components 724, which may be used by the vehicle 702 to navigate within the environment. For the purposes of this discussion, a map component may be any number of data structures modeled in two, three, or N dimensions, capable of providing information about the environment, such as but not limited to topology (e.g., intersections), streets, mountains, roads, terrain, and the general environment. In some cases, the map may include but is not limited to: texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), etc.), intensity information (e.g., lidar information, radar information, etc.), spatial information (e.g., image data projected onto a grid, individual "bins" (e.g., polygons associated with individual colors and / or intensities)), reflectivity information (e.g., specular reflection information, retroreflective information, BRDF information, BSSRDF information, etc.). In one example, the map may include a three-dimensional grid of the environment. In some examples, the vehicle 702 may be controlled at least in part based on the one or more map components 724. That is, the one or more map components 724 may be used in conjunction with the localization component 720, the perception component 722, the prediction component 728, and / or the planning component 730 to determine the position of the vehicle 702, detect objects in the environment, generate a route, determine actions and / or trajectories for navigating in the environment.

[0109] In some examples, the one or more map components 724 may be stored on one or more remote computing devices (e.g., computing device 734) that may be accessed via the one or more networks 732. In some examples, multiple map components 724 may be stored based on, for example, characteristics (e.g., type of entity, time of day, day of the week, season of the year, etc.). Storing multiple map components 724 may have similar memory requirements, but increases the speed at which data in the map may be accessed.

[0110] In at least one example, the vehicle computing device 704 may include one or more system controllers 726, which may be configured to control the steering, propulsion, braking, safety, transmitter, communication, and other systems of the vehicle 702. The one or more system controllers 726 may communicate with and / or control the corresponding systems of the one or more drive systems 714 and / or other components of the vehicle 702.

[0111] The prediction component 728 may generate one or more probability maps that represent the predicted probabilities of the possible locations of one or more objects in the environment. For example, the prediction component 728 may generate one or more probability maps for vehicles, pedestrians, animals, etc. within a threshold distance of the vehicle 702. In some cases, the prediction component 728 may measure the tracks of objects and generate discrete prediction probability maps, heat maps, probability distributions, discrete probability distributions, and / or trajectories of the objects based on the observed behavior and the predicted behavior. In some cases, one or more probability maps may represent the intentions of one or more objects in the environment.

[0112] In some examples, the prediction component 728 may include one or more ML prediction models, such as the active prediction model described above. The prediction component 728 may generate predicted trajectories of objects (e.g., objects) in the environment. For example, the prediction component 728 may generate one or more predicted trajectories for objects within a threshold distance of the vehicle 702. In some examples, the prediction component 728 may measure the trajectories of objects and generate the trajectories of the objects based on the observed behavior and the predicted behavior.

[0113] The planning component 730 may include various components and functions similar or identical to the planning component 112 described above. As described above, the planning component 730 may determine a trajectory for the vehicle 702 to traverse the environment to perform a driving operation from an off-route state to an on-route state. In various examples, the planning component 730 may determine various routes and trajectories at various levels of detail. For example, the planning component 730 may determine various trajectories for traveling from a first location (e.g., the current off-route location) to a second location (e.g., the target location on the route). For the purposes of discussion, a route may include a series of waypoints in a structured environment (e.g., including streets, intersections, etc.). The planning component 730 may generate instructions for guiding the vehicle 702 from the current off-route driving state to the specified on-route target state of the vehicle. In at least one example, the planning component 730 may determine how to guide the vehicle 702 from a first off-route state in a sequence of states to a second state in the sequence of states. In some examples, the instructions may be a candidate trajectory or a part of a trajectory. In some examples, multiple trajectories may be generated substantially simultaneously (e.g., within the technical tolerance) according to the receding horizon technique. One trajectory with the highest confidence level among the multiple trajectories in the receding data horizon may be selected to operate the vehicle. In various examples, the planning component 730 may select a trajectory for the vehicle 702.

[0114] In other examples, the planning component 730 may alternatively or additionally use data from the localization component 720, the perception component 722, the map component(s) 724, and / or the prediction component 728 to determine a trajectory for the vehicle 702 to follow through the environment. For example, the planning component 730 may receive data about objects associated with the environment (e.g., object data) from the localization component 720, the perception component 722, and / or the prediction component 728. In some examples, the planning component 730 receives data about relevant objects within the environment. Using this data, the planning component 730 may determine driving operations to travel from a first location (e.g., the current location) to a second location (e.g., a target location on a route) to avoid objects in the environment. In at least some examples, such a planning component 730 may determine that no such collision-free trajectory exists and, in turn, provide a trajectory that enables the vehicle 702 to safely enter a target state while avoiding all collisions and / or otherwise mitigating damage.

[0115] In some cases, aspects of some or all of the components discussed herein may include any model, technique, and / or machine learning technique. For example, in some cases, the components in the memory 718 (and the memory 738 discussed below) may be implemented as neural networks.

[0116] As described herein, an exemplary neural network is a technique that passes input data through a series of connected layers to produce an output. Each layer in a neural network may also include another neural network or may include any number of layers (whether convolutional or non-convolutional). As can be understood in the context of the present disclosure, neural networks may utilize machine learning, which may refer to a broad class of such techniques that generate outputs based on learned parameters.

[0117] Although discussed in the context of neural networks, any type of machine learning consistent with this disclosure may be used. For example, machine learning techniques may include, but are not limited to: regression techniques (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), instance-based techniques (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least angle regression (LARS)), decision tree techniques (e.g., classification and regression tree (CART), iterative dichotomiser 3 (ID3), chi-squared automatic interaction detection (CHAID), decision stump, conditional decision tree), Bayesian techniques (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, averaged one-dependent estimators (AODE), Bayesian belief network (BNN), Bayesian network), clustering techniques (e.g., k-means, k-medians, expectation maximization (EM), hierarchical clustering), association rule learning techniques (e.g., perceptron, backpropagation, Hopfield network, radial basis function network (RBFN)), deep learning techniques (e.g., deep Boltzmann machine (DBM), deep belief network (DBN), convolutional neural network (CNN), stacked autoencoder), dimensionality reduction techniques (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), mixture discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble techniques (e.g., boosting, bagging, adaptive boosting (AdaBoost), stacking, gradient boosting machine (GBM), gradient boosting regression tree (GBRT), random forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc.). Additional examples of architectures include neural networks such as ResNet50, ResNet101, VGG, DenseNet, PointNet, etc.

[0118] In at least one example, the (one or more) sensor systems 706 can include: lidar sensors, radar sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement unit (IMU), accelerometer, magnetometer, gyroscope, etc.), cameras (e.g., RGB, IR, intensity, depth, time-of-flight, etc.), microphones, wheel encoders, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), and the like. The (one or more) sensor systems 706 can include multiple instances of each of these or other types of sensors. For example, the lidar sensors can include individual lidar sensors located at the corners, front, rear, sides, and / or top of the vehicle 702. As another example, the camera sensors can include multiple cameras disposed at various positions around the exterior and / or interior of the vehicle 702. The (one or more) sensor systems 706 can provide inputs to the vehicle computing device 704. Additionally, or in an alternative, the sensor systems 706 can send sensor data to one or more computing devices 734 via one or more networks 732 after a predetermined period of time, near real-time, etc.

[0119] The vehicle 702 can also include one or more transmitters 708 for emitting light and / or sound. The transmitter 708 can include an internal audio transmitter and an internal visual transmitter to communicate with the passengers of the vehicle 702. By way of example and not limitation, the internal transmitters can include speakers, lights, signs, displays, touchscreens, haptic transmitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seatbelt tensioners, seat positioners, headrest positioners, etc.), and the like. The transmitter 708 can also include external transmitters. By way of example and not limitation, the external transmitters can include lights indicating the direction of travel or other indicators of vehicle actions (e.g., indicator lights, signs, light arrays, etc.), and one or more audio transmitters (e.g., speakers, speaker arrays, horns, etc.) for auditory communication with pedestrians or other nearby vehicles, one or more of which includes beam steering technology.

[0120] Vehicle 702 may also include one or more network interfaces 710 (or communication connections) that enable communication between vehicle 702 and one or more other local or remote computing devices. For example, network interface 710 may enable communication with (one or more) other local computing devices on vehicle 702 and / or drive system 714. Moreover, network interface 710 may allow the vehicle to communicate with other nearby computing devices (e.g., computing device 734, other nearby vehicles, etc.) and / or one or more remote sensor systems to receive sensor data. (One or more) network interfaces 710 may also enable vehicle 702 to communicate with a remote operation computing device or other remote services.

[0121] Network interface 710 may include a physical interface and / or a logical interface for connecting vehicle computing device 704 to another computing device or network, such as network 732. For example, network interface 710 may implement Wi-Fi-based communication, such as the frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies such as cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.) or any suitable wired or wireless communication protocol that enables the corresponding computing device to interact with other computing devices.

[0122] In at least one example, vehicle 702 may include one or more drive systems 714. In some examples, vehicle 702 may have a single drive system 714. In at least one example, if vehicle 702 has multiple drive systems 714, then a single drive system 714 may be positioned on opposite ends (e.g., front and rear, etc.) of vehicle 702. In at least one example, (one or more) drive systems 714 may include one or more sensor systems to detect conditions of (one or more) drive systems 714 and / or the surrounding environment of vehicle 702. By way of example and not limitation, the sensor system may include: one or more wheel encoders (e.g., rotary encoders) for sensing the rotation of the wheels of the drive module, inertial sensors (e.g., inertial measurement unit, accelerometer, gyroscope, magnetometer, etc.) for measuring the orientation and acceleration of the drive module, cameras or other image sensors, ultrasonic sensors for acoustically detecting objects in the surrounding environment of the drive module, lidar sensors, radar sensors, etc. Some sensors, such as wheel encoders, may be unique to (one or more) drive systems 714. In some cases, the sensor system on drive system 714 may overlap or supplement the corresponding system of vehicle 702 (e.g., sensor system 706).

[0123] The drive system 714 can include many systems in a vehicle system, including: a high-voltage battery, a motor for propelling the vehicle, an inverter for converting direct current from the battery into alternating current for use by other vehicle systems, a steering system including a steering motor and a steering rack (which can be electric), a braking system including a hydraulic actuator or an electric actuator, a suspension system including hydraulic components and / or pneumatic components, a stability control system for distributing braking force to mitigate loss of traction and maintain control, an HVAC system, a lighting system (e.g., lighting such as headlights / taillights for illuminating the external surroundings of the vehicle), and one or more other systems (e.g., a cooling system, a safety system, an on-vehicle charging system, other electrical components such as a DC / DC converter, high-voltage connectors, high-voltage cables, a charging system, a charging port, etc.). Additionally, the (one or more) drive system 714 can include a drive module controller, which can receive and preprocess data from the (one or more) sensor systems and control the operation of various vehicle systems. In some examples, the drive module controller can include one or more processors and a memory communicatively coupled to the one or more processors. The memory can store one or more modules to perform various functions of the drive system 714. Further, the drive system 714 can also include one or more communication connections that enable the respective drive module to communicate with one or more other local or remote computing devices.

[0124] In at least one example, the direct connection 712 can provide a physical interface to couple one or more drive systems 714 to the body of the vehicle 702. For example, the direct connection 712 can allow for the transfer of energy, fluid, air, data, etc. between the (one or more) drive systems 714 and the vehicle. In some cases, the direct connection 712 can further releasably secure the (one or more) drive systems 714 to the body of the vehicle 702.

[0125] In at least one example, the localization component 720, the perception component 722, the map component 724, the system controller 726, the prediction component 728, the planning component 730, the tree search component 302, the inertia-based trajectory generator 304, the route-based trajectory generator 306, and the trajectory optimizer 310 can process sensor data (as described above) and can send their respective outputs to the computing device 734 via one or more networks 732. In at least one example, the localization component 720, the perception component 722, the map component 724, the system controller 726, the prediction component 728, the planning component 730, the tree search component 302, the inertia-based trajectory generator 304, the route-based trajectory generator 306, and the trajectory optimizer 310 can send their respective outputs to the computing device 734 at a specific frequency, after a predetermined period of time has elapsed, near real-time, etc.

[0126] In some examples, vehicle 702 may send sensor data to one or more computing devices 734 via one or more networks 732. In some examples, vehicle 702 may receive sensor data from one or more computing devices 734 and / or one or more remote sensor systems via one or more networks 732. The sensor data may include raw sensor data and / or processed sensor data and / or a representation of the sensor data. In some examples, the (raw or processed) sensor data may be sent and / or received as one or more log files.

[0127] One or more computing devices 734 may include one or more processors 736 and a memory 738, and the memory 738 may include one or more prediction models 740, one or more search strategies 742, and / or one or more cost models / functions 744. In some examples, one or more computing devices 734 may store various prediction models 740, search strategies 742, and / or cost models / functions 744, which may be associated with various different autonomous vehicle models (e.g., having different capabilities and kinematic feasible trajectories), different driving environments (e.g., regions, driving scenario types, etc.), and / or different driving conditions (e.g., traffic conditions, road conditions, weather conditions, etc.). In these examples, computing device 734 may be configured to provide a combination of various mathematical functions and / or heuristics for generating an inertia-based reference trajectory and / or a route-based trajectory, as well as a search exploration strategy and / or a cost estimator component associated with various different vehicles (e.g., 702), depending on the type, model, characteristics, current driving environment, current driving conditions, etc. of the vehicle. Additionally, in some examples, memory 738 may store one or more components similar to the one or more components stored in memory 718 of vehicle 702. In such examples, one or more computing devices 734 may be configured to perform one or more methods described herein with respect to vehicle 702.

[0128] One or more processors 716 of vehicle 702 and one or more processors 736 of one or more computing devices 734 can be any suitable processors capable of executing instructions to process data and perform the operations described herein. By way of example and not limitation, a processor can include one or more central processing units (CPUs), a graphics processing unit (GPU), or any other device or portion of a device that processes electronic data to transform that electronic data into other electronic data that can be stored in registers and / or memory. In some examples, an integrated circuit (e.g., ASIC, etc.), a gate array (e.g., FPGA, etc.), and other hardware devices can also be considered processors configured to implement encoded instructions.

[0129] Memories 718 and 738 are examples of non-transitory computer-readable media. Memories 718 and 738 can store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods described herein and the functions attributable to the various systems. In various embodiments, the memories can be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash-type memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein can include many other logic devices, programming components, and physical components, and those shown in the figures are merely examples relevant to the discussion herein.

[0130] It should be noted that, although Figure 7 is shown as a distributed system, in alternative examples, the components of vehicle 702 can be associated with one or more computing devices 734 and / or the components of one or more computing devices 734 can be associated with vehicle 702. That is, vehicle 702 can perform one or more functions associated with one or more computing devices 734, and vice versa.

[0131] Figure 8 is a flowchart showing an example method 800 for determining and executing a trajectory to control a vehicle from an initial off-route state to a target state on a specified driving route. As described herein, the operations of method 800 can be performed by a planning component 112 (including various sub-components) and / or by calling relevant components running within autonomous vehicle 102 (e.g., including Figure 3 the components described therein).

[0132] At operation 802, the autonomous vehicle 102 may receive sensor data via the sensing component 110 to determine the current vehicle state in the driving environment. Based on the sensor data received via the sensing component 110, the planning component 112 may determine that the current vehicle state is a driving position on the route or an off-route position, such as a parking position, a passenger pick-up / drop-off position, a stopping position (e.g., for an emergency vehicle, etc.). Additionally, based on the vehicle's expected destination (e.g., a desired position communicated by a passenger or a dispatcher to the vehicle computing system), the planning component 112 may determine the driving route for the vehicle to reach the expected destination.

[0133] At operation 804, the planning component 112 may determine whether the current vehicle state is an on-route state. As described above, the on-route state may be a state where the vehicle is located on the road surface (e.g., within a threshold distance of the lane reference trajectory) and / or a state where the vehicle heading corresponds to the directionality of the driving lane / road (e.g., within a threshold heading angle of the lane direction). Based on determining that the current vehicle state is an on-route state (804: Yes), then at operation 806, the planning component 112 may control the autonomous vehicle at least in part based on the driving route structure (e.g., using the reference trajectory of the driving lane, waypoints connecting roads / lane segments, etc.).

[0134] In other examples, when determining that the current vehicle state is an off-route state (804: No), at operation 808, the planning component 112 may determine the target vehicle state on the driving lane of the driving route associated with the vehicle. For example, the planning component 112 may identify the portion of the driving route (e.g., road and / or driving lane) through which the vehicle can pass based on the current vehicle state, and determine a specific state in the lane reference trajectory of the road as the target state for the autonomous vehicle 102. As described above, the planning component 112 may determine the target state based on, for example, the lateral and / or longitudinal positions of the respective target states in the lane reference trajectory relative to the current vehicle state, the current heading of the vehicle relative to the driving direction of the driving lane, the current speed of the vehicle, and / or various characteristics or capabilities of the vehicle (e.g., turning radius, maximum steering angle, etc.).

[0135] At operation 810, the planning component 112 may determine a first candidate trajectory according to inertia (or an inertia-based reference system). In various examples, the planning component 112 may use a mathematical function (e.g., cubic spline curve, Bezier curve, a series of clothoid curves, etc.) to determine the first candidate trajectory based on the relative position, heading, and speed related to the current vehicle state and the target state. As described above, the first candidate trajectory determined at operation 810 may be determined without referring to the lane reference trajectory of the driving lane / road of the driving route.

[0136] At operation 812, the planning component 112 may determine whether additional candidate trajectories need to be generated to control the vehicle to travel between the current off-route state and the target state on the route. In some examples, the planning component 112 may set one or more thresholds for generating a specific number of candidate trajectories and / or a specific type of candidate trajectories. These thresholds may be based on the current position and / or heading of the vehicle relative to the target state or the road. For example, the planning component may use various heuristics to determine when the current vehicle state is relatively close to the target state (e.g., within a distance threshold, within a heading difference threshold, and / or within a speed difference threshold), or when it is relatively far from the target state. Based on such heuristics, the planning component 112 may determine the total number (e.g., minimum number) of candidate trajectories to be generated and the type of candidate trajectories to be generated (e.g., the number of route-based trajectories, the number of inertia-based trajectories, the number of perturbed route-based trajectories, the number of perturbed inertia-based trajectories, etc.).

[0137] When the planning component 112 determines that additional candidate trajectories should be generated for the tree search (812: Yes), then at operation 814, the planning component 112 may generate additional trajectories. As described above, the planning component 112 may use one or more of the inertia-based candidate action generator 304, the route-based candidate action generator 306, and / or the perturbed candidate action generator 308 to generate a dense set of candidate trajectories of the desired type.

[0138] After the planning component 112 has generated a sufficient number of candidate trajectories for the tree search (812: No), at operation 816, the planning component 112 may determine the candidate actions associated with the respective candidate trajectories. For example, the candidate actions may include refined vehicle control instructions such as speed, speed change, steering angle, steering angle change, yaw rate, yaw rate change, etc., which may be output by the planning component 112 and executed by the vehicle's drive system.

[0139] At operation 818, the planning component 112 may generate a search tree and determine a minimum-cost traversal (or multiple traversals) based on the candidate actions associated with the candidate trajectories. As described above, any number of tree search techniques may be used, during which the planning component 112 may evaluate a set of candidate trajectories between the current off-route vehicle state and the target state on the route. Various tree search techniques may iteratively determine a set of candidate actions based on the candidate trajectories, predict the potential future states of the vehicle (and the environment) in response to the candidate actions, and use a cost function to evaluate the candidate actions based on the predicted future states of the vehicle and the overall driving environment. Finally, at operation 820, the planning component 112 may use the output of the tree search to determine the control trajectory (or trajectories) that the autonomous vehicle 102 can follow to perform the operation of traveling from off-route to on-route and joining the driving route.

[0140] The methods described herein represent sequences of operations that can be implemented in hardware, software, or a combination thereof. In the context of software, these blocks represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the recited operations. In general, computer-executable instructions can include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and / or in parallel to implement the process. In some examples, one or more operations of the method can be entirely omitted. For example, the operations can include determining a first action and a second action of a vehicle relative to a selected trajectory without determining the corresponding costs of one or more actions of the vehicle. Additionally, the methods described herein can be combined with each other in whole or in part or with other methods.

[0141] The various techniques described herein can be implemented in the context of computer-executable instructions or software, such as program modules, that are stored in computer-readable storage and executed by a processor of one or more computing devices such as those shown in the figures. In general, program modules include routines, programs, objects, components, data structures, etc. and define the operational logic for performing particular tasks or implementing particular abstract data types.

[0142] Other architectures can be used to implement the described functionality and are intended to fall within the scope of the present disclosure. Additionally, while specific distributions of responsibilities have been defined above for purposes of discussion, the various functions and responsibilities can be distributed and divided in different ways depending on the circumstances.

[0143] Similarly, software can be stored and distributed in various ways and using different means, and the above-described specific software storage and execution configurations can be changed in many different ways. Accordingly, the software implementing the above techniques can be distributed on various types of computer-readable media, not limited to the forms of memory specifically described.

[0144] Example Clauses

[0145] A. A vehicle, comprising: one or more processors; and one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform the following operations, including: determining that there are no lane constraints associated with the current vehicle state of the vehicle; determining a target vehicle state of the vehicle associated with a portion of an environment having lane constraints; determining a first candidate trajectory of the vehicle at least in part based on: the current vehicle state, the target vehicle state, the distance between the current vehicle state and the driving route, the heading associated with the current vehicle state, and the direction associated with the target vehicle state; determining a second candidate path of the vehicle at least in part based on the current vehicle state and the target vehicle state, wherein determining the second candidate path is at least in part based on a route-based reference system associated with the lane constraints; determining a first candidate action for controlling the movement of the vehicle at least in part based on the first candidate path; determining a second candidate action for controlling the movement of the vehicle at least in part based on the second candidate path; generating a tree structure at least in part based on the first candidate action and the second candidate action; determining a traversal of the tree structure associated with a minimum cost; determining a control path of the vehicle at least in part based on the traversal of the tree structure; and controlling the vehicle at least in part based on the control path.

[0146] B. The vehicle according to clause A, wherein the operations further include: determining a third candidate path of the vehicle, wherein determining the third candidate path includes perturbing at least one of the first candidate path or the second candidate path; determining a third candidate action for controlling the movement of the vehicle at least in part based on the third candidate path; wherein generating the tree structure is also at least in part based on the third candidate action.

[0147] C. The vehicle according to clause A, wherein the operations further include: receiving a previous control path associated with a previous planning cycle of the vehicle as the third candidate path; and determining a third candidate action for controlling the movement of the vehicle at least in part based on the third candidate path, wherein generating the tree structure is also at least in part based on the third candidate action.

[0148] D. The vehicle according to clause A, wherein generating the tree structure includes: associating the current vehicle state with a first node of the tree structure; determining a cost associated with the first candidate action; determining the first candidate action for further exploration in the tree structure at least in part based on the cost; and determining a second node of the tree structure at least in part based on the first candidate action.

[0149] E. The vehicle according to clause D, wherein the cost is associated with one or more of the following: safety cost, schedule cost, comfort cost, or energy efficiency cost.

[0150] F. A method, comprising: determining a first candidate trajectory of a vehicle at least in part based on a current vehicle state and a target vehicle state, wherein the target vehicle state is on a driving lane of a driving route associated with the vehicle and the current vehicle state is deviated from the driving lane; determining a first candidate action for controlling the movement of the vehicle at least in part based on the first candidate trajectory; generating a tree structure at least in part based on the first candidate action; determining a control trajectory of the vehicle at least in part based on the tree structure; and controlling the vehicle at least in part based on the control trajectory.

[0151] G. The method according to clause F, wherein determining the first candidate trajectory is at least in part based on an inertia-based reference system, and wherein the method further comprises: determining a second candidate trajectory at least in part based on a route-based reference system associated with the driving lane.

[0152] H. The method according to clause F, wherein determining the first candidate trajectory is at least in part based on an inertia-based reference system, and wherein the method further comprises: perturbing the first candidate trajectory by using at least one of the following to determine a second candidate trajectory: a lateral offset parameter; or a speed scaling parameter.

[0153] I. The method according to clause F, wherein determining the first candidate trajectory is at least in part based on an inertia-based reference system, and wherein the method further comprises: determining a second candidate trajectory at least in part based on a previous control trajectory associated with a previous planning period of the vehicle.

[0154] J. The method according to clause F, wherein the first candidate trajectory comprises: a cubic spline curve; an S-shaped curve; a Bezier curve; or a series of clothoid curves.

[0155] K. The method according to clause F, wherein determining the control trajectory comprises determining a traversal of the tree structure associated with a minimum cost.

[0156] L. The method according to clause F, wherein generating the tree structure comprises: associating the current vehicle state with a first node of the tree structure; determining a cost associated with the first candidate action; selecting the first candidate action at least in part based on the cost for further exploration in the tree structure; and determining a second node of the tree structure at least in part based on the first candidate action.

[0157] M. The method according to clause F, wherein the cost is associated with one or more of the following: safety cost, schedule cost, comfort cost, or energy efficiency cost.

[0158] N. One or more non - transitory computer - readable media storing instructions executable by a processor, wherein when the instructions are executed, they cause the processor to perform the following operations, which include: determining a first candidate trajectory of a vehicle at least in part based on a current vehicle state and a target vehicle state, wherein the target vehicle state is in a driving lane of a driving route associated with the vehicle, and the current vehicle state is deviated from the driving lane; determining a first candidate action for controlling the movement of the vehicle at least in part based on the first candidate trajectory; generating a tree structure at least in part based on the first candidate action; determining a control trajectory of the vehicle at least in part based on the tree structure; and controlling the vehicle at least in part based on the control trajectory.

[0159] O. The one or more non - transitory computer - readable media according to clause N, wherein determining the first candidate trajectory is at least in part according to an inertia - based reference system, and wherein the method further includes: determining a second candidate trajectory at least in part based on a route - based reference system associated with the driving lane.

[0160] P. The one or more non - transitory computer - readable media according to clause N, wherein determining the first candidate trajectory is at least in part according to an inertia - based reference system, and wherein the method further includes: perturbing the first candidate trajectory to determine a second candidate trajectory by using at least one of the following: a lateral offset parameter; or a speed scaling parameter.

[0161] Q. The one or more non - transitory computer - readable media according to clause N, wherein determining the first candidate trajectory is at least in part according to an inertia - based reference system, and wherein the method further includes: determining a second candidate trajectory at least in part based on a previous control trajectory associated with a previous planning period of the vehicle.

[0162] R. The one or more non - transitory computer - readable media according to clause N, wherein the first candidate trajectory includes: a cubic spline curve; an S - shaped curve; a Bezier curve; or a series of clothoid curves.

[0163] S. The one or more non - transitory computer - readable media according to clause N, wherein determining the control trajectory includes determining a traversal of the tree structure associated with a minimum cost.

[0164] T. One or more non-transitory computer-readable media according to clause N, wherein the generation of the tree structure includes: associating the current vehicle state with a first node of the tree structure; determining a cost associated with the first candidate action; selecting, at least in part based on the cost, the first candidate action for further exploration in the tree structure; and determining a second node of the tree structure, at least in part based on the first candidate action.

[0165] Although the example clauses described above are described with respect to specific embodiments, it should be understood that, in the context of this document, the content of the example clauses can be implemented via methods, devices, systems, computer-readable media, and / or another embodiment. Additionally, any one of Examples A - T can be implemented alone or in combination with any other one or more of Examples A - T.

[0166] Conclusion

[0167] Although one or more examples of the techniques described herein have been described, various changes, additions, permutations, and equivalents are included within the scope of the techniques described herein.

[0168] In the description of the examples, reference has been made to the accompanying drawings that form a part thereof, which illustrate specific examples of the claimed subject matter by way of illustration. It should be understood that other examples can be used and changes or alterations such as structural changes can be made. Such examples, changes, or alterations do not necessarily depart from the scope of the subject matter intended to be claimed. Although the steps herein can be presented in a certain order, in some cases, the order can be changed such that certain inputs are provided at different times or in a different order, without changing the functionality of the described systems and methods. The disclosed processes can also be executed in a different order. Additionally, the various calculations herein do not need to be performed in the order disclosed, and other examples using alternative orders of calculation can be easily implemented. In addition to being reordered, the calculations can be broken down into sub-calculations with the same result.

[0169] Although the subject matter of the present invention has been described in language specific to structural features and / or method acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the above specific features or acts. Rather, these specific features and acts are disclosed as example forms of implementing the claims.

[0170] The components described herein represent instructions that may be stored on any type of computer-readable medium and implemented in software and / or hardware. All of the methods and processes described above may be embodied in software code modules and / or computer-executable instructions executed by one or more computers or processors, hardware, or some combination thereof and be fully automated therewith. Some or all of the methods may alternatively be embodied in dedicated computer hardware.

[0171] Unless otherwise expressly stated, conditional language such as “can,” “could,” “may,” or “might” is understood in context to mean that certain examples include certain features, elements, and / or steps, while other examples do not. Thus, such conditional language is generally not intended to imply that certain features, elements, and / or steps are in any way required for one or more examples or that one or more examples must include logic for deciding, with or without user input or prompting, whether to include or to perform certain features, elements, and / or steps in any particular example.

[0172] Unless otherwise expressly stated, conjunctive language such as the phrase “at least one of X, Y, or Z” shall be understood to mean items, terms, etc. that can be X, Y, or Z, or any combination thereof, including multiples of each element. “A” means singular and plural unless expressly described as singular.

[0173] Any routine descriptions, elements, or boxes in the flowcharts described herein and / or depicted in the figures shall be understood potentially to represent modules, segments, or portions of code that include one or more computer-executable instructions for implementing specific logical functions or elements in the routine. Alternative implementations are included within the scope of the examples described herein where elements or functions may be deleted, or executed out of the order shown or discussed, including substantially synchronously, in reverse order, with additional operations, or with omitted operations, depending on the functionality involved, as would be understood by one of ordinary skill in the art.

[0174] Numerous variations and modifications may be made to the examples described above, and the elements thereof are to be understood as being in other acceptable examples. All such modifications and variations are intended to be included within the scope of this disclosure and protected by the following claims.

Claims

1. A vehicle, comprising: one or more processors; and one or more non-transitory computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to perform operations including: determining a first candidate trajectory of the vehicle at least in part based on a current vehicle state and a target vehicle state, wherein the target vehicle state is on a driving lane of a driving route associated with the vehicle and the current vehicle state is deviated from the driving lane; determining a first candidate action for controlling the movement of the vehicle at least in part based on the first candidate trajectory; generating a tree structure at least in part based on the first candidate action; determining a control trajectory of the vehicle at least in part based on the tree structure; and controlling the vehicle at least in part based on the control trajectory.

2. The vehicle according to claim 1, wherein, Determining the first candidate trajectory is at least in part according to an inertia-based reference system, and wherein the operations further include: determining a second candidate trajectory at least in part based on a route-based reference system associated with the driving lane.

3. The vehicle according to claim 1 or claim 2, wherein, Determining the first candidate trajectory is at least in part according to an inertia-based reference system, and wherein the operations further include: determining a second candidate trajectory by perturbing the first candidate trajectory using at least one of: a lateral offset parameter; or a speed scaling parameter.

4. The vehicle according to any one of claims 1 to 3, wherein, Determining the first candidate trajectory is at least in part according to an inertia-based reference system, and wherein the operations further include: determining a second candidate trajectory at least in part based on a previous control trajectory associated with a previous planning cycle of the vehicle.

5. The vehicle according to any one of claims 1-4, wherein, The first candidate trajectory includes: a cubic spline curve; an S-shaped curve; a Bezier curve; or a series of clothoid curves.

6. The vehicle according to any one of claims 1-5, wherein, Determining the control trajectory includes determining a traversal of the tree structure associated with a minimum cost.

7. The vehicle according to any one of claims 1-6, wherein, Generating the tree structure includes: associating the current vehicle state with a first node of the tree structure; determining a cost associated with the first candidate action; selecting the first candidate action for further exploration in the tree structure at least in part based on the cost; and determining a second node of the tree structure at least in part based on the first candidate action.

8. The vehicle according to claim 7, wherein, The cost is associated with one or more of the following: a safety cost, a progress cost, a comfort cost, or an energy efficiency cost.

9. A method, comprising: determining a first candidate trajectory of a vehicle at least in part based on a current vehicle state and a target vehicle state, wherein the target vehicle state is on a driving lane of a driving route associated with the vehicle and the current vehicle state is deviated from the driving lane; determining a first candidate action for controlling the movement of the vehicle at least in part based on the first candidate trajectory; generating a tree structure at least in part based on the first candidate action; determining a control trajectory of the vehicle at least in part based on the tree structure; and controlling the vehicle at least in part based on the control trajectory.

10. The method according to claim 9, wherein, Determine that the first candidate trajectory is at least partially based on an inertia-based reference system, and wherein the method further comprises: Determine a second candidate trajectory based at least in part on a route-based reference system associated with the driving lane.

11. The method according to claim 9 or claim 10, wherein Determine that the first candidate trajectory is at least partially based on an inertia-based reference system, and wherein the method further comprises: Determine a second candidate trajectory by perturbing the first candidate trajectory using at least one of the following: A lateral offset parameter; or A speed scaling parameter.

12. The method according to any one of claims 9-11, wherein, Determine that the first candidate trajectory is at least partially based on an inertia-based reference system, and wherein the method further comprises: Determine a second candidate trajectory based at least in part on a previous control trajectory associated with a previous planning period of the vehicle.

13. The method according to any one of claims 9 - 12, wherein, The first candidate trajectory comprises: A cubic spline curve; An S-curve; A Bezier curve; or A series of clothoid curves.

14. The method according to any one of claims 9 - 13, wherein, Determine that the control trajectory includes determining a traversal of the tree structure associated with a minimum cost.

15. One or more non-transitory computer-readable media comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 9-14.

Images