Cost-Based Routing

Abstract

The vehicle computing system may implement techniques for determining actions for the vehicle to take based on associated costs. The costs may be based in part on the impact of the actions on objects operating in the environment (e.g., another vehicle, a bicyclist, a pedestrian, etc.). The vehicle computing system may detect objects based on sensor data and determine object trajectories based on the predicted reaction of the objects to the vehicle performing the actions. The vehicle computing system may determine costs associated with safety, comfort, progress, and / or operational rules for each action the vehicle can take based on the actions and / or predicted object trajectories. In one example, the action with the lowest cost may be selected for the vehicle to execute.

Description

[Technical field]

[0001] The present invention relates to cost-based routing. [Background technology]

[0002] This PCT international patent application is a continuation of, and claims the benefit of priority to, U.S. patent application Ser. No. 16 / 539,928, entitled "COST-BASED PATH DETERMINATION," filed on August 13, 2019, the entire contents of which are incorporated herein by reference.

[0003] Planning systems in autonomous and semi-autonomous vehicles determine actions to take for the vehicle in an operating environment. The actions for the vehicle may be determined in part based on avoiding objects present in the environment. For example, an action may be generated to go around a double parked vehicle, change lanes to avoid another vehicle on the road, etc. Conventional planning systems may select an action for the vehicle based on a determination that the action is the most conservative action. However, such conventional planning systems may adopt a vehicle-centric view of the environment that does not consider the impact of the vehicle's actions on other objects in the environment. As such, conventional planning systems may adversely affect other objects operating in the environment, such as other vehicles traveling on the same road. [Brief description of the drawings]

[0004] The detailed description will be set forth with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number identifies the drawing in which the reference number first appears. Use of the same reference number in different drawings indicates similar or identical components or features.

[0005] [Figure 1]FIG. 1 illustrates an autonomous vehicle in an environment, where an exemplary cost-based routing system can be used to determine a route for the autonomous vehicle for use in the environment. [Diagram 2] FIG. 1 illustrates various candidate actions for a vehicle to use in an environment and the computed costs associated with each candidate action. [Diagram 3] 1 is a diagram illustrating a vehicle in an environment, where an example cost-based routing system can determine a cost associated with an action based on the action's effect on a secondary object. [Figure 4] 1 is a diagram illustrating a vehicle in an environment, where an example cost-based routing system can be configured to identify occluded areas and determine actions to take based in part on the occluded areas. [Diagram 5] FIG. 1 illustrates a vehicle in an environment, where an exemplary cost-based routing system can be configured to determine an action to take based on a determination that a path of an object having priority is blocked. [Figure 6] FIG. 1 illustrates a vehicle in an environment configured to determine an action to take based at least in part on an action reward. [Figure 7] FIG. 1 is a block diagram of an example system for implementing the techniques described herein. [Figure 8] FIG. 1 illustrates an example process for controlling a vehicle according to an action based at least in part on a cost associated with the action. [Figure 9] FIG. 1 illustrates an example process for controlling a vehicle according to an action based at least in part on a safety cost associated with the action. [Figure 10] FIG. 1 illustrates an example process for controlling a vehicle based at least in part on a probability that an object is present in an occlusion area. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] The present disclosure is directed to techniques for dynamically determining an action (e.g., a path to travel) for a vehicle to take based on a cost associated with the action. The cost may be based in part on the impact of the action on objects (e.g., another vehicle, a bicycle, a pedestrian, etc.) operating in an environment including the vehicle. In at least some examples, such a cost may be based at least in part on the predicted motion of other objects in the environment in response to performing the potential action. The cost may be based on safety (e.g., avoiding a collision between the vehicle and the object), comfort (e.g., lack of sudden movements), progress (e.g., movement to a destination), operating rules (e.g., rules of the road, laws, ordinances, regulations, etc.), and the like. A vehicle computing system may determine one or more potential actions for the vehicle to take and may determine a cost associated with each potential action. In various examples, the actions may be selected based on an optimal cost (e.g., lowest cost, highest cost, cost below a threshold, etc.) associated therewith.

[0007] The vehicle may include an autonomous or semi-autonomous vehicle having a vehicle computing system configured to detect one or more objects in the environment. The objects may include static objects (e.g., buildings, bridges, signs, etc.) and dynamic objects such as other vehicles (e.g., cars, trucks, motorcycles, mopeds, etc.), pedestrians, bicyclists, etc. Unless indicated to the contrary, the term "object" herein refers to dynamic objects (e.g., objects that are stationary for a moment but are in motion and / or capable of moving). In one example, the object may be detected based on sensor data from a sensor of the vehicle (e.g., camera, motion detector, lidar, radar, time of flight, etc.). In one example, the object may be detected based on sensor data received from a remote sensor, such as, for example, a sensor associated with another vehicle or a sensor mounted in the environment configured to share data with multiple vehicles. In one example, the vehicle computing system may be configured to semantically classify the detected object. The classification may include another vehicle (e.g., automobile, pickup truck, semi-trailer truck, tractor, bus, train, etc.), a pedestrian, a bicyclist, a horse rider, etc. For example, a vehicle computing system may detect two objects and may classify the first object as a car and the second object as a pickup truck.

[0008] In various examples, the vehicle computing system may determine one or more actions that the vehicle can take while operating in an environment having a detected object. The actions may represent one or more possible paths (e.g., one or more vehicle trajectories) that the vehicle may take through the environment. The actions may include one or more reference actions (e.g., one of a group of operations that the vehicle is configured to perform in response to a dynamic operating environment), such as lane change right, lane change left, staying in the lane, going around an obstacle (e.g., double-parked vehicles, traffic cones, etc.), and the like. The actions may further include one or more sub-actions, such as a change in speed (e.g., maintain speed, accelerate, decelerate, etc.), a change in position (e.g., change position in the lane), and the like. For example, the actions may include staying in the lane (reference action) and adjusting the vehicle's position in the lane from a center position to operating on the left side of the lane (sub-action). For another example, the action may include accelerating from an initial position at a stop sign (sub-action) while remaining in the lane for two seconds (first reference action) followed by a lane change to the left (second reference action). In such an example, the action may include a sequence of actions.

[0009] In one example, the vehicle computing system may be configured to determine actions (e.g., reference actions and sub-actions) for the vehicle to take in a situation where the vehicle does not have the priority (e.g., an object has the priority). In one example, the vehicle computing system may be configured to determine a means by which the vehicle may safely proceed in a situation where a vehicle with priority may be prevented from exercising the priority. For example, the vehicle may approach a four-way stop sign behind a car. However, a pedestrian may cross the road in front of the car, thereby preventing the car from exercising the priority and passing in front of the vehicle. The vehicle computing system may identify the pedestrian blocking the car and may determine one or more actions for the vehicle to take based on the car being blocked. A first action may include remaining at the stop sign (e.g., maintaining position in the lane and maintaining substantially zero speed) and waiting for a car with priority to pass in front of the vehicle. A second action may include maintaining the vehicle's position in the lane and accelerating from the stop sign, with priority, to cross through the intersection in front of the car.

[0010] In various examples, the vehicle computing system may be configured to determine reference actions and / or sub-actions that are applicable to a vehicle in the environment. For example, changing lanes may not be a reference action that is applicable to a vehicle operating on a one lane road. In another example, accelerating may not be a sub-action that is applicable to a vehicle operating at a speed limit.

[0011] In various examples, the vehicle computing system may be configured to determine an initial position of each object detected in the environment. In various examples, the vehicle computing system may determine one or more predicted trajectories associated with each detected object, such as from the associated initial position together. In one example, the one or more predicted trajectories may be determined based on sensor data. Each predicted trajectory may represent a possible path that the detected object may travel through the environment. The one or more predicted trajectories may be based on a predicted reaction of the object to an action of the vehicle (e.g., active prediction). For example, the action may include the vehicle performing a lane change maneuver into a lane in which another vehicle is currently traveling. The vehicle computing system may make a determination that the predicted trajectory of the other vehicle may include a slight negative acceleration (deceleration) to provide space for the vehicle to perform a lane change.

[0012] In one example, the one or more predicted trajectories may be determined using a probabilistic heat map (e.g., a discretized probability distribution) for predicting the behavior of an object, such as described in U.S. patent application Ser. No. 15 / 807,521, filed Nov. 8, 2017, entitled "Probabilistic Heat Maps for Behavior Prediction," the entire contents of which are incorporated herein by reference. In one example, the one or more predicted trajectories may be based on a top-down representation of the environment, such as by utilizing techniques described in U.S. patent application Ser. No. 16 / 151,607, filed Oct. 4, 2018, entitled "Trajectory Prediction on Top-Down Scenes," and U.S. patent application Ser. No. 16 / 504,147, filed Jul. 5, 2019, entitled "Prediction on Top-Down Scenes based on Action Data," the entire contents of which are incorporated herein by reference. In various examples, the one or more predicted trajectories may be determined using tree search techniques, temporal logic, and / or machine learning techniques.

[0013] Additionally or alternatively, the vehicle computing system may be configured to identify a blocked area (e.g., also referred to as a blocked region) where a view of the environment (which may include oncoming traffic) may be blocked, such as by an obstacle. The blocked area may define an area in which the vehicle computing system is not able to detect objects. In various examples, the vehicle computing system may be configured to identify a relevant section of a road associated with the blocked area. In such examples, the section of the road may include one or more lanes, sidewalks, bike lanes, etc. along which the object is configured to operate in a direction that may be relevant to the vehicle. For example, the blocked area may include two lanes of the road along which the object may travel toward the vehicle. In such examples, any additional lanes of the road along which the object travels in a direction away from the vehicle may not be included in the blocked area.

[0014] In various examples, the vehicle computing system can determine a probability that the object will travel through the occluded area based on the relevant section of the road. In one example, the probability that the object will travel through the occluded area may be based on stored data associated with the relevant section of the road. The stored data may include sensor data associated with the relevant section of the road captured over time. In one example, the probability may be based on time of day, day of the week, month, season, etc. In one example, the probability may be determined using machine learning techniques. In such an example, a machine learning model may be trained with training data such as sensor data associated with the relevant section of the road captured over time. Based on the probability that the object may be present, the vehicle computing system may generate a predicted object having a predicted object trajectory and determine an action to take.

[0015] For each possible vehicle action (e.g., reference action and sub-action), the vehicle computing system may simulate a future state (e.g., estimated state) by projecting the vehicle and object forward in the environment for a time period (e.g., 5 seconds, 8 seconds, 12 seconds, etc.). The vehicle computing system may project the object forward (e.g., estimate the future position of the object) based on one or more predicted trajectories associated together. The vehicle computing system may project the vehicle forward (e.g., estimate the future position of the vehicle) based on one or more vehicle trajectories associated with the action. The estimated state may represent an estimated location (e.g., estimated position) of the vehicle, an estimated location of the object at a future time, and / or any other corresponding kinematic description of the object (velocity, acceleration, yaw, yaw rate, etc.). In various examples, the vehicle computing system may determine the estimated state at a predetermined rate (e.g., 10 Hertz, 20 Hertz, 50 Hertz, etc.) throughout the time period. In at least one example, the estimated states may be performed at a rate of 10 Hertz (e.g., 80 estimated states over an 8 second time period). In some examples, the vehicle computing device may determine estimated states of the vehicle and objects at various time intervals in a substantially parallel manner. In such examples, the vehicle computing device may reduce the time required to generate a set of estimated states over a time period.

[0016] The vehicle computing system may determine a cost associated with each estimated state based on the estimated positions of the vehicle and the object relative to one another. In one example, the vehicle computing system may analyze each estimated state based on one or more factors and apply or otherwise determine a cost value to the estimated state. In such an example, the cost of the estimated state may include a sum of the costs associated with each of the factors. The one or more factors may include vehicle and / or object safety (e.g., collision avoidance between the vehicle and the object), comfort (e.g., lack of sudden movements, e.g., by penalizing high accelerations when less than the shortest distance between the vehicle and the object), progress (e.g., movement to a destination), operating rules (e.g., rules of the road, laws, ordinances, regulations, etc.), etc.

[0017] The safety of the vehicle and / or object may include a likelihood of a collision between the vehicle and the object (e.g., a probability of a collision). The likelihood of a collision may be calculated based on a distance between the vehicle and the object (e.g., within 5 feet, 2 meters, 0.5 meters, etc.), a converged trajectory (e.g., an object trajectory that will substantially intersect with a vehicle trajectory associated with the action), a convergence rate between the vehicle and the object (e.g., 2 meters per second, 10 feet per second, etc.), etc. In one example, the likelihood of a collision may be based on a threshold associated with the distance and / or the convergence rate. For example, a distance between an estimated state associated with the vehicle and an estimated position associated with the object may be less than or equal to a threshold distance (e.g., 6 feet, 2.5 meters, etc.). Thus, the vehicle computing system may determine that a likelihood of a collision exists between the vehicle and the object. For another example, a trajectory associated with the vehicle and a trajectory associated with the object may converge at a rate equal to or greater than a threshold rate of convergence (e.g., convergence of 6 feet, 2.5 meters, 1.5 meters per second, etc.). Thus, the vehicle computing system may determine that there is a likelihood of a collision between the vehicle and the object. In one example, the likelihood or probability of a collision may be based on the amount by which the actual distance and / or convergence rate is below or above a threshold. In such an example, the probability of a collision may increase the closer the object is to the vehicle and is in an estimated state (e.g., 95% collision probability if within 6 inches of each other) and / or the amount by which the convergence rate is above a threshold (e.g., 90% collision probability with a convergence rate of 10 meters per second). In one example, the probability of a collision may be determined using machine learning techniques. In such an example, a machine learning model may be trained to determine the probability of a collision based on training data including situations in which the vehicle and the object interact in the environment.

[0018] In various examples, the cost associated with the safety factor may be based on the probability of a collision. In one example, if the probability of a collision is higher than a predefined threshold (e.g., 30%, 50%, etc.) indicating that a collision may occur, it may include a fixed cost (e.g., 50, 75, 90, etc.). In one example, the cost may be based on the probability of a collision determination. For example, the vehicle computing system may determine that the probability of a collision in an estimated state is 95%, and the vehicle computing system may evaluate a cost of 95 for the safety factor of the estimated state.

[0019] In various examples, the factors may include comfort of the object and / or the vehicle. In some examples, the cost of comfort may be determined based on the cost of safety. In some examples, the cost of comfort may include a portion of the cost of safety (e.g., 50%, 20%, etc.). In some examples, the cost of safety may include a base cost, such as based on a probability of collision. In such examples, to determine the total cost of safety for the estimated state, the vehicle computing system may calculate a first polynomial function (e.g., x 4 , x 6 ) may be applied to the base cost, and a second polynomial function (e.g., x 2 , x 3 For example, the vehicle computing system may determine a base cost of 4 associated with the estimated state, such as based on the probability of collision. The cost of safety of the estimated state may be calculated using a factor of 256 (base cost 4 ), and the comfort cost may be 16 (the base cost 2 For another example, the vehicle computing system may determine that a safety cost associated with the estimated state is 40 and a comfort cost associated with the estimated state is 20 (50% of the safety cost).

[0020] In one example, the comfort may include an estimated acceleration (e.g., positive, negative, lateral, etc.) and / or change in acceleration (e.g., jerk) associated with the object and / or vehicle caused by a vehicle action. For example, a vehicle action may cause an object to brake with significant force, causing an acceleration (e.g., deceleration) of -2 meters per second per second to avoid a collision. In one example, the comfort cost may be based on a value associated with the acceleration. For example, a negative acceleration of 2 feet per second (e.g., -2 feet per second) may result in a comfort cost of 2. Of course, the illustrative acceleration values ​​and / or costs are not intended to be limiting, and other values ​​and / or costs are contemplated herein.

[0021] In any such examples recited herein, the comfort cost and the safety cost may be determined based on the same parameters and / or the same functions of such parameters, but the safety cost significantly exceeds the comfort cost in the safety critical regime (proximity to object, proximity to object class, speed, etc.).

[0022] In various examples, the comfort cost may be associated with a distance (e.g., 10 feet, 5 feet, 2 meters, etc.) between the relative positions of the vehicle and the object at the estimated state. In one example, the distance may include the closest point of approach between the vehicle and the object over a period of time. The distance may be associated with the comfort of the occupants in the vehicle and / or the object as the object and the vehicle approach the closest point of approach to one another. In one example, the comfort cost may be calculated based on the distance, such as by applying a cost based on the inverse of the distance, or any other calculation that captures the idea that cost increases as distance decreases. In various examples, the comfort cost may be associated with a threshold distance. In such an example, the comfort cost may be evaluated based on the fact that the relative positions of the vehicle and the object are within a threshold distance of each other.

[0023] In various examples, the comfort cost may be associated with the closure rate between the vehicle and the object at the estimated state. In such examples, the comfort cost may be based on the relative speed associated with the object trajectory and / or the vehicle trajectory. For example, an object and vehicle with a closure rate of 20 miles per hour may include a comfort cost of 4, and an object and vehicle with a closure rate of 5 miles per hour may include a comfort cost of 1. In various examples, the cost and / or distance may be based on the classification of the object. For example, a vehicle position 5 feet away from another vehicle may include a comfort cost of 1, and a vehicle position 5 feet away from a pedestrian may include a comfort cost of 5.

[0024] In various examples, the factors may include the progression of a moving object and / or vehicle to a destination. In some examples, the cost may be determined based on a change in the speed of the object and / or vehicle (e.g., slowing down, stopping, etc.) and / or a delay in the movement of the vehicle from a stopped position, such as to avoid a collision. In various examples, the cost of progression may include a value substantially similar to the delay due to the estimated condition. For example, an action may include the vehicle waiting at an intersection for 11 seconds while the object approaches the intersection and passes through the intersection. The cost of progression associated with the action may include a value of 11, although other values ​​are contemplated herein.

[0025] In various examples, the factors may include the cost of a driving rule. The cost of a driving rule may be based on the rules of the road (e.g., Department of Transportation laws, ordinances, regulations, etc.), rules of good driving, local driving habits (e.g., general driving practices), driving etiquette (e.g., adjusting one's position in a lane to provide space for another vehicle to make a right turn without occupying a bike lane, etc.). In various examples, the cost of a driving rule associated with the estimated state may be calculated based on one or more rules that are broken and / or fulfilled. In such examples, the cost of a driving rule may be increased based on rules that are fulfilled or fulfilled, and / or increased based on rules that are broken or not fulfilled, and / or decreased (e.g., rules, habits, etiquette, etc.). For example, a route may include a vehicle changing lanes at an intersection. Although not illegal in some circumstances, changing lanes at an intersection may be discouraged by rules of good driving and therefore may impose a penalty cost of 5. The lane change may be within a threshold distance (e.g., 20 feet, 30 feet, 10 meters, etc.) of another vehicle that is considered the minimum distance for lane change etiquette. A lane change within the threshold distance may impose an additional penalty cost of 10. Thus, the total operational rule cost associated with the estimated state may be 15. For another example, a route may include a vehicle adjusting its position in the lane to the left while coming to a stop at a red light. The adjusted position may provide space for another vehicle behind to travel next to the vehicle and execute a right turn without having to wait for the light to turn green. The etiquette of adjusting the position may result in a reward of -3 (e.g., reducing the total cost associated with the estimated state).

[0026] In one example, the factors may be ranked in order of criticality. In such an example, at least one of the factors may include a cost that is weighted higher than other factors. In one example, the cost of safety may be weighted higher than other factors. In such an example, the vehicle computing system may emphasize safety over other factors in determining what action the vehicle should take. In one example, the initial cost of safety may be multiplied by 2x, 3x, 4x, etc. In one example, the vehicle computing system may apply a polynomial function (e.g., quadratic, cubic, etc.) to the initial cost of safety. For example, the vehicle computing system may multiply alpha*(cost of safety) 2 A total cost associated with the estimated state may be determined based on +Beta*(cost of comfort)+Gamma*(cost of progression)+Delta*(cost of operating rules).

[0027] In one example, the vehicle computing system may multiply the cost associated with each factor by a reduction amount based on the ranking. For example, the vehicle computing system may multiply the cost of safety by 4 times, the cost of comfort by 3 times, etc. In one example, the vehicle computing system may apply a polynomial function to the factors based on the ranking. For example, the vehicle computing system may multiply the cost of safety by 4 times, the cost of comfort by 3 times, etc. 4 +Beta*(cost of comfort) 3 +Gamma*(Cost of Progression) 2 Based on +Delta*(cost of operating rules), a total cost associated with the estimated state may be determined.

[0028] In one example, the vehicle computing system may sum the costs corresponding to each estimated state associated with the action to determine a total cost for the action. In various examples, the vehicle computing system may determine a total cost associated with each factor and an overall total cost. In such examples, the vehicle computing system may determine a total cost associated with the safety, comfort, progress, and / or operational rules associated with the action, as well as an overall total cost associated with the action.

[0029] In various examples, the vehicle computing system may compare a total cost associated with each action and may determine an action to take (e.g., control the vehicle) based on the total cost. In some examples, the actions selected may include the actions with the lowest cost associated with them. In such examples, safe actions that have minimal impact on other objects may include a low score. In some examples, the actions selected may include the actions with the highest cost associated with them. In such examples, safe actions that have minimal impact on other objects may include a high score, whereby penalties may be rewarded with negative values ​​and rewards may be rewarded with positive values.

[0030] In various examples, the vehicle computing system may determine the action to take by ranking the actions based on the cost associated with each factor individually. In at least one example, the vehicle computing system may first rank the actions based on safety cost. In such an example, the vehicle computing system may select a first set of actions that corresponds to the safest actions that can be processed against further costs (e.g., comfort cost, cost of travel, etc.). In one example, the first set of actions may include actions that have safety costs above or below a predetermined value (e.g., a threshold). In one example, the first set of actions may include a predetermined number and / or percentage of the safest actions (e.g., actions with the lowest or highest ranked safety costs, depending on how the costs are calculated). For example, the first set of actions may include the five actions with the lowest safety costs (e.g., the five safest actions). In another example, the first set of actions may include actions with the top 25% highest safety costs (25% of the safest actions).

[0031] In various examples, the vehicle computing system may determine, for each factor of the set of actions, a cost associated with a second ranked factor of one or more factors (e.g., factors ranked below safety). In at least one example, the second factor may include comfort of the object and / or vehicle. In some examples, the vehicle computing system may rank the set of actions based on the costs associated with the second ranked factors. In various examples, the vehicle computing system may identify a second set of actions based on the costs associated with the second ranked factors. In various examples, the second set of actions may include a predetermined number and / or percentage of actions having the highest (or lowest) costs associated with the second factors. For example, the second set of actions may include four actions of the first set of actions having the lowest comfort costs (e.g., the four most comfortable actions).

[0032] In various examples, the vehicle computing system may determine, for each of the second set of actions, a cost associated with a third ranked factor of the one or more factors. The vehicle computing system may rank the second set of actions and identify the third set of actions based on the costs associated with the third ranked factors as described above. The vehicle computing system may continue to determine a set of actions based on the costs associated with the next ranked factor until each of the factors has been considered. In one example, the vehicle computing system may determine an action to take based on the highest ranked action after each of the factors has been considered. In such an example, the vehicle computing system may control the vehicle based on the actions.

[0033] The techniques described herein may include improvements to the field of autonomous and / or semi-autonomous vehicle control planning. Conventionally, in autonomous vehicle control planning, a vehicle computing device may consider a variety of actions to take and select the most conservative path for the vehicle, such as one of a variety of actions that results in the longest distance being maintained from a detected object. The most conservative path ensures the safety of the vehicle and other objects, while often sacrificing other factors, such as progress and comfort. However, the techniques described herein include a means by which the vehicle computing system may select an action that optimizes safety, comfort, progress, and operating rules for both vehicles. Thus, the techniques described herein improve the field of autonomous and / or semi-autonomous vehicles.

[0034] Furthermore, as described above, conventional control planning techniques are vehicle-centric in that they do not consider the impact of an action on objects operating in the environment. Thus, a vehicle computing system employing conventional control planning techniques may ignore the adverse effects that an action may have on other objects operating in the environment. However, the techniques described herein include a means by which a vehicle computing system may select actions that optimize safety, comfort, progress, and operating rules for the vehicle as well as detected objects. Thus, the techniques described herein improve the field of autonomous and / or semi-autonomous vehicles.

[0035] The techniques described herein may be implemented in a number of ways. Example implementations are provided below with reference to the following drawings. Although described in the context of an autonomous vehicle, the methods, apparatus, and systems described herein may be applied to a variety of systems (e.g., sensor systems, or robotic platforms) and are not limited to autonomous vehicles. In one example, similar techniques may be utilized in driver-controlled vehicles where such systems may provide an indication of whether it is safe to perform various operations. In another example, the techniques may be utilized in an aviation or marine context, or in any system that uses planning techniques.

[0036] 1 is an illustration of an autonomous vehicle 102 in an environment 100, where an example cost-based routing system may be utilized to determine a route for the autonomous vehicle 102 for use in the environment 100. A vehicle computing device may execute the cost-based routing system for the vehicle 102. In one example, the cost-based routing techniques described herein may be implemented at least in part by or in association with a planning component of the vehicle computing device, such as planning component 724 of FIG.

[0037] In various examples, the vehicle computing device may be configured to detect one or more dynamic objects 104 (e.g., object 104) in the environment 100, such as via a perception system. The vehicle computing system may detect the object 104 based on sensor data received from one or more sensors. In some examples, the sensors may include sensors onboard the vehicle 102, such as, for example, cameras, motion detectors, lidar, radar, etc. In some examples, the sensors may include sensors onboard another autonomous vehicle and / or one or more remote sensors, such as sensor 106, mounted in the environment 100.

[0038] In various examples, the vehicle 102 may be configured to transmit and / or receive data from other autonomous vehicles and / or sensors 106. The data may include sensor data, such as data regarding objects 104 detected in the environment 100. In various examples, the environment 100 may include sensors 106 for traffic monitoring, collision avoidance, and the like. In one example, the sensors 106 may be mounted in the environment in areas of reduced visibility, such as, for example, intersections with poor or somewhat poor visibility, to provide additional visibility. For example, an intersection in the environment 100 may be determined to have a poor visibility intersection, where an approaching vehicle 102 may not be able to perceive an object 104 and / or other vehicles approaching from the left or right on an intersecting road. Thus, the intersection in the environment may include sensors 106 that provide sensor data to the approaching vehicle 102 regarding objects 104, such as objects 104(1) and 104(2) located on the intersecting road. In various examples, the vehicle computing system may receive the sensor data and may determine the type of object 104 (e.g., classify the type of object), such as whether the object 104 is a car, a truck, a bus, a semi-trailer truck, a motorcycle, a moped, a bicyclist, a pedestrian, etc.

[0039] In various examples, the vehicle computing system may determine one or more actions 110 for the vehicle 102 operating in the environment with the detected object 104. The action 110 may represent one or more possible paths (e.g., one or more vehicle trajectories) that the vehicle 102 may take through the environment 100. In one example, the action 110 may be determined based on the detected object 104. For example, the vehicle computing system may detect one or more objects 104, such as objects 104(1) and 104(2), approaching an intersection from the right. The vehicle computing system may determine that a first action 110(1) includes remaining at the initial position 112 and a second action 110(2) includes accelerating from the initial position 112 and passing through the intersection in front of the objects 104(1) and 104(2).

[0040] The actions 110 may include one or more reference actions (e.g., one of a group of maneuvers that the vehicle is configured to perform in response to a dynamic operating environment), such as changing lanes to the right, changing lanes to the left, staying in the lane, going around an obstacle (e.g., a double-parked vehicle, a traffic cone, etc.), etc. The actions 110 may further include one or more sub-actions, such as changing speed (e.g., maintaining speed, accelerating, decelerating, etc.), changing position (e.g., changing position in a lane), etc. For example, an action 110, such as the second action 110(2), may include staying in the lane (reference action) and accelerating from the initial position 112 with a first acceleration (sub-action), while the third action 110(3) may include staying in the lane (reference action) and accelerating from the initial position 112 with a second acceleration (sub-action). In another example, an action 110, such as action 110(4), may include accelerating (sub-action) from an initial position 112 while remaining in the lane for two seconds (first reference action) and then changing lanes left (second reference action). In yet another example, an action 110, such as action 110(5), may include accelerating at a first acceleration (sub-action) while changing lanes left (reference action) and accelerating at a second acceleration when established in the left lane.

[0041] In one example, the vehicle computing system may be configured to determine actions 110 (e.g., reference actions and sub-actions) for the vehicle 102 to take in a situation in which the vehicle 102 does not have priority (e.g., the object 104 has priority). In such an example, the vehicle computing system may be configured to determine a manner in which the vehicle 102 may safely proceed sooner than would be possible using traditional control planning techniques, such as using a most conservative approach. For example, the vehicle computing system may identify actions 110, such as action 110(5), that would allow the vehicle 102 to safely pass an intersection in front of the objects 104(1) and 104(2) without adversely affecting the objects 104(1) and 104(2), such as by requiring the vehicles 104(1) and 104(2) to slow down to avoid a collision with the vehicle 102. As such, vehicle 102 may be configured to proceed to the destination safer and faster than would otherwise be possible using the most conservative approach, which would require the vehicle to remain at initial position 112 until objects 104(1) and 104(2) have passed the intersection in front of vehicle 102.

[0042] In one example, the vehicle computing system may make a decision to block an object 104 with priority at an intersection from asserting the right of way. For example, the vehicle 102 may approach a four-way stop sign behind the object 104 (e.g., a car). However, a pedestrian may be crossing the road in front of the object 104, thereby blocking the object 104 from asserting the right of way and passing through the intersection in front of the vehicle 102. The vehicle computing system may identify the pedestrian blocking the object 104 and may determine one or more actions 110 for the vehicle 102 to take based on the object 104 being blocked. A first action 110(1) may include remaining at the stop sign (e.g., at an initial position 112) and waiting for the object 104 with priority to pass in front of the vehicle 102. A second action 110(2) may include maintaining the position of the vehicle 102 in the lane and accelerating to pass from the stop sign into the intersection ahead of the object 104 with priority.

[0043] In various examples, the vehicle computing system may be configured to determine actions 110 (e.g., reference actions and / or sub-actions) that are applicable to the vehicle 102 in the environment 100. For example, changing lanes may not be a reference action that is applicable to a vehicle operating on a one lane road. In another example, accelerating may not be a sub-action that is applicable to a vehicle operating at the speed limit.

[0044] In various examples, the vehicle computing system may determine one or more predicted object trajectories 108 (trajectories 108) based on the actions 110, sensor data, and / or type of object 104, such as trajectories 108(1), 108(2), and 108(3) associated with objects 104(1), 104(2), and 104(3), respectively. In various examples, the trajectories 108 may be determined based on a predicted motion of the object, as determined by a prediction system of the vehicle. In one example, the trajectories 108 associated with the object 104 may be based on active prediction. Active prediction includes determining the likelihood and / or possible reaction that the object 104 may have based on possible actions of the vehicle 102. For example, the vehicle computing system may determine that an object 104, such as object 104(3), on the same road as vehicle 102 but traveling in the opposite direction, may adjust its position in the lane in response to vehicle 102 performing a lane change, such as performing action 110(4) or 110(5) at an intersection. Thus, the vehicle computing system may determine trajectory 108(3), which may include a slight deviation to the right (relative to the direction of travel of object 104(3)).

[0045] In various examples, the trajectory 108 may be determined utilizing one or more machine learning algorithms. In such examples, the vehicle computing system may receive sensor data associated with the object 104, such as via a predictive system, and may predict the behavior of the object 104 based on the sensor data. For example, the vehicle computing system may be trained on captured image data of the movement of the object 104 over time, such that the vehicle computing system recognizes behaviors that the object 104 may take and may suggest future actions (e.g., trajectories 108). Additionally or alternatively, the vehicle computing system may utilize heat maps (e.g., discretized probability distributions), top-down representations of the environment, tree search methods, and / or temporal logic formulas to determine one or more trajectories 108 of the object 104, such as those described in the patent applications incorporated by reference above, although any other representation of the object's future state and / or uncertainty is contemplated.

[0046] For each vehicle action 110, such as actions 110(1), 110(2), 110(3), 110(4), and 110(5), the vehicle computing system may simulate a future state (e.g., an estimated state) by projecting the vehicle and objects forward in the environment for a time period (e.g., 5 seconds, 8 seconds, 12 seconds, etc.). The vehicle computing system may project the objects forward (e.g., estimate the future position of the object 104) based on one or more predicted trajectories associated with the respective action 110. For example, the vehicle computing system may determine that the second action 110(2) may result in an object trajectory 108(1) that includes a deceleration (e.g., a negative acceleration) of 1 mph while maintaining a constant heading. The vehicle computing system may project the vehicle 102 forward (e.g., estimate the future position of the vehicle) based on one or more vehicle trajectories associated with the action 110. As illustrated in FIG. 2, the estimated states may represent an estimated location (e.g., estimated position) of the vehicle 102 and an estimated position of the object 104 at some time in the future. In various examples, the vehicle computing system may determine the estimated states at a predetermined rate (e.g., 10 Hertz, 20 Hertz, 50 Hertz, etc.) through a set of estimated states (e.g., a group of estimated states representing an estimated position of the vehicle and / or object through a time period). In at least one example, the estimated states may be run at a rate of 10 Hertz (e.g., 80 estimated states over an 8 second time period). In certain examples, the vehicle computing device may determine the estimated states of the vehicle and object at various time intervals in a substantially parallel manner. In such examples, the vehicle computing device may reduce the time required to generate a set of estimated states over a time period.

[0047] In various examples, the vehicle computing system may be configured to identify one or more objects 104, such as 104(4), that are not related to the vehicle 102. An object 104 may be related to the vehicle 102 if the object 104 and the vehicle 102 are capable of potentially occupying the same space or coming within a threshold distance of one another over a period of time (e.g., potential collision). In various examples, an object may be determined to be not related to the vehicle based on a determination that a trajectory 108 associated with the object 104, such as trajectory 108(4) associated with the object 104(4), does not result in intersecting and / or converging with a vehicle trajectory associated with the action 110. For example, the trajectory 108(4) associated with the object 104(4) includes moving away from the vehicle 102 on a road that is substantially perpendicular to the direction of travel of the vehicle 102. In one example, an object 104 may make a determination that it is not relevant to the vehicle 102 based on a determination that the estimated position of the object 104 will be greater than a threshold distance from the estimated position of the vehicle 102 over a period of time (e.g., a set of estimated states). Based on a determination that the object 104 is not relevant to the vehicle 102, the object 104 may be ignored in an estimated state associated with an action 110. For example, the vehicle computing system may make a determination that the object 104(4) is not relevant to the vehicle 102 and may not include in the estimated state a trajectory 108(4) associated with the object 104(4). By ignoring an irrelevant object 104, such as object 104(4), the techniques described herein may improve the functionality of the vehicle computing system by providing additional storage and processing capabilities for determining estimated positions associated with related objects and other functions of the vehicle computing system, such as determining costs associated with different actions 110, controlling the vehicle, etc.

[0048] The vehicle computing system may determine a cost associated with each estimated state based on the estimated positions of the vehicle 102 and the object 104 relative to one another. In one example, the vehicle computing system may analyze each estimated state based on one or more factors and apply a cost value to the estimated state. In such an example, the cost of the estimated state may include a sum of the costs associated with each of the factors. The one or more factors may include safety of the vehicle 102 and / or the object 104 (e.g., avoiding a collision between the vehicle 102 and the object 104), comfort (e.g., lack of sudden movements by the vehicle 102 and / or the object 104), progress (e.g., movement of the vehicle 102 and / or the object 104 to a destination), operating rules (e.g., rules of the road, laws, ordinances, regulations, etc.), etc. In at least one example, the individual costs (e.g., at a specific state) may be compared to one another.

[0049] The safety of the vehicle 102 and / or the object 104 may include a likelihood of a collision (e.g., a probability of a collision) between the vehicle 102 and the object 104. The likelihood of a collision may be calculated based on a distance between the vehicle 102 and the object 104 (e.g., within 5 feet, 2 meters, 0.5 meters, etc.), a convergence trajectory (e.g., a trajectory 108 of the object 104 that substantially intersects with a vehicle trajectory associated with the action 110), a convergence rate between the vehicle 102 and the object 104 (e.g., 2 meters per second, 10 feet per second, etc.), etc. In one example, the likelihood of a collision may be based on a threshold associated with the distance and / or the convergence rate. For example, a distance between an estimated state associated with the vehicle 102 and an estimated position associated with the object 104 may be less than or equal to a threshold distance (e.g., 6 feet, 2.5 meters, etc.). Thus, the vehicle computing system may determine that a likelihood of a collision exists between the vehicle and the object. For another example, the vehicle trajectory associated with the action 110 and the trajectory 108 may converge at a rate equal to or greater than a threshold rate of convergence (e.g., convergence of 8 feet per second, 3 meters per second, 2 meters per second, etc.). As such, the vehicle computing system may determine that a likelihood of a collision exists between the vehicle 102 and the object 104. In one example, the likelihood or probability of a collision may be based on the amount by which the actual distance and / or convergence rate is below or above a threshold. In such an example, the probability of a collision may increase the closer the object is to the vehicle and is in an estimated state (e.g., 95% chance of collision if within 6 inches of each other) and / or the amount by which the convergence rate is above a threshold (e.g., 90% chance of collision with a convergence rate of 10 meters per second). In various examples, the probability of a collision may be determined using machine learning techniques. In such an example, a machine learning model may be trained utilizing training data that includes situations in which the vehicle 102 and the object 104 collided or did not collide.

[0050] In various examples, the cost associated with the safety factor (e.g., cost of safety) may be based on the probability of collision. In one example, if the probability of collision is higher than a predefined threshold (e.g., 40%, 51%, etc.) indicating that a collision may occur, a fixed cost (e.g., 60, 80, 100, etc.) may be included. In one example, the fixed cost may include a cost value above the threshold such that the vehicle computing system is not able to select an action to be associated with. In such an example, the probability of collision may include a constraint on the actions 110 that the vehicle 102 may take. In one example, the cost may be based on the probability of a collision decision. In one example, the correlation between the cost of safety and the probability of collision may be a 1:1, 2:1, 1:2 correlation, etc. For example, the vehicle computing system may make a determination that the probability of collision in an estimated state is 95%, and the vehicle computing system may evaluate a cost of 95 for the safety factor of the estimated state. In one example, the cost of safety may include a polynomial function of the probability of collision. In one example, the cost of safety may be determined based on a threshold probability of collision. In such an example, the cost of safety associated with a probability of collision below the threshold is determined based on a first relationship (e.g., 1:1, 2:1, x 2 ), and the cost of safety associated with a probability of collision above a threshold may include a second relationship (e.g., 10:1, x 4 etc.)

[0051] In one example, the threshold cost value above which the vehicle computing device may not select an action 110 (e.g., a constraint) may be associated with a particular probability of collision. For example, the threshold cost value may be set at 60% such that actions 110 having a probability of collision of 60% or higher may not be selected by the vehicle computing system. The threshold cost value may provide a means by which the vehicle computing system may emphasize safety over other factors in determining which action 110 the vehicle 102 will take.

[0052] In various examples, the safety cost may be based, at least in part, on the classification associated with the object 104. For example, the probability of an identical collision between two vehicles may result in a lower safety cost than the probability of a collision between a vehicle and a motorcycle. In one example, the probability of a collision with a pedestrian may exceed the cost associated with the probability of a collision with another vehicle, all else being equal. In one example, the vehicle computing system may determine a safety cost associated with the object and may increase the safety cost by a classification factor (e.g., 1.3 for motorcycles, 1.5 for pedestrians, etc.).

[0053] In various examples, the vehicle computing system may emphasize the safety of the other factors listed above by multiplying the determined safety cost, such as that based on the probability of collision, by a factor of two, three, four, etc., or by applying a polynomial function (e.g., quadratic, cubic, etc.) to the determined safety cost. For example, the vehicle computing system may calculate the total safety cost=(determined safety cost) 2 A quadratic equation may be applied to the cost of safety such as:

[0054] In various examples, the factors may include the comfort of the object 104 and / or the vehicle 102. In one example, the comfort of the object 104 may include an estimated acceleration (e.g., positive, negative, lateral, etc.) and / or an estimated change in acceleration (e.g., jerk) associated with the trajectory 108 of the object 104 as it responds to the action 110. The comfort of the vehicle 102 may include an estimated acceleration associated with the trajectory corresponding to the action 110 (lateral and / or longitudinal). For example, an object 104, such as object 104(1), may be predicted to apply brakes with a significant force, causing an acceleration (e.g., deceleration) of -2 meters per second per second in response to an action 110(2) that includes the vehicle 102 accelerating slowly from an initial position 112. In one example, the cost may be based on a value associated with the acceleration. For example, a negative acceleration of 2 feet per second per second (e.g., -2 feet per second per second per second) may result in a comfort cost of 2. In other examples, any other state / control of the object 104 being considered may be used in determining the comfort cost (e.g., nearest neighbor or next nearest neighbor, steering rate, turning rate, speed, how close it is to other vehicles as may be calculated taking into account road rule considerations, e.g., how close it is to or exceeds the speed limit required for the other object to drive away, etc.).

[0055] In one example, the comfort cost may be determined based on the distance between the vehicle 102 and the object 104 at an estimated state. In one example, the comfort cost associated with the action 110 may be based on the closest approach point between the estimated position of the vehicle 102 and the estimated position of the object 104 over a period of time. In various examples, the comfort cost may be based on the relative speed between the trajectory 108 and the vehicle trajectory associated with the action. In such examples, the comfort cost may be associated with the closure rate between the vehicle 102 and the object 104. For example, an object and vehicle with a closure rate of 30 miles per hour may include a comfort cost of 20, and an object and vehicle with a closure rate of 6 miles per hour may include a comfort cost of 2.

[0056] In one example, the distance and / or relative speed associated with the amenity cost may be based on a classification associated with the object 104. In one example, a classification may have a range of distances associated with it such that if the estimated location of the vehicle 102 is within a particular range at the estimated state, the associated amenity cost is added to the amenity cost of the estimated state. For example, a pedestrian may have a first range of 1 yard in any direction from the pedestrian with an associated amenity cost of 50 and a second range of 3 yards in any direction from the pedestrian with an associated amenity cost of 20.

[0057] In various examples, the cost of comfort may be determined based at least in part on the cost of safety. In some examples, the cost of comfort may include a portion of the cost of safety (e.g., 50%, 20%, etc.). In some examples, the cost of safety may include a cost based on a probability of collision or the like. In such examples, to determine the total cost of safety for the estimated state, the vehicle computing system may calculate a first function (e.g., x 4 , x 6) may be applied to the base cost, and a second function (e.g., x 2 , x 3 For example, the vehicle computing system may determine a base cost of 4 associated with the estimated state, such as based on the probability of collision. The cost of safety of the estimated state may be multiplied by 256 (base cost). 4 and the comfort cost is 16 (base cost) 2 For another example, the vehicle computing system may determine that a safety cost associated with the estimated state is 40 and a comfort cost associated with the estimated state is 20 (50% of the safety cost). In one example, the safety cost may include a first function of the object state (e.g., position, velocity, acceleration) and / or a relative state between the object 104 and the vehicle 102 (e.g., distance between the respective positions, relative velocity, relative acceleration, etc.), and the comfort cost may include a second function of the object state and / or a relative state between the object 104 and the vehicle 102.

[0058] In some examples, the vehicle computing system may assess a comfort cost based on the respective acceleration direction. In such examples, the cost of negative acceleration (e.g., deceleration) and / or lateral acceleration may include a higher cost than positive acceleration. In some examples, the vehicle computing system may multiply the negative acceleration and / or lateral acceleration by a factor of two, three, etc., or apply a polynomial function (e.g., quadratic, etc.) to the negative acceleration and / or lateral acceleration. In some examples, accelerations (e.g., positive, negative, lateral) may have costs associated with them based on accelerations exceeding a threshold (e.g., threshold positive, negative, and / or lateral accelerations). In such examples, the vehicle computing system may apply the costs based on a determination that the accelerations are equal to or exceed the respective threshold accelerations. In various examples, the comfort costs associated with the accelerations may be based on a determination that the accelerations are within one or more acceleration ranges. In such examples, the ranges may be associated with comfort cost values. For example, one acceleration range of -0.5 feet per second to -2 feet per second may have a cost value of 5, a second acceleration range of -2.1 feet per second to -5 feet per second may have a cost value of 10, and a third acceleration range of -5.1 feet per second to -10 feet per second may have a cost value of 15.

[0059] In various examples, the vehicle computing system may be configured to emphasize comforts associated with the object 104 over comforts associated with the vehicle 102. In such examples, the vehicle computing system may multiply the comfort cost of the object by 2x, 3x, 4x, etc., or may apply a polynomial function to the cost of the object. In various examples, the total comfort cost of a given estimated state may include the comfort cost of the object (whether adjusted for emphasis or not) and / or the comfort cost of the vehicle.

[0060] In various examples, the factors may include the progression of the moving object 104 and / or vehicle 102 to the destination. In one example, the cost may be determined based on a change in speed of the object 104 and / or vehicle 102 (e.g., decelerating, stopping, accelerating, decelerating, etc.) and / or a delay in the movement of the vehicle 102 from a stopped position (e.g., vehicle time delay), such as to avoid a collision. In various examples, the cost of progression may include a value substantially similar to the number of seconds of delay due to the action 110 and / or the estimated condition (e.g., object time delay, vehicle time delay, etc.). In an illustrated example, action 110(1) may include the vehicle waiting at an intersection (e.g., a stop sign) for 11 seconds while the object 104 approaches and passes through the intersection. The cost of progression associated with action 110(1) may include a value of 11. In one example, the cost of progression may be associated with a level of delay (e.g., a range of delay times). For example, a very short delay (eg, less than 1 second) may result in a cost of 1 step, while a short delay (eg, 1-3 seconds) may result in a cost of 2 steps, and so on.

[0061] In various examples, the cost of proceeding may be calculated based on a determined nominal speed profile of the object 104 operating in the environment 100. The nominal speed profile may include a profile of a baseline speed at which the object 104 may travel through the environment 100 in the absence of the vehicle 102. In certain examples, the nominal speed profile may be based on speed limits associated with the environment 100, such as based on pre-recorded data, known characteristics of the object 104 operating in the environment 100, etc. In various examples, the vehicle computing system may determine a difference between the nominal speed profile and the object trajectory 108 to determine the cost of proceeding. In certain examples, the vehicle computing system may apply a polynomial function or factor to the difference to determine a cost of proceeding associated with an action. In at least certain examples, such a cost may be based, at least in part, on the difference between a predicted trajectory of the object in which the vehicle 102 is present and taking a possible action.

[0062] In various examples, the factors may include the cost of a driving rule. The cost of a driving rule may be based on the rules of the road (e.g., Department of Transportation laws, ordinances, regulations, etc.), rules of good driving, local driving habits (e.g., general driving practices), driving etiquette (e.g., adjusting one's position in a lane to provide space for another vehicle to make a right turn without occupying a bike lane, etc.). In various examples, the cost of a driving rule associated with the estimated state may be calculated based on one or more rules that are broken and / or fulfilled. In such examples, the cost of a driving rule may be increased based on rules that are fulfilled or fulfilled, increased based on rules that are broken or not fulfilled, and / or decreased (e.g., rules, habits, etiquette, etc.). For example, an action 110, such as action 110(4), may include the vehicle 102 changing lanes at an intersection. Although not illegal in some circumstances, changing lanes at an intersection may be discouraged by rules of good driving and therefore may impose a penalty cost of 10. The lane change may be within a threshold distance (e.g., 20 feet, 30 feet, 10 meters, etc.) of another vehicle that is considered the minimum distance for lane change etiquette. A lane change within the threshold distance may impose an additional penalty cost of 10. Thus, the total operational rule cost associated with the estimated state may be 15. For another example, the action 110 may include the vehicle 102 adjusting its position in the lane to the left while coming to a stop at a red light. The adjusted position may provide space for an object approaching from behind the vehicle 102 to travel next to the vehicle 102 and execute a right turn without having to wait for the light to turn green. The etiquette of adjusting the position may result in a reward of -3 (e.g., reducing the total cost associated with the estimated state).

[0063] In one example, the factors may be ranked in order of severity. In such an example, at least one of the factors may include a cost that is weighted higher than other factors. As noted above, the cost of safety may be weighted higher than other factors. In such an example, the vehicle computing system may emphasize safety over other factors in determining the action the vehicle should take. For example, the vehicle computing system may determine a total cost associated with the estimated condition using the following formula: ES cost = (cost of safety) 2 + (cost of comfort) + (cost of progress) + (cost of op. rules) (1)

[0064] For another example, the vehicle computing system may determine a total cost associated with an estimated state using the following formula: ES cost = [2 × (cost of safety)] + (cost of comfort) + (cost of progress) + (cost of op. rules) (2)

[0065] In one example, the vehicle computing system may multiply the cost associated with each factor by a reduction amount based on the ranking. For example, the vehicle computing system may multiply the cost of safety by 4 times, the cost of comfort by 3 times, the cost of progress by 2 times, and the cost of operating rules by 1 time. In one example, the vehicle computing system may apply a polynomial function to the factors based on the ranking. For example, the vehicle computing system may determine a total cost associated with the estimated state according to the following formula: ES cost = (cost of safety) 4 +(cost of comfort) 3 +(Cost of Progression) 2 +(op. cost of rules) (3)

[0066] In various examples, the vehicle computing system may determine that a cost associated with the estimated state (e.g., an ES cost) exceeds a threshold estimated state cost. In one example, the vehicle computing system may ignore data associated with an action based in part on the ES cost exceeding a threshold.

[0067] In one example, the vehicle computing system may sum costs corresponding to some or all of the estimated states associated with an action to determine a total cost for the action. As a non-limiting example, the vehicle computing system may determine the total action cost according to the following formula: Total Action Cost = (ES1 cost) + (ES2 cost) + … + (ES N Cost) (4)

[0068] Additionally or alternatively, the vehicle computing system may determine a total cost associated with each factor. In such an example, the vehicle computing system may determine a total cost associated with safety, comfort, progress, and / or operational rules associated with the action. For example, a total safety cost associated with an action may be expressed as: Total safety cost = (ES1 safety cost) + (ES2 safety cost) + … + (ES N (5) Cost of Safety

[0069] Of course, any other combination of costs (linear, non-linear, weighted, unweighted, machine learning, etc.) is contemplated.

[0070] In various examples, the vehicle computing system may compare the total cost associated with each action and may determine an action to take (e.g., control the vehicle) based on the total cost. In some examples, the selected action may include the action with the lowest cost associated with it. In such examples, safe actions that have minimal impact on other objects may include a low score. In some examples, the selected action may include the action with the highest cost associated with it. In such examples, the penalty may be rewarded with a negative value and the reward may be rewarded with a positive value such that safe actions that have minimal impact on other objects may include a high score. In various examples, the vehicle computing system may control the vehicle based on the determined (e.g., selected) action. In some examples, any one cost above a threshold may disqualify such trajectory from consideration.

[0071] 2 is a collection of drawings 200 of possible actions (also referred to as candidate actions) 204, 206, 208, and 210 for a vehicle 202, such as vehicle 102, to take in an environment, such as environment 100, along with respective costs 212, 214, 216, and 218 associated therewith. Although illustrated as four actions, a first action 204 may correspond to action 110(2) having a first acceleration, a second action 206 may correspond to action 110(2) having a second acceleration, a third action 208 may correspond to action 110(5), and a fourth action 210 may correspond to action 110(1), although a greater or lesser number of actions may be considered by the vehicle computing system.

[0072] In some examples, actions 204, 206, 208, and / or 210 may be predefined and / or stored in a memory of the vehicle computing device that is accessible by a planning component (e.g., planning component 724 of FIG. 7). For example, the planning component may store actions 204, 206, 208, and / or 210 as a predefined set of actions that vehicle 202 may take at a two-way stop (e.g., where crossing traffic has the right of way). In some examples, actions 204, 206, 208, and / or 210 may be determined by the planning component based on the environment, such as based on the number and / or proximity of objects 222(1) and 222(2) detected in the environment. In various examples, the vehicle computing system may determine whether an action is applicable to the vehicle in the environment at a particular time (e.g., from an initial position). In such an example, the vehicle computing system may access predefined actions and may determine whether each of the predefined actions is applicable to the environment 100. The actions may be applicable to the environment based on a determination that they are executable in the environment (e.g., changing to the left lane when the vehicle is operating in the right lane of a two lane road), are not executable in the environment (e.g., changing lanes while the vehicle is operating on a single lane road), are logical for the environment (e.g., accelerating slowly in a school zone), or are not logical for the environment (e.g., accelerating quickly in proximity to a playground).

[0073] As described above, actions 204, 206, 208, and / or 210 may include reference actions and / or sub-actions. Reference actions may include staying in lane, lane changing to the right, lane changing to the left, going around an obstacle, abandoning going around an obstacle, etc. Sub-actions may include maintaining speed, accelerating, decelerating, adjusting position in lane, etc. In the illustrative example, action 204 includes staying in lane (reference action) and accelerating at a first acceleration (sub-action), and action 206 includes staying in lane (reference action) and accelerating at a second acceleration (sub-action). Further, in the illustrative example, action 208 includes changing lanes while accelerating from initial position 220, and action 210 includes substantially maintaining position at initial position 220.

[0074] In various examples, for each of the actions 204, 206, 208, and 210, the vehicle computing system may determine one or more predicted object trajectories, such as trajectory 108, based on the respective actions 204, 206, 208, 210. In one example, the predicted object trajectories may be based in part on sensor data collected by one or more sensors on the vehicle and / or sensors in the environment (e.g., mounted on another vehicle and fixed in a fixed location, such as an intersection with poor visibility). In one example, the predicted object trajectories may be based in part on the respective classifications associated with the objects 222(1) and 222(2). For example, a first predicted object trajectory associated with the object 222(1) classified as a car may include a first deceleration corresponding to a first amount of pressure applied to the brakes in response to the vehicle 202 traveling through the intersection in front of the object 222(1). A second predicted object trajectory associated with object 222(1) classified as a semi-trailer truck may include a second deceleration corresponding to a second amount of pressure applied to the brakes (e.g., more pressure to stop a larger vehicle) in response to vehicle 202 crossing in front of object 222(1).

[0075] In various examples, the predicted object trajectory may be determined based on the predicted motion of objects 222(1) and 222(2), as determined by a prediction system of the vehicle. In one example, the predicted object trajectory associated with objects 222(1) and 222(2) may be based on active prediction. Active prediction involves determining possible and / or probable reactions that objects 222(1) and 222(2) may have based on their respective actions 204, 206, 208, and 210. For example, the vehicle computing system may determine that object 222(2), which is on the same road as vehicle 202 and traveling in the opposite direction, may adjust its position in the lane in response to executing a lane change at the intersection, as shown in third action 208. Thus, the vehicle computing system may determine that the predicted object trajectory may include a slight deviation to the right (as shown in estimated position 226(2)). In various examples, the predicted object trajectory may be determined using a top-down representation of the environment 100, heat maps, temporal logic formulas, tree search methods, and / or machine learning techniques, such as those described in the U.S. patent applications incorporated herein.

[0076] As illustrated in FIG. 2, for each action 204, 206, 208, and 210, the vehicle computing system can simulate a future state (e.g., an estimated state) by projecting vehicle 202 and object 222(1) and object 222(2) forward in the environment for a period of time (e.g., 5 seconds, 8 seconds, 12 seconds, etc.). The first estimated state may include a first estimated object position 224(1) (e.g., object position 224(1)) associated with object 222(1), a first estimated object position 226(1) associated with object 222(2), and a first estimated vehicle position 228(1), and the second estimated state may include a second estimated object position 224(2) associated with object 222(1), a second estimated object position 226(2) associated with object 222(2), and a second estimated vehicle position 228(2), etc. Estimated object positions 224(1), 224(2), 224(3) and 226(1), 226(2), 226(3) may be determined based on predicted object trajectories associated with the respective objects 222(1) and 222(2). Although illustrated as three locations 224(1), 224(2), 224(3), 226(1), 226(2), 226(3), and 228(1), 228(2), 228(3) associated with first object 222(1), second object 222(2), and vehicle 202, respectively, a greater or lesser number of estimated states (e.g., locations) may be determined for objects 222(1), 222(2), and vehicle 202 over a period of time as in the visual representation of the three estimated states.

[0077] In various examples, the vehicle computing system may determine estimated states (thus estimated object positions 224(1), 224(2), 224(3), and 226(1), 226(2), 226(3), and estimated vehicle positions 228(1), 228(2), and 228(3)) over a time period (e.g., 4 seconds, 8 seconds, 10 seconds, etc.) at a predetermined rate (e.g., 10 Hertz, 20 Hertz, 50 Hertz, etc.). In at least one example, the estimated states may be performed at a rate of 10 Hertz (e.g., 80 estimated states over an 8 second time period). In such examples, estimated positions 224(1), 224(2), 224(3), 226(1), 226(2), 226(3), and 228(1), 228(2), and 228(3) may be determined at 0.1 second intervals. In one example, the vehicle computing device may determine estimated states of vehicle 202 and objects 222(1) and 222(2) at various time intervals that are substantially parallel. In such examples, the vehicle computing device may reduce the time required to generate a set of estimated states over a period of time.

[0078] 2, the vehicle computing system may determine a first cost 212 associated with the first action 204, a second cost 214 associated with the second action 206, a third cost 216 associated with the third action 208, and a fourth cost 218 associated with the fourth action 210. In one example, the vehicle computing system may determine a cost associated with each estimated state based on the estimated positions of the vehicle 202 and the objects 222(1) and 222(2), and / or the object and vehicle trajectories associated with each action 204, 206, 208, 210. In such an example, the costs 212, 214, 216, and 218 may represent a sum of the costs associated with each estimated position. For example, the 16 safety costs associated with the first cost 212 may represent a sum of the respective safety costs calculated for each estimated state. For example, the first estimated state may include a first safety cost of at least 0 for a distance from first estimated vehicle position 228(1) to first estimated object position 224(1) associated with first estimated object position 222(1) and / or first estimated object position 226(1) associated with second object 222(2), the second estimated state may include a second safety cost of 2 as the respective distances between vehicle 202 and objects 222(1) and 222(2) approach a converging trajectory, and the third estimated state may include a third safety cost of 14 based on at least a limited distance between third estimated vehicle position 228(3) and third estimated object position 224(3). Of course, with reference to all costs described herein, any numerical values ​​are for illustrative purposes only to show how various costs may compare to one another and may not reflect actual numbers.

[0079] As described above, costs 212, 214, 216, and 218 may be determined based on one or more factors. The one or more factors may include safety of vehicle 202 and / or objects 222(1) and 222(2) (e.g., avoiding a collision between vehicle 202 and objects 222(1) and 222(2)), comfort (e.g., lack of sudden movements by vehicle 202 and / or objects 222(1) and 222(2)), progress (e.g., movement of vehicle 202 and / or objects 222(1) and 222(2) to a destination), operating rules (e.g., rules of the road, laws, ordinances, regulations, etc.), and the like.

[0080] Although described as a total cost associated with safety, comfort, progress, and / or rules of the road, each factor cost may include safety, comfort, progress, and / or rules of the road for vehicle 202 and / or objects 222(1) and 222(1). For example, second action 206 may include accelerating vehicle 202 from initial position 220 at a rate that may be slightly uncomfortable for the occupants. The vehicle computing system may determine that the occupant discomfort may include a comfort cost of 2 for vehicle 202. Further, the vehicle computing system may determine that object 222(1) may have a predicted object trajectory that includes a slight deceleration in response to vehicle 202 crossing an intersection. The vehicle computing system may evaluate a comfort cost of 2 for object 222(1). Combining the vehicle comfort cost and the object 222(1) comfort cost, the vehicle computing system may determine that the total comfort cost for the second action is 4. In another example, the vehicle computing system may determine that the third action 208 includes a slight lateral acceleration to change lanes from the right lane to the left lane. Based on the slight lateral acceleration while traveling from the initial position 220, the vehicle computing system may evaluate a comfort cost of the vehicle 1. Further, the vehicle computing system may predict that a predicted trajectory associated with the object 222(2), which may include a slight lateral acceleration, will slowly adjust its position in the lane to the right. The vehicle computing system may evaluate a comfort cost of the object 1 to the third action 208, resulting in a total comfort cost of 2. For yet another example, the fourth action 210 may result in a cost associated only with the vehicle 202 based on a determination that the fourth action 210 will not affect the objects 222(1) and 222(2). Thus, safety, comfort, traveling, and operating rules (labeled road rules) costs are associated with the vehicle.

[0081] In one example, each factor cost may include a safety, comfort, progress, and / or rules of the road associated with one or more of objects 222(1) and 222(2). In such an example, the vehicle computing system may be configured to determine which action 204, 206, 208, or 210 to take based on an impact that action 204, 206, 208, or 210 may have on objects 222(1) and 222(2). For example, the vehicle computing system may predict that in response to vehicle 202 crossing an intersection in a first action, object 222(1) may decelerate at a first deceleration. The first deceleration may include an uncomfortable deceleration, such as to an operator and / or passenger of object 222(1). Thus, the comfort cost may include a comfort cost of six of the objects. However, the second action 206 causes the object 222(1) to decelerate at a second deceleration, which may be predicted to be slightly lower than the first deceleration. Because the second deceleration is lower (e.g., less uncomfortable) than the first deceleration, the comfort cost of the object may be determined to be four. For another illustration, the first deceleration associated with the first object 222(1) in response to the first action 204 may result in a delay in the first object 222(1) reaching the destination. Thus, the vehicle computing system may evaluate a cost of travel of two. However, the second deceleration of the object 222(1) in response to the second action 206 may result in less delay, thereby resulting in a cost of travel of one. As described above, the cost of progress value may represent a number of seconds that object 222(1) is delayed in response to action 204, 206, 208, or 210, or may represent a value corresponding to a level of delay (e.g., a very short delay (e.g., less than 1 second) resulting in a cost of progress of 1, a short delay (e.g., 1-3 seconds) resulting in a cost of progress of 2, etc.).Any one or more of such costs may be determined for one or more of the predicted trajectories based on changes in actions and / or absence of the vehicle.

[0082] The cost of the operational rule (rule of the road) may be based on the rules of the road (e.g., Department of Transportation laws, ordinances, regulations, etc.), rules of good driving, local driving habits (e.g., general driving practices), driving etiquette (e.g., adjusting position in a lane to provide space for another vehicle to make a right turn without occupying a bike lane, etc.). In various examples, the cost of the operational rule associated with the estimated state may be calculated based on one or more rules that have been broken and / or fulfilled. In such examples, the cost of the operational rule may be increased based on rules that have been fulfilled or fulfilled, and / or increased based on rules (e.g., rules, habits, etiquette, etc.) that have been broken or not fulfilled, and / or decreased. In an illustrative example, the first action 204 and the second action 206 both include a cost of one operational rule based on an object 222(1) having priority at the intersection. The third action 208 includes a cost of six operational rules based on lane changes within the intersection.

[0083] In various examples, the vehicle computing system may compare costs 212, 214, 216, and 218 associated with each action 204, 206, 208, and 210 and determine an action to take (e.g., control the vehicle). In one example, the vehicle computing system may select an action to take based on the lowest cost associated together. In various examples, the vehicle computing system may identify actions 204, 206, 208, and 210 that have factor costs below a threshold level associated with the factor and select the lowest cost action from the identified actions. For example, the threshold safety cost may include a value of 10. Because the first action 204 includes a safety cost that exceeds the threshold safety cost, the vehicle computing system may ignore the first action 204 from planning considerations. In one example, the vehicle computing system may identify a factor cost associated with the action 204, 206, 208, or 210 as being above a threshold and may stop determining (e.g., calculating) a cost value associated with the action 204, 206, 208, or 210. For example, the vehicle computing system may determine that after the third estimated state, the first action 204 has a safety cost that is above a threshold. Based on the determination that the safety cost is above the threshold, the vehicle computing system may stop generating estimated states and may determine the associated costs together. In such an example, the techniques described herein may provide additional computing resources to determine costs associated with other (possible) actions, thereby improving the functionality of the vehicle computing system.

[0084] 3 is an illustration of a vehicle 302 in an environment 300, such as vehicle 102 in environment 100, where an example cost-based routing system can determine a cost associated with an action based on the action's impact on secondary object 304(2). As described above, the vehicle computing system may determine one or more actions, such as actions 110(1), 110(2), 110(3), 110(4), and 110(5) illustrated in FIG. 1, and / or actions 204, 206, 208, and 210 illustrated in FIG. 2. The actions illustrated in FIG. 3 include vehicle 302 accelerating from an initial position 306 at a two-way intersection 312, where objects 304 traveling on intersecting roads, such as primary object 304(1) and secondary object 304(2), have priority.

[0085] In various examples, the vehicle computing system may detect the object 304 based on sensor data captured by one or more sensors mounted on the vehicle, on another vehicle, and / or in the environment. In one example, the vehicle computing system may designate the object 304 as a primary object 304(1) and a secondary object 304(2). The secondary object 304(2) may be designated based on a determination that the secondary object 304(2) follows the primary object 304(1) (e.g., has a substantially similar heading and is located behind the primary object 304(1)). In one example, the designation as a secondary object 304(2) may be based on a determination that the secondary object 304(2) is within a threshold distance (e.g., 10 feet, 4 meters, etc.) of the primary object 304(1). In such an example, the threshold distance may be determined based on the speed of the object 304, a speed limit associated with the environment, etc. In various examples, the designation as a secondary object 304(2) may be based on a determination that the secondary object 304(2) is predicted to react in a manner similar to the primary object 304(1) in response to the actions of the vehicle 302 (e.g., a similar predicted trajectory between the objects, etc.).

[0086] In one example, the vehicle computing system may determine one or more predicted trajectories of the object 304 based on the action. In at least one example, the vehicle computing system may determine one or more predicted trajectories of the primary object 304(1) and the impact of the predicted trajectories associated with the primary object 304(1) on the secondary object 304(2). In various examples, the vehicle computing system may project the vehicle 302 and the object 304 forward at a specified rate (e.g., 5 Hertz, 10 Hertz, etc.) for a period of time (e.g., 6 seconds, 7 seconds, 8 seconds, etc.). Estimated vehicle positions 308(1), 308(2), and 308(3) and estimated object positions 310(1), 310(2), and 310(3) are provided for illustrative purposes and a greater or lesser number of estimated object positions may be determined by the vehicle computing system. Further, while described as estimated object positions 310(1), 310(2), and 310(3) associated with primary object 304(1), the vehicle computing system may additionally generate an estimated object position for secondary object 304(2).

[0087] Based in part on estimated vehicle positions 308(1), 308(2), and 308(3) and estimated object positions 310(1), 310(2), and 310(3), as described above, the vehicle computing system may determine a total action cost based on the impact the action has on the primary object 304(1). The total action cost may include costs associated with one or more factors (e.g., safety, comfort, progress, operating rules, etc.). In one example, the vehicle computing system may apply the total action cost associated with the primary object 304(1) (e.g., the total impact the action has on the primary object 304(1) taking into account the factors) to the secondary object 304(2). In such an example, the total action cost associated with the object 304 may be doubled.

[0088] In one example, the vehicle computing system may determine a cost of safety, a cost of comfort, a cost of progression, and a cost of operational rules associated with the primary object 304(1) and may apply some or all of the costs (safety, comfort, progression, operational rules) to the secondary object 304(2). In such an example, the cost of the selection may be associated with factors that will affect the secondary object 304(2), such as comfort and progression, in response to a predicted trajectory of the primary object 304(1) associated with the action. For example, as shown, the action may include the vehicle 302 crossing an intersection in front of the primary object 304(1). In response to the action, the vehicle computing device may determine that the primary object 304(1) will likely apply brakes to slow down before entering the intersection, thereby avoiding a collision with the vehicle 302. Based on the negative acceleration of the primary object 304(1), the vehicle computing system may determine that the cost of comfort of the object associated with the action is 4 and the cost of progression of the object associated with the action is 2. Additionally, the vehicle computing system may determine that a negative acceleration of primary object 304(1) may also cause secondary object 304(2) to decelerate, thereby avoiding a collision with primary object 304(1). As such, the vehicle computing system may apply an object comfort cost of 4 and an object progress cost of 2 to secondary object 304(2), resulting in a total object comfort cost of 8 and a total object progress cost of 4.

[0089] In various examples, the vehicle computing system may determine that the cost associated with the secondary object 304(2) includes a percentage (e.g., 50%, 75%, etc.) of some or all of the cost associated with the primary object 304(1). Using the example from above, the primary object 304(1) may be evaluated with an amenity cost of 4 and an object progression cost of 2. The vehicle computing system may determine that the secondary object 304(2) will not need to slow down as much and therefore will apply an object amenity cost of 3 and an object progression cost of 1.5 (e.g., 75% of the primary object cost). In one example, the percentage may be based on the distance between the primary object 304(1) and the secondary object 304(2). In such an example, a secondary object 304(2) that follows closely behind (e.g., within two car lengths, within 20 feet, etc.) may be affected more, and as such a higher percentage (e.g., 80%, 85%, etc.) may be applied, and a secondary object 304(2) that follows a greater distance behind the primary object 304(1) (e.g., 10 meters, three car lengths, etc.) may have a lower percentage (e.g., 50%, 25%, etc.) applied.

[0090] In various examples, the vehicle computing system may determine the percentage based on the number of vehicles away from primary object 304(1) (e.g., secondary object 304(2), tertiary object, etc.). In one example, the vehicle computing system may determine that a cost associated with secondary object 304(2) comprises a first percentage of the cost applied to primary object 304(1) and a cost associated with tertiary object 304(1) comprises a second percentage of the cost applied to primary object 304(1). In such an example, the second percentage may be less than the first percentage. For example, the cost associated with secondary object 304(2) may be 75% of the cost associated with primary object 304(1) and the cost associated with the tertiary object may be 50% of the cost associated with primary object 304(1). In one example, the vehicle computing system may determine that a cost associated with secondary object 304(2) comprises a percentage of the cost applied to primary object 304(1) and a cost associated with tertiary object 304(1) comprises a percentage of the cost applied to secondary object 304(2). For example, the cost associated with secondary object 304(2) may be 80% of the cost associated with primary object 304(1) and the cost associated with the tertiary object may be 80% of the cost associated with secondary object 304(2).

[0091] 4 is an illustration of a vehicle 402 in an environment 500, such as vehicle 102 in environment 100, where an example cost-based routing system may be configured to identify a blocked area 404 and determine an action to take based in part on the blocked area 404. The blocked area 404 may define an area in which a vehicle computing system may not be able to detect one or more objects 406, such as object 104, because one or more obstacles 408 block the area associated with the blocked area 404. In an illustrative example, the blocked area 404 may include a portion of a road 410 blocked by an obstacle 408 (e.g., a parked automobile). In other examples, the blocked area 404 may include a sidewalk, walkway, bike lane, or other area in which a view of one or more objects 406 may be blocked.

[0092] In various examples, the vehicle computing system may identify a forward boundary 412 and a rear boundary 414 of the blocked area 404. The forward boundary 412 may include a forward-most line of the blocked area 404, in front of which the vehicle computing system may be configured to detect the object 406. The rear boundary 414 may include a furthest rearward line of the blocked area 404, in front of which the vehicle computing system may be configured to detect the object 406. In various examples, the vehicle computing system may be configured to identify one or more blocked road regions 416. The blocked road regions 416 may extend from the forward boundary to the rear boundary 414. In an illustrative example, the blocked road regions 416 are each associated with an individual lane of the section of road 410.

[0093] In various examples, the vehicle computing system may be configured to determine an identifier 418 associated with the section of road 410. The identifier 418 may represent a unique identifier associated with a portion of a road network utilized by the vehicle computing device to operate in the environment 400. The portion of the road network may be predefined, such as in map data, and may be stored on and / or accessible to the vehicle computing device via a remote computing device.

[0094] In various examples, the vehicle computing system may be configured to determine a probability that the object 406 may travel in the occluded area 404. In some examples, the probability may be based on the identifier 418 and the pre-recorded sensor data associated therewith. In some examples, the probability may be based on the time of day, the day of the week, the month, etc. In at least one example, the probability may be determined utilizing machine learning techniques. In such examples, one or more machine learning models may be trained utilizing pre-recorded sensor data captured at various times to determine the probability that the object 406 may operate in the occluded area 404. In various examples, the vehicle computing system may be configured to determine the probability that the object 406 may operate in the occlusion area 404 according to the techniques described in U.S. patent application Ser. No. 16 / 147,177, filed Sep. 28, 2018, entitled "Radar Spatial Estimation," and U.S. patent application Ser. No. 16 / 011,436, filed Jun. 18, 2018, entitled "Occlusion Aware Planning," both of which are incorporated herein by reference.

[0095] In various examples, based on the probability that the object 406 may operate in the occluded area 404, the vehicle computing system may determine (e.g., generate the predicted object 420 and include data associated with the predicted object in determining an action) that it is likely that the predicted object 420 is operating in the occluded area 404. In various examples, the vehicle computing system may generate the predicted object 420 based on a determination that the probability that the object 406 may travel in the occluded area 404 exceeds a threshold probability (e.g., >20%, >30%, >50%, etc.). In an illustrative example, the vehicle computing system may determine that the probability exceeds a threshold probability and may generate the predicted object 420 in the right lane of the section of the road 410.

[0096] Further, based on the determination that the probability is above the threshold, the vehicle computing system may determine one or more predicted object trajectories 422. As described above, the predicted object trajectory 422 may include a path and / or a speed that the predicted object 420 may travel in response to an action to be taken by the vehicle 402. The predicted object trajectory 422 may be determined utilizing a machine learning algorithm trained on pre-recorded sensor data associated with the identifier 418 (e.g., a section of the road 410) and / or other techniques for determining object trajectories described herein. In various examples, the vehicle computing system may determine a predicted object trajectory 422 associated with each action that the vehicle 402 may take from the initial position 424.

[0097] In various examples, the vehicle computing system may determine a cost associated with each action based on the predicted object trajectory 422 in the environment 400 and / or a trajectory associated with the detected object 406. In such examples, the vehicle computing device may determine a safety cost, a comfort cost, a travel cost, and / or a regulation cost associated with the predicted object 420 and / or the detected object 406. In one example, the detected object 406 may be considered a secondary object to the predicted object 420, such as secondary object 304(2). In such examples, the co-associated costs may be determined using the secondary object cost determination techniques described above with respect to FIG.

[0098] In various examples, the vehicle computing device may determine a total cost associated with each action based on costs (e.g., safety costs, comfort costs, travel costs, operating rules costs, etc.) associated with the predicted objects 420, the detected objects 406, and / or the vehicle 402. In at least one example, the action having the lowest cost may be selected by the vehicle computing system to take for the vehicle. In one example, the vehicle computing device may cause the vehicle to be controlled according to the action.

[0099] 5 is an illustration of a vehicle 502 in an environment 500, such as vehicle 102 in environment 100, where an example cost-based routing system may be configured to determine an action to take based on a determination that a path 504 of an object 506 having priority is blocked. In an illustrative example, the path 504 of the object 506 is blocked by a pedestrian 508. In other examples, the path 504 of the object 506 may be blocked for any number of other reasons, such as a construction zone, an accident, a bicyclist passing in front, etc. As described herein, the blocking of the path 504 may result in a predicted object trajectory associated with the object 506 having a substantially zero speed.

[0100] In various examples, the vehicle 502 may detect an object 506, such as object 104, in the environment 500 based on sensor data captured by sensors on the vehicle 502 and / or one or more remote sensors. In one example, the vehicle computing system may determine that the object 506 has priority based on rules of the road associated with the environment 500 and / or an intersection 510. For example, the vehicle computing system may determine that the object 506 has arrived at the intersection 510, which is a four-way stop, before the vehicle 502, and therefore gives priority to the object 506. For another example, the intersection may include a two-way stop, where the vehicle 502 is required to stop but the object 506 is not, and therefore gives priority to the object 506.

[0101] In an illustrative example, the vehicle computing system may detect a pedestrian 508 blocking a path 504 of an object 506, such as based on sensor data. In one example, the pedestrian 508 may include a second object classified as a pedestrian by the vehicle computing system. In various examples, the vehicle computing system may determine one or more trajectories 512 associated with the pedestrian 508 based on the sensor data. In various examples, the trajectory 512 may be determined based on various factors, such as the classification as a pedestrian 508, a maximum speed associated with the classification, a certain kinematic movement (e.g., a certain speed, a heading, etc.). In various examples, the trajectory 512 may be determined utilizing a top-down representation of the environment 500, heat maps, temporal logic formulas, tree search techniques, machine learning techniques, and / or any other techniques for predicted object 506 and / or pedestrian 508 movement in the environment, such as those described in the U.S. patent applications incorporated herein. In one example, the vehicle computing system may determine the trajectory 512 based on the classification as a pedestrian 508. In such examples, the trajectory 512 may be based on an average speed of the pedestrian 508, a maximum speed of the pedestrian 508, etc. In some examples, the trajectory 512 may be determined based on one or more characteristics of the pedestrian 508. The characteristics may include body positioning, clothing, footwear, etc. For example, a trajectory 512 associated with a pedestrian 508 in business attire may be 2 miles per hour, while a trajectory 512 associated with a pedestrian 508 wearing running clothes may be 4 miles per hour (although any values ​​may be used herein).

[0102] In various examples, based on the trajectory 512, the vehicle computing system may determine a time delay (e.g., an object time delay) associated with a blockage of the path 504. The object time delay may include a time required for a pedestrian to cross the path 504, cross a lane associated with the object 506, and / or cross a road associated with the object 506. In various examples, the vehicle computing system may determine one or more actions to take at the intersection 510 based in part on the object time delay. For example, the vehicle computing system may determine that the vehicle 502 may remain at the initial position 514 to wait for the object 506 to cross (first action), may accelerate in front of the object 506 with a first acceleration (second action), and may accelerate in front of the object 506 with a second acceleration (third action).

[0103] In various examples, the vehicle computing system may determine a cost associated with each of the one or more actions based on one or more factors. As described above, the one or more factors may include safety, comfort, progress, and / or operating rules. In an illustrative example, the vehicle computing system may determine that at least the costs of the object 506 and the vehicle 502 associated with safety and operating rules, and the cost of the object 506 associated with comfort, may be the same for each action. As such, the vehicle computing system may determine a cost associated with the progress of the object 506 and / or the comfort and progress of the vehicle 502.

[0104] In various examples, a cost of travel associated with the object 506 may be determined based on a time delay of the object (e.g., a delay in the movement of the object 506 for a pedestrian 508). In one example, the vehicle computing system may determine a time required to accelerate out of the path 504 of the object 506 through an intersection 510. In such an example, the cost of travel may be associated with a difference between the time required to pass in front of the object 506 and the time delay of the object. The difference may represent an additional delay of the object 506 from a forward motion based on the vehicle 502 action. For example, an action with acceleration may cause an additional delay of the object 506 for one second, resulting in a cost of travel of one for the object 506. In another example, an action with acceleration causes no additional delay of the object 506, resulting in a cost of travel of zero for the object 506 associated with the action.

[0105] As described above, the comfort cost and / or the cost of progression of the vehicle 502 may be based on the acceleration from the initial position 514. For example, the vehicle computing system may determine that a comfort cost associated with an action having a first acceleration is 1 and a cost of progression is 1, and a second comfort cost associated with a second action having a second acceleration is 2 and a cost of progression is 1, where the first acceleration is slower (e.g., more comfortable) than the second acceleration.

[0106] In various examples, the vehicle computing system may determine an action to take based on a total cost of each of the one or more actions (e.g., the determined costs associated with one or more factors). In various examples, the vehicle computing system may cause the vehicle to be controlled based on the action.

[0107] 6 illustrates a vehicle 602 in an environment 600, such as the vehicle 102 in the environment 100, configured to determine an action 604 to take based at least in part on an action reward. In various examples, the action reward may include the opposite of the action penalty (e.g., an increase in cost). In such examples, the action reward may result in a decrease in a cost associated with the action 604.

[0108] In one example, the vehicle computing system may detect one or more objects 606 in the environment 600. The vehicle computing system may detect the object 606 based on sensor data received from one or more sensors. In one example, the sensor may include sensors mounted on the vehicle 602, such as, for example, a camera, a motion detector, a lidar, a radar. In one example, the sensor may include one or more remote sensors. As illustrated in FIG. 6, the vehicle computing device may detect an object 606(1) behind the vehicle 602 and determine that the object 606(1) intends to turn right at an intersection 608 (e.g., follow an object path 614) based on indicators 610, etc. (e.g., turn signals, blinkers, etc.).

[0109] In various examples, the vehicle computing system may be configured to determine one or more actions 604 to take based on the environment 600 and / or the detected object 606. For example, a vehicle 602 approaching an intersection 608 having a stop sign 612 is requested to slow down to a stop (an environment 600-based determination). The vehicle computing device may determine that the vehicle 602 may perform a first action 604(1) including maintaining its position in the lane, or may perform a second action 604(2) including adjusting its position in the lane while slowing down to a stop (an object-based determination). In an illustrative example, the vehicle computing device may determine that a lane change to the left is not a viable option based in part on the detection of a second object 606(2) in an adjacent lane. In other examples where the adjacent lane is unoccupied, the vehicle computing device may determine that a lane change may be a viable option.

[0110] As described above, the vehicle computing system may determine a cost associated with each action 604 based on one or more factors (e.g., safety, comfort, progress, operating rules, etc.). In various examples, the vehicle computing device may determine that the second action 604(2) may provide sufficient space for the object 606(1) to operate on the right side of the vehicle 602 in the lane, thereby reducing the time required for the object 606(1) to reach the destination with which it is associated. In some examples, the vehicle computing device may evaluate a progress reward (e.g., an action reward) for the action 604(2). In some examples, the progress reward may be applied as a negative cost. In such examples, the progress reward may result in a decrease in the total cost of the action 604(2). The vehicle computing device may determine the action to take based on a determination that the action 604 has the lowest cost with which it is associated. For example, a first action 604(1) and a second action 604(2) may have substantially the same safety, comfort, and / or operating rules costs. Thus, a reduction in the total cost resulting from the progression reward may result in the second action 604(2) having a lower cost associated therewith. As such, the vehicle computing system may select the second action 604(2) as the action 604 to be taken by the vehicle.

[0111] In various examples, the vehicle computing system may determine that the safety, comfort, and / or operational rule factors are substantially the same for actions 604(1) and 604(2). In such examples, the vehicle computing system may not assess the costs associated with the safety, comfort, and / or operational rule factors, instead focusing the determination of action 604 on whether an action reward is applied to the action. Thus, the vehicle computing system may determine action 604(2) based on the action reward applied to action 604(2) in a manner that adjusts position to provide object 606(1) the ability to progress faster through intersection 608 according to object path 614.

[0112] 7 is a block diagram of an example system 700 for implementing the techniques described herein. In at least one example, the system 700 may include a vehicle 702, such as vehicle 102.

[0113] The vehicle 702 may include a vehicle computing device 704 , one or more sensor systems 706 , one or more emitters 708 , one or more communication connections 710 , at least one direct connection 712 , and one or more drive systems 714 .

[0114] The vehicle computing device 704 may include one or more processors 716 and a memory 718 communicatively coupled to the one or more processors 716. In the illustrated example, the vehicle 702 is an autonomous vehicle, however, the vehicle 702 may be any other type of vehicle, such as a semi-autonomous vehicle, or any other system having at least an image capture device (e.g., a camera-enabled smartphone). In the illustrated example, the memory 718 of the vehicle computing device 704 stores a localization component 720, a perception component 722, a planning component 724 including a cost of action component 730 and a prediction component 732, one or more system controllers 726, and one or more maps 728. 7 as residing in memory 718 for illustrative purposes, it is contemplated that localization component 720, perception component 722, planning component 724, one or more system controllers 726, one or more maps 728, cost of action component 730, and prediction component 732 may additionally or alternatively be accessible to vehicle 702 (e.g., stored relative to memory 702 remote from vehicle 702, such as to memory 734 of one or more computing devices 736, or otherwise accessible from vehicle 702). For example, memory 734 may store cost of action component 738 accessible by planning component 724 of vehicle 702.

[0115] In at least one example, the localization component 720 may include functionality for receiving data from the sensor system 706 to determine the position and / or orientation of the vehicle 702 (e.g., one or more of an x-position, a y-position, a z-position, a roll, a pitch, or a yaw). For example, the localization component 720 may include and / or request / receive a map of the environment, such as from a map 728, and may continually determine the position and / or orientation of the autonomous vehicle within the map. In one example, the localization component 720 may utilize simultaneous localization and mapping (SLAM), calibration, localization and mapping, simultaneously (CLAMS), relative SLAM, bundle adjustment, nonlinear least squares optimization, etc., to receive image data, lidar data, radar data, IMU data, GPS data, wheel encoder data, etc., to accurately determine the position of the autonomous vehicle. In one example, the localization component 720 may provide data to various components of the vehicle 702 as described herein to determine an initial position of the autonomous vehicle for determining the relevance of the vehicle 702 to objects.

[0116] In one example, the perception component 722 may include functionality to perform object detection, segmentation, and / or classification. In one example, the perception component 722 may provide processed sensor data indicative of the presence of an object (e.g., an entity) proximate to the vehicle 702 and / or a classification of the object as a type of object (e.g., automobile, pedestrian, bicyclist, animal, structure, tree, road surface, curb, sidewalk, unknown, etc.). In one example, the perception component 722 may provide processed sensor data indicative of the presence of a stationary entity proximate to the vehicle 702 and / or a classification of the stationary entity as a type (e.g., structure, tree, road surface, curb, sidewalk, unknown, etc.). In additional or alternative examples, the perception component 722 may provide processed sensor data indicative of one or more features associated with a detected object (e.g., a tracked object) and / or an environment in which the object is located. In one example, features associated with an object may include, but are not limited to, x position (global and / or local position), y position (global and / or local position), z position (global and / or local position), orientation (e.g., roll, pitch, yaw), object type (e.g., classification), object velocity, object acceleration, object range (size), etc. Features associated with an environment may include, but are not limited to, the presence of other objects in the environment, the state of other objects in the environment, time of day, day of the week, season, weather conditions, darkness / light indication, etc.

[0117] In general, the planning component 724 may determine a path for the vehicle 702 to follow to traverse an environment. For example, the planning component 724 may determine various routes and vehicle trajectories, as well as various levels of detail. For example, the planning component 724 may determine a path to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For purposes of this description, the route may include successive waypoints for travel between the two locations. As non-limiting examples, the waypoints include roadways, intersections, Global Positioning System (GPS) coordinates, and the like. Additionally, the planning component 724 may generate instructions for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 724 may determine how to guide the autonomous vehicle from a first waypoint in the succession of waypoints to a second waypoint in the succession of waypoints. In one example, the instructions may be a vehicle trajectory, or may be a portion of a trajectory. In one example, the multiple trajectories may be generated substantially simultaneously (e.g., within technical tolerances) according to a receding horizon technique, where one of the multiple trajectories is selected for the vehicle 702 to navigate.

[0118] In various examples, the route may include one or more actions of the actions that the planning component 724 selects based on action costs, as described herein. In such examples, the planning component 724 may include an action cost component 730 configured to determine one or more actions applicable to the environment. The actions may include reference actions and sub-actions that the vehicle 702 may perform. In one example, the action cost component 730 may receive one or more predicted object trajectories associated with each action of the one or more actions. For example, the prediction component 732 may generate one or more predicted trajectories for an object within a threshold distance from the vehicle 702 based on a predicted reaction of the object to the vehicle 702 performing the action. In one example, the prediction component 732 may measure a trace of the object and generate a trajectory for the object based on the observation and predicted behavior. In one example, the prediction component 732 may utilize one or more of machine learning techniques, a top-down representation of the environment, heat maps, temporal logic formulas, and / or tree search methods, as described in the U.S. patent applications incorporated by reference herein, to determine a predicted object trajectory.

[0119] In various examples, the cost of actions component 730 may determine a cost associated with each action based on one or more factors, such as safety, comfort, progress, and / or operating rules. In some examples, the cost may be associated with an object, such as based on a predicted object trajectory and / or a cost associated with the vehicle associated with the action. In various examples, the planning component 724 may determine the actions to take based on the costs associated with each of the actions. For example, the planning component 724 may select the action associated with the lowest overall cost and may cause the vehicle 702 to perform the action.

[0120] In various examples, the cost of action component 738 stored on the computing device 736 may be configured to perform similar functions as the cost of action component 730 (e.g., determining one or more actions that the vehicle 702 may take and calculating a cost associated with each action, etc.). In one example, the cost of action component 738 may receive one or more predicted object trajectories from the prediction component 732. In one example, the cost of action component may receive sensor data (e.g., raw and / or processed sensor data) from the sensor system 706 and / or the perception component 722. In such an example, the cost of action component 738 may be configured to determine one or more predicted object trajectories based on one or more actions utilizing any of the techniques for determining predicted object trajectories described herein. The cost of action component 738 may utilize the predicted object trajectories (e.g., received and / or determined) to determine a cost associated with each action. In one example, the cost of actions component 738 may transmit a cost associated with each action to the vehicle computing device 704. In one example, the cost of actions component 738 may transmit a cost associated with the action with the lowest cost to the vehicle computing device 704.

[0121] In at least one example, vehicle computing device 704 may include one or more system controllers 726, which may be configured to control steering, propulsion, braking, safety, emitter, communications, and other systems of vehicle 702. System controller 726 may communicate with and / or control systems corresponding to drive system 714 and / or other components of vehicle 702.

[0122] The memory 718 may further include one or more maps 728 that may be used by the vehicle 702 to navigate within the environment. For purposes of this description, a map may be any number of data structures modeled in two, three, or N dimensions that may provide information about the environment, such as, but not limited to, topology (such as intersections), roadways, mountain ranges, roads, terrain, and the general environment. In one example, the map may include, but is not limited to, texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), etc.), intensity information (e.g., lidar information, radar information, etc.), spatial information (e.g., image data projected onto a mesh, individual "surfels" (e.g., polygons associated with individual colors and / or intensities), reflectivity information (e.g., specularity information, retroreflectivity information, BRDF information, BSSRDF information, etc.). In one example, the map may include a three-dimensional mesh of the environment. In one example, the vehicle 702 may be controlled at least in part based on the map 728. That is, the map 728 may be used in conjunction with the localization component 720, the perception component 722, and / or the planning component 724 to determine the position of the vehicle 702, detect objects in the environment, generate routes, determine actions and / or trajectories, and navigate within the environment.

[0123] In one example, one or more maps 728 may be stored on a remote computing device (such as computing device 736) accessible via one or more networks 740. In one example, multiple maps 728 may be stored, for example, based on characteristics (e.g., type of entity, time of day, day of the week, season, etc.). Storing multiple maps 728 may have similar memory requirements but may increase the speed at which data in the maps may be accessed.

[0124] As will be appreciated, the components described herein (e.g., localization component 720, perception component 722, planning component 724 including action cost component 730 and prediction component 732, one or more system controllers 726, one or more maps 728) are described as separated for illustrative purposes, however, the operations performed by the various components may be combined or performed in any other component.

[0125] In one example, some or all aspects of the components described herein may include any model, technique, and / or machine learning technique. For example, in one example, the components in memory 718 (and memory 734 described below) may be implemented as neural networks.

[0126] As described herein, an exemplary neural network is a biologically inspired technology that passes input data through successively connected layers to generate an output. Each layer in a neural network may also include another neural network, or may include any number of layers (whether convolutional or not). As can be understood in the context of the present disclosure, a neural network may utilize machine learning, which may refer to a broad class of such technologies in which an output is generated based on learned parameters.

[0127] Although described in the context of neural networks, any type of machine learning may be used consistent with this disclosure.For example, machine learning techniques include regression techniques (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), instance-based techniques (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), Elastic Net, least-angle regression (LARS)), decision tree techniques (e.g., classification and regression tree (CART), iterative dichotomiser 3 (ID3), chi-squared automated interaction detection (CHAID), decision strains, conditional decision trees), Bayesian techniques (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, average one-dependence (AODE)), and others). estimators), Bayesian confidence networks (BNNs), Bayesian networks), clustering techniques (e.g., k-means, k-medians, expectation maximization (EM), hierarchical clustering), association rule learning techniques (e.g., perceptrons, backpropagation, Hopfield networks, RBFNs (Radial Basis Function Network), deep learning techniques (e.g., deep Boltzmann machine (DBM), deep confidence network (DBN), convolutional neural network (CNN), stacked autoencoder), dimensionality reduction techniques (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), mixed discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble techniques (e.g., boosting, bootstrap aggregation (bagging), adaboost, hierarchical generalization (blending), gradient boosting machine (GBM), gradient boosted regression tree (GBRT), random forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc., but are not limited to these.Further example architectures include neural networks such as ResNet50, ResNet101, VGG, DenseNet, PointNet, etc.

[0128] In at least one example, the sensor system 706 may include a lidar sensor, a radar sensor, an ultrasonic transducer, a sonar sensor, a position sensor (e.g., GPS, compass, etc.), an inertial sensor (e.g., inertial measurement unit (IMU), accelerometer, magnetometer, gyroscope, etc.), a camera (e.g., RGB, IR, intensity, depth, time of flight, etc.), a microphone, a wheel encoder, an environmental sensor (e.g., temperature sensor, humidity sensor, light sensor, pressure sensor, etc.), etc. The sensors 706 may include various examples of each of these or other types of sensors. For example, the lidar sensor may include individual lidar sensors located at the corners, front, rear, sides, and / or top of the vehicle 702. As another example, the camera sensor may include various cameras positioned at various locations about the exterior and / or interior of the vehicle 702. The sensors 706 may provide input to the vehicle computing device 704. Additionally or alternatively, the sensor system 706 may transmit sensor data over one or more networks 740 to one or more computing devices 736 at a particular frequency, after a predetermined period of time, in near real-time, or the like.

[0129] The vehicle 702 may also include one or more emitters 708 for emitting light and / or sound. The emitters 708 may include interior audio and visual emitters for communicating with occupants of the vehicle 702. By way of example, and not by way of limitation, the interior emitters may include speakers, lights, signs, display screens, touch screens, haptic emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, head rest positioners, etc.), and the like. The emitters 708 may also include exterior emitters. By way of example, and not by way of limitation, the exterior emitters may include lights for signaling the direction of travel or other indicators of the vehicle's actions (e.g., indicator lights, signs, light arrays, etc.), as well as one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) for acoustically communicating with pedestrians or one or more other nearby vehicles, including acoustic beam steering technology.

[0130] Vehicle 702 may also include one or more communication connections 710 that enable communication between vehicle 702 and one or more other local or remote computing devices. For example, communication connections 710 may facilitate communication with other local computing devices on vehicle 702 and / or drive module 714. Communication connections 710 may also enable the vehicle to communicate with other nearby computing devices (e.g., computing device 736, other nearby vehicles, etc.) and / or one or more remote sensor systems 742 for receiving sensor data.

[0131] The communication connection 710 may include a physical and / or logical interface for connecting the vehicle computing device 704 to another computing device or network, such as the network 740. For example, the communication connection 710 may enable Wi-Fi based communications over frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies such as Bluetooth, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communication protocol that allows each computing device to interface with other computing devices.

[0132] In at least one example, the vehicle 702 may include one or more drive systems 714. In some examples, the vehicle 702 may have one drive system 714. In at least one example, when the vehicle 702 has multiple drive systems 714, the individual drive systems 714 may be located at opposite ends of the vehicle 702 (e.g., the front and the rear, etc.). In at least one example, the drive system 714 may include one or more sensor systems that detect conditions surrounding the drive system 714 and / or the vehicle 702. By way of example and not by way of limitation, the sensor systems may include one or more wheel encoders (e.g., rotary encoders) for sensing the rotation of the wheels of the drive system, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) for measuring the orientation and acceleration of the drive module, cameras or other imaging sensors, ultrasonic sensors for acoustically detecting objects surrounding the drive module, lidar sensors, radar sensors, etc. Certain sensors, such as wheel encoders, may be unique to the drive system 714. In some cases, a sensor system on drive system 714 may overlap with a corresponding system on vehicle 702 (e.g., sensor system 706) or may complement a corresponding system on vehicle 702.

[0133] The drive system 714 may include many of the vehicle systems, including a high voltage battery, a motor for propelling the vehicle, an inverter for converting direct current from the battery to alternating current for use by other vehicle systems, a steering system including a steering motor and a steering rack (which may be electric), a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for distributing braking force to mitigate loss of traction and maintain control, an HVAC system, lighting (e.g., lighting such as head / tail lights for illuminating the exterior surroundings of the vehicle), and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electrical components such as DC / DC converters, high voltage junctions, high voltage cables, charging systems, charging ports, etc.). Additionally, the drive system 714 may include a drive module controller for controlling the operation of various vehicle systems that may receive and pre-process data from the sensor systems. In one example, the drive module controller may include one or more processors and memory communicatively coupled to the one or more processors. The memory may store one or more systems and perform various functions of the drive system 714. Additionally, drive system 714 may also include one or more communication connections that enable each drive system to communicate with one or more other local or remote computing devices.

[0134] In at least one example, the direct connection 712 may provide a physical interface to couple one or more drive systems 714 with the body of the vehicle 702. For example, the direct connection 712 may enable the transfer of energy, fluid, air, data, etc. between the drive systems 714 and the vehicle. In one example, the direct connection 712 may further releasably secure the drive systems 714 to the body of the vehicle 702.

[0135] In at least one example, the localization component 720, the perception component 722, the planning component 724, the one or more system controllers 726, the one or more maps 728, the cost of action component 730, and the prediction component 732 may process data (e.g., sensor data) as described above and may transmit their respective outputs to the computing device 736 over one or more networks 740. In at least one example, the localization component 720, the perception component 722, the planning component 724, the one or more system controllers 726, the one or more maps 728, the cost of action component 730, and the prediction component 732 may transmit their respective outputs to the computing device 736 at a particular frequency, such as after the passage of a predetermined period of time, in near real-time, or the like.

[0136] In one example, vehicle 702 may transmit sensor data to computing device 736 over network 740. In one example, vehicle 702 may receive sensor data from computing device 736 and / or remote sensor system 742 over network 740. The sensor data may include raw sensor data and / or processed sensor data and / or representations of sensor data. In one example, the sensor data (raw or processed) may be transmitted and / or received as one or more log files.

[0137] The computing device 736 may include one or more processors 744 and a memory 734 that stores a cost of action component 738 and a sensor data processing component 746. In various examples, the sensor data processing component 746 may be configured to receive data from one or more remote sensors, such as the sensor system 706 and / or the remote sensor system 742. In one example, the sensor data processing component 746 may be configured to process the data and transmit the processed data to the cost of action component 738 for determining a cost associated with a possible or candidate action of the vehicle 702. In one example, the sensor data processing component 746 may be configured to process the data and transmit the processed sensor data to the vehicle computing device 704 for use by the cost of action component 730 and / or the prediction component 732. In one example, the sensor data processing component 746 may be configured to transmit raw sensor data to the vehicle computing device 704.

[0138] The processor 716 of the vehicle 702 and the processor 744 of the computing device 736 may be any suitable processor capable of executing instructions to process data and perform operations as described herein. For purposes of illustration and not limitation, the processors 716 and 744 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or portion of a device that processes electronic data and converts the electronic data into registers and / or other electronic data that may be stored in memory. In certain examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices may also be considered processors so long as they are configured to implement coded instructions.

[0139] The memories 718 and 734 are examples of non-transitory computer-readable media. The memories 718 and 734 may store an operating system and one or more software applications, instructions, programs, and / or data to implement the methods and functions attributed to the various systems described herein. In various implementations, the memories may be implemented using any suitable memory technology, such as static RAM (SRAM), synchronous DRAM (SDRAM), non-volatile / flash type memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein may include many other logical, programmatic, and physical components, of which those shown in the accompanying drawings are merely examples relevant to the description herein.

[0140] In some examples, memories 718 and 734 may include at least working memory and storage memory. For example, working memory may be a limited amount of high-speed memory (e.g., cache memory) used to store data manipulated by processors 716 and 744. In some examples, memories 718 and 734 may include storage memory, which may be a relatively large amount of slower memory used for long-term storage of data. In some examples, processors 716 and 744 may not be able to directly manipulate data stored in storage memory, as described herein, and data may need to be loaded into working memory to perform operations based on the data.

[0141] 7 is illustrated as a distributed system, it should be noted that in alternative illustrations, components of the vehicle 702 may be associated with the computing device 736 and / or components of the computing device 736 may be associated with the vehicle 702. That is, the vehicle 702 may perform one or more functions associated with the computing device 736, and vice versa.

[0142] 8-10 illustrate exemplary processes according to embodiments of the present disclosure. These processes are illustrated as logical flow graphs, with each operation representing a sequence of operations that may be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement a process.

[0143] 8 illustrates an example process 800 for controlling a vehicle according to an action based, at least in part, on a cost associated with the action. Some or all of process 800 may be performed by one or more components in FIG. 7 as described herein. For example, some or all of process 800 may be performed by vehicle computing device 704 and / or computing device 736.

[0144] At operation 802, the process may include detecting objects in the environment based at least in part on sensor data associated with sensors of the vehicle. The sensors may include cameras, motion detectors, lidar, radar, time-of-flight, etc. In one example, the vehicle computing system may receive sensor data from one or more remote sensors, such as, for example, sensors onboard another autonomous vehicle and / or sensors onboard the environment.

[0145] In various examples, the vehicle may be configured to transmit and / or receive data from other autonomous vehicles and / or sensors. The data may include sensor data, such as data about objects detected in the environment. In various examples, the environment may include sensors for traffic monitoring, collision avoidance, etc. In various examples, the vehicle computing system may receive the sensor data and determine a type of object (e.g., classify the type of object), such as whether the object is a car, truck, bus, semi-trailer truck, motorcycle, moped, bicyclist, pedestrian, etc.

[0146] At operation 804, the process may include determining actions that a vehicle, such as vehicle 102, may take in the environment. The actions may represent one or more possible paths (e.g., one or more vehicle trajectories) that the vehicle may take through the environment. The actions may include one or more reference actions (e.g., one of a group of maneuvers that the vehicle is configured to perform in response to a dynamic operating environment), such as change lane right, change lane left, stay in lane, go around an obstacle (e.g., double-parked vehicles, traffic cones, etc.), and the like. The actions may further include one or more sub-actions, such as a change in speed (e.g., maintain speed, accelerate, decelerate, etc.), a change in position (e.g., change position in lane), and the like. For example, the actions may include staying in lane (reference action) and adjusting the vehicle's position in the lane from a center position to operating on the left side of the lane (sub-action).

[0147] At operation 806, the process may include determining an object trajectory (e.g., a predicted trajectory) associated with the object based at least in part on the action. The object trajectory may be determined based on sensor data. The object trajectory may represent a possible path that the detected object may travel through the environment. The object trajectory may be based on a predicted reaction (e.g., an active prediction) of the object to a vehicle action. For example, the action may include the vehicle accelerating into an intersection where another vehicle approaches from the right and has the right of way. The vehicle computing system may determine that the predicted trajectory of the other vehicle may include a rapid negative acceleration (deceleration) to avoid a collision with the vehicle. In various examples, the object trajectory may be determined utilizing a top-down representation of the environment, heat maps, temporal logic formulas, tree search methods, machine learning techniques, or other means of determining active predictions associated with objects in the environment, as described in U.S. patent applications incorporated herein by reference. In such examples, one or more of the sensor data, a representation of the sensor data (e.g., bounding box, range, etc.), a map, road network information, and a path associated with an action (all of which may include a sequence of data over a period of time) may be input into a trained machine learning model to output a predicted trajectory of the selected object. In one example, such a cost may be calculated relative to a baseline cost for the predicted trajectory of the object without the vehicle (e.g., what the object would do if the vehicle were not present).

[0148] At operation 808, the process may include determining a cost associated with the action based at least in part on the object trajectory. The cost may be based in part on the impact of the action on objects (e.g., another vehicle, a bicyclist, a pedestrian, etc.) operating in the environment with the vehicle. As described above, the cost may be based on one or more factors. The factors may include safety (e.g., probability of collision, distance from other objects, etc.), comfort (e.g., lack of sudden movements, acceleration), progress (e.g., movement to a destination, delays, etc.), operating rules (e.g., rules of the road, laws, ordinances, regulations, etc.), although any number of other considerations are contemplated. The factors may be attributable to the vehicle and / or the object. For example, the action may cause a delay in progress to the vehicle and the object due to the vehicle and the object slowing down, etc.

[0149] In various examples, the vehicle computing system may determine the cost based on a cost associated with a future state (e.g., estimated state) of the vehicle and / or object as based on the respective trajectories. In such examples, the vehicle computing system may determine the estimated state by projecting the vehicle and object in the environment forward for a time period (e.g., 6 seconds, 8 seconds, 16 seconds, etc.). The vehicle computing system may project the object forward (e.g., estimate the future position of the object) based on the object trajectory. The vehicle computing system may project the vehicle forward (e.g., estimate the future position of the vehicle) based on one or more vehicle trajectories associated with the action. The estimated state may represent an estimated location (e.g., estimated position) of the vehicle and an estimated position of the object at a time in the future. In various examples, the vehicle computing system may determine the estimated state at a predetermined rate (e.g., 10 Hertz, 20 Hertz, 50 Hertz, etc.) throughout the time period. In at least one example, the estimated state may be performed at a rate of 10 Hertz (e.g., every 0.1 seconds).

[0150] In various examples, the vehicle computing system may determine a cost associated with each estimated state based on the estimated positions of the vehicle and the object relative to one another. In some examples, the vehicle computing system may analyze each estimated state based on one or more factors and apply a cost value to the estimated state. In such examples, the cost of the estimated state may include a sum of the costs associated with each of the factors. In various examples, the action cost may include a sum of each of the costs associated with each estimated state.

[0151] At operation 810, the process may include comparing the cost associated with the action to costs associated with other actions the vehicle may take (e.g., as may be calculated in a similar manner for the actions described above). In various examples, the vehicle computing system may compare the total cost of the action (e.g., the sum of the costs associated with each estimated state) to the costs associated with other actions. In one example, the vehicle computing device may compare the total cost of the action associated with the factors and / or vehicle and / or object (e.g., safety cost, comfort cost of the object, etc.) to the corresponding costs associated with other actions. For example, the vehicle computing device may compare a first total safety cost associated with a first action to a second total safety cost associated with a second action. In at least one example, a cost associated with a specific time step may be compared to a similar instantaneous cost associated with further actions.

[0152] At operation 812, the process may include determining whether a cost associated with the action is the lowest cost compared to other actions. In various examples, the vehicle computing system may determine whether a total cost associated with the action is the lowest cost. In one example, the vehicle computing system may determine that a total cost associated with factors (e.g., safety cost, comfort cost of the object, etc.) is the lowest cost action.

[0153] Based on a determination that the cost is the lowest cost action ("Yes" at operation 812), the process may include causing the process to control the vehicle based at least in part on the action at operation 814. In various examples, causing the process to control the vehicle based on the action may include causing the vehicle to travel along one or more trajectories associated with the action.

[0154] Based on a determination that the cost is not the lowest cost action ("No" at operation 812), the process causes the vehicle to be controlled based at least in part on another action at operation 816. The other action may include another action having a lower cost than the action. The other action may be associated with one or more other vehicle trajectories.

[0155] 9 illustrates an example process 900 to illustrate an example process for controlling a vehicle according to an action based at least in part on a cost of safety associated with the action. Some or all of the process 900 may be performed by one or more components in FIG. 7 as described herein. For example, some or all of the process 900 may be performed by the vehicle computing device 704.

[0156] At operation 902, the process may include determining actions that the vehicle may take in the environment. The actions may represent one or more possible paths (e.g., one or more vehicle trajectories) that the vehicle may take through the environment. The actions may include one or more reference actions (e.g., one of a group of maneuvers that the vehicle is configured to perform in response to a dynamic operating environment), such as lane change right, lane change left, staying in the lane, going around an obstacle (e.g., double-parked vehicles, traffic cones, etc.), and the like. The actions may further include one or more sub-actions, such as a change in speed (e.g., maintain speed, accelerate, decelerate, etc.), a change in position (e.g., change position in the lane), and the like. For example, the actions may include staying in the lane (reference action) and adjusting the vehicle's position in the lane from a center position to operating on the left side of the lane (sub-action). Such an action may be one of multiple actions, which may be evaluated substantially simultaneously in certain examples and which the vehicle may be capable of performing at any instant in time.

[0157] At operation 904, the process may include determining an object trajectory (e.g., a predicted object trajectory indicating where the object may go at various times in the future) associated with an object operating in the environment based at least in part on the action. The object trajectory may be determined based on sensor data. The object trajectory may represent a possible path that the detected object may travel through the environment. The object trajectory may be based on a predicted reaction (e.g., active prediction) of the object to a vehicle action. For example, the action may include the vehicle accelerating into an intersection where another vehicle approaches from the right and has the right of way. The vehicle computing system may determine that the predicted trajectory of the other vehicle may include a rapid negative acceleration (deceleration) to avoid a collision with the vehicle. In various examples, the object trajectory may be determined utilizing a top-down representation of the environment, heat maps, temporal logic formulas, tree search methods, machine learning techniques, or other means of determining active predictions associated with objects in the environment, as described in U.S. patent applications incorporated herein by reference. These examples are provided with respect to FIG. 8.

[0158] At operation 906, the process may include determining a cost of safety associated with the action based at least in part on the object trajectory. In various examples, the cost of safety may include a likelihood of a collision between the vehicle and the object (e.g., a probability of a collision). The likelihood of a collision may be calculated based on a distance between the vehicle and the object (e.g., within 5 feet, 2 meters, 0.5 meters, etc.), a convergence trajectory (e.g., an object trajectory that will substantially intersect with a vehicle trajectory associated with the action), a convergence rate between the vehicle and the object (e.g., 2 meters per second, 10 feet per second, etc.), etc. In one example, the likelihood of a collision may be based on a threshold associated with the distance and / or the convergence rate. For example, a distance between an estimated state associated with the vehicle and an estimated position associated with the object may be less than or equal to a threshold distance (e.g., 8 feet, 3 meters, etc.). Thus, the vehicle computing system may determine that a likelihood of a collision exists between the vehicle and the object. For another example, the trajectory associated with the vehicle and the trajectory associated with the object may converge at a rate equal to or greater than a threshold rate of convergence (e.g., convergence of 6 feet per second, 2.5 meters per second, 1.5 meters per second, etc.). As such, the vehicle computing system may determine that a likelihood of a collision exists between the vehicle and the object. In one example, the likelihood or probability of a collision may be based on the amount by which the actual distance and / or convergence rate is below or above the threshold. In such an example, the probability of a collision may increase the closer the object is to the vehicle and is in an estimated state (e.g., 95% probability of collision if within 6 inches of each other) and / or the amount by which the convergence rate is above the threshold (e.g., 90% probability of collision with a convergence rate of 10 meters per second). In one example, the probability of a collision may be determined utilizing machine learning techniques. In such an example, a machine learning model may be trained to determine the probability of a collision based on training data including situations in which the vehicle and the object interact in the environment.In one example, the cost of safety may be determined based on a convergence rate (e.g., 3 feet per second = 3 cost of safety, 3 feet per second = 9 cost of safety, etc.), although any other linear and / or non-linear combinations of such parameters are contemplated.

[0159] In various examples, the cost of safety may be based on the probability of a collision. In one example, if the probability of a collision is higher than a predefined threshold (e.g., 30%, 50%, etc.) indicating that a collision may occur, a fixed cost (e.g., 50, 75, 90, etc.) may be included. In one example, the cost may be based on the probability of a collision decision. For example, the vehicle computing system may determine that the probability of a collision in an estimated state is 95%, and the vehicle computing system may evaluate a cost of 95 against the safety factor of the estimated state, although any linear / non-linear function of probability (polynomial, product, etc.) is contemplated as above.

[0160] In various examples, the vehicle computing system may emphasize the safety of the other factors listed above by increasing the determined safety cost, such as that based on the probability of collision, by a factor of two, three, four, etc., or by applying a polynomial function (e.g., quadratic, cubic, etc.) to the determined safety cost. In such examples, the vehicle computing system may evaluate the safety cost at an initial value, such as that based on the likelihood of collision, and may increase the safety cost by applying a factorial and / or polynomial function. For example, the vehicle computing system may calculate the total safety cost=(determined safety cost) 2 A quadratic equation may be applied to the cost of safety such as:

[0161] At operation 908, the process may include determining whether the safety cost is above a threshold. The threshold may be defined to ensure safe operation of the vehicle. In various examples, the threshold may include a predetermined maximum cost associated with the safety factor. In one example, the threshold safety cost may be dynamically determined based on the number of objects detected in the environment, a classification of the detected objects, an operational area (e.g., school zone, construction area, highway, business district, etc.), and the like. For example, the vehicle computing system may set the threshold safety cost to 5 for vehicles operating in areas where a threshold number of pedestrians are detected and set the threshold safety cost to 10 for vehicles operating on highways.

[0162] Based on a determination that the cost of safety is above the threshold ("Yes" at operation 908), the process may include overriding the action in vehicle planning considerations at operation 910. In various examples, the vehicle computing system may select an alternative action for controlling the vehicle.

[0163] Based on the safety cost not exceeding the threshold ("No" at operation 908), the process may include determining a total cost associated with the action based on one or more factors at operation 912. As described above, the factors may include safety factors, comfort factors, progression factors, and / or operating rule factors associated with the vehicle and / or object. The total cost may include a total cost of the individual factors (e.g., safety cost, cost of progression of the object, etc.) and / or a total cost of the action including each of the factors.

[0164] At operation 914, the process may include an action in the vehicle planning considerations. In various examples, the vehicle computing system may compare a total cost associated with the action to a total cost associated with other actions the vehicle may take in the environment. In various examples, the vehicle computing system may select the least costly action of the actions considered. In such examples, the vehicle computing system may cause the vehicle to be controlled based on the least costly action.

[0165] 10 illustrates an example process 1000 for controlling a vehicle based at least in part on a probability that an object is present in an occlusion area. Some or all of the process 1000 may be performed by one or more components in FIG. 7 as described herein. For example, some or all of the process 1000 may be performed by the vehicle computing device 704.

[0166] At operation 1002, the process may include identifying a blocked area in an environment in which the vehicle operates. In various examples, the blocked area may be identified based on sensor data received from one or more sensors (e.g., lidar, radar, camera, etc.) of the vehicle. The blocked area may define an area in the environment in which the vehicle's perception system is not able to detect an object (e.g., blocked by an obstacle, blocked visibility, etc.). In various examples, the vehicle computing system may be configured to identify a relevant section of a road associated with the blocked area. In such examples, the section of the road may include one or more lanes, sidewalks, bike lanes, etc. in which an object may operate in a direction that may be relevant to the vehicle. As described above with respect to FIG. 4, the blocked area may include two lanes of a road in which an object may travel toward the vehicle.

[0167] At operation 1004, the process may include determining a probability that the object is operating in the occluded area. In various examples, the probability may be determined utilizing machine learning techniques and / or one or more techniques described in U.S. Patent Application No. 16 / 147,177 and U.S. Patent Application No. 16 / 011,436, which are incorporated by reference above. In various examples, the vehicle computing system may determine the probability that the object is operating in the occluded area based on the relevant section of the road. In one example, the probability that the object will be traveling through the occluded area may be based on stored data associated with the relevant section of the road. The stored data may include sensor data (e.g., pre-recorded data) associated with the relevant section of the road captured over time. In one example, the probability may be based on time of day, day of the week, month, season, etc. In one example, the probability may be determined utilizing machine learning techniques. In such examples, the machine learning model may be trained with training data such as sensor data associated with the relevant section of the road captured over time. Based on the probability that an object may be present, the vehicle computing system may generate a predicted object with a predicted object trajectory and determine the action to take.

[0168] At operation 1006, the process may include determining whether the probability is above a threshold. The threshold probability may be based on an environment, a section of road, a day of the week, a week number, etc. In various examples, the threshold probability may include a predetermined value. In certain examples, the threshold may be dynamically determined. In such examples, the threshold probability may be determined based on a section of road, a time of day, a day of the week, etc. while the vehicle is operating in the environment.

[0169] Based on a determination that the probability is not above the threshold value ("No" at operation 1006), the process may include determining, at operation 1008, that the object is not moving in an occluded area.

[0170] At operation 1010, the process may include controlling the vehicle based on determining that the object is not operating in the occluded area. In various examples, the vehicle computing system may determine an action to take based on one or more detected objects in the environment utilizing the techniques described above.

[0171] Based on a determination that the probability is above a threshold ("Yes" at operation 1006), the process may include data associated with the object in vehicle planning considerations at operation 1012. In various examples, the vehicle computing system may determine costs associated with one or more possible actions the vehicle may take based on the object such as utilizing the techniques described herein. In such examples, the vehicle computing system may determine an action (e.g., control planning) for the vehicle to take based on the action costs. In various examples, the vehicle computing system may control the vehicle based, at least in part, on objects (e.g., predicted objects) operating in the occluded area.

[0172] (Example clauses) A: a method for determining an environment in which a vehicle is capable of taking a first action in the environment based on at least part of the sensor data; determining a first object trajectory associated with the first action and a second object trajectory associated with the second action; and determining a first action cost associated with the first action based at least partly on the first object trajectory, wherein the first action is at least one of a first safety cost, a first comfort cost, a first progress cost, or a first operating rule cost. determining a second action cost associated with the second action based at least in part on the second object trajectory, where the second action cost is based at least in part on at least one of a second safety cost, a second comfort cost, a second progress cost, or a second operating rule cost; determining that the first action cost associated with the first action is lower than a second action cost associated with the second action; and a memory storing processor-executable instructions for configuring the vehicle to control the vehicle at least in part based on the first action based at least in part on determining that the first action cost is lower than the second action cost.

[0173] B: In a vehicle as described in paragraph A, the first safety cost is based at least in part on a first function of an object state associated with the first object trajectory or a relative state between the first object trajectory and the vehicle, the first comfort cost is based at least in part on a second function of an object state associated with the first object trajectory or a relative state between the first object trajectory and the vehicle, the first travel cost is based at least in part on a first object time delay of the object, and the first operating rule cost is based at least in part on one or more regulations associated with the environment.

[0174] C: A vehicle as described by either paragraph A or B, wherein the instructions further configure the vehicle to determine a third action the vehicle can take in the environment, determine a third object trajectory associated with the third action, determine a third cost associated with the third action, determine that the third cost is higher than a threshold cost, and ignore data associated with the third action from control planning considerations.

[0175] D: A vehicle as described by any of paragraphs A-C, wherein the instructions further configure the vehicle to determine an occluded area in the environment based at least in part on the sensor data, utilize machine learning techniques to determine a probability associated with a second object operating in the occluded area, determine that the probability is above a threshold, and determine a predicted object trajectory associated with the second object, wherein determining the first action cost and the second action cost is further based, at least in part, on the predicted object trajectory associated with the second object.

[0176] E: A vehicle as described in any of paragraphs A-D, wherein the first object trajectory and the second object trajectory are determined based at least in part on at least one of a machine learning algorithm, a top-down representation of the environment, a discretized probability distribution, a temporal logic formula, or a tree search method.

[0177] F: The method comprises determining an object at a first position in the environment based at least in part on sensor data, determining candidate actions for the vehicle to take in the environment, determining object trajectories associated with the candidate actions, determining action costs associated with the candidate actions based at least in part on the object trajectories and vehicle trajectories of the vehicle associated with the candidate actions, and controlling the vehicle based at least in part on the action costs associated with the candidate actions.

[0178] G: Determining an action cost in a manner as described by paragraph F includes at least one of determining a safety cost associated with the candidate action, determining a comfort cost associated with the candidate action based at least in part on the object trajectory, determining a progress cost associated with the candidate action based at least in part on the object trajectory, or determining a cost of an operational rule associated with the candidate action.

[0179] H: In a manner as described by either paragraph F or G, the action cost includes a comfort cost associated with the object, the comfort cost being based at least in part on at least one of a positive acceleration of the object, a negative acceleration of the object, or a lateral acceleration of the object.

[0180] I: In a manner as described by any of paragraphs F-H, the action cost includes a cost of safety, and the cost of safety is based at least in part on at least one of determining a distance between an estimated vehicle position of the vehicle and an estimated object position of the object, determining a convergence rate between the estimated position of the vehicle and the estimated position of the object over a period of time, or determining a probability of a collision between the vehicle and the object.

[0181] J: In a method as described in any of paragraphs F-I, the candidate action is a first candidate action and the object trajectory is a first object trajectory, the method further comprising determining a second candidate action for the vehicle to take in the environment, determining the second object trajectory based at least in part on the second candidate action, determining a safety cost associated with the second candidate action based at least in part on the second object trajectory, determining that the safety cost is higher than a threshold safety cost, and excluding data associated with the second candidate action from control planning considerations based at least in part on determining that the safety cost is higher than the threshold safety cost.

[0182] K: determining a first estimated state of the vehicle and the object in a manner as described in any of paragraphs F-I, the first estimated state including a first estimated position of the vehicle at a first time and a first estimated position of the object at the first time; determining a second estimated state of the vehicle and the object, the second estimated state including a second estimated position of the vehicle at a second time and a second estimated position of the object at the second time; and determining a first cost associated with the first estimated state and a second cost associated with the second estimated state, wherein the step of determining an action cost is based at least in part on the first cost and the second cost.

[0183] L: In a manner as described by any of paragraphs F-K, the candidate action is a first candidate action and the action cost is a first action cost, and controlling the vehicle based at least in part on the first action cost includes determining a second candidate action for the vehicle to take in the environment, determining a second object trajectory based at least in part on the second candidate action, determining a second action cost associated with the second candidate action based at least in part on the second object trajectory, determining that the first action cost is lower than the second action cost, and controlling the vehicle to follow the first action based at least in part on the first action being lower than the second action.

[0184] M: A method as described in any of paragraphs F-L, wherein the object is a first object, and the method further comprises determining an occlusion area in the environment based at least in part on the sensor data, and determining an action cost associated with the candidate action is based at least in part on the occlusion area.

[0185] N: In the manner described in paragraph M, the probability that the second object is operating in the blocked area is based at least in part on at least one of machine learning techniques, a section of the road associated with the blocked area, a number of lanes associated with the blocked area, a size of the blocked area, a time of day, or a day of the week.

[0186] O: The system or device includes a processor and a non-transitory computer-readable medium storing instructions that, when executed, cause the processor to perform a computer-implemented method described in any one of paragraphs F-N.

[0187] P: The system or device comprises a means for processing and a means for storing coupled to the means for processing, the means for storing comprising instructions that configure one or more devices to perform the computer-implemented method described in any one of paragraphs F-N.

[0188] Q: A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to perform operations including determining an object at a first position in an environment based at least in part on sensor data; determining candidate actions for a vehicle to take in the environment; determining object trajectories associated with the candidate actions; determining a cost associated with the candidate actions based at least in part on at least one of an acceleration associated with the object trajectory or a distance between the object trajectory and the vehicle trajectory associated with the candidate action; and controlling the vehicle based, at least in part, on the cost associated with the candidate action.

[0189] R: A non-transitory computer-readable medium as described in paragraph Q, wherein the object is a primary object and the action further includes determining a secondary object in the environment, the secondary object being positioned behind the primary object in the environment and traveling in the same direction as the primary object, determining a first cost associated with the primary object, the first cost being determined based, at least in part, on an acceleration of the object responsive to the candidate action, and determining a second cost associated with the secondary object, the second cost comprising a percentage of the first cost, the cost comprising at least the first cost and the second cost.

[0190] S: A non-transitory computer-readable medium as described in either of paragraphs Q or R, wherein the candidate action is a first candidate action, the object trajectory is a first object trajectory, and the operation further includes determining a second candidate action for the vehicle to take in the environment, determining the second object trajectory based at least in part on the second candidate action, determining a safety cost associated with the second candidate action based at least in part on the second object trajectory, determining that the safety cost is higher than a threshold safety cost, and eliminating data associated with the second candidate action from control planning considerations based at least in part on determining that the safety cost is higher than the threshold safety cost.

[0191] T: A non-transitory computer-readable medium as described in any of paragraphs Q-S, wherein the candidate action is a first candidate action, the object trajectory is a first object trajectory, and the operation further includes determining a second candidate action for the vehicle to take in the environment, determining an object comfort cost associated with the second candidate action based at least in part on an acceleration associated with the second object trajectory, determining that the object comfort cost is higher than a threshold comfort cost, and eliminating data associated with the second candidate action from control planning considerations based at least in part on determining that the comfort cost is higher than the threshold comfort cost.

[0192] U: A non-transitory computer-readable medium as described in any of paragraphs Q-T, wherein determining the action cost includes at least one of determining a safety cost associated with the candidate action, determining a safety cost associated with the candidate action based at least in part on an object trajectory, determining a progress cost associated with the candidate action based at least in part on an object trajectory, or determining a cost of an operational rule associated with the candidate action.

[0193] V: A non-transitory computer-readable medium as described in any of paragraphs Q-U, wherein the operations further include determining occlusion areas in the environment based at least in part on the sensor data, and determining the cost is further based at least in part on the occlusion areas.

[0194] While the exemplary clauses as described above are described with respect to one specific implementation, it should be understood that in the context of this application, the contents of the exemplary clauses can also be implemented via a method, device, system, computer-readable medium, and / or another implementation. Furthermore, any of the exemplary clauses A-V may be implemented alone or in combination with one or more other exemplary clauses A-V.

[0195] (Conclusion) While one or more examples of the technology described herein have been described, various modifications, additions, permutations, and equivalents of those examples are included within the scope of the technology described herein.

[0196] In the description of the examples, reference is made to the accompanying drawings, which form a part of this application, showing certain examples of the claimed subject matter. It is understood that other examples can be used and that modifications or variations, such as structural changes, can be made. Such examples, modifications or variations do not necessarily depart from the scope of the intended inventive subject matter. While steps in this specification are provided in a particular order, in some cases the order can be changed such that certain inputs are provided at different times or in a different order without changing the functionality of the systems and methods described. The disclosed procedures can also be performed in different orders. Furthermore, the various calculations in this specification need not be performed in the order disclosed, and other examples using alternative orders of calculations can be easily implemented. In addition to being reordered, the calculations can also be decomposed into partial calculations with the same results.

Claims

1. A vehicle, A sensor, one or more processors; When executed by one or more of said processors, receiving environmental sensor data from the sensor; identifying an object at a first position in the environment based at least in part on the sensor data; determining a first action and a second action that the vehicle is capable of taking in the environment; determining occlusion areas in the environment based at least in part on the sensor data; determining a first object trajectory associated with the first action and a second object trajectory associated with the second action; determining a probability that the first object trajectory is in the occlusion area; determining a first action cost associated with the first action based at least in part on the first object trajectory and a determination that the probability exceeds a threshold probability, the first action being based at least in part on at least one of a first safety cost, a first comfort cost, a first progress cost, or a first operating rule cost; determining a second action cost associated with the second action based at least in part on the second object trajectory, the second action cost being based at least in part on at least one of a second safety cost, a second comfort cost, a second progress cost, or a second operating rule cost; determining that the first action cost associated with the first action is lower than the second action cost associated with the second action; and and controlling the vehicle based at least in part on the first action based at least in part on determining that a cost of the first action is lower than a cost of the second action. a memory storing processor-executable instructions for configuring the vehicle; A vehicle equipped with.

2. the first safety cost is based, at least in part, on a first function of a state of an object associated with the first object trajectory or a relative state between the first object trajectory and the vehicle; the first comfort cost is based, at least in part, on a second function of a state of the object associated with the first object trajectory or the relative state between the first object trajectory and the vehicle; the cost of the first progression is based, at least in part, on a first object time delay of the object; and a cost of the first operational rule is based, at least in part, on one or more regulations associated with the environment; 2. The vehicle of claim 1.

3. The instruction: determining a third action that the vehicle is capable of taking in the environment; determining a third object trajectory associated with the third action; determining a third cost associated with the third action; determining that the third cost is greater than a threshold cost; and to ignore data associated with the third action from control planning considerations; The vehicle according to claim 1 or 2, further comprising:

4. The instruction: utilizing machine learning techniques to determine a probability associated with a second object operating in the occluded area; determining that the probability is above a threshold; determining a predicted object trajectory associated with the second object; and determining the first action cost and the second action cost is further based, at least in part, on the predicted object trajectory associated with the second object; The vehicle according to any one of claims 1 to 3, further comprising:

5. The first object trajectory and the second object trajectory are Machine learning algorithms, a top-down representation of the environment for generating a predicted trajectory of the object within the environment; Discretized Probability Distributions, A temporal logic formula for evaluating trajectories using temporal logic, or determined at least in part based on at least one of the tree search methods; A vehicle according to any one of claims 1 to 4.

6. determining an object at a first location in the environment based at least in part on the sensor data; determining candidate actions for a vehicle to take in the environment; determining an object trajectory associated with the candidate action; determining an occlusion area in the environment; determining a probability that the object trajectory is in the occlusion area; determining an action cost associated with the candidate action based at least in part on the object trajectory of the vehicle associated with the candidate action, the probability that the object trajectory is in the occlusion area, and vehicle trajectory; controlling the vehicle based, at least in part, on the action costs associated with the candidate actions; A method for providing the above.

7. The step of determining the action cost includes: determining a cost of safety associated with said candidate actions; determining amenity costs associated with the candidate actions based at least in part on the object trajectory; determining a cost of progression associated with said candidate action based at least in part on said object trajectory; or determining a cost of an operational rule associated with the candidate action; The method according to claim 6.

8. The action cost includes a cost of an amenity associated with the object, the amenity cost being: the positive acceleration of said object, a negative acceleration of the object, or based at least in part on at least one of a lateral acceleration of the object; 8. The method according to claim 6 or 7.

9. The action costs include safety costs; The cost of said safety is determining a distance between an estimated vehicle position of the vehicle and an estimated object position of the object; determining a convergence rate indicative of the rate at which the vehicle trajectory and the object trajectory substantially intersect based at least in part on the estimated position of the vehicle and the estimated position of the object over a period of time; or determining a probability of a collision between the vehicle and the object; 9. The method according to any one of claims 6 to 8.

10. the candidate action is a first candidate action, the object trajectory is a first object trajectory, The method comprises: determining a second candidate action for the vehicle to take in the environment; determining a second object trajectory based at least in part on the second candidate action; determining a safety cost associated with the second candidate action based at least in part on the object trajectory; determining that the cost of safety is greater than a threshold cost of safety; excluding from control planning consideration data associated with the second candidate action based, at least in part, on determining that the safety cost is greater than the threshold safety cost; The method of any one of claims 6 to 9, further comprising:

11. determining a first estimated state of the vehicle and the object, the first estimated state including a first estimated position of the vehicle at a first time and a first estimated position of the object at the first time; determining a second estimated state of the vehicle and the object, the second estimated state including a second estimated position of the vehicle at a second time and a second estimated position of the object at the second time; and determining a first cost associated with the first estimated state and a second cost associated with the second estimated state; determining the cost of the action is based, at least in part, on the first cost and the second cost; 11. The method according to any one of claims 6 to 10.

12. the candidate action is a first candidate action, the action cost is a first action cost, Controlling the vehicle based at least in part on the first action cost comprises: determining a second candidate action for the vehicle to take in the environment; determining a second object trajectory based at least in part on the second candidate action; determining a second action cost associated with the second candidate action based at least in part on the second object trajectory; determining that the first action cost is less than the second action cost; and controlling the vehicle to follow the first candidate action based, at least in part, on the first action cost being less than the second action cost.

12. The method according to any one of claims 6 to 11.

13. the object is a first object, The method comprises: determining an occlusion area in the environment based at least in part on the sensor data; determining the action cost associated with the candidate action is based at least in part on the occlusion area.

13. The method according to any one of claims 6 to 12.

14. The second object is Machine learning technology, a section of road associated with said blocked area; the number of lanes associated with said blocked area; the size of the occlusion area; Time, or determining that the vehicle is operating in an occluded area based at least in part on at least one of a day of the week; The method of claim 13.

15. A non-transitory computer readable medium storing instructions that, when executed, cause one or more processors to perform the method of any one of claims 6 to 14.

Images