Vehicle trajectory control using tree search

Abstract

Trajectory generation for controlling the movement or other behavior of an autonomous vehicle may include alternating between determining candidate actions and predicting a future state based on the candidate actions. The technique may include determining a cost associated with the candidate actions, which may include an estimate of the transition cost from a current or previous state of the vehicle to a next state. The cost estimate may be a floor or ceiling cost, and the tree search may apply the floor or ceiling cost alternately, exclusively, or according to a ratio or a changing ratio. The prediction of the future state may be based at least in part on the classification of the dynamic object of the machine learning model as being a reactive object or being an inert object, which may change how the dynamic object is modeled for the prediction.

Description

Technical Field

[0001] The present disclosure relates to vehicle trajectory control using tree search.

Background Art

[0002] Autonomous vehicles may not be able to drive accurately and / or efficiently when the reference operating conditions are changed, such as when road signs are difficult to see (e.g., due to snow, garbage, sand), deteriorated (e.g., broken lights, worn lane markings), and / or invalidated (e.g., an obstacle partially blocks a lane, traffic signs and / or traffic cones indicate an alternative lane that does not match the original lane marking). Moreover, various environmental factors as well as the behavior of humans and animals may be irregular or unpredictable, which may further complicate autonomous vehicle driving.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Patent Document 6

Patent Document 7

Patent Document 8

[0004] Detailed explanations are provided with reference to the attached diagrams. In the diagrams, the leftmost digit of the reference number identifies the diagram in which that reference number first appears. The same reference number in different diagrams indicates similar or identical items.

[0005] [Figure 1] This figure illustrates an autonomous vehicle and an exemplary scenario in which lane references (whether previously mapped or detected) may not be reliable in determining commands to control the vehicle's movement. [Figure 2] This is a block diagram of an exemplary autonomous vehicle architecture that includes a guidance system for planning unstructured routes. [Figure 3A] An illustrative flowchart of an exemplary process for generating a path for controlling an autonomous vehicle using a tree search that alternately uses agent filters and / or estimated costs that may be generated from sensor data, which may include an occupation map showing static objects, and / or a map of predicted positions and / or velocity probabilities for dynamic objects. [Figure 3B] An illustrative flowchart of an exemplary process for generating a path for controlling an autonomous vehicle using a tree search that alternately uses agent filters and / or estimated costs that may be generated from sensor data, which may include an occupation map showing static objects, and / or a map of predicted positions and / or velocity probabilities for dynamic objects. [Figure 3C]An illustrative flowchart of an exemplary process for generating a path for controlling an autonomous vehicle using a tree search that alternately uses agent filters and / or estimated costs that may be generated from sensor data, which may include an occupation map showing static objects, and / or a map of predicted positions and / or velocity probabilities for dynamic objects. [Figure 4] This figure shows a three-dimensional representation of the set of different candidate trajectories generated for different time steps / positions in the environment associated with the branching of the tree search discussed herein. [Figure 5A] This is a top-down diagram of the various candidate actions that can be generated by a tree search for different action nodes at the same level of the tree search. [Figure 5B] This is a top-down diagram of the various candidate actions that can be generated by a tree search for different action nodes at the same level of the tree search. [Figure 6A] This is a top-down diagram of a single prediction node or two different prediction environment states associated with two different prediction nodes, which can be determined at least partially based on the candidate actions shown in Figure 5A. [Figure 6B] This is a top-down diagram of a single prediction node or two different prediction environment states associated with two different prediction nodes, which can be determined at least partially based on the candidate actions shown in Figure 5A. [Figure 7A] Figure 6A shows a top-down diagram of two different candidate actions generated based on the state associated with the first prediction node. Figure 7A also shows the results of agent filtering, which changes the classification of dynamic objects from reactive objects to inactive objects for use by the prediction and / or simulation components when generating the next level of prediction nodes. [Figure 7B]Figure 6A shows a top-down diagram of two different candidate actions generated based on the state associated with the first prediction node. Figure 7B also shows the results of agent filtering, which changes the classification of dynamic objects from reactive objects to inactive objects for use by the prediction and / or simulation components when generating the next level of prediction nodes. [Figure 8A] This is a top-down diagram of curved roads, lane-based, and offset-based criteria used to probabilistically generate actions for tree searches. [Figure 8B] This graph illustrates sampling techniques for determining actions for tree searches, which can be used to ensure that a sufficient parameter space is sampled to find the appropriate path. [Modes for carrying out the invention]

[0006] As discussed above, due to the occurrence of abnormal behavior and changes in road conditions, it can be difficult to prepare autonomous vehicles for all unforeseen circumstances. These situations may result in the autonomous vehicle stumbling or hesitating, coming to a complete stop where a human driver could have navigated the situation, and / or needing to transmit a request for assistance from a remote operator (or “remote operator”). This application relates to a technology for increasing the number of scenarios in which an autonomous vehicle can travel safely and effectively, for example, without stopping, without stumbling, without the need to request assistance from a remote operator, and / or in particular, by reducing the likelihood of collisions occurring with respect to abnormal situations and also with respect to baseline driving conditions. For example, the technologies discussed herein can reduce the occurrence of autonomous vehicles stopping or staggering in reference situations such as when objects such as traffic cones or vehicles are knocked into the middle of a lane and obstruct part of two lanes, when there is junk on the street, when there are many vehicles and pedestrians at a complex intersection, or when driving in a gravel area without lane markings.

[0007] The technologies discussed herein may include autonomous vehicle guidance systems that generate paths for controlling an autonomous vehicle based at least in part on a tree search technique that alternately determines candidate actions and predicts future states of the environment associated with the autonomous vehicle in response to those candidate actions. In some examples, the tree search technique may include detecting static and dynamic objects in the environment associated with the autonomous vehicle and classifying the dynamic objects as reactive or inactive objects via a machine learning (ML) model. Reactive objects may be more likely (as measured by the reliability output of the ML model) to change their movement in response to the actions of the autonomous vehicle, while the movement of inactive objects may be less likely to change based on the movement of the autonomous vehicle.

[0008] ML models can receive tracks or at least poses (e.g., position and / or orientation) associated with autonomous vehicles, tracks associated with dynamic objects, and / or sensor data associated with dynamic objects, and can be trained to output confidence scores indicating the likelihood that the actions of dynamic objects will respond to / be affected by the actions of autonomous vehicles. Tracks can identify the past, current, and / or predicted posture, velocity, acceleration, and object classification of the detected object (which may or may not be immutable; for example, the classification "vehicle" is likely to be immutable, while "cyclist" may change because a cyclist may dismount and become a pedestrian). In some cases, if the confidence score determined by the ML model meets or exceeds a confidence threshold, the detected object may be classified as a reactive object; otherwise, it may be classified as an inactive object. In some cases, classifying a detected object as a dynamic object may be handled by a different ML model, such as the ML model that determines the tracks associated with the detected object. Nevertheless, more simply, dynamic objects may be objects that are moving, have been moved, and / or are associated with object classifications associated with movable objects. In some examples, another ML model may determine static object maps (e.g., maps showing space occupied by static objects and / or "free space" in the environment) and / or one or more dynamic object maps (e.g., maps showing locations in the environment that may be occupied by moving objects or stationary objects that may move at a future point in time). These maps may be used for the cost determination unit of a tree search performed by a vehicle navigation system.

[0009] The tree search discussed herein can alternately determine candidate actions, predicted environmental states associated with those candidate actions in future time steps (e.g., at least partially responsive), other candidate actions based on those predicted environmental states, and second predicted environmental states associated with those additional candidate actions in further future time steps, up to a planned period or a specified number of actions. Candidate actions may, for example, represent a trajectory for controlling the vehicle's movement, or activating the vehicle's emitters (e.g., turn signals, headlights, speakers). Each candidate action may be associated with a separate action node, and each predicted environmental state may be associated with a prediction node in the tree. As an initial operation, the tree search can determine the current state of the environment associated with the autonomous vehicle, which may include dynamic and / or static objects, at least partially based on sensor data. This initial state may be associated with a root node, which in at least one example may be a prediction node. The root node and / or any other prediction node may identify dynamic objects as reactive or inactive objects as part of the environmental state indicated by the prediction node. In some examples, the state of the environment can be represented by the data structure associated with the root node / prediction node.

[0010] Using this initial state, the tree search may determine one or more candidate actions for exploration. Candidate actions may include coarse actions such as "stay in the same lane", "change lane to the left", "execute a right turn", "stop", etc., and / or fine instructions such as curves that define and / or are associated with the position, steering angle, steering speed, speed, and / or acceleration that the vehicle controller tracks. In some examples, determining one or more candidate actions for exploration may include transmitting the initial environmental state (or the state indicated by a particular prediction node of a branch being explored in a prediction node deeper than the initial node) to the vehicle's planning component and receiving a set of candidate actions from the planning component. The planning component may be the vehicle's nominal planning component that generates a trajectory for controlling the vehicle's movement and / or operation, in contrast to a contingent planning component that controls the vehicle during abnormal or emergency situations, although the contingent planning component may additionally or alternatively provide candidate actions for use by the tree search. and It is considered possible to generate. The tree search component may associate one or more candidate actions from the set received from the planning component with an action node.

[0011] In some examples, each of the candidate actions can be associated with its own action node. For example, a candidate action can be associated with an action node that is subordinate to a root node indicating the state on which the candidate action is based. In one example, each of the candidate actions can be associated with an individual action node, but in additional or alternative examples, a machine learning (ML) model can cluster the candidate actions together and determine a representative candidate action to represent two or more of the candidate actions in the cluster. In such examples, nodes associated with separate candidate actions of the same cluster can be merged into a single action node. The action node can, in some examples, identify the representative candidate action, or in other examples, the action node can indicate the representative and two or more candidate actions associated therewith. In an example where action nodes are merged, the lowest cost candidate action among the two or more candidate actions represented by the action node can be selected for inclusion in subsequent exploration and / or paths.

[0012] Tree search can transmit the candidate actions and the state on which the candidate actions are based (i.e., the initial state in this example) to a simulation and / or prediction system of the vehicle's perception components. The simulation system can be part of a prediction component for generating an estimated future trajectory for an object. The simulation system can determine the predicted state of the environment, at least in part, based on the candidate actions. This can include predicting how inactive and / or reactive dynamic objects will act, at least in part, based on the candidate actions. The predictions generated by the simulation system can be associated with a future point in time and used to generate a prediction node indicating the predicted state of the environment that can result from implementing the candidate actions, subordinate to the action node associated with the candidate actions.

[0013] Tree search may, as an addition or alternative, determine the cost associated with a candidate action. This cost may be based at least in part on predictions of how close the candidate action will bring the autonomous vehicle to static and / or dynamic objects, the autonomous vehicle's proximity to impassable surfaces, the speed, acceleration, and / or jerk associated with the candidate action, and short-term objective costs (e.g., displacement / progress along the route, parking, and soft objectives indicated by remote control devices, i.e., achieving objectives that the autonomous vehicle is not required to meet but are included in the cost determination to increase the likelihood that the autonomous vehicle will operate in a particular manner). In at least one example, a cost may be determined for each action node, and in relation to the action node, a total cost may be determined that includes the sum of the nodes that constitute the branch of the data structure containing that action node (e.g., the cost of the action node, and, tracing back to the root node, the costs of all parent nodes to which the action node is subordinate).

[0014] In at least one example, determining the cost can be based on cost-to-go, which may be a prediction of how much it would cost to move an autonomous vehicle in a particular manner (e.g., steering speed, steering angle, velocity, acceleration) and / or to a particular location. For example, cost-to-go may be an estimate that does not require running simulations to determine a more accurate prediction of cost-to-go, thereby increasing the number of candidate actions that can be explored and / or reducing the computational resources consumed by tree search. In some examples, the techniques discussed herein are for cost-to-go lower limit Estimated value (lower border Using estimates and cost-to-go upper limit Estimated value (upper border This may include alternating between using (estimate) and / or alternating according to a ratio. For example, the first action node of the first layer is lower limitEstimates can be used, and the second action node of the second layer, which is deeper than the first layer, upper limit Estimates can be used. The ratio depends on how many layers there are. upper limit Use estimates, or how many layers are there? upper limit Use estimates, or for example, with respect to two layers. upper limit Before switching to using estimates, the two layers lower limit Using estimates, for one layer upper limit Before switching to using estimates, the two layers lower limit It may be possible to set up the use of estimated values, etc. In at least one example, lower limit The estimated value could be zero. upper limit The estimate may be the cost associated with default actions such as maintaining the vehicle's steering angle and / or speed, or driving straight at a specified speed. In some examples, the default action may be determined at least partially based on the vehicle's track, perceptual data, etc.

[0015] The technology may, as an addition or alternative, include changing the status of a dynamic object from a reactive object to an inactive object, or vice versa, based at least partially on the latest predicted state of the environment. For example, the predicted track of an object associated with a predicted state, and / or candidate actions for the next layer, may be provided as input to a machine learning model of an agent filter discussed herein, which may classify the dynamic object as either a reactive or inactive object, which may differ from how the object was previously classified. In other words, the predicted action of a dynamic object may cause the dynamic object to be far enough away from an autonomous vehicle and / or further candidate actions associated with the autonomous vehicle, thereby making it less likely that the dynamic object will change its behavior based on the autonomous vehicle's actions. Conversely, a dynamic object that has previously been too far away, or directed in a particular direction, or has experienced environmental conditions that have prevented interaction with the autonomous vehicle (e.g., a red light rather than a turn when the dynamic object is stopped), may not have been affected by the actions of the autonomous vehicle, but may now or in the future be likely to be affected. In the first example, determining the classification of a dynamic object may be done in conjunction with determining the root node of the data structure, and this classification may persist throughout each of the layers of the data structure. However, in another example, the classification may be updated in each of the layers of prediction nodes and / or every n layers, where n is a positive integer.

[0016] The techniques discussed herein involve agent filtering (e.g., showing dynamic objects as inactive or reactive objects) by alternating between cost estimation and using simulations to determine more accurate costs, and / or instead of more intensive techniques for cost determination such as simulations. upper limit Estimated cost (upper border The computational load for determining a path to control an autonomous vehicle can be reduced, for example, by using default control instructions to estimate the estimated cost. The technique can increase the number of candidate actions that can be explored and, therefore, the number of paths that can be explored when deciding how to control the autonomous vehicle, in which case the paths may be a series of candidate actions forming a continuous series. Increasing the number of candidate actions and / or paths to be explored can increase the granularity at which the autonomous vehicle can be controlled and increase the number of scenarios in which the autonomous vehicle can travel successfully and safely.

[0017] Exemplary Scenario Figure 1 shows an exemplary scenario 100, which includes vehicle 102. In some examples, exemplary scenario 100 may be a real-world scenario, and / or exemplary scenario 100 may be a representation of a real-world scenario modeled as a simulated scenario. In examples where exemplary scenario 100 is a simulated scenario, exemplary scenario 100 may be determined at least in part on inputs received at the user interface of a computing device (for example, the user of the computing device may define an environment, objects within it, and / or their properties), and / or exemplary scenario 100 may be determined at least in part on log data received from one or more autonomous vehicles. The log data may be determined at least in part on sensor data received by the autonomous vehicles, perceptual data generated by perceptual components, and / or commands generated by planning components. In some examples, the autonomous vehicles may store the log data and / or periodically transmit the log data to a remote computing device.

[0018] In some cases, vehicle 102 may be an autonomous vehicle configured to operate according to the Level 5 classification issued by the U.S. National Highway Traffic Safety Administration, which describes a vehicle capable of performing all safety-critical functions throughout its entire journey without any expectation of the driver (or occupants) taking control of the vehicle at any point. However, in other examples, vehicle 102 may be a fully or partially autonomous vehicle with any other level or classification. The technologies discussed herein are thought to apply beyond robotic control, such as for autonomous vehicles. For example, the technologies discussed herein may be applied to mining, manufacturing, augmented reality, etc. Furthermore, although vehicle 102 is shown as a land vehicle, vehicle 102 could be a spacecraft, ship, etc. In some examples, vehicle 102 may be represented in a simulation as a simulated vehicle. For simplicity, the discussion herein does not distinguish between simulated vehicles and real-world vehicles. Therefore, references to “vehicle” may refer to simulated and / or real-world vehicles.

[0019] According to the technologies discussed herein and the example in which Scenario 100 is a real-world example, a vehicle 102 may receive sensor data from sensors 104 on the vehicle 102. For example, sensors 104 may include position sensors (e.g., Global Positioning System (GPS) sensors), inertial sensors (e.g., accelerometer sensors, gyroscope sensors, etc.), magnetic field sensors (e.g., compass), position / velocity / acceleration sensors (e.g., speedometer, drive system sensors), depth position sensors (e.g., LIDAR sensors, RADAR sensors, sonar sensors, time-of-flight (ToF) cameras, depth cameras, and / or other depth-sensing sensors), image sensors (e.g., cameras), audio sensors (e.g., microphones), and / or environmental sensors (e.g., barometers, hygrometers, etc.). In some examples, a simulated sensor may correspond to at least one of the sensors 104 on the vehicle 102, and in the simulation, one or more of the sensors 104 may be simulated. In some examples, the simulated sensor position may correspond to the relative position of one of the sensors 104 with respect to the vehicle 102.

[0020] Sensor 104 may generate sensor data, which may be received by a computing device 106 associated with the vehicle 102. However, in other examples, some or all of the sensors 104 and / or computing devices 106 may be separate from the vehicle 102 and / or located remotely from the vehicle 102, and data acquisition, processing, commands, and / or control may be communicated to and from the vehicle 102 by one or more remote computing devices via wired and / or wireless networks.

[0021] The computing device 106 may include memory 108 for storing a perception component 110, a planning component 112, a guidance system 114, and / or a controller 116. In some examples, the planning component 112 may include the guidance system 114. The perception component 110, the planning component 112, the guidance system 114, and / or the controller 116 may include one or more machine learning (ML) models and / or other computer executable instructions. Generally, the perception component 110 may determine what is in the environment surrounding the vehicle 102, and the planning component 112 may determine how to operate the vehicle 102 according to the information received from the perception component 110. For example, the planning component 112 may determine the trajectory 118 based at least in part on perceptual data and / or other information, such as one or more maps, location information (for example, where the vehicle 102 is in the environment relative to features detected by the maps and / or the perceptual component 110), and / or a route generated by the guidance system 114. The trajectory 118 may be one of the candidate actions determined by the guidance system 114.

[0022] The track 118 may include commands for the autonomous vehicle 102's controller 116 to actuate the vehicle 102's drive components to achieve a steering angle and / or steering speed that results in a vehicle position, vehicle speed, and / or vehicle acceleration following a path generated by the guidance system. For example, the track 118 may include a target direction, target steering angle, target steering speed, target position, target speed, and / or target acceleration for the controller to follow as part of the path. For example, a rough path generated by the guidance system 114 according to the techniques discussed herein may indicate the vehicle's position, direction, speed, and / or entry / exit curvature at a time interval of 500 milliseconds, and a smooth path output by the guidance system 114 may include such points at intervals of 10 or 100 milliseconds, which may correspond to the time interval associated with the track 118. In some examples, the controller may include software and / or hardware for actinguate the vehicle 102's drive components sufficiently to follow the track 118 (and / or a path that may include a number of tracks in one example).

[0023] In an exemplary scenario 100, the autonomous vehicle 102 receives and / or determines a route 120 that defines a starting position 122, an ending position 124, and a curve between the starting position 122 and the ending position 124 (note that the curve may include straight lines and / or one or more curves). For example, the planning component 112 may determine the route 120 based at least in part on sensor data and the ending position received as part of a mission (e.g., from the occupant, from the command center). As used herein, references to “position” may include both position and / or attitude (e.g., the position and / or orientation of the vehicle). In some examples, the route may not include the ending position 124 and may include, additionally or alternatively, target positions such as a target lane, a target relative position (e.g., 10 feet from the edge of the road), or a target object (e.g., following the vehicle, following the occupant, moving towards the individual calling the vehicle).

[0024] As the vehicle operates to reach the destination position 124, the autonomous vehicle 102 may encounter a scenario like exemplary scenario 100 in which a planner that relies on lane criteria (e.g., relative spatial designation determined at least partially on a map and / or locating the autonomous vehicle 102) to generate a route may not be able to generate a route accurately and / or efficiently. For example, various objects (e.g., an obstructing vehicle 126, a toolbox 128, and a fallen traffic cone 130) cumulatively block all three lanes of the indicated road, which may cause another planner to stop the vehicle and / or call for remote control because none of the lanes has enough room for the autonomous vehicle.

[0025] However, the guidance system 114 discussed herein may generate a path 132 based at least in part on environmental data 134 generated from sensor data acquired by sensor 104. For example, a perceptual component 110 may generate all or part of the environmental data 134, which may include static and / or dynamic data. For example, static data may indicate the possibility that an object exists at a location in the environment, and dynamic data may indicate that an object occupies or will occupy a location in the environment. In some cases, dynamic data may include a number of frames associated with separate time steps at intervals up to a prediction range (i.e., the maximum time / distance for which the dynamic data is predicted). In some examples, the guidance system 114 may be always operational, i.e., the guidance system may be a nominal planning component, or in alternative examples, the guidance system 114 may be an incidental planning component, or a planning component for special circumstances (e.g., when the nominal planning component is not capable of finding a valid path).

[0026] The techniques discussed herein may, in addition or as an alternative, include determining dynamic data based at least in part on a mode of vehicle operation. For example, the dynamic data may include a first prediction associated with a first vehicle mode and a first time, and a second prediction associated with a second vehicle mode and a first time. Vehicle modes may include mission-level modes such as picking up passengers, transporting passengers, dropping off passengers, etc.; route or trajectory-level modes such as maintaining a trajectory, slowing down to a stop, changing lanes, performing a right turn, etc.; and / or signaling modes such as activating speakers, activating turn signals, turning on headlights or high beams, etc. The actions and signals of the autonomous vehicle may influence decisions and actions made by other entities near the autonomous vehicle 102, thereby influencing the predicted movement of other objects.

[0027] In some examples, the perceptual component 110 may receive sensor data from the sensor 104 and determine data related to objects near the vehicle 102, such as static and / or dynamic data. The shown example of environmental data 134, including static and / or dynamic data, is a top-down representation of such data, but any representation of static and / or dynamic data is possible, such as heatmaps, object classifications associated with detected objects, instance segmentation, semantic segmentation, two-dimensional and / or three-dimensional bounding boxes, tracks, etc. The perceptual component 110 may also determine route data specifying the vehicle's destination, global map data identifying road characteristics (e.g., features detectable by various sensor modalities that help locate the autonomous vehicle), local map data identifying characteristics detected near the vehicle (e.g., the location and / or dimensions of buildings, trees, fences, fire hydrants, stop signs, and any other features detectable by various sensor modalities), track data (e.g., environmental representations, object detection, and / or tracks discussed herein), etc.

[0028] In some examples, the perceptual component 110 may receive sensor data from the sensor 104 and determine data related to objects near the vehicle 102, such as static and / or dynamic data. The shown example of environmental data 134, including static and / or dynamic data, is a top-down representation of such data, but any representation of static and / or dynamic data is possible, such as heatmaps, object classifications associated with detected objects, instance segmentation, semantic segmentation, two-dimensional and / or three-dimensional bounding boxes, tracks, etc. The perceptual component 110 may also determine route data specifying the vehicle's destination, global map data identifying road characteristics (e.g., features detectable by various sensor modalities that help locate the autonomous vehicle), local map data identifying characteristics detected near the vehicle (e.g., the location and / or dimensions of buildings, trees, fences, fire hydrants, stop signs, and any other features detectable by various sensor modalities), track data (e.g., environmental representations, object detection, and / or tracks discussed herein), etc.

[0029] In some examples, the perception component 110 may include a hardware and / or software pipeline, which may include one or more GPUs, ML models, Kalman filters, etc. In some examples, the perception component 110 may monitor as much of the environment surrounding the autonomous vehicle as possible, but this may be limited by the capabilities of the sensors, object and / or environmental occlusion (e.g., buildings, elevation changes, objects in front of other objects), and / or environmental influences such as fog, snow, etc. For example, the sensor data may include RADAR data, which the perception component 110 may receive as input. The perception component 110 may be configured to detect as much information as possible about objects and the environment in order to avoid failing to consider events or object behaviors that should be taken into account by the planning component 112 when determining the trajectory 118.

[0030] Object classification may be part of the environmental data 134 and may be determined by the perceptual component 110, which may distinguish between various object types, such as passenger cars, pedestrians, cyclists, delivery trucks, semi-trucks, traffic signs, etc. The truck may include past, current, and / or predicted object positions, velocities, accelerations, and / or directions. In some examples, the perceptual component 110 may include a prediction component, which may determine a predicted portion of the truck, which may include predicted positions, directions, steering speeds, velocities, accelerations, classifications (with respect to susceptible classifications, such as a cyclist that could become a pedestrian), etc. The prediction component may include simulation components, machine learning models, or, in additional or alternative examples, the prediction component may include a Kalman filter, etc. The perceptual component 110 may be used to determine environmental states indicated by prediction nodes as part of a tree search discussed herein. For example, environmental states may include current and / or predicted environmental data 134.

[0031] The data generated by the perceptual component 110 can be collectively referred to as perceptual data. As soon as / when the perceptual component 110 generates perceptual data, it may provide that perceptual data to the planning component 112 and / or the guidance system 114. In some examples, the perceptual data may include the outputs of sensor-specific pipelines (e.g., vision, LIDAR, RADAR) and / or hybrid sensor pipelines (e.g., vision LIDAR, RADAR-LIDAR).

[0032] The planning component 112 may determine one or more trajectories, control the movement of the vehicle 102 to move around along a path or route, and / or control the vehicle 102's behavior in other ways, using perceptual data received from the perception component 110 and / or paths received from the guidance system 114, any such behavior may be performed by various other components (for example, localization may be performed by a localization component, which may be based at least in part on perceptual data). For example, the planning component 112 determines a route for the vehicle 102 from a first position to a second position, determines a smoother trajectory from a rough trajectory received from the guidance system 114, and substantially simultaneously generates multiple potential trajectories for controlling the movement of the vehicle 102 according to a receding horizon technique (e.g., 1 microsecond, 0.5 seconds, 2 seconds, 5 seconds, 10 seconds, or any other short time period) to control the vehicle to follow the route (e.g., to avoid any of the detected objects), and may select one of the potential trajectories as the trajectory 118 of the vehicle 102, which can be used to generate drive control signals that can be transmitted to the controller 116 to actuate the drive components of the vehicle 102. Figure 1 shows an example of such a trajectory 118, which is represented as arrows indicating a target direction, target steering angle, target steering speed, target position, target speed, and / or target acceleration for the controller to follow, but the trajectory itself may contain commands to the controller, which may sequentially activate the drive system of the vehicle 102.

[0033] In some examples, the guidance system 114 may be a secondary guidance system to the nominal guidance system of the planning component 112. In some examples, the planning component 112 may generate one or more candidate actions for the tree search discussed herein, associated with action nodes. The planning component 112 may receive a track or route from the nominal guidance system and may receive a route generated by the guidance system 114 as discussed herein. The route received from the guidance system 114 may be treated as an incidental route by the planning component 112. For example, the planning component 112 may rely on the output of the nominal guidance system unless the output of the nominal guidance system would cause the vehicle 102 to stop, or cause the vehicle 102 to invoke remote control, or fail to output a feasible, collision-free, and / or economical route or track, or until such situations occur. In such cases, the planning component 112 may switch to using the route output by the guidance system 114. In additional or alternative examples, the planning component 112 may use the paths output by the guidance system 114 to generate trajectories that are at least partially based on sensor data and / or perception data, for all time periods, in a specific area (e.g., in an area without lane references). In some examples, the guidance system 114 may perform a tree search as discussed herein, which may include managing the generation of action nodes (e.g., by sending requests to an application programming interface (API) associated with the planning component 112 and receiving candidate actions in response), and / or managing the generation of prediction nodes (e.g., by sending requests to an API associated with the prediction component of the perception component 110 and receiving current and / or prediction environmental state data).

[0034] The guidance system 114 may include one or more GPUs, or may be communicably coupled to one or more GPUs (for example, via a publish / subscribe messaging system or via a data bus), and the techniques discussed herein may be parallelized and disseminated to the threads of those GPUs, although the techniques discussed herein may include at least a serial portion.

[0035] Exemplary System Figure 2 shows a block diagram of an exemplary system 200 implementing the technology discussed herein. In some cases, the exemplary system 200 may include a vehicle 202, which may correspond to vehicle 102 in Figure 1. In some cases, vehicle 202 may be an autonomous vehicle configured to operate according to the Level 5 classification issued by the U.S. National Highway Traffic Safety Administration, which describes a vehicle capable of performing all safety-critical functions throughout the entire journey without any expectation of the driver (or occupants) taking control of the vehicle at any point. However, in other examples, vehicle 202 may be a fully or partially autonomous vehicle having any other level or classification. Furthermore, in some cases, the technology described herein may also be usable by non-autonomous vehicles.

[0036] The vehicle 202 may include a vehicle computing device 204, a sensor 206, an emitter 208, a network interface 210, and / or a drive component 212. The vehicle computing device 204 may correspond to the computing device 106, and the sensor 206 may correspond to the sensor 104. The system 200 may include the computing device 214 as an addition or replacement.

[0037] In some cases, sensor 206 may correspond to sensor 104 and may include LIDAR sensors, RADAR sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., Global Positioning System (GPS), compass, etc.), inertial sensors (e.g., inertial measuring unit (IMU), accelerometer, magnetometer, gyroscope, etc.), image sensors (e.g., red-green-blue (RGB), infrared (IR), intensity, depth, time-of-flight cameras, etc.), microphones, wheel encoders, environmental sensors (e.g., thermometers, hygrometers, light sensors, pressure sensors, etc.). Sensor 206 may include numerous instances of each of these or other types of sensors. For example, the RADAR sensor may include individual RADAR sensors located on the corners, front, rear, sides, and / or top of vehicle 202. As another example, the camera may include numerous cameras located at various positions around the exterior and / or interior of vehicle 202. Sensor 206 may provide input to vehicle computing device 204 and / or computing device 214. The positions associated with the simulated sensors discussed herein may correspond to the position and / or origin (e.g., focal point) of the sensor's field of view relative to the vehicle 202, and / or the direction of movement of the vehicle 202.

[0038] Vehicle 202 may also include emitters 208 for emitting light and / or sound, as described above. In this example, emitters 208 may include internal audio and visual emitters for communicating with occupants of vehicle 202. Internal emitters may include, but are not limited to, speakers, lights, signs, display screens, touchscreens, haptic emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, headrest positioners, etc.). In this example, emitters 208 may also include external emitters. External emitters may include, but are not limited to, lights or other indicators of vehicle action for indicating the direction of movement (e.g., indicator lights, signs, light arrays, etc.), and one or more audio emitters for audible communication with pedestrians or other nearby vehicles (e.g., speakers, speaker arrays, horns, etc.), one or more of which audio emitters include acoustic beam steering technology.

[0039] The vehicle 202 may also include a network interface 210 that enables communication between the vehicle 202 and one or more other local or remote computing devices. For example, the network interface 210 may facilitate communication with other local computing devices and / or drive components 212 on the vehicle 202. Additionally, or alternatively, the network interface 210 may enable the vehicle to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic signals, etc.). Additionally, or alternatively, the network interface 210 may enable the vehicle 202 to communicate with a computing device 214. In some examples, the computing device 214 may include one or more nodes of a distributed computing system (e.g., a cloud computing architecture).

[0040] The network interface 210 may include physical and / or logical interfaces for connecting the vehicle computing device 204 to another computing device or a network such as network 216. For example, the network interface 210 may enable Wi-Fi-based communication, such as via frequencies defined by the IEEE 200.11 standard, short-range radio frequencies such as Bluetooth®, cellular communication (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communication protocol that enables each computing device to interface with other computing devices. In some cases, the vehicle computing device 204 and / or sensor 206 may transmit sensor data to the computing device 214 via network 216 at a specific frequency, after a predetermined period of time, or in near real-time.

[0041] In some cases, the vehicle 202 may include one or more drive components 212. In some cases, the vehicle 202 may have a single drive component 212. In some cases, the drive component 212 may include one or more sensors for detecting the conditions around the drive component 212 and / or the vehicle 202. Examples, but not limited to, include, one or more wheel encoders (e.g., rotary encoders) for sensing the rotation of the drive component's wheels, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) for measuring the orientation and acceleration of the drive component, cameras or other image sensors, ultrasonic sensors, LIDAR sensors, RADAR sensors, etc. for acoustically detecting objects around the drive component. Some sensors, such as wheel encoders, may be specific to the drive component 212. In some cases, the sensors on the drive component 212 may overlap with or complement the corresponding systems of the vehicle 202 (e.g., sensor 206).

[0042] The drive component 212 may include many of the vehicle systems, including a high-voltage battery, a motor for propelling the vehicle, an inverter for converting DC current from the battery into AC current for use by other vehicle systems, a steering system including a steering motor and steering rack (which may be electric), a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stabilization control system for distributing braking force to reduce traction loss and maintain control, an HVAC system, lighting (e.g., headlights / taillights for illuminating the exterior of the vehicle), and one or more other systems (e.g., a cooling system, safety systems, an on-board charging system, and other electrical components, such as a DC / DC converter, a high-voltage junction, high-voltage cables, a charging system, a charging port, etc.). In addition, the drive component 212 may include a drive component controller, which can receive and preprocess data from sensors to control the operation of various vehicle systems. In some cases, the drive component controller may include one or more processors and memory communicably coupled to one or more processors. The memory may store one or more components for performing various functionalities of the drive component 212. Furthermore, the drive component 212 may also include one or more communication connections that enable each drive component to communicate with one or more other local or remote computing devices.

[0043] The vehicle computing device 204 may include a processor 218 and a memory 220 communicably coupled to one or more processors 218. The memory 220 may correspond to memory 108. The computing device 214 may also include a processor 222 and / or memory 224. The processors 218 and / or 222 may be any suitable processor capable of processing data and executing instructions to perform operations as described herein. For example, but not limited to, the processors 218 and / or 222 may include one or more central processing units (CPUs), graphics processing units (GPUs), integrated circuits (e.g., application-specific integrated circuits (ASICs)), gate arrays (e.g., field-programmable gate arrays (FPGAs)), and / or any other devices or parts of devices that process electronic data and convert that electronic data into other electronic data that can be stored in registers and / or memories.

[0044] Memory 220 and / or 224 may be examples of non-temporary computer-readable media. Memory 220 and / or 224 may store operating systems and one or more software applications, instructions, programs, and / or data for implementing the methods and functions attributed to various systems described herein. In various implementation embodiments, memory may be implemented using any suitable memory technology, such as static random-access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein may include many other logical, programmatic, and physical components, of which those shown in the accompanying figures are only examples relevant to the discussion herein.

[0045] In some cases, memory 220 and / or memory 224 may store a localization component 226, a perception component 228, a planning component 230, a guidance component 232, a simulation component 234, an agent filter 236, and / or a system controller 238, where zero or more of any of these may be hardware such as a GPU, CPU, and / or other processing units. The perception component 228 may correspond to the perception component 110, the planning component 230 may correspond to the planning component 112, and the guidance component 232 may correspond to the guidance system 114.

[0046] In at least one example, the localization component 226 may include hardware and / or software for receiving data from sensors 206 to determine the position, velocity, and / or orientation of the vehicle 202 (e.g., one or more of x, y, z position, roll, pitch, or yaw). For example, the localization component 226 may include and / or request / receive a map of the environment and be able to continuously determine the position, velocity, and / or orientation of the autonomous vehicle within that map. In some cases, the localization component 226 may receive image data, LIDAR data, RADAR data, IMU data, GPS data, wheel encoder data, etc., using SLAM (simultaneous localization and mapping), CLAMS (calibration, localization, and mapping, simultaneously), relative SLAM, bundle adjustment, nonlinear least-squares optimization, etc., to accurately determine the position, attitude, and / or velocity of the autonomous vehicle. In some cases, the positioning component 226 may provide data to various components of the vehicle 202 to determine the initial position of the autonomous vehicle for the purpose of generating a trajectory and / or map data. In some examples, the positioning component 226 may provide the perception component 228 with the position and / or orientation of the vehicle 202 relative to the environment and / or associated sensor data.

[0047] In some cases, the perceptual component 228 may include a primary perceptual system and / or predictive system implemented in hardware and / or software. For example, the predictive component may be a simulation component 234, although in additional or alternative examples, the predictive component and the simulation component 234 may be separate components. Nevertheless, the perceptual component 228 (and its predictive and / or simulation component 234) may work with the guidance component 232 to determine the environmental conditions discussed herein. The perceptual component 228 may detect objects in the environment surrounding the vehicle 202 (e.g., identify that an object is present), classify objects (e.g., determine the object type associated with a detected object), segment sensor data and / or other representations of the environment (e.g., identify portions of sensor data and / or representations of the environment as associated with a detected object and / or object type), and determine characteristics associated with an object (e.g., tracks that identify the current, predicted, and / or previous position, orientation, velocity, and / or acceleration associated with the object). The perceptual component 228 may include a predictive component that predicts the actions / states of dynamic components of the environment, such as moving objects. In some examples, the perceptual component 228 may include a simulation component 234 as part of the predictive portion of the perceptual component 228. The data determined by the perceptual component 228 is called perceptual data. Although the agent filter 236 is shown as part of the simulation component 234, in additional or alternative examples, the perceptual component 228 may include the agent filter 236.

[0048] The planning component 230 may receive the position and / or orientation of the vehicle 202 from the positioning component 226 and / or perceptual data from the perception component 228, and may determine commands to control the vehicle 202's operation based at least partially on any of this data. In some examples, determining a command may involve determining the command at least partially on a format associated with the system to which the command is associated (for example, a first command to control the movement of an autonomous vehicle may be formatted in a first format of messages and / or signals that the drive component 212 can parse / execute (e.g., analog, digital, pneumatic, kinematic, such as those generated by the system controller of the drive component 212), and a second command relating to the emitter 208 may be formatted according to a second format associated with it). In some examples, if the planning component 230 may include hardware / software in-alloop in a simulation (e.g., for testing and / or training the planning component 230), the planning component 230 may generate commands that can be used to control a simulated vehicle. These instructions may be used, either additionally or as an alternative, to control the movement of the real-world version of vehicle 202, for example, when running a simulation on the vehicle while vehicle 202 is in operation.

[0049] The planning component 230 may be the primary component for determining control commands relating to the vehicle 202, such as during the operation of the vehicle 202 under nominal conditions, but the planning component 230 may further include a guidance component 232, and / or the vehicle 202 may include the guidance component 232 as an additional component separate from the planning component 230. The guidance component 232 may determine trajectories and / or paths to control the vehicle concurrently with the planning component 230, such as when the trajectory determined by the planning component 230 fails to generate (for example, when the planning component 230 is unable to determine a suitable trajectory to avoid an object), and / or to determine incidental trajectories and / or paths to control the vehicle 202 in the event of a violation of comfort metrics such as threshold acceleration and / or jerk, or road rules. The guidance component 232 may perform a tree search as discussed herein and may manage to determine action nodes and / or prediction nodes of the tree search by transmitting a request asking the planning component to generate candidate actions based at least in part on the environment determined in relation to the prediction node. The guidance component 232 may receive the initial state of the environment from the perception component 228 (i.e., in relation to the root node of the tree search), transmit this initial environmental state to the planning component 230, and receive one or more candidate actions from the planning component 230. The guidance component 232 may transmit at least one of these one or more candidate actions to the simulation component 234 and / or the prediction component of the perception component 228, and the simulation component 234 and / or the prediction component of the perception component 228 may determine a predicted state of the environment based at least in part on the candidate action. This process may be repeated until a planned period, distance, progress, target location, and / or a suitable path along the route are reached / determined.

[0050] For example, the planning period could be the length of time from the current time to the future (e.g., 500 milliseconds, 1 second, 2 seconds, 5 seconds, 8 seconds, 10 seconds). This length of time could be associated with controlling the vehicle over the next m units of time, where m is a positive integer. Distance could define the total distance covered by the constituent actions that make up the path, while progress along the route could be the displacement along / relative to the route. In additional or alternative examples, a target location may be used to terminate a tree search. For example, once a path to reach a target location in the environment is determined, the tree search may terminate by outputting that path. In additional or alternative examples where a guidance system is used when the nominal planning component fails to create a valid trajectory or path, the guidance system may terminate once it has determined a valid path (e.g., a path that is collision-free and conforms to a rule set, which may specify a comfort metric, compliance with laws, etc.). In additional examples, iterations may continue until a goal is achieved (e.g., a successful lane change, a successful merge, or any other completed action). In any one or more examples, any combination of the above may be further used as decision points for branching the tree.

[0051] The simulation component 234 may operate on the vehicle 202 and / or on the computing device 214. The simulation component 234 may simulate the environment and / or the vehicle 202, such as simulating the vehicle 202 performing candidate actions, and may determine the predicted state of the environment in response to the vehicle 202 performing candidate actions, which is at least partially based on the passage of time. For example, the simulation may include representations of the position, orientation, movement, and / or quality of parts of the environment and / or the vehicle 202. The environment may include agents such as other vehicles, pedestrians, vegetation, buildings, signs, etc.

[0052] The simulation component 234 may receive candidate actions and environmental states (which may be the current environmental state determined by the perception component 228, or a predicted environmental state determined by the prediction component of the perception component 228, or by the simulation component 234) from the guidance component 232 to determine the simulation data, which may be a two-dimensional or three-dimensional representation of the scenario. The simulation data may be used to instantiate and run the simulation. Candidate actions may be used to control the simulated movement of the vehicle 202 during the execution of the simulation. The three-dimensional representation may include position, orientation, geometric data (e.g., polygon representation, digital wire mesh representation), and / or motion data associated with one or more objects in the environment, as well as / or material, lighting, and / or lighting data, although in other examples this data may be omitted. In additional or alternative examples, the simulation component 234 may include computational structures (e.g., algorithmic and / or mathematical representations used by a computing device when performing a described operation that is not intended to be visualized (and / or impossible to visualize)).

[0053] Scenario data may include a two-dimensional representation of the environment associated with the scenario, the objects contained within it, and the properties associated with them, all of which may be part of the scenario associated with the log data. For example, scenario data may identify the location of an object, the area occupied by the object, the velocity and / or acceleration associated with the object, whether the object is static or dynamic, the object type associated with the object (e.g., classifications such as "pedestrian," "cyclist," "vehicle," "large vehicle," "traffic light," "traffic sign," "building," "road," "crosswalk," "sidewalk"), and / or other kinematic properties associated with the object and / or object type (e.g., coefficient of friction, elasticity, malleability). Regarding the environment itself, scenario data may identify the topology of the environment, the weather conditions associated with the environment, the lighting conditions (e.g., sunny, cloudy, night), the location of light sources, etc. In some examples, the topology associated with the scenario data, the location and dimensions of fixed objects (e.g., buildings, trees, signs), etc., may be generated based at least partially on a map. In some examples, scenario data may be used to instantiate a three-dimensional representation of an object (for example, by simulation component 234), and / or the simulated environment may be instantiated at least partially based on map data (for example, which may define the topology of the environment, the locations and / or dimensions of fixed objects such as signs, plants, and / or buildings) and / or scenario data.

[0054] In addition, or alternatively, the simulation may include, in addition to or instead of, the prediction component of the nominal prediction component or perception component 110 of the simulation component, a simulated object controlled by an agent behavior model, which is incorporated herein by reference in further detail in Patent Document 1 filed June 1, 2020. The agent behavior model may control the simulated motion of a simulated representation of a dynamic object, such as a reactive dynamic object. In some examples, the simulation may be performed as part of an expectation / prediction operation, and therefore one or more simulations may be performed to determine an expected scenario (e.g., predicted environmental state data) based on candidate actions generated according to a tree search as discussed herein.

[0055] In some examples, simulated sensors may determine simulated sensor data based at least in part on a simulation performed by simulation component 234. For example, Patent Document 2, filed September 24, 2019, which is incorporated herein by reference in its entirety, discusses this in more detail. In additional or alternative examples, the simulation itself performed by the simulation component may include simulated sensor data. Perception component 228 (for example, a copy thereof may include software and / or hardware, which may include a hardware-in-the-loop simulation) may receive such sensor data and / or output perception data provided as input to planning component 230 with the simulated sensor data. The planning component may use the perception data to determine commands for controlling the movement of vehicle 202, which may be used to control at least a simulated representation of vehicle 202 in the simulation, and in some examples, such as in an example where simulation component 234 is performed on the vehicle during real-world operation, it may be used to control the real-world movement of vehicle 202.

[0056] In some examples, the simulation component 234 may, additionally or as an alternative, store a set of rules and, at least in part, determine whether a candidate action passed or failed a scenario. The set of rules may be associated with and / or contain behavioral constraints. In some examples, the set of rules may be part of, or supersede, an event detection system (the entirety of which is incorporated herein by reference in Patent Document 3 filed November 13, 2019) and / or a collision monitoring system (Patent Document 4 filed December 4, 2019).

[0057] The simulation component 234 may include an agent filter 236, which may include a machine learning model trained to classify dynamic objects detected by the perception component 228 as either reactive or inactive. In at least one example, the classification does not have to be binary and could be a score such as a number between 0 and 1, where 0 indicates the object is inactive and 1 indicates the object is reactive. In some examples, the agent filter 236 may receive as input the track associated with the dynamic object and candidate actions for the vehicle, and may determine whether the dynamic object is inactive or reactive according to binary classification or regression as described above. A purely inactive agent will not change its behavior at all based on the actions / candidate actions of the vehicle 202, while actions taken by reactive objects are very likely to be influenced by the actions of the vehicle 202. In additional or alternative examples, the agent filter 236 may determine (e.g., covariance, posterior probability) the likelihood that a dynamic object will modify the action associated with it based on the actions of the vehicle 202. A dynamic object associated with the possibility of meeting or exceeding a threshold may be classified as reactive, while another dynamic object associated with the possibility of not meeting the threshold may be classified as inactive.

[0058] In some examples, training agent filter 236 may involve receiving log data from a vehicle, which may include perceptual data and trajectories implemented by the vehicle. Perceptual data may identify data related to dynamic objects, such as dynamic objects and / or tracks associated with dynamic objects. To identify labels indicating that a dynamic object was reactive to a vehicle action, or to generate a score indicating such a reaction, a computing device may determine a correlation score based on changes over time in the track of the dynamic object relative to the trajectory implemented by the vehicle. In some examples, filters may be applied to dynamic objects in the log data to limit dynamic objects that are less likely to modify their own behavior based on the vehicle's actions. For example, the filter may be based on (the speed of the dynamic object, in some cases , motion Based on the direction of the target object, etc. ) Dynamic objects located at a distance greater than a threshold distance from the vehicle can be removed.

[0059] The predicted state of the environment may be determined by the prediction component and / or simulation component 234 of the perception component 228, at least in part, based on whether any dynamic objects in the environment are classified as reactive. The predicted state of reactive objects may be simulated by the simulation component 234, while the predicted state of inert objects may be determined by the prediction component of the perception component 228, such as a Kalman filter for such predictions, or other simple propagation of the current state of inert objects (e.g., dead reckoning). The inert object prediction component may use less computation, while the dynamic object prediction component (e.g., simulation component 234) may, in some cases, require more computation.

[0060] Memory 220 and / or 224 may, as an addition or alternative, store a mapping system, a planning system, a passenger management system, etc. Although the perception component 228, the planning component 230, and / or the simulation component 234 are shown as being stored in memory 220 and / or 224, the perception component 228, the planning component 230, the guidance component 232, the simulation component 234, and / or the agent filter 236 may include processor-executable instructions, machine learning models (e.g., neural networks), and / or hardware.

[0061] In this specification, the localization component 226, the perception component 228, the planning component 230, the guidance component 232, the simulation component 234, the agent filter 236, and / or other components of System 200 may include one or more ML models. For example, the localization component 226, the perception component 228, the planning component 230, the guidance component 232, the simulation component 234, and / or the agent filter 236 may each include separate ML model pipelines. In some examples, the ML models may include neural networks. An exemplary neural network is a biologically inspired algorithm that passes input data through a series of connected layers to produce an output. Each layer in the neural network may include another neural network, or it may include any number of layers (whether convolutional or not). As can be understood in the context of this disclosure, a neural network may utilize machine learning, and machine learning may refer to a broad class of such algorithms in which an output is produced based on learned parameters.

[0062] Although discussed in the context of neural networks, any type of machine learning can be used in accordance with this disclosure. For example, machine learning algorithms include regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression spline (MARS), local estimation scatter plot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, least absolute contraction and selection operator (LASSO), elastic networks, least angular regression (LARS)), and decision tree algorithms (e.g., classification and regression tree (CART), iterative bisection 3). (ID3), Chi-squared automatic interaction detection (CHAID), decision stump, conditional decision tree), Bayesian algorithms (e.g., Naive Bayes, Gaussian Naive Bayes, Polynomial Naive Bayes, Average One Dependence Estimators (AODE), Bayesian Belief Network (BNN), Bayesian Network), Clustering algorithms (e.g., k-means, k-median, Expectation Maximization (EM), Hierarchical Clustering), Correlation Rule Learning algorithms (e.g., P This may include, but is not limited to, septrons, backpropagation, Hopfield networks, radial basis function networks (RBFNs), deep learning algorithms (e.g., deep Boltzmann machines (DBMs), deep belief networks (DBNs), convolutional neural networks (CNNs), stacked autoencoders), dimensionality reduction algorithms (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), summon mapping, multidimensional scaling (MDS), projection tracking, linear discriminant analysis (LDA), mixture discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble algorithms (e.g., boosting, bootstrap aggregation (bagging), AdaBoost, stacked generalization (blending), gradient boosting machines (GBMs), gradient boosted regression trees (GBRTs), random forests), SVMs (support vector machines), supervised learning, unsupervised learning, semi-supervised learning, etc.Further examples of architectures include neural networks such as ResNet-50, ResNet-101, VGG, DenseNet, and PointNet. In some examples, the ML models discussed herein may include PointPillars, SECOND, top-down feature layers (for example, the whole of which is incorporated herein by reference, see Patent Document 5), and / or VoxelNet. Architecture latency optimizations may include MobilenetV2, Shufflenet, Channelnet, and Peleenet. In some examples, the ML models may include residual blocks such as Pixor.

[0063] Memory 220 may, in addition or as an alternative, store one or more system controllers (which may be part of the drive components) that may be configured to control the steering, propulsion, braking, safety, emitter, communication, and other systems of the vehicle 202. These system controllers may communicate with and / or control the corresponding systems of the drive components 212 and / or other components of the vehicle 202. For example, a planning component 230 may generate commands based at least in part on perceptual data generated by a perceptual component 228 and / or simulated perceptual data, and transmit those commands to a system controller, which may control the operation of the vehicle 202 at least in part on those commands.

[0064] Although Figure 2 is shown as a distributed system, it should be noted that in alternative examples, components of vehicle 202 may be associated with computing device 214, and / or components of computing device 214 may be associated with vehicle 202. That is, vehicle 202 may perform one or more of the functions associated with computing device 214, and vice versa.

[0065] Exemplary route generation using tree search and / or agent filtering Figures 3A–3C illustrate an illustrative flowchart of an exemplary process 300 for generating a path to control a vehicle (e.g., vehicle 202) using a tree search that alternately uses agent filters and / or estimated costs that can be determined at least in part on sensor data and / or perception data, which may include an occupation map showing static objects and / or a map of predicted positions and / or velocity probabilities for dynamic objects. In some examples, the exemplary process 300 may be performed by the guidance component of the autonomous vehicle (e.g., guidance component 232), but in at least some examples, the exemplary process 300 may be performed, additionally or alternatively, by the planning, simulation, perception, and / or prediction components of the autonomous vehicle. In the latter example, the guidance component may coordinate the operations of various components, such as by transmitting API requests to APIs associated with each of the components and using the API responses to perform the tree search discussed herein. In another example, the guidance component may coordinate the operations of components using messages transmitted to and from various components via a publish-subscribe network. In some examples, the tree search performed by the guidance component may include performing an improved Monte Carlo tree search (MCTS), partially observable Monte-Carlo planning (POMCP), Markov decision processes (MDP), such as partially observable MDP (POMDP), using the techniques discussed herein, including agent filtering, upper / lower cost estimation, and / or defaulting to a default policy.

[0066] In Operation 302, the exemplary process 300 may include receiving a route associated with at least a starting position in the environment, in accordance with any of the techniques discussed herein. Figure 3A shows the environment 304 in which a vehicle 306 performing the exemplary process 300 is located. The starting position may be associated with the current position of the vehicle 306, and the route may specify an ending position and may include intervening objectives or operations, such as aiming to exit a highway, attempt to stay in a particular lane, or park in a particular block (but not a specific location, although in some examples a particular part of the block may be identifiable).

[0067] In Operation 308, exemplary process 300 may include receiving sensor data from one or more sensors in accordance with any of the techniques discussed herein. The sensors may be associated with a vehicle and / or another computing device. Operation 308 may also include, or alternatively, determining environmental state data based at least in part on the sensor data. In some examples, a perceptual component may determine environmental state data 310 (abbreviated as "state 310" in the figure for space reasons), and for any dynamic object detected by the perceptual component, an agent filter may determine whether the dynamic object is reactive or inactive (or, as above, an associated score). The environmental state data 310 may be associated with the most recently received set of sensor data (e.g., the current time, however there may be a slight delay between receiving the sensor data and determining the perceptual data).

[0068] Furthermore, the environmental state data 310 may include the position, orientation, and / or characteristics of the vehicle 306 in the environment, and the environmental state data 310 may correspond to the real-time operation of the autonomous vehicle. The environmental state data 310 may additionally or alternatively include a representation of the object type associated with one or more objects (e.g., passenger car 312, heavy vehicle 314, passenger car 316, building 318, building 320) and / or characteristics associated with one or more objects (e.g., position, velocity, acceleration, orientation, material type, kinematic coefficients). It should be noted that although the environmental state data 310 is represented as a two-dimensional image, in additional or alternative examples, the environmental state data 310 may include data structures such as public submessages, three-dimensional representations, etc. In some examples, the environmental state data 310 may further include a prediction of whether or not there are occluded objects, or something similar, which is incorporated herein by reference in its entirety and discussed in more detail in Patent Document 6 filed May 8, 2019. In additional or alternative examples, the prediction of whether an obscured object exists may be determined by a machine learning model, which receives environmental state data as input and outputs a probability field. Any area of ​​the environment associated with a probability of meeting or exceeding a threshold may be output as a potential false negative, which can be used as part of candidate action generation.

[0069] Environmental state data may include objects classified as dynamic by a perceptual component. For example, dynamic objects, also referred to herein as agents, may include vehicles, cyclists, pedestrians, balls, plastic bags being blown by the wind, and / or any other movable objects, or objects that are likely to move within a time period. Objects such as benches or tables may be movable, but are unlikely to move within a time period related to the vehicle's movement and may be considered static objects. Environmental state data 310 may include dynamic objects and may include dynamic object classifications and / or probabilities determined by an agent filter in relation to the dynamic objects. For example, the classification may include whether the dynamic object is inactive or reactive, and / or their probabilities. As described above, the agent filter may include an ML model trained to receive object tracks associated with the dynamic object, the current state of the vehicle and / or candidate actions further discussed herein, and / or sensor data associated with the dynamic object, and to determine classifications and / or confidence scores (e.g., posterior probability, probability) of whether the dynamic object is inactive or reactive, by a neural network or any of the other ML techniques described above. In some cases, if the reliability score determined by the ML model meets or exceeds a reliability threshold, the detected object may be classified as a reactive object; otherwise, it may be classified as an inactive object. In yet another case, the ML model may output the classification itself, either in addition to or instead of the reliability score, from the final layer, either as an addendum or as an alternative.

[0070] Inactive objects may be dynamic objects that are unlikely to change their own behavior (for example, having a reliability score below a reliability score threshold) and / or whose behavior depends at least partially on the actions of vehicle 306, while reactive objects may be dynamic objects that are likely to change their own behavior based on the actions of vehicle 306. Inactive objects are shown in the figure using the cross-hatching indicated by 322. Reactive objects are shown in the figure using the cross-hatching indicated by 324. Static objects are shown in the figure using the cross-hatching indicated by 326. Note that in Figure 3A, traffic is moving to the right, and therefore large vehicle 314 may have already substantially passed vehicle 306, and therefore may be unlikely to change the action it takes based on the actions of vehicle 306. Vehicles 312 and 316, on the other hand, have not yet passed vehicle 306 and may be significantly more likely to base their own actions at least partially on the actions of vehicle 306.

[0071] Referring to Figure 3B, in Operation 328, the exemplary process 300 may include determining the root node 330 of the tree search based at least partially on sensor data, according to one of the techniques discussed herein. In some examples, determining the root node may include determining the data structure 332 for the tree search, which may include setting up and storing directed acyclic graphs (DAGs), upper confidence bounds applied to trees (UCTs), determined sparse partially observable trees (DESPOTs), etc., to model control and environmental states. The root node may be associated with the current time and / or the most recent sensor data or batch of sensor data. Thus, the root node may be associated with perceptual data, which may or may not include predictive data. In other words, the root node may identify environmental state data, including the current position, orientation, velocity, acceleration, classification, etc., of static and / or dynamic objects in the environment (including similar information about the vehicle, which may be generated by the vehicle's localization component), and may, additionally or alternatively, include historical data of the same.

[0072] Predictions of how an object will behave in the future, and how this data will change accordingly, may be associated with prediction nodes discussed herein, and in some examples, prediction data for the current time step may be associated with a root node. In other words, a root node may contain prediction data that identifies one or more possible future states of the environment, including the current state of the environment containing the object within it, location data related to the vehicle (e.g., determined by SLAM), and / or the position, orientation, velocity, acceleration, classification of the object associated with future time.

[0073] The diagram shows prediction nodes (and root nodes, which may also be prediction nodes) as rectangles, and action nodes as circles. The dashed lines and circles 334 represent the relationships between root node 330 and action nodes that have not yet been discovered / determined based on root node 330. Root node 330 may identify environmental state data 310 and one or more predicted environmental scenarios. For simplicity and space limitations, only current environmental state data is shown in Figures 3A-3C, but predicted environmental state data, such as the predicted environmental states shown in Figures 6A and 6B, may be further associated with root node 330.

[0074] In Operation 336, the exemplary process 300 may include determining a first candidate action for controlling the vehicle's movement (at least partially based on a previous prediction node) according to one of the techniques discussed herein. The candidate action determined in Operation 336 may be determined at least partially based on the prediction node of the most recently determined layer of the prediction node. For example, Figure 3B shows only the first layer of the prediction node, which includes only the root node 330. Figure 3C shows the second layer of the prediction node, which includes prediction nodes 350 and 352. Determining a first candidate action may include providing the planning component with environmental state data associated with the prediction node on which the candidate action is based. For example, the first action node 338 indicates one or more candidate actions based on the environmental state data indicated by the root node 330. death Figure 3B shows one such candidate action, candidate action 340, which involves controlling the vehicle to move straight forward.

[0075] The environmental state data may be the current environmental state data (if the prediction node is the root node) or the associated prediction environmental state data, as described above. Regardless, determining the first candidate action in the planning component may involve a nominal method of trajectory planning. In additional or alternative examples, determining a candidate action based at least partially on environmental data may involve a trajectory determination system separate from the nominal trajectory generation system of the planning component. This separate system may determine a candidate action based at least partially on lane reference type, target type, extension variables, offset, multiplier, and / or trend type. This is also discussed in more detail in Figures 8A and 8B.

[0076] The lane reference type may indicate whether the lane reference for generating candidate actions should be generated using sensor data or using a predefined lane reference, such as one shown in a pre-generated map. The lane reference may or may not be associated with the center of the lane, as shown in Figure 8 (for example, the lane reference may be the center of the lane for straight sections of the lane, but for curves, the lane reference may be biased towards the inside or outside of the curve).

[0077] Objective types can define the types of actions needed to achieve the current route or mission. For example, objective types can specify the vehicle's current lane, adjacent lanes, parking spaces, or open spaces (e.g., where there are no lane markings).

[0078] Extended variables can identify weights, distances, coefficients, and / or other limits regarding how far unoccupied space can be explored laterally (and / or vertically in some cases) (for example, how far a candidate action can move a vehicle laterally). For example, extended variables could be general constraints on how much candidate actions can differ.

[0079] The offset may identify a predetermined distance from a lane reference used to iterate through the search for candidate actions. The distance may be determined dynamically, at least in part, based on sensor data such as vehicle speed, environmental complexity (see Patent Document 7, filed February 24, 2021, which is incorporated herein by reference).

[0080] The multiplier can be a coefficient between 0 and 1, which, when multiplied by the current maximum speed permitted by law, can determine the maximum speed associated with a candidate action. The multiplier may be randomized, modified according to a pattern, and / or constrained at least partially by a range set by the planning component, at least partially based on environmental condition data and the vehicle's previous trajectory.

[0081] Tendency types may be associated with higher curvature, velocity, and / or acceleration, and may identify constraints on curvature, velocity, and / or acceleration associated with various behavioral types such as “assertive,” which may be required when the perceiving component detects a complex environment or other aggressive traffic; “nominal,” which may provide a standard for typical interactions with other agents; “conservative”; and / or “submissive.” The perceiving engine and / or planning component may work together to determine the tendency type to be used, as discussed in more detail in Patent Document 8, filed August 28, 2020, which is incorporated herein by reference.

[0082] When the planning component generates a first candidate action, the guidance component may update the data structure 332 to include a first action node 338 that identifies the first candidate action. Figure 3B also shows two additional action nodes 342 and 344, which are shown with dashed lines because they may not be generated in cases where the tree search algorithm finds a low-cost path with minimal exploration. In other words, action nodes 342 and 344 may not have been explored yet, but may be generated by further iteration of operation 336 to enumerate additional candidate actions.

[0083] In some examples, a first candidate action may be associated with controlling the vehicle over a first time period. As discussed below, a candidate action in a deeper layer than the one associated with the first candidate action (for example, it includes action nodes 342 and 344) may be associated with controlling the vehicle over a second time period. In some examples, the time periods associated with each subsequent layer of an action node may be equal, or, in additional or alternative examples, the length of the time period may increase (for example, exponentially or logarithmically). For example, a first candidate action may be associated with controlling the vehicle over a 1-second period, a second candidate action associated with an action node one layer deeper than the first layer (for example, action node 356) may control the vehicle over 1.1 seconds, and a third layer may control the vehicle over a 1.25-second period, and so on. This extended time period can ensure greater precision and / or accuracy in imminent actions, while also ensuring that more distant actions do not control the vehicle in a way that results in higher costs / negative consequences.

[0084] In Operation 346, exemplary process 300 may include determining a first cost associated with a first candidate action using a lower cost, in accordance with any of the techniques discussed herein. In some examples, determining the first cost may be part of determining the first candidate action in Operation 336, and / or the cost determination may occur simultaneously using separate processing units, or when the first candidate action is received. In some examples, the guidance system may determine the cost, which may be based at least in part on environmental state data. In detail, the cost may be based at least in part on various subcosts such as proximity cost, safety cost, comfort cost, and / or progress cost. These subcosts may be based at least in part on environmental state data indicated by the last predicted node (whether the last predicted node is the root node or another predicted node). The proximity cost may be based at least in part on the minimum, average, or other distance that the candidate action moves the vehicle from static and / or dynamic objects. Safety costs may include a score indicating compliance with road rules, proximity to other objects and / or speed associated with the candidate action (for example, safety costs may penalize candidate actions that are close to an object (e.g., within a threshold distance from the object) and moving at high speed, while candidate actions that are close to an object but associated with low speed may not be penalized or only receive a small penalty, and high-speed candidate actions that are far from other objects may not be penalized by this cost), and / or proximity to impassable surfaces (e.g., sidewalks, buildings, closed lanes). In examples where safety costs include variable costs based on speed and lateral distance to an object, the costs may be determined at least in part on a hinge function such as an L1 or L2 hinge function.In some cases, the hinge point in a hinge function where a penalty begins to be applied may be based on the distance to the object, the velocity associated with the candidate action, the object track, and / or the object type. For example, a penalty may begin to be applied further away from the motorcycle rider than from the vehicle, and / or the penalty may be higher / severeer with respect to the motorcycle rider than with respect to the vehicle. Furthermore, if the candidate action is within the vehicle's threshold distance (e.g., the hinge point in the hinge function), the faster the velocity associated with the candidate action, the more severe the penalty may be. In at least one example, the threshold distance for applying a penalty specified by an L1 or L2 hinge function may be based at least partially on the velocity associated with the candidate action. In other words, faster candidate actions will be penalized further from the object than slower candidate actions, and the closer the faster candidate action is to the object, the more severe the L1 or L2 penalty may be compared to the same distance from the slower candidate action to the object (e.g., the steeper the slope in the L1 case, the larger the coefficient and / or squared value).

[0085] Comfort costs may be based at least in part on the speed, jerk, and / or acceleration associated with a candidate action, and / or whether the candidate action would violate threshold jerk and / or acceleration. Progress costs may be based at least in part on the completion of a mission or sub-objective (e.g., parking on a portion of a block, parking within a block, changing lanes), and / or the vehicle's displacement along the route. For example, progress costs may reward the further the vehicle travels along the route if the candidate action is performed. Costs calculated as rewards may have the opposite sign to other sub-costs. For example, if there is a positive cost for a candidate action that would violate a comfort metric (e.g., if that candidate action would exceed a threshold jerk), the reward may be a negative sub-cost. Further details on how costs are determined are discussed in Patent Document 9, filed May 11, 2020, which in whole is incorporated herein by reference.

[0086] In at least one example, the cost associated with a particular action node may include the cost of arrival (e.g., for any action node deeper than the first layer, the sum of the costs of all action nodes leading up to that action node), the cost of performing the action (e.g., this may include the aforementioned costs such as comfort cost, progress cost, etc.), and the cost of moving further beyond that action node (this may also be characterized as the cost of transitioning to a different state in the future). Modeling this future cost, also known as cost-to-go, can be complex and may require significant computational power if the number of action nodes explored in a tree search is taken into account. In reinforcement learning, cost-to-go is also called the "value" of being in a particular state.

[0087] In some examples, instead of determining all of the subcosts, Operation 346 may include using a lower or upper cost to serve as a substitute for determining at least some of the subcosts, such as cost-to-go. For example, the lower cost could be 0, and the upper cost could be the cost of using the default action. The lower cost could be a predefined heuristic, but in additional or alternative examples, the lower cost could be determined by a machine learning model trained at least in part on simulating or operating a vehicle and determining the minimum cost of the actions taken by the vehicle for similar scenarios. This machine learning model could determine the lower cost at least in part on environmental scenario data and / or tracks associated with the vehicle (i.e., that data could be provided as input). In yet another example, the lower cost may be updated after all or most of the candidate actions based on the prediction node have been determined. In such a case, the lower cost may be updated to be the cost of the candidate action with the lowest cost.

[0088] The default action could be to maintain the current speed, direction, steering speed, etc., or it could be a predefined action, such as driving straight at the speed limit detected as being associated with the section of road where the vehicle is located. In another example, the default action could be determined by a dedicated machine learning model for determining the default action, at least in part, based on perceptual data. Note that in the examples shown in Figures 3A-3C, the lower cost is used in the first action layer after the root node. This is done as an example only, and the upper cost could be used instead of the lower cost. Figures 3A-3C merely show that the lower and upper costs can be used alternately. In some examples, the lower and upper costs may alternate according to ratios such as 3 lower costs for 2 upper costs, 2 upper costs for 2 lower costs, or 4 lower costs for 1 upper cost.

[0089] In some cases, ramping ratios can be used to change the ratio of lower costs to upper costs used in successive layers. For example, before introducing lower costs and increasing the frequency with respect to successive layers, upper costs may be used more or exclusively in the lowest layers (e.g., the first two or three) (or vice versa). ) In some cases where the tree is sufficiently deep, the ramping ratio is lower bound costA steady state can be reached when the costs are used exclusively or in a specific ratio (e.g., equal in a 1:1 ratio). The lower cost forces the tree search algorithm to explore more of the tree, so purely using the lower cost guarantees finding the optimal route. However, by incorporating the upper cost, the tree search algorithm becomes more greedy, and can be tuned by balancing the ratio of lower cost usage to upper cost usage. In other words, tuning a tree search algorithm can involve balancing the algorithm between the completeness of the amount of space explored / the time and amount of computation required to find the path, and finding the best path.

[0090] In some cases, search parameters such as the ratio of lower cost used to upper cost, or whether lower cost or upper cost is used exclusively, can be determined at least partially based on the perceived data on which the machine learning model is used. For example, training data can be generated by experimentally changing the ratio used, or by using either lower cost or upper cost exclusively, and by remembering the generated paths, the time it took to compute them, the computation cycles, and / or the number of nodes and / or layers, the costs associated with the paths, and / or how the lower / upper parameters were set. The machine learning model can be trained to output tree search parameters that are predicted to reduce the computation cycles used, the number of nodes explored, and / or the costs associated with the paths, based on the perceived data available to the machine learning model, such as environmental state data indicated by the root node. Parameters may include, as an addition or alternative, the depth of the tree search, the width of the tree search, sampling parameters (such as how to modify predictions and the number of predictions made, which are discussed in more detail with reference to Figures 8A and 8B), parameters for determining whether to group prediction nodes into a single prediction node (e.g., whether an exact match of dependent candidate actions is required, the threshold distance used to identify what qualifies as “similar”, and / or k-means clustering parameters), whether dynamic objects may be reclassified during the tree search, and / or how many layers the search may explore before reclassification.

[0091] By replacing at least a portion of the costs with lower or upper cost estimates, it may be unnecessary to run simulations or prediction components before the first candidate action is output, and / or this replacement may reduce computational costs and increase the speed of the tree search algorithm.

[0092] Referring to Figure 3C, in Operation 348, exemplary process 300 may include determining a first prediction node associated with a future state of the environment, at least in part on a first candidate action, according to one of the techniques discussed herein. In at least one example, determining the first prediction node may include running a prediction component, which may run a Kalman filter, a machine learning model, or a simulation. Running a simulation may include instantiating a simulation, at least in part on environment state data and a first candidate action. Instantiation may use a set of template models that may include representations of various objects, such as three different passenger car models, four different pedestrian models, etc., which may be scaled and positioned as necessary to match the environment state data indicated by the prediction node on which the candidate action is based. Any number of different models may exist. In some examples, the template model may include a three-dimensional model of the surface of an object without any texturing, but in additional or alternative examples, the model may include textures. The template model may include a polygon mesh, a triangular mesh, etc. In some cases, models associated with dynamic objects may have a higher polygon count than models associated with static objects. In yet another case, objects in a simulation may be simplified to basic two- or three-dimensional geometric shapes, corresponding to the dimensions of the simulation. For example, vehicles and / or pedestrians may be represented as rectangles of various shapes, and pedestrians may be represented as cylinders.

[0093] In some examples, the instantiated simulated environment may include lighting and / or textures, but the techniques described herein work without lighting and / or textures. For two-dimensional simulations, lighting and textures may not be used. Furthermore, lighting and textures may not be used for three-dimensional simulations either. The simulated environment may include a model of the vehicle 306 itself. In some examples, instantiating the environmental portion of the simulated environment may be obtained at least partially based on map data and / or perceptual data (which may be indicated by environmental state data associated with the prediction node on which the first candidate action is based), and objects in the simulated environment may be obtained at least partially based on environmental state data indicated by the prediction node on which the first candidate action is based.

[0094] Running a simulation may involve controlling a simulated representation of the vehicle 306 according to candidate actions and based on the simulation instantiation described above, which may involve using environmental state data indicated by the prediction node on which the candidate actions are based. Running a simulation may also involve, as an addition or alternative, simulating the movement of dynamic objects in the simulated environment. Simulating the movement of dynamic objects may depend on whether the dynamic object is indicated as reactive or inactive, as determined by the agent filter, which may be indicated in the environmental state data associated with the prediction node on which the candidate actions are based. If the agent filter classifies the dynamic object as inactive, a Kalman filter or other simpler prediction method may be used to determine the predicted trajectory of the dynamic object. However, if the agent filter classifies the dynamic object as reactive, the simulation component may use a separate instance of the planning component, the agent behavior model (see Patent Document 1). ) The motion of a dynamic object may be controlled using one or more ML models trained to control the simulation of the dynamic object based at least partially on candidate actions. For example, a component controlling the simulated motion of a dynamic object may generate commands to control the simulated dynamic object based at least partially on the simulated execution of candidate actions by the simulated vehicle 306. However, in another example, the simulation may be the output of a mathematical structure or a machine learning model.

[0095] Once the simulation is complete (for example, when a candidate action is completed), the predicted tracks resulting from objects in the environment (e.g., position, orientation, etc., as discussed above), including the track resulting from vehicle 306 performing the first candidate action, may be used to determine updated environmental state data. The data structure 332 may be updated to include this updated environmental state data and a prediction node 350 that shows the predicted state of the environment that may result from implementing the first candidate action. In some examples, the simulation may be rerun with slightly different variables (e.g., changing the tendency of dynamic objects from "reserved" to "aggressive," "obedient," or "nominal") to determine a second updated environmental data associated with a different prediction node, namely prediction node 352. In some examples, the simulation component may output a number of potential scenarios, each of which may be associated with a probability. In such examples, the guidance component may create prediction nodes for each potential (predicted) scenario associated with a probability that meets or exceeds a probability threshold.

[0096] In another example, one or more predictions may be grouped together in a single node, at least in part, on the basis of determining similarity between predictions and / or determining similarity of candidate actions determined based on those predictions. In other words, after the next set of candidate actions has been determined in a subsequent action node layer (e.g., a layer containing a second action node 356), the process may include determining similarity between the candidate actions in that layer. In some examples, this may include determining the Euclidean distance between the spatial and / or spatiotemporal positions defined by two candidate actions (e.g., the Euclidean distance at a particular time, such as the time associated with the action node layer). In some examples, perfect matching may be required, thresholding may be used (e.g., candidate actions that identify positions less than a threshold distance from each other may be identified as similar), and / or k-means clustering may be used to determine that two (or more) candidate actions are similar (on the grounds that they are clustered into the same cluster).

[0097] If any two candidate actions are identified as similar, the process may involve associating the predictions from which those two (or more) similar candidate actions were generated / on which they depended with the same prediction node. In some examples, if multiple predictions are associated with the same prediction node, the process may involve determining weights associated with each different prediction. The weights may indicate the degree to which a prediction exists with that prediction node. Determining these weights may be based at least in part on the similarity between the candidate actions generated from the predictions and one or more actions associated with the various associated candidate actions determined from the other predictions. Grouping predictions into a single prediction node may be used by the process to determine a smaller subset of candidate actions to explore, such as one or the top p candidate actions ranked by cost, where p is a positive integer. For example, the top two candidate actions ranked by cost may be associated with a prediction node that identifies multiple predictions. Future exploration may be based at least in part on these two candidate actions.

[0098] In yet another example, the predictions of a particular prediction layer may be associated with all of the prediction nodes in that layer. In other words, there may be as many prediction nodes as there are prediction nodes, and each prediction node may represent all of the predictions. In such an example, weights may be associated with each prediction, indicating the likelihood of the prediction occurring and / or how similar the prediction is to the target prediction. Each prediction node may identify a different target prediction; in other words, prediction nodes may cycle among different predictions as a criterion for comparison.

[0099] Operation 348 may include a join operation that, as an addition or alternative, may result in a single predictor node representing two or more environmental states, and / or a single predictor node representing a single environmental state representing two or more environmental states associated with two or more previous predictor nodes. For example, when generating a second predictor node in the same layer as the first predictor node, such as predictor node 352 for predictor node 350, exemplary process 300 may include determining the similarity of at least two predictor nodes by an ML model. In such an example, the ML model may be a set of perceptrons and / or many layers of perceptrons (e.g., a CNN), a support vector machine (SVM), etc. The ML model may be trained to output a score indicating the similarity between two environmental states. The ML model may be trained according to a supervised method that labels environmental states as similar or dissimilar. If the score output by the ML model meets or exceeds a threshold, the two (or more) predictor nodes may be joined into a single predictor node; otherwise, those predictor nodes may remain separate. Based on similarity, further computational resources may be saved because the exploration of both nodes will yield substantially similar results.

[0100] In yet another example, an ML model may determine the embeddings associated with a prediction node, for example, by determining the embeddings associated with the environmental state data, which may involve transforming the environmental state data into a higher-dimensional data structure and projecting that higher-dimensional data structure into the embedding space. In such an example, the ML model may include embedding algorithms such as Uniform Manifold Approximation and Projection (UMAP), t-Distributed Stochastic Neighborhood Embedding (t-SNE), ISO Map, Local Linear Embedding (LLE), or other such algorithms. In the example,An exemplary process 300 determines an embedding for each of two prediction nodes and determines the distance (e.g., Euclidean distance) between the two embeddings in the embedding space. ru. If the distance is less than the threshold distance, the prediction nodes may be merged; otherwise, they may remain separate.

[0101] In Operation 354, exemplary process 300 may include determining a second candidate action for controlling the vehicle's movement based at least in part on environmental condition data indicated by a preceding prediction node, in accordance with any of the techniques discussed herein. For example, determining a second candidate action may be based at least in part on environmental condition data indicated by a prediction node 350. This relationship between a prediction node and a candidate action based thereon is indicated by an arrow, and in some examples, determining a second candidate action based at least in part on a prediction node 350 may be based at least in part on determining that the simulation that yielded the updated environmental condition data associated with the prediction node 350 did not result in a violation of operational constraints, did not exceed costs, or had no impact.

[0102] Without being limited, for example, motion constraints may include kinematic and / or qualitative metrics associated with the vehicle's behavior, such as the minimum distance between the vehicle and the nearest object, the acceleration required to avoid a collision, the time to collision, deviation from the route, whether the vehicle's jerk and / or acceleration meets or exceeds a threshold jerk and / or threshold acceleration, the predicted likelihood of a collision, etc. Additionally, or alternatively, motion constraints may include weights associated with various minimum distances that can be used to reduce or increase rewards based at least in part on the minimum distance the simulated autonomous vehicle has traveled from any object during the simulation.

[0103] In some examples, the final cost associated with the first candidate action may be determined after and / or simultaneously with the generation of the prediction node 350. In some examples, the decision to generate a second candidate action may be based at least in part on this final cost. For example, other final costs may be determined in relation to action nodes 342 and / or 344 and / or prediction nodes dependent thereon from the first action node 338 (prediction node 35 0 The decision to generate a second candidate action branching (via ) may be at least in part based on the decision that the first action node 338 is associated with a total cost of actions that is less than the total cost of taking another action. The total cost refers to the cost of the candidate action in question plus the total cost of all preceding actions in the branch leading to the candidate action in question. In the case of the second candidate action, the total cost would be the final cost associated with the second candidate action plus the final cost associated with the first candidate action.

[0104] The data structure 332 may be updated to include a second action node 356 and action nodes associated with any other candidate action determined at least partially based on the prediction node 350. Note that Figures 3B and 3C illustrate a simplified process in which only one branch of each layer is explored. Further actions may be determined from the same prediction node or from separate prediction nodes, as indicated by action nodes 342 and 344, and so on. Since the root node 330 itself may contain prediction environment state data, prediction nodes 350 and 352 can be considered to be in a second layer of prediction nodes.

[0105] An exemplary process 300 may initiate operations 358 and / or 360 at least in part on determining a second candidate action, but in some examples, operation 358 may be determined at least in part on operation 348, in addition or as an alternative.

[0106] In operation 358, the exemplary process 300 may include determining whether a dynamic object can be represented as an inactive agent or as a reactive agent, according to any of the techniques discussed herein. Operation 358 may include reclassifying at least one of the dynamic objects indicated by the most recent prediction node, such as prediction node 350 or 352. In some examples, operation 358 may be part of operation 348, but in some examples, operation 358 may receive candidate actions as input, which may include waiting for the next candidate action to be generated, at least partially, based on the prediction node. In the example shown, updating the dynamic object classification with respect to prediction node 350 may include using a second candidate action determined in relation to a second action node 356. In some examples, an agent filter may be trained to classify dynamic objects without using candidate actions as input, in which case operation 358 may be part of operation 348.

[0107] Regardless, the track associated with a dynamic object, indicated by the environmental state data associated with the prediction node, may be provided as input to the agent filter along with the vehicle's attitude and / or track. Note that at this stage in the tree search, the environmental state data includes the predicted track of the dynamic object. The agent filter may treat the predicted track of the dynamic object as the current track, and the current track identified by the root node as the past track, or the agent filter may treat the predicted track as the predicted track. In other words, the agent filter may be trained to handle a number of time steps of a track, including past tracks, current tracks, and / or predicted tracks. In some cases, the agent filter may use a portion of the available tracks associated with a time window, such as the last n seconds, where n is a positive number, relative to the most recent time in the tree search space.

[0108] If the classification of a dynamic object changes, as determined by the agent filter, the classification of that object may be modified in the environmental state data associated with the prediction node. For example, a reactive object may be reclassified as an inactive object, and vice versa. In operation 362, exemplary process 300 may include changing such status in the appropriate prediction node in the data structure. In some examples, this new classification may be used by the agent filter in operation 364 as part of a simulation to determine the subsequent prediction node. For example, an unshown prediction node determined based on the second action node 356 may use the updated classification determined in operations 358 and 362 as part of the simulation. If the classification of any of the dynamic objects identified in the environmental state data identified by the prediction node does not change, exemplary process 300 may continue without modifying that prediction node.

[0109] In Operation 360, exemplary process 300 may include using an upper cost to determine a second cost associated with a second candidate action, in accordance with any of the techniques discussed herein. As stated above, purely as an example, the use of a lower cost in Operation 346, and alternating between the lower and upper costs in a 1:1 ratio, would determine that the upper cost is used in Operation 360. However, as stated above, the upper cost may be used first, and the ratio may be any other ratio other than 1:1. Nevertheless, determining a second cost may include using an upper cost, which may be a predetermined cost associated with a default action.

[0110] In Operation 366, exemplary process 300 may include controlling a vehicle on at least partly on a route that includes a first candidate action and a second candidate action, at least partly on the total cost associated with them, in accordance with any of the techniques discussed herein. For example, a tree search may continue to alternately determine action nodes and prediction nodes until a termination event occurs, such as determining a predefined number of action nodes, a time limit on calculations, a limit on calculations or storage, that a displacement along the route is achieved, that several time steps have been reached (e.g., that actions have been determined up to the planning period), that an action is determined to reach the target location, or that the route to the target location is below a cost threshold.

[0111] The guidance component may determine one or more consecutive paths through a set of action nodes when the set of nodes is generated and / or when an end event occurs. In some examples, the guidance component may trace one or more potential paths through a set of action nodes when they are generated and backtrace the lowest-cost path (i.e., from the node at the end of the potential path in the last layer of nodes to the starting / root node, and / or from the node at the vehicle's current position) to ensure there are no collisions. Nevertheless, the guidance system may determine a consecutive path through a set of nodes that is associated with the lowest cost and / or advances the most along the route. A consecutive path of action nodes is a set of nodes connected by a dependency relationship in data structure 332. Intervening prediction nodes are not taken into account for path planning beyond the subcosts they may indicate. If two action nodes are connected by an intervening prediction node, they are dependent, which indicates that the lower-level action node (e.g., the second action node 356) starts from the end position of the higher-level action node (e.g., the first action node 338).

[0112] The guidance system may search for a path from the root node to the last layer of data structure 332. Searching may involve determining a contiguous set of connections between nodes of various sets of nodes from the root node to the action node in the deepest layer of data structure 332. Determining the path may involve searching for a solution in a multivariate space that maximizes (or satisfies the diversity heuristic) a combination of displacement along the route and lateral / azimuth diversity among the solutions, and minimizes the cost based at least partially on a cost map over a given time interval. For example, the search algorithm may include algorithms such as D*, D*lite, Focused Dynamic A*, A*, LPA*, Dijkstra's algorithm, etc., but other search algorithms for searching and / or generating directed graphs and / or weighted directed graphs may be used. In some examples, a search may consist of a set of rules that may include one or more rules specifying, for example, boundaries for determining a path within a range (for example, boundaries may be determined at least in part based on sensor data and / or maps), node connection rules (for example, a node may have only one parent node), and so on. In some examples, a search may include determining a directed graph between nodes of a set of nodes. The directed graph may include connections (e.g., edges) between a first node and a second node, and / or weights (e.g., costs) associated with those connections.

[0113] In some examples, before performing a complete route search, the guidance component may include determining whether previous routes and / or connections determined by the guidance system are feasible (e.g., satisfying current constraints such as speed, maximum steering angle, and / or boundaries, being collision-free, and having a cost below a cost threshold).

[0114] In some cases, the path determined by the guidance system may be coarse. For example, a coarse path may identify the approaching position, direction, speed, and / or curvature of the vehicle at intervals of 1 second or 500 milliseconds, but the vehicle's components may require or be controllable at finer time intervals (e.g., 10 milliseconds, 100 milliseconds). In other words, a coarse path may not be smooth enough for the vehicle to follow without significant errors. In some cases, a first type of processor (e.g., a graphics processing unit (GPU)) may determine prediction and action nodes, and / or the path, and a second type of processor may smooth the path generated by the GPU, and / or determine a trajectory for controlling the vehicle based at least partially on the smoothed path.

[0115] The guidance systems discussed herein may identify a route as feasible based at least in part on the costs discussed herein, and / or determine a reliability score associated with that route. The guidance system may output routes and / or reliability scores, which an autonomous vehicle may use to control its movement, for example, by generating a trajectory based at least in part on the routes. In some examples, the guidance system may output primary routes and / or incidental routes. For example, the guidance system may determine incidental routes based at least in part on generating a set of candidate routes, determining that the set contains two groups of candidate routes (for example, the two groups may be two distinct homotopic groups) based at least in part on a threshold distance, and selecting a primary route from the first group and an incidental route from the second group. In some examples, a primary route may be selected based at least in part on determining that the primary route is associated with a first total cost that is smaller than a second total cost associated with an incidental route. The primary path may be associated with a first total cost, and / or the incidental path may be associated with a second total cost that is below a cost threshold and / or the minimum cost of each associated group.

[0116] Figure 4 shows a three-dimensional representation 400 of four different sets of candidate actions (i.e., trajectories in the example shown) generated in four different action layers of the tree search. The first set of candidate actions 402 was generated at least partially based on the vehicle's position 404. These candidate actions 402 may, additionally or alternatively, be determined at least partially based on orientation, velocity, acceleration, steering speed, and environmental state data (discussed in more detail above with respect to static / dynamic objects, etc.) associated with the vehicle's movement. The space occupied by the vehicle is represented as a dashed line 406. Figure 4 also shows two road edges, road edge 408 and road edge 410. The height of a candidate action indicates the velocity and / or acceleration associated with that candidate action.

[0117] A second set of candidate actions 412 may be generated at least in part on selecting a first candidate action from a first set of candidate actions 402 for exploration, and at least in part on environmental state data, including the final position 414, orientation, speed, steering speed, etc., that the first candidate action will cause the vehicle to achieve when it completes its execution. The second set of candidate actions 412 may also be determined at least in part on environmental state data indicated by prediction nodes determined at least in part on the first candidate action.

[0118] A third set of candidate actions 416 may similarly be based at least in part on the selection of a second candidate action from the second set of candidate actions 412, the environmental state data generated in connection therewith, and / or the final position 418, orientation, velocity, steering speed, etc., that the second candidate action will affect. A fourth set of candidate actions 420 may similarly be based at least in part on the selection of a third candidate action from the third set of candidate actions 416, the environmental state data generated in connection therewith, and / or the final position 422, orientation, velocity, steering speed, etc., that the third candidate action will affect.

[0119] Representation 400 may be a visual representation of a determined, sparse, partially observable tree (DESPOT) determined according to a partially observed Markov decision process (POMDP).

[0120] Figures 5A and 5B show top-down diagrams of various candidate actions that can be generated by a tree search for different action nodes at the same level of the tree search. For example, in Figure 5A, candidate action 500 may be generated based on environmental state data 502, which may be represented by a prediction node. In Figure 5B, candidate action 504 may be generated based on environmental state data 502, which may be represented by the same prediction node. For example, candidate action 500 may represent candidate action 340 associated with the first action node 338, and candidate action 504 may be associated with another action node at the same layer of the data structure, such as action node 344.

[0121] Figures 6A and 6B show top-down diagrams of two different predicted environmental states, environmental state data 600 and environmental state data 602, associated with a single prediction node or two different prediction nodes, which can be determined at least partially on the candidate action 500 shown in Figure 5A. Differences between environmental states can be at least partially on changing the simulation parameters of the candidate action 500, as described above with respect to operations 348 and / or 364. In a first simulation that may result in determining the environmental state data 600, both reactive objects, vehicles 604 and 606, may be assigned a tendency of "nominal" or "moderate," which may cause vehicle 606 to come to a complete stop, vehicle 604 to come to a complete stop or slow down, considering the simulated execution of the candidate action 500 by vehicle 306, which may result in the final position 608 of vehicle 306.

[0122] Referring to Figure 6B, the environmental state data 602 may be determined at least in part based on a simulation in which two reactive objects, vehicle 604 and vehicle 606, can both be assigned a tendency to be "aggressive" or "low attention," which may cause those vehicles to ignore vehicle 306, fail to consider vehicle 306, or attempt to block vehicle 306's path when vehicle 306 performs candidate action 500, resulting in vehicles 604 and 606 being positioned in front of vehicle 306.

[0123] In at least one example, environmental state data 600 may be associated with a first prediction node, and environmental state data 602 may be associated with a second prediction node, because the join operation may determine that the two environmental states should be separate because they can be sufficiently different. For example, environmental state data 600 may be associated with prediction node 350, and environmental state data 602 may be associated with prediction node 352. Note that both of these prediction nodes are subordinate to the same action node, namely the first action node 338, because the environmental state data associated with both were determined at least partially based on the same action, namely candidate action 500 (which may correspond to candidate action 340). Note also that these prediction nodes are on the same layer of data structure 332. In additional or alternative examples where the resulting environmental states are more similar, both environmental states can be represented by the same prediction node, or one environmental state can represent both by selecting a representative environmental state, or by averaging the environmental states.

[0124] Figures 7A and 7B, shown in Figure 6A, show top-down diagrams of two different candidate actions, candidate action 700 and candidate action 702, respectively, generated based on environmental state data 600 associated with a first prediction node, for example, prediction node 350. For example, candidate action 700 may be associated with a second action node 356, and candidate action 702 may be associated with an unshown action node that, like action node 356, is also dependent on prediction node 350.

[0125] Figures 7A and 7B also show the results of agent filtering that changes the classification of a dynamic object from a reactive object to an inactive object for use by the prediction and / or simulation components when generating the next level prediction node based on candidate action 700 or candidate action 702, respectively. Operations 358 and / or 362 may bring about this change. Compare the classification of vehicle 604 in Figures 6A and 6B, where vehicle 604 is classified as a reactive object, with the classification of vehicle 604 in Figures 7A and 7B, where vehicle 604 is reclassified as an inactive object. In such an example, operations 358 and 362 may include classifying vehicle 604 as an inactive object. This may be because vehicle 604 has reached a point where it is unlikely, or will no longer, change its behavior based on the actions of vehicle 306. This means that in subsequent prediction operations, vehicle 604 can be simulated in a simpler way, regardless of the candidate actions of vehicle 306.

[0126] Figures 8A and 8B provide further details regarding the determination of candidate actions. Figure 8A shows a top-down view of the curved road 800, lane criterion 802, and offset criterion 804 used to probabilistically generate actions for tree search. The shown road 800 may be a single lane for simplicity. The lane criterion 802 may be predetermined and stored in relation to the map, or the lane criterion 802 may be determined by the perception component of the vehicle 306. The offset 806 may be determined probabilistically or iteratively, as discussed in more detail in relation to Figure 8B. The offset may be used to determine the offset criterion 804, which may be a displacement of the lane criterion 802 by the amount defined by the offset 806. The guidance component and / or planning component may determine candidate actions based at least partially on the lane criterion 802 when the offset is 0, which would involve generating a trajectory that follows the lane criterion 802. In complex environments as determined in Patent Document 7, etc., the maximum permissible legal speed (or a predefined speed below the maximum permissible legal speed) ) Following the lane standard at a speed equal to, or accelerating / decelerating to equal to, a default policy used by tree search may be this default policy. This default policy may be used in some examples to determine the upper limit cost. The default policy may be the best-case action, such as when all other agents are modeled as careful or obedient (yielding to vehicles) or thereby minimizing comfort costs and / or overall costs.

[0127] In cases where the offset is non-zero, determining a candidate action may involve determining a trajectory 808 by drawing a curve, Bézier curve, gradient, etc., bounded according to smoothness constraints associated with the vehicle's comfort metric and / or steering capability, from the starting position 810 to a position on the offset criterion 804.

[0128] Figure 8B shows Graph 812 illustrating a sampling technique for determining actions for a tree search that can be used to ensure that a sufficient parameter space is sampled to find a suitable path. This can ensure diversity of candidate actions indicated by action nodes, thereby eliminating the need for separate action node join operations. In additional or alternative examples, join operations, such as those described above with respect to prediction nodes, may be used to combine similar action nodes to streamline the data structure. Graph 812 shown only shows two parameters that can change between candidate actions: velocity 814 (represented on the y-axis) in the displacement direction along the route and offset 816 (represented on the x-axis) from the lane reference. It should be understood that additional or alternative parameters, such as steering speed, final direction, position in the environment, acceleration, lateral and / or longitudinal velocity (instead of or in addition to velocity along the route), etc., may be used to generate candidate actions. Graph 812 includes samples, such as sample 818, shown as a diamond, which are sampled according to intervals of 3 meters per second and 0.2 meters per second. Any other interval may be used to sample space, and the interval may be a predetermined value based on the type of parameter being sampled, and may be chosen so that the candidate actions are sufficiently different. In some examples, the sample may be limited to an area such as region 820. Region 820 may be determined based at least in part on perceptual data, environmental state data, the current trajectory, and / or previous actions. In the example shown, the vehicle may currently be moving fast and therefore it may not be feasible to decelerate to 0 meters per second in the next time step. At most, the vehicle may be able to decelerate to 8 meters per second in the next time step. Also, the higher the speed of the vehicle, the more likely it is to violate the curvature / steering speed constraint, which may depend on the speed and thereby limit the offset value.In other words, the faster the vehicle moves, the more the candidate actions can be narrowed (laterally) so as not to exceed lateral acceleration or jerk.

[0129] Exemplary clause A: A step of receiving route data associated with a start and end position in the environment; a step of receiving sensor data from a sensor; a step of determining a first candidate action for controlling the movement of a vehicle, at least partially based on the sensor data and route data; a step of determining a first cost associated with the first candidate action, wherein the first cost is at least partially based on a lower limit cost, the lower limit cost is an estimate of the minimum cost of an action to change a first state of the vehicle at a time associated with the start of the first candidate action; a step of determining a first prediction associated with a first state of the environment, at least partially based on the first candidate action and sensor data, wherein the first prediction includes a first state of an object in the environment at a future time; and a step of determining a first prediction and route data A method comprising: determining a second candidate action for controlling the movement of a vehicle, based at least partially on; determining a second cost associated with the second candidate action, wherein the second cost is based at least partially on an upper cost, where the upper cost is an estimate of the cost of performing a default action; alternately applying either the lower cost or the upper cost to one or more subsequent candidate actions; and controlling the vehicle along a path including the first candidate action and the second candidate action, and at least partially on determining that a first total cost including the first cost and the second cost is less than a threshold or less than a second total cost associated with a second path including at least one candidate action different from at least one of the first candidate action or the second candidate action.

[0130] B: The method of paragraph A, further comprising the steps of determining whether an object is a reactive object by a machine learning model and at least in part on sensor data, and determining a first prediction, at least in part on performing a first simulation using the determination that the object is a reactive object, wherein performing a first simulation using the determination that the object is a reactive object includes determining the motion of the object's representation, at least in part on a first candidate action.

[0131] C: The method according to either paragraph A or B, further comprising the steps of determining that an object is an inert object by a machine learning model and at least in part on sensor data, and determining a first prediction at least in part on modeling the motion of the inert object, wherein modeling the motion of the inert object includes determining the motion of the inert object at least in part on the state of the object and excluding a first candidate action.

[0132] D: The method according to any one of paragraphs A to C, further comprising the step of assigning a first candidate action and a second candidate action to the same prediction node in a data structure, wherein the first and second predictions are associated with the same prediction layer as the first prediction, at least partially based on determining a first set of candidate actions on the first prediction and a second set of candidate actions on the second prediction at least partially based on determining a first action from a first set associated with a first minimum cost from among the costs associated with the first set, determining a second action from a second set associated with a second minimum cost from among the costs associated with the first set, and determining that the first and second actions are associated with locations that are within a distance from each other that is less than a threshold distance.

[0133] E: The method according to any one of paragraphs A to D, further comprising the steps of determining, at least in part, the likelihood that an object will modify its behavior in response to one or more of a first or second candidate action; determining, at least in part, that the likelihood meets or exceeds a threshold, to classify the object as a reactive agent; or determining, at least in part, that the likelihood is below a threshold, to classify the object as an inactive agent.

[0134] F: The method described in any one of paragraphs A to E, wherein the lower cost is zero and the upper cost is determined at least in part on at least one of the costs associated with the second candidate action or the default trajectory.

[0135] G: A system comprising one or more processors and a memory storing processor-executable instructions, wherein, when executed by one or more processors, the system receives route data associated with a start and end position in an environment, receives sensor data from a sensor, determines a first candidate action for controlling the movement of a vehicle, at least in part based on the sensor data and route data, and determines a first cost associated with the first candidate action, the first cost being at least in part based on a lower limit cost, the lower limit cost being an estimate of the minimum cost of an action to change a first state of the vehicle at a time associated with the start of the first candidate action, and determines a first cost associated with a first state of the environment, at least in part based on the first candidate action and sensor data. A system for performing an operation comprising: determining a first prediction to be attached, the first prediction including a first state of an object in the environment at a future time; determining a second candidate action for controlling the movement of a vehicle, at least based on the first prediction and route data; determining a second cost associated with the second candidate action, the second cost being at least based on an upper limit cost, where the upper limit cost is an estimate of the cost to perform a default action; alternately applying either the lower limit cost or the upper limit cost to one or more subsequent candidate actions; and controlling the vehicle, at least based on determining a route including the first candidate action and the second candidate action, and a first total cost associated with the first candidate action and the second candidate action.

[0136] H: The system described in paragraph G, wherein the operation further includes determining whether an object is a reactive object by a machine learning model and at least in part on sensor data, and determining a first prediction at least in part on performing a first simulation using the determination that the object is a reactive object, and performing a first simulation using the determination that the object is a reactive object at least in part on first candidate actions.

[0137] I: The operation further includes determining that an object is an inert object based at least in part on a machine learning model and sensor data, and determining a first prediction based at least in part on modeling the motion of the inert object, the modeling of the motion of the inert object including determining the motion of the inert object based at least in part on the state of the object and excluding a first candidate action, as described in either paragraph G or H.

[0138] J: The system described in any one of paragraphs G to I, the operation of assigning a first candidate action and a second candidate action to the same prediction node of a data structure, wherein the first and second predictions are associated with the same prediction layer as the first prediction, at least partially based on determining a first set of candidate actions based at least partially based on the first prediction and a second set of candidate actions based at least partially based on the second prediction; determining a first action from a first set associated with a first minimum cost from among the costs associated with the first set; determining a second action from a second set associated with a second minimum cost from among the costs associated with the first set; and determining that the first and second actions are associated with locations that are within a distance from each other that is less than a threshold distance.

[0139] K: The system described in any one of paragraphs G to J, further comprising determining, at least in part, the likelihood that an object modifies its behavior in response to one or more of a first or second candidate action, based at least in part on sensor data, and determining to classify the object as a reactive agent, based at least in part on whether that likelihood meets or exceeds a threshold, or determining to classify the object as an inactive agent, based at least in part on whether that likelihood is below a threshold.

[0140] L: A system described in any one of paragraphs G to K, where the lower limit cost is zero and the upper limit cost is determined at least in part on at least one of the costs associated with the second candidate action or the default trajectory.

[0141] M: A system according to any one of paragraphs G to L, wherein controlling a vehicle at least in part on a route is at least in part further on determining that a first total cost is less than a threshold, or less than a second total cost associated with a second route that includes at least one candidate action different from at least one of the first or second candidate actions.

[0142] N: A non-temporary computer-readable medium storing processor-executable instructions, the processor-executable instructions, when executed by one or more processors, determine to one or more processors: receive route data associated with a start and end position in an environment; receive sensor data from a sensor; determine a first candidate action for controlling the movement of a vehicle, at least in part based on the sensor data and route data; determine a first cost associated with the first candidate action, the first cost being at least in part based on a lower cost or an upper cost; and determine a first prediction associated with a first state of the environment, at least in part based on the first candidate action and sensor data. A non-temporary computer-readable medium causing an operation to be performed which includes determining a first state of an object in a future time environment, determining a second candidate action for controlling the movement of a vehicle, at least in part on the first prediction and route data, and determining a second cost associated with the second candidate action, the second cost being at least in part on a lower limit cost or an upper limit cost, alternately applying one of the lower limit cost or the upper limit cost to one or more subsequent candidate actions, and controlling the vehicle, at least in part on determining a route including the first candidate action and the second candidate action, and a first total cost associated with the first candidate action and the second candidate action.

[0143] O: Operation further includes determining that an object is a reactive object based at least in part on a machine learning model and sensor data, and determining a first prediction based at least in part on performing a first simulation using the determination that the object is a reactive object, and performing a first simulation using the determination that the object is a reactive object, and determining the motion of the object's representation based at least in part on a first candidate action, as described in paragraph N of the non-temporal computer-readable media.

[0144] P: The operation further includes determining that an object is an inert object based at least in part on a machine learning model and sensor data, and determining a first prediction based at least in part on modeling the motion of the inert object, the modeling of the motion of the inert object including determining the motion of the inert object based at least in part on the state of the object and excluding a first candidate action, as described in either paragraph N or O of the non-temporary computer-readable medium.

[0145] Q: The operation is to assign a first candidate action and a second candidate action to the same prediction node of a data structure, further comprising assigning the first and second predictions to the same prediction layer as the first prediction, at least partially based on determining a first set of candidate actions on the first prediction and a second set of candidate actions on the second prediction, determining a first action from a first set associated with a first minimum cost from among the costs associated with the first set, determining a second action from a second set associated with a second minimum cost from among the costs associated with the first set, and determining that the first and second actions are associated with locations that are within a distance from each other that is less than a threshold distance, in a non-temporary computer-readable medium as described in any one of paragraphs N to P.

[0146] R: An operation further comprising determining, at least in part, the likelihood that an object modifies its behavior in response to one or more of a first or second candidate action, based on sensor data; determining, at least in part, that the likelihood meets or exceeds a threshold, to classify the object as a reactive agent; or determining, at least in part, that the likelihood is below a threshold, to classify the object as an inactive agent, as described in any one of paragraphs N to Q.

[0147] S: The lower limit cost is an estimate of the minimum cost of an action to change the vehicle's first state at the time associated with the start of the first candidate action, and the upper limit cost is determined at least in part on at least one of the costs associated with the second candidate action or the default trajectory, as described in any one of paragraphs N to R in a non-temporal computer-readable medium.

[0148] T: Controlling a vehicle at least in part based on a route is at least in part further based on determining that a first total cost is less than a threshold or less than a second total cost associated with a second route which includes at least one candidate action different from at least one of the first candidate action or the second candidate action, as described in any one of paragraphs N to S.

[0149] While the exemplary clauses described above illustrate one specific implementation, it should be understood that, in the context of this document, the content of those exemplary clauses can also be implemented through methods, devices, systems, computer-readable media, and / or other implementations. In addition, any of Examples A-T may be implemented alone or in combination with any one or more other Examples A-T.

[0150] knot While this subject matter has been described using language specific to structural features and / or methodological actions, it should be understood that the subject matter as defined in the attached claims is not necessarily limited to the specific features or actions described. Rather, those specific features and actions are disclosed as exemplary forms that implement the claims.

[0151] The components described herein correspond to instructions that can be stored in any type of computer-readable medium and implemented in software and / or hardware. All of the methods and processes described above can be embodied in software code components and / or computer-executable instructions executed by one or more computers or processors, hardware, or any combination thereof, and can be fully automated through them. Some or all of these methods can, alternatively, be embodied in dedicated computer hardware.

[0152] At least some of the processes discussed herein are presented as logical flow graphs, where each of their operations represents a sequence of operations that can be implemented in hardware, software, or a combination thereof. In a software context, the operations represent computer-executable instructions stored on one or more non-temporary computer-readable storage media, which, when executed by one or more processors, cause a computer or autonomous vehicle to perform the enumerated operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a particular function or implement a particular abstract data type. The order in which the operations are described is not intended to be interpreted as restrictive, and any number of the described operations can be combined in any order and / or in parallel to implement a process.

[0153] Among the many options, there are "may" and "could". ,Ma Conditional statements such as "might" are understood in context to indicate that certain features, elements, and / or steps are included in certain examples, while other examples do not, unless otherwise specified. Therefore, such conditional statements are not generally intended to mean that certain features, elements, and / or steps are required in some way for one or more examples, or that one or more examples necessarily contain logic, with or without user input or prompting, to determine whether a particular feature, element, and / or step is included in or performed in any of those examples.

[0154] Connecting phrases such as "at least one of X, Y, or Z" should be understood to indicate that an item, term, etc., may include any combination of X, Y, Z, or multiple elements of each of them, unless otherwise specified. Unless explicitly stated, "a" can mean singular or plural.

[0155] Any routine description, element, or block in the flowcharts described herein and / or shown in the accompanying figures should be understood as potentially representing a module, segment, or portion of code containing one or more computer-executable instructions for implementing a particular logical function or element in that routine. The examples described herein include alternative implementations, in which, as will be understood by those skilled in the art, elements or functions may be omitted or executed in a manner deviating from the order shown or discussed, substantially synchronously, in reverse order, with or without further operations, etc. Note that the term "substantially" can indicate a scope. For example, "substantially simultaneously" may mean that two activities occur within each other's time ranges, and "substantially the same dimensions" may mean that two elements have dimensions within each other's time ranges, and so on.

[0156] Many variations and modifications can be made to the above-described examples, and their elements should be understood as being among other acceptable examples. All such modifications and variations are intended to be included in the scope of this disclosure and to be protected by the following claims.

Claims

1. One or more processors, A system comprising a memory storing processor-executable instructions, wherein when a processor-executable instruction is executed by one or more processors, the system provides the following: Receiving route data associated with the start and end positions in the environment, Receiving sensor data from a sensor, Based at least partially on the aforementioned sensor data and route data, a first candidate action for controlling the vehicle's movement is determined. The first cost associated with the first candidate action is determined to be at least partially based on a lower limit cost, the lower limit cost being an estimate of the minimum cost of an action to change the first state of the vehicle at a time associated with the start of the first candidate action. The first prediction is determined based at least in part on the first candidate action and the sensor data, and the first prediction includes determining the first state of an object in the environment at a future time. Based at least partially on the first prediction and the route data, a second candidate action for controlling the vehicle's movement is determined. Determining a second cost associated with the second candidate action, wherein the second cost is at least in part based on an upper limit cost, the upper limit cost being an estimate of the cost of performing the default action. Applying either the lower cost or the upper cost alternately to one or more subsequent candidate actions, A path including the first candidate action and the second candidate action, And, To determine the first total cost associated with the first candidate action and the second candidate action. A system for performing operations that includes controlling the vehicle, at least in part, based on the above.

2. The aforementioned operation is The determination that the object is a reactive object is made by a machine learning model and at least partially based on sensor data, The further includes determining the first prediction, at least in part, based on performing the first simulation using the determination that the object is a reactive object. The system according to claim 1, wherein performing the first simulation using the determination that the object is a reactive object includes determining the movement of the representation of the object, at least in part, based on the first candidate action.

3. The aforementioned operation is The determination that the object is an inactive object is made by a machine learning model and at least partially based on the sensor data, The method further includes determining the first prediction, at least in part, based on modeling the motion of the inert object. The system according to claim 2, wherein modeling the motion of the inert object includes determining the motion of the inert object based at least in part on the state of the object and excluding the first candidate action.

4. The operation involves assigning the first candidate action and the second candidate action to the same prediction node in the data structure, wherein the first prediction and the second prediction are Determining a first set of candidate actions based at least partially on the first prediction, and a second set of candidate actions based at least partially on the second prediction, Determining a first action from the first set associated with the first minimum cost among the costs associated with the first set, Determining a second action from the second set associated with the second minimum cost among the costs associated with the first set, It is determined that the first action and the second action are associated with positions that are within a distance of each other that is less than a threshold distance, The system according to claim 3, further comprising assigning, at least in part, that is associated with the same prediction layer as the first prediction.

5. The aforementioned operation is Based at least partially on the sensor data, it is determined that the object may modify its behavior in response to one or more of the first candidate actions or the second candidate actions. To decide to classify the object as a reactive agent, at least in part on the fact that the aforementioned possibility meets or exceeds a threshold, The system according to claim 4, further comprising deciding to classify the object as an inactive agent, at least in part on the fact that the probability is below the threshold.

6. The system according to claim 5, wherein the lower limit cost is zero, and the upper limit cost is determined at least in part on at least one of the costs associated with the second candidate action or the default trajectory.

7. The system according to claim 6, wherein controlling the vehicle at least in part on the route is at least in part further on determining that the first total cost is less than a threshold or less than a second total cost associated with a second route which includes at least one candidate action different from at least one of the first candidate action or the second candidate action.

8. A non-temporary computer-readable medium storing processor-executable instructions, wherein when a processor-executable instruction is executed by one or more processors, the one or more processors... Receiving route data associated with the start and end positions in the environment, Receiving sensor data from a sensor, Based at least partially on the aforementioned sensor data and route data, a first candidate action for controlling the vehicle's movement is determined. The first cost associated with the first candidate action is determined to be at least partially based on a lower cost or an upper cost. The first prediction is determined based at least in part on the first candidate action and the sensor data, and the first prediction includes determining the first state of an object in the environment at a future time. Based at least partially on the first prediction and the route data, a second candidate action for controlling the vehicle's movement is determined. Determining a second cost associated with the second candidate action, wherein the second cost is determined to be at least in part based on the lower limit cost or the upper limit cost. Applying either the lower cost or the upper cost alternately to one or more subsequent candidate actions, A path including the first candidate action and the second candidate action, Based at least in part on determining the first total cost associated with the first candidate action and the second candidate action, A non-temporary computer-readable medium that enables the execution of operations including controlling the aforementioned vehicle.

9. The aforementioned operation is The machine learning model, and at least partially based on the sensor data, determines that the object is a reactive object. The further includes determining the first prediction, at least in part, based on performing the first simulation using the determination that the object is a reactive object. The non-temporary computer-readable medium according to claim 8, wherein performing the first simulation using the determination that the object is a reactive object includes determining the movement of the representation of the object, at least in part, based on the first candidate action.

10. The aforementioned operation is The determination that the object is an inactive object is made by a machine learning model and at least partially based on the sensor data, The method further includes determining the first prediction, at least in part, based on modeling the motion of the inert object. The non-temporary computer-readable medium according to claim 9, wherein modeling the motion of the inert object includes determining the motion of the inert object based at least in part on the state of the object and excluding the first candidate action.

11. The operation involves assigning the first candidate action and the second candidate action to the same prediction node in the data structure, wherein the first prediction and the second prediction are Determining a first set of candidate actions based at least partially on the first prediction, and a second set of candidate actions based at least partially on the second prediction, Determining a first action from the first set associated with the first minimum cost among the costs associated with the first set, Determining a second action from the second set associated with the second minimum cost among the costs associated with the first set, It is determined that the first action and the second action are associated with positions that are within a distance of each other that is less than a threshold distance, A non-temporary computer-readable medium according to claim 10, further comprising assigning to the same prediction layer as the first prediction, at least in part, based on the above.

12. The aforementioned operation is Based at least partially on the sensor data, it is determined that the object may modify its behavior in response to one or more of the first candidate actions or the second candidate actions. To decide to classify the object as a reactive agent, at least in part on the fact that the aforementioned possibility meets or exceeds a threshold, The non-temporary computer-readable medium according to claim 11, further comprising deciding to classify the object as an inactive agent, at least in part on the fact that the probability is below the threshold.

13. The lower limit cost is an estimate of the minimum cost of an action to change the first state of the vehicle at a time associated with the start of the first candidate action. The non-temporary computer-readable medium according to claim 12, wherein the upper limit cost is determined at least in part on at least one of the costs associated with the second candidate action or the default trajectory.

14. The non-temporary computer-readable medium according to claim 13, wherein controlling the vehicle at least in part on the aforementioned route is at least in part further on determining that the first total cost is less than a threshold or less than a second total cost associated with a second route which includes at least one candidate action different from at least one of the first candidate action or the second candidate action.

15. The non-temporary computer-readable medium according to claim 14, wherein the lower limit cost is zero and the upper limit cost is determined at least in part on at least one of the costs associated with the second candidate action or the default trajectory.

Images