Detecting a blocked stationary vehicle

Abstract

A method and system for determining whether a stationary vehicle is a congested vehicle to improve control of an autonomous vehicle. A perception engine can detect stationary vehicles in the environment of the autonomous vehicle using sensor data received from the vehicle. In response to this detection, the perception engine can determine feature values ​​of the vehicle environment (e.g., features of the stationary vehicle, other objects, the environment itself) based on the sensor data. The autonomous vehicle can then input these feature values ​​into a machine learning model to determine the probability that the stationary vehicle is a congested vehicle and use this probability to generate a trajectory to control the movement of the autonomous vehicle.

Description

[0001] This application is a divisional application of the invention patent application filed on February 1, 2019, with application number 201980013149.4 and invention title "Detection of Stationary Vehicles in Distress".

[0002] Cross-reference to related applications

[0003] This PCT international application claims priority to U.S. Patent Application 15 / 897,028, filed February 14, 2018, which is incorporated herein by reference. Background Technology

[0004] Stationary objects, such as vehicles on the road, can interfere with the autonomous operation of a vehicle. For example, stationary vehicles in front of an autonomous vehicle could be parked side-by-side or otherwise malfunctioning, thus blocking the autonomous vehicle. Detecting such obstructing vehicles can be limited by sensor visibility, as it may be impossible to "see" in front of a potential obstructing vehicle to determine if it is indeed obstructing.

[0005] Furthermore, environmental cues can increase the complexity of detection. As two examples, a vehicle stopped at an intersection with a detected red light might be obstructing traffic and not actually waiting for the light. Similarly, a vehicle stopped near an intersection with a detected green light might actually be in a long queue waiting to turn, rather than obstructing traffic. Automated operation based on improperly detected obstructed vehicles could create other problems, such as the autonomous vehicle failing to re-enter its original lane. Attached Figure Description

[0006] The accompanying drawings are described in detail. In the drawings, the leftmost numeral of the reference numeral indicates the drawing in which that reference numeral first appears. The same reference numerals in different drawings indicate similar or identical items.

[0007] Figure 1A An example scenario involving a stationary vehicle is shown, which could be a vehicle that is blocking traffic.

[0008] Figure 1B Show depiction Figure 1A An example of a view of an autonomous vehicle in a scene, to show what can be discerned from sensor data collected from an autonomous vehicle. Figure 1A Part of the scene.

[0009] Figure 2 A graphical flowchart illustrating an example process for detecting blocked vehicles is shown.

[0010] Figures 3A-3F Examples of sensor data and features derived from it for detecting congested vehicles are shown.

[0011] Figure 4 A block diagram of an example architecture for detecting congested vehicles, including an example vehicle system, is shown.

[0012] Figure 5 A flowchart illustrating an example process for training a machine learning model for congested vehicles, based on the techniques discussed in this paper.

[0013] Figure 6 A schematic flowchart illustrating an example process for using a machine learning model to detect congested vehicles, based on the techniques discussed in this paper.

[0014] Figure 7 A flowchart illustrating an example process for training a machine learning model for congested vehicles, based on the techniques discussed in this paper. Detailed Implementation

[0015] As described above, obstructing objects (including vehicles) include objects that prevent autonomous vehicles from proceeding along a planned route or path. For example, side-by-side parking is a common practice in many urban environments. Such side-by-side parked vehicles need to be detected as obstructing vehicles to be handled separately from stationary vehicles. Specifically, while an autonomous vehicle can be instructed to wait for a stopped vehicle to move, it can also be instructed to drive around such side-by-side parked vehicles. General rules, such as treating all stationary vehicles at a green light as obstructions, are generally inaccurate and / or insufficient for safely operating autonomous vehicles and / or operating them in a manner that more closely mimics human operation. This disclosure generally addresses techniques (e.g., machines, programs, processes) for determining whether a stationary vehicle is an obstructing vehicle or an object in order to control an autonomous vehicle based on that determination. In some examples, the techniques discussed herein include machine learning (ML) models configured to receive sensor data to determine whether a stationary vehicle is an obstructing vehicle. Instead of making this determination based on conditional rules (e.g., indicating that a vehicle is blocked if it is indeed a green light and the vehicle is indeed stopped), the technique discussed in this paper can use an ML model with sensor data as input to determine the probability that a stationary vehicle is a blocked vehicle.

[0016] In some examples, the techniques discussed herein may include: receiving raw and / or processed sensor data (e.g., sensor data processed by another machine learning model and / or software / hardware module of the autonomous vehicle) from one or more sensors and / or software and / or hardware modules of the autonomous vehicle; determining that a vehicle in the autonomous vehicle's path is stationary; determining feature values ​​from the sensor data (e.g., values ​​indicating features such as distance to the next intersection in the road; distance from the stationary vehicle to the next vehicle in front of it; the stationary vehicle's speed, brake light condition, height, size, and / or yaw; classification of the stationary vehicle and / or objects (one or more) near it; traffic flow data); and determining the probability that the stationary vehicle is a congested vehicle. In some examples, an ML model may be used to determine the probability.

[0017] In some examples, an autonomous vehicle may include a perception engine and / or a planner for controlling the autonomous vehicle. The perception engine may include one or more ML models and / or other computer-executable instructions for detecting, identifying, classifying, and / or tracking objects using sensor data collected from the autonomous vehicle's environment. In some examples, the perception engine may include an ML model (hereinafter referred to as a "BV model") configured to determine the probability that a stationary vehicle is a blocking vehicle, although the general discussion of ML models also applies to BV models. The planner may include one or more ML models, algorithms, etc., for route planning, trajectory planning, evaluation decisions, etc. The planner may be configured to generate a trajectory for controlling the movement of the autonomous vehicle, based on data received from other components of the autonomous vehicle, such as the perception engine and sensors (one or more).

[0018] The techniques discussed in this paper improve the operation of autonomous vehicles by increasing the accuracy of detection compared to previous solutions (e.g., using conditional rules). In practice, these techniques prevent autonomous vehicles from unnecessarily stopping behind obstructing vehicles, thus reducing power consumption and wasted computational cycles. When the stationary vehicle is a non-obstructing stationary vehicle, the techniques also prevent the autonomous vehicle from unnecessarily changing lanes or routes. This similarly reduces power consumption and wasted computational cycles. These techniques can also improve the safety of autonomous vehicle operation for autonomous passengers and / or environmental entities by more accurately perceiving situations. For example, these techniques can prevent autonomous vehicles from: changing lanes and only having to return to their original lane occupied by a row of cars previously undetected by the autonomous vehicle (e.g., potentially increasing the risk of a collision); encountering objects for which non-obstructing stationary vehicles have already stopped; anticipating that the doors (one or more) of a non-obstructing stationary vehicle may open and avoiding collisions with the doors, etc.

[0019] As used herein, a blocking vehicle is a stationary vehicle on a drivable surface that impedes the progress of other vehicles to some extent. Not all stationary vehicles on a drivable surface are blocking vehicles. For example, a non-blocking stationary vehicle can be a vehicle that has stopped moving forward on a drivable road due to a traffic light signaling a red light, yielding to another vehicle, and / or waiting for another vehicle to proceed. Conversely, a blocking vehicle can be a vehicle parked side-by-side, a delivery truck parked for transport, a vehicle without a driver, a stopped police car, a malfunctioning vehicle, etc. The difference between a non-blocking stationary vehicle and a blocking vehicle can be ambiguous and difficult to determine, as in most cases the difference may simply be a matter of the length of time a stationary vehicle has stopped. However, the key difference between a stationary vehicle and a blocking vehicle is that the movement of a vehicle would be possible and / or permitted unless a blocking vehicle is present. In some examples, a blocking vehicle can be classified as a stationary vehicle that does not comply with accepted road rules (such as traveling along a lane). The techniques discussed in this paper improve the operation of autonomous vehicles by preventing autonomous vehicles from stopping behind blocking vehicles for unsustainable periods of time.

[0020] Example Scenario

[0021] Figure 1AThis is a schematic bird's-eye view of Example Scenario 100, illustrating one of many instances to which the techniques discussed herein can be applied. In the example scenario, an autonomous vehicle 102 approaches a road intersection 104, which includes traffic lights 106. In some examples, the autonomous vehicle 102 may be an autonomous vehicle configured to operate according to a Level 5 classification issued by the National Highway Traffic Safety Administration (NHTSA), which describes a vehicle capable of performing all safety-critical functions throughout the journey, without the driver (or occupant) expected to control the vehicle at any time. However, in other examples, the autonomous vehicle 102 may be a fully or partially autonomous vehicle with any other level or category, now existing or to be developed in the future. Furthermore, in some examples, the techniques described herein for identifying obstructing vehicles may also be used by non-autonomous vehicles.

[0022] Autonomous vehicle 102 may receive sensor data from one or more sensors of autonomous vehicle 102. Autonomous vehicle 102 may use this sensor data to determine a trajectory for controlling the movement of the autonomous vehicle. In some examples, autonomous vehicle 102 may also include a planner and / or perception engine that receives data from the sensors, among other hardware and software modules. For example, this data may include: the position of autonomous vehicle 102 determined by Global Positioning System (GPS) sensors, data relating to objects near autonomous vehicle 102, route data specifying the vehicle's destination, global map data identifying road features (e.g., features detectable in different sensor modes that can be used to locate the autonomous vehicle), local map data identifying features detected near the vehicle (e.g., the location and / or size of buildings, trees, fences, fire hydrants, parking signs, and any other features detectable in various sensor modes), and so on. The planner may use this data to generate a trajectory for controlling the movement of the vehicle. In some examples, autonomous vehicle 102 may include a perception engine that receives sensor data from one or more sensors of vehicle 102, determines perception data from the sensor data, and transmits the perception data to the planner for use in localization. The location of the autonomous vehicle 102 is determined on a global map, one or more trajectories are identified, and the movement of the autonomous vehicle 102 is controlled to traverse the path or route. For example, the planner can determine the route of the autonomous vehicle 102 from a first location to a second location, generate potential trajectories for controlling the movement of the autonomous vehicle 102 within a time window (e.g., 1 microsecond, half a second) to control the vehicle to traverse the route, and select one of the potential trajectories as the trajectory of the autonomous vehicle 102. The trajectory can be used to generate drive control signals that can be transmitted to the drive components of the autonomous vehicle 102.

[0023] The perception engine may include one or more ML models and / or other computer-executable instructions for detecting, identifying, segmenting, classifying, and / or tracking objects using sensor data collected from the environment of the autonomous vehicle 102. For example, the perception engine may detect objects in the environment and classify said objects (e.g., passenger cars, semi-trucks, pickup trucks, people, children, dogs, balls). The perception engine may also determine the trajectory of objects (e.g., the object's history, current and / or predicted direction of travel, position, speed, and / or acceleration).

[0024] The perception engine can determine whether a vehicle is stationary based at least in part on sensor data received from one or more sensors of the autonomous vehicle. In some examples, the perception engine can receive sensor data and, at least in part on the sensor data, detect objects in the environment of the autonomous vehicle 102, classify the objects as a certain type of vehicle, and determine that the sensor data indicates the speed of the detected vehicle does not exceed a preset threshold speed (e.g., the sensed speed of the vehicle is not greater than or equal to 0.1 m / s or 0.05 m / s). As used herein, a vehicle can be, for example, but not limited to, physical means of transport, such as passenger cars, delivery trucks, bicycles, drones used for transporting objects, etc. In some examples, the perception engine can classify a detected vehicle as stationary based on determining that the speed of a stationary vehicle does not meet (e.g., meets, exceeds) a preset threshold speed and / or another condition is met, such as duration, traffic light status, sensing distance to an intersection meeting a preset threshold distance, etc. For example, the perception engine can classify a detected vehicle as a stationary vehicle based at least in part on received sensor data, which is determined to indicate that the detected vehicle is traveling at a speed below a preset threshold rate (e.g., the vehicle stops) and that the vehicle is traveling at a speed below the preset threshold rate for a preset period of time (e.g., the vehicle stops for 20 seconds), while traffic lights indicate a green light and / or a distance of 100 meters from the intersection.

[0025] return Figure 1AIn the example scenario, autonomous vehicle 102 may approach intersection 104, which includes traffic lights 106, and may encounter an object classified as a stationary vehicle (e.g., vehicle 108), also indicated by a question mark in the diagram. A technical challenge arises when the perception engine lacks sufficient information to determine whether a stationary vehicle is merely paused (i.e., a non-blocking stationary vehicle), whether it is obstructed by another object and / or legal constraints (e.g., a stop light), or whether it is actually a blocking vehicle. As used herein, a blocking vehicle is a vehicle stopped or moving on a drivable surface at a rate less than a preset threshold speed that impedes the progress of other vehicles. For example, a blocking vehicle might be a vehicle parked side-by-side, a delivery truck unloading cargo, a vehicle where the driver is absent or waiting to pick up passengers, a stopped police car, a malfunctioning vehicle (e.g., a vehicle with a faulty drive system, a vehicle with a flat tire, a vehicle involved in an accident, a vehicle whose occupants have left the vehicle), or a meter-reading patrol vehicle. As used herein, “drivable surface” can include those parts of a road associated with regulated driving conditions, as opposed to non-drivable surfaces such as shoulders, parking lanes, and / or bicycle lanes.

[0026] The difference between a congested vehicle and a non-congested stationary vehicle is that a non-congested stationary vehicle will stop when no other vehicle can proceed under any circumstances (or according to generally accepted driving rules), and therefore, a non-congested stationary vehicle itself does not obstruct other vehicles (e.g., a traffic light is red, an object has entered the road, and there is another vehicle in front of the stationary vehicle). As described herein, a congested vehicle can differ from a non-congested stationary vehicle because a congested vehicle obstructs the trajectory determined by the autonomous vehicle 102 (e.g., the congested vehicle occupies at least a portion of the same lane as the autonomous vehicle 102, and / or coincides with the path and / or trajectory of the autonomous vehicle 102).

[0027] Without resolving the ambiguity of whether the detected vehicle is a non-blocking stationary vehicle or a blocked vehicle, the perception engine may not be able to provide the planner with enough data to enable the planner to generate a trajectory suitable for controlling the motion of the autonomous vehicle 102 in a given scenario.

[0028] Figure 1B Show Figure 1A A bird's-eye view of the same example scenario 100, and reflecting further complexity of example scenario 100. Specifically, Figure 1BThis is a limited and imperfect view of the scene that can be used by the autonomous vehicle 102 via sensor data. Future advancements in sensors and perception may increase the portion of the scene reflected in the sensor data, but it is still likely, at least at some point, that the autonomous vehicle 102 will not be informed of 100% of the states, objects, etc., in or affecting the scene. Therefore, Figure 1B An example portion of scenario 100 is shown, reflected by sensor data received by the planner of autonomous vehicle 102. For example, global map data and GPS location received from sensors of autonomous vehicle 102 may indicate that intersection 104 is 100 meters ahead of autonomous vehicle 102; sensor data may indicate that traffic light 106 is green, and in conjunction with global map data, LiDAR data, and / or camera data (although any other form of sensor may be considered), the perception engine may confirm that autonomous vehicle 102 is in the lane authorized to enter intersection 104 based on the green light; autonomous vehicle 102 may receive sensor data from which it determines that vehicle 108, as well as vehicles 110 and 112, are stationary vehicles.

[0029] Due to limitations of sensors (one or more), autonomous vehicle 102 may not receive sensor data to detect the presence of vehicles 114-122. For example, vehicles 114-122 may be outside the sensor's field of view, beyond the sensor's reliable operating distance, or obstructed. In some examples, the perception engine may include conditional rules that specify the conditions for its output of a blocked vehicle indication. In some examples, the conditional rules may be pre-hardcoded by a programmer or administrator. For example, a hardcoded rule may specify that the perception engine outputs a blocked vehicle indication if: the detected vehicle has been classified as a stationary vehicle; a green light is detected and remains green for a preset time period, the stationary vehicle does not move during the preset time period; and the stationary vehicle is more than 15 meters from intersection 104.

[0030] However, such hard-coded rules may not correctly address many potential scenarios. For example, the exemplary hard-coded rules given above might apply to an example where a stationary vehicle 108 has lost functionality and is therefore a blocked vehicle 108. However, as... Figure 1AAs shown, there may actually be a long line of cars in front of the stationary vehicle 108, which the autonomous vehicle 102 may not be able to detect. Therefore, providing a congested vehicle indication to the planner, such as in scenario 100, could result in erroneous or troublesome operation by the autonomous vehicle 102. For example, the planner could initiate a lane change, assuming that the autonomous vehicle 102 can return to the same lane after passing the stationary vehicle 108, which is indicated as a congested vehicle by the perception engine. Due to the long line of cars in front of the stationary vehicle, the autonomous vehicle 102 may not be able to return to its original lane. Therefore, classifying the stationary vehicle 108 as a congested vehicle in this case would be a false affirmation. False negations could similarly interfere with the operation of the autonomous vehicle 102. Hard-coded rules could therefore generate an unacceptable ratio of false affirmations and / or false negations.

[0031] Furthermore, introducing new features into a set of hard-coded rules (e.g., checking if the perception engine detects that a hazard warning light on a stationary vehicle is flashing) would be time-consuming and require manual coding and / or reconfiguration of the perception engine. This is undesirable because such reconfiguration might require halting the operation of the autonomous vehicle 102 and / or waiting for human reconfiguration to complete, which could necessitate multiple iterations of developing and testing new reconfigurations of the perception engine.

[0032] In some examples, the techniques discussed in this paper can be used instead to probabilistically determine whether a stationary vehicle is a congested vehicle using ML models, which are not limited to this. This paper describes the method in conjunction with the BV ML model.

[0033] Example process

[0034] Figure 2 A graphical flowchart of example process 200 is shown, which is used to determine whether a stationary vehicle is a blocking vehicle at an autonomous vehicle in order to control the autonomous vehicle. At operation 202, example process 200 may include receiving sensor data from one or more sensors of the autonomous vehicle 204. In some examples, sensor data may be received additionally or alternatively from remote sensors, such as sensors of another vehicle, sensors of a remote computing device (e.g., remote operations services, weather stations, traffic control services, emergency services), and / or sensors located in the infrastructure (e.g., sensors located on lampposts, buildings).

[0035] At operation 206, example process 200 may include detecting a stationary vehicle 208 from sensor data. This may include detecting an object, classifying the object as a vehicle, and determining that the vehicle's rate (or speed) is less than a threshold rate (or speed). In some examples, the threshold may be a preset threshold (e.g., the vehicle is stopped or moving at a speed less than 0.05 meters per second), while in other examples, the threshold may be relative (e.g., the vehicle is moving at less than 20% of the average traffic rate). In some examples, operation 206 may additionally or alternatively include determining that other characteristics determined from the sensor data satisfy one or more conditions, such as the vehicle being at a certain distance from the intersection, the vehicle being a specific type (e.g., a bicycle, a passenger car), or a traffic light being green.

[0036] Operation 206 may include detecting all stationary vehicles within the "field of view" of the sensors of the autonomous vehicle 204 (i.e., those vehicles within the field of view of one or more sensors). In additional or alternative examples, operation 206 may include detecting stationary vehicles within a preset threshold distance. For example, the autonomous vehicle 204 may detect stationary vehicles within 50 meters. By limiting the range at which the autonomous vehicle 204 can detect stationary vehicles, the autonomous vehicle 204 saves processing and storage resources. Furthermore, by detecting not only whether stationary vehicles are in the same lane as the autonomous vehicle 204, the planner of the autonomous vehicle 204 is able to make more complex decisions regarding generating and / or selecting a trajectory to control the autonomous vehicle 204.

[0037] At operation 208, example process 200 may include determining feature value 210 based at least in part on sensor data. In some examples, feature value 210 may correspond to a feature specified by a Blocked Vehicle (BV) ML model. The BV ML model may be configured to determine the probability that a stationary vehicle 208 is a blocked vehicle based on feature value 210 determined by the perception engine of the autonomous vehicle 204. In some examples, the autonomous vehicle 204 may attempt to determine feature value 210 for at least a subset of possible features for which the BV ML model is configured. For example, the following shows an example of feature value 210 determined by the perception engine, which corresponds to a feature that the BV ML relies on to determine the probability that a stationary vehicle 208 is a blocked vehicle. In a given example, some feature values ​​210 may not be determined by the perception engine, or may be inapplicable, or may not be available from the sensor data (i.e., "blocked by another object", "behavior of another object"). Figures 3A-3F Some of these features and others are discussed.

[0038] feature Eigenvalues Distance from the intersection 14.3 meters Brake lights on 0 SV speed 001m / s Traffic flow normality .87 Blocked by another object - People near the vehicle 0 Other object behaviors - Proxy type Passenger vehicles

[0039] Although several example features are listed above (and relative to) Figures 3A-3F However, the number and type of features are not limited to this. For example, any number of features can be considered, such as object bounding box size, object color, object height, object size, object width, object velocity (i.e., speed and / or direction), object yaw (e.g., orientation relative to a vehicle), lane markings (e.g., left, center, right), GPS location, detected logos and / or text associated with stationary vehicles, etc. Any feature can be represented by a simple Boolean value (i.e., whether the feature exists), a real number (e.g., detected speed), text (e.g., classification), or any other data representation.

[0040] In some examples, the feature values ​​210 determined by the perception engine may include: the speed of the stationary vehicle 208; traffic signal status (e.g., traffic light status, presence of traffic signs); traffic flow data; the relationship between the stationary vehicle data and the traffic flow data (e.g., whether the stationary vehicle data is within a normal distribution of the traffic flow data); the probability that the stationary vehicle 208 is blocked by another object (e.g., based on residuals in the detection sensor data, which may indicate the presence of an object in front of the stationary vehicle); and the probability of the presence of an occluding object, which may be indicated by sensor residuals (e.g., noise radar data). Sensor data (which can indicate objects in front of the stationary vehicle 208); people detected near the stationary vehicle 208 and related data (e.g., how close a person is to the stationary vehicle, whether the person leaves and returns to the stationary vehicle 208); the behavior of other objects (e.g., other vehicles that go around the stationary vehicle 208, other vehicles that do not go around the stationary vehicle 208); classification labels (types) of the stationary vehicle 208 (e.g., police car, passenger car, delivery truck) and / or other objects in the environment; door opening / closing status (e.g., the rear door or a hole of the stationary vehicle 208 is opened / closed); and so on.

[0041] At operation 212, example procedure 200 may include using BV ML model 214 and determining the probability that a stationary vehicle 208 is a blocked vehicle based on feature values. For example, feature values ​​210 may be input into BV ML model 214, and BV ML model 214 may respond to and output the probability that a stationary vehicle 208 is a blocked vehicle based on the configuration of BV ML model 214. In some examples, BV ML model 214 may include a decision tree or any arrangement thereof, such as a random forest and / or an augmented ensemble of decision trees; a directed acyclic graph (DAG) (e.g., in which nodes are organized as a Bayesian network); one or more deep learning algorithms, such as an artificial neural network (ANN), a deep belief network (DBN), a deep stacked network (DSN), or a recurrent neural network (RNN); and so on. In some examples, BV ML model 214 may include a gradient-boosted decision tree.

[0042] For example, in the case where the BV ML model 214 includes decision trees, the decision trees can output positive or negative numbers to indicate whether the stationary vehicle 208 is a blocked vehicle or not. In some examples, the BV ML model 214 may include multiple decision trees that can be weighted. For example, one decision tree can be weighted to output -1.63 or +1.63, and a second decision tree can be weighted to output -0.76 or +0.76. Once all decision trees have output numbers, the perception engine can sum their outputs to determine the total probability that the stationary vehicle 208 is a blocked vehicle.

[0043] In the case where the BV ML model 214 includes a neural network, the BV ML model 214 may include an input layer for a node, one or more hidden layers for the node, and an output layer for the node. In some examples, the input layer of a node may be configured to receive one or more feature values ​​and activate nodes in one or more hidden layers. The output layer may be configured to receive stimuli from nodes in one or more hidden layers and output an indication based on the most activated node in the output layer. In some examples, the output layer may include two nodes (a positive indication that a stationary vehicle is a blocked vehicle, and a negative indication), and / or the output layer may provide an output between a strong confidence that a stationary vehicle is a blocked vehicle and a strong confidence that a stationary vehicle is not a blocked vehicle. In some examples, the decision tree may be a classification tree, in which the output includes whether the detected vehicle is a blocked vehicle.

[0044] In some examples, a BV ML model 214 can be generated (learned) from labeled feature data. For example, the BV ML model 214 may include a deep learning model that learns the probability that a stationary vehicle 208 is a blocked vehicle based on the feature values ​​of the input samples associated with the label, which indicates whether the sample feature value comes from a scenario where the stationary vehicle 208 is a blocked vehicle (i.e., a ground reality label). Figure 2 BV ML 214 is shown as a decision tree, where the shaded nodes are the nodes reached by the input feature values. The decision tree shown can output a weighted probability of 0.623 that stationary vehicle 208 is a blocked vehicle. In some examples, a positive value can indicate that stationary vehicle 208 is a blocked vehicle, and a negative value can indicate that stationary vehicle 208 is not a blocked vehicle, but any sign value (e.g., the opposite) can be considered.

[0045] At operation 216, example procedure 200 may include transmitting probabilities determined by the perception engine to a planner, which determines a trajectory for controlling the autonomous vehicle, as depicted at 218 and 220. The planner may use the probability indicating that the stationary vehicle 208 is a blocked vehicle to generate a trajectory for controlling the autonomous vehicle 204. For example, Figure 2 Two example trajectories that the planner can determine in alternative scenarios are shown.

[0046] In example scenario 218, the perception engine may have output an indication that the stationary vehicle 208 is a congested vehicle. In response to receiving this indication, the planner for the autonomous vehicle 204 can generate a trajectory 222 that allows the autonomous vehicle 204 to merge into another lane. In some examples, the perception engine may additionally provide the planner with a classification of the congested vehicle (e.g., police car, meter reading vehicle, delivery vehicle). In some examples, the classification may be a semantic label. This semantic label may be included as one of the feature values.

[0047] In example scenario 220, the perception engine may have output an indication that the stationary vehicle 208 is not a blocking vehicle (i.e., the stationary vehicle 208 is a non-blocking stationary vehicle). In response to receiving this indication, the planner of the autonomous vehicle 204 can generate a trajectory 224 that causes the autonomous vehicle 204 to move slowly forward in the same lane.

[0048] In some examples, the perception engine and / or planner may determine to transmit signals to a remote computing device to receive remote assistance. For example, the autonomous vehicle 204 may transmit signals to a remote computing device, which may have greater computing power, enabling the remote computing device to determine a probability or indication for a human remote operator to input whether the stationary vehicle 208 is a blocked vehicle. If the probability does not meet a threshold of negative or positive probability, the planner may transmit signals to the remote computing device indicating a low confidence level that the stationary vehicle 208 is or is not a blocked vehicle (e.g., the threshold could be 0.25 to limit the probability to be too low to rely on for trajectory generation). In some examples, the autonomous vehicle 204 may also transmit sensor data and / or feature values ​​to the remote computing device.

[0049] Example features

[0050] Figures 3A-3F By depicting various features, the BV ML model can be configured to generate the probability that a stationary vehicle is or is not a blocked vehicle. The feature values ​​discussed below can be determined by the perception engine from sensor data. Furthermore, the operations discussed below can be used as... Figure 2 This is implemented as part of operation 212.

[0051] Figure 3AExample scenario 300 is depicted, in which an autonomous vehicle 302 approaches a stationary vehicle 304 and an intersection 306 including a traffic signal 308 (a traffic light in this example). The autonomous vehicle 302 may correspond to the autonomous vehicle 204 and related discussions. According to any of the techniques discussed herein, the autonomous vehicle 302 can detect that vehicle 304 is a stationary vehicle via a perception engine. In response to the detected stationary vehicle 304, the autonomous vehicle 302 can determine feature values ​​from sensor data via the perception engine. For example, the autonomous vehicle 302 may determine: the status of the lights of the stationary vehicle 304 (e.g., hazard warning lights on / off, brake lights on / off), as shown at 310; the distance 312 between the stationary vehicle 304 (and / or the autonomous vehicle) and the intersection 306 and / or the distance from the stationary vehicle 304 to the next vehicle in front of the stationary vehicle (e.g., which may be determined in the case of a road bend, or where sensor residuals are available that can indicate the presence of a vehicle in front of the stationary vehicle 304 (e.g., radar reflections), etc.); and / or the status of traffic signals 308, which in some examples may include the presence or absence of traffic lights and / or traffic signs (e.g., green light, yellow light, red light, presence of stop sign, presence of yield sign, no detected sign, no detected stop sign). In some examples, the autonomous vehicle 302 may determine the height and / or size of the stationary vehicle 304. In some cases, the height of an autonomous vehicle can be determined using a BV ML model to indicate the likelihood that a stationary vehicle (304) is a blocking vehicle (e.g., semi-trucks and delivery trucks tend to be higher and tend to stop for longer periods in a way that obstructs part of the road).

[0052] In some examples, the perception engine may additionally or alternatively identify logos and / or text associated with stationary vehicle 304 as one of the feature values. For example, the planner's machine learning algorithm may determine that stationary vehicle 304 is associated with a pizza delivery sign (e.g., on the top of the vehicle), a taxi service sign, text and / or logo (e.g., UPS text and / or logo, Uber text and / or logo). In some examples, the text and / or logo may be categorized or used to enhance the confidence of the classification of stationary vehicle 304 as a type of delivery vehicle (e.g., a transport vehicle, a public transport vehicle).

[0053] In some examples, the perception engine may additionally or alternatively determine traffic flow data from sensor data. Traffic flow data may include data for additional objects detected by the perception engine and classified as vehicles (i.e., vehicles 314, 316, and 318). This data may include vehicle speed 320, distance 322 between a vehicle and the next and / or previous vehicle, etc.

[0054] For example, Figure 3B Example distribution 324 of traffic flow data is shown. In some examples, such a distribution can reflect the frequency of various detected feature values ​​related to other objects (e.g., as a distribution histogram). This can include the rate distribution of other vehicles determined by the perception engine. Example distribution 324 shows that the perception engine has determined that most of the detected vehicles are moving at speeds between approximately 55 km / h and 110 km / h. Although in Figure 3B The rates are depicted and discussed here, but it should be understood that any other characteristic value that is unique to each vehicle can also be represented in the distribution. For example, the distance between vehicles, the opening / closing of doors, the proximity of pedestrian vehicles, traffic light indications, etc., may be unique to each vehicle (or vehicle's lane), while some traffic light indications, etc., may not be unique. For clarity, when referring to the frequency distribution of other vehicle characteristics, these characteristics are referred to as "traffic flow data" in this paper.

[0055] The perception engine can output feature values ​​using the frequency distribution of traffic flow data associated with the detected vehicle, which include at least one of the following: percentile of the traffic flow data associated with stationary vehicle 304; distribution characteristics (e.g., whether the distribution includes a long tail; whether the long tail is eliminated by reducing the traffic flow data reflected in the distribution to vehicles in a specific lane(s); the width of the distribution; the height of the distribution); whether the traffic flow data associated with stationary vehicle 304 is within the tail; and so on. For example, Figure 3B Thresholds 326 and 328 are defined, which can indicate quartile positions and / or percentiles (e.g., the 5th and 95th percentiles, respectively). The perception engine can use these thresholds 326 and / or 328 to indicate whether the rate and / or another characteristic associated with the stationary vehicle 304 is within the main body of the distribution (of characteristic values ​​of other observed vehicles) or in the tail defined by thresholds 326 and / or 328. For example, the perception engine can determine whether the rate and / or another characteristic associated with the stationary vehicle 304 is within or outside two standard deviations of the mean of a Gaussian distribution. Regardless of the method used, if the perception engine determines that the rate and / or other characteristic value is outside the normal range (as described above), the perception engine can determine that the rate and / or other characteristic value 304 of the stationary vehicle is anomalous.

[0056] It should be understood that the perception engine can additionally use traffic flow data to identify the corresponding data for stationary vehicle 304 as anomalous. Note that while stationary vehicle 304 is more likely to fall into the lower tail section, in some examples, the higher tail section can be used for other purposes, such as identifying unstable vehicles.

[0057] In some examples, the perception data can generate traffic flow data for vehicles of the same type or general classification as the stationary vehicle. For instance, if stationary vehicle 304 has been classified as a bicycle, the perception engine can generate traffic flow data for other bicycles that the perception engine has already detected. In another example, if stationary vehicle 304 has been classified as a passenger vehicle, the perception engine can generate traffic flow data for objects classified as vehicles, passenger vehicles, and / or motor vehicles.

[0058] Figure 3C Additional or alternative features 330 are depicted for which the perception engine can determine feature values. The perception engine of the autonomous vehicle 302 can generate feature values ​​indicating the behavior of other objects relative to the stationary vehicle 304. For example, the perception engine can store the tracks 334 of another vehicle, such as vehicle 332, and can output said tracks as feature values.

[0059] In additional or alternative examples, the perception engine can output indications classifying tracks as feature values. For example, this indication could include that vehicle 332 is changing lanes, remaining stationary, etc. Similar to traffic flow data, the perception engine can convey the frequency of repeated behaviors by other objects. In some examples, this frequency can be limited to objects in the same lane as autonomous vehicle 302, or it can be more heavily weighted towards objects exhibiting behaviors originating from the lane of autonomous vehicle 302. For example, vehicles 336 and 338 might be parked vehicles. Since vehicles 336 and 338 are in different lanes, constraining the determination of the frequency of behaviors exhibited by other vehicles to the same lane as autonomous vehicle 302 can improve the accuracy of the trajectory generated by the planner in response to the received determined frequency. For example, if the determination is not limited to the same lane as autonomous vehicle 302, the feature value could indicate that two vehicles (66% of exhibiting vehicle behaviors) remain stationary and one vehicle (33% of exhibiting vehicle behaviors) has passed the stationary vehicle. However, by determining the frequency of behavior constrained to the same lane as the autonomous vehicle 302, this characteristic value can indicate that one vehicle (exhibiting 100% of the vehicle behavior) has passed the stationary vehicle.

[0060] In an additional or alternative example, the autonomous vehicle 302 may detect all stationary vehicles (i.e., not just those confined to the same lane) within the range of the autonomous vehicle 302's sensors or within a preset threshold distance (e.g., 50 meters, 100 meters) of the autonomous vehicle 302. In such an example, information about stationary vehicles in other lanes can be used to plan how other vehicles should react (e.g., planning routes to enter the autonomous vehicle's lane and around vehicles parked side-by-side). Feature values ​​may reflect the location and / or other feature values ​​associated with these other detected stationary vehicles.

[0061] Figure 3D Describe additional or alternative features 340. Figure 3D The shaded area indicates an environment region 342 that is occluded by at least one of the sensors of the autonomous vehicle 302 (e.g., occluded by the surface of the stationary vehicle 304). The perception engine of the autonomous vehicle 302 can generate a feature value indicating the probability that an occluding object 344 is present in front of the stationary vehicle 304 or otherwise in the occluded region 342.

[0062] In some examples, the perception engine may determine that sensor data includes residuals that can indicate the presence of an occluding object 344. For example, this residual may include a portion of an image / video indicating the presence of an occluding object that cannot be classified, or a sonar (SONAR) and / or radar (RADAR) anomaly. Taking radar as an example, the perception engine may determine the probability of the presence of an occluding object 344 based on identifying a portion of radar data attributable to the vehicle and / or other identified environmental objects (e.g., road), and determining that residuals in the radar data can indicate the presence of the occluding object 344. For example, RADAR data may include reflected noise indicating the presence of an object at a distance greater than the distance from the autonomous vehicle 302 to the stationary vehicle 304. These noise reflections may, in some cases, be refracted from under the chassis of the stationary vehicle 304 and / or via nearby objects (e.g., walls) from the occluding object to the autonomous vehicle's radar sensors.

[0063] In some examples, the feature value may include the distance from the stationary vehicle 304 to the object 344. This may include examples of the object 344 being obscured from the direct sensor "field of view," or examples of the object 344 being at least partially within the sensor's "field of view."

[0064] Figure 3EDepicting additional or alternative features 346, the perception engine can determine feature values ​​for these features. The perception engine of the autonomous vehicle 302 can generate feature values ​​indicating the presence of a person 348 near a stationary vehicle 304. In some examples, these feature values ​​may indicate: the distance between the person 346 and the stationary vehicle 304, whether the person 348 has left and returned to the stationary vehicle 304, many people near the stationary vehicle 304, and / or whether a door or other opening of the stationary vehicle 304 is open. In some examples, the feature values ​​may additionally or alternatively indicate the yaw 350 of the stationary vehicle 304. Yaw may be determined relative to the attitude of the autonomous vehicle 302 and / or relative to the direction of the lane. Figure 3E An example is depicted in which, since the heading and lane are parallel in the depicted example, a yaw of 350 is determined relative to the heading and lane of the autonomous vehicle 302.

[0065] Figure 3F The perception engine can determine feature values ​​for additional or alternative features 352. The perception engine of the autonomous vehicle 302 can generate feature values ​​that indicate the classification of detected objects (e.g., delivery truck, cone, sign, torch) and / or a meta-classification describing a set of classifications (e.g., delivery, construction area, disabled vehicle).

[0066] In summary, the perception engine can determine one or more feature values ​​(e.g., 15 meters from a stationary vehicle to an intersection; "delivery truck"; green light; traffic flow data, including the rates of other detected vehicles, and / or indications of whether the stationary vehicle's rate is anomalous compared to all other vehicles, vehicles in the same lane, or another subset of detected vehicles), said one or more feature values ​​corresponding to features(s), against which the BV ML model has been trained. The BV ML model can push these, along with any other feature values, through nodes of the BV ML model to determine the probability that a stationary vehicle is a congested vehicle.

[0067] Example Architecture

[0068] Figure 4 This is a block diagram of an example architecture 400 based on any of the technologies discussed herein, which includes an example vehicle system 402 for controlling the operation of at least one vehicle, such as an autonomous vehicle. In some examples, vehicle system 402 may represent at least a portion of autonomous vehicles 204 and / or 302. In some examples, this architecture may be used to control an autonomous vehicle encountering a stationary vehicle.

[0069] In some examples, vehicle system 402 may include processor(s) 404 and / or memory 406. These components are... Figure 4The components are shown in combination, but it is understood that in some examples these components may be individual components of vehicle system 402 and components of the system may be implemented as hardware and / or software.

[0070] Processor(s) 404 may comprise a single-processor system or a multi-processor system, wherein the single-processor system comprises one processor, or the multi-processor system comprises multiple processors (e.g., two, four, eight, or another suitable number). Processor(s) 404 may be any suitable processor capable of executing instructions. For example, in various embodiments, the processor may be a general-purpose or embedded processor implementing an architecture (ISA) of various instruction sets such as x86, PowerPC, SPARC, or MIPS ISA, or any other suitable ISA. In a multi-processor system, each processor 404 may collectively, but not necessarily, implement the same ISA. In some examples, processor(s) 404 may comprise a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a combination thereof.

[0071] Example vehicle system 402 may include memory 406. In some examples, memory 406 may include a non-transient computer-readable medium configured to store executable instructions / modules, data, and / or data items accessible to processor(s) ... Program instructions and data that can be stored via a non-transient computer-readable medium can be transmitted via a transmission medium or signals such as electrical, electromagnetic, or digital signals, which can be transmitted via communication media such as networks and / or wireless links, such as via network interface 410.

[0072] Furthermore, despite Figure 4While shown as a single unit, it should be understood that the processor(s) 404 and memory 406 may be distributed among multiple computing devices in the vehicle and / or among multiple vehicles, data centers, remote operation centers, etc.

[0073] In some examples, the input / output (“I / O”) interface 408 may be configured to coordinate I / O communications between processors (one or more) 404, memory 406, network interface 410, sensors (one or more) 412, I / O devices 414, drive system 416, and / or any other hardware of vehicle system 402. In some examples, I / O devices 414 may include external and / or internal speakers (one or more), displays (one or more), occupant input devices (one or more), etc. In some examples, the I / O interface 408 may perform protocol, timing, or other data conversions to convert data signals from one component (e.g., a non-transient computer-readable medium) into a format suitable for another component (e.g., processors (one or more)). In some examples, the I / O interface 408 may include support for devices connected via various types of peripheral buses (e.g., the Peripheral Component Interconnect (PCI) bus standard, the Universal Serial Bus (USB) standard, or variations thereof). For example, in some implementations, the functionality of the I / O interface 408 may be divided into two or more separate components, such as a northbridge and a southbridge. Moreover, in some examples, some or all of the functionality of the I / O interface 408, such as the interface to the memory 406, can be directly integrated into the processor(one or more) 404 and / or one or more other components of the vehicle system 402.

[0074] Example vehicle system 402 may include a network interface 410 configured to establish a communication link (i.e., a “network”) between vehicle system 402 and one or more other devices. For example, network interface 410 may be configured to allow data exchange between vehicle system 402 and another vehicle 418 (e.g., vehicles (one or more) 104(2) and (3)) via a first network 420, and / or between vehicle system 402 and telecomputing system 422 via a second network 424. For example, network interface 410 may enable wireless communication between another vehicle 418 and / or telecomputing device 422. In various embodiments, network interface 410 may support communication via wireless general data networks such as Wi-Fi networks and / or wireless general data networks such as cellular communication networks, satellite networks, etc.

[0075] In some examples, the sensor data discussed herein can be received at a first vehicle and transmitted to a second vehicle. In some examples, sensor data received from different vehicles can be combined into feature values ​​determined by the perception engine. For example, sensor data received from the first vehicle can be used to populate feature values ​​that are not available to the second vehicle and / or to weight feature values ​​determined by the second vehicle from sensor data received at the second vehicle.

[0076] The example vehicle system 402 may include one or more sensors 412, configured to position the vehicle system 402 in an environment to detect one or more objects in the environment, to sense movement of the example vehicle system 402 through its environment, to sense environmental data (e.g., ambient temperature, pressure, and humidity), and / or to sense conditions inside the example vehicle system 402 (e.g., number of passengers, interior temperature, noise level). Sensors 412 may include, for example, one or more lidar sensors, one or more cameras (e.g., RGB cameras, intensity (grayscale) cameras, infrared cameras, depth cameras, stereo cameras), one or more magnetometers, one or more radar sensors, one or more sonar sensors, one or more microphones for sensing sound, one or more IMU sensors (e.g., including accelerometers and gyroscopes), one or more GPS sensors, one or more Geiger counter sensors, one or more wheel encoders, one or more drive system sensors, speed sensors, and / or other sensors related to the operation of the example vehicle system 402.

[0077] Example vehicle system 402 may include perception engine 426, BV ML model 428 and planner 430.

[0078] The perception engine 426 may include instructions stored in memory 406 that, when executed by processor 404, configure processor(s) 404 to receive sensor data from sensors(s) 412 as input and output data representing, for example, one or more poses (e.g., position and orientation) of objects in the environment surrounding the example vehicle system 402, object trajectories associated with those objects (e.g., historical position, velocity, acceleration, and / or direction of travel of the object over a period of time (e.g., 5 seconds), and / or object classifications associated with the objects (e.g., pedestrians, vehicles, cyclists, etc.). In some examples, the perception engine 426 may be configured to predict object trajectories for more than one or more objects. For example, the perception engine 426 may be configured to predict multiple object trajectories based on, for example, probabilistic determination or multimodal distributions of predicted position, trajectory, and / or velocity associated with the objects.

[0079] The perception engine 426 may include instructions stored in memory 406 that, when executed by processor 404, configure processor 404 to receive sensor data from sensor(s)(s) 412 as input and output an indication that the perception engine has detected a stationary vehicle from the sensor data and may output one or more feature values. These feature values ​​may also be stored in memory 406. For example, this may include instructions that configure processor 404 to determine the distance between a stationary vehicle and a traffic light from images and / or a cloud of LiDAR points. The perception engine 426 may transfer the feature values ​​to BV ML model 428.

[0080] BV ML model 428 may include instructions stored in memory 406 that, when executed by processor 404, configure processor(s) 404 to receive feature values ​​associated with elements of the environment in which vehicle system 402 is located and to determine the probability that a stationary vehicle is a blocked vehicle. BV ML model 428 may include decision trees(s) with nodes and / or deep learning algorithms(s) that can drive feature values ​​through the nodes to determine and output.

[0081] The perception engine 426 can transmit the probability that a stationary vehicle is a blocked vehicle, along with any other additional information that the planner 430 can use to generate a trajectory (e.g., object classification, object trajectory, vehicle attitude), to the planner 430. In some examples, the perception engine 426 and / or the planner 430 may, at least in part, transmit a blocked vehicle indication via network interface 410 to a remote computing device 422, and / or via network 420 to another vehicle 418, based on the probability determined by the perception engine 426. In some examples, if vehicle 418 encounters a stationary vehicle at the same location indicated by the perception engine 426 of vehicle system 402, that indication can be used as a feature value by another vehicle 418. In some examples, this may include temporarily modifying a global map to include blocked vehicle indications, wherein the global map can be obtained by the convoy via a network.

[0082] In some examples, the perception engine and / or BV ML model 428 may be located at another vehicle 418 and / or a remote computing device 422. In some examples, the perception engine and / or remote computing device 422 located at another vehicle 418 may coordinate with the perception engine 426. For example, the other vehicle 418 and / or remote computing device 422 may determine one or more feature values ​​and / or probabilities. In the example where the other vehicle 418 and / or remote computing device 422 determines one or more feature values, the other vehicle 418 and / or remote computing device 422 may transmit one or more feature values ​​to the vehicle system 402 via networks 420 and / or 424, respectively. The perception engine 426 may include one or more feature values ​​received from the other vehicle 418 and / or remote computing device 422 in the feature values ​​pushed through the BV ML model 428 by the perception engine 426. In some examples where the BV ML model 428 is located at another vehicle 418 and / or a remote computing device 422, the other vehicle 418 and / or the remote computing device 422 may receive one or more feature values ​​from the vehicle system 402 via networks 420 and 424 respectively, and may determine the probability that a stationary vehicle is a blocked vehicle. The other vehicle 418 and / or the remote computing device 422 may then transmit this probability back to the planner 430 of the vehicle system 402.

[0083] In some examples, the remote computing device 422 may include a remote operating device. The remote operating device may be a device configured to indicate whether a stationary vehicle is a blocked vehicle in response to sensor data and / or one or more characteristic values. In additional or alternative examples, the remote operating device may display information relating to the sensor data and / or the one or more characteristic values, used to receive input from a remote operator (“remote operator”) confirming or identifying whether a stationary vehicle is / is not a blocked vehicle. In such examples, the remote operating device may include an interface for receiving input from the remote operator, such as an indication of whether the stationary vehicle is a blocked vehicle as correctly affirmed or incorrectly affirmed. In some examples, the remote operating device may confirm the indication or identify the indication as incorrectly affirmed in response to an autonomous vehicle and / or other autonomous vehicles.

[0084] In some examples, a remote operator may input feature values ​​into a remote computing device 422, which may then be transmitted to a vehicle system 402 for use by a BV ML model 428; and / or input feature values ​​into a BV ML model located at the remote computing device 422.

[0085] Planner 430 may include instructions stored in memory 406 that, when executed by processor(s) 404, configure processor(s) 404 to generate data representing a trajectory of the example vehicle system 402, for example, using data representing the position of the exemplary vehicle system 402 in its environment and other data (such as local attitude data), and the probability that a stationary vehicle is a blocked vehicle. In some examples, planner 430 may generate multiple potential trajectories for controlling the example vehicle system 402 substantially continuously (e.g., every 1 or 2 milliseconds, although any rolling time is contemplated) and select one of the trajectories for controlling the vehicle. This selection may be based at least in part on the current route, the probability that a stationary vehicle is a blocked vehicle, the current vehicle trajectory, and / or the trajectory data of detected objects. When selecting a trajectory, planner 430 may transmit the trajectory to drive system 416 to control the example vehicle system 402 according to the selected trajectory.

[0086] In some examples, the perception engine 426, the BV ML model 428, and / or the planner 430 may also include dedicated hardware, such as a processor suitable for running the perception engine (e.g., a graphics processor, FPGA).

[0087] Example process

[0088] Figure 5 A flowchart of an example process 500 for training a BV ML model according to the techniques discussed herein is shown.

[0089] At operation 502, according to any of the techniques discussed herein, example process 500 may include receiving samples that include sensor data associated with a tag indicating a stationary, non-blocking vehicle or a stationary, blocked vehicle. For example, thousands of samples may be received, each sample including sensor data for a discrete scene and a tag associated with the sensor data (e.g., blocked or non-blocking).

[0090] At operation 504, according to any of the techniques discussed herein, example process 500 may include determining feature values ​​for the samples based at least in part on sample sensor data. In some examples, determining feature values ​​from sensor data may include: receiving a feature, training a BV ML model on the feature, which may correspond to a feature that a perception engine can classify or otherwise determine (e.g., traffic light status, object trajectory and / or classification, distance to an intersection); and determining feature values ​​from sample data (e.g., for samples including sensor data, such as video of a green traffic light, the perception engine may generate feature values ​​corresponding to a green light, such as the word "green" or a numerical value symbolizing a green light). Feature value determination may be repeated for all samples received at operation 502.

[0091] At operation 506, according to any of the techniques discussed herein, example procedure 500 may include generating a BV ML model configured to output the probability that a stationary vehicle is a blocked vehicle, at least in part based on one or more feature values ​​and labels associated with a sample. For example, and depending on the type of the generated BV ML model (e.g., decision tree(s), deep learning model), training the BV ML model may include generating nodes, connection weights, node layers, and / or layer types that map input feature values ​​to labels. Thus, the resulting BV ML model can receive a set of feature values ​​from the perception engine at runtime and output an indication that a stationary vehicle is or is not a blocked vehicle. In some examples, the indication may include probabilities (e.g., real-valued numbers) that a planner can use to generate trajectories for controlling the autonomous vehicle.

[0092] For example, given a high positive indication that a stationary vehicle is a blocked vehicle, such as a probability equal to or greater than 1, the planner might generate a trajectory that causes the autonomous vehicle to merge into a different lane. For a low positive indication that a stationary vehicle is a blocked vehicle, such as a probability less than 1, the planner might determine a trajectory to remain in place for a few more seconds before reassessing, or to transmit a request for remote operational assistance to a remote computing device. For a negative indication that a stationary vehicle is a blocked vehicle, either a high value (greater than or equal to 1) or a low value (less than 1), the planner might determine a trajectory to remain in place. It is conceivable that the actual values ​​used by the planner to take different actions will depend on the planner's configuration.

[0093] Figure 6 A flowchart of an example process 600 for detecting a blocked vehicle is shown. For example, the operation of example process 600 may be performed by one or more processors of the autonomous vehicle or other components of the autonomous vehicle, as described below.

[0094] At operation 602, according to any of the techniques discussed herein, example process 600 may include receiving sensor data 604 from at least one sensor 412. For example, sensor data 604 may be received at a perception engine 426.

[0095] At operation 606, according to any of the techniques discussed herein, example process 600 may include detecting stationary vehicles in the environment of the autonomous vehicle, at least in part, based on sensor data 604. For example, this may include: detecting the presence of an object in the environment of the autonomous vehicle, classifying the object as a vehicle, determining the speed of the vehicle, and determining that the vehicle's speed does not meet a preset threshold speed. In additional or alternative examples, this may include determining that the vehicle is obstructing a previously generated trajectory of the autonomous vehicle and / or determining that the vehicle is obstructing another vehicle also in the environment of the autonomous vehicle. One or more of these operations may include inputting sensor data 604 into a stationary vehicle detector 608, which may include one or more machine learning algorithms of perception engine 426 or other components of the vehicle. In some examples, a stationary vehicle indication 610 may be generated (e.g., changing a register or flag value, transmitting a command to another component of perception engine 426).

[0096] At operation 612, according to any of the techniques discussed herein, example process 600 may include determining one or more feature values ​​614. For example, determining one or more feature values ​​614 may include (1) detecting one or more other vehicles on the road besides stationary vehicles and autonomous vehicles, and (2) determining feature values ​​indicating the speed of the stationary vehicles and the speed of one or more other vehicles. In some examples, the feature value may include an indication of whether the speed of the stationary vehicle is anomalous compared to one or more other vehicles, and / or an indication of the distribution of traffic data indicating the speed of the stationary vehicle and the speed of one or more other vehicles. In some examples, in Figure 6 The collection of various components of the perception engine 426, which is typically referred to as the feature value generator 616 (e.g., various machine learning algorithms that perform natural language processing, object detection, object classification, and object tracking), can determine the feature value 614 at least in part based on sensor data 604 and / or stationary vehicle indication 610.

[0097] In some examples, depending on any of the techniques discussed herein, operation 612 may additionally include providing one or more feature values ​​614 as input to the ML model (e.g., BV ML model 428).

[0098] At operation 618, according to any of the techniques discussed herein, example procedure 600 may include receiving an indication 620 from the ML model that a stationary vehicle is a blocked vehicle or a non-blocking vehicle (i.e., Figure 6The BV indicator 620 in the diagram. For example, indicator 616 may include labels (e.g., “blocked vehicles”, “unblocked vehicles”) and / or probabilities. In some examples, depending on any of the techniques discussed herein, perception engine 426 may receive indicator 620 and / or BV ML model 428, or perception engine 426 may transmit indicator 620 to planner 430. In some examples, planner 430 may additionally receive at least one of sensor data 604, data from perception engine 426 (e.g., object classification, object tracks), etc.

[0099] At operation 622, according to any of the techniques discussed herein, example process 600 may include generating a trajectory 624 for controlling the motion of an autonomous vehicle. For example, planner 430 may generate candidate trajectories at least in part based on instruction 616 and select one of the candidate trajectories to control the autonomous vehicle. Planner 430 may transmit the selected trajectory to drive system 416 of the autonomous vehicle.

[0100] Figure 7 A flowchart of an example process 700 for detecting a blocked vehicle is shown. For example, the operation of example process 700 may be performed by one or more processors or other components of the autonomous vehicle, and as described below, and / or the operation may be performed by a remote computing system such as another autonomous vehicle and / or a remote operating device.

[0101] At operation 702, the example process 700 may include receiving sensor data, depending on any of the techniques discussed herein.

[0102] At operation 704, based on any of the techniques discussed herein, example process 700 can identify a stationary vehicle (i.e., the "Yes" branch in the flowchart). In some examples, if no stationary vehicle is identified, process 700 can return to operation 702.

[0103] At operation 706, according to any of the techniques discussed herein, example process 700 may include determining one or more feature values ​​based at least in part on sensor data. Any feature values ​​discussed herein may be included. For example, one of the one or more feature values ​​may include determining (706(A)) traffic flow data indicating the speed of one or more vehicles detected from the sensor data. This may include the speed of stationary vehicles and / or the speed of other detected vehicles.

[0104] At operation 708, according to any of the techniques discussed herein, example procedure 700 may include providing one or more feature values ​​to a machine learning model.

[0105] At operation 710, according to any of the techniques discussed herein, example procedure 700 may include outputting the probability that a stationary vehicle is a blocked vehicle via a machine learning model.

[0106] At operation 712, according to any of the techniques discussed herein, example process 700 may include controlling the vehicle based at least in part on probability. For example, this may include controlling the vehicle to pass through a blocked vehicle (712(A)) or controlling the vehicle to wait for a non-blocking vehicle (712(B)).

[0107] Example Clause 1

[0108] A. An autonomous vehicle, comprising: at least one sensor; a drive system for controlling physical operations of the autonomous vehicle; and a perception engine configured to perform operations including: receiving sensor data from the at least one sensor; and detecting stationary vehicles in the environment of the autonomous vehicle based at least partially on the sensor data; determining one or more feature values ​​based at least partially on the sensor data, the determination of the one or more feature values ​​including detecting one or more other vehicles and speeds associated with the one or more other vehicles and the stationary vehicles based at least partially on the sensor data, wherein the one or more feature values ​​include a distribution of traffic flow data indicating the speeds of the stationary vehicles and the speeds of the one or more other vehicles; providing the one or more feature values ​​as input to a machine learning model; receiving from the machine learning model an indication that the stationary vehicle is a congested vehicle or a non-congested vehicle; transmitting the indication to a planner, wherein the planner is configured to perform operations including: receiving the indication; and generating a trajectory for controlling the movement of the autonomous vehicle.

[0109] B. According to the autonomous vehicle in paragraph A, detecting a stationary vehicle includes: detecting the presence of an object in the environment of the autonomous vehicle; classifying the object as a vehicle; determining the speed of the vehicle; and determining that the speed of the vehicle does not meet a preset threshold speed.

[0110] C. According to paragraph A or B, the detection of stationary vehicles further includes: detecting one or more other stationary vehicles within a preset threshold distance of the autonomous vehicle.

[0111] D. An automated vehicle based on any segment of paragraph AC, wherein the one or more feature values ​​further include at least the speed of a stationary vehicle and the traffic signal status.

[0112] E. An autonomous vehicle according to any paragraph AD, wherein the obstructing vehicle is an object detected by the perception engine that prevents the autonomous vehicle or at least one of the other vehicles from moving forward.

[0113] F. A computer-implemented method for controlling a vehicle, comprising: receiving sensor data from one or more sensors of the vehicle; identifying a stationary vehicle based at least in part on the sensor data; determining one or more feature values ​​based at least in part on the sensor data, one of the one or more feature values ​​including traffic flow data indicating the speed of one or more vehicles detected from the sensor data; providing the one or more feature values ​​to a machine learning model; outputting, through the machine learning model, a probability that the stationary vehicle is a congested vehicle; and controlling the vehicle based at least in part on the probability, wherein controlling the vehicle includes controlling the vehicle to pass through the congested vehicle or controlling the vehicle to wait for a non-congested vehicle.

[0114] G. According to paragraph F, the autonomous vehicle, wherein the one or more characteristic values ​​include at least one of the following: the speed of the stationary vehicle, lane markings, traffic signal status, distance from the stationary vehicle to the next object in front of the stationary vehicle, distance from the stationary vehicle to the next intersection, traffic flow data, vehicle tracks indicating the movement of another vehicle relative to the stationary vehicle, sensor residuals indicating the presence of occluded objects, classification of at least one of the stationary vehicle or objects in the environment, bounding box associated with the stationary vehicle, light status of the stationary vehicle, indication of someone near the vehicle, status of doors or openings of the stationary vehicle, size of the stationary vehicle, or yaw of the stationary vehicle.

[0115] H. According to paragraph F or paragraph G, for an autonomous vehicle, wherein the machine learning model includes multiple decision trees configured to: receive the one or more feature values; and push the one or more feature values ​​through the nodes of the decision trees to reach an output node associated with a weighted value; and sum the weighted values ​​to determine a probability.

[0116] I. A computer-implemented method according to any paragraph FH, wherein the method further includes detecting stationary vehicles in the vehicle's environment based at least in part on sensor data.

[0117] J. A computer-implemented method according to any paragraph FI, wherein detecting a stationary vehicle includes: detecting the presence of objects in the vehicle's environment; classifying the objects as vehicle-based objects; determining the vehicle's speed; and determining that the vehicle's speed does not meet a preset threshold speed.

[0118] K. A computer-implemented method according to any paragraph of FJ, wherein the one or more feature values ​​indicate one or more characteristics of at least one of the vehicle's environment, a stationary vehicle, or objects in the environment.

[0119] L. A computer-implemented method according to any paragraph of FK, wherein: a probability is output based at least in part on the detection of a stationary vehicle; and the probability indicates the probability that the stationary vehicle obstructs the vehicle.

[0120] M. A computer-implemented method according to any paragraph in paragraph FL, wherein the one or more feature values ​​include at least one of the following: the speed of a stationary vehicle, the state of the lights of a signal indicator associated with the stationary vehicle, the distance to the next intersection, the speed of at least one other vehicle, the distance between the stationary vehicle and the next vehicle in front of the stationary vehicle, or the state of a traffic signal.

[0121] N. A computer-implemented method according to any paragraph of paragraph FM, wherein the method further comprises: determining, at least in part based on traffic flow data, that the speed of a stationary vehicle detected by the perception engine is anomalous compared to the speeds of two or more other vehicles; and indicating that the speed of the stationary vehicle is anomalous as one of the one or more feature values.

[0122] O. A non-transient computer-readable medium having a set of instructions that, when executed, cause one or more processors to perform operations, the operations comprising: one or more processors; and a memory storing thereon the set of instructions, the one or more processors performing operations when executed, the operations comprising: receiving sensor data from at least one sensor; detecting a stationary vehicle; receiving one or more feature values; transmitting the one or more feature values ​​to a machine learning model; receiving from the machine learning model a probability that the stationary vehicle is a blocked vehicle; and transmitting the probability to a planner for an autonomous vehicle, the planner being configured to control the motion of the autonomous vehicle at least in part based on the probability.

[0123] According to paragraph P, in a non-transient computer-readable medium, detecting a stationary vehicle includes: detecting the presence of an object in the environment near the autonomous vehicle; classifying the object as a vehicle; determining the speed of the vehicle; and determining that the speed of the vehicle does not meet a preset threshold speed.

[0124] Q. According to paragraph O or P, in a non-transient computer-readable medium, detecting a stationary vehicle also includes determining that the vehicle obstructs a trajectory previously determined by an autonomous vehicle.

[0125] R. According to any paragraph of the OQ in the non-transient computer-readable medium, wherein the operation further includes: receiving multiple pairs of sample feature values ​​and sample indications, wherein the individual sample indications indicate blocked or unblocked vehicles, and deriving the pairs from sample sensor data; and training a machine learning model from the multiple pairs by: generating an input layer of nodes configured to receive sample feature values; generating one or more hidden layers, wherein the input layer of nodes is configured to activate nodes in the one or more hidden layers; and generating an output layer, wherein the output layer is configured to receive stimuli from nodes in the one or more hidden layers and output probabilities.

[0126] S. According to any paragraph of paragraph OR, in a non-transient computer-readable medium, wherein the one or more feature values ​​include at least one of the following: speed of a stationary vehicle, traffic signal status, distance from a stationary vehicle to the next object; distance from a stationary vehicle to the next intersection, lane markings, bounding box of a stationary object, traffic flow data, vehicle tracks indicating the movement of another vehicle relative to a stationary vehicle, sensor residuals indicating the presence of an obstructing object, classification of at least one of a stationary vehicle or an object in the environment, light status of a stationary vehicle, indication of a person near a vehicle, status of a door or opening of a stationary vehicle, height of a stationary vehicle, or yaw of a stationary vehicle.

[0127] T. Based on any non-transient computer-readable medium in paragraph OS, wherein the machine learning model is one or more decision tree or deep learning models.

[0128] Example Clause 2

[0129] 1. A system comprising: one or more processors; and a memory storing processor-executable instructions, said instructions causing said one or more processors to perform operations when executed by said one or more processors, said operations including: receiving sensor data from one or more sensors of a vehicle; detecting stationary vehicles at least partially based on the sensor data; determining one or more features at least partially based on the sensor data, one of said one or more features indicating traffic flow data; providing said one or more features to a machine learning model; outputting a probability from the machine learning model that a stationary vehicle is a congested vehicle; and controlling the vehicle at least partially based on said probability, wherein controlling the vehicle includes: controlling the vehicle to pass over congested vehicles, or controlling the vehicle to wait for non-congested vehicles.

[0130] 2. The system according to Clause 1, wherein identifying a stationary vehicle includes: detecting the presence of an object in the environment of the vehicle; classifying the object as a vehicle category; determining the speed of the object; and determining that the speed of the object does not meet a preset threshold speed.

[0131] 3. The system according to Clause 1, wherein the one or more features indicate at least one of the following: speed of a stationary vehicle, lane markings, traffic signal status, distance from a stationary vehicle to the next object in front of the stationary vehicle, distance from a stationary vehicle to the next intersection, traffic flow data, vehicle tracks indicating the movement of another vehicle relative to the stationary vehicle, sensor residuals indicating the presence of obstructing objects, classification of at least one of the stationary vehicle or objects in the environment, bounding box associated with the stationary vehicle, light status of the stationary vehicle, indication of someone near the vehicle, status of doors or openings of the stationary vehicle, size of the stationary vehicle, or yaw of the stationary vehicle.

[0132] 4. The system according to Clause 1, wherein identifying a stationary vehicle further includes detecting one or more other stationary vehicles within a preset threshold distance of the autonomous vehicle.

[0133] 5. The system according to Clause 1, wherein the machine learning model includes a plurality of decision trees configured to: receive the one or more features; push the one or more features through the nodes of the decision trees to reach an output node associated with a weighted value; and sum the weighted values ​​to determine the probability.

[0134] 6. The system according to Clause 1, wherein the one or more features indicate one or more characteristics of at least one of the vehicle's environment, a stationary vehicle, or objects in the environment.

[0135] 7. The system according to Clause 1, wherein: the probability is output based at least in part on the detection of stationary vehicles; and the probability indicates the likelihood that a stationary vehicle obstructs a vehicle.

[0136] 8. The system according to Clause 1, wherein the operation further comprises: determining, at least in part based on traffic flow data, that the speed of a stationary vehicle is anomalous compared to the speeds of two or more other vehicles detected by the perception engine; and, as one of the one or more features, indicating that the speed of the stationary vehicle is anomalous.

[0137] 9. A method comprising: receiving sensor data from at least one sensor; detecting a stationary vehicle; receiving one or more features; transmitting the one or more features to a machine learning model; receiving from the machine learning model a probability that the stationary vehicle is a blocked vehicle; and transmitting the probability to a planner for an autonomous vehicle, the planner being configured to control the motion of the autonomous vehicle at least in part based on the probability.

[0138] 10. The method according to Clause 9, wherein detecting a stationary vehicle comprises: detecting the presence of an object in the environment near the autonomous vehicle; classifying the object as a vehicle category; determining the speed of the object; and determining that the speed of the object does not meet a preset threshold speed.

[0139] 11. The method according to Clause 10 further comprises: receiving multiple pairs of sample features and sample indications, wherein a single sample indication indicates a blocked vehicle or a non-blocked vehicle, and deriving the pairs from sample sensor data; and training a machine learning model from the multiple pairs by: inputting sample features into an input layer of a node; receiving output probabilities from an output layer; determining the difference between the output probabilities and the sample indications; and changing one or more parameters of the machine learning model to minimize the difference.

[0140] 12. The method according to Clause 9, wherein the one or more features indicate at least one of the following: speed of a stationary vehicle, traffic signal status, distance from a stationary vehicle to the next object, distance from a stationary vehicle to the next intersection, lane markings, bounding box of a stationary object, traffic flow data, vehicle tracks indicating the movement of another vehicle relative to a stationary vehicle, sensor residuals indicating the presence of an obstructing object, classification of at least one of a stationary vehicle or an object in the environment, light status of a stationary vehicle, indication of someone near the vehicle, status of a door or opening of a stationary vehicle, height of a stationary vehicle, or yaw of a stationary vehicle.

[0141] 13. The method according to Clause 9 further includes: determining, at least in part, based on traffic flow data, that the speed of a stationary vehicle is anomalous compared to the speeds of two or more other detected vehicles; and providing an indication that the speed of the stationary vehicle is anomalous as one of the one or more features.

[0142] 14. The method according to Clause 9, wherein the one or more features indicate one or more characteristics of at least one of the vehicle's environment, a stationary vehicle, or objects in the environment.

[0143] 15. A non-transient computer-readable medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform a method according to any one of clauses 9 to 14.

[0144] Although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described. Rather, specific features and actions are disclosed as exemplary forms for implementing the claims.

[0145] The modules described herein represent instructions that can be stored in any type of computer-readable medium and can be implemented in software and / or hardware. All the methods and processes described above can be embodied and fully automated by software code modules and / or computer-executable instructions executed by one or more computers or processors, hardware, or some combination thereof. Some or all of these methods may alternatively be embodied in dedicated computer hardware.

[0146] Unless otherwise explicitly stated, conditional language (e.g., “may,” “can,” “may,” or “will”) should be understood in context as indicating that some examples include certain features, elements, and / or steps while others do not. Therefore, such conditional language is not generally intended to imply that one or more examples require certain features, elements, and / or steps in all circumstances, or that one or more examples must include logic to determine whether certain features, elements, and / or steps are included or will be performed in any particular example, with or without user input or prompts.

[0147] Unless otherwise specified, phrases such as “at least one of X, Y, or Z” should be understood to mean that an item, term, etc., can be X, Y, or Z, or any combination thereof, including multiples of each element. Unless explicitly stated as singular, “one” indicates both singular and plural.

[0148] Any general descriptions, elements, or blocks in the flowcharts described herein and / or depicted in the accompanying drawings should be understood as potentially representing a portion of a module, segment, or code, which includes one or more computer-executable instructions for implementing a particular logical function or element in a routine. Alternative embodiments are included within the scope of the examples described herein, wherein elements or functions may be removed from or performed out of order of the shown or discussed elements or functions, including substantially synchronous, in reverse order, with additional operations, or omitted operations, depending on the functionality involved as would be understood by those skilled in the art.

[0149] It should be emphasized that many variations and modifications can be made to the above examples, and these examples and elements should be understood as being among other acceptable examples. All such modifications and variations are intended to be included within the scope of this disclosure and protected by the appended claims.

Claims

1. A method for controlling vehicle operation, comprising: Receive sensor data associated with the vehicle's environment; Based at least in part on sensor data, determine indications of traffic flow associated with objects within a part of the environment, wherein the objects are not stationary objects; The instructions are provided as input to the machine learning model; The probability of a stationary object blocking a vehicle's path or a portion of the road associated with the vehicle is received from the machine learning model; and Vehicle control is based at least in part on probability.

2. The method according to claim 1, wherein, Traffic flow instructions include at least one of the following: A first distribution of at least one of the motion, direction of travel, or speed of an object in the aforementioned part of the environment; A second distribution of at least one of the motion, direction of travel, or speed of objects of the same category in the aforementioned part of the environment; Tail characteristics of the first or second distribution; An indication of whether at least one of the forward direction or velocity associated with a stationary object is located in the tail of the first or second distribution; Based at least in part on the percentage of objects that have the same direction or speed of movement as stationary objects in either the first or second distribution; At least in part based on the percentiles of the velocity of a stationary object associated with the first or second distribution; or The frequency with which another object in the aforementioned part of the environment performs a certain type of action.

3. The method according to claim 1, further comprising: Based at least in part on sensor data, determine an object trajectory associated with an object, wherein the object trajectory includes at least one of the object's current or historical direction of travel and speed; and Based at least in part on the object's trajectory, determine that the object's current or past direction of movement is the same as the vehicle's direction of travel, wherein: Traffic flow indications are determined at least in part based on object trajectories and by determining that the object is currently or has been moving in the same direction as the direction of travel.

4. The method according to claim 1, further comprising: Based at least in part on sensor data, determine object detection associated with objects, wherein object detection includes classification associated with objects; and Based at least in part on two or more object detections indicating classification, a distribution of object trajectory features associated with the two or more object detections is determined, wherein the object trajectory features include at least one of the current, historical, or predicted position, direction of travel, or speed indicated by the object trajectories associated with the two or more object detections. Distribution is part of the indication of traffic flow.

5. The method of claim 1, further comprising determining a weight associated with traffic flow indication based at least in part on the proximity of the object to the vehicle, wherein, Weights are included in traffic flow indications.

6. The method according to claim 1, further comprising: The vehicle is controlled to pass through a stationary object based at least in part on a probability that is determined to reach or exceed a threshold; otherwise... Control the vehicle to maintain its current movement or stop it.

7. A system for controlling vehicle operation, comprising: One or more processors; as well as A memory storing processor-executable instructions, which, when executed by the one or more processors, cause the system to perform the following operations, including: Receive sensor data indicating the environment associated with the vehicle; Based at least in part on sensor data, determine indications of movement of objects within a portion of the environment, wherein the objects are not stationary objects; The instructions are provided as input to the machine learning model; The probability of a stationary object blocking a vehicle's path or at least a portion of the road associated with the vehicle is received from a machine learning model; and Vehicle control is based at least in part on probability.

8. The system according to claim 7, wherein, The aforementioned portion of the environment includes at least one of a first lane or a second lane, with the vehicle located in the first lane and the second lane located adjacent to the first lane, and associated with an object trajectory indicating that a second object is moving in the second lane in the same direction as the vehicle's direction of travel.

9. The system according to claim 7, wherein, The instructions for movement include at least one of the following: A first distribution of at least one of the motion, direction of travel, or speed of an object in the aforementioned part of the environment; A second distribution of at least one of the actions, directions of travel, or speeds of objects of the same category in the aforementioned part of the environment; Tail characteristics of the first or second distribution; An indication of whether at least one of the forward direction or velocity associated with a stationary object is located in the tail of the first or second distribution; The percentage of objects that have the same direction or speed of movement as stationary objects, based at least in part on the first or second distribution.

10. The system according to claim 7, wherein, The operation also includes: Based at least in part on sensor data, determine an object trajectory associated with an object, wherein the object trajectory includes at least one of the object's current or historical direction of travel and speed; and Based at least in part on the object's trajectory, determine that the object's current or past direction of movement is the same as the vehicle's direction of travel, wherein: The indication of movement is determined at least in part based on the object's trajectory, and the direction in which the object is currently or has been moving is the same as the direction of travel.

11. The system according to claim 7, wherein, The operation also includes: Based at least in part on sensor data, determine object detection associated with objects, wherein object detection includes classification associated with objects; and Based at least in part on two or more object detections indicating classification, a distribution of object trajectory features associated with the two or more object detections is determined, wherein the object trajectory features include at least one of the current, historical, or predicted position, direction of travel, or speed indicated by the object trajectories associated with the two or more object detections. Distribution is part of the indication of movement.

12. The system according to claim 7, wherein, The operation also includes determining weights associated with traffic flow indications, at least in part based on the proximity of objects to vehicles, wherein the weights are included in the traffic flow indications.

13. The system according to claim 7, wherein, The operation also includes: The vehicle is controlled to pass through a stationary object based at least in part on a probability that is determined to reach or exceed a threshold; otherwise... Control the vehicle to maintain its current movement or stop it.

14. A non-transitory computer-readable medium storing processor-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform the following operations: Receive sensor data indicating the environment associated with the vehicle; Based at least in part on sensor data, determine indications of movement of objects within a portion of the environment, wherein the objects are not stationary objects; The instructions are provided as input to the machine learning model; The probability of a stationary object blocking a vehicle's path or at least a portion of the road associated with the vehicle is received from a machine learning model; and Vehicle control is based at least in part on probability.

15. The non-transient computer-readable medium according to claim 14, wherein, The aforementioned portion of the environment includes at least one of a first lane or a second lane, with the vehicle located in the first lane and the second lane located adjacent to the first lane, and associated with an object trajectory indicating that a second object is moving in the second lane in the same direction as the vehicle's direction of travel.

16. The non-transient computer-readable medium according to claim 14, wherein, The instructions for movement include at least one of the following: A first distribution of at least one of the motion, direction of travel, or speed of an object in the aforementioned part of the environment; At least one second distribution of the motion, direction of travel, or speed of objects of the same category in the aforementioned part of the environment; Tail characteristics of the first or second distribution; An indication of whether at least one of the forward direction or velocity associated with a stationary object is located in the tail of the first or second distribution; The percentage of objects that have the same direction or speed of movement as stationary objects, based at least in part on the first or second distribution.

17. The non-transient computer-readable medium according to claim 14, wherein, The operation also includes: Determine, at least in part, an object trajectory associated with an object based on sensor data, wherein the object trajectory includes at least one of the object's current or historical direction of travel and speed; and Based at least in part on the object's trajectory, determine that the object's current or past direction of movement is the same as the vehicle's direction of travel, wherein: The indication of movement is determined at least in part based on the object's trajectory, and the direction in which the object is currently or has been moving is the same as the direction of travel.

18. The non-transient computer-readable medium according to claim 14, wherein, The operation also includes: Based at least in part on sensor data, determine object detection associated with objects, wherein object detection includes classification associated with objects; and Based at least in part on two or more object detections indicating classification, a distribution of object trajectory features associated with the two or more object detections is determined, wherein the object trajectory features include at least one of the current, historical, or predicted position, direction of travel, or speed indicated by the object trajectories associated with the two or more object detections. Distribution is part of the indication of movement.

19. The non-transient computer-readable medium according to claim 14, wherein, The operation also includes determining weights associated with traffic flow indications, at least in part based on the proximity of objects to vehicles, wherein the weights are included in the traffic flow indications.

20. The non-transient computer-readable medium according to claim 14, wherein, The operation also includes: The vehicle is controlled to pass through a stationary object based at least in part on a probability that is determined to reach or exceed a threshold; otherwise... Control the vehicle to maintain its current movement or stop it.

Images