Planning for particulate matter regulation

Abstract

Techniques for detecting objects in an environment, determining a probability that the object is a region of particulate matter, and controlling a vehicle based on the probability. Regions of particulate matter may include vapors (e.g., emitted from manhole covers, dryer vents, etc.), exhaust from vehicles (e.g., cars, trucks, motorcycles, etc.), dust, environmental gases (e.g., resulting from sublimation, mist, evaporation, etc.), etc. Based on the associated probability that the object is a region of particulate matter, the vehicle computing system may substantially maintain the vehicle trajectory, modify the vehicle's trajectory to ensure that the vehicle does not collide with the object, stop the vehicle, or otherwise control the vehicle to ensure that the vehicle continues to proceed in a safe manner. The vehicle controller may continually adjust the trajectory based on additional obtained sensor data and the associated region probability.

Description

[Technical field]

[0001] The present invention relates to planning adjustments to particulate matter, and more particularly to planning adjustments to particulate matter in the control of a vehicle. [Background technology]

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS This PCT international patent application is a continuation of, and claims the benefit of priority to, U.S. patent application Ser. No. 16 / 586,650, entitled "PLANNING ACCOMMODATIONS FOR PARTICULATE MATTER," filed Sep. 27, 2019, the entire contents of which are incorporated herein by reference.

[0003] Various methods, apparatus, and systems are utilized to guide an autonomous vehicle through an environment that includes various static and dynamic objects. For example, an autonomous vehicle may include sensors, such as LIDAR, RADAR, cameras, etc., to detect moving and stationary objects in the environment. The sensors may further collect data associated with particulate matter, such as vapors emanating from a manhole cover or exhaust from a vehicle. In some examples, an autonomous vehicle may utilize the sensor data to make control decisions, such as determining where to drive based in part on the detected objects. However, an autonomous vehicle may recognize particulate matter as an object to be avoided, which may cause the vehicle to stop or adjust its path to avoid the particulate matter, preventing forward movement. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent Application Serial No. 16 / 023,878 [Patent Document 2] U.S. Patent Application Serial No. 16 / 539,928 [Brief description of the drawings]

[0005] The detailed description will be set forth with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number identifies the figure in which the reference number first appears. Use of the same reference number in different figures indicates similar or identical components or features according to examples of the present disclosure. [Figure 1] An example of a vehicle computing system of an autonomous vehicle that may detect objects and control the vehicle based on the probability that the object is associated with an area of ​​particulate matter (e.g., water vapor, etc.) according to an example of the present disclosure. [Diagram 2] 1 is an exemplary process for controlling a vehicle based on a probability that an object in the vehicle's path is associated with a region of particulate matter according to an example of the present disclosure. [Diagram 3] 1 depicts a block diagram of an example system for implementing the techniques described herein. [Figure 4] 1 depicts an example process for controlling a vehicle based on a probability that a detected object is associated with a region of particulate matter. [Diagram 5] 10 depicts an example process for slowing a vehicle to gather additional data about an object to determine whether the object is associated with a region of particulate matter. [Figure 6] 1 depicts an example process for controlling a vehicle to avoid objects that are likely (e.g., low probability) not associated with an area of ​​particulate matter. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0006] The present disclosure relates to techniques for detecting an object in an environment, determining a probability that the object is a region of particulate matter (e.g., a cloud), and controlling a vehicle based on the probability. The region of particulate matter may include steam (e.g., emitted from manhole covers, dryer vents, etc.), exhaust from a vehicle (e.g., a car, truck, motorcycle, etc.), environmental gases (e.g., resulting from sublimation, mist, evaporation, etc.), dust clouds, or other types of particulate matter that may be located in the vehicle's environment and do not affect driving behavior (e.g., an autonomous vehicle may safely pass through the particulate matter without affecting the platform). Based on a first probability that the object is a region of particulate matter, the vehicle computing system may substantially maintain the vehicle trajectory, thereby improving efficient operation of the vehicle. Based on a second probability that the object is a region of particulate matter, the vehicle computing system may modify the vehicle's trajectory to ensure that the vehicle does not collide with the object, thereby increasing safety of the vehicle and other objects in the environment.

[0007] The vehicle may include an autonomous or semi-autonomous vehicle that utilizes sensor data collected by one or more sensors to detect objects in the environment and navigate the environment based on the objects. The sensor data captured by the vehicle may include data captured by a LIDAR sensor, as well as data captured by other sensors, such as camera sensors, RADAR sensors, time-of-flight sensors, and sonar sensors. In some examples, the sensor data may be provided to a perception system configured to determine or classify the type of object in the environment (e.g., vehicle, pedestrian, bicycle, motorcycle, animal, parked car, tree, building, cloud of particulate matter, etc.). In various examples, the perception system may identify one or more features associated with the object and determine a classification based on the one or more features. In various examples, the perception system may utilize techniques described in U.S. Patent Application Publication No. 2013 / 0133994, filed June 29, 2018, entitled "Point Cloud Filtering with Semantic Segmentation," the entire contents of which are incorporated herein by reference. In some examples, the perception system may include a machine-learned model configured to identify features and classify the object. In various examples, the perception system may provide a confidence associated with the classification. In such examples, the confidence may include a probability that the object is associated with the classification. For example, based on features associated with a detected object, the perception system may determine a 70% (or any) probability that the object is associated with (e.g., classified as) an area of ​​particulate matter. In various examples, the object may not be identified as being associated with a particular type of object (e.g., not a pedestrian, car, truck, etc.). In such examples, the perception system may be configured to determine a probability that the object is associated with an area of ​​particulate matter.

[0008] In some examples, the probability may be based on one or more characteristics associated with the object. The characteristics may include the location of the object in the environment (e.g., proximity to a surface, height from a surface, etc.), the size of the object (e.g., width, height, depth, etc.), shape (e.g., symmetric, asymmetric, circular, rectangular, etc.), proximity to another vehicle, proximity to a location on another vehicle (e.g., a location associated with an exhaust, etc.), or the track of the object (e.g., movement captured over time), etc. For example, a vehicle computing system may detect an object in the environment and determine that the bottom edge of the object is approximately 6 inches from the surface of the road and within a threshold distance to another vehicle operating on the same road segment as the vehicle in front of the vehicle. Based on the height characteristic and / or the threshold distance to the other vehicle, the vehicle computing system may determine that there is a high probability that the object is a region of particulate matter.

[0009] In various examples, the probability that the object is associated with the area of ​​particulate matter may be based on map data accessed by the vehicle computing system. The map data may represent information about the environment in which the vehicle operates, based in part on pre-recorded sensor data from one or more vehicles. In some examples, the vehicle computing system may utilize the map data to navigate the vehicle within the environment. The map data may include data associated with drivable surfaces (e.g., streets, roads, road sections, intersections, etc.) and / or known obstacles in the environment (e.g., fixed objects in the environment, etc.). In some examples, the map data may further include known areas of particulate matter emissions, such as manhole covers and dryer vents (e.g., industrial dryers, domestic dryers, etc.). In such examples, the probability that the object is associated with the area of ​​particulate matter may be based in part on the known areas of particulate matter emissions. For example, the vehicle computing system may identify the object at a location in the environment. The vehicle computing system may access the map data to determine that the location is associated with a manhole cover that is often associated with particulate matter emissions (e.g., steam from underground).

[0010] In various examples, the probability may be determined utilizing machine learning techniques. In such examples, a machine-learned model may be trained with training data to determine the probability that an object is associated with an area of ​​particulate matter. The training data may include previously captured sensor data from one or more vehicles that has been annotated to indicate a ground truth condition of one or more areas of particulate matter present in the environment.

[0011] In various examples, the vehicle computing system may determine an action to take based on a probability that the object is associated with an area of ​​particulate matter. The action may include maintaining a course and speed (e.g., a trajectory) associated with the vehicle's path, slowing the vehicle to a predetermined speed, slowing and stopping the vehicle, changing lanes, and / or adjusting a position within the lane. In various examples, the action may be determined based on the probability being higher and / or lower than one or more thresholds. In some examples, the thresholds may be associated with various levels of probability, such as a high probability, a medium probability, or a low probability. In such examples, the level of probability may have one or more actions associated therewith. For example, based on a determination that the probability is equal to or greater than a first threshold probability (e.g., a high probability) that the object is associated with an area of ​​particulate matter, the vehicle computing system may cause the vehicle to maintain a trajectory associated with the vehicle's path. In another example, based on a determination that the probability is less than or equal to a first threshold probability and greater than or equal to a second threshold (e.g., a medium probability), the vehicle computing system may slow the vehicle to a predetermined speed (e.g., 5 miles per hour, 10 miles per hour, 15 kilometers per hour, etc.). In yet another example, based on a determination that the probability is less than or equal to a second threshold probability, the vehicle computing system may stop the vehicle, such as to avoid a collision with an object. These are merely illustrative examples and other actions and / or levels of probability and actions associated therewith are contemplated.

[0012] In various examples, the action may be determined based on a cost associated with the action. In some examples, the vehicle computing system may identify one or more actions to take and may determine a cost associated with each of the actions (e.g., an action cost). In various examples, the cost may be based on one or more of a safety cost (e.g., vehicle and / or object safety, avoiding a collision between the vehicle and the object), a comfort cost (e.g., eliminating abrupt movements, for example, by penalizing large accelerations that are less than a minimum distance between the vehicle and the object), a progress cost (e.g., movement toward a destination), an operational rule cost (e.g., rules of the road, laws, regulations, etc.), or costs described in U.S. Patent Application Publication No. 2019 / 0133634, entitled “Cost-Based Path Determination,” filed on August 13, 2019, the entire contents of which are incorporated herein by reference, and the like. In some examples, the action selected may include the action associated with the lowest cost. In such examples, the action may be based on cost optimization.

[0013] In various examples, actuation may be determined based on a function associated with the probability that the object is associated with a region of particulate matter, the size of the region, the distance to the region, the vehicle speed, and / or any other factor associated with vehicle travel in the environment. In some examples, one or more factors (e.g., probability, size, distance, vehicle speed, etc.) may be weighted more heavily than other factors, e.g., to have a greater impact on actuation selection. For example, the size of the region of particulate matter may be weighted more heavily than the distance to the region. Thus, size may have a greater impact on actuation selection than the distance to the region.

[0014] In various examples, the vehicle computing system may detect an object that may be associated with the region of particulate matter and may determine that the distance to the object is greater than a threshold distance (e.g., 200 feet, 100 yards, 100 meters, etc.). The vehicle computing system may determine an initial probability that the object is associated with the region of particulate matter. Based on a determination that the initial probability is greater than a threshold probability (e.g., 95%, 99%, etc.), the vehicle may proceed along substantially the same trajectory (e.g., speed and / or direction). Based on a determination that the initial probability is less than the threshold probability, the vehicle computing system may modify the trajectory of the vehicle. In some examples, the vehicle computing system may cause the vehicle to change lanes, adjust its position within the lane, or take another action to avoid the object. In some examples, the vehicle computing system may slow the vehicle to a predetermined speed to capture additional data associated with the object. In such examples, the vehicle computing system may determine an updated probability that the object is associated with the region of particulate matter based on the additional data. For example, the vehicle computing system may detect that an object located 100 meters ahead of the vehicle is associated with a 20% probability that the object is associated with an area of ​​water. The vehicle computing system may determine that the distance of 100 meters is above a threshold distance and the 20% probability is below the threshold probability, and may slow the vehicle to 10 miles per hour to capture additional data associated with the object. The vehicle computing system may capture the additional data and may determine an updated probability of 90% that the object is associated with an area of ​​particulate matter. Based on the determination of the updated probability, the vehicle computing system may cause the vehicle to proceed according to the original trajectory (e.g., the trajectory before slowing).

[0015] In various examples, the vehicle computing system may detect an object that may be associated with an area of ​​particulate matter. The vehicle computing system may determine that the probability that the object is associated with the area of ​​particulate matter is below a threshold (e.g., unlikely to be particulate matter). The vehicle computing system may determine a distance to the object and may determine a deceleration rate (e.g., -3 miles per hour per second, -5 miles per hour per second, etc.) based on the distance. The deceleration rate may be associated with pressure applied to the vehicle's brakes, regenerative braking, downshifting, engine braking, or any other method of stopping the vehicle before reaching a location associated with the object. By determining the deceleration rate based on the distance, the vehicle computing system can ensure that the vehicle does not contact the object, thereby significantly improving the safe operation of the vehicle. For example, the vehicle computing system may determine that an object located 25 feet in front of the vehicle has a low probability of being associated with particulate matter. The vehicle computing system may slow the vehicle at a rate of 15 feet per second per second to stop before the object. In some examples, the vehicle computing system may be configured to slow the vehicle at a maximum deceleration rate (e.g., 20 feet per second per second, 25 feet per second per second, etc.) to minimize damage to the vehicle and / or objects.

[0016] In some examples, based on a determination that the probability that the object is associated with an area of ​​particulate matter falls below a threshold, the vehicle computing system may determine a lateral acceleration (e.g., 2 meters per second, 8 feet per second, etc.) required to avoid the object. In such examples, the vehicle computing system may be configured to cause the vehicle to adjust the position of the vehicle laterally (e.g., swerve) from a path that may intersect with the object. In various examples, the vehicle may determine, prior to or substantially simultaneously with the determination of the lateral acceleration, that the lateral path is clear of other objects. In some examples, the decision to adjust the position laterally (e.g., change lanes, swerve, etc.) may be based on the distance to the object being less than a threshold distance (e.g., 30 meters, 40 feet, etc.). In some examples, the threshold distance may include a threshold stopping distance. In such examples, the vehicle computing system may determine that stopping the vehicle before the object is not a viable option, but an adjustment in position may be used to avoid the object. For example, the vehicle computing system may determine that an object is 15 feet in front of a vehicle traveling 25 miles per hour, and that the vehicle cannot stop ahead of the object at its maximum rate of deceleration. Thus, the vehicle computing system may determine the lateral acceleration required to avoid the object by swerving into an adjacent lane.

[0017] The techniques discussed herein can improve autonomous vehicle control in several ways. For example, conventional autonomous vehicle controllers may rely heavily on LIDAR to detect and avoid objects in the environment. Particulate matter in the environment, such as that generated by exhaust, steam, or environmental factors such as evaporation or sublimation, may generate light beam returns associated with the LIDAR. The returns may cause the LIDAR to detect objects that may otherwise be unidentifiable to the autonomous vehicle controller. To avoid the unidentified object, the autonomous vehicle's controller may brake and / or stop the vehicle to avoid the object, which may result in erratic and / or inconsistent driving. In contrast, the techniques described herein detect the unidentified object, determine a probability that the object is associated with particulate matter, and control the vehicle based on the probability. Thus, the techniques described herein result in improved control of the autonomous vehicle by making it more effective and efficient. Additionally, by relying on the probabilities generated utilizing many detected features in the environment, the vehicle computing system may improve safety associated with autonomous vehicle operation.

[0018] The techniques described herein can be implemented in several ways. Exemplary implementations are presented below with reference to the following figures. Although discussed in the context of an autonomous vehicle, the methods, apparatus, and systems described herein can be applied to a variety of systems (e.g., sensor systems or robotic platforms) and are not limited to autonomous vehicles. In one example, the techniques described herein may be utilized in a driver-controlled vehicle, and such a system may provide an indication to the driver of the vehicle of whether it is safe to perform various maneuvers. In another example, these techniques can be utilized in an aviation or nautical context, or in any system that includes objects or entities that may be associated with behaviors unknown to the system. In addition, the techniques described herein can be used with real data (e.g., captured using sensors), simulated data (e.g., generated by a simulator), or any combination of the two.

[0019] 1 is a diagram of an environment 100 in which one or more computing systems 102 of an autonomous vehicle 104 (e.g., the vehicle 104) may detect an object 106 and control the vehicle 104 based on a probability that the object 106 is associated with an area of ​​particulate matter 108, according to an example of the disclosure. The autonomous vehicle 104 may detect the object 106 based on sensor data captured by one or more sensors 110 of the vehicle 104. The sensor data captured by the sensors 110 may include data captured by a LIDAR sensor, as well as data captured by other sensors, such as camera sensors, RADAR sensors, time-of-flight sensors, and sonar sensors. In at least one example, the sensor data may include LIDAR data including an optical signal reflected off the object 106. In such an example, the optical signal may reflect off water molecules present in the one or more areas of particulate matter 108.

[0020] In some examples, the sensor data can be provided to a perception component 112 configured to determine a classification associated with the object 106 (e.g., vehicle, pedestrian, bicycle, motorcycle, animal, parked car, tree, building, area of ​​particulate matter, etc.). In various examples, the perception component 112 may determine the object classification based on one or more features associated with the object 106. The features may include the size (e.g., width, height, depth, etc.) of the object 106, the shape (e.g., geometric shape, symmetry, etc.), the location of the object 106 (e.g., proximity to a surface, height from a surface, etc.), the trajectory of the object 106 (e.g., movement over time, speed, direction, etc.), proximity to another object 106, or proximity to a location on another vehicle (e.g., a location associated with exhaust, etc.). For example, perception component 112 may recognize the shape, size, and trajectory associated with objects 106(4) and 106(5) and may determine that objects 106(4) and 106(5) are vehicles. In another example, perception component 112 may determine that the shape, location, size, and trajectory associated with objects 106(1), 106(2), and 106(3) are associated with an area of ​​particulate matter.

[0021] In various examples, the perception component 112 may utilize one or more machine-learned models configured to output a classification based on features associated with the object 106. In some examples, the machine-learned models may be configured to output a probability that the object is associated with a classification (e.g., a confidence that the object is properly classified). In such examples, the machine-learned models may be trained utilizing training data that includes annotated features associated with regions of particulate matter 108, such as those captured by one or more vehicles 104 in pre-recorded sensor data. For the purposes of this discussion, we focus on determining the probability that objects 106, such as objects 106(1), 106(2), and 106(3), are associated with regions of particulate matter. However, the perception component 112 may be configured to make the same or similar confidence (e.g., probability) determinations for objects 106 of other classifications.

[0022] In various examples, the probability that the object 106 is associated with a region of particulate matter 108 (e.g., a cloud of particulate matter, an area of ​​particulate matter, etc.) may be based on features associated with the object 106. For example, the perception component 112 may detect an object 106(1) in the environment and determine that the bottom edge of the object 106(1) is at a height (H) of approximately 6 inches (15.24 cm) above the drivable road surface 114 and within a threshold distance to another vehicle (e.g., object 106(4)) operating in front of the vehicle 104 on the same road segment as the vehicle 104. Based on the height characteristic and / or the threshold distance to the other vehicle, the vehicle computing system may determine that there is a high (e.g., 93%) probability that the object is a region of particulate matter. In some examples, the probability may be increased based on a number of features that match those associated with the region of particulate matter 108.

[0023] In various examples, the probability that the object 106 is associated with the region 108 of particulate matter may be based on map data accessed by the perception component 112. The map data may represent information about the environment 100 based in part on pre-recorded sensor data from one or more vehicles. The map data may include data associated with drivable surfaces 114 (e.g., streets, roads, sections of roads, intersections, etc.), known obstacles (e.g., fixed objects in the environment, etc.), and / or other surfaces in the environment (e.g., sidewalks, etc.). In various examples, the map data may further include known areas of particulate matter emissions, such as areas 116. The known areas of particulate matter emissions 116 may include manhole covers, dryer vents (e.g., industrial dryers, domestic dryers, etc.), etc. In such examples, the probability that the object is associated with the region of particulate matter may be based in part on the known areas of particulate matter emissions. For example, the vehicle computing system may identify the object at a location in the environment, such as a sewer drain, etc. For example, multiple vehicles may travel within environment 100 and capture sensor data at a location that includes an area 108(3) of particulate matter emitted from drivable surface 114. A computing system configured to generate map data (e.g., a vehicle computing system, a remote computing system, etc.) may determine that the area 108(3) of particulate matter is associated with a known area 116 of particulate matter emissions based on the sensor data captured from the multiple vehicles.

[0024] In some examples, the perception component 112 may compare locations associated with the object 106 and determine that there is an increased probability that the object 106 is associated with the region of particulate matter 108 based on proximity to the known area of ​​particulate matter emission 116. For example, the perception component 112 may detect an object 106(3) in the environment 100. The perception component 112 may access map data to determine that locations associated with the known area of ​​particulate matter emission 116 are proximate (e.g., within a threshold distance, the source of emission, etc.) to the object 106(3). The perception component 112 may determine that there is an increased probability (e.g., 90%, 95%, etc.) that the object 106(3) is associated with the region of particulate matter 108(3) based in part on the known area of ​​particulate matter emission 116.

[0025] In various examples, the probability that an object 106 is associated with a region of particulate matter 108 may be based on a trajectory 118 associated therewith. In various examples, the perception component 112 may collect perception data over a period of time and determine a trajectory 118 associated with the object 106. The trajectory 118 may include a detected movement of the object 106 over a period of time, e.g., a distance traveled, a speed, etc. The perception component 112 may determine the probability based in part on the trajectory 118. In an illustrative example, the perception component 112 may detect the object 106(2) as the object 106(2) moves over time along the trajectory 118. Based in part on the movement of the object 106(2), the perception component 112 may determine that the object 106(2) is associated with the region of particulate matter 108(2).

[0026] In various examples, the probability may additionally or alternatively be based in part on the shape of the object 106(2) along the trajectory 118 and / or the size of the object 106(2). In such examples, the perception component 112 may identify a dissipation of particulate matter and a decrease in size of the object 106(2) over time. In various examples, the perception component 112 may determine a probability that the object 106(2) is associated with the region of particulate matter 108(2) based on the shape of the object 106(2) along the trajectory 118 and / or the size of the object 106(2).

[0027] In various examples, the planning component 120 of the vehicle 104 may be configured to receive data from the perception component 112 and control the vehicle 104 based on the data. In various examples, the data may include a classification of the object 106 as a region of particulate matter 108 and / or a probability that the object 106 is associated with the region of particulate matter 108. In various examples, the planning component 120 may determine an actuation to take based on the probability that the object 106 is associated with the region of particulate matter 108. The actuation may include maintaining a course and speed (e.g., a trajectory) associated with the path of the vehicle 104, slowing the vehicle 104 to a predetermined speed (e.g., 10 miles per hour, 20 kilometers per hour, etc.), stopping the vehicle 104, changing lanes, and / or adjusting a position within the lane. In various examples, the planning component 120 may determine one or more trajectories for the vehicle 104 to travel based on the actuation.

[0028] In various examples, an action may be determined based on the probability being higher and / or lower than one or more threshold probabilities. In some examples, the thresholds may be associated with various levels of probability, such as a high probability, a medium probability, and a low probability. In some examples, each level of probability (e.g., a probability higher / lower than a different threshold) may have one or more actions associated therewith. For example, based on a determination that there is a high probability (e.g., a probability equal to or greater than a first threshold) that the object 106(3) is associated with the region of particulate matter 108(3), the planning component 120 may determine that the action includes maintaining the course and speed of the vehicle 104 (e.g., maintaining a trajectory). In another example, based on a determination that there is a low probability (e.g., a probability equal to or less than a second threshold) that the object 106(3) is associated with the region of particulate matter 108(3), the planning component 120 may determine that the action includes stopping the vehicle. In yet another example, based on a determination that there is a medium probability (e.g., a probability below a first threshold and above a second threshold) that the object 106(3) is associated with the region of particulate matter 108(3), the planning component 120 may determine to slow the vehicle to a predetermined speed (e.g., 15 kilometers per hour, 9 miles per hour, etc.).

[0029] In various examples, the planning component 120 may be configured to determine an actuation for the vehicle 104 to choose based on a function corresponding to the probability that the object 106 is associated with a region of particulate matter, the size of the region, the distance to the region, the vehicle speed, the width of a lane associated with the vehicle trajectory, and / or additional buffers (e.g., safety buffers, distance to maintain from the object 106, etc.), and / or any other factors associated with the vehicle traveling in the environment. In some examples, one or more factors (e.g., probability, size, distance, vehicle speed, etc.) may be weighted more heavily than other factors, e.g., to have a greater impact on the actuation selection. For example, the size of the region of particulate matter may be weighted more heavily than the distance to the region. Thus, size may have a greater impact on the actuation selection than distance to the region.

[0030] In various examples, the planning component 120 may be configured to determine two or more maneuvers that the vehicle 104 may take based on a probability, and may select the maneuver based on a cost analysis associated with each maneuver, such as utilizing techniques described in U.S. Patent Application Publication No. 2010 / 0133994, filed Aug. 13, 2019, incorporated herein above. In at least some such examples, for example, such costs may include costs based on available width in the lane (e.g., such that costs grow exponentially with respect to vehicle width and / or additional buffers). As a non-limiting example, the cost may be a function of the width of the lane relative to a threshold width, polynomially, exponentially, or in other forms. In some examples, such costs may be based on a distance to the nearest object, which may include a boundary of the drivable surface.

[0031] In some examples, the cost associated with safety (e.g., safety cost) may be based on a probability of a collision between the vehicle 104 and the object 106. In some examples, the safety cost may include a fixed cost (e.g., 60, 80, 100, etc.) when the probability of a collision is higher than a predefined threshold (e.g., 40%, 51%, etc.) indicating that a collision may occur. In some examples, the fixed cost may include a cost value that is above a threshold and thus prevents the vehicle computing system from selecting an action associated therewith. In such examples, the probability of a collision may include a constraint on the actions that the vehicle 104 may take. In various examples, the safety cost may include a function of the probability that the object 106 is associated with the region 108 of particulate matter. For example, the safety cost may increase as the probability that the object 106 is associated with the region 108 of particulate matter decreases. In some examples, the safety cost may be based on the relative positions and / or trajectories / trajectories of the vehicle 104 and the object 106. In such an example, the safety cost may increase as the vehicle 104 (traveling on the vehicle track) approaches the object 106 .

[0032] In various examples, the comfort cost may be associated with an estimated acceleration (e.g., positive, negative, lateral, etc.) and / or an estimated change in acceleration (e.g., jerk) associated with the vehicle trajectory determined in response to the detection of the object 106. In some examples, the comfort cost may be associated with a distance between the vehicle 104 performing the associated actuation and the object 106. In such examples, the comfort cost may be associated with a closest point of approach between the vehicle 104 and the object 106. In some examples, the comfort cost may be associated with a passing speed associated with the actuation. In some examples, the comfort cost may be associated with a relative speed between the vehicle 104 and the object 106. In other examples, any other state / control of the vehicle 104 and / or the considered object 106 may be used to determine the comfort cost (e.g., proximity of other vehicles, which may be calculated as a nearest neighbor or next nearest neighbor, steering rate, turning rate, speed, etc.).

[0033] In various examples, the cost of travel may include the progress of the vehicle 104 traveling toward the destination. In some examples, the cost of travel may be calculated based on changes in the speed of the vehicle 104 and / or delays of the vehicle 104 caused by stopping, slowing down, etc. In some examples, the cost of travel may include a value substantially similar to the number of seconds of delay due to the actuation. In such examples, an actuation that includes slowing and / or stopping the vehicle before a location associated with the object 106 may be associated with a higher cost of travel than one associated with an actuation that includes navigation that avoids the object 106.

[0034] In various examples, the operational rule cost may be based on rules of the road (e.g., Department of Transportation laws, rules, regulations, etc.), rules of good driving, local driving habits (e.g., general driving practices), driving etiquette (e.g., adjusting position in a lane to provide space for another vehicle to pass and turn right without occupying a bike lane, etc.). In various examples, the operational rule cost associated with an actuation may be calculated based on one or more rules that have been broken and / or fulfilled. In such examples, the operational rule cost may be increased based on a rule that has been broken or not fulfilled (e.g., a regulation, custom, etiquette, etc.) and / or decreased based on a rule that has been fulfilled or fulfilled. For example, an actuation associated with changing lanes to oncoming traffic may include breaking a rule that the vehicle 104 should remain in a lane (e.g., original vehicle path, trajectory, etc.) designated for traffic maneuvering in a direction associated with the planned vehicle trajectory. This actuation may involve a higher operational rule cost than an actuation that involves adjusting the position of the vehicle 104 in a lane.

[0035] In various examples, the operational rule cost may be determined based on rules, regulations, etc. associated with a lane marker that corresponds to the lane in which the vehicle is traveling (e.g., during detection of object 106). In such examples, the color of the lane marker and / or whether the lane marker is a solid and / or dashed line may contribute to the operational rule cost associated with the actuation. For example, an actuation associated with crossing a double yellow line may have a high operational rule cost associated with it (e.g., 40, 50, etc.). In another example, a dashed white line may have a low operational rule cost associated with it (e.g., 5, 10, etc.).

[0036] In various examples, the costs may be ranked or weighted in order of importance. In some examples, at least one of the costs (e.g., safety, comfort, advancement, operating rules) may include a cost that is weighted more heavily than other costs. For example, safety may be weighted more heavily than other factors. In such examples, the planning component 120 may emphasize safety over other factors in determining the operation for the vehicle 104 to select.

[0037] In some examples, the planning component 120 may determine which action to choose based on the lowest cost associated therewith. In such examples, the action may be based on cost optimization. For example, the planning component 120 may receive an indication that there is a 70% probability that the object 106(1) is associated with the region of particulate matter 108(1). The planning component 120 may determine that the 70% probability is lower than a first threshold probability and higher than a second threshold probability. Based on this determination, the planning component 120 may determine that the vehicle 104 may take a first action including slowing down to a predetermined speed or a second action including changing lanes to avoid the object 106(1). The planning component 120 may determine that a cost associated with slowing down the vehicle 104 is lower than a cost associated with changing lanes and may slow the vehicle 104 to a predetermined speed based on the cost analysis.

[0038] In various examples, the perception component 112 may provide the planning component 120 with a distance (D) between the vehicle 104 and the object 106(1) in addition to a probability that the object 106(1) is associated with the region 108(1) of particulate matter. In various examples, the planning component 120 may determine that the distance (D) exceeds a threshold distance (e.g., 150 feet, 80 meters, etc.). In some examples, the threshold distance may include a predetermined threshold distance programmed into the computing system 102. In some examples, the threshold distance may be dynamically determined based on one or more factors, such as, for example, vehicle speed, a speed limit associated with the drivable surface 114, a number of detected objects 106, a type of object 106 detected within the environment 100, weather (e.g., rain, snow, fog, etc.), or an area or zone associated with the environment (e.g., a school zone, a residential area, a construction zone, etc.), etc.

[0039] In some examples, based on a determination that the distance (D) is greater than a threshold distance, the planning component 120 may determine that the probability corresponds to the initial probability. In various examples, based on a determination that the initial probability is equal to or greater than a threshold probability (e.g., 91%, 99%, etc.), the planning component 120 may cause the vehicle 104 to maintain the trajectory. In such examples, the vehicle 104 may maintain a course and speed associated with a planned vehicle path through the environment 100. For example, the planning component 120 may determine with a high probability that the object 106(1) is associated with the region of particulate matter 108(1). Thus, the planning component 120 may maintain the vehicle trajectory to operate through the region of particulate matter 108(1).

[0040] In some examples, based on a determination that the initial probability is equal to or less than the threshold probability, the planning component 120 may cause the vehicle 104 to reduce (e.g., slow down) its speed. In such examples, the reduced speed may provide time for the perception component 112 to gather additional data regarding the object 106 to determine an updated probability. In some examples, the speed may be dynamically determined based on one or more factors (e.g., vehicle speed, speed limit, weather, etc.) as described above. In various examples, the speed may include a predetermined speed (e.g., 10 miles per hour, 8 miles per hour, 18 kilometers per hour, etc.) and is programmed into the computing system 102, for example. In some examples, the predetermined speed may be based on one or more safety factors, such as minimized impact on occupants of the vehicle 104 or the object 106, to maximize safe operation of the vehicle 104.

[0041] In various examples, the planning component 120 may receive the updated probability after a period of time (e.g., 0.1 seconds, 0.5 seconds, 1 second, etc.). In some examples, the updated probability may be based on data captured between a first time associated with the initial probability and a second time associated with the updated probability. In some examples, the updated probability may be associated with data captured at a second time for a portion of the distance (D). In various examples, the planning component 120 may compare the updated probability to one or more thresholds to determine an action for the vehicle 104 to take. For example, based on a determination that the updated probability is equal to or greater than an upper threshold, the planning component 120 may accelerate the vehicle back to an initial speed (e.g., a speed before deceleration, a speed associated with the planned vehicle path through the environment 100). In another example, based on a determination that the updated probability is equal to or less than an upper threshold and the adjacent lane is clear, the planning component 120 may determine to have the vehicle 104 change lanes into the adjacent lane, such as to avoid the object 106(1). In yet another example, based on a determination that the updated probability is less than or equal to a lower threshold, planning component 120 may stop the vehicle ahead of object 106(1) (e.g., to avoid a collision with object 106(1)).

[0042] In various examples, planning component 120 may receive a distance (D) between vehicle 104 and object 106(1) and may determine that the distance (D) is less than a threshold distance and therefore insufficient to obtain additional data and determine an updated probability. In some examples, planning component 120 may receive a probability that object 106(1) is associated with region 108(1) of particulate matter. Based on a determination that the probability is less than a threshold probability (e.g., a low probability), planning component 120 may determine to stop the vehicle prior to object 106(1), such as to avoid a collision with object 106(1).

[0043] In various examples, the planning component may determine a deceleration rate (e.g., a negative acceleration) (e.g., −3 miles per hour per second, −5 miles per hour per second, etc.) based on the distance (D) between the vehicle 104 and the object 106(1). The deceleration rate may be associated with pressure applied to one or more brakes of the vehicle 104, regenerative braking, downshifting, engine braking, or any other method of stopping the vehicle 104 before reaching a location associated with the object 106(1). By determining the deceleration rate based on the distance (D), the planning component 120 can ensure that the vehicle 104 does not contact the object 106(1), thereby significantly improving the safe operation of the vehicle 104. For example, the planning component 120 may determine that the distance (D) between the vehicle and the object 106(1) is 35 feet and that there is a low probability that the object 106(1) is associated with the region of particulate matter 108(1). The planning component 120 may slow the vehicle 104 at a rate of 15 feet per second to stop it in front of the object 106(1).

[0044] In some examples, planning component 120 may be configured to slow the vehicle at a maximum deceleration rate (e.g., 20 feet per second per second, 25 feet per second per second, etc.). In such examples, the maximum deceleration rate may be based on a maximum amount of pressure that can be applied to the brakes, maximum regenerative braking, minimum downshifts, maximum engine braking, etc., to, for example, slow vehicle 104 as far ahead of a location as possible to minimize damage to vehicle 104 and / or object 106(1).

[0045] In various examples, based on a determination that the probability that object 106(1) is associated with region 108(1) of particulate matter falls below a threshold, planning component 120 may determine to adjust position laterally to avoid object 106(1), etc. In some examples, the decision to move laterally may be based on a determination that the lateral path is clear of object 106, such as another vehicle, motorcycle, or bicyclist. In such examples, the determination that the lateral path is clear may be based on sensor data and / or an indication from the perception system that at least a threshold area of ​​the adjacent lane is clear of object 106. The threshold area may be associated with an area required to transition into the adjacent lane without coming within a threshold distance in front of or behind object 106. For example, the threshold area may ensure that vehicle 104 does not obstruct another vehicle operating in the adjacent lane. In another example, the threshold area may ensure that the vehicle 104 does not change lanes to follow behind another vehicle at an uncomfortable distance (eg, less than five feet).

[0046] In some examples, the decision to move laterally, such as to avoid a path that may intersect with object 106(1), may be based on the distance (D) and / or speed of vehicle 104. In such examples, if the distance (D) between vehicle 104 and object 106 is less than a threshold and / or less than the distance required to stop vehicle 104 (traveling at a particular speed) before reaching a location associated with object 106(1), planning component 120 may move vehicle 104 laterally. For example, planning component 120 may determine that the distance (D) to object 106(1) is 15 feet in front of vehicle 104 traveling at 20 miles per hour and that vehicle 104 cannot stop in front of object 106(1) at a maximum deceleration rate. Thus, planning component 120 may determine the lateral acceleration required to move into an adjacent lane to avoid object 106(1). In some examples, the lateral acceleration, maximum deceleration rate, and / or the decision to move laterally may be based on a cost-based analysis as described above. In such examples, the decision to steer the path of object 106(1) and / or the lateral acceleration associated with the movement may be based on one or more of a safety cost, a comfort cost, a progress cost, and / or an operating rule cost associated with the operation.

[0047] In various examples, planning component 120 may determine a lateral acceleration (e.g., 2 meters per second, 8 feet per second, etc.) associated with a lateral movement such as that required to avoid object 106(1). The lateral acceleration may be based on the distance (D) and / or the speed of vehicle 104. In such examples, the closer object 106(1) is to vehicle 104 and / or the faster vehicle 104 is traveling in environment 100, the greater the lateral acceleration required to avoid object 106(1).

[0048] FIG. 2 is an example process 200 for controlling a vehicle 104 based on a probability that an object 106(1) in the path of the vehicle is associated with a region 108(1) of particulate matter, according to an example of the disclosure.

[0049] Operation 202 includes detecting objects 106(1) and 106(4) in environment 100. In various examples, objects 106(1) and 106(4) may be detected based on data captured from one or more sensors. The sensor data captured by the sensors may include data captured by a LIDAR sensor as well as data captured by other sensors, such as camera sensors, RADAR sensors, time-of-flight sensors, and sonar sensors. In at least one example, the sensor data may include LIDAR data including optical signals reflected off objects 106(1) and 106(4). In some examples, the optical signals may reflect off water molecules present in region 108 of particulate matter.

[0050] In various examples, a vehicle computing system, such as computing system 102, may be configured to determine a classification associated with objects 106(1) and 106(4). The classification may include a class or type associated with objects 106(1) and 106(4), e.g., pedestrian, car, truck, bicyclist, moped, train, area of ​​particulate matter (e.g., cloud-like), etc. As an illustrative example, the vehicle computing system may identify object 106(4) as a car and object 106(1) as an area of ​​particulate matter. In some examples, the vehicle computing system may process LIDAR returns (e.g., light reflecting off object 106(1)) but may be unable to determine a classification associated with an object, such as object 106(1). In such examples, based on the lack of a classification, the vehicle computing system may determine that object 106(1) may include area of ​​particulate matter 108(1).

[0051] Operation 204 may include determining a probability that the object is associated with the region of particulate matter 108(1). In various examples, the probability may include a confidence that the object 106(1) is properly classified as the region of particulate matter 108(1). In such examples, the vehicle computing system may determine the confidence (e.g., probability) substantially simultaneously with the classification. In some examples, the probability may represent a probability that the object 106(1) includes the region of particulate matter 108(1) in the absence of classification. In such examples, the probability may include a likelihood that the object 106(1) is particulate matter 108(1).

[0052] In various examples, the vehicle computing system may determine the probability based on one or more features associated with the object 106(1). The features may include the location of the object 106(1) in the environment (e.g., proximity to a surface, height from a surface, etc.), the size of the object 106(1) (e.g., width, height, depth, etc.), the shape (e.g., symmetric, asymmetric, circular, rectangular, etc.), the proximity of the object 106(1) to vehicles such as the object 106(4) (e.g., proximity to exhaust-emitting vehicles (gasoline or diesel cars, trucks, etc.)), the proximity to locations on other vehicles (e.g., locations associated with exhaust, etc.), or the trajectory (e.g., movement captured over time) of the object 106(1), etc. For example, the vehicle computing system may detect the object 106(1) and determine that there is an 80% probability that the object 106(1) is associated with the region of particulate matter 108(1) based on the cloud-like shape and the proximity to the object 106(4).

[0053] In some examples, the vehicle computing system may determine the probability based on map data associated with the environment 100. The map data may represent information about the environment 100 in which the vehicle operates. In some examples, the map data may be generated based on pre-recorded sensor data from one or more vehicles 104. The map data may include data associated with drivable surfaces (e.g., streets, roads, sections of roads, intersections, etc.) and / or known obstacles in the environment (e.g., fixed objects in the environment, etc.). In some examples, the map data may further include known areas of particulate matter emissions, such as manhole covers and dryer vents (e.g., industrial dryers, domestic dryers, etc.). In such examples, the probability that the object 106(1) is associated with the region of particulate matter 108(1) may be based in part on the known areas of particulate matter emissions.

[0054] In various examples, the vehicle computing system may utilize machine learning techniques to determine the probability. In such examples, the machine learned model may be trained with training data to determine the probability that object 106(1) is associated with region of particulate matter 108(1). In some examples, the training data may include pre-recorded data annotated with identified regions of particulate matter, such as region of particulate matter 108(1). In some examples, the annotations may include features. In various examples, the machine learned model may be trained to identify region of particulate matter 108(1) based on map data as described above.

[0055] The operations 206 include determining an action 210 for the vehicle to take based at least in part on the probability. In various examples, the vehicle computing system can determine a trajectory (e.g., an initial trajectory, a first trajectory, etc.) for the vehicle, which is determined, for example, at a time associated with the initial position 208. The initial trajectory may include a direction of travel and a speed (or speeds) associated with the vehicle traveling through the environment, such as along a route to the destination. In various examples, the action 210, such as the first action 210(1), may include maintaining the initial trajectory. In some examples, the vehicle computing system may cause the vehicle to take the first action 210(1) based on a determination that a probability that the object 106(1) is associated with the region 108(1) of particulate matter exceeds a first threshold probability (e.g., 95%, 98%, etc.). In such examples, the vehicle computing system may determine that the vehicle 104 may be able to safely proceed along the initial trajectory to the destination based on a high probability that the detected object 106(1) includes particulate matter.

[0056] In various examples, operations 210, such as operations 210(2), 210(3), and 210(4), may include modifying the trajectory to a second (modified) trajectory. In an illustrative example, second operation 210(2) includes modifying the trajectory by reducing a speed associated therewith (e.g., compared to the initial trajectory). In various examples, the modified (or second) speed associated with the modified trajectory may include a predetermined speed (e.g., 9 miles per hour, 15 kilometers per hour, etc.). In such examples, the predetermined speed may be associated with a safety protocol for vehicle operation, thereby maximizing safety of the vehicle, vehicle occupants, and others operating within environment 100. In some examples, the speed associated with the modified trajectory may be dynamically determined based on one or more factors. The factors may include the speed of the vehicle 104, the speed limit associated with the environment 100, the number of objects 106 detected, the type of objects 106 detected in the environment 100, the weather (e.g., rain, snow, fog, etc.), or the area or zone associated with the environment 100 (e.g., school zone, residential area, construction zone, etc.). In such an example, the vehicle computing system may determine a safe speed to approach the object 106(1) in the environment 100, thereby ensuring safe operation in any conditions (e.g., different environments, weather, etc.). For example, the vehicle computing system may detect the object 106(1) and determine that the vehicle 104 is in a school zone. Based on the school zone, the vehicle computing device may slow the vehicle down to 7 miles per hour to allow the vehicle to continue moving forward, but may be able to stop a very short distance if necessary for safety reasons.

[0057] In some examples, the vehicle computing system may determine to perform one or more of the second action 210(2), the third action 210(3), and / or the fourth action 210(4) based on a cost analysis. In such examples, the vehicle computing system may calculate a cost associated with each action (e.g., a safety cost, a comfort cost, a progress cost, and / or an operating rule cost, etc.) and may select to perform the action with the lowest cost. In some examples, the vehicle computing system may select an action for the vehicle 104 to select based at least in part on cost optimization.

[0058] In various examples, the vehicle computing system may determine to perform the second action 210(2) based on a determination that the probability is less than the first threshold probability. In some examples, the second action 210(2) may be performed in response to a determination that the probability is less than the first threshold distance and greater than or equal to a second threshold (e.g., 50%, 60%, etc.). In such examples, the vehicle computing may slow the vehicle 104 based on a determination that there is a medium probability that the object 106(1) is associated with the region of particulate matter 108(1).

[0059] In various examples, the vehicle computing system may determine to perform the second operation 210(2) based in part on the distance (D) between the vehicle 104 and the object 106(1). In some examples, based on a determination that the distance (D) exceeds a threshold distance (e.g., 100 feet, 100 meters, etc.), the vehicle computing system may cause the vehicle 104 to perform the second operation 210(2). In such examples, the vehicle computing system may be configured to collect additional data regarding the object 106(1) to determine an updated probability that the object 106(1) is associated with the region of particulate matter 108(1). In some examples, the vehicle computing system may determine a high confidence that the object 106(1) is associated with the region of particulate matter 108(1) based on the updated probability. Based on the updated probability, the vehicle computing system may cause the vehicle 104 to resume an initial trajectory and / or determine an updated trajectory for the vehicle 104 to travel to a destination within the environment 100. For example, the vehicle computing system may determine that the updated trajectory exceeds a first threshold probability and may accelerate the vehicle to a speed associated with the initial trajectory and / or another trajectory associated with the vehicle's 104 progression toward the destination.

[0060] In various examples, the vehicle computing system may slow the vehicle at a predetermined deceleration rate (e.g., 3 feet per second per second, 2 meters per second per second, etc.) from the initial trajectory to the revised trajectory associated with the second actuation 210(2). The deceleration rate may be associated with the amount of pressure applied to the vehicle's brakes, regenerative braking, downshifting, engine braking, or any other method of slowing the vehicle down to a speed associated with the revised trajectory. In some examples, the vehicle computing system may determine the deceleration rate based on a probability. For example, the vehicle computing system may slow the vehicle by 5 feet per second per second based on a determination that the probability is 50% and by 3 feet per second per second based on a determination that the probability is 80%. In at least some examples, such deceleration may not be predetermined but may be based at least in part on a probability, the relative location of the steam, etc.

[0061] In addition to or instead of slowing the traveling speed of the vehicle 104, the vehicle computing system may change the driving direction associated with the initial trajectory, as shown as third action 210(3). The third action 210(3) is shown as a lane change, but may include any other safe lateral maneuver that enables navigation to avoid the object 106(1), such as modifying the position in the lane, or operating partially or completely on the shoulder. In some examples, the vehicle computing system may cause the vehicle 104 to perform the third action 210(3) based on the probability falling below a first threshold probability. In some examples, the third action 210(3) may be selected based on the probability falling below a first threshold and above a second threshold. In such examples, the vehicle computing system may cause the vehicle to perform the third action 210(3) based on a medium probability that the object 106(1) is associated with the region of particulate matter 108(1). Although described herein in terms of thresholds, of course, this is not meant to be so limiting. For example, while some examples may set fixed thresholds for decision making, it should be understood that the discussion of thresholds herein may also refer to cost formulas set such that a transition occurs between two outcomes based on optimizing for cost near such thresholds. In various examples, a combination of hard constraints (e.g., fixed thresholds for decisions) and thresholds that represent transitions at cost optimization are contemplated.

[0062] In various examples, the vehicle computing system may determine that the lateral path (e.g., adjacent lane) is clear of other vehicles, motorcyclists, bicyclists, etc. In some examples, the determination that the lateral path is clear may be based on sensor data and / or an indication from the perception system that at least a threshold area 212 of the adjacent lane (or other operating area, such as a shoulder) is clear (e.g., unoccupied). In some examples, the threshold area 212 may be associated with an area required to transition into the adjacent lane without coming within a threshold distance in front of or behind the object 106. For example, the threshold area 212 may ensure that the vehicle 104 does not impede (e.g., block, etc.) the movement of another vehicle operating in the adjacent lane. In some examples, confirming that the threshold area 212 is clear may ensure that the vehicle 104 does not encroach on other drivers (e.g., other vehicles, motorcyclists, bicyclists, etc.), such as when the vehicle 104 adjusts its position in the lane to be closer to the adjacent lane. In such instances, the techniques described herein may ensure that the impact of vehicle 104 travel on other drivers is minimal, both in terms of safety and comfort.

[0063] In various examples, the vehicle computing system may determine a lateral acceleration (e.g., 1 meter per second per second, 5 feet per second per second, etc.) associated with the third actuation 210(3). In some examples, the lateral acceleration may include a predetermined lateral acceleration (e.g., 3 feet per second per second, 1 meter per second per second, etc.) such as specified for a lane-changing maneuver. In some examples, the lateral acceleration may be dynamically determined based on one or more factors associated with the environment 100, such as the speed of the vehicle 104, a speed limit, weather, or an area or zone. In various examples, the lateral acceleration may be based on the distance (D) between the vehicle 104 and the object 106(1) and / or the speed of the vehicle 104. In such examples, the closer the object 106(1) is to the vehicle 104 and / or the faster the vehicle 104 is traveling in the environment 100, the greater the lateral acceleration.

[0064] In various examples, the vehicle computing system may cause the vehicle 104 to modify its initial trajectory to stop the vehicle 104 prior to a location associated with the object 106(1), for example, as shown in FIG. 2 as fourth action 210(4). In some examples, the decision to perform the fourth action 210(4) may be based on the probability falling below a second (or lower) threshold (e.g., 25%, 10%, etc.). In such examples, based on a determination that there is a low probability that the object 106(1) is associated with the region of particulate matter 108(1), the vehicle computing system may stop the vehicle 104 prior to a location associated with the object 106(1).

[0065] In various examples, the vehicle computing system may determine a deceleration rate (e.g., a negative acceleration) of the vehicle 104 (e.g., -6 miles per hour per second, -10 miles per hour per second, etc.). In some examples, the deceleration rate may include a predetermined rate. The predetermined rate may include a deceleration rate stored in the vehicle computing system for use when the vehicle 104 should be stopped prior to a location associated with the object 106(1). In some examples, the predetermined rate may include a fixed deceleration rate. In some examples, the vehicle computing system may cause a fixed amount of pressure to be applied to the brakes based on the decision to perform the fourth act 210(4). In some examples, the vehicle computing system may effect a predetermined amount of engine braking (e.g., a predetermined number of revolutions per minute, etc.), or a predetermined downshift (e.g., one gear down, two gears down, etc.), or the like. In some examples, the predetermined rate may be based on the distance (D) and / or the speed of the vehicle 104. For example, for a distance of 20 feet and a speed of 25 miles per hour, the vehicle computing device may determine that the deceleration rate should be 20 feet per second per second.

[0066] In various examples, the deceleration rate may be dynamically determined based on the distance (D) and one or more factors associated with the environment 100, as described herein. In such examples, the vehicle computing device may process data associated with the factors and determine a deceleration rate appropriate for the environment 100. For example, the vehicle computing device may determine that the drivable surface of the road is wet (e.g., rain, sprinkler overflow, etc.) and that the conditions may require a deceleration rate of 15 feet per second per second to stop the vehicle 104 (dry road conditions may require a lower rate).

[0067] In some examples, the deceleration rate may be based on whether the vehicle is occupied. In such examples, the vehicle computing system may determine that one or more occupants are present in the vehicle 104, such as via one or more sensors in the passenger compartment, and adjust the deceleration rate based on the occupants. For example, the vehicle computing system may use an increased deceleration rate for an empty vehicle than an occupied vehicle, or vice versa. In some examples, the deceleration rate may be based on a type or class of occupant (e.g., human (e.g., adult, child, infant, etc.), animal (e.g., dog, cat, etc.), etc.). In such examples, the vehicle computing system may be configured to determine a class or type of occupant based on sensor data received from sensors in the passenger compartment and determine a deceleration rate based on the class or type. For example, the vehicle computing system may determine that the passenger compartment houses an adult and a dog. The vehicle computing system may determine that the deceleration rate may be lower than a deceleration rate associated with the vehicle 104 in a similar scenario without a dog occupant to prevent the dog from becoming a projectile.

[0068] In some examples, the vehicle computing system may be configured to slow the vehicle 104 at a maximum deceleration rate (e.g., 20 feet per second per second, 25 feet per second per second, etc.). In some examples, the maximum deceleration rate may be based on a maximum amount of pressure that can be applied to the brakes, a maximum downshift, an engine braking limit, or the like, e.g., to slow the vehicle 104 as far in advance of a location as possible to minimize damage to the vehicle 104 and / or the object 106(1). In some examples, the maximum deceleration rate may include a predetermined maximum rate. In some examples, the maximum deceleration rate may be associated with a predetermined maximum amount of pressure that can be applied to the brakes, a maximum downshift, a maximum amount of engine braking, or the like. In such examples, the maximum deceleration rate may be based on factors. For example, the maximum deceleration rate may be lower in wet and / or snowy conditions than in dry conditions. In some examples, the maximum deceleration rate may be determined based on whether the vehicle 104 is occupied. In such examples, the vehicle computing system may determine that one or more occupants are present in the vehicle 104 and may adjust the maximum deceleration rate based on the occupants. For example, the maximum deceleration rate may be higher for an empty vehicle 104 than for an occupied vehicle 104. In some examples, the maximum deceleration rate may be determined based on the type or class of occupants occupying the vehicle. In such examples, the vehicle computing system may be configured to maximize safety of occupants of the vehicle 104, such as by reducing the maximum deceleration rate for vulnerable occupants (e.g., young, frail, etc.).

[0069] In various examples, based on a determination that the vehicle 104 cannot stop before the location associated with the object, the vehicle computing system may cause the vehicle to perform a third action 210(3) as described above. For example, the vehicle computing system may determine that the vehicle cannot stop at the distance (D) at a maximum deceleration rate. As described above, the vehicle computing system may determine that the threshold area 212 is not occupied and may determine a lateral acceleration required to avoid (e.g., swerve) the object 106(1). In such examples, the lateral acceleration may be based on the distance (D) and factors associated with the environment 100, such as the speed of the vehicle 104, road conditions, weather, etc.

[0070] The vehicle computing system may select actuation 210 based on safety considerations for the occupants of the vehicle and / or others present in the environment (e.g., passengers of other vehicles, etc.). Thus, the techniques described herein may significantly improve the safety of autonomous vehicle operation.

[0071] 3 shows a block diagram of an example system 300 for implementing the techniques described herein. In at least one example, the system 300 may include a vehicle 302, such as the vehicle 104. The vehicle 302 may include an autonomous vehicle, a semi-autonomous vehicle, or a manually controlled vehicle.

[0072] The vehicle 302 may include a vehicle computing device 304, one or more sensor systems 306, such as the sensors 110 of FIG. 1, one or more emitters 308, one or more communication connections 310, at least one direct connection 312, and one or more drive systems 314. The vehicle computing device 304 may include one or more processors 316, and a memory 318 communicatively coupled to the one or more processors 316. In the illustrative example, the vehicle 302 is an autonomous vehicle, although the vehicle 302 may be any other type of vehicle or robotic platform having at least an image capture device (e.g., a camera-enabled smartphone). In the illustrated example, the memory 318 of the vehicle computing device 304 stores a localization component 320, a perception component 322, a planning component 324, one or more system controllers 326, and one or more maps 328. While shown in FIG. 3 as residing in memory 318 for illustrative purposes, the localization component 320, the perception component 322, the planning component 324, the one or more system controllers 326, and the one or more maps 328 may additionally or alternatively be accessible to the vehicle 302 (e.g., may be stored in memory remote from the vehicle 302 or may be accessible by the vehicle 302 in other form).

[0073] In at least one example, the localization component 320 may include functionality to receive data from the sensor system 306 to determine a position and / or orientation (e.g., one or more of an x ​​position, a y position, a z position, a roll, a pitch, or a yaw) of the vehicle 302. For example, the localization component 320 may include and / or request / receive a map of the environment and may continually determine the location and / or orientation of the autonomous vehicle within the map. In some examples, the localization component 320 may receive image data, LIDAR data, RADAR data, time-of-flight data, IMU data, GPS data, wheel encoder data, etc., utilizing SLAM (simultaneous localization and mapping), CLAMS (calibration, localization and mapping, simultaneously), relative SLAM, bundle adjustment, nonlinear least squares optimization, etc., to accurately determine the location of the autonomous vehicle. In some examples, the localization component 320 can provide data to various components of the vehicle 302 to determine an initial position of the autonomous vehicle, such as for generating a trajectory and for determining to retrieve map data, as discussed herein.

[0074] In some examples, the perception component 322 may include functionality for object detection, segmentation, and / or classification. In some examples, the perception component 322 may provide processed sensor data indicative of the presence of an entity or object proximate the vehicle 302 and / or the classification of the entity as an entity type (e.g., vehicle, pedestrian, bicyclist, animal, building, tree, road surface, curb, sidewalk, stop signal, stop sign, lane marker, vehicle light, unknown, etc.). In additional or alternative examples, the perception component 322 may provide processed sensor data indicative of one or more characteristics associated with a detected entity (e.g., a tracked object) and / or the environment in which the entity is located. In some examples, characteristics associated with an entity may include, but are not limited to, x-position (global and / or local position), y-position (global and / or local position), z-position (global and / or local position), orientation (e.g., roll, pitch, yaw), entity type (e.g., classification), velocity of the entity, acceleration of the entity, range (size) of the entity, etc. Characteristics associated with an environment may include, but are not limited to, the presence of another entity in the environment, the state of another entity in the environment, time of day, day of the week, season, weather conditions, dark / light indication, etc.

[0075] In examples where the perception component 322 performs detection, the perception component 322 may output detections of objects and / or object features detected in the image. Such detections may include two-dimensional bounding boxes (which may be used to later crop the image) and / or masks of the detected objects. In some examples, such detections may utilize machine learning techniques (e.g., Scale Invariant Feature Transform (SIFT), Histogram of Oriented Gradients (HOG), etc.) followed by a Support Vector Machine (SVM) to classify objects shown in images received from a camera and / or LIDAR data received from a LIDAR sensor of the sensor system 306. Alternatively or additionally, the detections may utilize deep learning techniques based on Convolutional Neural Networks (CNN) to classify objects shown in sensor data captured by the sensor system 306. In various examples, the perception component 322 may utilize machine learning techniques to determine the probability that a detected entity is associated with an area of ​​particulate matter. In such an example, the perception component may provide probabilities to the action determination component 330 of the planning component 324 to determine what action to take using the techniques described herein.

[0076] In general, the planning component 324 may determine a path for the vehicle 302 to follow to traverse an environment. For example, the planning component 324 may determine various routes and vehicle trajectories as well as various levels of detail. For example, the planning component 324 may determine a route for traveling from a first location (e.g., a current location) to a second location (e.g., a goal location). For purposes of this discussion, the route may include a sequence of waypoints for traveling between the two locations. As non-limiting examples, the waypoints may include roads, intersections, Global Positioning System (GPS) coordinates, and the like. Additionally, the planning component 324 may generate instructions for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 324 may determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints. In some examples, the instructions may be a vehicle trajectory, or a portion of a trajectory. In some examples, multiple trajectories can be generated substantially simultaneously (e.g., within technical tolerances) according to receding horizon techniques, and one of the multiple trajectories is selected for vehicle 302 to navigate.

[0077] In various examples, the actuation determination component 330 of the planning component 324 may receive data associated with the detected entity, such as a probability that the entity includes an area of ​​particulate matter. In such examples, the actuation determination component 330 may be configured to determine an actuation for the vehicle 302 to take based on the probability, as described herein. The actuation may include maintaining the trajectory or modifying the speed and / or direction of travel associated with the trajectory. In some examples, the modification may include slowing and stopping the vehicle 302 prior to a location associated with the detected entity. For example, based on a low probability that the entity includes particulate matter, the actuation determination component 330 may determine to stop the vehicle 302 before contacting the entity. In another example, based on a high probability that the entity includes particulate matter, the actuation determination component 330 may determine that the vehicle actuation should include maintaining the trajectory associated therewith.

[0078] Additionally, and as discussed above, the actuation may include slowing the vehicle 302 to capture additional data associated with the entity and determine an updated probability. In such an example, the actuation determination component 330 may determine that the entity is more than a threshold distance away from the vehicle 302. The actuation determination component 330 may cause the vehicle 302 to slow its speed (to a predetermined speed or a dynamically determined speed). The perception component 322 may collect additional data associated with the entity and may determine an updated probability. In response to receiving the updated probability from the perception component 322, the actuation determination component 330 may determine a second action to take, such as stopping the vehicle, changing lanes, or accelerating to a speed associated with the vehicle 302 traveling through the environment.

[0079] In various examples, the actuation determination component 330 may be configured to determine an acceleration. The acceleration may include a positive acceleration, such as associated with a determination to accelerate the vehicle 302 to a speed associated with traveling through the environment, or a negative acceleration, such as associated with a determination to slow and / or stop the vehicle. In some examples, the acceleration may include a predetermined acceleration associated with the vehicle 302, or the environment, such as based on one or more factors associated with the environment. For example, the predetermined acceleration may be set to 3 feet per second per second in school zones and 5 feet per second per second in areas not associated with schools.

[0080] In various examples, the actuation determination component 330 may be configured to dynamically determine the acceleration based on one or more factors associated with the environment. As discussed above, the factors may include the speed of the vehicle 302, a speed limit associated with the environment, a number of detected objects, a type of object detected in the environment, weather (e.g., rain, snow, fog, etc.), or an area or zone associated with the environment (e.g., a school zone, a residential area, a construction zone, etc.), etc. In various examples, the determined acceleration may include a maximum acceleration (positive or negative). In at least one example, the maximum acceleration may include a maximum deceleration rate. In such an example, the actuation determination component 330 may determine to utilize a maximum deceleration rate to stop the vehicle in a short distance and / or to slow the vehicle as much as possible before reaching a location associated with the entity.

[0081] In various examples, the actuation determination component 330 may determine that the distance between the vehicle 302 and the entity does not allow the vehicle 302 to stop. In such examples, the actuation determination component 330 may determine whether an area, such as an adjacent lane or a road shoulder adjacent to the vehicle 302, is clear. Based on a determination that the adjacent area is clear, the actuation determination component 330 may cause the vehicle to correct its trajectory and move into and / or toward the adjacent area. In various examples, the actuation determination component 330 may determine a lateral acceleration associated with the lateral movement (trajectory correction). In some examples, the lateral acceleration may include a predetermined lateral acceleration, such as associated with changing lanes or avoiding an object. In some examples, the actuation determination component 330 may dynamically determine the lateral acceleration. In some examples, the dynamically determined lateral acceleration may be based on one or more features associated with the environment. In some examples, the dynamically determined lateral acceleration may be based on the distance between the vehicle 302 and the entity. In at least one example, the lateral acceleration may be based on the distance and the speed of the vehicle 302. For example, a vehicle traveling 30 miles per hour in close proximity to an entity may be required to aggressively swerve into an adjacent lane to avoid the entity based on a low probability that the entity is associated with an area of ​​particulate matter. Specific features of actuation determination component 330 are discussed in connection with FIG. 3, although it will be understood that actuation determination component 330 may be configured to perform any of the functions associated with determining an actuation based on a probability that an object or entity is associated with an area of ​​particulate matter, as described herein.

[0082] The memory 318 may further include one or more maps 328 that may be used by the vehicle 302 to navigate within the environment and / or to determine the location of features in the environment. For purposes of this discussion, a map may be any number of data structures modeled in two, three, or N dimensions that may provide information about the environment, such as, but not limited to, topology (such as intersections), streets, mountain ranges, roads, terrain, and the environment in general. In some examples, the map may include, but is not limited to, texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), etc.), intensity information (e.g., LIDAR information, RADAR information, etc.), spatial information (e.g., image data projected onto a mesh, individual "surfels" (e.g., polygons associated with individual colors and / or intensities), reflectance information (e.g., specularity information, retroreflectance information, BRDF information, BSSRDF information, etc.). In one example, the map may include a three-dimensional mesh of the environment. In some examples, the map may be stored in a tile format, with individual tiles of the map representing separate portions of the environment, and may be loaded into the working memory as needed. In at least one example, the one or more maps 328 may include at least one map (e.g., an image and / or a mesh). In some examples, the vehicle 302 may be controlled based at least in part on the map 328. That is, the map 328 may be used in conjunction with the localization component 320, the perception component 322, the planning component 324, and / or the actuation decision component 330 to determine a location of the vehicle 302, identify objects and / or environmental features in the environment, and / or generate a route and / or trajectory for navigating within the environment.

[0083] In some examples, one or more maps 328 may be stored on a remote computing device (such as computing device 332) accessible via network 334. In some examples, multiple maps 328 may be stored, for example, based on a characteristic (e.g., type of entity, time of day, day of the week, season, etc.). Storing multiple maps 328 may have similar memory requirements but increase the speed at which data in the maps can be accessed. In some examples, one or more maps 328 may store sizes or dimensions of objects associated with individual locations in the environment. For example, as the vehicle 302 traverses the environment, a map representing an area proximate to the vehicle 302 is loaded into memory, as well as one or more sizes or dimensions of objects associated with the locations.

[0084] In at least one example, the vehicle computing device 304 can include one or more system controllers 326, which can be configured to control steering, propulsion, braking, safety, emitter, communication, and other systems of the vehicle 302. The system controllers 326 can communicate with and / or control corresponding systems of the drive system 314 and / or other components of the vehicle 302.

[0085] As can be appreciated, the components discussed herein (e.g., localization component 320, perception component 322, planning component 324 including actuation decision component 330, one or more system controllers 326, and one or more maps 328) are described as separated for purposes of illustration, however, operations performed by various components may be combined or performed in any other component.

[0086] In at least one example, the sensor system 306 may include LIDAR sensors, RADAR sensors, ultrasonic transducers, sonar sensors, location sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), cameras (RGB, IR, intensity, depth, time-of-flight, etc.), microphones, wheel encoders, environmental sensors (temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), and the like. The sensor system 306 may include multiple instances of each of these or other types of sensors. For example, the LIDAR sensors may include individual LIDAR sensors located at the corners, front, rear, sides, and / or top of the vehicle 302. As another example, the camera sensors may include multiple cameras positioned at various locations on the exterior and / or interior of the vehicle 302. In some examples, the camera sensors located inside the vehicle may provide sensor data to the perception component 322 for determining whether an occupant occupies the passenger compartment and / or the type or class associated with the occupant. In such examples, the actuation determination component 330 may utilize the occupant information in determining the actuation to select and / or in determining the acceleration (e.g., positive, negative, lateral) associated with the actuation. The sensor system 306 can provide input to the vehicle computing device 304. Additionally or alternatively, the sensor system 306 can transmit sensor data over one or more networks 334 to one or more computing devices 332 at a particular frequency, after a predetermined period of time, in near real-time, etc.

[0087] The vehicle 302 may also include one or more emitters 308 for emitting light and / or sound, as described above. The emitters 308 in this example include internal audio and visual emitters for communicating with occupants of the vehicle 302. By way of example and not limitation, the internal emitters may include speakers, lights, signs, display screens, touch screens, haptic emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, head rest positioners, etc.), and the like. The emitters 308 in this example also include external emitters. By way of example and not limitation, the external emitters in this example include lights (e.g., indicator lights, signs, light arrays, etc.) for signaling a direction of travel or other indicators of vehicle operation, and one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) for audibly communicating with pedestrians or other nearby vehicles, one or more of which include acoustic beam steering technology.

[0088] The vehicle 302 may also include one or more communication connections 310 that enable communication between the vehicle 302 and one or more other local or remote computing devices. For example, the communication connections 310 may facilitate communication with other local computing devices on the vehicle 302 and / or the drive system 314. The communication connections 310 may also enable the vehicle to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic signals, etc.). The communication connections 310 may also enable the vehicle 302 to communicate with remote teleoperation computing devices or other remote services.

[0089] The communication connection 310 may include physical and / or logical interfaces for connecting the vehicle computing device 304 to another computing device or network, such as the network 334. For example, the communication connection 310 may enable Wi-Fi based communications, such as over frequencies defined by the IEEE 802.11 standard, short-range radio frequencies such as Bluetooth, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communication protocol that enables each computing device to interface with other computing devices.

[0090] In at least one example, the vehicle 302 may include one or more drive systems 314. In some examples, the vehicle 302 may have a single drive system 314. In at least one example, when the vehicle 302 has multiple drive systems 314, the individual drive systems 314 may be located at both ends of the vehicle 302 (e.g., the front and the rear, etc.). In at least one example, the drive system 314 may include one or more sensor systems for detecting conditions surrounding the drive system 314 and / or the vehicle 302. By way of example and not limitation, the sensor systems may include one or more wheel encoders (e.g., rotary encoders) that sense the rotation of the wheels of the drive module, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) that measure the orientation and acceleration of the drive module, cameras or other image sensors, ultrasonic sensors that acoustically detect objects surrounding the drive module, LIDAR sensors, RADAR sensors, etc. Some sensors, such as the wheel encoders, may be unique to the drive system 314. In some cases, a sensor system on the drive system 314 may overlap or complement a corresponding system on the vehicle 302 (eg, sensor system 306).

[0091] The drive system 314 may include many of the vehicle systems, including a high-voltage battery, a motor to propel the vehicle, an inverter to convert direct current from the battery to alternating current for use in other vehicle systems, a steering system including a steering motor and steering rack (which may be electric), a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for braking force distribution to mitigate loss of traction and maintain control, an HVAC system, lighting (e.g., lighting such as head / tail lights to illuminate the vehicle's exterior environment), and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electrical components such as DC / DC converters, high-voltage junctions, high-voltage cables, charging systems, charging ports, etc.). Additionally, the drive system 314 may include a drive system controller that may receive and preprocess data from the sensor systems to control the operation of various vehicle systems. In some examples, the drive system controller may include one or more processors and a memory communicatively coupled to the one or more processors. The memory may store one or more components for performing various functionalities of the drive system 314. Additionally, drive system 314 may include one or more communication connections that enable each drive system to communicate with one or more other local or remote computing devices.

[0092] In at least one example, the direct connection 312 may provide a physical interface coupling one or more drive systems 314 with the body of the vehicle 302. For example, the direct connection 312 may enable the transfer of energy, fluid, air, data, etc. between the drive systems 314 and the vehicle. In some examples, the direct connection 312 may further releasably secure the drive systems 314 to the body of the vehicle 302.

[0093] In at least one example, the localization component 320, the perception component 322, the planning component 324, the one or more system controllers 326, and the one or more maps 328 may process the sensor data as described above and may transmit their respective outputs to the one or more computing devices 332 via one or more networks 334. In at least one example, the localization component 320, the perception component 322, the planning component 324, the one or more system controllers 326, and the one or more maps 328 may transmit their respective outputs to the computing device 332 at a particular frequency, after a predetermined period of time, in near real-time, etc.

[0094] In some examples, the vehicle 302 may transmit sensor data over the network 334 to one or more computing devices 332. In some examples, the vehicle 302 may transmit raw sensor data to the computing device 332. In other examples, the vehicle 302 may transmit processed sensor data and / or representations of the sensor data to the computing device 332. In some examples, the vehicle 302 may transmit sensor data at a particular frequency, after a predetermined period of time, in near real-time, etc. In some cases, the vehicle 302 may transmit the sensor data (raw or processed) to the computing device 332 as one or more log files.

[0095] The computing device 332 may include a processor 336 and a memory 338 that stores the actuation component 340 and the training component 342. The actuation component 340 may be configured to perform similar functionality as the actuation determination component 330, as described above. In some examples, the actuation component 340 may store one or more predetermined accelerations (e.g., positive, negative, lateral) for use by the actuation determination component 330. In such examples, the computing device 332 may transmit the predetermined accelerations from the actuation component 340 to the actuation determination component 330.

[0096] In some examples, the training component 342 may include functionality to train one or more models to detect objects and / or object features, determine (classify or regress) objects and / or object features, determine the location of objects in the environment, etc. As described herein, the sensor data can be annotated or displayed as representing regions of particulate matter, and the data so annotated or displayed can be identified as training data. The training data can be input to a machine learning model, where known results (e.g., ground truth, such as known "future" attributes) can be used to tune the weights and / or parameters of the machine learning model to minimize errors.

[0097] In some examples, the training component 342 can communicate information generated by one or more models to the vehicle computing device 304 to revise how the vehicle 302 is controlled in response to different conditions.

[0098] For example, some or all aspects of the components discussed herein may include any models, algorithms, and / or machine learning algorithms. For example, in some examples, the components in memory 338 (and memory 318 described above) may be implemented as neural networks. In some examples, the training component 342 may utilize a neural network to generate and / or execute one or more models to improve various aspects of object trajectory estimation for use in trajectory planning of the vehicle 302.

[0099] As described herein, an exemplary neural network is a biologically inspired algorithm that passes input data through a series of connected layers to generate an output. Each layer of a neural network may include another neural network, or may include any number of layers (whether convolutional or not). As can be understood in the context of the present disclosure, a neural network may utilize machine learning, which may refer to a broad class of such algorithms in which an output is generated based on learned parameters.

[0100] Although discussed in the context of neural networks, any type of machine learning can be used consistent with this disclosure. For example, machine learning algorithms may include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), local estimation scatterplot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic net, least angle regression (LARS)), decision tree algorithms (e.g., classification and regression trees (CART), iterative dichotomy 3 (ID3), chi-squared automated interaction detection (CHAID), decision stump, conditional decision tree), Bayesian algorithms (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, average-one dependent estimator (AODE), Bayesian belief network (BNN), Bayesian network), clustering algorithms (e.g., k-means, k-median, expectation maximization (EM), hierarchical clustering), association rule learning algorithms (e.g., perceptron, bacon, sparse ensemble ... The learning algorithms may include deep learning algorithms (e.g., deep Boltzmann machines (DBM), deep belief networks (DBN), convolutional neural networks (CNN), stacked autoencoders), dimensionality reduction algorithms (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), mixed discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble algorithms (e.g., boosting, bootstrap aggregation (bagging), Adaboost, stacked generalization (blending), gradient boosting machines (GBM), gradient boosting regression trees (GBRT), random forests), support vector machines (SVM), supervised learning, unsupervised learning, semi-supervised learning, etc.

[0101] Additional examples of architectures include neural networks such as ResNet50, ResNet101, VGG, DenseNet, and PointNet.

[0102] The processor 316 of the vehicle 302 and the processor 336 of the computing device 332 may be any suitable processor capable of executing instructions to process data and perform operations as described herein. By way of example and not limitation, the processors 316 and 336 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or portion of a device that processes electronic data and converts it to other electronic data that may be stored in registers and / or memory. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices may also be considered processors so long as they are configured to implement encoded instructions.

[0103] The memories 318 and 338 are examples of non-transitory computer-readable media. The memories 318 and 338 may store an operating system, as well as one or more software applications, instructions, programs, and / or data to implement the methods and functions attributed to the various systems described herein. In various implementations, the memories may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash type memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein may include many other logical, programmatic, and physical components, of which the ones shown in the accompanying drawings are merely examples relevant to the discussion herein.

[0104] 3 is shown as a distributed system, it should be noted that in alternative examples, components of the vehicle 302 may be associated with the computing device 332 and / or components of the computing device 332 may be associated with the vehicle 302. That is, the vehicle 302 may perform one or more of the functions associated with the computing device 332, and vice versa.

[0105] 4-6 illustrate exemplary processes according to embodiments of the present disclosure. These processes are illustrated as logical flow graphs, each of which represents a sequence of operations that may be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable recording media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement a process.

[0106] 4 illustrates an example process 400 for controlling a vehicle based on a probability that a detected object is associated with an area of ​​particulate matter. For example, some or all of the process 400 may be performed by one or more components in FIG. 3 as described herein. For example, some or all of the process 400 may be performed by the vehicle computing device 304, the computing device 332, or any other computing device or combination of computing devices. Additionally, any of the operations described in the example process 400 may be performed in parallel, in a different order than depicted in the process 400, omitted, combined with other processes, etc.

[0107] At operation 402, the process may include detecting an object in the environment. In some examples, the vehicle computing system may receive sensor data from one or more sensors of the vehicle. As described above, the sensor data may include data captured by one or more LIDAR sensors, RADAR sensors, cameras, time-of-flight sensors, or the like. In at least one example, the sensor data may include LIDAR data. The vehicle computing system may be configured to receive the sensor data and detect the object based on the sensor data. The vehicle computing system may be further configured to classify the object (e.g., determine a type or classification of the object) based on the sensor data.

[0108] At operation 404, the process may include determining a probability that the object is associated with the region of particulate matter. In various examples, the vehicle computing system may determine a classification of the object as a region of particulate matter and a confidence associated with the classification. In such examples, the confidence may correspond to a probability that the object is associated with the region of particulate matter. In some examples, the vehicle computing system may be unable to determine a classification associated with the detected object. In such examples, the vehicle computing system may determine a probability that the object is associated with the region of particulate matter. For example, the vehicle computing system may receive LIDAR data in which an object is detected (e.g., light reflected from a surface or molecule). The vehicle computing system may process the data but may not be able to classify the object as a particular classification of object. As such, the vehicle computing system may process the sensor data to determine a probability that the object is associated with the region of particulate matter.

[0109] In various examples, the probability may be determined based on one or more features associated with the object, such as those described above. In some examples, the probability may be determined based on map data associated with the environment. In some examples, the probability may be determined utilizing a machine learning model that is trained to determine a classification of the object and a confidence associated with the classification.

[0110] At operation 406, the process may include determining whether the probability is equal to or greater than a first threshold (e.g., 85%, 90%, 95%, etc.). In some examples, the first threshold may include an upper threshold above which there is a high probability that the object is associated with an area of ​​particulate matter. As discussed above, although described herein in terms of thresholds, this is not meant to be so limiting. While some examples may set a fixed threshold for decision making, it should be understood that the description of thresholds herein may refer to a cost formula set such that a transition occurs between two outcomes based on optimizing for cost near such threshold. In various examples, a combination of hard constraints (e.g., fixed thresholds for decision making) and thresholds representing transitions in cost optimization are contemplated.

[0111] Based on a determination that the probability is equal to or greater than the first threshold ("Yes" at operation 406), the process may include controlling the vehicle according to a first operation at operation 408. In some examples, the first operation may include causing the vehicle to maintain a trajectory associated with traveling through the environment. In such examples, based on a determination that there is a high probability that the object is associated with the region of particulate matter, the vehicle may proceed past (or through) the object while traveling through the environment. In some examples, the first operation may include slowing a speed associated with the vehicle. In such examples, the vehicle computing system may slow the vehicle to a predetermined or dynamically determined speed to pass and / or travel through the region of particulate matter. In such examples, the vehicle computing system may be configured to safely react to and / or navigate around another object located behind the object associated with the particulate matter. In some examples, the first operation may additionally or alternatively include modifying a position in a lane or making a lane change to avoid the object. In such examples, despite a high probability that the object is associated with an area of ​​particulate matter, the vehicle computing system may cause the vehicle to navigate to avoid the object, such as to maximize operational safety.

[0112] Based on a determination that the probability is not greater than or equal to the first threshold ("No" at Operation 406), the process may include determining whether the probability is greater than or equal to a second threshold (e.g., 55%, 60%, 70%, etc.) at Operation 410. In some examples, the second threshold may be associated with a medium probability that the object is associated with a region of particulate matter.

[0113] Based on a determination that the probability is equal to or greater than the second threshold ("Yes" at operation 410), the process may include controlling the vehicle according to a second action at operation 412. The second action may include modifying a trajectory associated with the vehicle. As discussed above, the second action may include modifying a trajectory to slow the speed of the vehicle. The modified speed may include a predetermined speed or a speed dynamically determined by the vehicle computing system, such as based on one or more factors associated with the environment. In various examples, based on a distance between the vehicle and the object, the modified speed may provide an opportunity for the vehicle computing system to incorporate additional data and determine an updated probability associated with the object. The vehicle may determine an action to take (e.g., the first action, the second action, or the third action) based on the updated data.

[0114] In various examples, the second actuation may include modifying the trajectory to move the vehicle laterally. In such examples, the vehicle computing system may determine a lateral acceleration associated with the second actuation as described above.

[0115] Based on a determination that the probability is not greater than or equal to the second threshold ("NO" at Operation 410), the processor may include controlling the vehicle according to a third operation at Operation 414. In some examples, the third operation may include modifying the trajectory of the vehicle to stop the vehicle. In some examples, the vehicle computing system may determine a deceleration rate associated with the third operation. In such examples, the deceleration rate may include a predetermined rate or a dynamically determined rate as described above. In some examples, the deceleration rate may be based on a determination that the vehicle is occupied and / or a type or classification of a passenger occupying the vehicle.

[0116] In some examples, the third act may include modifying the vehicle's trajectory to move laterally into an adjacent lane, shoulder, etc., such as based on a determination that the vehicle cannot stop prior to a location associated with the object. As discussed above, the vehicle computing system may determine that an area into which the vehicle will move laterally is clear of the object. In various examples, the vehicle computing system may determine a lateral acceleration associated with the third act. The lateral acceleration may be based on factors associated with the environment, or the distance between the vehicle and the object, etc.

[0117] 5 illustrates an example process 500 for slowing a vehicle to gather additional data about an object to determine whether the object is associated with an area of ​​particulate matter. For example, some or all of the process 500 may be performed by one or more components in FIG. 3 as described herein. For example, some or all of the process 500 may be performed by the vehicle computing device 304, the computing device 332, or any other computing device or combination of computing devices. Additionally, any of the operations described in the example process 500 may be performed in parallel, in a different order than depicted in the process 500, omitted, combined with other processes, etc.

[0118] At operation 502, the process may include detecting an object in the environment based at least in part on sensor data from a sensor of the vehicle. As described above, the sensor data may include data captured by one or more LIDAR sensors, RADAR sensors, cameras, time-of-flight sensors, or the like. In at least one example, the sensor data may include LIDAR data. The vehicle computing system may be configured to receive the sensor data and detect the object based on the sensor data. The vehicle computing system may be further configured to classify the object (e.g., determine a type or classification of the object) based on the sensor data.

[0119] At operation 504, the process may include determining a distance between the vehicle and the object. In some examples, the distance may include a distance between a location associated with the vehicle and a location associated with the object.

[0120] At operation 506, the process may include determining whether the distance is greater than or equal to a threshold distance (e.g., 100 feet, 100 meters, etc.), which may represent a distance sufficient to allow the vehicle to collect additional data about the object to ensure an accurate probability that the object is associated with an area of ​​particulate matter.

[0121] Based on a determination that the distance is not greater than or equal to the third threshold distance ("no" at operation 506), the process may include determining a probability that the object is associated with the region of particulate matter at operation 508. In various examples, the vehicle computing system may determine a classification of the object as a region of particulate matter and a confidence associated with the classification. In such examples, the confidence may correspond to a probability that the object is associated with the region of particulate matter. In some examples, the vehicle computing system may be unable to determine a classification associated with the detected object. In such examples, the vehicle computing system may determine a probability that the object is associated with the region of particulate matter. For example, the vehicle computing system may receive LIDAR data in which an object is detected (e.g., light reflected off a surface or molecule). The vehicle computing system may process the data but may not be able to classify the object as a particular classification of object. As such, the vehicle computing system may process the sensor data to determine a probability that the object is associated with the region of particulate matter.

[0122] In various examples, the probability may be determined based on one or more features associated with the object, such as those described above. In some examples, the probability may be determined based on map data associated with the environment. In some examples, the probability may be determined utilizing a machine learning model that is trained to determine a classification of the object and a confidence associated with the classification.

[0123] At operation 510, the process may include controlling the vehicle based at least in part on the probability. In various examples, the vehicle computing system may control the vehicle according to any of the operations described above.

[0124] Based on a determination that the distance is greater than or equal to the threshold distance ("YES" at operation 506), the process may include determining an initial probability that the object is associated with the region of particulate matter at operation 512. The initial probability may be determined as described above in connection with operation 508. For example, the initial probability may be determined based on one or more features associated with the object, map data, and / or the use of machine learning.

[0125] At operation 514, the process may include determining whether the initial probability is less than or equal to a first threshold (e.g., 95%, 98%, etc.). In various examples, the first threshold may include a probability above which the object has a high probability of being associated with a region of particulate matter.

[0126] Based on a determination that the initial probability is above the first threshold (NO at operation 514), the process may include maintaining a trajectory associated with the vehicle at operation 516. The trajectory may include an initial trajectory determined by a planning component of the vehicle computing system for the vehicle to travel through the environment, such as along a path from a first location to a second location or destination, etc.

[0127] Based on a determination that the initial probability is less than or equal to the first threshold ("YES" at operation 514), the process may include slowing the vehicle to capture additional sensor data at operation 518. In some examples, the vehicle computing system may modify the vehicle's trajectory to a predetermined speed (e.g., 10 miles per hour, 15 kilometers per hour, etc.). In some examples, the vehicle computing system may dynamically determine a speed associated with the modified trajectory. In such examples, the speed may be determined based on one or more factors in the environment as described above.

[0128] At operation 520, the process may include determining updated probabilities based at least in part on the additional sensor data. In various examples, the vehicle computing system may process the additional sensor data to determine the updated probabilities utilizing the techniques described above with respect to operation 508 and / or operation 512.

[0129] At operation 522, the process may include controlling the vehicle based at least in part on the updated probability. In various examples, based on a determination that the updated probability is equal to or greater than the first threshold probability, the vehicle computing system may accelerate the vehicle to an updated trajectory to proceed toward the destination. The updated trajectory may include an initial speed, such as associated with the initial trajectory or a different speed. In various examples, based on a determination that the updated probability is equal to or less than the first threshold probability and / or equal to or greater than the second threshold probability, the vehicle computing system may determine to maintain a low speed associated with the modified trajectory, such as described above in connection with operation 516.

[0130] In various examples, based on a determination that the updated probability is less than or equal to the first threshold probability and / or greater than or equal to the second threshold probability, the vehicle computing system may determine to modify the vehicle trajectory to move the vehicle into an adjacent area, such as an adjacent lane, a shoulder, etc. In such an example, as discussed above, the vehicle computing system may determine that the adjacent area is clear of other objects before moving the vehicle laterally. In some examples, the vehicle computing system may determine a lateral acceleration associated with the lateral movement.

[0131] In various examples, based on a determination that the probability is less than or equal to the second threshold probability, the vehicle computing system may stop the vehicle. In some examples, the vehicle computing system may determine a deceleration rate, brake pressure, engine braking, downshift, etc., associated with stopping the vehicle prior to a location associated with the object. As discussed above, the deceleration rate may be predetermined or dynamically determined by the vehicle computing system based on one or more factors associated with the environment and / or the distance between the object and the vehicle.

[0132] In various examples, based on a determination that the vehicle cannot stop before the location associated with the object, the vehicle computing system may additionally or alternatively cause the vehicle to adjust its position laterally to avoid the object. In such examples, the vehicle computing system may determine a lateral acceleration required to avoid the object. The lateral acceleration may be based on the distance between the vehicle and the object and / or factors associated with the environment, such as, for example, vehicle speed. The vehicle computing system may be configured to modify a trajectory (or trajectories) associated with the vehicle to maximize safety of the vehicle and objects operating within the environment.

[0133] 6 illustrates an example process 600 for controlling a vehicle to avoid objects that are likely (e.g., low probability) not associated with an area of ​​particulate matter. For example, some or all of the process 600 may be performed by one or more components in FIG. 3 as described herein. For example, some or all of the process 600 may be performed by the vehicle computing device 304, the computing device 332, or any other computing device or combination of computing devices. Additionally, any of the operations described in the example process 600 may be performed in parallel, in a different order than depicted in the process 600, omitted, combined with other processes, etc.

[0134] At operation 602, the process may include detecting an object in the environment based at least in part on sensor data from a sensor of the vehicle. As described above, the sensor data may include data captured by one or more LIDAR sensors, RADAR sensors, cameras, time-of-flight sensors, or the like. In at least one example, the sensor data may include LIDAR data. The vehicle computing system may be configured to receive the sensor data and detect the object based on the sensor data. The vehicle computing system may be further configured to classify the object (e.g., determine a type or classification of the object) based on the sensor data.

[0135] At operation 604, the process may include determining that a probability that the object is associated with the region of particulate matter is below a threshold probability. As described above, the probability may be determined based on one or more features associated with the object as described above. In some examples, the probability may be determined based on map data associated with the environment. In some examples, the probability may be determined utilizing a machine learning model that is trained to determine a classification of the object and a confidence associated with the classification. In various examples, a threshold probability (e.g., 50%, 40%, etc.) may be associated with a low probability that the object is associated with the region of particulate matter.

[0136] At operation 606, the process may include determining a first cost associated with the first operation of the vehicle. As discussed above, the first cost may include one or more of a safety cost, a progress cost, a comfort cost, and an operating rule cost associated with the first operation. In various examples, the first operation may include slowing and / or stopping the vehicle prior to a location associated with the object. In some examples, the first cost may be based at least in part on a deceleration rate (negative acceleration) associated with the vehicle stopping and / or slowing. In various examples, the deceleration rate may include a predetermined rate. In some examples, the predetermined rate may be based on a speed associated with the vehicle. In such examples, the vehicle computing system may determine a speed associated with the initial trajectory and may determine the deceleration rate based on the speed. For example, the vehicle computing system may store a table of predetermined deceleration rates associated with different vehicle speeds. In some examples, the deceleration rate may include a maximum deceleration rate. In some examples, the maximum deceleration rate may include a maximum amount of pressure that can be applied to the brakes, a maximum downshift, or an engine braking limit, etc.

[0137] At operation 608, the process may include determining a second cost associated with the second actuation of the vehicle. As discussed above, the second cost may include one or more of a safety cost, a proceeding cost, a comfort cost, and an operating rule cost associated with the second actuation. In various examples, the second actuation may include causing the vehicle to adjust its position laterally to navigate to avoid the object, such as adjusting its position in a lane, changing lanes partially, and / or changing lanes completely to navigate to avoid the object. In various examples, the second cost may be associated with a lateral acceleration and / or a horizontal acceleration associated with the second actuation.

[0138] In various examples, the second cost may be based on a determination of whether the lateral area is clear of objects. In such examples, the vehicle computing system may process sensor data received from one or more sensors of the vehicle to determine whether the lateral area is clear. In some examples, the lateral area may include an area adjacent to the vehicle into which the vehicle may legally enter. For example, the lateral area may include an adjacent lane, a shoulder, or the like.

[0139] At operation 610, the process may include determining whether the first cost is lower than the second cost.

[0140] Based on a determination that the first cost is less than the second cost (YES at operation 610), the process may include controlling the vehicle according to the first actuation at operation 612.

[0141] Based on a determination that the first cost is greater than the second cost (NO at operation 610), the process may include controlling the vehicle according to the second actuation at operation 614. Example clauses A: A system comprising one or more processors and one or more computer-readable media storing instructions executable by the one or more processors, the instructions which, when executed, cause the system to perform operations including capturing sensor data of an environment using a sensor in an autonomous vehicle; determining that the sensor data represents an object in the environment; determining a probability that the object includes a region of particulate matter; and controlling the autonomous vehicle according to a first trajectory based on a first determination that the probability is greater than or equal to a threshold probability, or controlling the autonomous vehicle according to a second trajectory based on a second determination that the probability is less than the threshold probability.

[0142] B: The system described in paragraph A, wherein the first trajectory includes maintaining a speed of the autonomous vehicle and the second trajectory includes at least one of slowing a speed of the autonomous vehicle, stopping the autonomous vehicle prior to a location associated with the object, or adjusting a lateral position of the autonomous vehicle to avoid the object.

[0143] C: The system of paragraphs A or B, wherein the autonomous vehicle is controlled according to a second trajectory, the second trajectory including a second speed that is slower than the first speed associated with the first trajectory, and the operations further include: capturing additional sensor data representing the object in the environment; determining an updated probability that the object includes a region of particulate matter based at least in part on the additional sensor data; and controlling the autonomous vehicle according to a third trajectory based at least in part on the updated probability, the third trajectory including a third speed that is faster than the second speed.

[0144] D: The system of any one of paragraphs A to C, wherein the operations further include determining a distance between the autonomous vehicle and a location associated with the object and determining a deceleration rate for stopping the autonomous vehicle before the location associated with the object, and controlling the autonomous vehicle according to the second trajectory includes controlling the autonomous vehicle based on the deceleration rate.

[0145] E: The system of any one of paragraphs A to D, wherein the autonomous vehicle is controlled according to a second trajectory, and the operation further includes determining that an area adjacent to the autonomous vehicle is not occupied by other objects and determining a lateral acceleration associated with movement of the autonomous vehicle into the area, and the second trajectory includes the lateral acceleration.

[0146] F: A computer-implemented method comprising receiving sensor data captured by a sensor in an environment, determining that the sensor data represents an object in the environment, utilizing machine learning techniques to determine a probability that the object includes an area of ​​particulate matter, and controlling a vehicle based at least in part on the probability.

[0147] G: The computer-implemented method of paragraph F, wherein controlling the vehicle includes at least one of: determining that the probability is associated with a first probability that the object is associated with the area of ​​particulate matter and stopping the vehicle prior to a location associated with the object; determining that the probability is associated with a second probability that the object is associated with the area of ​​particulate matter and slowing the speed of the vehicle, the second probability being higher than the first probability; or determining that the probability is associated with a third probability that the object is associated with the area of ​​particulate matter and maintaining the speed of the vehicle, the third probability being higher than the second probability.

[0148] H: The computer-implemented method of paragraph F or G, further including determining a distance between the vehicle and a location associated with the object and determining that the distance is greater than a threshold distance, and controlling the vehicle includes slowing the vehicle to capture additional sensor data associated with the object.

[0149] I: The computer-implemented method of any one of paragraphs F-H, further comprising determining an updated probability based at least in part on the additional sensor data, and increasing a speed of the vehicle based at least in part on the updated probability.

[0150] J: The computer-implemented method of any one of paragraphs F-I, further including determining that the probability is lower than a threshold probability, determining a distance between the vehicle and a location associated with the object, and determining a deceleration rate based at least in part on the distance, and controlling the vehicle includes slowing the vehicle at the deceleration rate.

[0151] K: The computer-implemented method of any one of paragraphs F through J, further including determining that the probability is lower than a threshold probability, determining that the vehicle is occupied by an occupant, and determining a deceleration rate based at least in part on the occupant, and controlling the vehicle includes slowing the vehicle at the deceleration rate.

[0152] L: The computer-implemented method of any one of paragraphs F-K, wherein controlling the vehicle includes determining a first action for the vehicle to take based at least in part on the probability, determining a second action for the vehicle to take based at least in part on the probability, determining a first cost associated with the first action and a second cost associated with the second action, determining that the first cost associated with the first action is lower than the second cost associated with the second action, and causing the vehicle to perform the first action based at least in part on the first cost being lower than the second cost.

[0153] M: The computer-implemented method of any one of paragraphs F-L, wherein at least one of the first operation or the second operation includes at least one of maintaining the vehicle's trajectory, reducing the vehicle's speed to a predetermined speed, reducing the vehicle's speed to a dynamically determined speed, stopping the vehicle prior to a location associated with the object, or adjusting the vehicle's lateral position to avoid the object.

[0154] N: The computer-implemented method of any one of paragraphs F-M, further including determining that the probability is lower than a threshold probability, determining a first distance between the vehicle and a location associated with the object, determining a second distance at which the vehicle stops at a deceleration rate, and determining that the second distance exceeds the first distance, and controlling the vehicle includes adjusting a lateral position of the vehicle to avoid the object.

[0155] O: A system or device comprising a processor and a non-transitory computer-readable medium storing instructions, which, when executed, cause the processor to perform the computer-implemented method described in any one of paragraphs F to N.

[0156] P: A system or device comprising a means for processing and a means for storing coupled to the means for processing, the means for storing including instructions for configuring one or more devices to perform the computer-implemented method described in any one of paragraphs F to N.

[0157] Q: One or more computer-readable media storing instructions that, when executed by one or more processors, perform operations including receiving sensor data captured by a sensor in an environment; determining that the sensor data represents an object in the environment; determining a probability that the object includes an area of ​​particulate matter, where the probability is determined at least in part based on at least one of a size, shape, trajectory, or location associated with the object; and controlling a vehicle based at least in part on the probability.

[0158] R: The one or more computer-readable media of paragraph Q, wherein controlling the vehicle includes maintaining a trajectory associated with the vehicle.

[0159] S: The one or more computer-readable media of paragraph Q or R, wherein controlling the vehicle includes modifying a trajectory associated with the vehicle, and modifying the trajectory associated with the vehicle includes at least one of reducing a speed of the vehicle to a predetermined speed, stopping the vehicle, or adjusting a lateral position of the vehicle to avoid an object.

[0160] T: One or more computer-readable media described in any one of paragraphs Q-S, wherein the operation further includes determining that a distance between the vehicle and a location associated with the object is greater than a threshold distance, where controlling the vehicle includes driving the vehicle at a first speed; capturing additional sensor data representative of the object in the environment; determining an updated probability that the object includes a region of particulate matter based at least in part on the additional sensor data; and accelerating the vehicle to a second speed based at least in part on the updated probability, where the second speed is faster than the first speed.

[0161] U: One or more computer-readable media of any one of paragraphs Q-T, wherein the operations further include determining a first action to be taken by the vehicle based at least in part on the probability, determining a second action to be taken by the vehicle based at least in part on the probability, determining a first cost associated with the first action and a second cost associated with the second action, determining that the first cost associated with the first action is lower than the second cost associated with the second action, and causing the vehicle to perform the first action based at least in part on the first cost being lower than the second cost.

[0162] V: One or more computer-readable media described in any one of paragraphs Q-U, wherein at least one of the first operation or the second operation includes at least one of maintaining the vehicle's trajectory, reducing the vehicle's speed to a predetermined speed, reducing the vehicle's speed to a dynamically determined speed, stopping the vehicle prior to a location associated with the object, or adjusting the vehicle's lateral position to avoid the object.

[0163] Although the exemplary provisions set forth above are described with respect to one particular implementation, it should be understood that in the context of this specification, the contents of the exemplary provisions can be implemented via a method, device, system, computer-readable medium, and / or another implementation. Furthermore, any of the exemplary examples A-V may be implemented alone or in combination with one or more of the others of the examples A-V. Conclusion Having described one or more examples of the techniques described herein, various modifications, additions, permutations, and equivalents thereof fall within the scope of the techniques described herein.

[0164] In describing the examples, reference is made to the accompanying drawings, which form a part of this application, which show, by way of illustration, specific examples of the claimed subject matter. It is to be understood that other examples may be used and that modifications or alterations, such as structural changes, may be made. Such examples, modifications, or alterations do not necessarily depart from the intended scope of the claimed subject matter. Although individual examples are described herein as having particular features or components, the features and components of the individual examples may be combined and used together. Although operations herein may be presented in a particular order, in some cases the order may be changed such that certain inputs are provided at different times or in a different order without changing the functionality of the systems and methods described. The disclosed procedures could also be performed in a different order. Furthermore, the various calculations herein need not be performed in the order disclosed, and other examples using alternative orders of calculations could be readily implemented. In addition to reordering, calculations could also be decomposed into sub-calculations having the same results.

Claims

1. 1. A system comprising: one or more processors; one or more computer-readable media storing instructions executable by said one or more processors; the instructions, when executed, cause the system to: Capturing sensor data of an environment using sensors in the autonomous vehicle; determining that the sensor data represents an object in the environment; determining a probability that the object includes a region of particulate matter; controlling the autonomous vehicle according to a first trajectory based on a first determination that the probability is greater than or equal to a threshold probability; and controlling the autonomous vehicle according to a second trajectory based on a second determination that the probability is lower than the threshold probability; Executing an operation including The first trajectory includes maintaining a velocity of the autonomous vehicle; The second trajectory is slowing the speed of the autonomous vehicle; stopping the autonomous vehicle prior to a location associated with the object; or adjusting a lateral position of the autonomous vehicle to avoid the object. Contains at least one of A system characterized by:

2. The autonomous vehicle is controlled according to the second trajectory, the second trajectory including a second speed that is slower than a first speed associated with the first trajectory, and the operation includes: capturing additional sensor data representative of the objects in the environment; and determining an updated probability that the object includes the region of particulate matter based at least in part on the additional sensor data; and controlling the autonomous vehicle according to a third trajectory based at least in part on the updated probabilities, the third trajectory including a third speed that is greater than the second speed; and The system of claim 1 further comprising:

3. The operation includes: determining a distance between the autonomous vehicle and a location associated with the object; determining a deceleration rate to stop the autonomous vehicle before the location associated with the object; Further comprising:

3. The system of claim 1 or 2, wherein controlling the autonomous vehicle according to the second trajectory comprises controlling the autonomous vehicle based on the deceleration rate.

4. The autonomous vehicle is controlled according to the second trajectory, and the operation comprises: determining that an area adjacent to the autonomous vehicle is not occupied by other objects; determining a lateral acceleration associated with movement of the autonomous vehicle into the area; Further comprising:

3. The system of claim 1, wherein the second trajectory includes the lateral acceleration.

5. Receiving sensor data captured by sensors in the environment; determining that the sensor data represents an object in the environment; Utilizing machine learning techniques to determine a probability that the object includes a region of particulate matter; and controlling the vehicle based at least in part on the probability. Including, Controlling the vehicle includes: determining that the probability is associated with a first probability that the object is associated with the region of particulate matter and stopping the vehicle prior to a location associated with the object; determining that the probability is associated with a second probability that the object is associated with the region of particulate matter and slowing the speed of the vehicle, the second probability being greater than the first probability; and determining that the probability is associated with a third probability that the object is associated with the region of particulate matter and maintaining a speed of the vehicle, the third probability being greater than the second probability.

6. determining a distance between the vehicle and a location associated with the object; determining that the distance is greater than a threshold distance; Further comprising:

6. The method of claim 5, wherein controlling the vehicle includes slowing the vehicle and capturing additional sensor data associated with the object based on a determination that the distance is greater than a threshold distance.

7. determining updated probabilities based at least in part on the additional sensor data; and increasing a speed of the vehicle based at least in part on the updated probability; and The method of claim 6 further comprising:

8. determining that the probability is less than a threshold probability; determining a distance between the vehicle and a location associated with the object; determining a deceleration rate based at least in part on the distance; and Further comprising:

6. The method of claim 5, wherein controlling the vehicle includes slowing the vehicle at the deceleration rate.

9. determining that the probability is less than a threshold probability; determining that the vehicle is occupied by a passenger; determining a deceleration rate based at least in part on the occupant; and Further comprising:

6. The method of claim 5, wherein controlling the vehicle includes slowing the vehicle at the deceleration rate.

10. Controlling the vehicle includes: determining a first action to be taken by the vehicle based at least in part on the probability; determining a second action to be taken by the vehicle based at least in part on the probability; and determining a first cost associated with the first operation and a second cost associated with the second operation; determining that the first cost associated with the first operation is less than the second cost associated with the second operation; causing the vehicle to perform the first operation based at least in part on the first cost being less than the second cost; and 6. The method of claim 5, comprising:

11. At least one of the first operation or the second operation includes: maintaining the trajectory of said vehicle; reducing the speed of the vehicle to a predetermined speed; reducing the speed of the vehicle to a dynamically determined speed; stopping the vehicle prior to a location associated with the object; or adjusting the lateral position of the vehicle to avoid the object.

11. The method of claim 10, comprising at least one of:

12. determining that the probability is less than a threshold probability; determining a first distance between the vehicle and a location associated with the object; determining a second distance at which the vehicle stops at a deceleration rate; determining that the second distance exceeds the first distance; Further comprising:

6. The method of claim 5, wherein controlling the vehicle includes adjusting a lateral position of the vehicle to avoid the object.

13. A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 5 to 12.

Images