Collision Avoidance Planning System

Abstract

Techniques for controlling a vehicle based on a collision avoidance algorithm are described herein. The vehicle can receive sensor data and determine that the sensor data represents an object in an environment in which the vehicle is moving. A computing device associated with the vehicle determines a collision probability between the vehicle and the object at a first time point at a predicted position of the vehicle and the object. Updated positions of the vehicle and the object can be determined, and a second collision probability can be determined. The vehicle is controlled based at least in part on the collision probability.

Description

[Technical Field]

[0001] The present invention relates to a collision avoidance planning system. [CROSS-REFERENCE TO RELATED APPLICATIONS] This patent application claims priority to U.S. Utility Patent Application No. 17 / 086,047, filed October 30, 2020, which is incorporated herein by reference in its entirety. [Background technology]

[0002] Various methods, devices, and systems are utilized by autonomous vehicles to guide such autonomous vehicles through an environment that may have other objects, including other vehicles. In some examples, predictive techniques can be used to determine the likelihood of a collision between two entities in an environment. Current predictive techniques may have limitations, including a lack of ability to adapt to spatial constrictions associated with an environment. Limitations in current predictive techniques may result in an inability for an autonomous vehicle and another vehicle to pass each other while avoiding a collision. [Brief explanation of the drawings]

[0003] The detailed description will be set forth with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number identifies the drawing in which the reference number first appears. The use of the same reference number in different drawings indicates similar or identical components or features.

[0004] [Figure 1] FIG. 1 is a pictorial flow diagram of an exemplary process for determining a probability of collision and controlling a vehicle based at least in part on the probability of collision. [Figure 2] FIG. 2 is a diagram of an exemplary model of a vehicle and an object for determining a collision probability associated with the vehicle and the object. [Figure 3] FIG. 3 is a flow diagram of an exemplary process for determining acceleration and controlling a vehicle based at least in part on the acceleration. [Figure 4A] FIG. 4A is an illustration of exemplary vehicle trajectories and actions that can be selected based on collision probabilities associated with the vehicles. [Figure 4B] FIG. 4B is an illustration of exemplary vehicle trajectories and actions that can be selected based on collision probabilities associated with the vehicles. [Figure 5] FIG. 5 is a block diagram of an exemplary system for implementing the techniques described herein. [Figure 6] FIG. 6 is an exemplary process for determining a probability of collision and controlling a vehicle based at least in part on the probability of collision. DETAILED DESCRIPTION OF THE INVENTION

[0005] [Detailed explanation] Techniques for evaluating vehicle and object trajectories to avoid collisions are discussed herein. In some examples, a vehicle, such as an autonomous vehicle, can determine a vehicle trajectory to navigate through an environment. The vehicle can also determine an object trajectory for an object (e.g., another vehicle) in the environment. The vehicle can predict vehicle and object positions based on the vehicle trajectory and the object trajectory. Upon determining a collision probability associated with a predicted intersection (e.g., an intersection between the vehicle trajectory and the predicted object trajectory, a distance between the vehicle and the object less than a threshold, an overlap between the vehicle and the object greater than a threshold, etc.), a planning system of the vehicle can determine a position of the vehicle along the vehicle trajectory to avoid the collision and promote the progress of the object along the object trajectory. For example, the planning system can "back up" the vehicle along the vehicle trajectory and "advance" the object along the object trajectory until the planning system determines the probability of collision is less than a threshold. The planning system can modify the vehicle trajectory to slow down the vehicle or stop the vehicle at a location in the environment to avoid the collision.

[0006] In some examples, a vehicle's planning system can utilize a collision avoidance algorithm to reduce the risk of collision with objects in the environment. The vehicle can determine a probability of collision based on the collision avoidance algorithm. The probability of collision can be output by the collision avoidance algorithm and used to control the vehicle. The collision avoidance algorithm can utilize at least one model (e.g., a geometric model, a machine learning model, etc.) to minimize the amount of computation required for tasks and motion planning associated with the vehicle.

[0007] In some cases, the probability of a collision can be determined based on the vehicle and object being located in a constricted area of ​​the environment. The vehicle can be located partially or completely within a lane of a road. The lane in which the vehicle is located can be adjacent to the lane in which the object is located partially or completely. The lane in which the vehicle is located can be utilized to allow traffic movement (e.g., oncoming traffic) in a direction opposite to the movement of traffic in the lane in which the object is located. The road including the lane in which the vehicle is located and the lane in which the object is located can be at least partially constricted.

[0008] The collision avoidance algorithm can determine the risk of a collision based on sensor data received by sensors in the vehicle. The sensor data can include data associated with each of the vehicles and objects in the environment (e.g., vehicles, pedestrians, bicycles, animals, parked cars, trees, buildings, etc.). The sensor data can include information associated with physical characteristics, positions, and / or movements associated with each of the vehicles and objects. Additional information associated with the objects can be determined based on the sensor data, such as, for example, the position, speed, acceleration, and direction of each of the vehicles and objects. The sensor data can be utilized to generate inputs for the collision avoidance algorithm. By way of example, the inputs can include the trajectories of the vehicles and objects, as well as the time period, time step size, distance step size, probability threshold, etc. for searching for a collision.

[0009] In some cases, the collision avoidance algorithm may utilize various geometric models, which may vary based on at least one of processing power, accuracy, tendency to under- / over-estimate collision probability, etc. In some examples, the geometric models may be utilized to determine intersections and / or collision probabilities associated with the vehicles and objects. The intersections and / or collision probabilities may be determined based on respective predicted positions of the vehicles and objects. The predicted positions of the vehicles and objects at the intersections may be determined based on trajectories of the vehicles and objects. The planning system may determine the predicted positions at the intersections based on respective geometric representations of the vehicles and objects. The planning system may determine an overlap between the geometric representation of the vehicle and the geometric representation of the object, as well as a probability distribution associated with the object. The overlap may be determined for the predicted positions of the vehicles and objects associated with the intersections, as well as subsequent predicted positions of the vehicles and objects determined after the vehicle "backs up" and the object "forwards."

[0010] Starting from an initial state (e.g., associated with a collision or collision probability between the vehicle and the object that exceeds a threshold), the collision avoidance algorithm can be utilized to simulate a "backward movement" of a trajectory associated with the vehicle at the intersection (e.g., simulating a deceleration (e.g., deceleration) of the vehicle). In these examples, the "backward movement" of a trajectory associated with the vehicle can include determining vehicle characteristics associated with a new position prior to its position at the initial state. Next, the collision avoidance algorithm can simulate an "advancement" of the trajectory of the object (e.g., simulating a change in the object's position). In these examples, the "advancement" of the trajectory of the object can include determining object characteristics associated with the object at a new time subsequent to the time of the initial state. Next, the collision avoidance algorithm can determine whether a geometric representation of the vehicle at the new time overlaps with a geometric representation of the object at the new position. The "backward movement" and "forward movement" can then be repeated until it is determined that there is no overlap or the collision probability is below a threshold. The vehicle can be controlled to ensure there is no collision and to ensure the object is able to proceed.

[0011] In some examples, the probability of a collision can be determined based on a spatial relationship between a geometric representation of the vehicle and a geometric representation of the object. The probability of a collision can be determined based on a position probability distribution associated with the object. The position probability distribution can be associated with the object as two uncorrelated probability distributions, such as two uncorrelated probability distributions including a probability distribution in a first direction (e.g., across the x-axis) and a probability distribution in a second direction (e.g., across the y-axis). The position probability distribution can be associated with the object as a single correlated probability distribution in both directions (e.g., across both the x-axis and the y-axis). For example, an object such as another vehicle can be represented as a rectangle based on the range of the object determined from sensor data. The probability of a collision can be determined based on an integral associated with each probability distribution and a geometric model associated with the vehicle. The position probability distribution is not limited to the geometric models described above and can include one or more other models. Other models can include, for example, a model based on a velocity associated with the object, a probability related to the velocity associated with the object, uncertainty related to the acceleration associated with the object, uncertainty related to the steering associated with the object, uncertainty related to the range of the object itself, a heat map associated with the object, a tube associated with the object's location, etc. Examples of heatmap-based probability distributions are described, for example, in U.S. Patent Application No. 16 / 151,607, entitled "Trajectory Prediction on Top-Down Scenes," filed October 4, 2018, the entire contents of which are incorporated herein by reference.

[0012] In some examples, the collision avoidance algorithm can utilize a machine learning model to determine a probability of collision associated with the positions of the vehicle and the object. In some examples, the machine learning model can include a convolutional neural network (CNN), which may include one or more recurrent neural network (RNN) layers, such as, but not limited to, a long short-term memory (LSTM) layer. A probability of collision associated with the vehicle can be determined based on the positions of the vehicle and the object, respectively, at the intersection. A probability of collision associated with the vehicle can be determined based on the positions of the vehicle and the object, respectively, after the vehicle "backs up" and the object "forwards." A predicted speed associated with the vehicle can be determined based on the positions of the vehicle and the object at the intersection. A predicted acceleration associated with the vehicle can be determined based on the speed and position of the vehicle and the object after the vehicle "backs up" and the object "forwards."

[0013] In some examples, a collision avoidance algorithm may be utilized to determine an acceleration for controlling the vehicle based on a predicted speed and / or a predicted position associated with the vehicle. The acceleration for controlling the vehicle may be determined based on the following options: i) stopping the vehicle; or ii) correcting the speed of the vehicle to a reference speed at a position along the vehicle's trajectory. The acceleration for controlling the vehicle may be based on the most conservative option.

[0014] By utilizing collision avoidance algorithms, the vehicle can be controlled to avoid a collision without updating steering inputs (e.g., without requiring the vehicle to change lanes). The vehicle can be controlled to stop or slow down, allowing an object moving in the opposite direction to safely continue along its object trajectory and / or pass by the vehicle. Even if the environment associated with the vehicle is constricted, the object can continue on its trajectory and / or pass by the vehicle. These and other improvements to vehicle operation are described herein.

[0015] The techniques described herein can improve the functionality of computing devices in several additional ways. In some cases, collision avoidance algorithms can enable rapid evaluation of trajectories to determine the probability of collision along the trajectory. The use of simple geometric models and probability distributions allows for simple calculations that accurately represent the risk associated with such trajectories. Thus, such collision avoidance algorithms can be run in parallel to evaluate multiple trajectories and / or actions to safely guide a vehicle, thereby improving general safety outcomes and traffic flow. The use of such strategies can improve computer performance, requiring less memory, and reduce processing requirements, thus enabling such systems to be employed in real-world vehicle safety systems that must make critical safety decisions in short periods of time with limited amounts of processing power and memory.

[0016] The techniques described herein can be implemented in many ways. Exemplary embodiments are provided below with reference to the following figures. While described in the context of an autonomous vehicle, the methods, apparatus, and systems described herein are applicable to a variety of systems and are not limited to autonomous vehicles. In another example, the techniques can be utilized in an aviation or nautical context, or in any system configured to input data to determine movement relative to objects in an environment. Additionally, while described in the context of a road having a constricted area, the techniques can be applicable to any type or road and are not limited to the specific examples described herein. Furthermore, the techniques described herein can be used with real data (e.g., captured using sensors), simulated data (e.g., generated by a simulator), or any third of the two.

[0017] FIG. 1 is a pictorial flow diagram of an exemplary process for determining a probability of collision and controlling a vehicle based at least in part on the probability of collision.

[0018] Operation 102 may include determining a vehicle trajectory and an object trajectory. The vehicle trajectory and the object trajectory may be determined based on sensor data received by one or more sensors on the vehicle. Examples of vehicle trajectories and object trajectories are described, for example, in U.S. patent application Ser. No. 16 / 151,607, entitled "Trajectory Prediction On Top-Down Scenes," filed October 4, 2018, and U.S. patent application Ser. No. 15 / 843,512, entitled "Trajectory Generation Using Curvature Segments," filed December 15, 2017, and incorporated herein by reference in their entireties. In some examples, the vehicle and the object may be located in a constricted area of ​​the environment. The vehicle may travel on a road and may travel in a direction opposite (e.g., on the opposite side of) the movement of the object traveling on the road. At least a portion of the road on which the vehicle and the object are located may be at least partially constricted (e.g., by double-parked vehicles, a door opening toward traffic, etc.).

[0019] Example 104 shows an environment in which a vehicle 106 and an object 108 are navigating the environment. The vehicle 106 and the object 108 may each be at a starting location. The vehicle 106 may have a vehicle trajectory 110, and the object 108 may be associated with an object trajectory 112.

[0020] Operation 114 may include determining a collision probability associated with vehicle 106 and object 108. For example, operation 114 may include proceeding (e.g., simulating) the movement of vehicle 106 and object 108 along predicted trajectories 110 and 112, respectively. As shown in example 116, the collision probability may be determined at location 118. In some examples, the collision probability may be determined to exceed a collision probability threshold. In some examples, this collision probability threshold may be a first threshold that triggers the initiation of a "backtracking" algorithm.

[0021] In some examples, the collision probability is calculated based on the predicted intersection point (e.g., time T N The predicted intersection point may be associated with the location of the object 108 in the narrowed portion of the road.

[0022] Example 116 shows an environment in which a vehicle 106 and an object 108 are navigating the environment at a predicted location associated with a predicted intersection. The vehicle 106 can be associated with a predicted location (e.g., location S) 118 at the predicted intersection. The object 108 can be associated with a predicted time (e.g., time T) at the predicted intersection. N )120.

[0023] Operation 122 may include determining an updated predicted position associated with the vehicle and the object. The updated predicted position associated with vehicle 106 may be determined by simulating a "setback" of vehicle trajectory 110 (e.g., simulating a deceleration (e.g., a deceleration) of the vehicle). In these examples, the "setback" of vehicle trajectory 110 may include determining vehicle characteristics associated with a new predicted position prior to predicted position 118.

[0024] An updated predicted position associated with the object 108 can be determined by simulating an “advancement” of the object trajectory 112 (e.g., simulating the object changing its position). In these examples, “advancing” the object trajectory 112 may include determining object properties associated with the object at a new time after the predicted time 120.

[0025] Example 124 shows a simulated instance in which the vehicle 106 and object 108 are represented with updated predicted positions. The vehicle 106 can be associated with an updated predicted position (e.g., position SD (e.g., position S minus some distance D)) 126. The updated predicted position 126 associated with the vehicle 106 can be determined by simulating a "backward movement" of the vehicle trajectory 110. The updated predicted position of the object 108 is determined by simulating an updated predicted time 128 (e.g., time T N+M (e.g., time M plus an additional time N). An updated predicted position associated with object 108 at updated predicted time 128 can be determined by simulating the "forward movement" of object trajectory 112.

[0026] Operation 130 may include determining whether the collision probability exceeds a collision probability threshold. In some examples, this collision probability threshold may be a second threshold different from the threshold (e.g., the first threshold) described above with respect to operation 114. In some examples, this threshold may be considered an "exit" threshold for terminating the "backtracking" algorithm. The second threshold may be less than or equal to the first threshold. The number of collision probabilities determined by the collision avoidance algorithm is not limited to two and may include any number of collision probabilities associated with each predicted position associated with the vehicle and each predicted time associated with the object. As the number of collision probabilities increases (or the distance between positions for determining collision probability decreases below the threshold), the probability approach associated with the determined collision probabilities utilized by the collision avoidance algorithm approaches a sequential probability approach. The sequential probability approach includes a sequential probability of collision associated with each point on a sequential path by "backtracking" from the first predicted position of the vehicle to the vehicle's starting position. Operation 130 may determine, for each collision probability associated with the successive paths, whether the collision probability exceeds a collision probability threshold until a collision probability is determined that is less than or equal to a second threshold.

[0027] Act 132 may include determining an updated vehicle trajectory. The updated vehicle trajectory may be determined based on a new acceleration profile. The new acceleration profile may be determined based on a collision probability (e.g., the collision probability determined in Act 114). The new acceleration profile may include accelerating (e.g., decelerating) the vehicle. In some examples, the acceleration may include slowing or stopping the vehicle to arrive at or before a location (e.g., a first location associated with a vehicle that is stopped or has a slowest speed, or a second location associated with a vehicle that is slowing).

[0028] Thus, as described herein, a vehicle approaching an object on a road can be controlled to reduce the risk of a collision with the object. The risk of a collision can be reduced based on a predicted position of the object that may result in a collision between the vehicle and the object. The predicted position of the object can be based on the object's changing trajectory due to a constricted area in a portion of the road along which the object is traveling. A collision probability can be determined based on a predicted intersection between the vehicle and the object. The predicted intersection can be associated with a predicted position of the vehicle and a predicted position of the object. The predicted intersection can be utilized to simulate the vehicle's "backward" in time and the object's "forward" in position. Based on the collision probability being below a second threshold, the vehicle and object simulation can be repeated to determine updated predicted positions of the vehicle and the object. The updated predicted position can be associated with a reduced risk of a collision between the vehicle and the object. Based on the updated predicted position, the vehicle can be controlled. The vehicle can be controlled to slow down or stop to reach or before the associated updated predicted position. Slowing down or stopping the vehicle can allow the object to pass by the vehicle.

[0029] FIG. 2 is a diagram illustrating an example of a vehicle and object model for determining a collision probability associated with the vehicle and the object.

[0030] 2, the exemplary representation 202 may include a vehicle having a vehicle trajectory and an object having an object trajectory. In some examples, the exemplary representation 202 may include the vehicle 106, the object 108, the vehicle trajectory 110, and the object trajectory 112, as described above in connection with FIG.

[0031] The exemplary representation 204 illustrates a geometric model utilized to determine a collision probability associated with the vehicle 106. The geometric model may include a geometric representation 206 associated with the vehicle 106 and a geometric representation 208 associated with the object 108. In some examples, the collision probability may be determined based on the distance between the geometric representation 206 and the geometric representation 208. In these examples, the collision probability may be determined based on the overlap between the geometric representation 206 and the geometric representation 208. The overlap may be associated with an intersection between the vehicle 106 and the object 108. The overlap may be determined in relation to the probability of no correlation in the x and y directions. The overlap may be utilized to determine an x-direction collision probability associated with the x direction and a y-direction collision probability associated with the y direction. The x-direction collision probability may be multiplied by the y-direction collision probability to determine the collision probability associated with the vehicle 106 and the object 108. As described in more detail below, different models of the vehicle 106 may be utilized to facilitate the calculation. A machine learning model may then be utilized to determine a correction factor based on a Monte Carlo simulation.

[0032] In some examples, the vehicle 106 may be represented by a geometric representation 206 shown as a circle. The radius of the circle may be based on the width of the vehicle 106. In some examples, the center of the circle may be based on or associated with the center of the front bumper of the vehicle 106.

[0033] In some examples, the center of the geometric representation 206 may be determined to be closer to another portion of the vehicle 106 (e.g., closer to or farther from the center of the vehicle) to provide a more conservative probability of collision for that portion of the vehicle 106.

[0034] In some examples, the object 108 may be represented by a geometric representation 208, such as a rectangle. The rectangle may be based on the extents of the object (e.g., the width and length of the object).

[0035] In example 204, the offset from the center of representation 208 to the center of representation 206 can be expressed as a variable "k" in the first dimension and a variable "h" in the second dimension.

[0036] The representation 208 may also be associated with a cumulative distribution function (e.g., a normal distribution function) that represents the probability that the object 108 is at a particular location at a particular time. The collision probability of the vehicle 106 colliding with the object 108 may be determined based on an x-direction collision probability associated with the x-direction and a y-direction collision probability associated with the y-direction. The collision probability in the x-direction of the vehicle 106 colliding with the object 108 may be determined based on a horizontal offset. The collision probability in the x-direction may be determined based on a mean, a standard deviation, and an area under the curve associated with the normal distribution function. The collision probability in the y-direction of the vehicle 106 colliding with the object 108 may be determined based on a vertical offset. The collision probability in the y-direction may be determined based on a mean, a standard deviation, and an area under the curve associated with the normal distribution function. The collision probability in the x-direction may be multiplied by the collision probability in the y-direction to determine the collision probability associated with the vehicle 106 and the object 108.

[0037] In some examples, the probability of collision between the vehicle 106 and the object 108 can be determined based on the vertical and horizontal offsets and position probabilities described above. An initial value of the collision probability can be determined to be associated with a predicted intersection point between the vehicle 106 and the object 108. An initial value of the collision probability can be determined to be associated with an overlap between the geometric representation 206 and the geometric representation 208. The overlap can be determined based on the size of each of the vertical and horizontal offsets being less than or equal to the size of the radius of the geometric representation 206. In some examples, the value of the collision probability can be determined to be a maximum value based on the maximum size of the overlap (e.g., the center of the geometric representation 206 having the same predicted position as the center of the geometric representation 208). In these examples, the value of the collision probability can be determined to decrease based on a decrease in the size of the overlap (e.g., an increase in the size of one or more of the vertical or horizontal offsets).

[0038] In some examples, a collision probability associated with each position of vehicle 106 and object 108 can be determined based on geometric representation 206 and based on geometric representation 208, as shown in exemplary representations 210-216. The collision probability can be determined based on an overlap between geometric representation 206 and geometric representation 208, as shown in exemplary representations 210-216. The overlap associated with each position of vehicle 106 and object 108 can have a size based on the area of ​​geometric representation 206 and the probability distribution associated with graphical representation 208.

[0039] Although the graphical representation 206 is described in the context of a circle, the vehicle 106 may be represented by other shapes (such as a square and / or a rectangle) to reduce computational complexity. Additional models of the vehicle 106 are shown as examples 212, 214, and 216.

[0040] Model 212 is represented as a square with dimensions based on the width of the circle. Figure 2 shows model 212 as a square surrounding a circle to illustrate the relative sizes of the models. Because the area of ​​model 212 is larger than the area of ​​circle 210, this model overestimates the probability of collision.

[0041] Model 214 is represented as a square with dimensions based on the width of the circle. Figure 2 shows model 214 as a square surrounded by a circle. Because the area of ​​model 214 is smaller than the area of ​​circle 210, this model underestimates the probability of collision.

[0042] Model 216 is represented as a square with a rectangle on each side of the square, with the square having dimensions based on the width of the circle. Figure 2 shows model 216 as a rectangle bounded by a square and a circle. Because the area of ​​model 216 is smaller than the area of ​​circle 210, this model underestimates the collision probability. Because the area of ​​model 216 is larger than the area of ​​model 214, the amount by which this model underestimates the collision probability is less than the amount by which model 214 underestimates the collision probability. This model estimates the collision probability more accurately than model 214.

[0043] Although four examples of geometric representation 206 have been described, geometric representation 206 is not limited thereto and may include any additional number of polygons (e.g., rectangles) in addition to the rectangles on each side of the square in example 216. In some examples, geometric representation 206 according to example 216 may include a square, a circle, and a rectangle in each gap enclosed by each of the rectangles on each side of the square.

[0044] In some examples, the collision probability associated with each position of vehicle 106 and object 108 can be determined as a combined collision probability based on collision probabilities associated with two or more geometric representations 206. For example, determining the collision probability using model 212 results in an overestimation of the collision probability. Determining the collision probability using models 214 or 216 results in an underestimation of the collision probability. In some examples, these overestimations and underestimations can be combined to determine an aggregate or combined collision probability.

[0045] In some examples, the over- and under-estimations can be adjusted by a correction factor to generate a more accurate collision probability. In some examples, the correction factor can be generated by, or at least partially based on, a machine learning model trained to output a correction factor based on a model used to represent the range of the vehicle 106, the object 208, and the offset between the vehicle 106 and the object 108. For example, the ground truth for training the machine learning model can be generated by a simulation (e.g., a Monte Carlo simulation), and the simulation can be evaluated to determine whether a collision occurred. Based on a large number of samples (e.g., tens of thousands), a ground truth collision probability can be determined for a particular model parameter. As described above, a correction factor can be learned based on the difference between the actual ground truth collision probability and the over- or under-estimation. The collision probability can be determined based on a simulation as described above, but is not limited to this, and can be determined based on any other type of model. For example, the collision probability can be determined based on a correction factor determined using a heuristic model. The heuristic model can utilize any combination of correction and aggregation. As an example, an initial collision probability associated with any type of geometric representation (e.g., any of geometric representations 206) can be determined, then corrected with a correction factor determined based on the initial collision probability, and then aggregated with other collision probabilities (e.g., any collision probabilities associated with other geometric representations (e.g., any of geometric representations 206)). As another example, an initial collision probability associated with any type of geometric representation can be determined, then aggregated with the other collision probabilities, and then corrected with a correction factor determined based on the aggregated collision probabilities. As another example, an initial collision probability associated with any type of geometric representation can be determined, then corrected with a correction factor determined based on the initial collision probability, then aggregated with the other collision probabilities, and then corrected with a corrected factor determined based on the aggregated collision probabilities, and so on.

[0046] When a machine learning model is used to determine the correction factor, the correction factor can be determined by the machine learning model or represented by a static means, such as a look-up table. The correction factor can be predicted based on the error by fitting a quadratic curve based on a normal distribution function associated with each of the over-estimated and under-estimated collision probabilities to the error. The correction factor can be utilized to determine an updated over-estimated collision probability and an updated under-estimated collision probability. The updated over-estimated collision probability and the updated under-estimated collision probability can be averaged together to determine an average collision probability associated with the positions of the vehicle 106 and the object 108. The average collision probability can provide a more accurate collision probability than either the over-estimated or under-estimated collision probability.

[0047] Thus, as described herein, a vehicle and an object can each be modeled as a geometric representation. A vehicle can be modeled as a circle, a square surrounding a circle, a square surrounded by a circle, or a combination of a square and a rectangle surrounded by a circle. An object can be modeled as a rectangle. The geometric representations can be used to predict an intersection associated with a predicted collision between the vehicle and the object. The overlap between the geometric representations at the overlap can be associated with a collision probability. The overlap can be used to overestimate the collision probability based on a vehicle modeled as a square surrounding a circle. The overlap can be used to underestimate the collision probability based on a vehicle modeled as a square surrounded by a circle. The overestimates and underestimates of the collision probability can be combined to determine an average collision probability. The average collision probability can estimate the collision probability with greater accuracy than the overestimates and underestimates of the collision probability. The vehicle can be controlled based on the average collision probability to reduce the risk of a collision with the object.

[0048] FIG. 3 is a pictorial flow diagram of an exemplary process for determining acceleration and controlling a vehicle based at least in part on the acceleration.

[0049] Operation 302 may include determining a collision probability for a position (e.g., a predicted position) associated with each of the vehicle and the object. If the collision probability exceeds a first threshold (e.g., an entry threshold), operation 302 may include executing a collision avoidance algorithm including "backing up" the vehicle and "forward" the object. If the collision probability falls below a second threshold (e.g., an exit threshold), the collision avoidance algorithm may be terminated.

[0050] As described above, a collision avoidance algorithm can be utilized to determine a collision probability based on simulating a "reverse" vehicle trajectory and an "forward" object trajectory. The collision probability based on the "reverse" and "forward" can be determined based on an earlier predicted location associated with the vehicle and a subsequent predicted time associated with the object. The collision avoidance algorithm can be utilized to continue simulating the "reverse" and "forward" until the collision probability is determined to be less than or equal to a second threshold (e.g., 10%).

[0051] Example 304 illustrates determining the collision probability according to a collision avoidance algorithm. Although two scenarios are shown, any number of scenarios can be simulated based on the collision probability exceeding a second threshold.

[0052] The example 304 can include a scenario 306 that includes an environment in which a collision probability can be determined at a time associated with a predicted intersection. The collision probability can be determined by utilizing a collision avoidance algorithm. The collision probability can be determined based on an initial time (e.g., T N ) 308, and an initial predicted position (e.g., S1) 310 associated with the vehicle 106 at the predicted intersection. In some examples, the probability of collision may be determined to exceed a first threshold that triggers the initiation of a "backtracking" algorithm.

[0053] The example 304 can include a scenario 312 that includes an environment in which a collision probability can be determined at a time after a time associated with a predicted intersection. A collision avoidance algorithm can be utilized to determine a collision probability based on simulating a "retreat" vehicle trajectory and an "advance" object trajectory. The "retreat" and "advance" based collision probability can be calculated based on an earlier predicted position (e.g., S2) 314 associated with the vehicle 106 and a later predicted time (T N+M ) 316. The collision avoidance algorithm can determine if the collision probability is less than or equal to a second threshold to terminate the "backtracking" algorithm.

[0054] Operation 318 may include determining a speed associated with the vehicle at each of the initial predicted position and the earlier predicted position. In some examples, the speed and collision probability associated with the initial predicted position may be determined based on a collision probability determined to be above a first threshold. The speed and collision probability associated with the earlier predicted position may be determined based on interpolation between a speed associated with the vehicle at a first threshold (e.g., speed 0) and a speed associated with the vehicle at a second threshold (e.g., the speed of the original vehicle trajectory). As described above, the first threshold may be associated with a stationary vehicle. The second threshold may be associated with a vehicle maintaining a speed (e.g., a speed unchanged from the speed of the original vehicle trajectory). In some examples, a "backtracking" algorithm may include a minimum speed below which the vehicle will not move. A collision avoidance algorithm may be associated with a minimum speed. In these examples, the first threshold may be associated with the vehicle moving at a minimum speed instead of speed 0.

[0055] Example 320 shows a graph plotting vehicle speed with respect to collision probability. A speed 322 associated with the vehicle 106 at the initial predicted position (e.g., S1) 310 can be determined. A speed 324 associated with the vehicle 106 at the early predicted position (e.g., S2) 314 can be determined. A collision avoidance algorithm can be utilized to determine a collision probability 326 associated with the speed 322 associated with the vehicle 106 at the initial predicted position 310. A collision avoidance algorithm can be utilized to determine a collision probability 328 associated with the speed 324 associated with the vehicle 106 at the early predicted position 314. The speed 324 and collision probability 328 associated with the early predicted position 310 can be determined based on interpolation between the speed 322 associated with the vehicle 106 at a first threshold and the speed associated with the vehicle at a second threshold (e.g., the speed of the original vehicle trajectory). The interpolation can include any type of interpolation, such as linear interpolation, quadratic interpolation, etc. As mentioned above, the first threshold can be associated with a vehicle that is stationary or moving at a minimum speed, and the second threshold can be associated with a vehicle that maintains speed (e.g., does not change speed).

[0056] Operation 332 may include determining an acceleration based on a velocity associated with the vehicle and a current location (e.g., a starting location). The acceleration for controlling the vehicle may include a first acceleration utilized to control the vehicle to have a first velocity when the vehicle reaches a first location. In some examples, the first acceleration may be utilized to stop the vehicle or to control the vehicle to have a first velocity (e.g., a minimum velocity (e.g., 0 m / s, 1 m / s, 2 m / s, etc.)) at the first location. Alternatively, the acceleration may include a second acceleration utilized to control the vehicle to decelerate to have a second velocity at a second location. The acceleration for controlling the vehicle may include a most conservative acceleration associated with a location determined by the collision avoidance algorithm to be closest to the current location. The most conservative acceleration may include a first acceleration associated with a first location based on the first location being closer to the current location than the second location, or vice versa.

[0057] Operation 334 may include controlling the vehicle based on the acceleration (e.g., the most conservative acceleration). In some examples, the most conservative acceleration may be determined as the acceleration associated with the vehicle moving a minimum distance in the same amount of time. The most conservative acceleration may be determined as the acceleration associated with the vehicle stopping and not moving beyond a position that results in the shortest distance between the vehicle's position after deceleration or stopping and the vehicle's current position. In some examples, the most conservative acceleration may include a first acceleration associated with a first position based on the first position being closer to the current position than the second position. In some examples, the most conservative acceleration may include a second acceleration associated with a second position based on a second position that is closer to the current position than the first position. The first acceleration may be associated with the vehicle being controlled to stop at the first position or to have a first speed (e.g., a minimum speed). Alternatively, the second acceleration may be associated with the vehicle being controlled to decelerate to have a second speed at the second position.

[0058] Thus, as described herein, a collision avoidance algorithm can be utilized to determine a collision probability and a speed and / or acceleration associated with the vehicle. The acceleration utilized to control the vehicle is the most conservative acceleration and can include: i) stopping the vehicle or slowing the vehicle to a minimum speed; ii) slowing the vehicle to a speed determined by interpolation; or iii) allowing the vehicle to continue at the original vehicle trajectory speed. Optimizing forward movement of the vehicle while determining that there is no high risk of a collision with an out-of-lane object (e.g., an object returning into a lane adjacent to the lane in which the vehicle is located).

[0059] 4A and 4B are diagrams illustrating example vehicle trajectories and actions that can be selected based on a collision probability associated with a vehicle. As described above in connection with FIG. 1, a vehicle trajectory can be associated with a vehicle 106, and an object trajectory can be associated with an object 108.

[0060] As shown in FIG. 4A , a vehicle planning system can determine multiple trajectories for a vehicle to travel along a single lane. That is, the planning system can consider multiple trajectories based on the vehicle's position within the environment. Each trajectory can be evaluated by a collision avoidance algorithm to determine the vehicle's safe operation throughout the environment. In some examples, acceleration determined by the techniques described herein can be used when evaluating the cost associated with a trajectory, and thus a trajectory can be selected or determined based on the lowest cost. For example, a trajectory biased toward the object 108 can have a reduced speed because it increases the probability of collision. Thus, if nothing else is considered, a trajectory biased away from the object 108 can be selected because it results in a higher speed (e.g., reduced travel time).

[0061] FIG. 4B illustrates a similar operation in which different actions (e.g., change lane left, stay in lane, change lane right) can be evaluated based on a collision avoidance algorithm, as described herein.

[0062] Thus, as described herein, collision avoidance algorithms can be used to make more complex decisions utilized to control a vehicle. If a road with lanes does not contain other lanes with traffic moving in the same direction, the vehicle can be controlled to change position within the lane in which it is located. Or, if the cost of changing lanes is lower than the cost of remaining in the lane, the vehicle can be controlled to change lanes instead of slowing down or stopping. The cost of changing lanes can be lower if the vehicle can be controlled to have a higher speed by changing lanes than by remaining in the same lane.

[0063] 5 is a block diagram of an example system for implementing the techniques described herein. In at least one example, the system 500 can include a vehicle 502. In the illustrated example system 500, the vehicle 502 is an autonomous vehicle, although the vehicle 502 can be any other type of vehicle.

[0064] Vehicle 502 may be an unmanned vehicle, such as an autonomous vehicle configured to operate according to a Level 5 classification issued by the U.S. Department of Transportation's National Highway Traffic Safety Administration, which describes a vehicle capable of performing all safety-critical functions throughout the entire journey and does not expect a driver (or passenger) to be in control of the vehicle at any time. In such an example, vehicle 502 may not include a driver and / or controls for driving vehicle 502, such as a steering wheel, accelerator pedal, and / or brake pedal, as it may be configured to control all functions from the start to the completion of the journey, including all parking functions. This is merely an example, and the systems and methods described herein may be incorporated into any land, air, or water vehicle, ranging from vehicles that must be manually controlled by a driver at all times to vehicles that are partially or fully autonomously controlled.

[0065] Vehicle 502 may include one or more computing devices 504, one or more sensor systems 506, one or more emitters 508, one or more communication connections 510 (also referred to as communication devices and / or modems), at least one direct connection 512 (e.g., for physically coupling with vehicle 502 to exchange data and / or provide power), and one or more drive systems 514. One or more sensor systems 506 may be configured to capture sensor data associated with an environment.

[0066] The one or more sensor systems 506 may include time-of-flight sensors, position sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), lidar sensors, radar sensors, sonar sensors, infrared sensors, cameras (e.g., RGB, IR, intensity, depth, etc.), microphone sensors, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), ultrasonic transducers, wheel encoders, etc. The one or more sensor systems 506 may include multiple instances of each of these or other types of sensors. For example, a time-of-flight sensor may include individual time-of-flight sensors positioned at the corners, front, rear, sides, and / or top of the vehicle 502. As another example, a camera sensor may include multiple cameras positioned at various locations around the exterior and / or interior of the vehicle 502. The one or more sensor systems 506 may provide input to the computing device 504.

[0067] The vehicle 502 may also include one or more emitters 508 for emitting light and / or sound. The one or more emitters 508 in this example include interior audio and visual emitters for communicating with occupants of the vehicle 502. By way of example and not limitation, the interior emitters may include speakers, lights, signs, display screens, touchscreens, haptic emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, headrest positioners, etc.). The one or more emitters 508 in this example also include exterior emitters. By way of example and not limitation, the exterior emitters in this example include lights (e.g., indicator lights, signs, light arrays, etc.) that signal direction of travel or other indicators of vehicle operation, and one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) for audibly communicating with pedestrians or other nearby vehicles, one or more of which may include acoustic beam steering technology.

[0068] Vehicle 502 may also include one or more communication connections 510 that enable communication between vehicle 502 and one or more other local or remote computing devices (e.g., remote teleoperated computing devices) or remote services. For example, one or more communication connections 510 may facilitate communication with other local computing devices on vehicle 502 and / or one or more drive systems 514. One or more communication connections 510 may also enable vehicle 502 to communicate with other nearby computing devices (e.g., other nearby vehicles, traffic signals, etc.).

[0069] The one or more communications connections 510 may include physical and / or logical interfaces for connecting the computing device 504 to another computing device or to one or more external networks 542 (e.g., the Internet). For example, the one or more communications connections 510 may enable Wi-Fi®-based communications, such as over frequencies defined by the IEEE 802.11 standard, short-range radio frequencies such as Bluetooth®, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), satellite communications, dedicated short-range communications (DSRC), or any suitable wired or wireless communications protocol that enables each computing device to interface with other computing devices.

[0070] In at least one example, the vehicle 502 can include one or more drive systems 514. In some examples, the vehicle 502 can have a single drive system 514. In at least one example, if the vehicle 502 has multiple drive systems 514, an individual drive system 514 can be located at each end of the vehicle 502 (e.g., the front and rear, etc.). In at least one example, the drive system 514 can include one or more sensor systems 506 for detecting conditions surrounding the drive system 514 and / or the vehicle 502. By way of example and not limitation, the sensor systems 506 can include one or more wheel encoders (e.g., rotational encoders) for sensing the rotation of the wheels of the drive system, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) for measuring the orientation and acceleration of the drive system, cameras or other imaging sensors, ultrasonic sensors, lidar sensors, radar sensors, etc. for acoustically detecting objects surrounding the drive system. Some sensors, such as the wheel encoders, can be specific to the drive system 514. In some cases, the sensor systems 506 on the drive system 514 may overlap or supplement corresponding systems (e.g., sensor systems 506 ) of the vehicle 502 .

[0071] The drive system 514 can include many vehicle systems, including a high-voltage battery, a motor for propelling the vehicle, an inverter for converting direct current from the battery to alternating current used by other vehicle systems, a steering system (which can be electric) including a steering motor and a steering rack, a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for distributing braking force to mitigate loss of traction and maintain control, an HVAC system, lighting (e.g., lighting such as head / tail lights for illuminating the exterior surroundings of the vehicle), and one or more other systems (e.g., a cooling system, a safety system, an on-board charging system, other electrical components such as DC / DC converters, high-voltage junctions, high-voltage cables, a charging system, a charge port, etc.). Additionally, the drive system 514 can include a drive system controller that can receive and preprocess data from the sensor system 506 and control the operation of various vehicle systems. In some examples, the drive system controller can include one or more processors and a memory communicatively coupled to the one or more processors. The memory can store one or more components for performing various functions of the drive system 514. Additionally, drive system 514 also includes one or more communication connections that enable the respective drive system to communicate with one or more other local or remote computing devices.

[0072] Computing device 504 may include one or more processors 516 and memory 518 communicatively coupled to the one or more processors 516. In the depicted example, memory 518 of computing device 504 stores a position estimation component 520, a perception component 522, a prediction component 534, a planning component 536 including collision avoidance component 524, a map component 538, and one or more system controllers 540. While shown as residing within memory 518 for illustrative purposes, it is contemplated that position estimation component 520, perception component 522, collision avoidance component 524, prediction component 534, planning component 536, map component 538, and one or more system controllers 540 may additionally or alternatively be accessible to computing device 504 (e.g., stored in different components of vehicle 502) and / or accessible to vehicle 502 (e.g., stored remotely).

[0073] In the memory 518 of the computing device 504, a position estimation component 520 may include functionality for receiving data from the sensor system 506 to determine the position of the vehicle 502. For example, the position estimation component 520 may include and / or request / receive a three-dimensional map of the environment and continuously determine the position of the autonomous vehicle within the map. In some examples, the position estimation component 520 may receive time-of-flight data, image data, lidar data, radar data, sonar data, IMU data, GPS data, wheel encoder data, or any combination thereof, using SLAM (Simultaneous Localization and Mapping) or CLAMS (Simultaneous Calibration, Localization and Mapping) to accurately determine the position of the autonomous vehicle. In some examples, the position estimation component 520 may provide data to various components of the vehicle 502 to determine an initial position of the autonomous vehicle for generating a trajectory, as described herein.

[0074] The perception component 522 may include functionality to perform object detection, segmentation, and / or classification. In some examples, the perception component 522 may provide processed sensor data indicating the presence of an entity proximate to the vehicle 502 and / or the classification of the entity as an entity type (e.g., vehicle, pedestrian, bicycle, building, tree, road surface, curb, sidewalk, unknown, etc.). In additional and / or alternative examples, the perception component 522 may provide processed sensor data indicating one or more characteristics associated with the detected entity and / or the environment in which the entity is located. In some examples, features associated with the entity may include, but are not limited to, x-position (global position), y-position (global position), z-position (global position), orientation, entity type (e.g., classification), entity speed, entity range (size), etc. Features related to the environment may include, but are not limited to, the presence of another entity in the environment, the state of another entity in the environment, time of day, day of the week, season, weather conditions, darkness / light indication, etc.

[0075] As described above, the perception component 522 can use a perception algorithm to determine a perception-based bounding box associated with an object in the environment based on the sensor data. For example, the perception component 522 can receive image data and classify the image data to determine that an object is represented in the image data. Then, using a detection algorithm, the perception component 522 can generate a two-dimensional bounding box and / or a perception-based three-dimensional bounding box associated with the object. The perception component 522 can further generate a three-dimensional bounding box associated with the object. As described above, the three-dimensional bounding box can provide additional information, such as a position, orientation, pose, and / or size (e.g., size, width, height, etc.), associated with the object.

[0076] The perception component 522 may include functionality to store the sensory data generated by the perception component 522. In some examples, the perception component 522 may determine a trajectory corresponding to an object classified as an object type. By way of example only, the perception component 522 may capture one or more images of an environment using the sensor system 506. The sensor system 506 may capture images of the environment including an object, such as a pedestrian. The pedestrian may be at a first location at time T and at a second location at time T+t (e.g., movement during a time period t after time T). In other words, the pedestrian may move from the first location to the second location during this time span. Such movement may be recorded, for example, as stored sensory data associated with the object.

[0077] The stored sensory data, in some examples, may include fused sensory data captured by the vehicle. The fused sensory data may include a fusion or other combination of sensor data from sensor system 506, such as image sensors, lidar sensors, radar sensors, time-of-flight sensors, sonar sensors, global positioning system sensors, internal sensors, and / or any combination thereof. The stored sensory data may additionally or alternatively include classification data including a semantic classification of an object represented in the sensor data (e.g., pedestrian, vehicle, building, road surface, etc.). The stored sensory data may additionally or alternatively include trajectory data (position, orientation, sensor features, etc.) corresponding to the movement of an object classified as a dynamic object through the environment. The trajectory data may include multiple trajectories of multiple different objects over time. This trajectory data may be mined to identify images of particular types of objects (e.g., pedestrians, animals, etc.) when the object is stationary (e.g., standing still) or moving (e.g., walking, running, etc.). In this example, the computing device determines a trajectory corresponding to a pedestrian.

[0078] The collision avoidance component 524 may include functionality to calculate or otherwise determine accelerations for controlling the vehicle using a collision avoidance algorithm. The collision avoidance component 524 may determine a collision probability associated with the vehicle and the object. The collision avoidance component 524 may determine the collision probability based on a predicted intersection associated with the vehicle and the object. The collision avoidance component 524 may determine that the collision probability is equal to or greater than a first threshold. The collision avoidance component 524 may simulate a "reverse" vehicle trajectory and a "forward" object trajectory. The collision avoidance component 524 may simulate the "reverse" and "forward" to determine a collision probability associated with an updated predicted position associated with the vehicle and an updated predicted time associated with the object. The collision avoidance component 524 may iteratively repeat the "reverse" and "forward" simulations to determine an updated predicted position of the vehicle and the object based on a collision probability that is less than a second threshold.

[0079] The collision avoidance component 524 can determine the speed of the vehicle with respect to the probability of collision. A speed associated with the vehicle at an initial predicted location can be determined. A speed associated with the vehicle at an early predicted location can be determined based on interpolation between the speed associated with the vehicle at the initial predicted location and the original speed associated with the vehicle at the current location (e.g., start location).

[0080] The collision avoidance component 524 can determine an acceleration for controlling the vehicle based on a speed associated with the vehicle and a current location. The acceleration for controlling the vehicle can include a first acceleration utilized to control the vehicle to have a first speed when the vehicle reaches a first location. In some examples, the first acceleration can be utilized to stop the vehicle or to control the vehicle to have a first speed (e.g., a minimum speed (e.g., 0 m / s, 1 m / s, 2 m / s, etc.)) at the first location. Alternatively, the acceleration can include a second acceleration utilized to control the vehicle to decelerate to have a second speed at a second location. The acceleration for controlling the vehicle can include a most conservative acceleration associated with a location determined by the collision avoidance algorithm to be closest to the current location. The most conservative acceleration can include a first acceleration associated with a first location based on the first location being closer to the current location than the second location, or vice versa.

[0081] The prediction component 534 can generate one or more probability maps that represent predicted probabilities of possible locations of one or more objects in the environment. For example, the prediction component 534 can generate one or more probability maps of vehicles, pedestrians, animals, etc. within a threshold distance from the vehicle 502. In some examples, the prediction component 534 can measure the trajectories of the objects and generate a discretized predicted probability map, heat map, probability distribution, discretized probability distribution, and / or trajectory for the objects based on the observed and predicted behavior. In some examples, the one or more probability maps can represent the intent of one or more objects in the environment.

[0082] The planning component 536 can determine a path for the vehicle 502 to follow to navigate through an environment. For example, the planning component 536 can determine various routes and paths, as well as various levels of detail. In some examples, the planning component 536 can determine a route for traveling from a first location (e.g., a current location) to a second location (e.g., a target location). For purposes of this description, the route can be a series of waypoints for traveling between the two locations. As non-limiting examples, the waypoints include streets, intersections, Global Positioning System (GPS) coordinates, etc. Furthermore, the planning component 536 can generate instructions for guiding the autonomous vehicle along at least a portion of the route from the first location to the second location. In at least one example, the planning component 536 can determine how to guide the autonomous vehicle from a first waypoint in the series of waypoints to a second waypoint in the series of waypoints. In some examples, the instructions can be a path or a portion of a path. In some examples, multiple routes can be generated substantially simultaneously (i.e., within technical tolerances) according to a receding horizon technique, and one route among the multiple routes in a receding data horizon with the highest confidence level can be selected for driving the vehicle.

[0083] In other embodiments, the planning component 536 can alternatively or additionally use data from the perception component 522 and / or the prediction component 534 to determine a path for the vehicle 502 to follow to navigate through the environment. For example, the planning component 536 can receive data regarding objects associated with the environment from the perception component 522 and / or the prediction component 534. Using this data, the planning component 536 can determine a route to travel from a first location (e.g., a current location) to a second location (e.g., a target location) to avoid the objects in the environment. In at least some examples, such a planning component 536 can determine that such a collision-free path does not exist and then provide a path that leads the vehicle 502 to a safe stop that avoids all collisions and / or otherwise mitigates damage.

[0084] The memory 518 may further include one or more maps 538 that can be used by the vehicle 502 to navigate within the environment. For purposes of this description, a map may be any number of data structures modeled in two, three, or N dimensions and may provide information about the environment, such as, but not limited to, topology (such as intersections), streets, mountain ranges, roads, terrain, and the general environment. In some examples, the map may include, but is not limited to, covariance data (e.g., represented in a multi-resolution voxel space), texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), etc.), intensity information (e.g., LIDAR information, RADAR information, etc.), spatial information (e.g., image data projected onto a mesh, individual “surfels” (e.g., polygons associated with individual colors and / or intensities), reflectance information (e.g., specularity information, retroreflectivity information, BRDF information, BSSRDF information, etc.). In one example, the map may include a three-dimensional mesh of the environment. In some examples, as described herein, the map can be stored in a tile format and loaded into working memory as needed, such that individual tiles of the map represent distinct portions of the environment. In at least one example, the one or more maps 538 can include at least one map (e.g., an image and / or a mesh). In some examples, the vehicle 502 can be controlled at least in part based on the map 538. That is, the map 538 can be used in conjunction with the position estimation component 520, the perception component 522 (and subcomponents), the prediction component 534, and / or the planning component 536 to determine the position of the vehicle 502, identify objects in the environment, generate predictions associated with the objects and / or the vehicle 502, and / or generate a route and / or trajectory for navigating the environment.

[0085] In at least one example, the computing device 504 can include one or more system controllers 540, which can be configured to control steering, propulsion, braking, safety, emitter, communication, and other systems of the vehicle 502. These system controllers 540 can communicate with and / or control corresponding systems of the drive system 514 and / or other components of the vehicle 502, which can be configured to operate according to the path provided by the planning component 536.

[0086] Vehicle 502 may include one or more processors 546 and a memory 548 communicatively coupled to one or more processors 546, which may be connected to computing device 544 via network 542. In at least one example, one or more processors 546 may be similar to processor 516, and memory 548 may be similar to memory 518. In the illustrated example, memory 548 of computing device 544 stores teleoperation component 550 and / or model component 552. In at least one example, model component 552, after empirical testing and / or simulation, may generate ML models used by perception component 522, as described herein. While shown as residing in memory 548 for purposes of illustration, it is contemplated that teleoperation component 550 and model component 552 may additionally or alternatively be accessible to computing device 544 (e.g., stored in a different component of computing device 544) and / or accessible to computing device 544 (e.g., stored remotely).

[0087] The model component 552 may include functionality to generate a model for determining the height / classification of multi-channel image data, as described herein.

[0088] Processor 516 of computing device 504 and processor 546 of computing device 544 may be any suitable processor capable of processing data and executing instructions to perform operations, as described herein. By way of example and not limitation, processors 516 and 546 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or portion of a device that processes electronic data and converts it into other electronic data that can be stored in registers and / or memory. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices may also be considered processors so long as they are configured to implement encoded instructions.

[0089] Memory 518 of computing device 504 and memory 548 of computing device 544 are examples of non-transitory computer-readable media. Memories 518 and 548 can store an operating system and one or more software applications, instructions, programs, and / or data for implementing the methods and functionality attributed to the various systems described herein. In various embodiments, memories 518 and 548 can be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein include many other logical, programmatic, and physical components, and those shown in the accompanying drawings can be merely examples relevant to the description herein.

[0090] In some examples, some or all aspects of the components discussed herein may include any model, algorithm, and / or machine learning algorithm. For example, in some examples, the components in memories 518 and 548 may be implemented as neural networks.

[0091] As described herein, an exemplary neural network is a biologically inspired algorithm that passes input data through a series of connected layers to generate an output. Each layer in a neural network can contain another neural network and can include any number of layers (convolutional or not). As can be understood in the context of this disclosure, a neural network can refer to a broad class of algorithms that utilize machine learning and in which output is generated based on learned parameters.

[0092] Although discussed in the context of neural networks, any type of machine learning can be used consistent with the present disclosure. For example, machine learning or machine learning algorithms can include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), local estimation scatterplot smoothing (LOESS)), instance-based algorithms (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic nets, least angle regression (LARS)), decision tree algorithms (e.g., classification and regression trees (CART), ID3 (Iterative Dichotomizer)), and others. 3), Chi-squared Automatic Correlation Detection (CHAID), Decision Stamps, Conditional Decision Trees), Bayesian algorithms (e.g., Naive Bayes, Gaussian Naive Bayes, Multinomial Naive Bayes, Average-of-One Dependence Estimator (AODE), Bayesian Belief Networks (BNN), Bayesian Networks), Clustering algorithms (e.g., k-means, k-medians, Expectation Maximization (EM), Hierarchical Clustering), Association Rule Learning Algorithms (e.g., Perceptron, Backpropagation, Hopfield Networks, Radial Basis Function Networks (RBFN)), Deep Learning Algorithms (e.g., Deep Boltzmann Machines (DBM), Deep Belief Networks (DBN), Convolutional Neural Networks (CNN) , stacked autoencoders), dimensionality reduction algorithms (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), mixed discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble algorithms (e.g., boosting, bootstrap aggregation (bagging), Adaboost, stacked generalization (blending), gradient boosting machines (GBM), gradient boosting regression trees (GBRT), random forests), support vector machines (SVMs), supervised learning, unsupervised learning, semi-supervised learning, etc.

[0093] FIG. 6 is an exemplary process for determining a collision prediction and controlling an autonomous vehicle based at least in part on the collision prediction.

[0094] At operation 602, the process may include receiving sensor data captured by sensors on the autonomous vehicle within an environment. The sensor data may include data associated with an environment in which the autonomous vehicle is traveling based on the autonomous vehicle being located at a vehicle start position.

[0095] At operation 604, the process may include determining that the sensor data represents an object in the environment. The sensor data may include data associated with an object located at the object start position.

[0096] At operation 606, the process may include determining a predicted object trajectory associated with the object. The predicted object trajectory may include a movement of the object.

[0097] In operation 608, the process may include determining a vehicle trajectory associated with the autonomous vehicle. The vehicle trajectory may include the movement of the autonomous vehicle.

[0098] In operation 610, the process may include determining whether a first collision probability associated with the vehicle trajectory and the object trajectory exceeds a threshold, which may be a first threshold that triggers the initiation of a "backtracking" algorithm.

[0099] At operation 612, the process may include determining a first predicted position of the autonomous vehicle. The first predicted position may be associated with a predicted intersection of the vehicle with a narrowed portion of the road.

[0100] At operation 614, the process may include determining a second predicted position of the object at a third time after the first time. The second predicted position of the object may be associated with a predicted intersection point of the vehicle at the predicted time.

[0101] At operation 616, the process may include determining a second collision probability associated with the first predicted position of the autonomous vehicle. The second collision probability may be determined by utilizing a collision avoidance algorithm to simulate a "backward" vehicle trajectory and an "advance" object trajectory. The second collision probability may be determined based on an updated predicted position associated with the vehicle (e.g., the first predicted position minus a distance) and an updated predicted time associated with the object (e.g., the predicted time plus an additional time).

[0102] At operation 618, the process may include controlling the autonomous vehicle based on the new acceleration profile. The new acceleration profile may be determined based on a collision probability (e.g., a first collision probability or a second collision probability). The new acceleration profile may include an acceleration (e.g., a deceleration) of the vehicle. In some examples, the acceleration may include slowing or stopping the vehicle to arrive at or before the updated predicted position.

[0103] [Example of invention content] A. A system including one or more processors and one or more computer-readable media storing instructions executable by the one or more processors, the instructions, when executed, causing the system to: receive sensor data captured by sensors of an autonomous vehicle within an environment; determine that the sensor data represents an object within the environment; determine a predicted object trajectory associated with the object; determine a vehicle trajectory associated with the autonomous vehicle; and determine a first collision probability associated with the autonomous vehicle at a first location and the object at a first time based at least in part on the predicted object trajectory and the vehicle trajectory; and determine a first collision probability associated with the autonomous vehicle at a first location and the object at a first time based at least in part on the predicted object trajectory and the vehicle trajectory; determining a first predicted position of the autonomous vehicle at a second location along the vehicle trajectory, the second location being before the first location; determining a second predicted position of the object at a second time along the predicted object trajectory, the second time being after the first time; determining a second collision probability associated with the autonomous vehicle and the object based at least in part on the first predicted position and the second predicted position, the second collision probability being less than or equal to the first collision probability; and controlling the autonomous vehicle based at least in part on the second collision probability.

[0104] B. The system described in paragraph A, wherein the operations further include associating the first location with a first velocity, determining a second velocity associated with the autonomous vehicle at the second location based at least in part on interpolation based on the first velocity, and determining an acceleration associated with the autonomous vehicle based at least in part on the first velocity, and controlling the autonomous vehicle includes controlling the autonomous vehicle based at least in part on the acceleration.

[0105] C. The system of paragraph A or B, wherein the operations further include determining the first collision probability based at least in part on a position probability distribution associated with the object.

[0106] D. The system described in any of paragraphs A-C, wherein the operations further include determining a first collision probability based at least in part on a first geometric model of the autonomous vehicle associated with overestimating an actual collision probability and a second geometric model of the autonomous vehicle associated with underestimating the actual collision probability, wherein the first collision probability represents an aggregated collision probability based at least in part on the first geometric model and the second geometric model.

[0107] E. The system of any of paragraphs A-D, wherein the operations further include: determining a third predicted position of the autonomous vehicle at a third position along the vehicle trajectory, the third position being between the first position and the second position; determining a fourth predicted position of the object at a third time along the predicted object trajectory, the third time being between the first time and the second time; determining a third collision probability associated with the autonomous vehicle and the object based at least in part on the third predicted position and the fourth predicted position; determining an acceleration associated with the third collision probability; and determining that the third collision probability meets or exceeds a second threshold.

[0108] F: A method comprising: receiving sensor data captured by sensors on a vehicle in an environment; determining that the sensor data represents an object in the environment; determining a first collision probability associated with a first predicted position of the vehicle and a second predicted position of the object at a first time; determining a third predicted position of the vehicle, the second predicted position being before the first predicted position; determining a fourth predicted position of the object at a second time, the second time being after the first time; determining a second collision probability associated with the vehicle and the object based at least in part on the third predicted position and the fourth predicted position; and controlling the vehicle based at least in part on the second collision probability.

[0109] G: The method of paragraph F, further including receiving a predicted object trajectory associated with the object, and determining the second predicted position and the fourth predicted position based at least in part on the predicted object trajectory.

[0110] H: The method of paragraph F or G, further including receiving a predicted vehicle trajectory associated with the vehicle, and determining the first predicted position and the third predicted position based at least in part on the predicted vehicle trajectory.

[0111] I: The method of any of paragraphs F-H, further comprising representing the object using a geometric model that includes a position probability distribution associated with the object.

[0112] J: The method of paragraph I, wherein the geometric model is a first geometric model, the method further includes representing the vehicle as a second geometric model, and the first collision probability is based at least in part on the first geometric model and the second geometric model.

[0113] K: The method of any of paragraphs F-J, further comprising determining a first collision probability based at least in part on a first geometric model of the vehicle associated with determining a first probability and a second geometric model of the vehicle associated with determining a second probability, wherein the first collision probability represents an aggregated collision probability based at least in part on the first geometric model and the second geometric model.

[0114] L: The method of any of paragraphs F-K, further comprising determining the first collision probability based at least in part on a correction factor.

[0115] M: One or more non-transitory computer-readable media storing instructions executable by a processor that, when executed, cause the processor to perform operations including receiving sensor data captured by sensors on a vehicle in an environment; determining that the sensor data represents an object in the environment; determining a first collision probability associated with a first predicted position of the vehicle and a second predicted position of the object at a first time; determining a third predicted position of the vehicle, the second predicted position being before the first predicted position; determining a fourth predicted position of the object at a second time, the second time being after the first time; determining a second collision probability associated with the vehicle and the object based at least in part on the third predicted position and the fourth predicted position; and controlling the vehicle based at least in part on the second collision probability.

[0116] N. The one or more non-transitory computer-readable media described in paragraph M, wherein the operations further include receiving a predicted object trajectory associated with the object; and determining the second predicted position and the fourth predicted position based at least in part on the predicted object trajectory.

[0117] O. The one or more non-transitory computer-readable media described in paragraphs M or N, wherein the operations further include receiving a predicted vehicle trajectory associated with the vehicle; and determining the first predicted position and the third predicted position based at least in part on the predicted vehicle trajectory.

[0118] P: The one or more non-transitory computer-readable media of any of paragraphs M-O, further comprising representing the object using a geometric model that includes a position probability distribution associated with the object.

[0119] Q: The one or more non-transitory computer-readable media described in paragraph P, wherein the geometric model is a first geometric model, the operation further includes representing the vehicle as a second geometric model, and the first collision probability is based at least in part on the first geometric model and the second geometric model.

[0120] R: The one or more non-transitory computer-readable media described in any of paragraphs M-Q, wherein the operations further include determining the first collision probability based at least in part on a first geometric model of the vehicle associated with determining a first probability and a second geometric model of the vehicle associated with determining a second probability, and the first collision probability represents an aggregated collision probability based at least in part on the first geometric model and the second geometric model.

[0121] S. The one or more non-transitory computer-readable media of any of paragraphs M-R, wherein the operation further includes determining the first collision probability based at least in part on a correction factor.

[0122] T. One or more non-transitory computer-readable media described in any of paragraphs M-S, wherein controlling the vehicle further includes controlling the vehicle to slow down or stop based at least in part on a second collision probability.

[0123] Although the exemplary subject matter described above is described with respect to one particular embodiment, it should be understood in the context of this specification that the exemplary subject matter can also be implemented via a method, device, system, computer-readable medium, and / or other embodiments. Furthermore, any of Examples A-T can be implemented alone or in combination with any other one or more Examples A-T.

[0124] [Conclusion] One or more examples of the technology described herein have been described; however, various modifications, additions, permutations, and equivalents thereof fall within the scope of the technology described herein.

[0125] In the illustrative description, reference is made to the accompanying drawings, which form a part hereof, and which show, by way of illustration, specific examples of the claimed subject matter. It is understood that other examples may be used, and modifications or substitutions, such as structural changes, may be made. Such examples, modifications, or substitutions do not necessarily depart from the intended scope of the claimed subject matter. While steps herein may be presented in a particular order, in some cases the order can be changed so that certain inputs are provided at different times or in a different order without changing the functionality of the described systems and methods. Disclosed procedures can also be performed in a different order. Furthermore, the various calculations herein need not be performed in the order disclosed, and other examples using alternative orders of calculations can be readily implemented. In addition to reordering, calculations can also be decomposed into sub-calculations that produce the same result.

Claims

1. receiving sensor data captured by sensors associated with a vehicle in an environment; determining that the sensor data represents an object in the environment; determining a first collision probability associated with a first predicted position of the vehicle and a second predicted position of the object at a first time based at least in part on a correction factor; determining a third predicted position of the vehicle, the third predicted position being before the first predicted position; determining a fourth predicted position of the object at a second time, the second time being after the first time; and determining a second collision probability associated with the vehicle and the object based at least in part on the third predicted position and the fourth predicted position; controlling the vehicle based at least in part on the second probability of collision; A method comprising:

2. receiving a predicted object trajectory associated with the object; determining the second predicted position and the fourth predicted position based at least in part on the predicted object trajectory; The method of claim 1 further comprising:

3. receiving a predicted vehicle trajectory associated with the vehicle; determining the first predicted position and the third predicted position based at least in part on the predicted vehicle trajectory; 3. The method of claim 1 or 2, further comprising:

4. Receiving sensor data captured by sensors associated with a vehicle in an environment; determining that the sensor data represents an object in the environment; determining a first collision probability associated with a first predicted position of the vehicle and a second predicted position of the object at a first time; determining a third predicted position of the vehicle, the third predicted position being before the first predicted position; determining a fourth predicted position of the object at a second time, the second time being after the first time; and determining a second collision probability associated with the vehicle and the object based at least in part on the third predicted position and the fourth predicted position; controlling the vehicle based at least in part on the second probability of collision; representing the object using a first geometric model including a position probability distribution associated with the object, wherein the first collision probability is determined based on the position probability distribution; representing the vehicle as a second geometric model, the first collision probability being based at least in part on the first geometric model and the second geometric model; A method comprising:

5. Receiving sensor data captured by sensors associated with a vehicle in an environment; determining that the sensor data represents an object in the environment; determining a first collision probability associated with a first predicted position of the vehicle and a second predicted position of the object at a first time; determining a third predicted position of the vehicle, the third predicted position being before the first predicted position; determining a fourth predicted position of the object at a second time, the second time being after the first time; and determining a second collision probability associated with the vehicle and the object based at least in part on the third predicted position and the fourth predicted position; controlling the vehicle based at least in part on the second probability of collision; determining the first probability of collision based at least in part on a first geometric model of the vehicle associated with determining a first probability and a second geometric model of the vehicle associated with determining a second probability; Including, the first collision probability represents an aggregate collision probability based at least in part on the first geometric model and the second geometric model. method.

6. the vehicle is an autonomous vehicle, and before determining the first probability of collision, determining a predicted object trajectory associated with the object; determining a vehicle trajectory associated with the autonomous vehicle; determining a third collision probability associated with the autonomous vehicle at a first location and the object at a third time based at least in part on the predicted object trajectory and the vehicle trajectory; a first predicted position of the autonomous vehicle is determined at a second position along the vehicle trajectory, the second position being before the first position; the second predicted position of the object is determined along the predicted object trajectory at the first time, the first time being after the third time; 6. The method according to any one of claims 1 to 5.

7. Receiving sensor data captured by sensors associated with a vehicle in an environment; determining that the sensor data represents an object in the environment; determining a first collision probability associated with a first predicted position of the vehicle and a second predicted position of the object at a first time; determining a third predicted position of the vehicle, the third predicted position being before the first predicted position; determining a fourth predicted position of the object at a second time, the second time being after the first time; and determining a second collision probability associated with the vehicle and the object based at least in part on the third predicted position and the fourth predicted position; controlling the vehicle based at least in part on the second probability of collision; determining a fifth predicted position of the autonomous vehicle at a third position along the vehicle trajectory, the third position being between the first position and the second position; determining a sixth predicted position of the object at a fourth time along the object trajectory, the fourth time being between the first time and the third time; and determining a third collision probability associated with the autonomous vehicle and the object based at least in part on the fifth predicted position and the sixth predicted position; and determining an acceleration associated with the third probability of crash; determining that the third collision probability meets or exceeds a threshold; and A method comprising:

8. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause one or more computing devices to perform the method of any one of claims 1 to 7.

9. one or more processors; one or more computer-readable media storing instructions executable by the one or more processors; wherein the instructions, when executed, receiving sensor data captured by sensors associated with a vehicle in an environment; determining that the sensor data represents an object in the environment; determining a first collision probability associated with a first predicted position of the vehicle and a second predicted position of the object at a first time based at least in part on a correction factor; determining a third predicted position of the vehicle, the third predicted position being before the first predicted position; determining a fourth predicted position of the object at a second time, the second time being after the first time; and determining a second collision probability associated with the vehicle and the object based at least in part on the third predicted position and the fourth predicted position; controlling the vehicle based at least in part on the second probability of collision; causing the system to perform an action including

10. The operation is receiving a predicted vehicle trajectory associated with the vehicle; determining the first predicted position and the third predicted position based at least in part on the predicted vehicle trajectory; receiving a predicted object trajectory associated with the object; determining the second predicted position and the fourth predicted position based at least in part on the predicted object trajectory; The system of claim 9 further comprising:

11. One or more processors; one or more computer-readable media storing instructions executable by the one or more processors; wherein the instructions, when executed, receiving sensor data captured by sensors associated with a vehicle in an environment; determining that the sensor data represents an object in the environment; determining a first collision probability associated with a first predicted position of the vehicle and a second predicted position of the object at a first time; determining a third predicted position of the vehicle, the third predicted position being before the first predicted position; determining a fourth predicted position of the object at a second time, the second time being after the first time; and determining a second collision probability associated with the vehicle and the object based at least in part on the third predicted position and the fourth predicted position; controlling the vehicle based at least in part on the second probability of collision; representing the object using a first geometric model including a position probability distribution associated with the object; representing the vehicle as a second geometric model, the first collision probability being based at least in part on the first geometric model and the second geometric model; causing the system to perform an action including

12. One or more processors; one or more computer-readable media storing instructions executable by the one or more processors; wherein the instructions, when executed, receiving sensor data captured by sensors associated with a vehicle in an environment; determining that the sensor data represents an object in the environment; determining a first collision probability associated with a first predicted position of the vehicle and a second predicted position of the object at a first time; determining a third predicted position of the vehicle, the third predicted position being before the first predicted position; determining a fourth predicted position of the object at a second time, the second time being after the first time; and determining a second collision probability associated with the vehicle and the object based at least in part on the third predicted position and the fourth predicted position; controlling the vehicle based at least in part on the second probability of collision; determining the first probability of collision based at least in part on a first geometric model of the vehicle associated with determining a first probability and a second geometric model of the vehicle associated with determining a second probability; causing the system to perform an action including the first collision probability represents an aggregate collision probability based at least in part on the first geometric model and the second geometric model. system.

13. Controlling the vehicle includes:

13. The system of claim 9, further comprising controlling the vehicle to slow down or stop based at least in part on the second probability of collision.

Images