Vehicle, method, and non-transitory computer-readable medium for obstacle avoidance - Patents.com
The vehicle computing system in autonomous vehicles dynamically navigates around obstacles by predicting trajectories and optimizing costs, addressing inefficiencies and safety issues in traditional planning systems.
Patent Information
- Application Number
- JP2022517417
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-09-27
- Filing Date
- 2020-09-21
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-09-21
AI Technical Summary
Traditional planning systems in autonomous and semi-autonomous vehicles often prioritize conservative actions to avoid obstacles, leading to reduced vehicle progress and safety, causing secondary collisions and inefficiencies.
A vehicle computing system dynamically determines actions based on the likelihood of an obstacle blocking the path, using sensor data and machine learning to predict object trajectories, emit signals, and optimize costs such as safety, comfort, and progress to navigate around obstacles effectively.
Enhances vehicle safety and efficiency by optimizing actions to avoid obstacles while minimizing delays and collisions, ensuring safe and comfortable operation.
Smart Images

Figure 0007588639000001 
Figure 0007588639000002 
Figure 0007588639000003
Abstract
Description
[Technical field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This PCT international patent application is a continuation of, and claims the benefit of priority to, U.S. patent application Ser. No. 16 / 586,604, entitled "BLOCKING OBJECT AVOIDANCE," filed Sep. 27, 2019, the entire contents of which are incorporated herein by reference. [Background technology]
[0002] Planning systems in autonomous and semi-autonomous vehicles determine actions to take for the vehicle as it enters the operating environment. The actions for the vehicle may be determined based in part on avoiding objects present in the environment. For example, actions such as stopping for a pedestrian blocking the vehicle's path or slowing down behind a cyclist operating in its path may be generated. A conventional planning system may select an action for the vehicle based on determining that the action is the most conservative action, such as remaining stopped or staying behind the cyclist until the pedestrian is completely out of its path. However, such a conventional planning system may actually prevent the vehicle from operating safely in the environment (as such actions may cause a secondary collision) and may significantly impede the progress of the vehicle. As such, a conventional planning system may adversely affect the progress of the vehicle without necessarily improving operational safety. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] U.S. Patent Application Serial No. 16 / 151,607 [Patent Document 2] U.S. Patent Application Serial No. 16 / 504,147 [Patent Document 3] U.S. Patent Application Serial No. 15 / 807,521 [Patent Document 4] U.S. Patent Application Serial No. 16 / 539,928 [Patent Document 5] U.S. Patent Application Serial No. 15 / 982,694 [Brief description of the drawings]
[0004] The detailed description will be set forth with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number identifies the drawing in which that reference number first appears. Use of the same reference number in different drawings indicates similar or identical components or features. [Figure 1] FIG. 1 is an illustration of an environment in which an exemplary object avoidance system may be used by an autonomous vehicle to bypass block objects in the environment. [Diagram 2] FIG. 2 is a diagram of a process for determining to navigate a vehicle around a dynamic block object. [Diagram 3] FIG. 3 is an illustration of a process for determining to navigate a vehicle around a block object while staying in an operating lane. [Figure 4] FIG. 4 is a block diagram of an example system for implementing the techniques described herein. [Diagram 5] FIG. 5 illustrates an example process for determining to navigate around an object in an environment. [Figure 6] FIG. 6 illustrates an example process for controlling a vehicle around a block object according to an action based at least in part on a cost associated with the action. [Figure 7] FIG. 7 illustrates an example process for controlling a vehicle around a block object and yielding to a second object detected while navigating around the block object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0005] The present disclosure is directed to techniques for dynamically determining an action to be taken by a vehicle (e.g., a path to travel) based on a detected object obstructing the path (e.g., trajectory) of the vehicle. In one example, a vehicle computing system may detect an object at a location and determine that the location is at least partially within the path of the vehicle. Based on the likelihood that the object will continue to block the path, the vehicle computing system may determine whether to wait for the object to move out of the path or to alter the vehicle's trajectory to navigate around the object.
[0006] The vehicle may include an autonomous vehicle or a semi-autonomous vehicle having a vehicle computing system configured to detect one or more objects in the environment. The objects may include static objects (e.g., buildings, bridges, signs, etc.) and dynamic objects such as other vehicles (e.g., cars, trucks, motorcycles, mopeds, etc.), pedestrians, bicyclists, etc. Unless indicated to the contrary, the term "object" herein refers to a dynamic object (e.g., an object that is stationary for a moment but is moving and / or capable of moving). In an example, the object may be detected based on sensor data from a sensor of the vehicle (e.g., a camera, a motion detector, a lidar, a radar, time of flight, etc., or a combination thereof). In an example, the object may be detected based on sensor data received from a remote sensor, such as, for example, a sensor associated with another vehicle or a sensor mounted in the environment configured to share data with multiple vehicles. In an example, the vehicle computing system may be configured to semantically classify the detected object. The classification may include another vehicle (e.g., a car, a pickup truck, a semi-trailer truck, a tractor, a bus, a train, etc.), a pedestrian, a bicyclist, a horse rider, etc. For example, a vehicle computing system may detect two objects and classify the first object as a bicyclist and the second object as a pedestrian.
[0007] In various examples, the vehicle computing system can detect an object at a location within the environment. The vehicle computing system can determine that the object is blocking a path associated with the vehicle through the environment. In one example, the determination that the object at least partially blocks the path is based on a determination that the location is within a drivable surface along which the vehicle plans to travel (e.g., associated with the vehicle's trajectory).
[0008] In one example, the vehicle computing system may determine that an object is obstructing a path based on a determination that a vehicle traveling along the vehicle trajectory passes by the object less than a threshold distance (e.g., a minimum safe distance). In one example, the threshold distance may be based on a classification associated with the object. For example, a vehicle computing system programmed to maintain a minimum of four feet (or other distance) from pedestrians may detect a pedestrian standing in a bike lane. The vehicle computing system may determine that a vehicle traveling along the vehicle trajectory will pass three feet from the pedestrian, i.e., less than the minimum (e.g., threshold) distance. The vehicle computing system may determine that the pedestrian is a blocking object.
[0009] In various examples, the vehicle computing system can determine the likelihood that the object will continue to obstruct the path of the vehicle. In one example, the likelihood can be determined based on the object's location, a location relative to the object, the object's size, a level of stability (e.g., amount of movement, etc.), the object's velocity, the object's velocity change (e.g., velocity covariance), detected object actions, etc.
[0010] In one example, the likelihood may be based on a predicted object trajectory (e.g., one or more object trajectories) and / or a confidence associated with the trajectory. In one example, the predicted object trajectory may be based on a top-down representation of the environment, for example, by utilizing techniques described in U.S. Patent Application Publication No. 2010 / 0133634, entitled “Trajectory Prediction on Top-Down Scenes,” filed October 4, 2018, and U.S. Patent Application Publication No. 2010 / 0133634, entitled “Prediction on Top-Down Scenes based on Action Data,” filed July 5, 2019, the entire contents of which are incorporated herein by reference. In one example, the predicted trajectory may be determined using probabilistic heat maps (e.g., discretized probability distributions), tree search methods, and / or temporal logic formulae for predicting object behavior, such as those described in U.S. Patent Application Publication No. 2010 / 0133634, entitled “Probabilistic Heat Maps for Behavior Prediction,” filed November 8, 2017, the entire contents of which are incorporated herein by reference. In various examples, the predicted object trajectory may be determined utilizing tree search techniques, temporal logic, and / or machine learning techniques. In various examples, the confidence may represent a level of certainty that a vehicle computing system may have in the accuracy of the predicted object trajectory.
[0011] In various examples, the vehicle computing system may be configured to emit audio and / or visual signals toward the blocking object. In such examples, the audio and / or visual signals may be designed (e.g., frequency, volume, etc.) to alert the blocking object to the vehicle and encourage the object to move out of the vehicle's path. In various examples, the likelihood of the object moving out of the vehicle's path may be based on a detected reaction to the emitted signal from the vehicle. In such examples, the vehicle computing system may determine whether the object is likely to move out of the vehicle's path (e.g., change the object's trajectory) based on sensor data collected from the sensor. For example, the vehicle computing system may determine that a pedestrian obstructing the path adjusts its position in response to the emitted signal. The vehicle computing system may determine that the likelihood of the object continuing to block the path may be reduced based on the adjusted position, and that the vehicle should wait for the pedestrian to move out of the path.
[0012] In an example, the vehicle computing system may determine that the likelihood of the object leaving the vehicle's path is equal to or greater than a threshold likelihood (e.g., 50%, 60%, etc.). In an example, based on a determination that the likelihood exceeds the threshold, the vehicle computing system may determine to slow the forward speed of the vehicle to provide time for the object to leave the vehicle's path. In an example, based on a determination that the likelihood exceeds the threshold, the vehicle computing system may slow the vehicle to a speed substantially similar to the speed of the object and wait for the object to completely leave the vehicle's path. In an example where the object is dynamic, the vehicle may substantially match a speed associated with the object and maintain a safe distance behind the object. In an example where the object is stationary, the vehicle computing system may cause the vehicle to stop and / or maintain a stopped position.
[0013] In various examples, the vehicle computing system can determine that a likelihood that the object will deviate from the vehicle's path is below a threshold, and the vehicle computing system can determine to modify the vehicle trajectory to navigate around the object. In one example, the decision to modify the vehicle trajectory to navigate around the object can be based on determining that the likelihood is below a second (e.g., minimum) threshold (e.g., 20%, 30%, etc.). In various examples, the modification of the vehicle trajectory can include changing position within a lane, changing lanes, changing the speed of the vehicle, etc.
[0014] In various examples, the vehicle computing system can determine a cost associated with each action, such as that described in U.S. Patent Application Publication No. 2010 / 0133634, filed Aug. 13, 2019, entitled “Cost-Based Path Determination,” the entire contents of which are incorporated herein by reference. The cost can be based on one or more of a safety cost (e.g., vehicle and / or object safety, collision avoidance between the vehicle and the object), a comfort cost (e.g., eliminating sudden movements by penalizing large accelerations less than a minimum distance between the vehicle and the object), a progress cost (e.g., travel to a destination), a behavior rule cost (e.g., rules of the road, laws, rules, regulations, etc.), and the like. In an example, the vehicle computing system can select an action to perform based on a cost associated therewith. In an example, the vehicle computing system can select an action based on the action with the lowest cost associated therewith compared to other actions. In an example, the vehicle computing system can determine an action to take based on a cost associated therewith being less than a threshold cost.
[0015] For example, the vehicle computing system may determine that the lane in which the vehicle is operating is blocked by an object standing in the photo lane, but that adjacent lanes are free of other objects. The vehicle computing system may calculate a cost associated with operating in the adjacent lane to navigate around the block object. Based on a determination that the cost is equal to or less than a threshold, the vehicle computing system may cause the vehicle to navigate into the adjacent lane around the block object. Based on a determination that the cost is above a threshold, the vehicle computing system may cause the vehicle to stop (or maintain a stopped position) before a location associated with the block object.
[0016] The technology discussed herein may include improvements to the field of autonomous and / or semi-autonomous vehicle control planning. Traditionally, in autonomous vehicle control planning, a vehicle computing device may detect an object blocking a path, such as a pedestrian on a road, and stop the vehicle to wait for the pedestrian to move out of the way. Stopping to wait for the blocking object while ensuring the safety of the pedestrian and the vehicle may sacrifice other factors such as the progress and comfort of the vehicle, the vehicle's occupants, and other vehicles on the road (e.g., traffic congestion while waiting for the pedestrian to move). However, the technology described herein includes a means by which the vehicle computing system can select an action that optimizes the safety, comfort, and progress of the vehicle. Thus, the technology described herein improves the field of autonomous and / or semi-autonomous vehicles.
[0017] The techniques described herein may be implemented in a number of ways. Exemplary implementations are provided below with reference to the accompanying drawings. Although described in the context of an autonomous vehicle, the methods, apparatus, and systems described herein may be applied to a variety of systems (e.g., sensor systems, or robotic platforms) and are not limited to autonomous vehicles. In one example, similar techniques may be utilized in driver-controlled vehicles where such systems may provide indications of whether it is safe to perform various operations. In another example, the techniques may be utilized in an aviation or nautical context, or in any system that uses planning techniques.
[0018] 1 is an illustration of an environment 100 in which one or more computing systems 102 of an autonomous vehicle 104 (e.g., the vehicle 104) may detect one or more objects 106 (e.g., a block object 106) blocking the path of the vehicle 104 and control the vehicle 104 based on the likelihood that the block object 106 will deviate from the path of the vehicle 104. The block objects 106 may include those that impede the forward progress of the vehicle 104 along a predetermined path and / or that determine a trajectory (e.g., a speed and / or direction of movement) associated with travel from a location to a destination. Such determinations may be based on a threshold speed of the vehicle and / or a cost of travel associated with the vehicle, as discussed elsewhere herein. The block object may be substantially stationary (e.g., a pedestrian standing in front of the vehicle 104 and / or moving around, etc.) and / or dynamic (e.g., a bicyclist in the roadway in front of the vehicle 104 traveling at a speed slower than the speed associated with the trajectory of the vehicle 104, a pedestrian walking in the roadway in front of the vehicle 104, such as a pedestrian navigating a construction zone). The vehicle 104 can detect the block object 106 and one or more other objects 108 based on sensor data captured by one or more sensors 110 of the vehicle 104. The sensor data captured by the sensors 110 may include data captured by a lidar sensor, a camera sensor, a radar sensor, a time-of-flight sensor, a sonar sensor, etc.
[0019] In an example, the sensor data may be provided to a perception component 112 configured to determine a classification 114 associated with the block object 106 and / or other objects 108 (e.g., vehicle, pedestrian, bicycle, motorcycle, animal, etc.). In various examples, the perception component 112 may determine the object classification 114 based on one or more features associated with the block object 106 and / or other objects 108. The features may include size (e.g., width, height, depth, etc.), shape (e.g., geometry, symmetry, etc.), and / or other distinguishing features of the object 106 and / or object 108. For example, the perception component 112 may recognize a size and / or shape of a block object 106 corresponding to a pedestrian and another object 108 corresponding to a cyclist.
[0020] In various examples, the perception component 112 may be configured to determine that the block object 106 is blocking a vehicle path 116 (path 116) associated with the vehicle 104 passing through the environment 100. In an example, the vehicle path 116 may include a path of the vehicle 104 from a current location to a destination. In an example, the path 116 may include a drivable surface (e.g., a drivable area) associated with the movement of the vehicle 104 to the destination. In an example, the drivable surface may include a width of the vehicle 104 and / or a safety margin on either side of the vehicle 104. In an example, the drivable surface may include a width of the lane 118.
[0021] In an example, the perception component 112 may determine that a block object 106 is blocking the vehicle path 116 based on a determination that a location associated with the block object 106 is at least partially within the drivable area and / or the vehicle's path 116. In an illustrative example, at least a portion of each of three block objects 106 is located within the vehicle path 116. In another example, based on a determination that at least a portion of at least one of the block objects 106 is located within the path 116, the perception component 112 may determine that a group of three pedestrians are designated as block objects 106. In an example, based on a determination that at least a portion of at least one of the block objects 106 is located within the path 116, the perception component 112 may generate a bounding box 120 around the group of block objects 106. In such an example, the perception component 112 may consider the group of block objects 106 together as a whole.
[0022] In various examples, the perception component 112 may determine that the block object 106 is blocking the vehicle path 116 based on a determination that the vehicle cannot go around the block object 106 in the lane 118. In such examples, the distance D may be less than the width of the vehicle plus a minimum safe distance (e.g., a safety margin) from the block object 106. In an example, the minimum safe distance may be based on a classification 114 associated with the block object 106. For example, a minimum safe distance associated with a pedestrian may be 1 meter, and a minimum safe distance associated with a bicyclist may be 1.2 meters. In various examples, the minimum safe distance may be based on the speed of the vehicle. In such examples, the faster the vehicle 104 moves, the greater the minimum safe distance, or vice versa, the slower the vehicle 140 moves, the smaller the minimum safe distance. For example, a vehicle traveling at 10 miles per hour may include a minimum safe distance of 3 feet from a pedestrian, and a vehicle traveling at 20 miles per hour may include a minimum safe distance of 5 feet. The distances and speeds are merely illustrative examples, however, other speeds and / or distances may be contemplated.
[0023] In various examples, based on a determination that the block object 106 is blocking the path 116, the perception component 112 may blend the block object 106 into the drivable area of the vehicle 104, as described in U.S. Patent Application Publication No. 2013 / 0133999, entitled “Drive Envelope Determination,” filed May 17, 2018, the entire contents of which are incorporated herein by reference. In such examples, blending the block object 106 includes modifying the drivable area such that the location occupied by the block object 106 is removed from the available drivable surface (e.g., is no longer in an area in which the vehicle 104 can move). In one example, a reference line (e.g., a centerline of vehicle movement) associated with the vehicle path 116 may be shifted a distance from the block object 106. The distance may include a minimum safe distance or a greater distance from the block object 106.
[0024] In various examples, the perception component 112 may be configured to provide sensor data (e.g., a determination of a block object, a classification of the block object 106, etc.) to the planning component 122 to determine an action to be taken by the vehicle. In one example, the action may be determined based on the likelihood that the block object 106 will continue to block the vehicle path 116. In one example, the determination of continued blocking may be based on the likelihood that the block object 106 will remain (at least partially) on the path 116 for a threshold time (e.g., 15 seconds, 30 seconds, 1 minute, etc.). In an example, the likelihood may be determined based on the location of the block object 106, one or more locations associated with the block object 106 (e.g., known locations of tourist interest, known locations of photography, etc.), the classification 114 of the block object 106, the size of the block object 106 (e.g., width, height, number of pedestrians in a group, etc.), the level of stability (e.g., amount of movement, etc.), the speed of the block object 106, the speed change of the block object 106 (e.g., speed covariance), a detected object action (e.g., taking a photo, pointing at another object, etc.), a known time associated with a detected object action (e.g., a time associated with taking a photo at a landmark, etc.), and the like. For example, the block object 106 may be present at a location associated with a tourist activity and may be determined to be taking a photo with an image capture device. The planning component 122 may determine that the block object 106 is likely to continue blocking the vehicle's path 116 (e.g., continued blocking is likely) based on the location and the detected activity (e.g., action) associated with the block object 106.
[0025] In various examples, the planning component 122 may be configured to determine the likelihood that the block object 106 will continue to block the path 116 by utilizing machine learning techniques. In such examples, one or more machine learning algorithms may be trained utilizing training data including a plurality of block objects 106. In various examples, the machine learning algorithm may be trained according to one or more characteristics of the block object 106, such as the location, size, level of stability, detected object actions, speed, speed changes, etc., described above. In such examples, the machine learning algorithm may be configured to determine the likelihood that the block object 106 will continue to block the vehicle path 116 based on the characteristics.
[0026] In an example, the likelihood may be based on one or more predicted trajectories (e.g., object trajectories) associated with the block object 106 and / or a confidence associated with the object trajectory. In such an example, the prediction component of the planning component 122 may be configured to determine a predicted object trajectory. For example, based on a determination that the predicted object trajectory includes little or no velocity in a direction that deviates from the path 116, the planning component 122 may determine that the block object 106 is likely to remain on the path 116. Conversely, based on a determination that the object trajectory includes a direction of movement that deviates from the path 116, the planning component 122 may determine that the block object 106 is unlikely to remain on the path 116.
[0027] In one example, the object trajectory can be based on a top-down representation of the environment, such as by utilizing techniques described in U.S. Patent Nos. 5,993,333 and 5,949,623, the entire contents of which are incorporated herein by reference. In one example, the predicted object trajectory can be determined using probabilistic heat maps (e.g., discretized probability distributions), tree search methods, temporal logic formulas, and / or machine learning techniques for predicting object behavior, such as those described in U.S. Patent No. 5,993,333, the entire contents of which are incorporated herein by reference. In various examples, the confidence can represent a level of certainty that the planning component 122 can have in the accuracy of the predicted object trajectory.
[0028] In an example, the planning component 122 can determine an action to take based on the likelihood. In an example where the object is dynamic, the vehicle can substantially match a speed associated with the object and maintain a safe distance behind the object. In an illustrative example, the block object 106 is substantially static (e.g., standing in the lane 118). In such an example, the action can include stopping the vehicle 104 (and / or maintaining a stopped position) before a location associated with the block object 106. In an example, the planning component 122 can determine to stop the vehicle 104 and / or maintain a stopped position before the location based on a determination that the block object 106 is unlikely to remain in the vehicle path 116. In an example, the decision to stop (remain stopped) can be based on a determination that the likelihood is equal to or less than a likelihood threshold (e.g., 25%, 35%, 50%, etc.).
[0029] In various examples, the action may include modifying the vehicle trajectory such that the vehicle 104 travels a second path (e.g., a different path). In one example, the modification of the vehicle trajectory to travel the second path may be based on a determination that a likelihood that the block object 106 will remain on the first path 116 is equal to or greater than a likelihood threshold (e.g., 50%, 60%, etc.). The second path may include a vehicle path 116 that detours around the block object 106. The second path and / or associated vehicle trajectory may result in the vehicle 104 continuing forward toward the destination while avoiding the block object 106 by at least a minimum safe distance. The vehicle trajectory and / or second path may be associated with adjusting the position of the vehicle 104 within the lane 118, a lane change (e.g., to an adjacent lane 126), and / or adjusting the position of the vehicle 104 at least partially outside the lane 118 (e.g., to a shoulder, a bike lane, etc.) to safely navigate around the block object 106.
[0030] In various examples, the action may include adjusting a position within the lane 118 to navigate around the block object 106. In one example, the planning component 122 may determine whether the vehicle 104 can proceed around the block object 106 in the lane 118 (e.g., whether adjusting a position is a viable action). In such an example, the planning component 122 may determine a distance (D) between the block object 106 and / or its associated bounding box 120 and a lane marker 124 (e.g., a road marking that delineates the edge of a lane, etc.). The planning component 122 may determine whether the distance (D) is greater than or equal to a width of the vehicle 104 and / or a safety buffer (e.g., a minimum safe distance) from the block object 106. As described above, the minimum safe distance may be based on a classification 114 associated with the block object 106.
[0031] In various examples, based on a determination that the distance (D) is greater than or equal to the width of the vehicle 104 plus a minimum safe distance, the planning component 122 can determine that adjusting the position within the lane 118 is a possible action. In such an example, a vehicle 104 that is at a maximum lateral distance from a block object 106 within the lane 118 may be at least a minimum safe distance from the block object 106. In various examples, the planning component 122 can determine that adjusting the position within the lane 118 is a possible action based at least in part on an action cost associated therewith. As described in further detail below, the planning component 122 can determine an action cost associated with an action, such as adjusting the position within the lane on a first trajectory associated with a first distance and a first speed from the block object 106, adjusting the position within the lane on a second trajectory associated with a second distance and a second speed from the block object 106, and the like. In at least some such examples, such costs may include, for example, costs based on the width available in the lane (e.g., such that the cost increases exponentially with respect to the width of the vehicle and / or additional buffers). As a non-limiting example, the cost may be exponential or otherwise polynomial that is a function of the width of the lane relative to a width threshold. In an example, such costs may be based on a distance to the nearest object, which may include the boundary of the drivable surface. In various examples, the planning component 122 may determine an action cost associated with each potential action and select a behavior based on the respective action cost. In an example, the actions may be determined based on cost optimization. In such an example, the most cost-effective action for the vehicle may be selected.
[0032] In an example, based on the feasibility determination, the planning component 122 may determine one or more passing speeds associated with the vehicle navigating around the block object 106 at one or more passing distances. The passing distance may include one or more distances that are equal to or greater than a minimum safe distance between the vehicle 104 and the block object 106 within the lane 118. In other words, the passing distance may include any distance associated with the passing of the vehicle 104 outside the minimum safe distance within the lane 118. In various examples, the speed may be determined based on the cost optimization described above. In such examples, the distance and / or the speed may be determined based on the most cost-effective action (e.g., the lowest cost action) for the vehicle to take.
[0033] In various examples, the passing speed may be associated with the passing distance. In such examples, the greater the distance beyond the minimum safe distance, the faster the vehicle 104 may travel. In various examples, the vehicle 104 may be limited to a maximum passing speed while navigating around the block object 106. In such examples, the planning component 122 may limit the passing speed of the vehicle 104 to the maximum passing speed. In one example, the maximum passing speed may include a relative speed between the vehicle 104 and the block object 106. For example, the vehicle 104 may be limited to passing the block object 106 at 15 miles per hour (or other speed) relative to a speed associated with the object's trajectory.
[0034] In one example, the passing speed and / or the maximum passing speed can be based on the classification 114 associated with the block object 106. For example, the maximum passing speed at which the vehicle 104 can pass a cyclist can be 15 kilometers per hour, and the maximum passing speed at which the vehicle 104 can pass a pedestrian can be 10 kilometers per hour, although these are merely example passing speeds and any other speeds are contemplated herein. As another example, the planning component 122 can access a first table of passing distances and associated speeds corresponding to the passing of a pedestrian in the lane 118, and a second table of passing distances and associated speeds corresponding to the passing of a cyclist as shown in FIG.
[0035] In various examples, the action may include at least partially changing lanes into an adjacent lane 126. In an example, the lane change may include at least a portion of the vehicle 104 exiting the lane 118 and entering an adjacent lane 126, a shoulder of the road, a bike lane, etc. In various examples, the planning component 122 may determine to change lanes based on a determination that positioning within the lane 118 is not feasible. In an example, actions related to lane changing may be restricted by the color and / or style of the lane markers 124 (e.g., dashed, solid, double, etc.) and / or environmental factors (e.g., road curvature, speed limit, etc.). In such examples, the planning component 122 may be configured to determine whether a lane change is a feasible action based on the color and / or style of the lane markers 124 and / or environmental factors. For example, the vehicle 104 may cross a double solid yellow lane marker 124 based on a determination that the speed limit is below the speed limit threshold and the road is substantially straight (e.g., no turns).
[0036] In various examples, the planning component 122 can determine that the adjacent lane 126 is away from the other object 108. In an example, the planning component 122 can determine that a threshold area of the adjacent lane 126 is away from the other object 108. In an example, the threshold area can include at least a width of the vehicle 104 in the adjacent lane 126 and / or a safety buffer (e.g., minimum safe distance) from the other object 108. In an example, the threshold area can be associated with an area required to at least partially transition into the adjacent lane 126 without encroaching within a threshold distance (e.g., minimum safe distance) in front of, behind, or laterally of another object 108. In an example, the perception component 112 can provide data corresponding to occupancy of the adjacent lane 126 to the planning component 122.
[0037] In an example, the adjacent lane 126 may be associated with traffic traveling in the same direction as the vehicle 104. In such an example, the planning component 122 may determine to move into the adjacent lane 126 to go around the block object 106 and continue to the destination along the second path 116 in the adjacent lane 126. In an illustrative example, the adjacent lane 126 may be associated with traffic in the opposite direction. In such an example, the planning component 122 may determine, such as via sensor data provided by the perception component 112, that the adjacent lane 126 is away from oncoming traffic. In various examples, the threshold area associated with the adjacent lane 126 with opposite traffic may include an area of a distance that the vehicle 104 travels around the block object 106 plus a distance that another object 108 covers (e.g., at a speed limit associated with the environment 100) in the time required for the vehicle to go around the block object 106. In one example, based on a determination that another object 108 occupies the threshold area, the planning component 122 can cause the vehicle 104 to yield to the other object 108 before proceeding along the second path.
[0038] In various examples, the planning component 122 can determine the action to take based on the cost associated therewith. In such examples, the planning component 122 can compare costs associated with one or more different actions (e.g., maintaining a stopped position to wait for the block object 106 to move, adjusting a position within a lane, partially changing lanes, completely changing lanes, etc.) and select the action with the lowest cost. In various examples, the planning component 122 can determine costs associated with various iterations of each action. In such examples, the planning component 122 can determine costs associated with adjusting a position within a lane and / or partially or completely changing lanes at different distances and / or speeds. In various examples, the planning component 122 can determine the action to take based on optimizing the costs associated with each different action and / or iteration of the action.
[0039] In some examples, the cost may be based on one or more of safety (e.g., avoidance of collisions between the vehicle 104 and the block object 106 and other objects 108), comfort (e.g., lack of sudden movements of the vehicle 104), progress (e.g., movement of the vehicle 104 to a destination), operating rules (e.g., rules of the road, laws, regulations, etc.), and the like, as described in U.S. Patent No. 6,399,633, the entire contents of which are incorporated herein by reference.
[0040] In one example, the cost associated with safety (e.g., safety cost) may be based on a probability of collision between the vehicle 104 and the block object 106 and other objects 108. In one example, a fixed cost (e.g., 60, 80, 100, etc.) may be included if the probability of collision is greater than a predefined threshold (e.g., 40%, 51%, etc.) indicating the possibility of a collision. In one example, the fixed cost may include a cost value that exceeds a threshold such that the vehicle computing system cannot select an action associated therewith. In such an example, the probability of collision may include a constraint on the actions that the vehicle 104 may take.
[0041] In one example, the safety cost may be based on the relative position and / or trajectory between the vehicle 104 and the block object 106 and / or other objects 108. In such an example, the safety cost may increase the closer the vehicle 104 (moving on the vehicle trajectory) gets to the block object 106 and other objects 108. In one example, the safety cost may be based on the classification 114 associated with the block object 106 and other objects 108.
[0042] In various examples, the comfort cost may be associated with an estimated acceleration (e.g., positive, negative, lateral, etc.) and / or an estimated acceleration change (e.g., jerk) associated with the vehicle trajectory corresponding to the second path around the block object 106. In one example, the comfort cost may be associated with a distance between the vehicle 104 performing the associated action and the block object 106 and other objects 108. In such an example, the comfort cost may be associated with a closest approach point between the vehicle 104 and the block object 106 and other objects 108. In one example, the comfort cost may be associated with a passing speed associated with the action. In one example, the comfort cost may be associated with a relative speed between the vehicle 104 and the block object 106 and other objects 108. In one example, the distance and / or speed associated with the comfort cost may be based on the classification 114 associated with the block object 106 and other objects 108. In other examples, other states / controls of the vehicle 104 and / or considered objects 106 and / or 108 may be used in determining the comfort cost (which may be calculated, for example, as how close to other vehicles - nearest or next nearest neighbors, steering speed, turning speed, speed, etc.).
[0043] In various examples, the travel cost may include the progress of the vehicle 104 moving towards the destination. In one example, the travel cost may be calculated based on changes in the speed of the vehicle 104 and / or delays of the vehicle 104 caused by stopping, slowing down the vehicle, etc. In one example, the travel cost may include a value substantially similar to the number of seconds of delay caused by the action. In such an example, an action including stopping and / or maintaining a stopped position of the vehicle to wait for a block object 106 to move off the path 116 may be associated with a higher travel cost than one associated with an action including navigation around the block object 106.
[0044] In various examples, the behavior rule cost may be based on rules of the road (e.g., Department of Transportation laws, rules, regulations, etc.), rules of good driving, local driving habits (e.g., general driving practices), driving courtesy (e.g., adjusting position within a lane to provide space for another vehicle to turn right without occupying a bike lane, etc.). In various examples, the behavior rule cost associated with an action may be calculated based on one or more rules that have been broken and / or fulfilled. In such examples, the behavior rule cost may increase based on a rule that has been broken or not fulfilled (e.g., a regulation, custom, courtesy, etc.) and / or decrease based on a rule that has been fulfilled or fulfilled. For example, an action associated with changing lanes into oncoming traffic may include breaking a rule that the vehicle 104 should stay in a lane 118 designated for traffic operating in a direction associated with the vehicle path 116. The action may include a higher behavior rule cost than an action that includes adjusting the position of the vehicle 104 within the lane 118.
[0045] In various examples, the action rule cost may be determined based on rules, regulations, etc. associated with the lane markers 124. In such examples, the color of the lane markers 124 and / or whether the lane markers 124 are solid and / or broken may contribute to the action rule cost associated with the action. For example, an action associated with crossing a double yellow line may have a high action rule cost associated with it (e.g., 40, 50, etc.). As another example, a dashed white line may have a low action rule cost associated with it (e.g., 5, 10, etc.).
[0046] In various examples, the costs may be ranked or weighted in order of importance. In some examples, at least one of the costs (e.g., safety, comfort, progress, rules of operation) may include a cost that is weighted higher than the other costs. For example, safety may be weighted more heavily than other factors. In such examples, the planning component 122 may emphasize safety over other factors in determining the action to be taken by the vehicle 104.
[0047] In various examples, the planning component 122 may cause the vehicle 104 to perform actions associated with going around the block object 106. In such examples, the planning component 122 may continue to receive sensor data corresponding to the environment 100 from the perception component 112. The sensor data may include data associated with the block object 106 and / or other objects 108. In various examples, the perception component 112 may be configured to process the sensor data and determine a predicted object trajectory associated with the block object 106 and / or other objects 108. In such examples, the predicted object trajectory may include an updated object trajectory associated with the block object 106.
[0048] In various examples, the planning component 122 may determine that another object 108 may include an object trajectory that merges with a vehicle trajectory associated with the second path (e.g., navigating around the block object 106). Based on a determination that the object trajectories and / or the vehicle trajectory merge and / or the vehicle 104 comes within a threshold distance to the other object 108, the planning component 122 may cause the vehicle to slow down and / or stop before a position associated with the other object 108, for example, to avoid a collision associated therewith. The planning component 122 may cause the vehicle to continue to slow down and / or maintain a stopped position until it determines that the other object 108 is no longer a factor (e.g., the trajectories associated with the vehicle 104 and the other object 108 do not merge, there is no possibility of a collision between the vehicle 104 and the other object 108, etc.). For example, the vehicle 104 may determine to move into an adjacent lane 126 to navigate around the block object 106. While operating in adjacent lane 126, perception component 112 may detect other object 108(1) and determine that an object trajectory associated with the other object 108(1) crosses the vehicle trajectory associated with the action (e.g., associated with the second path). Planning component 122 may stop the vehicle at a location in front of the object trajectory (e.g., in front of the pedestrian crossing shown in environment 100).
[0049] In various examples, the planning component 122 can cause one or more signals to be emitted from the vehicle 104. In an example, the planning component 122 can attempt to move the block object 106 from the vehicle path 116 before the vehicle 104 determines an action to avoid the block object 106. The signal can include one or more of an audio signal or a visual signal (e.g., a light signal, a holographic image, etc.) emitted from an output device of the vehicle 104. In various examples, the signal can be emitted to alert the block object 106 of the presence of the vehicle 104. In an example, the signal can provide a means by which the block object 106 can determine that the vehicle 104 is present and / or that the block object 106 is blocking the path 116. Thus, the signal can be emitted with the intent of moving the block object 106 from the path 116. For example, the vehicle 104 can emit an audio signal, such as a gentle horn, to alert the block object 106 that is a pedestrian that they are in the vehicle path 116.
[0050] In an example, the signal may be emitted to inform the block object 106 of a potential direction in which the block object 106 may move out of the vehicle path 116. In such an example, the signal may be configured to assist the block object 106 in determining that the object trajectory moves out of the path 116. For example, the block object 106 may be a vehicle blocking an intersection of a first lane. The perception component 112 may determine that a second lane (e.g., a lane adjacent to the first lane) may include an area for the block object 106 to move into such that the block object 106 no longer blocks the vehicle path 116. The planning component 122 may receive information from the perception component 112 and emit a light (e.g., on a road visible to an operator of the block object 106) in the form of an arrow pointing in the direction of the second lane. Thus, the operator of the block object 106 may follow the arrow into the second lane to assist in determining a direction to move out of the way of the vehicle 104.
[0051] In an example, the perception component 112 may receive an indication that the block object 106 is not intended to move out of the way of the vehicle 104. The indication may include audio and / or visual signals received from the block object 106, such as via one or more sensors. In an example, the indication may include a movement or lack thereof indicating that the block object 106 is not intended to move out of the vehicle path 116. In an example, the indication may include a verbal indication, such as a pedestrian vocalizing "I'm not moving" or other phrase indicating an intent to remain in the vehicle path 116. In such an example, the indication may be received via one or more microphones on the vehicle 104 and processed by the computing system 102, which utilizes natural language processing and / or other means of language processing to determine one or more words spoken and / or their intent. In various examples, the indication may be utilized by the perception component 112 to determine the likelihood that the block object 106 will remain in the vehicle path 116. In such instances, the planning component 122 can determine an action to take based at least in part on the indication.
[0052] 2 is a diagram of a process 200 for determining to navigate a vehicle 104 around a dynamic block object 202, such as block object 106. In an illustrative example, the vehicle computing system may classify the dynamic block object 202 as a bicyclist. In other examples, the dynamic block object 202 may include objects of other classifications, such as a scooter, a moped, a pedestrian (e.g., a runner, a stroller, etc.).
[0053] At operation 204, the process may include detecting that the object is blocking a path associated with the vehicle trajectory 206, such as vehicle path 116. In various examples, the vehicle computing system may determine that the dynamic block object 202 is blocking the vehicle trajectory 206 based on a determination that at least a portion of the dynamic block object 202 is located in a lane 208 associated with the vehicle 104, such as lane 118. In an example, the vehicle computing system may determine that the dynamic block object 202 (e.g., one or more locations associated therewith) is at least partially within a drivable surface associated with the vehicle trajectory 206.
[0054] At operation 210, the process may include determining an object trajectory 212 associated with an object (e.g., the dynamic block object 202). In various examples, the object trajectory 212 may be determined based on sensor data captured over time. In such examples, the vehicle computing system may process the sensor data to determine a change in the position of the dynamic block object 202 over time and determine the object trajectory 212 (e.g., speed, change in speed, heading, etc.) based on the change in position over time. In one example, the vehicle computing system may determine the object trajectory 212 based on a top-down representation of the environment (e.g., determined utilizing the sensor data), such as by utilizing techniques described in U.S. Patent Nos. 5,993,333 and 5,993,363, the entire contents of which are incorporated herein by reference. In such examples, the object trajectory 212 may include a prediction of future dynamic block object 202 movement (e.g., direction of movement, speed, etc.). In some examples, the predicted object trajectory may be determined using probabilistic heat maps (e.g., discretized probability distributions), tree search methods, temporal logic formulas, and / or machine learning techniques for predicting object behavior such as those described in U.S. Patent Application Publication No. 2010 / 0239999, the entire contents of which are incorporated herein by reference. Similar to top-down representations, in such examples, the object trajectory may include a prediction of future dynamic block object 202 movement.
[0055] In various examples, based on a determination that the object trajectory 212 includes a heading direction substantially similar to the vehicle trajectory 206, the vehicle computing system may determine that the object is a dynamic block object 202. The dynamic block object 202 may include an object that blocks the vehicle path and is moving in a substantially similar direction, but at a slower speed than the vehicle 104.
[0056] In operation 214, the process may include determining that there is a likelihood that the dynamic block object 202 will continue to block the path of the vehicle 104, or greater than a threshold. In various examples, the likelihood may be determined based on a location of the dynamic block object 202, one or more locations associated with the dynamic block object 202 (e.g., known locations of tourist interest, known locations of photography, etc.), a classification of the block object 106, a size of the block object 106 (e.g., width, height, number of cyclists in a peloton, etc.), a level of stability (e.g., amount of movement, consistency of movement, etc.), a speed of the dynamic block object 202, a change in speed of the dynamic block object 202 (e.g., speed covariance), a detected object action (e.g., taking a photo, pointing at another object, etc.), a known time associated with the detected object action (e.g., a time associated with taking a photo at a landmark, etc.), and / or the like.
[0057] In various examples, the vehicle computing system may be configured to determine the likelihood that the dynamic block object 202 will continue to block the path of the vehicle 104 by utilizing machine learning techniques. In such examples, one or more machine learning models may be trained utilizing training data including a plurality of dynamic block objects 202. In various examples, the machine learning algorithms may be trained according to one or more features of the dynamic block object 202, such as position, size, level of stability, detected object actions, speed, speed changes, etc. In such examples, the machine learning models may be configured to determine the likelihood that the dynamic block object 202 will continue to block the path of the vehicle 104 based on the features.
[0058] In the depicted example, the dynamic block object 202, a cyclist, has an object trajectory 212 that has substantially the same heading direction as the vehicle trajectory 206 and has a substantially constant speed (e.g., minimal change in speed). Thus, the vehicle computing system can determine that the likelihood that the dynamic block object 202 will continue to block the path of the vehicle 104 exceeds a threshold likelihood (e.g., 55%, 65%, etc.).
[0059] At operation 216, the process may include determining a modified vehicle trajectory 218 (e.g., modified trajectory 218) based on the likelihood being equal to or greater than a threshold. In various examples, based on a determination that the likelihood that the object will continue to block the path of the vehicle 104 exceeds a threshold, the vehicle computing system may determine an action to take. In an example, the action may include slowing down to maintain a safe distance behind the dynamic block object 202 (e.g., slowing down to match a speed associated with the object trajectory 212), changing position within the lane 208, and / or at least partially changing lanes to an adjacent lane 220 to navigate around the dynamic block object 202. The modified trajectory 218 may be associated with the action.
[0060] In an illustrative example, the modified trajectory 218 includes a complete lane change into an adjacent lane. In such an example, the vehicle computing system may determine that a threshold area of the adjacent lane 220 is not occupied by another object. In other examples, the modified trajectory 218 may include a trajectory associated with another action, such as a partial lane change, an adjustment of position within the lane 208, or the like. As described above, the vehicle computing system may determine the action to perform and / or the modified trajectory 218 associated therewith based on a cost associated therewith (e.g., a safety cost, a comfort cost, a proceeding cost, a behavior rule cost, or the like). In such an example, the vehicle computing system may determine possible actions and determine to perform the action associated with the modified trajectory 218 based on the action having the lowest cost associated therewith. For example, the comfort cost and proceeding cost associated with maintaining a safe distance behind the dynamic block object 202 may cause the total cost associated with the action to be high. Because navigating around the dynamic block object 202 may be safer (if not relatively safer) than following behind the dynamic block object 202 and may have lower comfort and safety costs based on the increased speed at which the vehicle 104 may travel, the vehicle computing system may decide to navigate around the dynamic block object 202 according to the modified trajectory 218.
[0061] FIG. 3 is an illustration of a process 300 for determining to navigate a vehicle 104 around a block object 106 (e.g., object 106) while remaining within an operating lane 302 (e.g., lane 302), such as lane 118.
[0062] At operation 304, the process may include determining an object 106 that is at least partially blocking a first path 306 of the vehicle. As described above, the vehicle computing system may detect the block object 106 based on sensor data captured by one or more of the sensors mounted on the vehicle 104, other vehicles, and / or the operating environment. In various examples, the first path 306 may represent a drivable area in which the vehicle 104 plans to travel according to a planned path between a first location and a destination. In an example, the vehicle computing system may determine that the block object 106 is at least partially blocking the vehicle based in part on a determination that a location associated with the block object 106 is at least partially within the drivable area. In an example, based on a determination that the block object 106 is blocking the first path 306 of the vehicle 104, the vehicle computing system may fuse the block object 106 into a drivable area as described in U.S. Patent Application Publication No. 2010 / 0139994, which is incorporated by reference. In an illustrative example, the fusion is depicted as a cut-out of the drivable area. The cutout may include a buffer distance (eg, a minimum distance (eg, 3 inches, 6 inches, etc.)) for the vehicle 104 to maintain from the block object 106.
[0063] At operation 308, the process may include determining an action cost based at least in part on a distance (D) between the vehicle 104 at a position in the lane 302 adjacent the object 106 and a position associated with the object 106. The action cost may be associated with one or more of a safety cost, a comfort cost, a progress cost, or a motion rule cost associated with the action of going around the block object 106 while operating in the lane. In an example, the vehicle computing system may determine that the distance (D) is equal to or greater than a threshold distance. In such an example, the vehicle computing system may determine that going around the block object while remaining within the lane 302 is a viable action. Although shown with a position of the vehicle 104 at a maximum lateral distance from the block object 106, it is contemplated that the vehicle 104 may travel at another distance (D) from the block object 106 that meets or exceeds the threshold distance. Of course, while described herein in terms of thresholds, this is not meant to be so limiting. For example, an example may set a fixed threshold for decision making, but it should be understood that any discussion of a threshold herein may also refer to a cost expression set in such a way that a transition occurs between two outcomes based on optimizing for costs at or around such a threshold. Various examples contemplate a combination of hard constraints (e.g., fixed thresholds for decisions) as well as thresholds that represent transitions in cost optimization.
[0064] In various examples, the threshold distance (e.g., 10 inches, 2 feet, 1 yard, 1 meter, etc.) may represent a minimum safe distance for the vehicle to maintain from the block object 106 while operating in the environment. In one example, the threshold distance may be based on a classification associated with the block object 106. In such an example, the vehicle computing system may determine the classification and threshold distance associated therewith.
[0065] At operation 310, the process may include determining a vehicle speed associated with the distance. In various examples, the vehicle speed may be determined based on optimizing an action cost associated with going around the block object 106 at the distance (D). In such examples, the vehicle computing system may calculate action costs associated with one or more speeds and select the speed associated with the lowest action cost. For example, the vehicle computing system may determine a first action cost associated with passing 3 meters from the block object at 10 miles per hour, a second action cost associated with passing 3 meters from the block object at 15 miles per hour, and a third action cost associated with passing 3 meters from the block object at 20 miles per hour. The vehicle computing system may select the action with the lowest cost as the action to be performed by the vehicle.
[0066] In various examples, the vehicle speed may be based on a distance (D) that the vehicle 104 maintains from the block object 106. The distance that the vehicle 104 may maintain from the block object 106 may range from a threshold distance (e.g., a minimum safe distance) to a distance between the vehicle 104 and the block object 106 associated with the vehicle 104 at a maximum lateral position within the lane 302 (e.g., the vehicle 104 operating at a lane marker).
[0067] In various examples, the vehicle speed may be based on distance. In one example, the vehicle speed may increase as the vehicle increases distance away from the block object 106 (above a threshold distance). For example, the vehicle speed associated with the vehicle 104 navigating around the block object 106 at a minimum safe distance may be 5 miles per hour and 12 miles per hour at a maximum lateral position within the lane. The vehicle speeds are merely exemplary and any other speeds are contemplated herein.
[0068] In various examples, the vehicle speed may be based in part on a classification associated with the block object 106. In such examples, the vehicle computing system may determine a classification associated with the block object 106 and determine the vehicle speed based in part on the classification. In one example, a different classification (or taxonomy) of objects (e.g., wheeled vehicles (e.g., bicycles, scooters, mopeds, etc.), pedestrians (e.g., runners, walkers, people standing in place, etc.), etc.) may be determined.
[0069] In various examples, the vehicle speed associated with the distance and / or classification may be predetermined. For example, the vehicle computing system may store therein a table of vehicle speeds and distances associated with going around the block object 106. In an example, the table of vehicle speeds may be associated with a particular classification associated with the block object 106. In various examples, the vehicle speed associated with the distance and / or classification may be dynamically determined based on one or more environmental conditions. The environmental conditions may include weather considerations (e.g., rain, snow, etc.), road conditions (e.g., smooth surfaces, gravel, boreholes, etc.), density of objects (e.g., number of vehicles, pedestrians, etc. in the environment), proximity to other objects (e.g., adjacent lanes, etc.), etc. In various examples, the dynamic determination of the vehicle speed may include adjusting the predetermined vehicle speed associated with the distance and / or classification for the environmental condition. For example, the vehicle computing system may determine that the vehicle speed associated with the distance from the block object 106 is 10 kilometers per hour and that due to rain and wet road surfaces, the vehicle speed should be 8 kilometers per hour.
[0070] In various examples, in response to determining the distance and vehicle speed passing the block object 106, the vehicle computing system may determine a second path 312. The second path 312 may include a width of the vehicle traveling on the track associated with the distance and the vehicle speed. In various examples, the second path 312 may include a buffer distance (e.g., 6 inches, 8 inches, etc.) to the width of the vehicle. In such examples, the buffer distance may include a safety buffer on one or both sides of the vehicle.
[0071] In various examples, the second path may be determined based on action cost optimization. In such examples, the vehicle computing system may determine action costs associated with multiple actions and determine an action based on the lowest cost action. In one example, the vehicle computing system may determine a trajectory associated with the lowest cost action and drive the vehicle along the trajectory associated with the lowest cost action (e.g., the optimized action). For example, the actions may include navigating around the block object at various distances (e.g., distances associated with staying in the lane, partially changing lanes, completely changing lanes, etc.) and various speeds. The selected action (e.g., the optimal action) may include the action associated with the lowest overall cost (e.g., total cost including safety cost, progress cost, comfort cost, and / or behavior rule cost).
[0072] In operation 314, the process may include controlling the vehicle according to a modified vehicle trajectory 316 associated with the vehicle speed (determined in operation 310) and the distance (determined in operation 308). The vehicle computing system enables the vehicle to go around the block object 106 on the modified vehicle trajectory 316. As described in more detail below with respect to FIG. 4, the vehicle computing system can cause one or more drive systems to move the vehicle 104 along the second path 312 according to the modified vehicle trajectory 316.
[0073] In various examples, the vehicle computing system may determine to control the vehicle according to the modified vehicle trajectory 316 based on a determination that the action is the lowest cost action with a cost associated therewith (e.g., a cost associated with the action). In an example, the vehicle computing system may determine a cost associated with each action the vehicle can take. The actions may include going around the block object 106, going around the block object 106 traveling on another drivable surface (e.g., an adjacent lane, a shoulder, a bike path, etc.), and / or stopping before a location associated with the block object 106 and / or maintaining a stopped position. In various examples, the vehicle computing system may determine a cost associated with each possible (e.g., possible) action. In various examples, the vehicle computing system may determine that a cost associated with an action corresponding to the modified vehicle trajectory 316 comprises the lowest cost of the possible actions. In such an example, the action corresponding to the modified vehicle trajectory 316 may comprise the safest and / or most rapid (e.g., efficient) action for the vehicle 104 to travel to the destination.
[0074] 4 is a block diagram of an example system 400 for implementing the techniques described herein. In at least one example, the system 400 may include a vehicle 402, such as vehicle 104. The vehicle 402 may include one or more vehicle computing devices 404, one or more sensor systems 406, one or more emitters 408, one or more communication connections 410, at least one direct connection 412, and one or more drive systems 414.
[0075] The vehicle computing device 404 may include one or more processors 416 and a memory 418 communicatively coupled to the one or more processors 416. In the illustrated example, the vehicle 402 is an autonomous vehicle, however, the vehicle 402 may be any other type of vehicle, such as a semi-autonomous vehicle, or any other system having at least an image capture device (e.g., a camera-enabled smartphone). In the illustrated example, the memory 418 of the vehicle computing device 404 stores a positioning component 420, a perception component 422, a planning component 424 including a prediction component 426 and an action cost component 428, one or more system controllers 430, and one or more maps 432. 4 as residing in memory 418 for illustrative purposes, it is contemplated that the positioning component 420, the perception component 422, the planning component 424, the one or more system controllers 430, the one or more maps 432, the prediction component 426, and the action cost component 428 may additionally or alternatively be accessible to the vehicle 402 (e.g., may be stored on a memory remote from the vehicle 402, such as on memory 434 of one or more computing devices 436, or otherwise accessible from the vehicle 402). For example, the memory 434 may store the action cost component 428 accessible by the planning component 424 of the vehicle 402.
[0076] In at least one example, the positioning component 420 may include functionality for receiving data from the sensor system 406 to determine a position and / or orientation (e.g., one or more of an x position, a y position, a z position, a roll, a pitch, or a yaw) of the vehicle 402. For example, the positioning component 420 may include and / or request / receive a map of the environment, such as from a map 432, and may continually determine a position and / or orientation of the autonomous vehicle within the map. In one example, the positioning component 420 may utilize simultaneous localization and mapping (SLAM), calibration, localization and mapping, simultaneously (CLAMS), relative SLAM, bundle adjustment, nonlinear least squares optimization, etc., to receive image data, lidar data, radar data, IMU data, GPS data, wheel encoder data, etc., to accurately determine a position of the autonomous vehicle. In one example, the positioning component 420 may provide data to various components of the vehicle 402 as described herein to determine an initial position of the autonomous vehicle to determine if an object is associated with the vehicle 402.
[0077] In an example, a perception component 422, such as the perception component 112, may include functionality to perform object detection, segmentation, and / or classification. In an example, the perception component 422 may provide processed sensor data indicating the presence of an object (e.g., an entity) in proximity to the vehicle 402 and / or a classification of the object as an object type (e.g., a car, a pedestrian, a bicyclist, an animal, a structure, a tree, a road surface, a curb, a sidewalk, an unknown, etc.). In an example, the perception component 422 may provide processed sensor data indicating the presence of a stationary entity in proximity to the vehicle 402 and / or a classification of the stationary entity as a type (e.g., a building, a tree, a road surface, a curb, a sidewalk, an unknown, etc.). In various examples, the perception component 422 may provide processed sensor data indicating the presence of a block entity (e.g., a block object).
[0078] In an example, the perception component 422 may determine that a detected entity (e.g., an object) is a block object based on a determination that a location associated with the block object is at least partially within the drivable area and / or the vehicle's path. In various examples, the perception component 422 may determine that an object is a block object based on a determination that the object is impeding the vehicle's advancement along a predetermined path, and / or may determine a trajectory (e.g., speed and / or direction of travel) associated with movement from the location to the destination, or not. In various examples, the perception component 422 may determine that a detected entity is a block object based on a determination that the vehicle 402 cannot navigate around the block object in a lane associated with the vehicle's movement through the environment. In such an example, a distance between the detected entity and a lane marker associated with the lane may be less than a width of the vehicle 402 plus a minimum safe distance (e.g., a safety margin) from the detected entity. In an example, the minimum safe distance may be based on a classification associated with the detected entity.
[0079] In further or alternative examples, the perception component 422 can provide processed sensor data indicative of one or more characteristics associated with a detected entity (e.g., a tracked object, a block object, etc.) and / or an environment in which the object is located. In one example, the characteristics associated with the object can include, but are not limited to, an x position (global and / or local position), a y position (global and / or local position), a z position (global and / or local position), an orientation (e.g., roll, pitch, yaw), a type of object (e.g., classification), a velocity of the object, an acceleration of the object, an extent (size) of the object, etc. Characteristics associated with the environment can include, but are not limited to, a presence of another object in the environment, a state of another object in the environment, a time of day, a day of the week, a season, weather conditions, a darkness / lightness index, etc.
[0080] In general, the planning component 424 can determine a path for the vehicle 402 to follow to traverse an environment. For example, the planning component 424 can determine various paths and vehicle trajectories as well as various levels of detail. For example, the planning component 424 can determine a path to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For purposes of this description, the path may be a series of waypoints for traveling between the two locations. As non-limiting examples, the waypoints include roadways, intersections, Global Positioning System (GPS) coordinates, and the like. Additionally, the planning component 424 can generate instructions to guide the autonomous vehicle 402 along at least a portion of the path from the first location to the second location. In at least one example, the planning component 424 can determine how to guide the autonomous vehicle from a first waypoint in the series of waypoints to a second waypoint in the series of waypoints. In an example, the instructions may be a vehicle trajectory or may be a portion of a trajectory. In one example, the multiple trajectories may be generated substantially simultaneously (e.g., within technical tolerances) according to a receding horizon technique, where one of the multiple trajectories is selected for the vehicle 402 to navigate.
[0081] In various examples, the path may be based at least in part on the likelihood that the block object will remain in the path of the vehicle 402. In one example, the path may be associated with an action that the planning component 424 determines to take based on the block object. The action may include stopping and / or maintaining a stopped position before a position associated with the block object, slowing to a speed associated with the block object and maintaining a distance behind the block object (e.g., as described above with respect to FIG. 2), adjusting a position in a lane to go around the block object, and / or leaving a lane at least partially to go around the block object (e.g., changing lanes, operating in a bike lane, operating on the shoulder, etc.), etc.
[0082] In various examples, the planning component 424 may determine an action for the vehicle 402 to take based on the likelihood that the block object will remain in the vehicle's path. In an example, the prediction component 426 of the planning component 424 may determine the likelihood, such as by utilizing techniques described herein. In an example, the prediction component 426 may determine the likelihood that the block object will remain in the vehicle's path for a threshold period of time (e.g., 30 seconds, 1 minute, 2 minutes, etc.). In various examples, the prediction component 426 may be configured to determine the likelihood by utilizing machine learning techniques. In such an example, one or more machine learning models may be trained to determine the likelihood that the block object will remain in the vehicle's path based on one or more characteristics of the block object, such as object position, object size, level of stability, detected object action, speed, change in speed, etc.
[0083] In various examples, the likelihood may be determined based on one or more predicted object trajectories associated with the block object. In such examples, the prediction component 426 may be configured to determine a predicted object trajectory. In an example, the prediction component 426 may measure a trace of the object and generate a trajectory for the object based on the observation and predicted movement. In various examples, the predicted object trajectory may be determined based on the detected movement of the block object over time. In such examples, the prediction component 426 may observe the position of the block object over time to determine a predicted object trajectory as illustrated in FIG. 2.
[0084] In one example, the predicted object trajectory can be based on a top-down representation of the environment, such as by utilizing techniques described in U.S. Patent Application Nos. 16 / 151,607 and 16 / 504,147, the entire contents of which are incorporated herein by reference. In one example, the predicted object trajectory can be determined using probabilistic heat maps (e.g., discretized probability distributions), tree search methods, temporal logic formulas, and / or machine learning techniques for predicting object behavior, such as those described in U.S. Patent Application No. 2006 / 0139991, the entire contents of which are incorporated herein by reference. In various examples, the confidence can represent a level of certainty that the prediction component 426 can have in the accuracy of the predicted object trajectory.
[0085] In various examples, the planning component 424 may determine an action to take based at least in part on a cost associated therewith. In such examples, the action cost component 428 may be configured to determine a cost associated with each possible action. The possible actions may include actions that the vehicle may take within given constraints, such as not traveling closer than a minimum safe distance from an object, not violating a motion rule, etc. The costs may include one or more of a safety cost, a comfort cost, a progress cost, and / or a motion rule cost, as described above. In various examples, the planning component 424 may select the action with the lowest cost as the action to execute.
[0086] In at least one example, vehicle computing device 404 may include one or more system controllers 430, which may be configured to control steering, propulsion, braking, safety, emitter, communication, and other systems of vehicle 402. System controller 430 may communicate with and / or control corresponding systems of drive system 414 and / or other components of vehicle 402.
[0087] The memory 418 may further include one or more maps 432 that may be used by the vehicle 402 to navigate within the environment. For purposes of this description, a map may be any number of data structures modeled in two, three, or N dimensions that may provide information about the environment, such as, but not limited to, topology (such as intersections), streets, mountain ranges, roads, terrain, and the general environment. In one example, the map may include, but is not limited to, texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), etc.), intensity information (e.g., lidar information, radar information, etc.), spatial information (e.g., image data projected onto a mesh, individual "surfels" (e.g., polygons associated with individual colors and / or intensities), reflectivity information (e.g., specularity information, retroreflectivity information, BRDF information, BSSRDF information, etc.). In one example, the map may include a three-dimensional mesh of the environment. In one example, the vehicle 402 may be controlled based at least in part on the map 432. That is, the map 432 can be used in conjunction with the localization component 420, the perception component 422, and / or the planning component 424 to determine the position of the vehicle 402, detect objects in the environment, generate paths, and determine actions and / or trajectories to navigate within the environment.
[0088] In an example, one or more maps 432 may be stored on a remote computing device (such as in a map component 438 of a computing device 436) accessible via one or more networks 440. In an example, multiple maps 432 may be stored, for example, based on characteristics (e.g., type of entity, time of day, day of the week, season of the year, etc.). Storing multiple maps 432 may have similar memory requirements but may increase the speed at which data in the maps can be accessed.
[0089] As can be appreciated, the components discussed herein (e.g., localization component 420, perception component 422, planning component 424 including prediction component 426 and action cost component 428, one or more system controllers 430, one or more maps 432) are described as separated for illustrative purposes, however, the operations performed by the various components may be combined or performed in any other component.
[0090] In some examples, some or all aspects of the components described herein may include any models, techniques, and / or machine learning techniques. For example, in some examples, the components in memory 418 (and memory 434 described below) may be implemented as neural networks.
[0091] As described herein, an exemplary neural network is a biologically inspired technology that passes input data through successively connected layers to produce an output. Each layer in a neural network may also include another neural network, or may include any number of layers (whether convolutional or not). As may be understood in the context of the present disclosure, a neural network may utilize machine learning, which may refer to a broad class of technologies in which an output is generated based on learned parameters.
[0092] Although described in the context of neural networks, any type of machine learning may be used consistent with this disclosure.For example, machine learning techniques include regression techniques (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS)), instance-based techniques (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), Elastic Net, least-angle regression (LARS)), decision tree techniques (e.g., classification and regression tree (CART), iterative dichotomiser 3 (ID3), chi-squared automated interaction detection (CHAID), decision strains, conditional decision trees), Bayesian techniques (e.g., naive Bayes, Gaussian naive Bayes, multinomial naive Bayes, average one-dependence estimators), Bayesian confidence networks (BNNs), Bayesian networks), clustering techniques (e.g., k-means, k-medians, expectation maximization (EM), hierarchical clustering), association rule learning techniques (e.g., perceptrons, backpropagation, Hopfield networks, RBFNs (Radial Basis Function Network), deep learning techniques (e.g., deep Boltzmann machine (DBM), deep confidence network (DBN), convolutional neural network (CNN), stacked autoencoder), dimensionality reduction techniques (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), mixed discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble techniques (e.g., boosting, bootstrap aggregation (bagging), adaboost, hierarchical generalization (blending), gradient boosting machine (GBM), gradient boosted regression tree (GBRT), random forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc., but are not limited to these.Further examples of architectures include neural networks such as ResNet50, ResNet101, VGG, DenseNet, and PointNet.
[0093] In at least one example, the sensor system 406 may include lidar sensors, radar sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, time of flight, etc.), microphones, wheel encoders, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), and the like. The sensors 406 may include various examples of each of these or other types of sensors. For example, the lidar sensors may include individual lidar sensors located at the corners, front, rear, sides, and / or top of the vehicle 402. As another example, the camera sensors may include various cameras positioned at various locations about the exterior and / or interior of the vehicle 402. The sensors 406 may provide input to the vehicle computing device 404. Additionally or alternatively, the sensor system 406 may transmit sensor data over one or more networks 440 to one or more computing devices 436 at a particular frequency, after a predetermined period of time, in near real-time, etc.
[0094] The vehicle 402 may also include one or more emitters 408 for emitting light and / or sound. The emitters 408 include interior audio and visual emitters for communicating with passengers of the vehicle 402. By way of non-limiting example, the interior emitters may include speakers, lights, signs, display screens, touch screens, haptic emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, head rest positioners, etc.), and the like. The emitters 408 may include exterior emitters. By way of non-limiting example, the exterior emitters may include lights for signaling a direction of movement or other indicators of the vehicle's actions (e.g., indicator lights, signs, light arrays, etc.), as well as one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) for acoustically communicating with pedestrians or one or more other nearby vehicles, including acoustic beam steering technology.
[0095] In various examples, the planning component 424 may be configured to emit one or more signals towards the detected block object via the emitter 408. In an example, the planning component 424 may emit the signal before taking action as determined above based on the likelihood that the block object will remain in the vehicle's path. As described above, the signal may include an audio signal and / or a visual signal. In an example, the signal may signal an intent to warn the block object of the vehicle 104 in the environment. In an example, the signal may provide information to the block object, such as a direction in which the block object may move to deviate from the vehicle's path.
[0096] In various examples, in response to emitting the signal, the perception component 422 may receive sensor data corresponding to a response and / or indication of the block object's intent, whether or not to change position. In an example, the indication may be received via a microphone, speaker, or other sensor system 406. In at least one example, the indication may be received via an audio input via a microphone. In such an example, the perception component 422 may be configured to process the audio input to determine an intent and / or indication associated therewith, such as via natural language processing. In an example, the indication may be received via a camera sensor, such as an image. In such an example, the perception component 422 may be configured to determine the intent and / or indication by processing the image data. For example, the perception component 422 may indicate that the block object remains in a position and / or location and waves at the vehicle 402 in response to the emitted signal. The perception component 422 may determine that the indication includes an intent to remain in a position blocking the path of the vehicle 402.
[0097] The vehicle 402 may also include one or more communication connections 410 that enable communication between the vehicle 402 and one or more other local or remote computing devices. For example, the communication connections 410 may facilitate communication with other local computing devices on the vehicle 402 and / or the drive module 414. The communication connections 410 may also enable the vehicle to communicate with other nearby computing devices (e.g., computing device 436, other nearby vehicles, etc.) and / or one or more remote sensor systems 442 for receiving sensor data.
[0098] The communication connection 410 may include a physical and / or logical interface for connecting the vehicle computing device 404 to another computing device or network, such as the network 440. For example, the communication connection 410 may enable Wi-Fi-based communications over frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies such as Bluetooth, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 5G, etc.), or any suitable wired or wireless communication protocol that allows each computing device to interface with other computing devices.
[0099] In at least one example, the vehicle 402 may include one or more drive systems 414. In some examples, the vehicle 402 may have one drive system 414. In at least one example, if the vehicle 402 has multiple drive systems 414, the individual drive systems 414 may be located at opposite ends of the vehicle 402 (e.g., the front and the rear, etc.). In at least one example, the drive system 414 may include one or more sensor systems that detect conditions surrounding the drive system 414 and / or the vehicle 402. By way of non-limiting example, the sensor systems may include one or more wheel encoders (e.g., rotary encoders) for sensing the rotation of the wheels of the drive system, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) for measuring the orientation and acceleration of the drive module, cameras or other image sensors, ultrasonic sensors for acoustically detecting objects surrounding the drive module, lidar sensors, radar sensors, etc. Certain sensors, such as wheel encoders, may be unique to the drive system 414. In some cases, a sensor system on the drive system 414 may overlap or complement a corresponding system on the vehicle 402 (eg, sensor system 406).
[0100] The drive system 414 may include many of the vehicle systems, including a high voltage battery, a motor for propelling the vehicle, an inverter for converting direct current from the battery to alternating current for use by other vehicle systems, a steering system including a steering motor and a steering rack (which may be electric), a brake system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for distributing braking force to mitigate loss of traction and maintain control, an HVAC system, lighting (e.g., lighting such as head / tail lights for illuminating the exterior surroundings of the vehicle), and one or more other systems (e.g., cooling systems, safety systems, on-board charging systems, other electrical components such as DC / DC converters, high voltage junctions, high voltage cables, charging systems, charging ports, etc.). Additionally, the drive system 414 may include a drive module controller for controlling the operation of various vehicle systems, which may receive and preprocess data from the sensor systems. In an example, the drive module controller may include one or more processors and a memory communicatively coupled to the one or more processors. The memory may store one or more systems for performing various functions of the drive system 414. Additionally, drive system 414 may also include one or more communication connections that enable each drive system to communicate with one or more other local or remote computing devices.
[0101] In at least one example, the direct connection 412 may provide a physical interface for coupling one or more drive systems 414 with the body of the vehicle 402. For example, the direct connection 412 may enable the transfer of energy, fluid, air, data, etc. between the drive systems 414 and the vehicle. In one example, the direct connection 412 may further releasably secure the drive systems 414 to the body of the vehicle 402.
[0102] In at least one example, the positioning component 420, the perception component 422, the planning component 424, the one or more system controllers 430, the one or more maps 432, the prediction component 426, and the cost of action component 428 can process data (e.g., sensor data) as described above and transmit their respective outputs to the computing device 436 over one or more networks 440. In at least one example, the positioning component 420, the perception component 422, the planning component 424, the one or more system controllers 430, the one or more maps 432, the prediction component 426, and the cost of action component 428 can transmit their respective outputs to the computing device 436 in near real-time after a predetermined period of time and at a particular frequency.
[0103] In one example, the vehicle 402 can transmit sensor data to the computing device 436 over the network 440. In one example, the vehicle 402 can receive sensor data from the computing device 436 and / or the remote sensor system 442 over the network 440. The sensor data may include raw sensor data and / or processed sensor data and / or representations of the sensor data. In one example, the sensor data (raw or processed) may be transmitted and / or received as one or more log files.
[0104] The computing device 436 may include one or more processors 444 and memory 434 that stores the map component 438 as well as the sensor data processing component 446. In various examples, the sensor data processing component 446 may be configured to receive data from one or more remote sensors, such as the sensor system 406 and / or the remote sensor system 442. In an example, the sensor data processing component 446 may be configured to process the data and send the processed data to the planning component 424 to determine an action to be taken by the vehicle 402. In an example, the sensor data processing component 446 may be configured to process the data and send the processed sensor data to the vehicle computing device 404 for use by the action cost component 428 and / or the prediction component 426. In an example, the sensor data processing component 446 may be configured to send raw sensor data to the vehicle computing device 404.
[0105] The processor 416 of the vehicle 402 and the processor 444 of the computing device 436 may be any suitable processor capable of executing instructions to process data and perform operations as described herein. By way of non-limiting example, the processors 416 and 444 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or portion of a device that processes electronic data and converts the electronic data into registers and / or other electronic data that may be stored in memory. In certain examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices may also be considered processors so long as they are configured to execute encoded instructions.
[0106] The memories 418 and 434 are examples of non-transitory computer-readable media. The memories 418 and 434 can store an operating system and one or more software applications, instructions, programs, and / or data to execute the methods and functions attributed to the various systems described herein. In various implementations, the memories may be implemented using suitable memory technologies such as static RAM (SRAM), synchronous DRAM (SDRAM), non-volatile / flash type memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein may include many other logical, programmatic, and physical components, of which those shown in the accompanying drawings are merely examples relevant to the description herein.
[0107] In some examples, the memories 418 and 434 may include at least a working memory and a storage memory. For example, the working memory may be a limited capacity high-speed memory (e.g., cache memory) used to store data to be manipulated by the processors 416 and 444. In some examples, the memories 418 and 434 may include a storage memory, which may be a relatively large capacity slower memory used for long-term storage of data. In some examples, the processors 416 and 444 may not be able to directly manipulate data stored in the storage memory, as described herein, and the data may need to be loaded into the working memory to perform operations based on the data.
[0108] 4 is depicted as a distributed system, it should be noted that in alternative examples, components of the vehicle 402 may be associated with the computing device 436 and / or components of the computing device 436 may be associated with the vehicle 402. That is, the vehicle 402 may perform one or more functions associated with the computing device 436, and vice versa.
[0109] 5-7 illustrate exemplary processes according to embodiments of the present disclosure. These processes are illustrated as logical flow graphs, with each operation representing a sequence of operations that may be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to perform a process.
[0110] 5 shows an example process 500 for controlling a vehicle according to an action based at least in part on a cost associated with the action. Some or all of the process 500 may be performed by one or more components in FIG. 4 as described herein. For example, some or all of the process 500 may be performed by the vehicle computing device 404 and / or the computing device 436.
[0111] At operation 502, the process may include detecting an object in the environment that is blocking a path of the vehicle. In an example, the object may be detected based at least in part on sensor data associated with sensors of the vehicle. The sensors may include cameras, motion detectors, lidar, radar, time of flight, etc. In an example, the vehicle computing system may receive sensor data from one or more remote sensors, such as, for example, sensors onboard another autonomous vehicle and / or sensors onboard the environment.
[0112] In various examples, the vehicle computing system may determine that an object is blocking the path of the vehicle based on a determination that the object is impeding the forward movement of the vehicle. In such examples, forward movement may be impeded based on a determination that the vehicle must slow its forward speed and / or stop to prevent a collision with the object. The blocking object may be substantially stationary (e.g., a pedestrian standing and / or moving around in front of the vehicle, etc.) and / or dynamic (e.g., a bicyclist in the roadway in front of the vehicle moving at a speed slower than that associated with the vehicle's trajectory, a pedestrian walking in the roadway in front of the vehicle, such as a pedestrian navigating a construction zone, etc.). In various examples, the vehicle computing system may determine that an object is blocking the path of the vehicle based at least in part on a location associated with the object that is at least partially in the path of the vehicle and / or the drivable area. In various examples, the vehicle computing system may determine that an object is blocking the path of the vehicle based on a determination that the vehicle cannot go around the object while remaining in the lane. In such examples, the object may completely block a lane associated with travel of the vehicle.
[0113] At operation 504, the process may include determining an object trajectory (e.g., a predicted trajectory) associated with the object. The object trajectory may be determined based on sensor data. The object trajectory may represent a possible path that the detected object may travel through the environment. In various examples, the object trajectory may be determined utilizing a top-down representation of the environment, heat maps, temporal logic expressions, tree search methods, machine learning techniques, or other means of determining active predictions associated with objects in the environment, as described in U.S. patent applications incorporated herein by reference. In such examples, one or more of the sensor data, a representation of the sensor data (e.g., bounding boxes, ranges, etc.), a map, road network information, and a path associated with an action (all of which may include a sequence of data over a period of time) may be input into a machine learning model trained to output a predicted trajectory of the selected object. In various examples, an object trajectory, such as a trajectory associated with a dynamic block object, may be determined based on a change in the object's position over time. In such examples, the object trajectory may be based on the movement of the detected object over time.
[0114] At operation 506, the process may include determining a likelihood that the object will continue to block the path based at least in part on the object trajectory. In an example, the vehicle computing system may be configured to determine a likelihood that the object will continue to block the path for more than a threshold period (e.g., 45 seconds, 90 seconds, etc.). As described above, the likelihood may be determined based on a predicted object trajectory and / or a confidence level associated therewith. In various examples, the likelihood may be determined based on one or more factors, such as a location of the block object, one or more positions associated with the block object (e.g., known positions of tourist attention, photography, etc.), a classification of the block object, a size of the block object (e.g., width, height, number of pedestrians in a group, etc.), a level of stability (e.g., amount of movement, etc.), a speed of the block object, a change in speed of the block object (e.g., speed covariance), an action of the detected object (e.g., taking a photo, pointing at another object, etc.), a known time associated with an action of the detected object (e.g., a time associated with taking a photo at a landmark, etc.), etc. As described above, the likelihood may be determined utilizing machine learning techniques.
[0115] In operation 508, the process may include determining whether the likelihood exceeds a threshold (e.g., 45%, 57%, 62%, etc.). The threshold may include a threshold associated with a decision to stop and / or maintain a stopped position (e.g., for a substantially static object), or to slow to a speed substantially equal to the object's speed (e.g., for a dynamic object), to follow behind the dynamic object at a safe speed, or to determine a path that goes around a blocking object.
[0116] Based on a determination that the likelihood is below the threshold ("NO" at operation 508), the process may include controlling the vehicle to wait for the block object to move off the path at operation 510. In examples where the block object is a dynamic object, the vehicle may maintain substantially the same heading in the lane and slow down to substantially the same speed as the dynamic object. In various examples, the vehicle computing system may cause the vehicle to maintain a safe distance behind the vehicle. In such examples, the vehicle computing system may ensure that the vehicle is optimizing a safety protocol associated with the block object.
[0117] In examples where the block object is a substantially static object (e.g., maintains substantially the same position), the vehicle computing system may cause the vehicle to stop and / or maintain a stopped position in front of a position associated with the block object. In such examples, the vehicle computing system may cause the vehicle to wait for the block object to move out of the vehicle's path before continuing forward on the path.
[0118] Based on a determination that the likelihood is equal to or greater than a threshold value ("yes" at operation 508), the process may include controlling the vehicle around the object at operation 512. As discussed herein, the vehicle computing system may determine several possible actions the vehicle can take, such as adjusting a position in a lane around the block object, changing lanes, partially changing lanes, etc. In various examples, the vehicle computing system may determine a cost associated with each possible action (possibly even a cost associated with waiting for the block object to move out of the path) and control the vehicle around the object based on the cost determination (e.g., the lowest cost action).
[0119] 6 shows an example process for controlling a vehicle around a block object according to an action based at least in part on a cost associated with the action. Some or all of process 600 may be performed by one or more components in FIG. 4 as described herein. For example, some or all of process 600 may be performed by vehicle computing device 404.
[0120] In operation 602, the process may include determining that a likelihood that the object will continue to block a path associated with the vehicle exceeds a threshold likelihood (e.g., operation 512 of FIG. 5). In one example, the determination of continued blocking may be based on a likelihood that the blocking object will remain (at least partially) on the path associated with the vehicle for a threshold time (e.g., 15 seconds, 30 seconds, 1 minute, etc.).
[0121] In various examples, the likelihood may be determined based on one or more factors, such as the location of the block object, one or more positions associated with the block object (e.g., known positions of tourist attention, photography, etc.), the classification of the block object, the size of the block object (e.g., width, height, number of pedestrians in a group, etc.), the level of stability (e.g., amount of movement, etc.), the speed of the block object, the change in the speed of the block object (e.g., speed covariance), the action of the detected object (e.g., taking a photo, pointing at another object, etc.), the known time associated with the action of the detected object (e.g., time associated with taking a photo at a landmark, etc.), etc. As discussed above, the likelihood may be determined based on a predicted object trajectory and / or a confidence associated therewith. In various examples, the likelihood may be determined utilizing machine learning techniques or other methods to determine the likelihood that the object will continue to block the path of the vehicle.
[0122] At operation 604, the process may include determining a lateral distance between the vehicle at a position in the lane directly beside the object (e.g., a block object) and a position associated with the object. In some examples, the lateral distance from the object may include a maximum lateral distance. In such examples, the vehicle may operate at a position in the lane furthest from the block object. In other examples, the distance may be less than the maximum lateral distance.
[0123] In various examples, the vehicle computing system may determine the distance between the block object and the opposing edge of the lane (e.g., the distance from the block object to the lane marker). In such examples, the vehicle computing system may subtract the vehicle width and / or a safety buffer (e.g., 3 inches, 6 inches, etc.) from the distance between the block object and the lane marker to determine the lateral distance between the vehicle at a position directly beside the block object and a position associated with the block object.
[0124] At operation 606, the process may include determining whether the lateral distance within the lane is greater than or equal to a threshold distance. In various examples, the threshold distance may represent a minimum safe distance that the vehicle should maintain from the object. In one example, the threshold distance may be based on a classification associated with the block object. In such an example, the vehicle computing system may determine a classification associated with the block object and determine a threshold distance associated therewith.
[0125] Based on a determination that the lateral distance is greater than or equal to the threshold distance ("YES" at operation 606), the process at operation 608 may include determining a vehicle speed associated with the distance. As described above, the vehicle speed may be determined based on a cost optimization associated with the lateral distance (e.g., one or more lateral distances from a position associated with a block object). In various examples, the vehicle speed at the lateral distance may include the speed associated with the lowest action cost.
[0126] In various examples, the vehicle speed and lateral distance may be related such that the vehicle speed may increase as the lateral distance from the block object increases. In an example, the vehicle speed may include a predetermined vehicle speed associated with passing the block object in the lane. In an example, the predetermined vehicle speed may include a maximum safe speed for operating in proximity to the object. In various examples, the vehicle speed may be based on a classification associated with the block object. In an example, the first classification may include a first set of predetermined vehicle speeds associated with the passing, and the second classification may include a second set of predetermined vehicle speeds associated with the passing. In such examples, the first set of predetermined vehicle speeds and the second set of predetermined vehicle speeds may include at least some of the same vehicle speeds and / or different vehicle speeds.
[0127] At operation 610, the process may include determining whether an action cost associated with going around a blocking object in a lane is less than or equal to a threshold cost. The threshold cost may include a predetermined maximum cost associated with the action. The threshold cost may maximize safety and performance of the vehicle operating in the environment. In various examples, the threshold cost may be associated with a constraint on the vehicle action. In such examples, the constraint may prevent the vehicle computing system from selecting one or more actions based on high costs (e.g., safety, comfort, progress, motion rule costs, etc.). For example, the threshold cost may be associated with a maximum comfort cost to prevent the vehicle computing system from performing an action that is overly unpleasant for a passenger in the vehicle, a person associated with the object, etc.
[0128] Based on a determination that an action cost associated with going around a block object in the lane is less than or equal to a threshold cost ("YES" at operation 610), the process at operation 612 may include controlling the vehicle according to a trajectory associated with the lateral distance from the object and the vehicle speed. As described above, the action cost may be based on a safety cost, a comfort cost, a progression cost, and / or a motion rule cost associated with the action. In various examples, the threshold cost may include a predetermined maximum cost associated with performing the action. In such examples, the threshold cost may ensure that the action performed by the vehicle optimizes the safety, comfort, and progression of the vehicle while operating in the environment.
[0129] Based on a determination that the action cost associated with going around the block object in the lane exceeds the threshold cost ("NO" at operation 610), the process may include maintaining a minimum distance from the location associated with the block object at operation 614. In such an example, the vehicle computing system may determine to yield to the block object and wait for the block object to move out of the vehicle's path. In various examples, the vehicle computing system may stop the vehicle before the location associated with the block object. In such an example, the action may represent the most conservative action to ensure the safety of the block object, the vehicle, the vehicle's occupants, and other objects in the environment. In various examples, the vehicle computing system may slow the forward speed of the vehicle to substantially match the speed of the object, such as a dynamic object traveling in substantially the same direction as the vehicle. In such an example, the vehicle computing system may cause the vehicle to travel behind the block object by a distance (e.g., a minimum safe distance).
[0130] Based on a determination that the lateral distance is below the threshold distance ("NO" at operation 606), the process may include determining whether a threshold area of the adjacent lane is occupied at operation 616. In an example, the threshold area may include at least the width of a vehicle in the adjacent lane and / or a safety buffer (e.g., 4 inches, 8 inches, 16 inches, etc.) from other objects. In an example, the threshold area may be associated with an area required to at least partially transition into the adjacent lane without encroaching within a threshold distance (e.g., a minimum safe distance) in front of, behind, or laterally of another object 108.
[0131] In examples where the adjacent lanes include traffic moving in opposite directions, the threshold area may include an area encompassing the distance the vehicle travels around the block object and the distance another object covers (e.g., at a speed limit associated with the environment) in the time required for the vehicle to go around the block object. In one example, based on a determination that another object occupies the threshold area, the vehicle computing system may cause the vehicle to yield to the other object before proceeding around the block object.
[0132] Based on a determination that the threshold area of the adjacent lane is occupied (“Yes” at operation 616), the process may include stopping the vehicle before a location associated with the block object, as described at operation 614.
[0133] Based on a determination that the threshold area of the adjacent lane is unoccupied ("no" at operation 616), the process may include determining whether an action cost associated with traveling in the adjacent lane is greater than or equal to a threshold cost. As described above, the action cost may be based on a safety cost, a comfort cost, a progression cost, and / or an operation rule cost associated with the action. In various examples, the threshold cost may include a predetermined maximum cost associated with performing the action. In such examples, the threshold cost may ensure that the action performed by the vehicle optimizes the safety, comfort, and progression of the vehicle while operating within established rules associated with the environment.
[0134] Based on a determination that the action cost is greater than or equal to the threshold cost ("Yes" at operation 618), the process may include stopping the vehicle in front of a location associated with the block object, such as that described at operation 614.
[0135] Based on a determination that the action cost is below the threshold cost ("NO" at OPERATION 620), the process may include having the vehicle enter an adjacent lane to detour around the block object at OPERATION 620. In various examples, the vehicle computing device may determine one or more vehicle trajectories (e.g., modified vehicle trajectories) for the vehicle to follow to detour around the block object. In an example, the modified vehicle trajectory may be associated with a second path and / or a second drivable area around the block object. In various examples, the vehicle computing system may send a signal to a drive system of the vehicle to drive the vehicle according to the modified trajectory, such as along the second path and / or within the drivable area.
[0136] 7 illustrates an example process for controlling a vehicle around a block object and yielding to a second object detected while navigating around the block object. Some or all of process 700 may be performed by one or more components in FIG. 4 as described herein. For example, some or all of process 700 may be performed by vehicle computing device 404.
[0137] At operation 702, the process may include controlling the vehicle around a first object in the environment according to a first trajectory. In various examples, the first object may include a block object. In one example, the vehicle may be controlled as described above with respect to operation 512 of FIG. 5 and / or operations 612 and / or 620 of FIG. 6.
[0138] At operation 704, the process may include detecting a second object in the environment. In an example, the second object may be detected based at least in part on sensor data associated with sensors of the vehicle. The sensors may include cameras, motion detectors, lidar, radar, time of flight, etc. In an example, the vehicle computing system may receive sensor data from one or more remote sensors, such as, for example, sensors on board another autonomous vehicle and / or sensors on board the environment.
[0139] At operation 706, the process may include determining a second trajectory associated with the second object. The second object trajectory may be determined based on the sensor data. The second object trajectory may represent a possible path that the detected object may travel through the environment. In various examples, the second trajectory may be determined based on a change in the position of the second object over time. In such examples, the second object trajectory may be based on the movement of the detected second object over time. In various examples, the second object trajectory may be determined utilizing a top-down representation of the environment, a heat map, a temporal logic formula, a tree search method, machine learning techniques, or other means of determining an active prediction associated with an object in the environment, as described in U.S. patent applications incorporated herein by reference. In such examples, one or more of the sensor data, a representation of the sensor data (e.g., a bounding box, a range, etc.), a map, road network information, and a path associated with an action (all of which may include a sequence of data over a period of time) may be input into a machine learning model trained to output a predicted trajectory of the second object.
[0140] In operation 708, the process may include determining whether there is an intersection between the first trajectory (associated with the vehicle) and the second trajectory (associated with the second object). In one example, the intersection may represent a future state of the vehicle and the second object whose associated location is within a threshold distance of each other. In one example, the intersection may represent a future state of the vehicle and the second object occupying the same space at the same time while traveling on their respective trajectories.
[0141] Based on a determination that the second trajectory and the first trajectory do not merge ("NO" at operation 708), the process may include controlling the vehicle according to the first trajectory at operation 710. In an example, the vehicle may go around the block object and continue along the path to the destination without yielding to the second object.
[0142] Based on a determination that the second trajectory and the first trajectory merge ("YES" at operation 708), the process may include controlling the vehicle to yield to the second object at operation 712. In various examples, yielding to the second object may include slowing a speed associated with the first trajectory to allow the second object to cross the path of the vehicle. In an example, yielding to the second object may include stopping the vehicle before a location associated with the second object and / or one or more locations associated with the second trajectory.
[0143] (Example clauses) A. A vehicle having a sensor, one or more processors, and a memory storing processor-executable instructions, which when executed by the one or more processors, cause the vehicle to receive sensor data of an environment from the sensor; identify an object at a location within the environment based at least in part on the sensor data, where the object is at least partially blocking a first vehicle trajectory associated with the vehicle; determine a predicted object trajectory associated with the object; determine that the predicted object trajectory indicates a continued blocking of the first vehicle trajectory; determine a confidence level associated with the predicted object trajectory; and determine a second vehicle trajectory based at least in part on the predicted object trajectory and the confidence level, where the second vehicle trajectory is associated with the vehicle navigating around the object; and control the vehicle in accordance with the second vehicle trajectory.
[0144] B. The vehicle described in paragraph A, wherein the processor executable instructions further cause the vehicle to determine a first distance between a lane marker representing an edge of a lane and the location associated with the object, determine a second distance that includes the first distance minus a width of the vehicle, and determine a vehicle speed associated with the distance, and the second vehicle trajectory is associated with navigating around the object at the distance and at the vehicle speed.
[0145] C. The vehicle of paragraph A or paragraph B, wherein the processor-executable instructions further cause the vehicle to determine a classification associated with the object and determine a threshold distance between the vehicle and a location associated with the object based at least in part on the classification, and the second vehicle trajectory is based at least in part on the threshold distance.
[0146] D. The vehicle of any one of paragraphs A through C, wherein the processor-executable instructions further cause the vehicle to emit at least one of an audio signal or a visual signal in a direction associated with the object and determine that movement of the object follows the predicted object trajectory, and determining the second vehicle trajectory is based at least in part on determining that movement of the object follows the predicted object trajectory.
[0147] E. The vehicle of any one of paragraphs A to D, wherein the predicted object trajectory is based at least in part on at least one of a machine learning algorithm, a top-down representation of the environment, a discretized probability distribution, a temporal logic formula, or a tree search method.
[0148] F. A computer-implemented method comprising: identifying an object at a location in an environment based at least in part on sensor data; determining that the location is associated with the vehicle trajectory based at least in part on a determination that a vehicle traveling on the vehicle trajectory at least partially passes through the location; determining that the object at least partially blocks the vehicle trajectory based at least in part on the location being associated with the vehicle trajectory; determining a likelihood that the object will continue to at least partially block the vehicle trajectory; and controlling the vehicle based at least in part on the likelihood that the object will continue to at least partially block the vehicle trajectory.
[0149] G. The computer-implemented method of paragraph F, wherein the vehicle trajectory is a first vehicle trajectory, and determining that the object is at least partially blocking the vehicle trajectory is based at least in part on determining that the object is impeding progress of the vehicle along the first vehicle trajectory, the method comprising determining a threshold distance to navigate around the object and determining a second vehicle trajectory based at least in part on the threshold distance, and controlling the vehicle includes causing the vehicle to traverse the environment based at least in part on the second vehicle trajectory.
[0150] H. The computer-implemented method of paragraph F or G, wherein determining the likelihood the object will continue to at least partially block the vehicle trajectory comprises determining an object trajectory and determining, based at least in part on the object trajectory, that the object will substantially remain at the location associated with the vehicle trajectory and continue to impede progress of the vehicle traveling on the vehicle trajectory for at least a threshold time.
[0151] I. The computer-implemented method of any of paragraphs F-H, wherein controlling the vehicle comprises causing the vehicle to bypass the object in accordance with the second vehicle trajectory, the method further comprising identifying a second object based at least in part on the sensor data, determining a second object trajectory associated with the second object, determining an intersection between the second object trajectory and the second vehicle trajectory, and controlling the vehicle based at least in part on the intersection.
[0152] J. The computer-implemented method of any of paragraphs F through I, wherein the location of the object is a first location of the object, the method further comprising determining a second location of the object; determining that the second location of the object does not impede progress of the vehicle traveling on the vehicle trajectory; determining that the object does not at least partially block the vehicle trajectory based at least in part on the second location of the object not impeding progress of the vehicle; and controlling the vehicle according to the vehicle trajectory.
[0153] K. The computer-implemented method of any of paragraphs F through J, wherein the likelihood that the object will continue to block the vehicle trajectory is based at least in part on at least one of a classification of the object, a position of the object, a location of the object within the environment, a size of the object, a level of stability associated with the object, a velocity of the object, or a change in velocity of the object.
[0154] L. The computer-implemented method of any of paragraphs F through K, further comprising at least one of: emitting at least one of an audio signal or a visual signal in a direction based at least in part on a location of the object; and controlling the vehicle around the object's location based at least in part on a determination that the object's location remains substantially the same; or controlling the vehicle according to the vehicle trajectory based at least in part on determining a change in the object's location to a second location that does not at least partially block the vehicle trajectory.
[0155] M. The method of any one of paragraphs F-L, comprising receiving at least one of an audio signal via a microphone or a visual signal via a camera, and determining that the at least one of the audio signal or the visual signal comprises an indication that the object is not attempting to move from the vehicle trajectory, and wherein a likelihood that the object will continue to at least partially block the vehicle trajectory is based at least in part on the indication.
[0156] N. The method of any one of paragraphs F-M, wherein controlling the vehicle comprises determining an action for the vehicle to take, the action comprising at least one of maintaining a stationary position in the vehicle trajectory or determining a second vehicle trajectory around the object, and the action is determined at least in part based on at least one of a safety cost associated with the action that is based at least in part on a relative state between the object and the vehicle, a comfort cost associated with the action that is based at least in part on the relative state between the object and the vehicle, a travel cost associated with the action that is based at least in part on a vehicle delay associated with the action, a travel cost associated with the action, or a behavior rule cost associated with the action that is based at least in part on one or more regulations associated with the environment.
[0157] O. A system or device comprising a processor and a non-transitory computer-readable medium storing instructions that, when executed, cause the processor to perform the computer-implemented method described in any one of paragraphs F through N.
[0158] P. A system or apparatus comprising a processing means and a storage means coupled to the processing means, the storage means causing one or more devices to execute the computer-implemented method described in any one of paragraphs F to N.
[0159] Q. A non-transitory computer-readable medium storing instructions that, when executed, cause one or more processors to: identifying an object at a location within the environment based at least in part on the sensor data; determining that the location of the object is interfering with travel of the vehicle on a vehicle trajectory of the vehicle; determining that the object is at least partially blocking the vehicle trajectory based at least in part on the location of the object interfering with travel of the vehicle on the vehicle trajectory; determining a likelihood that the object will continue to at least partially block the vehicle trajectory; and controlling the vehicle based at least in part on the likelihood that the object will continue to at least partially block the vehicle trajectory. A non-transitory computer-readable medium for causing a program to be executed.
[0160] R. The non-transitory computer-readable medium of paragraph Q, wherein the instructions further cause the processor to determine a first distance between a lane marker representing an edge of a lane and a location of the object, determine a second distance that includes the first distance minus a width of the vehicle, and determine a second vehicle trajectory based at least in part on the second distance, and controlling the vehicle causes the vehicle to travel according to the second vehicle trajectory.
[0161] S. The non-transitory computer-readable medium of any one of paragraphs Q through R, wherein the instructions cause the processor to determine a vehicle speed associated with the distance, and the second vehicle trajectory comprises the vehicle speed.
[0162] T. The non-transitory computer-readable medium of any of paragraphs Q through S, wherein controlling the vehicle comprises determining a first trajectory for the vehicle to take to navigate around the object; determining a second trajectory for the vehicle to take to navigate around the object; determining a first cost associated with the first trajectory and a second cost associated with the second trajectory, the first cost and the second cost comprising at least one of a safety cost, a comfort cost, a progress cost, or a behavior rule cost associated with a respective trajectory; determining that the first cost associated with the first trajectory is lower than the second cost associated with the second trajectory; and causing the vehicle to navigate around the object according to the first trajectory.
[0163] U. The non-transitory computer-readable medium of any of paragraphs Q through T, wherein the instructions further cause the processor to receive an indication that the object does not attempt to move from the vehicle trajectory, and wherein a likelihood that the object will continue to at least partially block the vehicle trajectory is based at least in part on the indication.
[0164] V. The non-transitory computer-readable medium of any of paragraphs Q-U, wherein the likelihood that the object will continue to at least partially block the vehicle trajectory is based at least in part on at least one of a classification of the object, a position of the object, a location of the object, a size of the object, a level of stability associated with the object, a velocity of the object, or a change in velocity of the object.
[0165] Although the exemplary clauses as described above are described with respect to one specific implementation, it should be understood that in the context of this application, the contents of the exemplary clauses may also be implemented via a method, a device, a system, a computer-readable medium, and / or another implementation. Furthermore, any of the exemplary clauses A-V may be implemented alone or in combination with one or more other exemplary clauses A-V.
[0166] (Conclusion) While one or more examples of the technology described herein have been described, various modifications, additions, permutations, and equivalents of those examples are included within the scope of the technology described herein.
[0167] In the illustrative description, reference is made to the accompanying drawings, which form a part of this application, showing specific examples of the claimed subject matter. It is understood that other examples can be used and that modifications or variations, such as structural changes, can be made. Such examples, modifications or variations do not necessarily depart from the scope of the intended inventive subject matter. While steps herein are provided in a particular order, in some cases the order may be changed such that certain inputs may be provided at different times or in a different order without changing the functionality of the systems and methods described. The disclosed procedures may also be performed in different orders. Furthermore, the various computations herein need not be performed in the order disclosed, and other examples using alternative orders of computations may be readily implemented. In addition to being reordered, computations may also be decomposed into partial computations with the same results.
Claims
1. A sensor, one or more processors; A memory for storing processor-executable instructions; A vehicle equipped with The processor-executable instructions, when executed by the one or more processors, cause the vehicle to: receiving environmental sensor data from the sensor; identifying an object at a location within the environment based at least in part on the sensor data, the object at least partially blocking a first vehicle trajectory associated with the vehicle; and determining a predicted object trajectory associated with the object; determining that the predicted object trajectory indicates a continuous block of the first vehicle trajectory; determining a confidence level associated with the predicted object trajectory; determining a first distance between a lane marker representing an edge of a lane and the location associated with the object; determining a second distance that includes the first distance minus a width of the vehicle; determining a vehicle speed associated with the second distance; determining a second vehicle trajectory based at least in part on the predicted object trajectory and the confidence level, the second vehicle trajectory being associated with the vehicle navigating around the object at the second distance and at the vehicle speed; controlling the vehicle according to the second vehicle trajectory; A vehicle that performs the above.
2. The vehicle of claim 1, wherein the processor executable instructions further cause the vehicle to determine that the predicted object trajectory indicates a continuous block of the first vehicle trajectory for at least a predetermined threshold time.
3. The processor executable instructions include: determining a classification associated with the object; determining a threshold distance between the vehicle and a location associated with the object based at least in part on the classification; Then, The vehicle of claim 1 or 2, wherein the second vehicle trajectory is based at least in part on the threshold distance.
4. The processor executable instructions include: emitting at least one of an audio signal or a visual signal in a direction associated with the object; determining whether the object movement follows the predicted object trajectory; Then, The vehicle of claim 1 , wherein determining the second vehicle trajectory is based at least in part on determining that movement of the object follows the predicted object trajectory.
5. The predicted object trajectory may be: Machine learning algorithms, a top-down representation of said environment; Discretized Probability Distributions, a temporal logic expression, or Tree Search Method 5. The vehicle of claim 1 , based at least in part on at least one of:
6. A method performed by a vehicle computing system, comprising: identifying an object at a location within the environment based at least in part on the sensor data; determining that the location is associated with the vehicle track based at least in part on a determination that a vehicle traveling on the vehicle track at least partially passes through the location; determining that the object at least partially blocks the vehicle trajectory based at least in part on the location being associated with the vehicle trajectory; determining a likelihood that the object will continue to at least partially block the vehicle trajectory; controlling the vehicle based at least in part on a likelihood that the object will continue to at least partially block the vehicle trajectory, where controlling the vehicle comprises determining an action to be taken by the vehicle, the action comprising: maintaining a stationary position on the vehicle track; or determining a second vehicle trajectory around the object; At least one of The action is: a safety cost associated with the action that is based at least in part on a relative state between the object and the vehicle; a comfort cost associated with the action, the comfort cost being based at least in part on the relative state between the object and the vehicle; a travel cost associated with the action that is based at least in part on a vehicle delay associated with the action; or a behavior rule cost associated with the action that is based at least in part on one or more regulations associated with the environment; and A method for providing the above.
7. The vehicle trajectory is a first vehicle trajectory, and the object at least partially blocking the vehicle trajectory is based at least in part on determining that the object is impeding progress of the vehicle along the first vehicle trajectory, and the method further comprises: determining a threshold distance for navigating around the object; determining a second vehicle trajectory based at least in part on the threshold distance; Equipped with The method of claim 6 , wherein controlling the vehicle causes the vehicle to traverse the environment based at least in part on the second vehicle trajectory.
8. Determining the likelihood that the object will continue to at least partially block the vehicle trajectory includes: Determining an object trajectory; determining, based at least in part on the object trajectory, that the object remains substantially at the location associated with the vehicle trajectory and continues to impede progress of the vehicle traveling along the vehicle trajectory for at least a threshold time; The method of claim 6 or 7, comprising:
9. Controlling the vehicle comprises causing the vehicle to bypass the object according to a second vehicle trajectory, the method comprising: identifying a second object based at least in part on the sensor data; and determining a second object trajectory associated with the second object; determining an intersection between the second object trajectory and the second vehicle trajectory; controlling the vehicle based at least in part on the intersection; The method of claim 6 , further comprising:
10. The location of the object is a first location of the object, and the method further comprises: determining a second location of the object; determining that a second location of the object does not impede progress of the vehicle along the vehicle path; determining that the object is not at least partially blocking the vehicle trajectory based at least in part on the second location of the object not impeding progress of the vehicle; and controlling the vehicle according to the vehicle trajectory; The method of claim 6 , further comprising:
11. The likelihood that the object will continue to block the vehicle trajectory is A classification of the object; the position of said object, the location of the object within the environment; the size of the object, a level of stability associated with said object; the speed of said object, or The change in speed of the object 11. The method of claim 6, wherein the method is based at least in part on at least one of the following:
12. Receiving at least one of an audio signal via a microphone or a visual signal via a camera; determining that at least one of the audio signal or the visual signal comprises an indication that the object is not about to move from the vehicle trajectory; Equipped with The method of claim 6 , wherein a likelihood that the object will continue to at least partially block the vehicle trajectory is based at least in part on the indication.
13. emitting at least one of an audio signal or a visual signal in a direction based at least in part on a location of the object; controlling the vehicle around the location based at least in part on a determination that the location of the object remains substantially the same; or controlling the vehicle according to the vehicle trajectory based at least in part on determining a change in location of the object to a second location that does not at least partially block the vehicle trajectory; and performing at least one of The method of any one of claims 6 to 12, further comprising:
14. A non-transitory computer readable medium storing instructions that, when executed, cause one or more processors to perform the method of any one of claims 6 to 13.
Citation Information
Patent Citations
Course setting device and course setting method
JP2018176879A
Drive envelope determination
US10614717B2
Probabilistic heat maps for behavior prediction
US11055624B1
Trajectory prediction on top-down scenes
US11169531B2
Method for operating a driver assistance system, and driver assistance system
US20150210311A1
