Route planning using delta cost volume generated from movement restrictions and observed driving behavior.

Abstract

In one embodiment, the method includes determining an initial cost volume associated with a plurality of potential trajectories of a vehicle within an environment based on a set of travel constraints for the vehicle; generating a delta cost volume using the initial cost volume and environmental data associated with the environment, the delta cost volume being generated by determining an adjustment to the initial cost volume that incorporates observed driving behavior; and scoring one of the plurality of potential trajectories for the vehicle based on the initial cost volume and the delta cost volume.

Description

Background Art

[0001] Autonomous vehicles rely on effective and efficient route planning to travel in an urban environment along the safest, most convenient, and most economically beneficial vehicle trajectories. A vehicle trajectory is a series of states through which a vehicle, parameterized by time and speed, has passed. Trajectory planning or trajectory generation is a real-time planning of the movement of a vehicle from one achievable state to the next, which satisfies the kinematic limits of the vehicle based on its dynamics and constrained by the navigation mode. Since a route planning system needs to identify all agents, whether stationary or moving, and ensure that potential vehicle trajectories bypass these agents, finding potential vehicle trajectories is complex. Finding potential vehicle trajectories may be based on trajectory optimization, which is a process of determining a trajectory that satisfies a set of constraints while minimizing (or maximizing) some measure such as cost. The cost of a potential vehicle trajectory may be based on many factors including safety, comfort, efficiency, position, speed, acceleration, feasibility, legality, etc.

Brief Description of the Drawings

[0002] [Figure 1A] FIG. 1A shows an exemplary trajectory determination for a vehicle following a cargo truck. [Figure 1B] FIG. 1B shows an exemplary trajectory determination for a vehicle traveling near a bicycle lane. [Figure 1C] FIG. 1C shows an exemplary trajectory determination for a vehicle turning at an intersection. [Figure 1D] FIG. 1D shows an exemplary trajectory determination for a vehicle approaching a stop sign. [Figure 2] FIG. 2 shows an exemplary cost evaluation. [Figure 3] FIG. 3 shows an exemplary difference between an initial cost volume and a differential cost volume. [Figure 4]Figure 4 shows an example architecture for generating differential cost volumes. [Figure 5A] Figure 5A shows an illustrative cost comparison between two trajectories using the initial cost volume of the scenario in Figure 1A. [Figure 5B] Figure 5B shows an illustrative cost comparison between two trajectories using the initial cost volume of the scenario in Figure 1B. [Figure 5C] Figure 5C shows an illustrative cost comparison between two trajectories using the initial cost volume of the scenario in Figure 1C. [Figure 5D] Figure 5D shows an illustrative cost comparison between two trajectories using the initial cost volume of the scenario in Figure 1D. [Figure 6A] Figure 6A shows an illustrative cost comparison between the two trajectories using the differential cost volume for the scenario in Figure 1A. [Figure 6B] Figure 6B shows an illustrative cost comparison between the two trajectories using the differential cost volume for the scenario in Figure 1B. [Figure 6C] Figure 6C shows an illustrative cost comparison between the two trajectories using the differential cost volume for the scenario in Figure 1C. [Figure 6D] Figure 6D shows an illustrative cost comparison between the two trajectories using the differential cost volume for the scenario in Figure 1D. [Figure 7] Figure 7 shows an example method for determining the planned trajectory for a vehicle. [Figure 8] Figure 8 shows a block diagram illustrating an example of a transportation management environment that matches passengers with autonomous vehicles. [Figure 9] Figure 9 shows an example of a processing pipeline for autonomous driving. [Figure 10] Figure 10 shows an example of a computing system. [Modes for carrying out the invention]

[0003] The following description will explain various embodiments. Certain configurations and details are described to provide a full understanding of the embodiments for illustrative purposes. However, it will be apparent to those skilled in the art that embodiments may be carried out without certain details. Furthermore, well-known features may be omitted or simplified so as not to obscure the embodiments described. In addition, the embodiments disclosed herein are merely examples, and the scope of this disclosure is not limited thereto. A particular embodiment may include all or some of the components, elements, features, functions, operations, or steps of the embodiments disclosed herein, or it may not include any of them. Embodiments according to the present invention are specifically disclosed in appendix claims relating to methods, storage media, systems, and computer program products, in which case any feature mentioned in one claim category, such as a method, may similarly be claimed in another claim category, such as a system. Dependencies or backreferences in the appendix claims are selected solely for public reasons. However, any subject matter resulting from intentional backreferences to any prior claims (particularly in multiple dependencies) may also be claimed, insofar as any combination of claims and their features is disclosed and is claimed independently of the dependencies selected in the appendix claims. The claimable subject matter includes not only the combination of features described in the appendix claims but also any other combination of features in the claims, in which case each feature referred to in the claims may be combined with any other feature or other combination of features in the claims. Furthermore, any embodiments and features described or depicted herein may also be claimed in separate claims and / or in any combination with any embodiments or features described or depicted herein or with any features of the appendix claims.

[0004] Vehicle trajectory planning can be a foundation for autonomous driving. For example, and without limitation, a computing system may often need to determine the best forward trajectory for an autonomous vehicle in the next few seconds. For instance, the computing system may determine whether the vehicle should accelerate, go straight, or make a slight turn. Figures 1A to 1D illustrate different scenarios in which a planned trajectory needs to be determined for an autonomous vehicle. Figure 1A shows an example of trajectory determination for a vehicle following a cargo truck. Vehicle 100 may be traveling on a road 102 with two lanes, namely right lane 104 and left lane 106. Vehicle 100 may be following a cargo truck 108 in left lane 106. The computing system may need to determine whether vehicle 100 should continue following the cargo truck 108 or change to right lane 104, which is represented by two potential trajectories, namely trajectories 110 and 112, respectively. Figure 1B shows an example of trajectory determination for a vehicle traveling near a bicycle lane. Vehicle 100 may be traveling in lane 114 adjacent to bicycle lane 116. A cyclist 118 may be present in bicycle lane 116. Vehicle 100 may be approaching cyclist 118 from behind. Therefore, the calculation system may need to determine whether vehicle 100 should continue straight or move slightly to the left to avoid a collision with the cyclist and then return to the center of the lane, which is represented by two potential trajectories, namely trajectories 120 and 122, respectively. Figure 1C shows an example of trajectory determination for a vehicle turning at an intersection. Vehicle 100 may need to turn left from lane 124, while another vehicle 126 is also nearby, turning left from lane 128. A potential trajectory for vehicle 126 may be trajectory 130. The calculation system may need to determine whether to turn left using trajectory 132 which terminates in the center of lane 134, or using trajectory 136 which terminates near the lane boundary line 138. Figure 1D shows an example of trajectory determination for a vehicle approaching a stop sign.The calculation system may need to determine whether vehicle 100 should approach stop sign 140 at a steady speed, such as 25 mph, or at a gradually decreasing speed, such as 20 mph, then 10 mph, then 5 mph. These two options can be represented by trajectories 142 and 144, respectively. In the scenarios shown in Figures 1A to 1D, the calculation system can determine the planned trajectory by optimizing different sets of costs evaluated by different factors. For example, the cost of vehicle 100 colliding with another agent without limitation can be very high. Another example is the cost of crossing a lane boundary without limitation can be very high. Yet another example is the cost of vehicle 100 traveling with a large lateral acceleration or large lateral jerk without limitation, because this would result in discomfort for the occupants inside vehicle 100. Therefore, it is important to accurately determine the cost of each potential trajectory and select a planned trajectory based on that cost.

[0005] The determination of the cost of a potential trajectory may be based on a set of hand-engineered costs determined based on the movement limitations of vehicle 100. For example, without limitation, one of the costs may be based on penalized jerks. Penalized jerks may be the sum of all jerks along a trajectory that are penalized by some weight. Thus, the trajectory that minimizes all these different costs can be determined as the planned trajectory that vehicle 100 should adopt when moving forward. A cost volume can be generated based on the hand-engineered costs. The cost volume obtained based on space and time can provide cost information associated with each location associated with the potential trajectory at different points in time. Instead of using hand-engineered costs, the cost volume can also be learned directly from observed driving behavior by a machine learning model. With the cost volume, the computing system can look up the cost associated with each potential trajectory in the cost volume frequently, for example, after every 10 centimeters of travel, and then select the best trajectory based on the cost.

[0006] The use of manually designed costs to generate cost volumes may have limitations. Firstly, cost volumes generated from manually designed costs may not effectively address how vehicle 100 should interact with other agents in non-edge cases. Non-edge cases may be related to elegance, such as how smoothly to pass an agent. For example, should vehicle 100 slow down slightly before passing cyclist 118, or should vehicle 100 proceed at a higher speed, and what is the exact distance vehicle 100 should maintain from cyclist 118? Secondly, given the vast number of possible driving scenarios, determining manually designed costs can be difficult to scale. Thirdly, manually designed costs may require expert knowledge of the problem domain, which can be time-consuming. Finally, the design of manually designed costs may be limited by the complexity that humans can introduce. The use of machine learning models to learn about cost volumes may be useful in addressing the limitations of manually designed costs by learning from observed driving behavior. However, relying solely on observed driving behavior may also have limitations. Firstly, the interpretability of the trained machine learning model may be limited. Secondly, since the cost volume is generated based solely on observed driving behavior, it can only be as complete as the data provided. For example, if the observed driving behavior lacks certain scenarios without limitation, the cost volume may not adequately reflect the costs of such scenarios. Thirdly, this can lack flexibility, as defining costs is difficult because everything is learned from observed driving behavior. As a result, if it is desired to use certain movement restrictions that are not necessarily reflected in the observed driving behavior when calculating costs, it may be necessary to generate data incorporating these movement restrictions through simulation or manual execution, which can be costly and difficult to scale.Finally, there are often times when drivers do not want to learn from bad driving habits (for example, perhaps the observed driving behavior includes excessive aggressive driving).

[0007] To address the shortcomings of both of the two aforementioned methods for generating cost volumes, embodiments disclosed herein generate cost volumes based not only on observed driving behavior but also on manually designed costs, and determine the trajectory costs accordingly. The integration of manually designed costs and observed driving behavior can introduce advantages from both worlds. Embodiments disclosed herein can incorporate multiple manually designed costs into the training of a machine learning model. In certain embodiments, the computing system can determine an initial cost volume associated with multiple potential trajectories of vehicle 100 in the environment based on a set of movement constraints for vehicle 100. The computing system can then generate a delta cost volume using the initial cost volume and environmental data associated with the environment. In certain embodiments, the delta cost volume can be generated by determining adjustments to the initial cost volume incorporating observed driving behavior. The computing system can further score one of the multiple potential trajectories of vehicle 100 based on the initial cost volume and the delta cost volume.

[0008] Figure 2 shows an illustrative cost assessment. The cost assessment may be for a vehicle 100 traveling in a lane. The vehicle 100 can be represented by a black dot 202. The lateral direction may indicate the direction in relation to the lane boundary. As shown in Figure 2, the further the vehicle 100 deviates from the center 204 of the lane, the higher the cost. Figure 2 can indicate that as the cost gradually decreases, a favorable trajectory 206 for the vehicle 100 may be progressing toward the center 204.

[0009] Figure 3 shows an illustrative difference between the initial cost volume and the final cost volume. As shown in Figure 3, the cost volume 300 can be visualized as a three-dimensional volume based on space (indicated by the x and y axes) and time (indicated by the t axis). The cost volume 300 may have cost measurements at multiple locations along each of the associated potential trajectories with multiple timestamps (e.g., t0 and t1). In a particular embodiment, the initial cost volume may have initial cost measurements at multiple locations along each of the associated potential trajectories with multiple timestamps. The initial cost volume can be implemented as a three-dimensional lookup table having cells encoded by the initial cost measurements. As an example, without limitation, a slice 302 of the initial cost volume at t0 may have several cost measurements at different locations along the x and y axes. As another example, without limitation, a slice 304 of the initial cost volume at t1 may have several cost measurements at different locations along the x and y axes.

[0010] In certain embodiments, the calculation system can generate a final cost volume based on the sum of an initial cost volume and a delta cost volume. The final cost volume may have final cost measures at multiple locations associated with multiple timestamps. Each final cost measure may have adjustments to the initial cost measure. In certain embodiments, the calculation system can determine that the delta cost volume ensures that each final cost measure of the final cost volume exceeds a threshold measure. Similarly, the calculation system may also implement the final cost volume as a three-dimensional lookup table having cells encoded by the final cost measures. As an example, without limitation, a slice 306 of the final cost volume at t0 may have several cost measures at different locations along the x and y axes. By comparing slice 306 with slice 302, it can be seen that these cost measures are adjusted from those in the initial cost volume at the corresponding locations. For example, in the case of location 310 at t0, the initial cost measure may be 0.5 according to slice 302, while the final cost measure may be 0.4 according to slice 306, due to an adjustment of -0.1. As another example, without limitation, a slice 308 of the final cost volume at t1 may have several cost measures at different locations along the x and y axes. Furthermore, by comparing slice 308 with slice 304, it can be seen that these cost measures are also adjusted from those in the initial cost volume at the corresponding locations. For example, at location 320 at t1, the initial cost measure may be 0.3 according to slice 304, while the final cost measure may be 0.6 according to slice 308, due to the adjustment of 0.3. In certain embodiments, the cost measures associated with the initial or final cost volume may be based on one or more of the following: location cost, velocity cost, steering angle-based cost, acceleration cost, jerk cost, any appropriate cost, or any appropriate combination thereof.Speed ​​cost, steering angle-based cost, acceleration cost, or jerk cost may be manually designed or may be machine-learned from observed driving behavior. For example, without limitation, in the case of location 310 at t0, the location cost may indicate the cost of a vehicle located at this location at this point in time, while the speed cost may indicate the cost of a vehicle located at this location at this point in time traveling at a specific speed.

[0011] Figure 4 shows an exemplary architecture for generating the final cost volume. As shown in Figure 4, the initial cost volume 402 for vehicle 100 in the environment can be determined based on a set of movement restrictions for vehicle 100. For example, without limitation, the set of movement restrictions may have the restriction that vehicle 100 should maintain a distance from the lane boundary. For another example, without limitation, the set of movement restrictions may have the restriction that vehicle 100 should not get too close to other agents. For yet another example, without limitation, the set of movement restrictions may have the restriction that there should be no large jerks or large accelerations.

[0012] In certain embodiments, the computing system can access the environment and associated environmental data 404. The computing system can generate the environmental data 404 by rasterizing one or more top-down images of the vehicle 100 and agents around the vehicle 100 in the environment. Thus, the environmental data 404 may contain information such as the distance between the vehicle 100 and the agents, lane boundaries associated with the driving environment, the speeds of the vehicle 100 and the agents, the direction of travel of the vehicle 100 and the agents, the yield relationship between the vehicle 100 and the agents, the locations of the vehicle 100 and the agents, and one or more of these.

[0013] In certain embodiments, the computing system can then determine adjustments to the initial cost volume 402, which incorporates observed driving behavior, based on a machine learning model. The machine learning model may be trained on observed driving data (e.g., real-world road missions, simulations, etc.), thereby allowing it to incorporate observed driving behavior when determining adjustments. In certain embodiments, the machine learning model may be based on a convolutional neural network (CNN) 406. The CNN 406 can further determine adjustments, i.e., a delta cost volume 408, based on both the initial cost volume 402 and the environmental data 404, as shown in Figure 4. The computing system can then integrate the delta cost volume 408 and the initial cost volume 402 to generate a final cost volume 410.

[0014] Taking the scenario in Figure 1B as an example, the generation of the final cost volume 410 shown in Figure 4 can be intuitively explained as follows: The movement restriction may require that vehicle 100 not cross the lane boundary. Therefore, the initial cost volume 402 may have a large cost if the trajectory crosses the left lane boundary of lane 114. The environmental data 404 may include vehicle 100, other agents (including cyclist 118), and encoded lane boundaries. The CNN 406 can learn all such information from the environmental data 404. In the scenario of passing cyclist 118, the human driver may often drive away from cyclist 118, or pass cyclist 118, in which case slightly crossing the left lane boundary and then driving back to lane 114. The CNN 406 can learn such behavior, informing that vehicle 100 should pass cyclist 118 at a relative distance even if it has slightly crossed the left lane boundary. As a result, it may be desirable to slightly reduce the cost from the initial cost volume 402 for crossing the left boundary, for this reason, so that the vehicle 100 may be allowed to efficiently pass the cyclist 118 with perceived safety and elegance. For this purpose, the CNN 406 can determine an adjustment to the initial cost volume 402, i.e., a delta cost volume 408. By combining the delta cost volume 408 with the initial cost 402, the calculation system can generate a final cost volume 410 that reduces the cost when the trajectory 122 crosses the left boundary but increases the cost when the trajectory 120 is close to the cyclist 118. Thus, such a trajectory can be as close as possible to a human riding trajectory.

[0015] In certain embodiments, the computing system can train a machine learning model based on multiple training data that indicate observed driving behavior. These training data may include captured sensor data having images, videos, LiDAR point clouds, radar signals, or any combination thereof. The training process can be illustrated by the following example. The computing system can generate multiple trajectories for a particular driving environment. On the other hand, there may be default trajectories, such as a human driving trajectory associated with an environment that is considered the best trajectory. The machine learning model may have a delta cost function. Based on the initial cost volume 402 and the training data, the computing system can determine a predicted delta cost volume 408 by using the machine learning model. The computing system can then select one trajectory from the multiple trajectories based on the initial cost volume 402 and the predicted delta cost volume 408. Specifically, this may involve the step of applying the predicted delta cost volume 408 to the initial cost volume 402 to generate a temporary final cost volume 410. By using a temporary final cost volume of 410, the computing system can identify the best trajectory, i.e., the one with the smallest cost. The computing system can then compare the selected trajectory with a default trajectory to calculate the difference. The computing system can further update the machine learning model based on the comparison. The difference can be backpropagated to the machine learning model for its optimization. As an example, without limitation, the difference may be fed into a delta cost function that can output a loss. The computing system can then determine whether the loss has been minimized, which may be based on several iterations. For example, in the t-th iteration, the loss can be compared to the loss in the previous (t-1)-th iteration. If the loss in the t-th iteration exceeds the loss in the (t-1)-th iteration, the computing system can update the parameters of the machine learning model.The training can continue to the (t + 1)-th iteration. In the (t + 1)-th iteration, the computing system may update the predicted delta cost volume 408 by using the updated machine learning model, may reselect a trajectory based on the initial cost volume 402 and the updated predicted delta cost volume 408, may calculate the difference between the reselected trajectory and the predefined trajectory, may output a loss based on the difference, and may compare the loss with the loss in the t-th iteration. If the loss in the (t + 1)-th iteration is less than or equal to the loss in the t-th iteration, the computing system can terminate the training by the machine learning model as being optimized. If the loss in the (t + 1)-th iteration still exceeds the loss in the t-th iteration, the training process can proceed with further iterations until the point where the loss is minimized.

[0016] Figures 5A to 5D illustrate an illustrative cost comparison between two trajectories using the initial cost volume 402 of the scenario in Figures 1A to 1D. The x-axis represents the horizontal direction, the y-axis represents the vertical direction, and t represents time. Figure 5A illustrates an illustrative cost comparison between two trajectories using the initial cost volume 402 of the scenario in Figure 1A. Time t0 may be the starting point, in which case the calculation system needs to determine the cost for each potential trajectory. Thus, trajectories 110 and 112 may both be at the same location, and their costs may both be 0.1. As an example, without limitation, the calculation system can check a lookup table similar to slice 302 in Figure 3. The calculation system can identify the location of both trajectories and then obtain the initial cost measurement at that location. At time t1, trajectory 110 may remain straight, while trajectory 112 may be moving to the right to change to lane 104. According to the initial cost volume 402, the cost of trajectory 110 may be 0.3, and that of trajectory 112 may be 0.7 (i.e., the cost is higher during the lane change process). As an example, without limitation, the calculation system could check a lookup table similar to slice 304 in Figure 3. The calculation system could identify the locations of both trajectories and then obtain initial cost measurements at the individual locations of the two trajectories. At time t2, trajectory 110 may still be in a straight state, while trajectory 112 may be in the process of completing a lane change. According to the initial cost volume 402, the cost of trajectory 110 may be 0.5, and that of trajectory 112 may also be 0.5 (the lane change is complete, and vehicle 100 is again centered in lane 112 so that the cost decreases). The comparison may be the result of the initial cost volume 402 being generated based on a movement constraint that vehicle 100 should try to stay in the lane and avoid the lane change.

[0017] Figure 5B shows an exemplary cost comparison between two trajectories using the initial cost volume 402 of the scenario of FIG. 1B. Time point t0 may be the starting point, in which case the computing system needs to determine the cost for each potential trajectory to pass by cyclist 118. Thus, both trajectory 120 and trajectory 122 may be at the same location, and these costs may both be 0.1. At time point t1, trajectory 120 may maintain a straight state, while trajectory 122 may be moving slightly to the left beyond the left boundary. According to the initial cost volume 402, the cost of trajectory 120 may be 0.1, and that of trajectory 122 may be 0.3 (i.e., the cost is high because vehicle 100 is passing through the left lane boundary). At time point t2, trajectory 120 may still maintain a straight state, while trajectory 180 may be moving to the right to return to the center of lane 114. According to the initial cost volume 402, the cost of trajectory 120 may be 0.1, and that of trajectory 122 may be 0.2 (vehicle 100 is again located within the lane boundary so that the cost decreases). Note that both costs may decrease, which may be due to vehicle 100 having finished passing by cyclist 118. The comparison may be the result of the initial cost volume 402 being generated based on a movement restriction that vehicle 100 should not attempt to cross the lane boundary so as to stay within the lane. <![CDATA[ ]]><![CDATA[

[0018] ]]><![CDATA[ ]]>Figure 5C shows an illustrative cost comparison between two trajectories using the initial cost volume 402 of the scenario in Figure 1C. Time t0 may be the starting point, in which case the calculation system needs to determine the cost for each potential trajectory to turn left. Thus, both trajectories 132 and 136 may be located in the same place, and their costs may both be 0.1. At time t1, according to the initial cost volume 402, the cost of trajectory 132 may be 0.4 and that of trajectory 136 may be 0.6. At time t2, trajectory 132 may terminate in the middle of lane 134, while trajectory 136 may terminate relatively close to the lane boundary line 138. According to the initial cost volume 402, the cost of trajectory 132 may be 0.2 and that of trajectory 136 may be 0.5. The comparison may be a result of the initial cost volume 402 being generated based on the movement constraint that when turning left, vehicle 100 should attempt to terminate in the center of the target lane. The movement constraint may overlook a situation in which there is also another vehicle 126 turning left, and its potential trajectory 130 may impose a certain degree of risk on vehicle 100.

[0019] Figure 5D shows an illustrative cost comparison between two trajectories using the initial cost volume 402 of the scenario in Figure 1D. Time t0 may be the starting point, in which case the calculation system needs to determine the cost for each potential trajectory to approach the stop sign 140. Thus, both trajectories 142 and 144 may be at the same location, and their costs may both be 0.1. At time t1, trajectory 144 may reduce its speed, while trajectory 142 can maintain its original speed. According to the initial cost volume 402, the cost of trajectory 144 may be 0.2, and that of trajectory 142 may be 0.5. At time t2, trajectory 144 may continue to reduce its speed, while trajectory 142 can continue at its original speed. According to the initial cost volume 402, the cost of trajectory 144 may be 0.2, and that of trajectory 142 may be 0.5. The comparison may be a result of the initial cost volume 402 being generated based on the movement constraint that vehicle 100 should begin to decelerate relatively early in time when approaching stop sign 140. The initial cost volume 402 may impose a penalty for approaching stop sign 140 at an excessively high speed.

[0020] Figures 6A to 6D illustrate an illustrative cost comparison between two trajectories using the final cost volume 410 for the scenarios in Figures 1A to 1D. The x-axis represents the horizontal direction, the y-axis represents the vertical direction, and t represents time. Figure 6A illustrates an illustrative cost comparison between two trajectories using the final cost volume 410 for the scenario in Figure 1A. Time t0 may be the starting point, in which case the calculation system needs to determine the cost for each potential trajectory. Thus, both trajectories 110 and 112 may be at the same location, and their costs may both be 0.1. As an example, without limitation, the calculation system can check a lookup table similar to slice 306 in Figure 3. The calculation system can identify the location of both trajectories and then obtain the final cost measurement for that location. At time t1, trajectory 110 may remain in a straight line, while trajectory 112 may be moving to the right to change to lane 104. According to the differential cost volume 410, the cost of trajectory 110 may be 0.4, and that of trajectory 112 may be 0.3. As an example, without limitation, the calculation system could check a lookup table similar to slice 308 in Figure 3. The calculation system could identify the locations of both trajectories and then obtain the final cost measurements for the individual locations of the two trajectories. The cost of trajectory 112 being less than that of trajectory 110 may be due to an adjustment to the initial cost volume 402 that prompts a lane change while following the cargo truck 108. Such adjustments can be learned from observed driving behavior that signals a human driver to make a lane change to avoid continuing in close proximity to the cargo truck 108 and to have a relatively better field of view. At time t2, trajectory 110 may still be in a straight line, while trajectory 112 may be moving to the right to change to lane 104. According to the final cost volume 410, the cost of trajectory 110 may be 0.5, and the cost of trajectory 112 may be 0.3.This may be due to an adjustment to the initial cost volume 402 to reduce the cost of changing lanes while following cargo truck 108. The final cost volume 410 may be generated based on both the initial cost volume 402 and the observed driving behavior, which provides the advantage of ensuring not only safety but also elegance.

[0021] Figure 6B shows an illustrative cost comparison between two trajectories using the final cost volume 410 for the scenario in Figure 1B. Time t0 may be the starting point, in which case the calculation system needs to determine the cost of each potential trajectory to pass cyclist 118. Thus, both trajectories 120 and 122 may be located in the same place, and their costs may both be 0.1. At time t1, trajectory 120 may remain straight, while trajectory 120 may be moving slightly to the left beyond the left boundary. According to the final cost volume 410, the cost of trajectory 120 may be 0.2, and that of trajectory 122 may be 0.1. This may be due to an adjustment to the initial cost volume 402 that encourages vehicle 100 to move further away from cyclist 118 even when crossing the lane boundary. The adjustment to the initial cost volume 402 may also impose a penalty on vehicle 100 for getting too close to cyclist 118. Such adjustments can also be learned from observed driving behavior. At time t2, trajectory 120 may still remain straight, while trajectory 122 may be moving to the right to return to the center of lane 114. According to the final cost volume 410, the cost of trajectory 120 may be 0.1, and that of trajectory 122 may be 0.08. The final cost volume 410 may be generated based on both the initial cost volume 402 and the observed driving behavior, which has the advantage of guaranteeing not only safety but also elegance.

[0022] Figure 6C shows an illustrative cost comparison between two trajectories using the final cost volume 410 for the scenario in Figure 1. Time t0 may be the starting point, in which case the calculation system needs to determine the cost for each potential trajectory to turn left. Thus, both trajectories 132 and 136 may be located in the same place, and their costs may both be 0.1. At time t1, according to the final cost volume 410, the cost of trajectory 132 may be 0.45, and that of trajectory 136 may be 0.3. At time t2, trajectory 132 may terminate in the middle of lane 134, while trajectory 136 may terminate relatively close to the lane boundary line 138. According to the final cost volume 410, the cost of trajectory 132 may be 0.4, and that of trajectory 136 may be 0.2. The comparison may indicate that the adjustment to the initial cost volume 402 prompts vehicle 100 to move further away from another vehicle 126 that is also turning left. Alternatively, the adjustment to the initial cost volume 402 may also penalize vehicle 100 for getting too close to other vehicles 126 that are turning left. Such adjustments may also be learned from observed driving behavior, for humans can generally maintain a reasonable distance from other vehicles whenever possible, provided it is safe and their actions do not violate traffic regulations. As a result, the final cost volume 410 may have the advantage of guaranteeing not only safety but also elegance.

[0023] Figure 6D shows an illustrative cost comparison between two trajectories using the final cost volume 410 for the scenario in Figure 1D. Time t0 may be the starting point, in which case the calculation system needs to determine the cost for each potential trajectory to approach the stop sign 140. Thus, both trajectories 142 and 144 may be located in the same place, and their costs may both be 0.1. At time t1, trajectory 144 may reduce its speed, while trajectory 142 can maintain its original speed. According to the final cost volume 410, the cost of trajectory 144 may be 0.1, and that of trajectory 142 may be 0.6. At time t2, trajectory 144 may continue to reduce its speed, while trajectory 142 can continue at its original speed. According to the differential cost volume 410, the cost of trajectory 144 may be 0.1, and that of trajectory 142 may be 0.9. The comparison may indicate that the adjustment to the initial cost volume 402 encourages the vehicle 100 to begin decelerating relatively earlier in time when approaching the stop sign 140. The adjustment to the initial cost volume 402 may impose further penalties for approaching the stop sign 140 at excessive speed, which is reflected by the cost of the trajectory 142 being higher than that from the initial cost volume 402. As can be observed, the final cost volume 410 may be consistent with the initial cost volume 402 in evaluating how to approach the stop sign 140. The reason this makes sense is that approaching the stop sign 140 at a gradually decreasing speed can be determined not only by movement restrictions but also reflected by observed driving behavior (i.e., human drivers often decelerate when approaching the stop sign 140).

[0024] In a particular embodiment, the calculation system can score multiple remaining potential trajectories of vehicle 100 based on an initial cost volume 402 and a data cost volume 408. In other words, the calculation system can score potential trajectories using a final cost volume 410. A lower cost may result in a higher score. In a particular embodiment, scoring the trajectories of vehicle 100 may include a step of determining the cost of the trajectories. The determination may include the following steps: First, the calculation system can identify multiple candidate timestamps and associated candidate locations of the trajectory from multiple locations associated with multiple timestamps. Next, the calculation system can determine multiple final cost measures corresponding to the multiple candidate timestamps and associated candidate locations of the trajectory. The calculation system can further determine the cost of the trajectories based on the multiple final cost measures. Next, in a particular embodiment, the calculation system can rank multiple potential trajectories based on their individual scores. Using the individual scores, the calculation system can further select the top-ranked potential trajectories from the multiple potential trajectories as planned trajectories for vehicle 100. Taking the scenarios in Figures 6A to 6D as examples, the calculation system can determine the planned trajectory as follows. In the scenario of Figure 6A, the calculation system can use the final cost volume 410 to score trajectories 110 and 112. The calculation system may select trajectory 112 as the planned trajectory based on its score being higher than that of trajectory 110, which indicates that vehicle 100 should change lanes to avoid following cargo truck 108. In the scenario of Figure 6B, the calculation system can use the final cost volume 410 to score trajectories 120 and 122. The calculation system may select trajectory 122 as the planned trajectory based on its score being higher than that of trajectory 120, which indicates that vehicle 100 should travel further away from cyclist 118, even if it only slightly crosses the left boundary.In the scenario of Figure 6C, the calculation system can use the final cost volume 410 to score trajectories 132 and 136. The calculation system may select trajectory 136 as the planned trajectory based on its score being higher than that of trajectory 132, which indicates that vehicle 100 should turn left with as much distance as possible from other vehicles 126 that are also turning left. In the scenario of Figure 6D, the calculation system can use the final cost volume 410 to score trajectories 142 and 144. The calculation system may select trajectory 144 as the planned trajectory based on its score being higher than that of trajectory 142, which indicates that vehicle 100 should begin to decelerate earlier in time as it approaches the stop sign 140.

[0025] Figure 7 shows an exemplary method 700 for determining a planned trajectory for a vehicle. The method may begin in step 710, in which case the calculation system can determine an initial cost volume 402 associated with multiple potential trajectories of the vehicle 100 in the environment based on a set of movement constraints for the vehicle 100. In step 720, the calculation system may generate a delta cost volume 408 using the initial cost volume 402 and environmental data 404 associated with the environment, in which case the delta cost volume 408 is generated by determining adjustments to the initial cost volume 402 which incorporates the observed driving behavior. In step 730, the calculation system may generate a final cost volume 410 based on the sum of the initial cost volume 402 and the delta cost volume 408, in which case the final cost volume 410 has final cost measures. In step 740, the calculation system can determine whether the delta cost volume 408 ensures that each final cost measure of the final cost volume 410 exceeds a threshold measure. If not all final cost measurements exceed the threshold measurement, the method can repeat steps 720 to 740. If each final cost measurement of the final cost volume 410 exceeds the threshold measurement, the method can proceed to step 750. In step 750, the calculation system can score one of several potential trajectories of vehicle 100 based on the initial cost volume 402 and the delta cost volume 408. In step 760, the calculation system can score the remaining several potential trajectories of vehicle 100 based on the initial cost volume 402 and the delta cost volume 408. In step 770, the calculation system can rank the several potential trajectories based on their individual scores. In step 780, the calculation system can use the individual scores to select the top-ranked potential trajectory from the several potential trajectories as the planned trajectory of vehicle 100. Specific embodiments may, as appropriate, repeat one or more steps of the method shown in Figure 7.While this disclosure describes and illustrates specific steps of the method in Figure 7 as occurring in a particular order, this disclosure assumes that any suitable steps of the method in Figure 7 may occur in any suitable order. Furthermore, while this disclosure describes and illustrates an exemplary method for determining a vehicle's planned trajectory that includes specific steps of the method in Figure 7, this disclosure assumes any suitable method for determining a vehicle's planned trajectory that includes any suitable steps, which may, as appropriate, include all or some of the steps of the method in Figure 7, or none of them. Furthermore, while this disclosure describes and illustrates specific components, devices, or systems for performing specific steps of the method in Figure 7, this disclosure assumes any suitable combination of any suitable components, devices, or systems for performing any suitable steps of the method in Figure 7.

[0026] Figure 8 shows a block diagram illustrating a transportation management environment for matching passengers with autonomous vehicles. In a particular embodiment, the environment may include various computing entities such as a user computing device 830 for a user 801 (e.g., a passenger provider or passenger), a transportation management system 860, an autonomous vehicle 840, and one or more third-party systems 870. The computing entities can be connected fluently over any suitable network 810. As an example, without limitation, one or more parts of the network 810 may include an ad-hoc network, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), a part of the internet, a part of the public switched telephone network (PSTN), a cellular network, or any combination of the above. In a particular embodiment, any suitable network configuration and protocol may be used that enables the computing entities to communicate with one another. Figure 8 shows a single user device 830, a single transport management system 860, a single vehicle 840, multiple third-party systems 870, and a single network 810, but this disclosure assumes any appropriate number of each of these entities. As an example, without limitation, a network environment may include multiple users 801, user devices 830, transport management systems 860, autonomous vehicles 840, third-party systems 870, and a network 810.

[0027] The user device 830, the transportation management system 860, the autonomous vehicle 840, and the third-party system 870 may be connected to each other in whole or in part, or they may be located in the same place. These computing entities can communicate via different transmission technologies and network types. For example, the user device 830 and the vehicle 840 can communicate with each other via cable or short-range wireless communication (e.g., Bluetooth®, NFC, Wi-Fi, etc.), and together they can connect to the Internet via a cellular network accessible from any one of the devices (for example, the user device 830 may be a smartphone with LTE connectivity). On the other hand, the transportation management system 860 and the third-party system 870 can connect to the Internet via their respective LAN / WLAN networks and Internet service providers (ISPs). Figure 8 shows a transmission link 850 connecting the user device 830, the autonomous vehicle 840, the transportation management system 860, and the third-party system 870 to a communication network 810. This disclosure envisions any suitable transmit link 850, including, for example, wired connections (e.g., USB, Lightning, DSL (Digital Subscriber Line), or DOCSIS (Data Over Cable Service Interface Specification)), wireless connections (e.g., Wi-Fi, WiMAX, cellular, satellite, NFC, Bluetooth®), optical connections (e.g., SONET (Synchronous Optical Networking), SDH (Synchronous Digital Hierarchy)), any other wireless communication technologies, and any combination thereof. In certain embodiments, one or more links 850 may be connected to one or more networks 810, which may in part include, for example, ad-hoc networks, intranets, extranets, VPNs, LANs, WLANs, WANs, WWANs, MANs, PSTNs, cellular networks, satellite networks, or any combination thereof.Computation entities do not necessarily have to use the same type of transmission link 850. For example, user device 830 may communicate with the transportation management system via a cellular network and the internet, but may also communicate with the autonomous vehicle 840 via Bluetooth (registered trademark) or a physical wire connection.

[0028] In certain embodiments, the transport management system 860 can fulfill ride requests for one or more users 801 by dispatching appropriate vehicles. The transport management system 860 can receive any number of ride requests from any number of ride requesters 801. In certain embodiments, ride requests from ride requesters 801 may include identifiers that identify the ride requesters within the system 860. The transport management system 860 can use the identifiers to access and store information about the ride requesters 801 according to the requesters' privacy settings. Information about the ride requesters 801 may be stored in one or more data stores (e.g., relational database systems) that are associated with and accessible from the transport management system 860. In certain embodiments, the ride requester information may include profile information about a particular ride requester 801. In certain embodiments, a ride requester 801 may be associated with one or more categories or types, thereby the ride requester 801 may be associated with aggregated information about specific ride requesters of those categories or types. The passenger information may include, for example, preferred pick-up and drop-off locations, driving preferences (e.g., safety and comfort level, preferred speed, acceleration / deceleration rate, safe distance from other vehicles when traveling at various speeds, route, etc.), entertainment preferences and settings (e.g., preferred music genre or playlist, audio volume, display brightness, etc.), temperature settings, whether conversation with the driver is welcome, frequent destinations, historical ride patterns (e.g., time of travel, start and end locations, etc.), preferred language, age, gender, or any other appropriate information. In certain embodiments, the transport management system 860 may classify user 801 based on known information about the user 801 (e.g., using a machine learning classifier) ​​and may use the classification to obtain relevant aggregate information associated with that class.For example, system 860 can identify user 801 as a young adult and obtain relevant aggregate information associated with young adults, such as the types of music that young adults generally prefer.

[0029] Furthermore, the transportation management system 860 can store and access ride information. Ride information may include locations related to the ride, traffic data, route options, optimal pick-up or drop-off locations for the ride, or any other relevant information related to the ride. For example, without limitation, when the transportation management system 860 receives a request to travel from San Francisco International Airport (SFO) to Palo Alto, California, the system 860 may access or generate any relevant ride information for this particular ride request. Ride information may include, for example, a preferred pick-up location in SFO, an alternative pick-up location if the pick-up location is unsuitable for the passenger (e.g., the passenger has a disability and cannot access the pick-up location) or if the pick-up location is unavailable due to construction, traffic congestion, changes in pick-up / drop-off rules, or any other reason, one or more routes for navigating from SFO to Palo Alto, a preferred exit ramp for the user type, or any other relevant information related to the ride. In certain embodiments, a portion of the ride information may be based on historical data associated with historical rides facilitated by the system 860. For example, historical data may include aggregated information generated based on past ride information, which may include any ride information described herein and telemetry data collected by sensors in autonomous vehicles and / or user devices. Historical data may be associated with a specific user (e.g., that particular user's preferences, common routes, etc.), a user's category / class (e.g., based on demographics), and / or all users of system 860. For example, historical data unique to a single user may include information about past rides taken by that particular user, including where the user got on and off, the music the user wishes to listen to, traffic information associated with the ride, the time of day the user most frequently rode, and any other relevant information unique to the user.As another example, historical data associated with a user's category / class may include ride preferences common to or popular among users within that category / class, such as teenagers who prefer pop music, or ride requesters who frequently travel to the financial district, who may desire to listen to the news. As yet another example, historical data associated with all users may include general usage trends, such as traffic and ride patterns. By using historical data, system 860 in a particular embodiment can predict and provide ride suggestions in response to ride requests. In a particular embodiment, system 860 can use machine learning such as neural networks, regression algorithms, instance-based algorithms (e.g., k-nearest neighbors), decision tree algorithms, Bayesian algorithms, clustering algorithms, correlation rule learning algorithms, deep learning algorithms, dimensionality reduction algorithms, ensemble algorithms, and any other suitable machine learning algorithms known to those skilled in the art. Machine learning models can be trained using any suitable training algorithm, including supervised learning based on labeled training data, unsupervised learning based on unlabeled training data, and / or semi-supervised learning based on a mixture of labeled and unlabeled training data.

[0030] In certain embodiments, the transportation management system 860 may include one or more server computers. Each server may be a single server or a distributed server spanning multiple computers or multiple data centers. Servers may be of various types, such as, for example, a web server, a news server, a mail server, a message server, an advertising server, a file server, an application server, a switching server, a database server, a proxy server, another server suitable for performing the functions or processes described herein, or any combination thereof. In certain embodiments, each server may include hardware, software, or embedded logic components, or a combination of two or more such components, that perform the appropriate functions implemented or supported by the server. In certain embodiments, the transportation management system 860 may include one or more data stores. Data stores may be used to store various types of information, such as passenger information, passenger requester information, passenger provider information, historical information, third-party information, or any other appropriate type of information. In certain embodiments, the information stored in the data store may be organized according to a specific data structure. In certain embodiments, each data store may be a relational, columnar, correlated, or any other appropriate type of database system. While this disclosure describes or illustrates certain types of databases, it assumes any suitable type of database. Certain embodiments may provide interfaces that enable a user device 830 (which may belong to a passenger requester or provider), a transport management system 860, a vehicle system 840, or a third-party system 870 to process, transform, manage, retrieve, modify, add, or delete information stored in a data store.

[0031] In certain embodiments, the transport management system 860 may include an authorization server (or any one or more other suitable components) that allows a user 801 to opt in or opt out of having their information and actions logged, recorded, or detected by the transport management system 860 or shared with other systems (e.g., a third-party system 870). In certain embodiments, a user 801 may opt in or opt out by setting appropriate privacy settings. A user's privacy settings can determine what user-related information may be logged, how user-related information may be logged, when user-related information may be logged, who may log user-related information, who may share user-related information, and for what purposes user-related information may be logged or supplied. The authentication server may be used to enforce one or more of user 801's privacy settings in the transport management system 860 through blocking, data hashing, anonymization, or other suitable techniques, as appropriate.

[0032] In certain embodiments, the third-party system 870 may be a network-addressable computing system capable of providing HD maps or host GPS maps, customer reviews, music or content, weather information, or any other appropriate type of information. The third-party system 870 can generate, store, receive, and transmit relevant data, such as map data, customer review data from customer review websites, weather data, or any other appropriate type of data. The third-party system 870 can be accessed by other computing entities in the network environment, either directly or via the network 810. For example, user equipment 830 can access the third-party system 870 via the network 810 or via the transport management system 860. In the latter case, if credentials are required to access the third-party system 870, user 801 may provide such information to the transport management system 860, which can then act as a proxy for accessing content from the third-party system 870.

[0033] In certain embodiments, the user device 830 may be a mobile computing device such as a smartphone, tablet computer, or laptop computer. The user device 830 may include one or more processors (e.g., CPU and / or GPU), memory, and storage. An operating system, as well as applications such as a transportation management system 860 and related transportation applications, an application related to a third-party system 870, and an application related to the operating system, can be installed on the user device 830. The user device 830 may include a function to determine its location, direction, or orientation based on integrated sensors such as a GPS, compass, gyroscope, or accelerometer. The user device 830 may also include a wireless transceiver for wireless communication and may support wireless communication protocols such as Bluetooth®, Near Field Communication (NFC), Infrared (IR) communication, Wi-Fi, and / or 2G / 3G / 4G / LTE mobile communication standards. The user device 830 may also include one or more cameras, scanners, touchscreens, microphones, speakers, and any other suitable input / output devices.

[0034] In certain embodiments, the vehicle 840 may be an autonomous vehicle and may be equipped with a sensor array 844, a navigation system 846, and a ride service calculation system 848. In certain embodiments, a group of autonomous vehicles 840 may be managed by a transport management system 860. The group of autonomous vehicles 840 may be owned in whole or in part by entities associated with the transport management system 860, or they may be owned by third-party entities in relation to the transport management system 860. In either case, the transport management system 860 can control the operation of the autonomous vehicle 840, including, for example, dispatching a vehicle 840 selected to fulfill a passenger request, commanding the vehicle 840 to perform selected actions (e.g., going to a service center or charging / refueling station, stopping, emergency braking, self-diagnosticating, locking / unlocking compartments, changing music stations, changing temperature, and any other appropriate actions), and commanding the vehicle 840 to enter a selected operating mode (e.g., a normal operating mode, a reduced speed driving mode, a driving under human operator command mode, and any other appropriate operating mode).

[0035] In certain embodiments, the autonomous vehicle 840 can receive and transmit data to and from the transportation management system 860 and the third-party system 870. Examples of data received may include, for example, instructions, new software or software updates, maps, 3D models, trained or untrained machine learning models, location information (e.g., the location of passengers, the autonomous vehicle 840 itself, and other locations of the autonomous vehicle 840, as well as target destinations such as service centers), navigation information, traffic information, weather information, entertainment content (e.g., music, videos, and news), passenger information, passenger information, and any other relevant information. Examples of data transmitted from the autonomous vehicle 840 may include, for example, telemetry and sensor data, decisions / judgments based on such data, vehicle status or conditions (e.g., battery / fuel level, tire and brake status, sensor status, speed, odometer, etc.), location, navigation data, occupant input (e.g., occupants can transmit / receive data to / from the transportation management system 860 and / or the third-party system 870 through a user interface within the vehicle 840), and any other relevant data.

[0036] Furthermore, in certain embodiments, the autonomous vehicles 840 can communicate not only with each other but also with other conventional human-driven vehicles, including those managed or not managed by the transport management system 860. For example, one vehicle 840 can communicate with another vehicle data regarding its individual location, state, circumstances, sensor readings, and any other relevant information. In certain embodiments, inter-vehicle communication may occur over direct short-range wireless connections (e.g., Wi-Fi, Bluetooth®, NFC) and / or over a network (e.g., via the Internet or the transport management system 860 or a third-party system 870).

[0037] In certain embodiments, the autonomous vehicle 840 can acquire and process sensor / telemetry data. Such data can be captured by any suitable sensor. For example, the vehicle 840 may have a LiDAR sensor array of multiple light-detecting and ranging (LiDAR) transceivers configured to rotate 360°, emitting pulsed laser light and measuring reflected light from objects surrounding the vehicle 840. In certain embodiments, the signal-transmitting LiDAR may be steered by the use of a gated optical valve, which may be a MEMS device that guides a light beam using the principle of light diffraction. Such a device does not require the use of a gimbaled mirror to steer the light beam 360° around the autonomous vehicle. Rather, the gated optical valve may guide the light beam into one of several optical fibers, which can be configured so that the light beam can be guided to many distinct locations around the autonomous vehicle. Thus, data can be captured 360° around the autonomous vehicle, but without the need for rotating parts. LiDAR is an effective sensor for measuring distance to a target and can therefore be used to generate a three-dimensional (3D) model of the external environment of the autonomous vehicle 840. For example, without limitation, the 3D model may represent the external environment including objects such as other vehicles, curbs, rubble, objects, and pedestrians within the maximum distance of the sensor configuration (e.g., 50, 100, or 200 meters). As another example, the autonomous vehicle 840 may have optical cameras oriented in different directions. These cameras can be used, for example, to recognize roads, lane markings, road signs, traffic lights, police, other vehicles, and any other visible objects of interest. Infrared cameras may be installed to enable the vehicle 840 to "observe" at night. In certain embodiments, the vehicle may be equipped with stereo vision to detect hazards such as pedestrians or tree branches on the road. As yet another example, the vehicle 840 may have radar for detecting other vehicles and / or hazards at a distance.Furthermore, the vehicle 840 may have ultrasonic equipment for, for example, parking and / or obstacle detection. In addition to sensors that enable the vehicle 840 to detect, measure, and understand the external world around it, the vehicle 840 may be further equipped with sensors for detecting and self-diagnosing the vehicle's own state and condition. For example, the vehicle 840 may have, for example, wheel sensors for measuring speed, for example, a Global Positioning System (GPS) for determining the vehicle's current geolocation, and / or inertial measurement units, accelerometers, gyroscopes, and / or odometer systems for detecting movement or motion. While these descriptions of sensors provide specific examples of their usefulness, those skilled in the art will understand that the usefulness of sensors is not limited to these examples. Furthermore, while examples of usefulness may be described in relation to specific types of sensors, it should be understood that usefulness can be achieved using any combination of sensors. For example, the autonomous vehicle 840 can construct a 3D model of its surroundings based on data from its LiDAR, radar, sonar, and cameras, along with a pre-generated map obtained from the transportation management system 860 or a third-party system 870. Although the sensor 844 appears in a specific location on the autonomous vehicle 840 in Figure 8, the sensor 844 can be placed in any suitable location inside or on top of the autonomous vehicle 840. Exemplary locations for the sensor include the front and rear bumpers, doors, front windshield, side panels, or any other suitable location.

[0038] In certain embodiments, the autonomous vehicle 840 may be equipped with processing units (e.g., one or more CPUs and GPUs), memory, and storage. Thus, the vehicle 840 may be equipped to perform a variety of computational and processing tasks, including processing sensor data, extracting useful information, and performing corresponding operations. For example, based on images captured by its camera and machine vision model, the vehicle 840 may identify certain types of objects captured by the images, such as pedestrians, other vehicles, lanes, curbs, and any other objects of interest.

[0039] In certain embodiments, the autonomous vehicle 840 may have a navigation system 846 responsible for safely navigating the autonomous vehicle 840. In certain embodiments, the navigation system 846 may take as input any type of sensor data from, for example, a Global Positioning System (GPS) module, an Inertial Measurement Unit (IMU), a LiDAR sensor, an optical camera, a Radio Frequency (RF) transceiver, or any other suitable telemetry or sensory mechanism. The navigation system 846 may also utilize, for example, map data, traffic data, accident reports, weather reports, commands, target destinations, and any other suitable information to determine the navigation route and specific driving actions (e.g., deceleration, acceleration, stopping, turning, etc.). In certain embodiments, the navigation system 846 may use its determinations to guide the autonomous vehicle 840 to its destination without colliding with other objects in order to control the vehicle 840 to operate in a directed manner. While a physical embodiment of the navigation system 846 (e.g., a processing unit) appears in a specific location on the autonomous vehicle 840 in Figure 8, the navigation system 846 can be located in any suitable location inside or on top of the autonomous vehicle 840. Exemplary locations for the navigation system 846 include inside the cabin or passenger compartment of the autonomous vehicle 840, near the engine / battery, near the front seats, near the rear seats, or any other suitable location.

[0040] In certain embodiments, the autonomous vehicle 840 may be equipped with a ride service calculation device 848, which may be a tablet or any other suitable device installed by the transportation management system 860 to allow the user to interact with the autonomous vehicle 840, the transportation management system 860, other users 801, or a third-party system 870. In certain embodiments, the installation of the passenger service calculation unit 848 can be achieved by positioning the passenger service calculation unit 848 inside the autonomous vehicle 840 and configuring it to communicate with the vehicle 840 via a wired or wireless connection (for example, via Bluetooth®). Figure 8 shows a single passenger service calculation unit 848 in a specific location within the autonomous vehicle 840, but the autonomous vehicle 840 may include several passenger service calculation units 848 in several different locations within the vehicle. As an example, without limitation, the autonomous vehicle 840 may have four passenger service calculation units positioned in locations such as one in front of the front left passenger seat (e.g., the driver's seat in a conventional US automobile), one in front of the front right passenger seat, and one in front of each of the rear left and rear right passenger seats. The passenger service calculation device 848 may be included. In certain embodiments, the passenger service calculation device 848 may be detachable from any component of the autonomous vehicle 840. This allows the user to handle the passenger service calculation device 848 in a manner consistent with other tablet calculation devices. For example, without limitation, the user may move the passenger service calculation device 848 to any location within the cabin or passenger compartment of the autonomous vehicle 840, hold the passenger service calculation device 848, or handle the passenger service calculation device 848 in any other suitable manner. While this disclosure describes providing a particular calculation device in a particular manner, this disclosure is intended to provide any suitable calculation device in any suitable manner.

[0041] Figure 9 shows a block diagram illustrating an algorithmic navigation pipeline. In a particular embodiment, the algorithmic navigation pipeline 900 may include several computing modules, such as a sensor data module 905, a perception module 910, a prediction module 915, a planning module 920, and a control module 925. The sensor data module 905 can acquire and preprocess sensor / telemetry data provided to the perception module 910. Such data can be captured by any suitable sensors of the vehicle. As an example, without limitation, the vehicle may have a light detection and ranging (LiDAR) sensor configured to measure reflected signals from objects surrounding the vehicle to transmit pulsed laser beams in multiple directions. The time of flight of the optical signals can be used to measure the distance or depth from the LiDAR to the object. As another example, the vehicle may have optical cameras oriented in different directions to capture images of the vehicle's surroundings. Radar may also be used by the vehicle to detect other vehicles and / or hazards at a given distance. As a further example, a vehicle may be equipped with, for example, ultrasonic cameras for short-range object detection such as parking and obstacle detection, or infrared cameras for object detection in low-light conditions or darkness. In certain embodiments, the sensor data module 905 may suppress noise in the sensor data or normalize the sensor data.

[0042] The perception module 910 is responsible for correlating and merging data from different types of sensors in the sensor module 905 to model the vehicle's contextual environment. The perception module 910 can use information extracted by multiple independent sensors to provide information that would not be available from any single type of sensor. By combining data from multiple sensor types, the perception module 910 is allowed to leverage the intensities of different sensors to perceive the environment with relative accuracy and precision. As an example, without limitation, image-based object recognition may not function well in low-light conditions. This may be compensated for by sensor data from LiDAR or radar, which are effective sensors for measuring distance to a target in low-light conditions. As another example, image-based object recognition may incorrectly determine that an object depicted in a poster is an actual three-dimensional object in the environment. However, if depth information from LiDAR is also available, the perception module 910 could use that additional information to determine that the object in the poster is not actually a three-dimensional object.

[0043] The perception module 910 can process available data (e.g., sensor data, data from high-resolution maps, etc.) to derive information about the contextual environment. For example, the perception module 910 may include one or more agent modelers (e.g., object detectors, object classifiers, or machine learning models trained to derive information from sensor data) to detect and / or classify agents (e.g., other vehicles, pedestrians, moving objects) present in the vehicle's environment. The perception module 910 can also determine various characteristics of the agents. For example, the perception module 910 may track the velocity, direction of movement, acceleration, trajectory, relative distance, or relative position of these agents. In certain embodiments, the perception module 910 may also leverage information from high-resolution maps. High-resolution maps may include accurate three-dimensional models of the environment, including buildings, curbs, road signs, traffic lights, and any stationary fixtures in the environment. By using localization techniques based on the vehicle's GPS data and / or images (e.g., simultaneous localization and mapping, i.e., SLAM), the perception module 910 can determine the vehicle's pose (e.g., position and orientation) or the vehicle's sensors' pose within a high-resolution map. The pose information can then be used by the perception module 910 to determine objects expected to be present in the environment for querying the high-resolution map.

[0044] The perception module 910 can use sensor data and / or information derived therefrom from one or more types of sensors to generate a representation of the vehicle's contextual environment. For example, without limitation, the representation of the external environment may include objects such as other vehicles, curbs, rubble, objects, and pedestrians. The contextual representation may be limited to the maximum distance of the sensor array (e.g., 50, 100, or 200 meters). The representation of the contextual environment may include semantic information about traffic lanes, traffic rules, traffic signs, time, weather, and / or any other appropriate information, as well as information about agents and objects surrounding the vehicle. The contextual environment can be represented in any appropriate manner. For example, without limitation, the contextual representation can be encoded as a vector or matrix of numbers, such that each value in the vector / matrix corresponds to information in a predetermined category. For example, each agent in the environment can be represented by a sequence of values ​​starting with the agent's coordinates, classification (e.g., vehicle, pedestrian, etc.), orientation, speed, trajectory, etc. Alternatively, information about the contextual environment can be represented by a raster image visually depicting the agent, semantic information, etc. For example, the raster image may be a bird's-eye view of a vehicle and its surroundings up to a predetermined distance. The raster image may include visual information (e.g., bounding boxes, color-coded shapes, etc.) that represents various data about the object (e.g., vehicles, pedestrians, lanes, buildings, etc.).

[0045] The current contextual environment representation from the perception module 910 can be consumed by the prediction module 915 to generate one or more predictions of the future environment. For example, given a representation of the contextual environment at time t0, the prediction module 915 can output another contextual representation for time t1. For example, if the contextual environment at t0 is represented by a raster image, the output of the prediction module 915 may be another raster image (e.g., a snapshot of the current environment) depicting where the agent will be located at time t1 (e.g., a future snapshot). In certain embodiments, the prediction module 915 may include a machine learning model (e.g., a convolutional neural network, a neural network, a decision tree, a support vector machine, etc.) that can be trained on pre-recorded contextual and sensor data. For example, one training sample can be generated based on a sequence of actual sensor data captured by the vehicle at times t0 and t1. The captured data at times t0 and t1 can be used to generate a first contextual representation (training data) and a second contextual representation (the relevant ground truth used for training), respectively. During training, the machine learning model can process a first contextual representation using the model's current configuration parameters and output a predicted contextual representation. The predicted contextual representation can then be compared to a known second contextual representation (i.e., the ground truth at time t1). The comparison can be quantified by a loss value calculated using a loss function. This loss value can be used to update the machine learning model's configuration parameters (e.g., via backpropagation techniques) so that the loss is relatively small if the prediction needs to be repeated. The machine learning model can be iteratively trained using large sets of training samples until a convergence or termination condition is met. For example, training may terminate when the loss value falls below a predetermined threshold.Once trained, a machine learning model can be used to generate predictions of future contextual representations based on the current contextual representation.

[0046] The planning module 920 can determine the vehicle's navigation route and specific driving actions (e.g., deceleration, acceleration, stopping, turning, etc.) based on the predicted contextual representation generated by the prediction module 915. In certain embodiments, the planning module 920 can utilize predicted information encoded within the predicted contextual representation (e.g., predicted location or trajectory of agents, semantic data, etc.) and any other available information (e.g., map data, traffic data, accident reports, weather reports, target destinations, and any other relevant information) to determine one or more destinations or navigation instructions for the vehicle. As an example, without limitation, based on the predicted behavior of agents surrounding the vehicle and traffic data to a specific destination, the planning module 920 can determine a specific navigation route and associated driving actions for the vehicle to avoid possible collisions with one or more agents. In certain embodiments, the planning module 920 can generate several different plans (e.g., destinations or navigation instructions) for the vehicle based on a given predicted contextual presentation. For each plan, the planning module 920 can calculate a score representing the desirability of that plan. For example, if a plan is likely to result in the vehicle colliding with an agent at its predicted location, based on the predicted contextual representation, the plan's score may be penalized accordingly. Another plan that causes the vehicle to violate traffic rules or take a considerable detour to avoid a possible collision may also have a penalized score, although the penalty may not be as severe as the penalty applied to the previous plan that resulted in the collision. A third plan in which the vehicle simply stops or changes lanes to avoid a collision with an agent in the predicted future may receive the highest score.Based on the assigned score for planning, the planning module 920 can select the best plan to execute. While the above example uses collision as an example, the disclosure herein envisions the use of any appropriate scoring criteria, such as distance or time traveled, fuel consumption, changes in estimated time of arrival at destination, occupant comfort, proximity to other vehicles, and confidence scores associated with predicted contextual representations.

[0047] Based on a plan generated by the planning module 920, which may include one or more navigation paths or associated driving actions, the control module 925 can determine specific commands to be issued to the vehicle's actuators. The vehicle's actuators are components responsible for moving and controlling the vehicle. Actuators control the vehicle's driving functions, such as steering, rotation signals, deceleration (braking), acceleration, and gear shifting. As an example, without limitation, the control module 925 may send a command to the steering actuator to maintain a specific steering angle over a specific amount of time in order to move the vehicle along a specific trajectory to avoid an agent that is expected to enter the vehicle's area. As another example, the control module 925 may send a command to the accelerator actuator to cause the vehicle to safely avoid an agent that is expected to enter the vehicle's area.

[0048] Figure 10 shows an illustrative computer system 1000. In a particular embodiment, one or more computer systems 1000 perform one or more steps of one or more methods described or illustrated herein. In a particular embodiment, one or more computer systems 1000 provide functions described or illustrated herein. In a particular embodiment, software running on one or more computer systems 1000 performs one or more steps of one or more methods described or illustrated herein or provides functions described or illustrated herein. A particular embodiment includes one or more parts of one or more computer systems 1000. In this specification, a reference to a computer system may, as appropriate, include an arithmetic unit and vice versa. Furthermore, a reference to a computer system may, as appropriate, include one or more computer systems.

[0049] This disclosure assumes any suitable number of computer systems 1000. This disclosure assumes computer systems 1000 having any suitable physical form. For example, without limitation, computer system 1000 may be an embedded computer system, a system on a chip (SOC), a single-board computer system (SBC) (such as a computer on a module (COM) or system on a module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile phone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented / virtual reality device, or two or more combinations thereof. Where appropriate, computer system 1000 may consist of one or more computer systems 1000, may be a single system or distributed, may span multiple locations, may span multiple machines, may span multiple data centers, or may reside in a cloud which may include one or more cloud components within one or more networks. As appropriate, one or more computer systems 1000 may perform one or more steps of one or more methods described or illustrated herein without significant spatial or temporal limitations. For example, without limitation, one or more computer systems 1000 may perform one or more steps of one or more methods described or illustrated herein in real time or in batch mode. One or more computer systems 1000 may, as appropriate, perform one or more steps of one or more methods described or illustrated herein at different times or in different locations.

[0050] In a particular embodiment, the computer system 1000 includes a processor 1002, memory 1004, storage 1006, input / output (I / O) interface 1008, communication interface 1010, and bus 1012. While this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular configuration, this disclosure assumes any suitable computer system having any suitable number of any suitable components in any suitable configuration.

[0051] In certain embodiments, the processor 1002 includes hardware for executing instructions, such as those that constitute a computer program. For example, without limitation, to execute an instruction, the processor 1002 may retrieve (or fetch) the instruction from an internal register, internal cache, memory 1004, or storage 1006, decode and execute it, and then write one or more results to an internal register, internal cache, memory 1004, or storage 1006. In certain embodiments, the processor 1002 may include one or more internal caches for data, instructions, or addresses. This disclosure assumes a processor 1002 that includes any appropriate number of appropriate internal caches as appropriate. For example, without limitation, the processor 1002 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction cache may be copies of instructions in memory 1004 or storage 1006, and the instruction cache may accelerate the retrieval of these instructions by the processor 1002. The data in the data cache may be data in memory 1004 or storage 1006 that will be processed by computer instructions, the results of previous instructions executed by processor 1002 that are accessible to or written to memory 1004 or storage 1006 by a continuation instruction, or copies of any other suitable data. The data cache can accelerate read or write operations by processor 1002. The TLB can accelerate virtual address translation for processor 1002. In certain embodiments, processor 1002 may include one or more registers for data, instructions, or addresses. This disclosure assumes, as appropriate, a processor 1002 including any appropriate number of any appropriate internal registers. As appropriate, processor 1002 may include one or more compute logic units (ALUs), may be a multicore processor, or may include one or more processors 1002.While this disclosure describes and exemplifies specific processors, this disclosure assumes any suitable processor.

[0052] In certain embodiments, memory 1004 includes main memory for storing instructions for the processor 1002 to be executed or data for the processor 1002 to be processed. For example, without limitation, computer system 1000 may read instructions into memory 1004 from storage 1006 or another source (such as another computer system 1000). Processor 1002 can then read instructions from memory 1004 into internal registers or internal cache. To execute the instructions, processor 1002 can retrieve and decode the instructions from the internal registers or internal cache. During or after the execution of the instructions, processor 1002 may write one or more results (which may be intermediate or final results) into internal registers or internal cache. Processor 1002 can then write one or more of these results into memory 1004. In certain embodiments, the processor 1002 executes instructions only in one or more internal registers or internal caches or in memory 1004 (rather than in storage 1006 or elsewhere), and processes data only in one or more internal registers or internal caches or in memory 1004 (rather than in storage 1006 or elsewhere). One or more memory buses (each possibly including an address bus and a data bus) can connect the processor 1002 to memory 1004. Bus 1012 may include one or more memory buses, as will be described in more detail later. In certain embodiments, one or more memory management units (MMUs) are present between the processor 1002 and memory 1004 to facilitate access to memory 1004 requested by the processor 1002. In certain embodiments, memory 1004 includes random access memory (RAM). This RAM may, as appropriate, be volatile memory. This RAM may, as appropriate, be dynamic RAM (DRAM) or static RAM (SRAM). Furthermore, this RAM may be single-port or multi-port RAM, as appropriate.This disclosure assumes any suitable RAM. Memory 1004 may optionally include one or more memory 1004. While this disclosure describes and illustrates specific memory, this disclosure assumes any suitable memory.

[0053] In certain embodiments, storage 1006 includes mass storage for data or instructions. For example, without limitation, storage 1006 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or two or more combinations thereof. Storage 1006 may, as appropriate, include removable or non-removable (i.e., fixed) media. Storage 1006 may, as appropriate, be located inside or outside the computer system 1000. In certain embodiments, storage 1006 is non-volatile semiconductor memory. In certain embodiments, storage 1006 includes read-only memory (ROM). This ROM may, as appropriate, be a mask program ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically modifiable ROM (EAROM), or flash memory, or two or more combinations thereof. This disclosure assumes mass storage 1006 having any suitable physical form. The storage 1006 may optionally include one or more storage control units to facilitate communication between the processor 1002 and the storage 1006. The storage 1006 may optionally comprise one or more storage units. While this disclosure describes and illustrates specific storage, it assumes any suitable storage.

[0054] In certain embodiments, the I / O interface 1008 includes hardware, software, or both that provide one or more interfaces for communication between the computer system 1000 and one or more I / O devices. The computer system 1000 may, as appropriate, include one or more of these I / O devices. One or more of these I / O devices can enable communication between a person and the computer system 1000. As an example, without limitation, an I / O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touchscreen, trackball, video camera, another suitable I / O device, or two or more combinations of these. An I / O device may include one or more sensors. This disclosure assumes any suitable I / O device and any suitable I / O interface 1008 for them. As appropriate, the I / O interface 1008 may include one or more device or software drivers that enable the processor 1002 to drive one or more of these I / O devices. The I / O interface 1008 may optionally include one or more I / O interfaces 1008. While this disclosure describes and illustrates specific I / O interfaces, this disclosure assumes any suitable I / O interface.

[0055] In certain embodiments, the communication interface 1010 includes hardware, software, or both that provide one or more interfaces for communication (e.g., packet-based communication) between computer system 1000 and one or more other computer systems 1000 or one or more networks. For example, without limitation, the communication interface 1010 may include a network interface controller (NIC) or network adapter for communicating with Ethernet or any other wired-based network, or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network such as a Wi-Fi network. This disclosure assumes any suitable network and any suitable communication interface 1010 for it. For example, without limitation, computer 1000 may communicate with one or more parts of an ad-hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or the Internet, or two or more combinations thereof. One or more parts of one or more of these networks may be wired or wireless. For example, computer system 1000 can communicate with a wireless PAN (WPAN) (e.g., Bluetooth® WPAN), a Wi-Fi network, a Wi-MAX network, a cellular telephone network (e.g., a GSM (Global System for Mobile Communications) network), or any other suitable wireless network, or two or more combinations thereof. Computer system 1000 may optionally include any suitable communication interface 1010 for any of these networks. Communication interface 1010 may optionally include one or more communication interfaces 1010. While this disclosure describes and illustrates specific communication interfaces, this disclosure assumes any suitable communication interface.

[0056] In certain embodiments, bus 1012 includes hardware, software, or both that interconnect components of computer system 1000. For example, without limitation, bus 1012 may include an AGP (Accelerated Graphics Port) or any other graphics bus, an EISA (Enhanced Industry Standard Architecture) bus, an FSB (Front-Side Bus), an HT (HyperTransport) interconnect, an ISA (Industry Standard Architecture) bus, an INFINIBAND interconnect, an LPC (Low Pin-Count) bus, a memory bus, an MCA (Micro Channel Architecture) bus, a PCI (Peripheral Component Interconnect) bus, a PCI-Express (PCIe) bus, a SATA (Serial Advanced Technology Attachment) bus, a VLB (Video electronics standards association Local Bus) bus, or another suitable bus, or two or more combinations thereof. Bus 1012 may, as appropriate, include one or more buses 1012. While this disclosure describes and illustrates a specific bus, it assumes any suitable bus or interconnection.

[0057] In this specification, one or more computer-readable non-temporary storage media may, as appropriate, include one or more semiconductor-based or other types of integrated circuits (ICs) (such as field-programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disk drives (ODDs), magneto-optical discs, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM drives, secure digital cards or drivers, any other suitable computer-readable non-temporary storage media, or any two or more suitable combinations thereof. The computer-readable non-temporary storage media may, as appropriate, be volatile, non-volatile, or a combination of volatile and non-volatile.

[0058] In this specification, "or" is inclusive, not exclusive, unless explicitly indicated otherwise or indicated otherwise by the context. Thus, in this specification, "A or B" means "A, B, or both," unless explicitly indicated otherwise or indicated otherwise by the context. Furthermore, "and" means both joint and several, unless explicitly indicated otherwise or indicated otherwise by the context. Thus, in this specification, "A and B" means "A and B jointly and individually," unless explicitly indicated otherwise or indicated otherwise by the context.

[0059] The scope of this disclosure includes all changes, substitutions, modifications, alterations, and modifications to the exemplary embodiments described or illustrated herein, as would be expected by those skilled in the art. The scope of this disclosure is not limited to the exemplary embodiments described or illustrated herein. Furthermore, while this disclosure describes and illustrates individual embodiments herein as including specific components, elements, features, functions, operations, or steps, any of these embodiments may include any combination or permutation of any components, elements, features, functions, operations, or steps described or illustrated anywhere in this specification, as would be expected by those skilled in the art. Furthermore, references in the appended claims to devices or systems or components of devices or systems that are adapted, configured, capable, configured, enabled, operable, or freely operable to perform a particular function include such devices, systems, or components, regardless of whether their particular function is activated, turned on, or unlocked, as long as such devices, systems, or components are adapted, configured, capable, configured, enabled, operable, or freely operable. In addition, while this disclosure describes or illustrates certain embodiments as providing certain advantages, certain embodiments may provide some or all of these advantages, or none of them.

Claims

1. It is a method, The steps include: determining the potential trajectory of the vehicle in the environment and the associated initial cost volume based on a set of vehicle movement restrictions using one or more processors; A step of generating a delta cost volume using the initial cost volume and environmental data of the environment by one or more processors, wherein the delta cost volume is generated by determining adjustments to the initial cost volume which incorporates observed driving behavior. The steps include scoring one of the vehicle's potential trajectories by adding the initial cost volume and the delta cost volume as the final cost volume using one or more processors, A method for providing this.

2. The step of scoring the remaining potential trajectories in the potential trajectory based on the initial cost volume and the delta cost volume using one or more processors, The steps include: ranking the remaining potential trajectories based on their individual scores using one or more processors; The steps include: using one or more processors to select the top-ranked potential trajectory from the potential trajectories as a planned trajectory by using the individual scores; The method according to claim 1, further comprising:

3. A step of generating environmental data by rasterizing one or more top-down images of the vehicle and agents around the vehicle in the environment using one or more processors, The method according to claim 1, further comprising:

4. The data for the aforementioned environment is The distance between the vehicle and the agent, Lane boundaries associated with the aforementioned environment, The speed of the vehicle and the agent, The direction of travel of the vehicle and the agent, The yield relationship between the vehicle and the agent, or The location of the aforementioned vehicle and the aforementioned agent, The method according to claim 3, comprising information comprising one or more of the above.

5. The method according to claim 1, further comprising the step of training a machine learning model on training data that notifies the observed driving behavior using one or more processors, wherein the training data that notifies the observed driving behavior comprises captured sensor data comprising images, videos, LiDAR point clouds, radar signals, or any combination thereof.

6. The step of training the aforementioned machine learning model is: The steps include generating a trajectory using one or more processors, The steps include: determining the predicted delta cost volume based on the initial cost volume and the training data using one or more processors; The steps include: selecting a target trajectory from the trajectory based on the initial cost volume and the predicted delta cost volume using one or more processors; The steps include: comparing the target trajectory with a predetermined trajectory using one or more of the aforementioned processors; The steps include updating the machine learning model based on the comparison using one or more processors, The method according to claim 5, comprising:

7. The method according to claim 1, wherein the initial cost volume comprises initial cost measurement at locations along each of the associated potential trajectories with a timestamp.

8. The method according to claim 7, wherein the final cost volume comprises a final cost measurement at the location along each of the potential trajectories associated with the timestamp, and each final cost measurement comprises one of the adjustments to the initial cost measurement.

9. The method of claim 8, further comprising the step of determining by one or more processors that the delta cost volume ensures that each final cost measurement of the final cost volume exceeds a threshold measurement.

10. The method of claim 8, further comprising the step of performing the final cost volume as a three-dimensional lookup table having cells encoded by the final cost measurement using one or more processors.

11. The step of scoring the trajectory of the vehicle includes the step of determining the cost of the trajectory using one or more processors, in which case the step of determining the cost of the trajectory is The steps include: using one or more processors to identify candidate locations in the trajectory associated with candidate timestamps from the locations associated with candidate timestamps; The steps include: determining the final cost measurement corresponding to the candidate location associated with the candidate timestamp of the trajectory using one or more processors; The steps include: determining the cost of the trajectory based on the final cost measurement using one or more processors; The method according to claim 8, comprising:

12. A system comprising one or more processors and one or more computer-readable non-temporary storage media, wherein the one or more computer-readable non-temporary storage media comprises instructions that, when executed by the one or more processors, cause the system to perform an operation, and the operation is A step of determining the potential trajectory of the vehicle in the environment and the associated initial cost volume based on a set of vehicle movement restrictions, A step of generating a delta cost volume using the initial cost volume and environmental data of the environment, wherein the delta cost volume is generated by determining adjustments to the initial cost volume which incorporates the observed driving behavior. A step of scoring one of the vehicle's potential trajectories by adding the initial cost volume and the delta cost volume as the final cost volume, A system equipped with these features.

13. The one or more processors are capable of further operations when they have executed the instruction to perform an operation, and the operation is A step of scoring the remaining potential trajectories in the potential trajectory based on the initial cost volume and the delta cost volume, The steps include ranking the remaining potential trajectories based on their individual scores, The steps include selecting the top-ranked potential trajectory from the potential trajectories as the planned trajectory by using the individual scores mentioned above, The system according to claim 12, comprising:

14. The one or more processors are further capable of performing an operation when they have executed the instruction, and the operation is The system according to claim 12, further comprising the step of generating environmental data by rasterizing one or more top-down images of the vehicle and agents around the vehicle in the environment.

15. The data for the aforementioned environment is The distance between the vehicle and the agent, Lane boundaries associated with the aforementioned environment, The speed of the vehicle and the agent, The direction of travel of the vehicle and the agent, The yield relationship between the vehicle and the agent, or The location of the aforementioned vehicle and the aforementioned agent, The system according to claim 12, comprising information comprising one or more of the following.

16. A computer-readable, non-temporary storage medium containing software that is operable to generate an action when executed, wherein the action is: A step of determining the potential trajectory of a vehicle in the environment and the associated initial cost volume based on a set of vehicle movement restrictions, A step of generating a delta cost volume using the initial cost volume and environmental data of the environment, wherein the delta cost volume is generated by determining adjustments to the initial cost volume which incorporates the observed driving behavior. A step of scoring the trajectory of the potential trajectory by adding the initial cost volume and the delta cost volume as the final cost volume, A computer-readable, non-temporary storage medium equipped with [specific features / features].

17. The software is further operable to generate actions when executed, and these actions are A step of scoring the remaining potential trajectories in the potential trajectory based on the initial cost volume and the delta cost volume, The steps include ranking the remaining potential trajectories based on their individual scores, The steps include selecting the top-ranked potential trajectory from the potential trajectories as the planned trajectory by using the individual scores mentioned above, A computer-readable non-temporary storage medium according to claim 16, comprising:

18. The software is further operable to generate actions when executed, and these actions are The steps of generating the environmental data by rasterizing one or more top-down images of the vehicle and agents around the vehicle in the environment, A computer-readable non-temporary storage medium according to claim 16, comprising:

19. The data for the aforementioned environment is The distance between the vehicle and the agent, Lane boundaries associated with the aforementioned environment, The speed of the vehicle and the agent, The direction of travel of the vehicle and the agent, The yield relationship between the vehicle and the agent, or The location of the aforementioned vehicle and the aforementioned agent, A computer-readable non-temporary storage medium according to claim 16, comprising information comprising one or more of the above.

Images