Vehicle trajectory determination

Abstract

Techniques for determining a trajectory of a vehicle are described to operate the vehicle according to a planned path. In one example, a vehicle computing system may determine a position of the vehicle at a first time. Based on the position, the vehicle computing system may determine an estimated position of the vehicle at a second time, the estimated position of the vehicle including a lateral coordinate and a longitudinal coordinate. The vehicle computing system may determine the longitudinal coordinate based on a vehicle trajectory (e.g., a previously determined trajectory) associated with the first time, and may determine the lateral coordinate based on the planned path. The vehicle computing system may determine a second vehicle trajectory based in part on the estimated position and the first trajectory, and may control the vehicle according to the second vehicle trajectory.

Description

[Technical field]

[0001] The present disclosure relates to vehicle trajectory determination. [Background technology]

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This PCT international application claims priority to U.S. patent application Ser. No. 17 / 327,350, filed May 21, 2021, and entitled "VEHICLE TRAJECTORY DETERMINATION," the entire contents of which are incorporated herein by reference.

[0003]

[0002] Vehicles may be equipped with control systems for determining a trajectory to be followed by the vehicle, such as based on a planned path of the vehicle through an environment. These control systems often correct for discrepancies between the vehicle's planned path and the vehicle's physical position. For example, the control system may determine a trajectory to be followed by the vehicle by estimating a future state of the vehicle and merging a position- and velocity-based trajectory associated with the estimated future state. However, estimating a future state and merging a trajectory associated with the estimated future state may introduce noise or errors into the trajectory calculation, which may result in erratic or sporadic changes in the vehicle's direction. [Brief description of the drawings]

[0004]

[0003] The detailed description will now be made with reference to the accompanying drawings, in which the leftmost digit(s) of a reference number identifies the drawing in which the reference number first appears. Use of the same reference number in different drawings indicates similar or identical components or features. [Figure 1] FIG. 1 is a diagram illustrating a vehicle operating in an environment and equipped with a vehicle control system according to an example of the present disclosure. [Diagram 2]

[0005] FIG. 2 is a diagram illustrating a process for determining a vehicle trajectory according to an example of the disclosure. [Diagram 3]

[0006] FIG. 3 is a block diagram of an example system for implementing the techniques described herein. [Figure 4]

[0007] FIG. 4 illustrates an example process for determining a vehicle trajectory associated with a vehicle operating in an environment according to an example of the disclosure. [Diagram 5]

[0008] FIG. 5 illustrates an example process for determining a trajectory to be followed by a vehicle at a future time based on vehicle actions associated with the vehicle's operation in an environment, according to an example of the disclosure. [Figure 6]

[0009] FIG. 6 illustrates an example process for transmitting control signals associated with a vehicle trajectory based on actuation delays associated with corresponding vehicle components. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0005] Detailed Description

[0010] As described above, a vehicle control system may determine a trajectory for a vehicle to follow based on an estimated future state and a position- and speed-based trajectory associated with the estimated future state. However, noise and errors introduced by determining the estimated future state and its associated speed may result in irregular or sporadic changes in the vehicle's direction. As such, current implementations may be insufficient to provide a continuous signal to effectively track (e.g., follow) a planned trajectory while maintaining a smooth ride for the occupants. This application relates to techniques for improving vehicle control systems to provide and maintain a continuous trajectory for efficiently and effectively tracking a planned path.

[0006]

[0011] In some examples, the vehicle may be configured to travel a planned path in the environment. Such a path may be a geometric set of locations that the vehicle follows while traveling from a start location to a destination, or any portion thereof. The vehicle may include a control system configured to control the vehicle through the environment based in part on the planned path. In some examples, the control system may include a planning component. In various examples, the planning component may be configured to determine a planned trajectory for the vehicle to follow. In some examples, the planned trajectory may comprise one or more deviations from a predefined path associated with the traveling of the vehicle, such as deviations taken in response to objects (e.g., other vehicles, pedestrians, bicycles, etc.). In various examples, the planning component may be configured to determine and / or modify a trajectory that the vehicle follows when traveling according to the planned path. In various examples, the planning component may be configured to determine a trajectory that the vehicle follows at a predefined interval, such as every 0.1 seconds, every 0.05 seconds, etc. As used herein, the term planned trajectory may be used to describe a previously determined trajectory (e.g., a trajectory determined at a previous time interval). For example, the planning component may determine a new trajectory for the vehicle to follow at a given time interval based at least in part on a previous trajectory, and the planning component may pass the new trajectory to a tracking component of the control system at each time interval.

[0007]

[0012] In various examples, the tracking component may be configured to determine one or more control signals to send to the drive system to control the vehicle according to the new trajectory. In various examples, the control signals may include instructions to modify settings associated with one or more components of the drive system of the vehicle (e.g., motor, engine, transmission, steering components, braking components, etc.). As one non-limiting example, the tracking may inform one or more motor controllers of a particular current to apply to one or more wheels (and thus a desired acceleration or speed of the vehicle). In such an example, the tracking component may be configured to cause the vehicle to be controlled according to the new trajectory. As discussed herein, the vehicle control system may be associated with a vehicle computing system.

[0008]

[0013] In some examples, the vehicle computing system may determine a position of the vehicle, such as based on sensor data from one or more sensors on the vehicle and / or one or more remote sensors (e.g., sensors associated with other vehicles, sensors installed in the environment, etc.). The sensor data may include data associated with a current state of the vehicle, such as the speed, acceleration, position, and / or orientation of the vehicle. The position of the vehicle may include a current physical location of the vehicle operating within the environment, such as following a planned trajectory of the vehicle. In some examples, the vehicle computing system may determine whether the current location of the vehicle is within a threshold distance (e.g., 10 centimeters, 7 inches, etc.) laterally from the planned trajectory. In such examples, the threshold distance may represent a safety constraint that ensures the vehicle operates within predetermined safety parameters. In this manner, the techniques described herein may improve safe operation of the vehicle.

[0009]

[0014] Based on a determination that the current position is not within a threshold distance of the planned trajectory, the vehicle computing system may determine to stop further operation of the vehicle within the environment. In some examples, the vehicle computing system may determine a trajectory for the vehicle to follow such that the vehicle stops at a safe location. In such examples, the vehicle computing system may be configured to identify a safe location and navigate the vehicle through the environment to the safe location. In some examples, the vehicle computing system may be configured to connect to a remote operator, such as to receive control input from the remote operator.

[0010]

[0015] Based on the determination that the current position is within a threshold distance of the planned trajectory, the vehicle computing system may determine an estimated position of the vehicle at a future time. In such examples, the estimated position of the vehicle may include a projection of the vehicle's position at a future time (e.g., as if the vehicle perfectly followed the previously determined trajectory given the vehicle's current state estimate). In some examples, the time may be determined based on a predetermined rate for calculating the vehicle trajectory (e.g., every 10 milliseconds, every 50 milliseconds, etc.). Additionally or alternatively, in some examples, the future time associated with the estimated position may be determined based on a delay associated with a drive system component. The delay associated with the drive system component may include a delay (e.g., latency) in determining a control signal to send to the drive system component and / or a delay in actuating the drive system component based on the control signal. For example, a braking system of the vehicle may be associated with a first time delay between receiving a control signal and engaging the brakes based on the signal. The vehicle computing system may determine the future time associated with the estimated position based on the first time delay. In some examples, the delay associated with the drive system component may include an average delay associated with two or more drive system components. In such examples, the vehicle computing system may determine a future time associated with the estimated position based at least in part on the average delay associated with the operation of the two or more components of the drive system.

[0011]

[0016] The estimated position of the vehicle may include a longitudinal coordinate and a lateral coordinate. In various examples, the vehicle computing system may determine the longitudinal coordinate of the estimated position of the vehicle based at least in part on a previously determined trajectory. For example, the vehicle computing system may determine the longitudinal coordinate based on how far the vehicle is estimated to move during a time interval between a current time and a future time while traveling at one or more speeds associated with the previously determined trajectory. In various examples, the vehicle computing system may be configured to calculate the trajectory at a predetermined rate. In such examples, the estimated position of the vehicle may be determined based on a most recently (previously) determined trajectory calculated at the predetermined rate.

[0012]

[0017] The lateral coordinate of the estimated position of the vehicle may include the lateral position of the vehicle at a future time. In some examples, the vehicle computing system may determine the lateral coordinate of the estimated position based on a planned trajectory of the vehicle. In some examples, the lateral coordinate of the estimated position may be identical or substantially identical to a lateral coordinate associated with the planned trajectory at the future time (e.g., less than 2% difference, within 5 centimeters, etc.). In some examples, the lateral coordinate may be within a threshold lateral distance (e.g., within 10 centimeters, within 3 inches, etc.) from the planned trajectory of the vehicle. Thus, in at least one example, the vehicle computing system may be configured to constrain the lateral coordinate of the estimated position to be within a lateral range of the planned trajectory.

[0013]

[0018] In various examples, the vehicle computing system may determine a new vehicle trajectory based in part on an estimated position of the vehicle having a lateral coordinate constrained to the planned trajectory. In such examples, the new vehicle trajectory may be determined based on longitudinal information (e.g., speed, acceleration, etc.) rather than lateral information (e.g., position variation). In various examples, the longitudinal information associated with the vehicle trajectory determination may include one or more speeds associated with a vehicle action. The vehicle action may include an action determined by the vehicle computing system based on the conditions of the environment (e.g., road rules, detected objects, etc.). As one non-limiting example, the vehicle action may include maintaining a speed traveling through the environment, stopping at a stop sign, accelerating from a stopped position at an intersection, slowing down to yield to other vehicles, etc.

[0014]

[0019] In at least one example, an action may be determined based on objects detected in the environment to control the vehicle based on the objects. The objects may include pedestrians, bicycles, motorcycles, other vehicles, etc. In this manner, the vehicle computing system may be configured to determine an action based on a determination that the objects are related to the vehicle and / or based on predicted object trajectories associated therewith. The vehicle computing system is not limited to any of the U.S. patent applications which are incorporated herein by reference in their entirety, including but not limited to U.S. patent application Ser. No. 16 / 389,720, filed on April 19, 2019 and entitled “Dynamic Object Relevance Determination,” U.S. patent application Ser. No. 16 / 417,260, filed on May 20, 2019 and entitled “Object Relevance Determination,” U.S. patent application Ser. No. 15 / 807,521, filed on November 8, 2017 and entitled “Probabilistic Heat Maps for Behavior Prediction,” U.S. patent application Ser. No. 16 / 151,607, filed on October 4, 2018 and entitled “Trajectory Prediction on Top-Down Scenes Based on Action Data,” and U.S. patent application Ser. No. 16 / 151,607, filed on July 5, 2019 and entitled “Prediction on Top-Down Scenes Based on Action Data.” No. 16 / 504,147, entitled "Top-Down Scenes based on Action Data," the entire contents of each of which are incorporated herein by reference for all purposes, may be used to determine object relationships and predicted object trajectories.

[0015]

[0020] The speed associated with the vehicle action may represent one or more speeds associated with the vehicle performing the vehicle action. For example, the speed may include a speed associated with slowing down the vehicle to stop at a red light. However, this is merely exemplary and not intended to be limiting. In at least one example, the vehicle computing system may determine a vehicle trajectory associated with the estimated position based on a speed-based optimization of the vehicle movement. In some examples, the vehicle computing system may determine a vehicle trajectory associated with the estimated position of the vehicle utilizing techniques such as those described in U.S. Patent Application No. 16 / 805,118, filed February 28, 2020, entitled "System and Method for Adjusting a Planned Trajectory of an Autonomous Vehicle," the contents of which are incorporated herein by reference in their entirety for all purposes. In various examples, the vehicle computing system may control the vehicle according to the vehicle trajectory determined based on the estimated position and the speed-based optimization.

[0016]

[0021] The techniques discussed herein may improve the functionality of vehicle computing systems in many ways. As described above, the present trajectory determination systems include determining a vehicle trajectory by estimating a future vehicle position and determining a lateral- and longitudinal-based trajectory. These systems then integrate the lateral- and longitudinal-based trajectories into a single vehicle trajectory that the vehicle follows. However, by constraining the lateral coordinate of the vehicle's estimated position to its planned path, the techniques described herein can limit the trajectory determination to a velocity-based trajectory. In other words, the lateral constraint can remove the requirement to perform lateral optimization in the trajectory determination process and to integrate the lateral optimization with velocity-based optimization. In this manner, the techniques described herein reduce the total amount of computing resources required to determine a vehicle trajectory, thereby improving the vehicle computing system.

[0017]

[0022] Unlike conventional control systems, the control systems described herein may determine the trajectory the vehicle will follow based on an estimated position of the vehicle at a future time adjusted for drive system delays. By accounting for additional delays in the operation of the drive system, the techniques described herein may reduce errors introduced due to the operation of the drive system. In some examples, the reduction in errors may reduce the amount of computing resources required by the vehicle computing system to determine the vehicle trajectory. Additionally, accounting for the operation delays may enable the vehicle computing system to more effectively and efficiently maintain a continuous trajectory that tracks the planned path. Furthermore, by accounting for latency and delays in the operation of the control system, the techniques described herein may improve the safe operation of the vehicle.

[0018]

[0023] The techniques described herein may be implemented in many ways. Examples are provided below with reference to the following drawings. Although described in the context of an autonomous vehicle, the methods, apparatus, and systems described herein may be applied to a variety of systems (e.g., sensor systems or robotic platforms) and are not limited to autonomous vehicles. In one example, similar techniques may be utilized in driver-controlled vehicles where such systems may provide indications of whether it is safe to perform various maneuvers. In other examples, the techniques may be utilized in aviation or marine contexts, or in any system that uses planning techniques.

[0019]

[0024] FIG. 1 is a schematic diagram illustrating a vehicle 102 equipped with a control system in an exemplary environment 100 in which the vehicle 102 operates. In the illustrated example, the vehicle 102 is traveling through the environment 100, but in other examples, the vehicle 102 may be stationary (e.g., stopped at a stop sign, red light, etc.) and / or parked in the environment 100. In some examples, such as the example of FIG. 1, one or more objects 104 may additionally operate in the environment 100. For example, FIG. 1 shows an object 104 (e.g., a pedestrian) moving through a crosswalk 106. Although not shown, any number and / or type of objects may additionally or alternatively be present in the environment 100, including static objects, such as, for example, road signs, parked vehicles, fire hydrants, buildings, curbs, etc., and / or dynamic objects, such as, for example, pedestrians, animals, bicycles, trucks, motorcycles, other vehicles, etc.

[0020]

[0025] In various examples, the vehicle computing system 116 of the vehicle 102 may be configured to determine the object 104 in the environment 100 based on sensor data received from one or more sensors. The sensors may include cameras, motion detectors, lidar, radar, inertial sensors, etc. The sensors may be on-board the vehicle 102 and / or may be remote from the vehicle 102, such as on-board other vehicles and / or the environment 100. In examples where the sensors are remote sensors (e.g., on-board other vehicles, the environment 100), the vehicle computing system 116 may be configured to receive the sensor data over one or more networks. Additional details associated with the sensors are described below with respect to FIG. 3. In some examples, the vehicle computing system 116 may be configured to determine a position, orientation, and / or location information associated with the vehicle 102 based on the sensor data.

[0021]

[0026] In some examples, the vehicle 102 may be an autonomous vehicle configured to operate according to a Level 5 classification issued by the National Highway Traffic Safety Administration, which describes a vehicle capable of performing all safety-critical functions for the entire trip in situations where a driver (or passenger) is not expected to control the vehicle at any time. In such examples, the vehicle 102 may be configured to control all functions from start to stop, including all parking functions, and thus may be unmanned. In some examples, the vehicle 102 may include a semi-autonomous vehicle configured to perform at least a portion of the control functions associated with vehicle operation. Additional details associated with the vehicle 102 are described below.

[0022]

[0027] 1 illustrates a scenario in which a vehicle 102 is traveling in an environment 100 according to a planned path 108. The planned path 108 may include a general planned driving path for the vehicle 102 to travel from an initial location associated with the trip to a destination. In the illustrated example, the vehicle 102 is traveling in a first lane 110 of a road 112, the road including a first lane 110 associated with traffic traveling in a first direction and a second lane 114 associated with traffic traveling in a second (opposite) direction. This is merely an example, and the vehicle may be configured to operate at intersections, multi-lane roads, highways, etc.

[0023]

[0028] The vehicle 102 may include a vehicle computing system 116 configured to perform some or all of the functions described herein. The vehicle computing system 116 may include a planning component 118 configured to determine a planned path 108 and a vehicle trajectory 120 associated with the vehicle 102 operating according to the planned path 108. In various examples, the planning component 118 may be configured to determine the vehicle trajectory 120 at a predetermined rate (e.g., every 0.1 seconds, every 0.15 seconds, etc.). In such examples, the vehicle trajectory 120 may be determined at regular time intervals (ΔT). In some examples, the time intervals may be determined based on a time associated with the planning component 118 calculating a next vehicle trajectory. For example, the planning component 118 running a first vehicle trajectory 120(1) at a first time T1 may begin calculating a second vehicle trajectory 120(2) to be performed at a second time T2. The time interval ΔT1 between the first time and the second time T2 may be a fixed time interval determined to provide the planning component with sufficient time (e.g., calculation time + buffer) to determine the second vehicle trajectory 120(2) and enable its execution at the second time T2.

[0024]

[0029] In some examples, the time interval may be determined based on a delay time associated with initiating a correction to a drive system component associated with the next vehicle trajectory 120. In such examples, the delay time may include a predetermined time associated with a drive system component delay. The drive system components may include a motor, an engine, a transmission, a steering system component, a braking system component, etc. As described in more detail below, such delays or latencies may be aggregated or otherwise combined to determine a total delay between the trajectory determination and the final actuation command. In such examples, the determined overall delay or latency may vary from moment to moment based on which component (or combination of components) is actuated. In some examples, the drive system component delay may include a time associated with the tracking component 122 generating a control signal, the drive system component receiving the control signal, and / or the drive system component actuating the control signal and changing a setting associated with the drive system component. Continuing with the example, at a first time T1, the planning component 118 may begin calculating a second vehicle trajectory 120(2) associated with a second time T2. The second vehicle trajectory 120(2) may include a reduction in speed that requires activation of braking components of the drive system. The delay time may account for a delay in activating the braking components to slow the vehicle 102 as needed to follow the second vehicle trajectory 120(2).

[0025]

[0030] In some examples, the delay time associated with the drive system component can include a maximum delay time associated with the drive system component. In such examples, the delay time can include a delay associated with the drive system component having the longest delay associated therewith. In some examples, the delay time can include a minimum delay time associated with the drive system component. In such examples, the delay time can include a delay associated with the drive system component having the shortest delay. In some examples, the delay time can include an average delay time associated with the drive system component. In some examples, the delay time can include an average delay of two or more drive system components associated with a control signal based on the vehicle trajectory 120. In some examples, the delay time can include an average of a maximum delay time and a minimum delay time. For example, a delay associated with a motor that accelerates the vehicle can include a 50 millisecond delay time and a delay associated with a braking component can include a 20 millisecond delay time. The delay time associated with the drive system component can be 35 seconds. This is merely an example and other times and component delays are contemplated herein.

[0026]

[0031] In various examples, the planning component 118 may be configured to dynamically determine the time interval ΔT during vehicle operation. In some examples, the planning component 118 may dynamically determine the time interval ΔT based on a determined action for the vehicle to perform. In various examples, the planning component 118 may be configured to determine an action for the vehicle to perform with respect to the environment. In some examples, the planning component 118 may determine the action based on a cost-based action analysis. In such examples, the planning component 118 may determine the action utilizing techniques such as those described in U.S. Patent Application No. 17 / 202,795, filed February 24, 2021, entitled “Cost-Based Action Determination,” the contents of which are incorporated herein by reference in their entirety for all purposes. For example, the planning component 118 may detect an object 104 approaching the crosswalk 106 and may determine to yield to the object 104. Thus, the action includes decelerating the object 104 so that it can continue crossing the roadway 112 at the crosswalk 106. The planning component 118 may determine that the action includes a deceleration action that includes activation of a brake actuation system component. In this manner, the planning component dynamically determines the time interval ΔT based on a delay time associated with the brake actuation system.

[0027]

[0032] In at least one example, the time interval ΔT can include a delay associated with calculating the vehicle trajectory 120 and a delay associated with the operation of the drive system components. For example, the time intervals ΔT1 and ΔT2 can include a time associated with the vehicle trajectory calculation and a delay time associated with the brake system components, although this is merely an example and any other delay time associated with the drive system components is contemplated herein.

[0028]

[0033] As described above, the planning component 118 may be configured to determine an updated vehicle trajectory 120 for the vehicle 102 to travel through the environment 100 based on the time interval. The updated vehicle trajectory 120 may include a future trajectory associated with the vehicle at a future time. Thus, the planning component 118 may be configured to determine and provide a continuous trajectory for the vehicle to follow. For example, at a first time T1, the planning component 118 determines a second vehicle trajectory 120(2) for the vehicle to follow at a second (future) time T2, and at the second time T2, the planning component 118 determines a third vehicle trajectory 120(3) for the vehicle to follow at a third (future) time T3.

[0029]

[0034] In at least one example, the planning component 118 determines an updated vehicle trajectory 120 by determining an actual vehicle position 124 of the vehicle at a particular time and determining an estimated vehicle position 126 of the vehicle at a next time interval. In some examples, the planning component determines the estimated vehicle position 126 and / or the updated vehicle trajectory 120 based on a determination that the actual vehicle position 124 at a particular time is within a threshold distance 128 (e.g., 1 meter, 3 meters, 6 feet, etc.) of the planned trajectory. The planned trajectory may include previously determined vehicle trajectories 120 associated with previous time intervals. For example, at T2, the planning component 118 determines whether a second actual vehicle position 124(2) is within the threshold distance 128 of a first vehicle trajectory 120(1), at T3, the planning component 118 determines whether a third actual vehicle position 124(3) is within the threshold distance 128 of the second vehicle trajectory 120(2), and so on.

[0030]

[0035] In some examples, the planning component 118 may determine whether a distance between the actual vehicle position 124 and the planned trajectory meets or exceeds a threshold distance 128. In some examples, based on a determination that the actual vehicle position 124 is not within the threshold distance 128 of the planned path 108 (e.g., the distance meets or exceeds the threshold distance 128), the planning component 118 may determine to stop further operation of the vehicle 102 in the environment 100. In some examples, in response to determining to stop further operation, the planning component 118 may determine a trajectory for the vehicle 102 to stop in a safe position, such as parking to the side of the first lane 110. In some examples, the planning component 118 may be configured to call a remote operator based on a determination that the actual vehicle position 124 is away from the planned trajectory by more than the threshold distance 128. In such examples, the planning component 118 may receive a control signal from the remote operator, such as to ensure safe operation of the vehicle 102 through the environment 100.

[0031]

[0036] Based on a determination that the actual vehicle position is within a threshold distance 128 of the planned trajectory (e.g., the distance is less than the threshold distance 128), the planning component 118 may determine an estimated vehicle position 126 at a future time based in part on the time interval ΔT. For example, the planning component 118 determines a first estimated vehicle position 126(1) at a second time T2 based in part on the first actual vehicle position 124(1) at a first time T1 and the first time interval ΔT1. In various examples, the estimated vehicle position 126 may include a vertical coordinate (Y) and a horizontal coordinate (X). In various examples, the planning component 118 may determine the vertical coordinate of the estimated vehicle position 126 based in part on one or more speeds associated with the planned trajectory (e.g., a previously determined vehicle trajectory). For example, planning component 118 may determine a longitudinal coordinate based on how much vehicle 102 is estimated to move during a time interval ΔT between a current time and a future time while traveling at one or more speeds associated with a previously determined vehicle trajectory. For example, planning component 118 may determine a longitudinal coordinate associated with first estimated vehicle position 126(1) based on a longitudinal distance between first actual vehicle position 124(1), first vehicle trajectory 120(1), and a first time interval ΔT1.

[0032]

[0037] The lateral coordinate of the estimated vehicle position 126 may include a lateral position of the vehicle at a future time. In some examples, the planning component 118 may determine the lateral coordinate of the estimated vehicle position 126 based on a planned trajectory of the vehicle 102. In some examples, the lateral coordinate may represent an X-axis coordinate of the vehicle 102 associated with the complete tracking of the vehicle along a previously determined trajectory. In some examples, the lateral coordinate of the estimated vehicle position 126 may be identical or substantially identical (e.g., less than 2% difference, within 5 centimeters, etc.) to a lateral coordinate associated with the planned trajectory (e.g., the X-coordinate of the planned trajectory) at a future time point. In some examples, the lateral coordinate may be within a threshold lateral distance (e.g., within 10 centimeters, within 3 inches, etc.) from the planned trajectory. Thus, in at least one example, the planning component 118 may be configured to constrain the lateral coordinate of the estimated vehicle position 126 to within a lateral range of the planned trajectory.

[0033]

[0038] In various examples, the planning component 118 may determine a new or updated vehicle trajectory 120 based in part on an estimated vehicle position 126 with a lateral coordinate constrained to the planned trajectory. In such examples, the new vehicle trajectory 120 may be determined based on longitudinal information (e.g., speed, acceleration, etc.) rather than lateral information (e.g., position variation). In various examples, the longitudinal information associated with the determination of the new vehicle trajectory 120 may include one or more speeds associated with a determined vehicle action. As described above, the vehicle action may include an action determined by the vehicle computing system based on the conditions of the environment 100 (e.g., road rules, detected objects, etc.). As one non-limiting example, the vehicle action may include maintaining a speed to travel through the environment, stopping at a stop sign, accelerating from a stopped position at an intersection, slowing down to yield to another vehicle or other object, etc.

[0034]

[0039] In at least one example, the planning component 118 may determine an action, such as controlling the vehicle, based on the object 104 detected in the environment. As an illustrative, non-limiting example, the vehicle action may include the vehicle 102 yielding to an object 104 (e.g., a pedestrian) crossing the road 112, such as at a crosswalk 106. The object 104 may include a pedestrian, a bicycle, a motorcycle, another vehicle, etc. In some examples, the vehicle computing system may be configured to determine an action based on the object 104 based on a determination that the object 104 is related to the vehicle 102. In some examples, the determination of the relatedness of the object may be based on a predicted object trajectory 130 associated with the object. In such examples, the planning component 118 (e.g., a prediction component associated therewith) may be configured to determine the predicted object trajectory 130 and / or the relatedness of the object 104 associated therewith. In some examples, the planning component may determine object relationships using techniques such as those described in U.S. patent application Ser. Nos. 16 / 389,720 and / or 16 / 417,260, the contents of which are incorporated herein by reference for all purposes. In some examples, the planning component 118 may determine predicted object trajectories 130 using techniques such as those described in U.S. patent application Ser. Nos. 15 / 807,521, 16 / 151,607, and 16 / 504,147, the contents of which are incorporated herein by reference for all purposes.

[0035]

[0040] In various examples, the planning component 118 may determine one or more speeds associated with the new vehicle trajectory 120 based on the action. In some examples, the one or more speeds may be determined based on a previous vehicle trajectory 120 (e.g., a planned trajectory), such as one associated with a previous (consecutive) time interval. For example, the planning component 118 may begin determining a third vehicle trajectory 120(3) at a second time T2. The planning component 118 may determine a second estimated position 126(2) based on a second actual vehicle position 124(2) at the second time T2. The planning component 118 may determine an action including the vehicle 102 yielding to the object 104 and may determine that the vehicle should continue to reduce a forward speed associated with the second vehicle trajectory 120(2) to ensure that the vehicle 102 maintains a safe distance (e.g., 3 feet, 1 meter, 2 meters, etc.) from the crosswalk 106. Based on the second estimated vehicle position 126(2), the second vehicle trajectory 120(2), and the position of the crosswalk 106 (and / or the estimated future position of the object 104), the planning component 118 may determine a third vehicle trajectory 120(3) and / or one or more speeds associated therewith. In various examples, by constraining the estimated vehicle position 126 to the planned trajectory, and thus constraining the calculation of the vehicle trajectory 120 to longitudinal action-based movements (e.g., rather than lateral movements), the techniques described herein may improve the functionality of the vehicle computing system 116.

[0036]

[0041] In at least one example, the planning component 118 of the vehicle computing system 116 may determine a new vehicle trajectory 120 associated with an estimated vehicle position 126 based on a speed-based optimization of vehicle movement utilizing techniques such as those described in U.S. Patent Application No. 16 / 805,118, the contents of which are incorporated herein by reference for all purposes. In various examples, the planning component 118 may be configured to transmit the vehicle trajectory 120 to the tracking component 122. In various examples, the tracking component 122 may be configured to determine a position and / or orientation of the vehicle at a particular time associated with a particular vehicle trajectory, and to generate and transmit one or more control signals to one or more drive system components to control the vehicle according to the vehicle trajectory 120 received from the planning component 118. In this manner, the tracking component 122 may continuously monitor the current state of the vehicle 102 and determine control signals to ensure that the vehicle follows the vehicle trajectory 120 or maneuvers continuously back to the vehicle trajectory 120. For example, the tracking component 122 may receive the second vehicle trajectory 120(2) from the planning component 118, where the second vehicle trajectory 120(2) includes a deceleration action (e.g., one or more speeds associated with the vehicle 102 yielding to the pedestrian). In some examples, the tracking component 122 may generate and send a control signal to a braking system component of a vehicle drive system based on the second vehicle trajectory 120(2). The tracking component 122 may send the control signal to the braking system component such that the vehicle 102 is controlled according to the second vehicle trajectory at the second time. As another example, the tracking component 122 may determine a current position of the vehicle 102, such as a second actual vehicle position 124(2) at the second time, and may determine steering angles, motor and / or engine actions (e.g., speed up, maintain speed, slow down, etc.), braking actions, etc. to cause the vehicle 102 to follow the second vehicle trajectory 120(2) at time T2.

[0037]

[0042] In various examples, the tracking component 122 may receive the vehicle trajectory 120 prior to the time associated therewith. In some examples, the planning component 118 may transmit trajectory data to the tracking component 122 at a time interval prior to the time associated with the execution of the vehicle trajectory. In some examples, the time interval may be a time associated with a delay of a drive system component, as described above. In some examples, the tracking component 122 may be configured to transmit a signal at an appropriate time to activate one or more associated drive system components at a particular time corresponding to the vehicle trajectory 120. In such examples, the vehicle computing system 116 may be configured to modify a delay in the calculation and / or execution of the vehicle trajectory 120 to cause the vehicle 102 to more closely track the planned path 108, etc. For example, the planning component 118 may transmit the third vehicle trajectory 120(3) to the tracking component 122 at a time prior to a third time T3, which time includes a time delay associated with the braking system. The tracking component 122 may receive the third vehicle trajectory 120(3) and may generate a control signal based on the third vehicle trajectory 120(3) and a previous vehicle trajectory (e.g., the second vehicle trajectory 120(2)). The tracking component 122 may send a control signal to a braking component to control the vehicle according to the third vehicle trajectory 120(3) at a third time T3. By providing control signals to the drive system components at times that account for delays associated with the drive system components, the techniques described herein may enable the vehicle computing system 116 to more accurately and effectively control the vehicle 102 to maintain a continuous trajectory that tracks the planned path 108.

[0038]

[0043] FIG. 2 illustrates an example process 200 for determining a trajectory of a vehicle 102 .

[0039]

[0044] In operation 202, a vehicle computing system, such as vehicle computing system 116, determines a first position 204 of vehicle 102 traveling according to first vehicle trajectory 120(1) at a first time T1. In some examples, first position 204 may represent an actual vehicle position, such as first actual vehicle position 124(1). In some examples, vehicle computing system may determine first position 204 based on sensor data from one or more sensors. The sensor data may include data regarding a current state of vehicle 102, such as, for example, speed, acceleration, position, and / or direction of vehicle 102. In some examples, vehicle 102 may operate according to planned route 108. The planned route may be, for example, a general driving route associated with vehicle 102 traveling to a final destination.

[0040]

[0045] In operation 206, the vehicle computing system determines that the first position 204 is within a threshold distance 128 of a planned trajectory 207 of the vehicle 102. The planned trajectory 207 may include a previously determined vehicle trajectory, such as a vehicle trajectory associated with a time interval prior to T1. The threshold distance 128 may represent a distance (e.g., 3 feet, 1 meter, 2 meters, etc.) from the planned trajectory 207 that indicates that the vehicle 102 remains within a safe distance of the planned trajectory 207. For example, the threshold distance 128 may indicate that the vehicle 102 has not deviated from the planned trajectory 207. In some examples, the threshold distance 128 may represent a predetermined safety parameter associated with the operation of the vehicle 102. In such examples, by verifying that the first position 204 is within the threshold distance, the vehicle computing system may ensure safe operation of the vehicle 102.

[0041]

[0046] In operation 208, the vehicle computing system determines a second position 210 associated with the vehicle 102 at a second time after the first time based at least in part on the first position 204 and the first vehicle trajectory 120(1), the second position 210 having a lateral coordinate 212 and a longitudinal coordinate 214. In various examples, the second position 210 may be an estimated vehicle position, such as the first estimated vehicle position 126(1), associated with the second time. In some examples, the vehicle computing system may project the first position 204 onto the planned trajectory 207 to determine the second position 210. In such examples, the vehicle computing system may modify the lateral coordinate 212 of the first position 204 to be identical or substantially identical to a lateral component of the planned trajectory 207. The vehicle computing system may then determine the second position 210 based on the first position 204 projected onto the planned trajectory 207, such as by estimating a distance the vehicle will travel based on the first vehicle trajectory 120(1). In other words, the vehicle computing system may estimate a position of the vehicle 102 at a future time based on the movement of the vehicle 102 according to the planned trajectory 207.

[0042]

[0047] As described above, the second position 210 may include a lateral coordinate 212 and a vertical coordinate 214. The lateral coordinate of the second position 210 includes a lateral position of the vehicle 102 at a second (future) time. In some examples, the vehicle computing system determines the lateral coordinate 212 of the second position 210 based on the planned trajectory 207. In some examples, the lateral coordinate 212 of the second position 210 may be identical or substantially identical (e.g., less than 2% difference, within 5 centimeters, etc.) to a lateral coordinate associated with the planned trajectory 207 (e.g., the X coordinate of the planned trajectory 207) at the future time. In some examples, the lateral coordinate 212 may be within a threshold lateral distance (e.g., within 10 centimeters, within 3 inches, etc.) from the planned trajectory 207. Thus, in at least one example, the vehicle computing system may be configured to constrain the lateral coordinate 212 of the second position 210 to a lateral range of the planned trajectory 207.

[0043]

[0048] In various examples, the vehicle computing system may determine the longitudinal coordinate 214 of the second position 210 based on one or more velocities associated with a current vehicle trajectory, such as the first vehicle trajectory 120(1) at the first time T1. In some examples, the vehicle computing system may be configured to determine the vehicle trajectory 120 at a rate (e.g., every 100 milliseconds, every 126 milliseconds, etc.), for example, to provide a continuous trajectory and ensure a smooth ride for the vehicle occupants. In such examples, the vehicle computing system determines the longitudinal coordinate 214 based on the current trajectory associated with the vehicle while determining an updated trajectory associated with the second future time.

[0044]

[0049] In various examples, the second time may be a time interval after the first time. In some examples, the time interval may be based on a time associated with a calculation of a trajectory of the vehicle. Additionally, in some examples, the time interval may be determined based on one or more time delays associated with a vehicle drive component, such as generating a control signal and causing the vehicle drive component to modify one or more settings based on the control signal. In some examples, the time interval may be associated with a predetermined rate (e.g., 100 milliseconds, 150 milliseconds, etc.). As described above, in some examples, the vehicle computing system may be configured to dynamically determine the time interval, for example, based on the determined vehicle action 216. In such examples, the rate and the time interval associated therewith may be dynamically determined during operation of the vehicle.

[0045]

[0050] In operation 218, the vehicle computing system determines an action 216 associated with the operation of the vehicle 102. As described above, the vehicle computing system may determine the action 216 based on conditions in the environment 100 (e.g., road rules, detected objects, etc.). As one non-limiting example, vehicle actions may include maintaining speed to navigate the environment, slowing down to stop at a stop sign, accelerating from a stopped position, slowing down to yield to other vehicles or other objects, etc.

[0046]

[0051] In at least one example, the vehicle computing system determines an action based on a detected object, such as object 104, in the environment. For example, the vehicle computing system may determine to accelerate to get ahead of the detected object in a merging scenario. As another example, the vehicle computing system may determine to decelerate to give way to the object. As mentioned above, the vehicle computing system may determine an action 216 based on a determination that the detected object is related to the vehicle 102 using techniques such as those described in U.S. Patent Application Nos. 16 / 389,720 and / or 16 / 417,260, the contents of which are incorporated herein by reference for any purpose. In some examples, the vehicle computing system may determine an object relationship and / or action 216 based on a predicted object trajectory associated with the detected object. In such examples, the vehicle computing system may be configured to determine a predicted object trajectory using techniques such as those described in U.S. Patent Application Nos. 15 / 807,521, 16 / 151,607, and 16 / 504,147, the contents of which are incorporated herein by reference for any purpose.

[0047]

[0052] In operation 220, the vehicle computing system determines a second vehicle trajectory 120(2) associated with the second time based at least in part on the action 216 and the second position 210. The second vehicle trajectory 120(2) may include one or more speeds and / or headings associated with the operation of the vehicle at the second time. In various examples, the heading of the second vehicle trajectory 120(2) may correspond to the planned path 108. In such examples, the vehicle computing system may determine a trajectory to keep the vehicle 102 on or substantially on the planned path 108.

[0048]

[0053] In various examples, a speed associated with the second vehicle trajectory 120(2) may be determined based in part on the first vehicle trajectory 120(1) and the action 216. For example, the vehicle computing system may determine that the action 216 includes slowing down and stopping at a stop sign. The vehicle computing system may determine a distance from the second location 210 to a stop location associated with the stop sign and determine a deceleration rate associated with smoothly controlling the vehicle 102 to the stop location. The vehicle computing system may determine one or more speeds associated with the second location 210 based on the deceleration rate.

[0049]

[0054] In operation 222, the vehicle computing system controls the vehicle at the second time based at least in part on the second vehicle trajectory 120(2). In various examples, the vehicle computing system may generate and provide control signals to the drive system components to operate the vehicle 102 according to the second vehicle trajectory 120(2). In some examples, the vehicle computing system may transmit the control signals at the second time. In some examples, the vehicle computing system may be configured to transmit the control signals prior to the second time based on, for example, a time delay associated with the drive system components. In such examples, the vehicle computing system may be configured to run the vehicle according to the second vehicle trajectory 120(2) at the second time to, for example, prevent errors associated with the control signals and the operation of the drive system.

[0050]

[0055] 3 is a block diagram of an example system 300 for implementing the techniques described herein. In at least one example, the system 300 may include a vehicle 302, such as the vehicle 102.

[0051]

[0056] The vehicle 302 may include one or more vehicle computing devices 304, such as the vehicle computing systems described herein, one or more sensor systems 306, one or more emitters 308, one or more communication connections 310, at least one direct connection 312, and one or more drive systems 314.

[0052]

[0057] The vehicle computing device 304 may include one or more processors 316 and a memory 318 communicatively coupled to the one or more processors 316, although the vehicle 302 may be any other type of vehicle, such as a semi-autonomous vehicle, or any other system having at least an image capture device (e.g., a camera-enabled smartphone). In the illustrated example, the memory 318 of the vehicle computing device 304 stores a localization component 320, a perception component 322, a planning component 324, a tracking component 326, one or more system controllers 328, and one or more maps 330. While shown in FIG. 3 as residing in memory 318 for illustrative purposes, the localization component 320, the perception component 322, the planning component 324, the tracking component 326, the one or more system controllers 328, and the one or more maps 330 may additionally or alternatively be accessible to the vehicle 302 (e.g., stored in memory remote from the vehicle 302, e.g., memory 332 of a remote computing device 334, or otherwise accessible by memory remote from the vehicle 302).

[0053]

[0058] In at least one example, the localization component 320 may include functionality to receive data from the sensor system 306 to determine a position and / or orientation (e.g., x, y, z position, roll, pitch, or yaw) of the vehicle 302. For example, the localization component 320 may include and / or request / receive a map of the environment and continuously determine a position and / or orientation of the autonomous vehicle within the map. In some cases, the localization component 320 may receive image data using simultaneous localization and mapping (SLAM), calibration, localization and mapping, simultaneously (CLAMS), relative SLAM, bundle adjustment, nonlinear least squares optimization, etc., and may use LIDAR data, radar data, IMU data, GPS data, wheel encoder data, etc. to precisely determine a position of the autonomous vehicle. In some cases, the localization component 320 may provide data to various components of the vehicle 302 to determine an initial position of the autonomous vehicle for generating a path polygon (e.g., a vehicle path) associated with a vehicle path, as described herein.

[0054]

[0059] In some cases, the perception component 322 may include functionality to perform object detection, segmentation, and / or classification. In some examples, the perception component 322 may provide processed sensor data indicating the presence of an object (e.g., an entity) proximate to the vehicle 302 and / or the classification of the object as an object type (e.g., car, pedestrian, bicycle, animal, building, tree, road surface, curb, sidewalk, unknown, etc.). In some examples, the perception component 322 may provide processed sensor data indicating the presence of a stationary entity proximate to the vehicle 302 and / or the classification of the stationary entity as a type (e.g., building, tree, road surface, curb, sidewalk, unknown, etc.). In additional or alternative examples, the perception component 322 may provide processed sensor data indicating one or more characteristics associated with a detected object (e.g., a tracked object) and / or an environment in which the object is located. In some examples, characteristics associated with an object may include, but are not limited to, x-position (global and / or local position), y-position (global and / or local position), z-position (global and / or local position), orientation (e.g., roll, pitch, yaw), object type (e.g., classification), object velocity (e.g., object speed), object acceleration, object range (e.g., size), etc. Characteristics associated with an environment may include, but are not limited to, the presence of other objects in the environment, the state of other objects in the environment, time of day, day of the week, season, weather conditions, indications of darkness / light, etc.

[0055]

[0060] In general, the planning component 324 may determine a path for the vehicle 302 to follow to travel through an environment. For example, the planning component 324 may determine various paths and trajectories as well as various levels of detail. For example, the planning component 324 may determine a path to travel from a first location (e.g., a current location) to a second location (e.g., a target location). For purposes of this description, the path may include a sequence of waypoints to travel between the two locations. As a non-limiting example, the waypoints may include streets, intersections, Global Positioning System (GPS) coordinates, and the like. Additionally, the planning component 324 may generate instructions to guide the autonomous vehicle 302 from the first location to the second location along at least a portion of the path. In at least one example, the planning component 324 may determine how to guide the autonomous vehicle from a first waypoint in the sequence of waypoints to a second waypoint in the sequence of waypoints. In some examples, the instructions may be a trajectory or a portion of a trajectory. In some examples, multiple trajectories may be generated substantially simultaneously (e.g., within technical tolerances) according to a receding horizon technique, and one of the multiple trajectories is selected for navigation of vehicle 302.

[0056]

[0061] In some examples, the planning component 324 may include a prediction component that generates predicted trajectories associated with objects operating in the environment. For example, the prediction component may generate one or more predicted trajectories for objects within a threshold distance from the vehicle 302. In some examples, the prediction component may measure the trajectories of the objects and generate the trajectories of the objects based on the observed and predicted behavior. In various examples, the planning component 324 may be configured to determine an action to be taken by the vehicle based at least in part on the predicted trajectories of the objects in the environment. In such examples, the planning component 324 may select a vehicle trajectory for the vehicle to travel based at least in part on the action (e.g., based in part on the detected object and / or the predicted object trajectory associated therewith).

[0057]

[0062] In various examples, the planning component 324 may provide the selected vehicle trajectory to the tracking component 326. In various examples, the tracking component 326 may additionally receive position and / or orientation data as determined by the localization component 320. The tracking component 326, such as the tracking component 122, may be configured to determine a position and / or orientation relative to the planned trajectory based on, for example, a steering angle, a speed, an acceleration, a drive direction, a drive gear, and / or gravitational acceleration. The tracking component 326 may be configured to determine control signals that cause the vehicle to adjust one or more drive components, for example, to track the determined trajectory. The tracking component 326 may determine adjustments based on the current position and / or orientation data, for example, to cause the vehicle to accurately track the vehicle trajectory or to maneuver back to the vehicle trajectory.

[0058]

[0063] In at least one example, the vehicle computing device 304 may include one or more system controllers 328 that may be configured to control steering, propulsion, braking, safety, emitter, communication, and other systems of the vehicle 302. The system controllers 328 may communicate with and / or control corresponding systems of the drive system 314 and / or other components of the vehicle 302.

[0059]

[0064] The memory 318 may further include one or more maps 330 that may be used by the vehicle 302 to navigate within the environment. For purposes of this description, a map may be any number of data structures modeled in two, three, or N dimensions and may provide information about the environment, such as, but not limited to, topology (such as intersections), streets, mountain ranges, roads, terrain, and the environment in general. In some cases, the map may include texture information (e.g., color information (e.g., RGB color information, Lab color information, HSV / HSL color information), etc.), intensity information (e.g., LIDAR information, RADAR information, etc.), spatial information (e.g., image data projected onto a mesh, individual "surfels" (e.g., polygons associated with individual colors and / or intensities), reflectance information (e.g., specularity information, retroreflectivity information, BRDF information, BSSRDF information, etc.). In one example, the map may include a three-dimensional mesh of the environment. In some examples, the vehicle 302 may be controlled based at least in part on the map 330. That is, the map 330 may be used in conjunction with the localization component 320, the perception component 322, and / or the planning component 324 to determine the position of the vehicle 302, detect objects in the environment, and / or generate paths and / or trajectories for navigating within the environment.

[0060]

[0065] In various examples, the map 330 may be utilized by the vehicle computing device 304 to determine right-of-way, such as at an intersection. The right-of-way may indicate an entity (e.g., the vehicle 302 or an object) that has the right of way at an intersection or other junction. In various examples, the map 330 may indicate the right-of-way based on the vehicle's position, direction of travel, the object's position, the object's direction of travel, the object's projected trajectory, etc.

[0061]

[0066] In some examples, one or more maps 330 may be stored on a remote computing device (such as computing device 334) accessible via network 336, such as map component 338. In some examples, multiple maps 330 may be stored, for example, based on characteristics (e.g., type of entity, time of day, day of the week, season, etc.). Storing multiple maps 330 may require similar memory requirements, but increases the speed at which data in the maps may be accessed.

[0062]

[0067] As can be appreciated, the components described herein (e.g., the localization component 320, the perception component 322, the planning component 324, the tracking component 326, the one or more system controllers 328, and the one or more maps 330) are described separately for purposes of explanation, however, operations performed by the various components may be combined or performed in any other component.

[0063]

[0068] In some cases, some or all aspects of the components described herein may include any models, techniques, and / or machine learning techniques. For example, in some cases, the components in memory 318 (and memory 332, described below) may be implemented as neural networks.

[0064]

[0069] As described herein, an exemplary neural network is a biologically inspired algorithm that passes input data through a series of connected layers to generate an output. Each layer in a neural network may contain other neural networks and may contain any number of layers (convolutional or not). As can be understood in the context of this disclosure, a neural network may refer to a broad class of algorithms that may utilize machine learning and in which an output is generated based on learned parameters.

[0065]

[0070] Although discussed in the context of neural networks, any type of machine learning may be used consistent with this disclosure. For example, machine learning techniques may include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), local estimation scatter plot smoothing (LOESS)), instance-based techniques (e.g., ridge regression, least absolute shrinkage and selection operator (LASSO), elastic nets, least angle regression (LARS)), decision tree techniques (e.g., classification and regression trees (CART), iterative dichotomiser (ID3), etc.), and may be implemented in a variety of ways. 3), Chi-squared Automatic Correlation Detection (CHAID), Decision Stamps, Conditional Decision Trees), Bayesian techniques (e.g., Naïve Bayes, Gaussian Naïve Bayes, Multinomial Naïve Bayes, Average-of-One Dependent Estimator (AODE), Bayesian Belief Network (BNN), Bayesian Network), Clustering techniques (e.g., k-means, k-median, Expectation Maximization (EM), Hierarchical Clustering), Association Rule Learning Algorithms (e.g., Perceptron, Backpropagation, Hopfield Network, Radial Basis Function Network (RBFN)), Deep Learning techniques (e.g., Deep Boltzmann Machine (DBM), Deep Belief Network (DBN), Convolutional Neural Network (CN N), stacked autoencoders), dimensionality reduction techniques (e.g., principal component analysis (PCA), principal component regression (PCR), partial least squares regression (PLSR), Sammon mapping, multidimensional scaling (MDS), projection pursuit, linear discriminant analysis (LDA), mixed discriminant analysis (MDA), quadratic discriminant analysis (QDA), flexible discriminant analysis (FDA)), ensemble techniques (e.g., boosting, bootstrap aggregation (bagging), AdaBoost, stacked generalization (blending), gradient boosting machine (GBM), gradient boosting regression tree (GBRT), random forest), SVM (support vector machine), supervised learning, unsupervised learning, semi-supervised learning, etc.

[0066]

[0071] In at least one example, the sensor system 306 may include lidar sensors, radar sensors, ultrasonic transducers, sonar sensors, position sensors (e.g., GPS, compass, etc.), inertial sensors (e.g., inertial measurement units (IMUs), accelerometers, magnetometers, gyroscopes, etc.), cameras (e.g., RGB, IR, intensity, depth, time of flight, etc.), microphones, wheel encoders, environmental sensors (e.g., temperature sensors, humidity sensors, light sensors, pressure sensors, etc.), etc. The sensor system 306 may include multiple instances of each of these or other types of sensors. For example, a LIDAR sensor may include individual LIDAR sensors located at the corners, front, rear, sides, and / or top of the vehicle 302. As another example, a camera sensor may include multiple cameras located at various locations about the exterior and interior of the vehicle 302. The sensor system 306 may provide input to the vehicle computing device 304. Additionally or alternatively, the sensor system 306 may transmit sensor data via one or more networks 336 to one or more computing devices 334 at a particular frequency, after a predetermined period of time, in near real-time, etc.

[0067]

[0072] The vehicle 302 may also include one or more emitters 308 that emit light and / or sound. The emitters 308 in this example include in-vehicle audio and visual emitters for communicating with occupants of the vehicle 302. By way of example and not limitation, the in-vehicle emitters may include speakers, lights, signs, display screens, touch screens, haptic emitters (e.g., vibration and / or force feedback), mechanical actuators (e.g., seat belt tensioners, seat positioners, head rest positioners, etc.), and the like. The one or more emitters 308 in this example also include exterior emitters. By way of example and not limitation, the exterior emitters in this example may include lights (e.g., indicator lights, signs, light arrays, etc.) that indicate a direction of travel or other indication of vehicle actions, and one or more audio emitters (e.g., speakers, speaker arrays, horns, etc.) for audibly communicating with pedestrians or other nearby vehicles, one or more of which include acoustic beam steering technology.

[0068]

[0073] Vehicle 302 may also include one or more communication connections 310 that enable communication between vehicle 302 and one or more other local or remote computing devices. For example, communication connections 310 may facilitate communication with other local computing devices in vehicle 302 and / or drive system 314. Communication connections 310 may also allow the vehicle to communicate with other nearby computing devices (e.g., computing device 334, other nearby vehicles, etc.) and / or one or more remote sensor systems 340 to receive sensor data.

[0069]

[0074] The communication connection 310 may include physical and / or logical interfaces for connecting the vehicle computing device 304 to other computing devices or networks, such as the network 336. For example, the communication connection 310 may enable Wi-Fi®-based communications, such as over frequencies defined by the IEEE 802.11 standard, short-range wireless frequencies such as Bluetooth®, cellular communications (e.g., 2G, 3G, 4G, 4G LTE, 3G, etc.), or any suitable wired or wireless communication protocol that enables individual computing devices to interface with other computing devices.

[0070]

[0075] In at least one example, the vehicle 302 may include one or more drive systems 314. In some examples, the vehicle 302 may have a single drive system 314. In at least one example, if the vehicle 302 has multiple drive systems, the individual drive systems 314 may be located at opposite ends of the vehicle 302 (e.g., front and rear, etc.). In at least one example, the drive system 314 may include one or more sensor systems that detect conditions surrounding the drive system 314 and / or the vehicle 302. By way of example and not limitation, the sensor systems may include one or more wheel encoders (e.g., rotary encoders) that sense the rotation of the wheels of the drive module, inertial sensors (e.g., inertial measurement units, accelerometers, gyroscopes, magnetometers, etc.) that measure the orientation and acceleration of the drive module, cameras or other image sensors, ultrasonic sensors that acoustically detect objects in the vicinity of the drive module, LIDAR sensors, radar sensors, etc. Some sensors, such as wheel encoders, may be specific to the drive system. In some cases, the sensor systems in the drive system 314 may overlap or complement corresponding systems in the vehicle 302 (e.g., the sensor system 306).

[0071]

[0076] The drive system 314 may include many vehicle systems, including a high voltage battery, a motor for propelling the vehicle, an inverter for converting direct current from the battery to alternating current used by other vehicle systems, a steering system (which may be electric) including a steering motor and a steering rack, a braking system including hydraulic or electric actuators, a suspension system including hydraulic and / or pneumatic components, a stability control system for distributing braking force to mitigate loss of traction and maintain control, an HVAC system, lighting (e.g., lighting such as head / tail lights for illuminating the exterior surroundings of the vehicle), and one or more other systems (e.g., a cooling system, a safety system, an on-board charging system, other electrical components such as DC / DC converters, a high voltage junction, a high voltage cable, a charging system, a charging port, etc.). The non-limiting examples of vehicle systems listed above may be referred to herein, additionally or alternatively, as "components" of the drive system 314. In various examples, each of the components of the drive system 314 may include latency associated with processing control signals. In various examples, the vehicle computing device 304 may be configured to determine an updated vehicle trajectory and / or transmit control signals based on the latency of one or more components. For example, the planning component 324 may be configured to determine an updated trajectory at time intervals based in part on the latency of the components. As another example, the tracking component 326 may be configured to transmit signals to drive system components based in part on the associated latency.

[0072]

[0077] Additionally, drive system 314 may include a drive module controller that may receive and preprocess data from the sensor systems to control operation of various vehicle systems. In some examples, the drive module controller may include one or more processors and a memory communicatively coupled to the one or more processors. The memory may store one or more modules that perform various functions of drive system 314. Furthermore, drive system 314 may also include one or more communication connections that enable individual drive modules to communicate with one or more other local or remote computing devices.

[0073]

[0078] In at least one example, the direct connection 312 may provide a physical interface coupling one or more drive systems 314 to the body of the vehicle 302. For example, the direct connection 312 may allow for the transfer of energy, fluid, air, data, etc. between the drive systems 314 and the vehicle. In some examples, the direct connection 312 may also removably secure the drive systems 314 to the body of the vehicle 302.

[0074]

[0079] In at least one example, the localization component 320, the perception component 322, the planning component 324, the tracking component 326, the one or more system controllers 328, and the one or more maps 330, and their various components, may process the sensor data as described above and transmit their respective outputs over one or more networks 336 to the computing device 334. In at least one example, the localization component 320, the perception component 322, the planning component 324, the tracking component 326, the one or more system controllers 328, and the one or more maps 330 may transmit their respective outputs to the computing device 334 at a particular frequency, after a predetermined time has elapsed, in near real-time, etc.

[0075]

[0080] In some examples, the vehicle 302 may transmit sensor data to the computing device 334 over the network 336. In some examples, the vehicle 302 may receive sensor data from the computing device 334 over the network 336. The sensor data may include raw sensor data and / or processed sensor data and / or representations of the sensor data. In some examples, the sensor data (raw or processed) may be transmitted and / or received as one or more log files.

[0076]

[0081] The computing device 334 may include a processor 342 and a memory 332 that stores a map component 338 and a sensor data processing component 344. In some examples, the map component 338 may include functionality for generating maps of various resolutions. In such examples, the map component 338 may transmit one or more maps to the vehicle computing device 304 for navigation purposes. In various examples, the sensor data processing component 344 may be configured to receive data from one or more remote sensors, such as the sensor system 306 and / or the remote sensor system 340. In some examples, the sensor data processing component 344 may be configured to transmit raw sensor data to the vehicle computing device 304.

[0077]

[0082] The processor 316 of the vehicle 302 and the processor 342 of the computing device 334 may be any suitable processor capable of processing data and executing instructions to process data and perform operations as described herein. By way of example and not limitation, the processors 316 and 342 may include one or more central processing units (CPUs), graphics processing units (GPUs), or any other device or portion of a device that processes electronic data and converts the electronic data into other electronic data that can be stored in registers and / or memory. In some examples, integrated circuits (e.g., ASICs, etc.), gate arrays (e.g., FPGAs, etc.), and other hardware devices may also be considered processors so long as they are configured to implement encoded instructions.

[0078]

[0083] The memories 318 and 332 are examples of non-transitory computer-readable media. The memories 318 and 332 may store an operating system and one or more software applications, instructions, programs, and / or data for implementing the functions resulting from the methods and various systems described herein. In various embodiments, the memories may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), non-volatile / flash memory, or any other type of memory capable of storing information. The architectures, systems, and individual elements described herein may include many other logical, programmatic, and physical components, and those shown in the accompanying drawings are merely examples for the purposes of the description herein.

[0079]

[0084] In some examples, the memories 318 and 332 may include at least a working memory and a storage memory. For example, the working memory may be a high-speed memory with limited capacity (e.g., cache memory) used to store data manipulated by the processors 316 and 342. In some examples, the memories 318 and 332 may include a storage memory, which may be a relatively large, slow memory used for long-term storage of data. In some examples, the processors 316 and 342 may not directly manipulate data stored in the storage memory and may need to load the data into the working memory to perform operations based on the data, as described herein.

[0080]

[0085] 3 as a distributed system, in alternative examples, components of the vehicle 302 may be associated with the computing device 334 and / or components of the computing device 334 may be associated with the vehicle 302. That is, the vehicle 302 may perform one or more of the functions associated with the computing device 334, and vice versa.

[0081]

[0086] 4-6 illustrate example processes according to examples of the present disclosure. These processes are illustrated as logical flow graphs, in which each operation represents a sequence of operations that may be implemented in hardware, software, or a combination thereof. In the context of software, the operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more processors, perform the described operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform a particular function or implement a particular abstract data type language. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations may be combined in any order and / or in parallel to implement the process.

[0082]

[0087] 4 illustrates an example process 400 for determining a vehicle trajectory associated with operation of a vehicle in an environment, such as environment 100. Some or all of process 400 may be performed by one or more components in FIG. 3, as described herein. For example, some or all of process 400 may be performed by vehicle computing device 304.

[0083]

[0088] At operation 402, process 400 includes determining a first position at a first time of the vehicle in the environment, the vehicle operating according to a planned trajectory. The planned trajectory may include a previously determined trajectory of the vehicle operating in the environment, such as in a previous time interval. The vehicle computing system may determine the first position based on sensor data received from one or more sensors. The sensor data may indicate the position and / or movement of the vehicle in the environment. The sensors may include cameras, motion detectors, lidar, radar, time of flight, etc. The sensors may be onboard the vehicle and / or may include sensors remote from the vehicle (e.g., onboard another vehicle, onboard the environment, etc.).

[0084]

[0089] In various examples, the vehicle may operate according to a first trajectory (e.g., a first vehicle trajectory). The first trajectory may include a heading and one or more speeds. In various examples, the vehicle computing system determines the first trajectory based on an action associated with the operation of the vehicle in the environment. For example, the first trajectory may be associated with the vehicle slowing down to give way to an object located near the vehicle in the environment.

[0085]

[0090] In various examples, the vehicle computing system may determine whether the first position is within a threshold distance of the planned trajectory of the vehicle. The threshold distance may represent a safety parameter associated with the operation of the vehicle. Based on the determination that the first position is equal to or greater than the threshold distance, the vehicle computing system may determine to stop the operation of the vehicle, for example, to ensure safe operation of the vehicle. In some examples, the vehicle computing system may determine a safe location for the vehicle to move to (e.g., a parking position, etc.) and control the vehicle to the safe location. In some examples, the vehicle computing system may connect to a remote operator and receive control input from the remote operator to ensure safe operation of the vehicle. Based on the determination that the first position is equal to or less than the threshold distance, the vehicle computing system may determine to continue operation in the environment.

[0086]

[0091] At operation 404, the process 400 includes determining a second position associated with the vehicle at a second time after the first time, the second position including a lateral coordinate and a longitudinal coordinate associated with the planned trajectory. In some examples, the second position may represent an estimated future position of the vehicle at the second time (e.g., in the future). In some examples, the vehicle computing system may project the first position onto the planned trajectory to determine the second position. In such examples, the vehicle computing system may modify the lateral coordinate of the first position to be the same or substantially the same as the planned trajectory. The vehicle computing system may then determine the second position based on the first position projected onto the planned trajectory, such as by estimating a distance traveled by the vehicle based on the first trajectory (e.g., a speed associated with the first trajectory). In other words, the vehicle computing system may estimate a position of the vehicle at a future time (e.g., a second time) based on the movement of the vehicle along the planned trajectory. In various examples, the vehicle computing system may determine the longitudinal coordinate based on the distance and / or the first trajectory.

[0087]

[0092] At operation 406, process 400 includes determining a vehicle trajectory associated with the vehicle operating at the second time based at least in part on the second position and the state associated with the vehicle operating at the first time. The state of the vehicle operating at the first time may include a position, a speed, a steering angle, a turning rate, a heading, and / or other aspects of the vehicle state associated with the first time. The vehicle trajectory may include, for example, a heading and one or more speeds that the vehicle follows to track a planned path as the vehicle travels through an environment. For example, to maintain safe vehicle operation and a continuous path, the vehicle trajectory may take into account unforeseen conflicts in the environment.

[0088]

[0093] In various examples, the vehicle computing system may be configured to control the vehicle based at least in part on the vehicle trajectory. A tracking component of the vehicle computing system may receive the vehicle trajectory, such as from the planning component. The tracking component may determine an actual position of the vehicle at a second time and may determine one or more drive system components associated with operating the vehicle according to the second trajectory. The tracking component may operate the drive system components based on the second trajectory.

[0089]

[0094] 5 illustrates an example process 500 for determining a trajectory that a vehicle will follow in the future based on vehicle behavior associated with vehicle actions in the environment. Some or all of process 500 may be performed by one or more components in FIG. 3 as described herein. For example, some or all of process 500 may be performed by vehicle computing device 304.

[0090]

[0095] At operation 502, process 500 includes determining a first position of a vehicle operating according to a first trajectory (e.g., a first vehicle trajectory) in an environment at a first time. The vehicle computing system may determine the first position based on sensor data received from one or more sensors. The sensor data may indicate a position and / or movement of the vehicle in the environment. The sensors may include cameras, motion detectors, lidar, radar, time of flight, etc. The sensors may be onboard the vehicle and / or may include sensors remote from the vehicle (e.g., onboard another vehicle, onboard the environment, etc.).

[0091]

[0096] At operation 504, the process 500 includes determining whether the first position is within a threshold distance of a planned trajectory of the vehicle. The planned trajectory may include a previously determined trajectory associated with the operation of the vehicle in the environment. As discussed above, the planned trajectory may be determined by a planning component of the vehicle computing system, such as at a previous time interval. In various examples, the threshold distance (e.g., 1 foot, 0.5 meters, etc.) may represent a safety constraint to ensure that the vehicle operates within predetermined safety parameters.

[0092]

[0097] Based on a determination that the first location is not within the threshold distance ("No" at operation 504), process 500 includes identifying a second location in the environment for the vehicle to travel to at operation 506. Exceeding the threshold distance may represent a deviation from the planned trajectory beyond a predetermined safety parameter. The second location may include a safe location for the vehicle to travel to, such as outside of traffic flow. In various examples, the second location may include a parking location for the vehicle to cease operation.

[0093]

[0098] At operation 508, the process 500 includes controlling the vehicle to a second location. In some examples, the vehicle computing system may determine a new trajectory associated with controlling the vehicle to the second location. In such examples, the vehicle computing system may control the vehicle according to the new trajectory. In some examples, the vehicle computing system may establish a connection with a remote operator, such as via one or more networks. In response to establishing the connection, the vehicle computing system may enable the remote operator to control the vehicle to the second location, or to another location associated with the stopped vehicle operation. In at least one example, the remote operator may control the vehicle to a safe location to ensure the safety of the vehicle and other objects operating in the environment, while the vehicle computing system and / or the remote computing system perform troubleshooting operations to determine the cause of the deviation from the planned path.

[0094]

[0099] Based on the determination that the first position is within the threshold distance ("Yes" at operation 504), the process 500 includes, at operation 510, determining a second position of the vehicle associated with a second time, the second position including an estimated future position of the vehicle. The second position may include a lateral coordinate and a vertical coordinate (e.g., XY coordinate). The lateral coordinate of the second position may include a lateral position of the vehicle at the second (future) time. In some examples, the vehicle computing system determines the lateral coordinate of the second position based on the planned trajectory. In some examples, the lateral coordinate may be the same or substantially the same (e.g., less than 2% difference, within 5 centimeters, etc.) as a lateral coordinate associated with the planned trajectory at the future time (e.g., the X coordinate of the planned trajectory). In some examples, the lateral coordinate may be within a threshold lateral distance (e.g., within 10 centimeters, within 3 inches, etc.) from the planned trajectory. Thus, in at least one example, the vehicle computing system may be configured to limit the lateral coordinate of the second position to a lateral range of the planned trajectory.

[0095]

[0100] In various examples, the vehicle computing system may determine a longitudinal coordinate of the second position based on a trajectory associated with a first time (e.g., the first trajectory). In some examples, the vehicle computing system may be configured to determine the vehicle trajectory at a rate (e.g., every 50 milliseconds, every 100 milliseconds, etc.), e.g., to provide a continuous trajectory and ensure a smooth ride for the vehicle's occupants. In such examples, the vehicle computing system may determine the longitudinal coordinate based on a current trajectory associated with the vehicle while determining an updated trajectory associated with a second future time.

[0096]

[0101] In some examples, the time interval between the first time and the second time may be determined based at least in part on a rate associated with determining the vehicle trajectory. Additionally, in some examples, the time interval may be determined based on a time delay associated with the operation of a vehicle component (e.g., a drive system component). In such examples, the vehicle computing system may be configured to take into account the delay associated with the operation of the drive system component to, for example, provide a more accurate and continuous trajectory and ensure a smooth ride for the occupants.

[0097]

[0102] At operation 512, process 500 includes determining an action associated with the vehicle operating in the environment. The action may include an action determined by a vehicle computing system (e.g., planning component 118) based on conditions in the environment (e.g., road rules, detected objects, etc.). As a non-limiting example, the action may include maintaining a speed for traveling through the environment, stopping at a stop sign, accelerating from a stopped position at an intersection, slowing down to yield to other vehicles, etc.

[0098]

[0103] In at least one example, the vehicle computing system may determine an action based on an object detected in the environment, such as to control the vehicle based on the object. The object may include a pedestrian, a bicycle, a motorcycle, another vehicle, etc. In some examples, the vehicle computing system may be configured to detect an object in the environment and determine that the object is related to the vehicle. In such examples, the vehicle computing system may determine an action based on the related object. In some examples, the vehicle computing system may determine the relevance of the object utilizing techniques such as those described in U.S. Patent Application Nos. 16 / 389,720 and / or 16 / 417,260, the contents of which are incorporated herein by reference above for all purposes. In some examples, the determination of the relevance of the object may be based on a predicted object trajectory associated therewith. In such examples, the vehicle computing system (e.g., a prediction component associated therewith) may be configured to determine the predicted object trajectory and / or the relevance of the object associated therewith. In some examples, the vehicle computing system may determine predicted object trajectories using techniques such as those described in U.S. patent application Ser. Nos. 15 / 807,521, 16 / 151,607, and 16 / 504,147, the contents of which are incorporated herein by reference for all purposes.

[0099]

[0104] At operation 514, process 500 includes determining whether the action is related to a change in speed or direction of the vehicle. A change in speed of the vehicle may include accelerating or decelerating (e.g., negative acceleration). For example, the action may include accelerating entering an intersection from a stop sign. As another example, the action may include slowing down to yield to other vehicles. A change in direction may include turning, changing lanes, etc. For example, the action may include a lane change action that includes a change in the direction of travel of the vehicle.

[0100]

[0105] Based on a determination that the action is not associated with a change in speed or direction of the vehicle ("No" at operation 514), process 500 includes determining a second trajectory based in part on the first trajectory at operation 516. In various examples, the first trajectory and the second trajectory may be the same or substantially the same. In some examples, the second trajectory may include a correction to the driving direction associated with the first trajectory.

[0101]

[0106] Based on a determination that the action is related to a change in speed or direction of the vehicle ("Yes" at operation 514), process 500 includes determining a third trajectory associated with the second time based in part on the second position and the vehicle action at operation 518. In some examples, the third trajectory may be additionally determined based on the first trajectory.

[0102]

[0107] At operation 520, process 500 includes controlling the vehicle based at least in part on the second trajectory (determined at operation 516) or the third trajectory. In various examples, the vehicle computing system may identify one or more drive system components associated with the second trajectory or the third trajectory. In some examples, the vehicle computing system may generate one or more control signals to actuate the drive system components. In some examples, the vehicle computing system may transmit a control signal at the second time, such as to initiate a correction from the first trajectory to the second trajectory or the third trajectory at the second time. In some examples, the vehicle computing system may determine a delay (e.g., an actuation delay) associated with the drive system component. In such examples, the vehicle computing system may transmit a signal at a time prior to the second time, such as to actuate the drive system component for the second time, based at least in part on the delay associated with the system component.

[0103]

[0108] 6 illustrates an example process 600 for transmitting control signals associated with a vehicle trajectory based on actuation delays associated with corresponding vehicle components. Some or all of the process 600 may be performed by one or more components in FIG. 3 as described herein. For example, some or all of the process 600 may be performed by the vehicle computing device 304.

[0104]

[0109] At operation 602, process 600 includes determining a position of a vehicle operating according to a first trajectory (e.g., a first vehicle trajectory) in an environment at a first time. The vehicle computing system may determine the position based on sensor data received from one or more sensors. The sensor data may indicate the position and / or movement of the vehicle in the environment. The sensors may include cameras, motion detectors, lidar, radar, time of flight, etc. The sensors may include sensors onboard the vehicle or off-vehicle (e.g., onboard another vehicle, on-board the environment, etc.).

[0105]

[0110] At operation 604, process 600 includes determining an estimated position of the vehicle at a second time based at least in part on the first trajectory and the planned trajectory of the vehicle. The estimated position may include a lateral coordinate and a vertical coordinate (e.g., XY coordinates). The lateral coordinate of the estimated position includes a lateral position of the vehicle at the second (future) time. In some examples, the vehicle computing system determines the lateral coordinate of the estimated position based on the planned trajectory. In some examples, the lateral coordinate may be identical or substantially identical (e.g., less than 2% difference, within 5 centimeters, etc.) to a lateral coordinate associated with the planned trajectory (e.g., the X coordinate of the planned trajectory) at the future time. In some examples, the lateral coordinate may be within a threshold lateral distance (e.g., within 10 centimeters, within 3 inches, etc.) from the planned trajectory. Thus, in at least one example, the vehicle computing system may be configured to constrain the lateral coordinate of the second position to be within a lateral range of the planned trajectory.

[0106]

[0111] In various examples, the vehicle computing system may determine a longitudinal coordinate of the estimated position based on a vehicle trajectory associated with a first time (e.g., a first trajectory). In some examples, the vehicle computing system may be configured to determine the vehicle trajectory at a rate (e.g., every 50 milliseconds, every 100 milliseconds, etc.), such as to provide a continuous trajectory and ensure a smooth ride for vehicle occupants. In such examples, the vehicle computing system determines a longitudinal coordinate based on a current trajectory associated with a vehicle operating at a first time while determining an updated trajectory associated with a second future time.

[0107]

[0112] In some examples, the time interval between the first time and the second time is determined based at least in part on a rate associated with determining the vehicle trajectory. Additionally, in some examples, the time interval may be determined based on a time delay associated with the operation of a vehicle component (e.g., a drive system component). In such examples, the vehicle computing system may be configured to take into account the delay associated with the operation of the drive system component to provide a more accurate and continuous trajectory and ensure a smooth ride for the occupants.

[0108]

[0113] At operation 606, process 600 includes determining a vehicle action associated with the estimated location. The action may include an action that the vehicle would take at the estimated location and / or at a second time associated therewith. The action may include an action determined by the vehicle computing system (e.g., planning component 118) based on conditions in the environment (e.g., road rules, detected objects, etc.) and / or based on objects detected in the environment. As a non-limiting example, the action may include maintaining a speed to travel through the environment, stopping at a stop sign, accelerating from a stopped position at an intersection, slowing down to yield to other vehicles, etc.

[0109]

[0114] At operation 608, the process 600 includes determining a second trajectory associated with a second time based in part on the vehicle action and the first trajectory. In various examples, the second trajectory may have one or more speeds and / or headings associated with it. In such examples, the vehicle computing system may determine one or more speeds and / or headings associated with the second trajectory. In examples where the vehicle action includes a change in speed and / or heading, the vehicle computing system may determine the second trajectory utilizing one or more first speeds associated with the first trajectory. In examples where the vehicle action includes a change in heading, the vehicle computing system may determine the second trajectory utilizing one or more first headings associated with the first trajectory.

[0110]

[0115] At operation 610, process 600 includes determining whether the second trajectory is associated with a modification of a vehicle component. In some examples, the vehicle component may include a drive system component, as described above. In some examples, the drive system component may include a motor, an engine, a transmission, a steering system component, a braking system component, etc. In various examples, the vehicle computing system may determine a modification of the vehicle component based on a change in speed and / or heading between the first trajectory and the second trajectory.

[0111]

[0116] Based on a determination that the second trajectory is not associated with a modification of the vehicle component ("No" at operation 610), process 600 includes controlling the vehicle according to the second trajectory at operation 612. In this manner, the vehicle computing system may cause the vehicle to travel according to the second trajectory at the second time.

[0112]

[0117] Based on a determination that the second trajectory is associated with a modification of the vehicle component ("Yes" at operation 610), process 600 includes determining an actuation delay associated with the vehicle component at operation 614. In some examples, the modification may include a modification of two or more vehicle components. In some examples, the actuation delay may include an average actuation delay associated with the two or more vehicle components. In some examples, the actuation delay may include a maximum or minimum delay associated with an actuation of the vehicle component of the two or more components. In some examples, the actuation delay may include a predetermined delay associated with one or more drive system components (e.g., vehicle components).

[0113]

[0118] At operation 616, process 600 includes sending a control signal to a vehicle component based at least in part on the actuation delay and the second trajectory. In some examples, the control signal may actuate the vehicle component to cause the vehicle to travel according to the second trajectory at the second time.

[0114] Examples

[0119] A: A system comprising a sensor, one or more processors, and one or more non-transitory computer-readable media storing instructions executable by the one or more processors, the instructions which, when executed, cause the system to: receive a first vehicle trajectory; determine a first position of a vehicle operating in an environment at a first time based at least in part on sensor data from the sensor; determine a first projection position of the first position mapped to the first trajectory; and determine a second position of the vehicle at a second time after the first time based at least in part on the first projection position and the first vehicle trajectory. determining an action associated with the vehicle operating in the environment, the second position including an estimated future position of the vehicle, the second position including a lateral coordinate constrained to the first vehicle trajectory and a longitudinal coordinate determined based at least in part on a speed associated with the first vehicle trajectory; determining a second vehicle trajectory associated with the vehicle operating at the second time based at least in part on the second position and the action; and controlling the vehicle at the second time based at least in part on the second vehicle trajectory.

[0115]

[0120] B: The system described in A, further comprising determining that a distance from the first position of the vehicle to the first vehicle trajectory is less than or equal to a threshold distance, wherein determining the second position of the vehicle is based at least in part on determining that the distance is less than or equal to the threshold distance.

[0116]

[0121] C: The system of either paragraph A or B, further including: determining a third position of the vehicle at the second time, where the third position of the vehicle comprises an actual position of the vehicle at the second time; determining that a distance between the third position and the first vehicle trajectory exceeds a threshold distance; identifying a fourth position in the environment for the vehicle to move to based at least in part on the distance exceeding the threshold distance, where the fourth position is associated with a parking position; determining a third trajectory associated with the vehicle to operate to the fourth position; and controlling the vehicle according to the third trajectory.

[0117]

[0122] D: The system of any one of A to C, wherein the second time is based at least in part on at least one of a first time interval associated with calculation of a vehicle trajectory or a second time interval associated with an actuation delay corresponding to a vehicle component associated with control of the vehicle.

[0118]

[0123] E: The system described in any one of paragraphs A to D, further including: determining a vehicle component associated with control of the vehicle; determining an actuation delay associated with the vehicle component; and transmitting a signal to control the vehicle component based at least in part on the actuation delay.

[0119]

[0124] F: A method comprising: determining an estimated position of a vehicle operating in an environment at a first time at a future time after the first time based at least in part on a current position of the vehicle, the estimated position including a lateral coordinate based at least in part on a first vehicle trajectory associated with the vehicle operating in the environment and a projection of the current position onto the first trajectory, and an ordinate coordinate determined at least in part on a velocity associated with the first vehicle trajectory; and determining a second vehicle trajectory associated with the vehicle operating at the future time based at least in part on the estimated position and the velocity.

[0120]

[0125] G: The method of claim F, further comprising determining that a distance from the current position to the first vehicle trajectory is less than or equal to a threshold distance, and determining the estimated position of the vehicle is based at least in part on determining that the distance is less than or equal to the threshold distance.

[0121]

[0126] H: The method of any of paragraphs F or G, further comprising: determining a measured position of the vehicle at the future time; determining that a distance between the measured position and the first vehicle trajectory exceeds a threshold distance; and determining to move the vehicle to a parking position based at least in part on the distance exceeding the threshold distance.

[0122]

[0127] I: The method described in paragraph H, wherein moving the vehicle to the parking position includes at least one of controlling the vehicle based at least in part on a third trajectory associated with movement of the vehicle into the parking position, or controlling the vehicle based at least in part on a control input received from a remote operator.

[0123]

[0128] J: The method of any one of clauses F to I, wherein the future time is based at least in part on at least one of a first time interval associated with calculation of a vehicle trajectory or a second time interval associated with an actuation delay corresponding to a vehicle component associated with control of the vehicle.

[0124]

[0129] K: The method of any one of clauses F-J, further comprising: determining vehicle components associated with controlling the vehicle according to the second vehicle trajectory; and determining actuation delays associated with the vehicle components, wherein determining the second vehicle trajectory is further based at least in part on the actuation delays.

[0125]

[0130] L: The method of any one of clauses F to K, further comprising determining an action associated with the vehicle operating in the environment and determining a velocity associated with the action, wherein the second vehicle trajectory is determined based at least in part on the velocity.

[0126]

[0131] M: A method according to any one of clauses F to I, determining objects operating in the environment and determining an action to be taken by the vehicle based at least in part on the objects, wherein determining the second vehicle trajectory is further based at least in part on the action.

[0127]

[0132] N: The method of any one of paragraphs F to M, further comprising controlling the vehicle at the future time based at least in part on the second vehicle trajectory.

[0128]

[0133] O: A system or device comprising a processor and a non-transitory computer-readable medium storing instructions that, when executed, cause the processor to perform the computer-implemented method described in any one of items F to N.

[0129]

[0134] P: A system or device comprising a processing means and a storage means coupled to the processing means, the storage means containing instructions for configuring one or more devices to perform the computer-implemented method described in any one of items F to N.

[0130]

[0135] Q: One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations including: determining an estimated position of a vehicle operating in an environment at a first time at a future time after the first time based at least in part on a current position of the vehicle, the estimated position including a lateral coordinate based at least in part on a first vehicle trajectory associated with the vehicle operating in the environment and a projection of the current position onto the first vehicle trajectory, and a longitudinal coordinate determined at least in part based on a velocity associated with the first vehicle trajectory; and determining a second vehicle trajectory associated with the vehicle operating at the future time based at least in part on the estimated position and the velocity.

[0131]

[0136] R: The one or more non-transitory computer-readable media described in Q, further comprising determining that a distance from the current position to the first vehicle trajectory is less than or equal to a threshold distance, and determining an estimated position of the vehicle is based at least in part on determining that the distance is less than or equal to the threshold distance.

[0132]

[0137] S: The one or more non-transitory computer-readable media described in paragraph Q, wherein the future time is based at least in part on at least one of a first time interval associated with calculating a vehicle trajectory or a second time interval associated with an actuation delay corresponding to a vehicle component associated with controlling the vehicle.

[0133]

[0138] T: The one or more non-transitory computer-readable media described in paragraph Q, wherein the operations further include determining a vehicle component associated with controlling the vehicle according to the second vehicle trajectory, determining an actuation delay associated with the vehicle component, and transmitting a signal to actuate the vehicle component based at least in part on the actuation delay.

[0134]

[0139] U: The one or more non-transitory computer-readable media described in Q, wherein the operations further include determining an action associated with the vehicle operating in the environment at the future time and determining one or more velocities associated with the action, and the second vehicle trajectory is determined based at least in part on the one or more velocities.

[0135]

[0140] V: The one or more non-transitory computer-readable media described in paragraph Q, wherein the operations further include controlling the vehicle at the future time based at least in part on the second vehicle trajectory.

[0136]

[0141] Although the above example terms are described with respect to one particular implementation, it should be understood that in the context of this specification, the contents of the example terms may be implemented via methods, devices, systems, computer-readable media, and / or other implementations. Additionally, any of the example terms A-V may be implemented alone or in combination with any one or more of the other example terms A-V.

[0137] summary

[0142] Although one or more examples of the technology described herein have been described, various modifications, additions, permutations, and equivalents thereof fall within the scope of the technology described herein.

[0138]

[0143] In the exemplary description, reference is made to the accompanying drawings, which form a part of this specification, showing by way of example specific examples of the claimed subject matter. It should be understood that other examples can be used and modifications or substitutions, such as structural changes, can be made. Such examples, modifications or substitutions do not necessarily depart from the intended scope of the claimed subject matter. Although steps herein may be presented in a particular order, in some cases the order may be changed, such as providing certain inputs at different times or in a different order, without changing the functionality of the described systems and methods. The disclosed procedures may also be performed in different orders. Additionally, the various calculations herein need not be performed in the order disclosed, and other examples using alternative orders of calculations may be readily implemented. In addition to reordering, calculations may also be decomposed into sub-calculations that produce the same results.

Claims

1. A sensor; one or more processors; one or more non-transitory computer-readable media that store instructions executable by one or more processors; wherein the instructions, when executed, cause the system to: Receiving a first vehicle trajectory; determining a first position of a vehicle operating in an environment at a first time based at least in part on sensor data from the sensor; determining a first projection of the first location mapped onto the first vehicle trajectory; determining a second position of the vehicle at a second time after the first time based at least in part on the first projected position and the first vehicle trajectory, the second position comprising an estimated future position of the vehicle; The second position is a lateral coordinate constrained to the first vehicle trajectory; a longitudinal coordinate determined based at least in part on a velocity associated with the first vehicle trajectory; and determining an action associated with the vehicle operating in the environment; and determining a second vehicle trajectory associated with the vehicle operating at the second time based at least in part on the second position and the action; and and controlling the vehicle based at least in part on the second vehicle trajectory at the second time.

2. determining that a distance from the first position of the vehicle to the first vehicle trajectory is less than or equal to a threshold distance; The system of claim 1 , wherein determining the second position of the vehicle is based at least in part on determining that the distance is less than or equal to the threshold distance.

3. determining a third position of the vehicle at the second time, the third position of the vehicle comprising an actual position of the vehicle at the second time; and determining that a distance between the third location and the first vehicle trajectory exceeds a threshold distance; identifying a fourth location in the environment for the vehicle to travel to based at least in part on the distance exceeding the threshold distance, the fourth location being associated with a park location; and determining a third trajectory associated with the vehicle moving to the fourth location; The system of claim 1 or 2, further comprising: controlling the vehicle according to the third trajectory.

4. The second time is a first time interval associated with the calculation of a vehicle trajectory; or a second time interval associated with an actuation delay corresponding to a vehicle component associated with controlling the vehicle; The system of claim 1 or 2, based at least in part on at least one of:

5. determining a vehicle component associated with control of the vehicle; determining an actuation delay associated with the vehicle component; The system of claim 1 or 2, further comprising: transmitting a signal to control the vehicle component based at least in part on the actuation delay.

6. determining an estimated position of the vehicle at a future time after a first time based at least in part on a current position of the vehicle operating in an environment at the first time, the estimated position comprising: a lateral coordinate based at least in part on a first vehicle trajectory associated with the vehicle operating in the environment and a projection of a current position onto the first vehicle trajectory; a ordinate determined based at least in part on a speed associated with the first vehicle trajectory; and and determining a second vehicle trajectory associated with the vehicle operating at the future time based at least in part on the estimated position and the velocity.

7. determining that a distance from the current location to the first vehicle trajectory is less than or equal to a threshold distance; The method of claim 6 , wherein determining the estimated position of the vehicle is based at least in part on determining that the distance is less than or equal to the threshold distance.

8. determining a measured position of the vehicle at the future time; determining that a distance between the measured location and the first vehicle trajectory exceeds a threshold distance; and determining to move the vehicle into a park position based at least in part on the distance exceeding the threshold distance; The method of any one of claims 6 and 7, further comprising:

9. Moving the vehicle to the parking position includes: controlling the vehicle based at least in part on a third trajectory associated with movement of the vehicle into the park position; or controlling the vehicle based at least in part on control inputs received from a remote operator; The method of claim 8 , comprising at least one of:

10. The future time is a first time interval associated with the calculation of the vehicle trajectory, or at least one of a second time interval associated with an actuation delay corresponding to a vehicle component associated with controlling the vehicle; 8. The method of claim 6 or 7, based at least in part on

11. determining vehicle components associated with controlling the vehicle according to the second vehicle trajectory; determining an actuation delay associated with the vehicle component, determining the second vehicle trajectory is further based at least in part on the actuation delay; and The method of claim 6 or 7, further comprising:

12. determining an action associated with the vehicle operating in the environment; and determining a speed associated with the action; The method of claim 6 or 7, wherein the second vehicle trajectory is determined based at least in part on the velocity.

13. determining objects moving in the environment; determining an action to be taken by the vehicle based at least in part on the object; The method of claim 6 or 7, wherein determining the second vehicle trajectory is further based at least in part on the action.

14. The method of claim 6 or 7, further comprising controlling the vehicle at the future time based at least in part on the second vehicle trajectory.

15. 8. One or more non-transitory computer readable media storing instructions that, when executed, cause one or more processors to perform the method of claim 6 or 7.

Images