Autonomous vehicle trajectory planning for robust fallback mechanisms

Abstract

To enable autonomous vehicle (AV) trajectory planning for robust fallback mechanisms.SOLUTION: A method may include periodically receiving updates to a planned trajectory and updates to a fallback trajectory from a planning system of an AV. The planned trajectory may include a trajectory to a planned location of the AV, and the fallback trajectory may include a trajectory to a fallback stopping location in an environment of the AV. The method may include causing the AV to operate according to the updates to the planned trajectory. The method may include, upon determining that a threshold amount of time has passed since receiving a last update to the planned trajectory from the planning system of the AV, autonomously modifying operation of the AV according to a last update of the fallback trajectory received from the planning system of the AV.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] This specification relates generally to autonomous vehicles, and more particularly to autonomous vehicle trajectory planning for robust fallback response. [Background technology]

[0002] Autonomous vehicles (AVs), whether fully autonomous or partially autonomous, often operate by planning a trajectory along which the AV may travel. Planning a trajectory may involve sensing the external environment using various sensors (e.g., radar, optical, acoustic, humidity, etc.). This external environment may include other objects in the environment, some of which may be mobile. These objects may include other vehicles, bicyclists, pedestrians, animals, etc. [Brief explanation of the drawings]

[0003] The present disclosure is presented by way of example, and not by way of limitation, and may be more fully understood by reference to the following detailed description when considered in conjunction with the figures in which:

[0004] [Figure 1] FIG. 1 illustrates a block diagram of an example autonomous vehicle (AV) capable of AV trajectory planning for robust fallback response, in accordance with some implementations of the present disclosure. [Figure 2] FIG. 2 illustrates a flowchart of an example method for AV trajectory planning for robust fallback response, according to some implementations of the present disclosure. [Figure 3] FIG. 3 depicts a top-down view of an AV's operating environment, showing a representation of the planned trajectory and fallback trajectory, according to some implementations of the present disclosure. [Figure 4] FIG. 4 depicts a top-down view of an AV's operating environment, according to some implementations of the present disclosure, showing a representation of a planned trajectory, a fallback trajectory, and a guide curve used to assist in generating the fallback trajectory. [Figure 5]FIG. 5 illustrates another flowchart of an example method for AV trajectory planning for robust fallback response, according to some implementations of the present disclosure. [Figure 6] FIG. 6 depicts a top-down view of an AV's operating environment, showing a representation of a planned trajectory and multiple fallback trajectories, according to some implementations of the present disclosure. [Figure 7] FIG. 7 illustrates a block diagram of an example computing device capable of AV trajectory planning for robust fallback response, according to some implementations of the present disclosure. Summary of the Invention

[0005] In one implementation, a method for autonomous vehicle (AV) trajectory planning for robust fallback response is disclosed. The method may include periodically receiving updates to a planned trajectory and updates to a fallback trajectory from a planning system of the AV. The planned trajectory may include a trajectory to a planned location of the AV, and the fallback trajectory may include a trajectory to a fallback stop location within the AV's environment. The method may include operating the AV according to the updates to the planned trajectory. The method may include, upon determining that a threshold time has elapsed since receiving a last update to the planned trajectory from the planning system of the AV, autonomously modifying operation of the AV according to the last update to the fallback trajectory received from the planning system of the AV.

[0006] In another implementation, a system for AV trajectory planning for robust fallback response is disclosed. The system may include a memory and one or more processing devices coupled to the memory and configured to perform operations. The operations may include periodically receiving updates to a planned trajectory and updates to one or more fallback trajectories from a planning system of the AV. The planned trajectory may include a trajectory to a planned location of the AV, and each fallback trajectory of the one or more fallback trajectories may include a trajectory to a respective fallback stop location within the environment of the AV. The operations may include operating the AV according to the updates to the planned trajectory. The operations may include selecting one fallback trajectory from the one or more fallback trajectories based on selection metrics associated with the respective fallback trajectories. The operations may include autonomously modifying operation of the AV according to the last update of the selected fallback trajectory received from the planning system of the AV upon determining that a threshold time has elapsed since receiving the last update to the planned trajectory from the planning system of the AV.

[0007] In another implementation, a non-transitory computer-readable storage medium having executable instructions stored thereon is disclosed. The executable instructions can cause one or more processing devices to perform operations. The operations may include periodically receiving updates to a planned trajectory and updates to one or more fallback trajectories from a planning system of the AV. The planned trajectory may include a trajectory to a planned position of the AV, and each fallback trajectory of the one or more fallback trajectories may include a trajectory to a respective fallback stop position within the environment of the AV. The operations may include operating the AV according to the updates to the planned trajectory. The operations may include selecting one fallback trajectory from the one or more fallback trajectories based on selection metrics associated with the respective fallback trajectories. The operations may include autonomously modifying operation of the AV according to the last update of the selected fallback trajectory received from the planning system of the AV upon determining that a threshold time has elapsed since receiving the last update to the planned trajectory from the planning system of the AV. DETAILED DESCRIPTION OF THE INVENTION

[0008] An autonomous vehicle or a vehicle (AV) incorporating various driver assistance features may include a planning system that generates and updates a planned trajectory along which the AV may travel through its environment. The planning system may use various sensors, such as radar, optical devices (e.g., cameras), etc., to generate or update the planned trajectory. However, at some point while the AV is traveling through the environment around the AV, the planning system may be unable to generate or update the planned trajectory. For example, one or more of the sensors may fail (e.g., due to a hardware or software failure) and stop providing information to the planning system, or the planning system itself may fail (e.g., due to a software error or hardware failure). If the AV is unable to generate or update its planned trajectory, the AV often responds by stopping in the direction the AV is currently traveling.

[0009] This response has several drawbacks. For example, a stopping AV may end up stopping in an undesirable location, such as the center of a lane on a road. This may lead to the AV remaining stopped in a location where other drivers would not expect to encounter the stopped vehicle, resulting in a dangerous situation. Additionally, the above response does not consider any predictions about other objects (such as other vehicles) in the environment around the AV, or makes only very basic predictions about such objects and how they may respond to the stopping AV. The lack of accurate predictions about other objects may further increase the likelihood of a dangerous situation occurring.

[0010] Aspects and implementations of the present disclosure address these and other challenges of existing AV systems. Specifically, the present disclosure describes an AV including a planning system that generates and updates (1) a planned trajectory for the AV and (2) one or more fallback trajectories for the AV. The fallback trajectory may include a trajectory from the AV's location to a fallback stop location within the AV's environment. The fallback stop location may include a location determined by the AV to be a relatively safe place to stop the AV (e.g., a roadside). The planning system may generate the fallback trajectory based, at least in part, on the predicted trajectory of an object within the environment around the AV. The planning system may provide the planned trajectory and the one or more fallback trajectories to a control system of the AV, which may control the AV according to the planned trajectory or the fallback trajectory. A fallback subsystem of the AV may determine that a threshold time has elapsed since receiving an update to the planned trajectory (which may indicate that one or more sensors of the planning system or the AV have failed). In response, the fallback subsystem may select one of the fallback trajectories and cause the control system of the AV to operate the vehicle according to the selected fallback trajectory.

[0011] Advantages of the disclosed techniques and systems include, but are not limited to, operating an AV to follow a fallback trajectory if the AV encounters a malfunction. Rather than simply responding to the malfunction by stopping, the AV follows a predetermined fallback trajectory to a fallback stopping location. The fallback trajectory can take into account the predicted behavior of other objects around the AV. The fallback stopping location can be a safer stopping location than a roadway lane (e.g., a shoulder).

[0012] As used herein, the term "safe" may refer to a condition or state of low or minimal risk, or a condition or state that is preferable under given circumstances. A condition or state is not necessarily free of all risk. For example, a safe fallback stopping location for an AV may refer to a location (e.g., a shoulder) where the risk of a traffic accident or other adverse event is lower than another location (e.g., within a lane of the same road).

[0013] In these instances where the description of implementation refers to an AV, it should be understood that similar techniques can be used in various driver assistance systems that do not rise to the level of a fully autonomous driving system. More specifically, the disclosed techniques can be used in Society of Automotive Engineers (SAE) Level 2 driver assistance systems that implement steering, braking, acceleration, lane centering, adaptive cruise control, etc., and other driver support. Similarly, the disclosed techniques can be used in SAE Level 3 driver assistance systems that are capable of autonomous driving under limited (e.g., highway) conditions. Such systems can use rapid and accurate detection and tracking of moving objects to alert the driver of approaching objects and allow the driver to make final driving decisions (e.g., SAE Level 2 systems) or certain driving decisions such as slowing down or changing lanes (e.g., SAE Level 3 systems) without requiring driver feedback.

[0014] 1 is a diagram illustrating components of an example AV 100 capable of AV trajectory planning for robust fallback response, according to some implementations of the present disclosure. AV 100 may include a motor vehicle (such as a car, truck, bus, motorcycle, ATV, recreational vehicle, any specialized agricultural or construction vehicle, etc.), an aircraft (such as an airplane, helicopter, drone, etc.), a marine vessel (such as a ship, boat, yacht, submarine, etc.), or any other self-propelled vehicle capable of operating in an autonomous mode (with no or reduced human input) (e.g., a robot, a factory or warehouse robotic vehicle, a sidewalk delivery robotic vehicle, etc.).

[0015] The environment 101 around the AV 100 (sometimes referred to as the "driving environment") may include any objects (moving or non-moving) located outside the AV 100, such as roads, buildings, trees, bushes, sidewalks, bridges, mountains, other vehicles, pedestrians, animals, etc. The driving environment 101 may be an urban, suburban, rural, etc. In some implementations, the driving environment 101 may be an off-road environment (e.g., a farmland or other agricultural land). In some implementations, the driving environment 101 may be an indoor environment (e.g., an industrial plant environment, a shipping warehouse, a hazardous area of ​​a building, etc.). In some implementations, the driving environment 101 may be substantially flat, with various objects moving parallel to the surface (e.g., parallel to the surface of the Earth). In other implementations, the driving environment 101 may be three-dimensional and include objects capable of moving along all three directions (e.g., balloons, leaves, etc.). Hereinafter, the term "driving environment" should be understood to include all environments in which autonomous movement of a self-driving vehicle may occur. For example, an "operating environment" may include any possible flight environment of an aircraft or a marine environment of a marine vessel. Objects in the operating environment 101 may be located at any distance from the AV 100, from a close distance of a few feet (or less) to several miles (or more).

[0016] As described herein, in a semi-autonomous or partially autonomous driving mode, AV 100 assists with one or more driving maneuvers (e.g., steering, braking, and / or accelerating to perform lane centering, adaptive cruise control, advanced driver assistance systems (ADAS), or emergency braking), but a human driver is expected to maintain situational awareness of the AV 100's surroundings and supervise the assisted driving maneuvers. Here, while AV 100 may perform all driving tasks in certain situations, the human driver is expected to be responsible for assuming control as needed.

[0017] For simplicity and brevity, various systems and methods are described below in conjunction with AV 100, although similar technologies may be used in various driver assistance systems that fall short of a fully autonomous driving system. In the United States, the SAE defines different levels of automated driving operation to indicate how much or how little control the vehicle has over the driving; however, different organizations in the United States, or elsewhere, may classify the levels differently. More specifically, the disclosed systems and methods may be used in SAE Level 2 (L2) driver assistance systems that implement steering, braking, acceleration, lane centering, adaptive cruise control, and other driver support. The disclosed systems and methods may be used in SAE Level 3 (L3) driver assistance systems that are capable of autonomous driving under limited (e.g., highway) conditions. Similarly, the disclosed systems and methods may be used in vehicles using SAE Level 4 (L4) automated driving systems that operate autonomously under most normal driving conditions and require only occasional attention from a human operator. In all such driver assistance systems, accurate lane estimation may be performed automatically without driver input or control (e.g., while the vehicle is moving), resulting in improved reliability of vehicle positioning and navigation, and overall safety of autonomous, semi-autonomous, and other driver assistance systems. As noted above, in addition to the way SAE classifies levels of autonomous driving operation, other organizations in the United States or other countries may classify levels of autonomous driving operation differently. Without limitation, the systems and methods disclosed herein may be used in driver assistance systems defined by the levels of autonomous driving operation of these other organizations.

[0018] Example AV 100 may include sensing system 110. Sensing system 110 may include various electromagnetic (e.g., optical) and non-electromagnetic (e.g., audio) sensing subsystems and / or devices. Sensing system 110 may include radar 114 (or multiple radars 114), which may be any system that utilizes radio or microwave frequency signals to sense objects within the operating environment 101 of AV 100. Radar 114 may be configured to sense both the spatial location of objects (including their spatial dimensions) and the velocity of the objects (e.g., using Doppler shift techniques). Hereinafter, "velocity" refers to both how fast an object is moving (object speed) as well as the direction of the object's motion. Sensing system 110 may include lidar 112, which may be a laser-based unit that may determine the distance to and velocity of objects within operating environment 101. Each of the lidar 112 and radar 114 may include a coherent sensor, such as a frequency-modulated continuous wave (FMCW) lidar or radar sensor. For example, the radar 114 may use heterodyne detection for velocity determination. In some implementations, the ToF and coherent radar functionality is combined into a radar unit that can simultaneously determine both the distance to a reflecting object and the radial velocity of the reflecting object. Such a unit may be configured to operate in a non-coherent sensing mode (ToF mode) and / or a coherent sensing mode (e.g., a mode using heterodyne detection), or both modes simultaneously. In some implementations, multiple lidars 112 or radars 114 may be mounted on the AV 100.

[0019] As used herein, the term "object" may include any body, item, device, body, or thing (moving or non-moving) located outside of AV 100, such as other vehicles, bicyclists, pedestrians, animals, roads, buildings, trees, bushes, sidewalks, bridges, mountains, piers, embankments, runways, or other objects.

[0020] The Lidar 112 may include one or more light sources that generate and emit signals and one or more detectors for signals reflected back from objects. In some implementations, the Lidar 112 can perform a 360-degree scan in the horizontal direction. In some implementations, the Lidar 112 may be capable of spatial scanning along both the horizontal and vertical directions. In some implementations, the field of view may be up to 90 degrees vertically (e.g., at least a portion of the area above the horizon is scanned by the radar signal). In some implementations, the field of view may be spherical (consisting of two hemispheres).

[0021] The sensing system 110 may further include one or more cameras 118 configured to capture images of the driving environment 101. The images may be two-dimensional projections of the driving environment 101 (or portions of the driving environment 101) onto a projection surface (planar or non-planar) of the camera. Some of the cameras 118 of the sensing system 110 may be video cameras configured to capture a continuous (or quasi-continuous) stream of images of the driving environment 101. The sensing system 110 may also include one or more infrared (IR) sensors 119. The sensing system 110 may further include one or more sonars 116, which in some implementations may be ultrasonic sonars.

[0022] AV 100 may include data processing system 120. Data processing system 120 may include one or more computers or computing devices. Data processing system 120 may include hardware or software that receives data from sensing system 110, processes the received data, and determines how AV 100 should operate within driving environment 101. In some implementations, data processing system 120 may receive non-electromagnetic data, such as audio data (e.g., ultrasonic sensor data or data from a microphone picking up an emergency vehicle siren), temperature sensor data, humidity sensor data, pressure sensor data, weather data (e.g., wind speed and direction, precipitation data), etc.

[0023] For example, data processing system 120 may include recognition and planning system 130. Recognition and planning system 130 may be configured to detect and track objects in driving environment 101 and recognize the detected objects. For example, recognition and planning system 130 may analyze images captured by camera 118 and may be capable of detecting traffic signals, road signs, road layouts (e.g., lane boundaries, intersection topology, parking designations, etc.), the presence of obstacles, etc. Recognition and planning system 130 may also receive radar sensing data (Doppler data and ToF data) and determine the distances to various objects in environment 101 and the velocities of such objects (line of sight and, in some implementations, lateral). In some implementations, recognition and planning system 130 may use radar data in combination with data captured by camera 118.

[0024] The perception and planning system 130 may monitor how the driving environment 101 unfolds over time, for example, by tracking the position and velocity of moving objects (e.g., relative to the Earth and / or AV 100) and predicting how various objects will move over a particular time frame, e.g., 1-10 seconds or more. The perception and planning system 130 may also receive information from the positioning subsystem 122, which may include a GPS transceiver and / or an inertial measurement unit (IMU) configured to obtain information about the position of the AV 100 relative to the Earth and its surroundings. The positioning subsystem 122 may use the positioning data (e.g., GPS and IMU data) in conjunction with the sensory data to help accurately determine the position of the AV 100 relative to fixed objects in the driving environment 101 (e.g., roads, lane boundaries, intersections, sidewalks, crosswalks, road signs, curbs, surrounding buildings, etc.).

[0025] In some implementations, the perception and planning system 130 of the data processing system 120 may use data generated by the perception and planning system 130, the positional subsystem 122, and / or other systems and components to plan how the AV 100 will behave in various driving situations and environments. For example, the perception and planning system 130 may include a navigation subsystem for determining a global driving path to a destination. The perception and planning system 130 may also include an obstacle avoidance subsystem for safely avoiding various objects or other obstacles (such as stones, stranded vehicles, pedestrians ignoring traffic rules or signals) within the driving environment 101 of the AV 100. The obstacle avoidance system may be configured to assess the size of an obstacle and its trajectory (if the obstacle is moving) and select an optimal driving strategy (e.g., braking, steering, accelerating, etc.) to avoid the obstacle.

[0026] In one or more implementations, the perception and planning system 130 may include a trajectory subsystem 132. The trajectory subsystem 132 may generate a path through the current driving environment 101. Generating a path through the driving environment 101 may include selecting lanes, navigating traffic congestion, selecting locations to make U-turns, selecting trajectories for parking maneuvers, etc. In one implementation, the trajectory subsystem 132 may generate a planned trajectory through the current driving environment 101. The planned trajectory may include a primary or preferred trajectory for the AV 100 to follow as it travels toward a desired destination.

[0027] In some implementations, trajectory subsystem 132 may generate one or more fallback trajectories. A fallback trajectory may include a trajectory through operating environment 101 to a location where AV 100 can stop. For example, in response to AV 100 experiencing some type of malfunction, such that AV 100 should not continue operating but should stop at a safe location, AV 100 may operate according to the fallback trajectory. In some implementations, trajectory subsystem 132 that generates a trajectory (either a planned trajectory, a fallback trajectory, or some other type of trajectory) may include trajectory subsystem 132 that generates updates to a previously generated trajectory. An update to a trajectory may include a correction to a previously generated trajectory, an extension of a previously generated trajectory, or some other type of update to a previously generated trajectory.

[0028] In some implementations, data processing system 120 may include fallback subsystem 124. Fallback subsystem 124 may be configured to determine whether one or more components of AV 100 have experienced a failure such that AV 100 should operate according to a fallback trajectory. In some implementations, fallback subsystem 124 may determine that AV 100 has experienced a failure in response to a threshold time elapsed since perception and planning system 130 provided an update to the planned trajectory of AV 100 or in response to fallback subsystem 124 receiving an error notification. In response to fallback subsystem 124 determining that AV 100 has experienced a failure, fallback subsystem 124 may select a fallback trajectory and cause AV control system 140 (discussed below) to autonomously modify operation of AV 100 according to the selected fallback trajectory.

[0029] In some implementations, AV 100 may include AV control system (AVCS) 140. AVCS 140 may receive trajectory or other data from data processing system 120 and operate AV 100 according to the received data. Algorithms and modules of AVCS 140 may generate control outputs for use by various systems and components of AV 100, such as powertrain, braking, and steering 150, vehicle electronics 160, signaling 170, and other systems and components not explicitly shown in FIG. 1 . These systems and components may modify the operation of AV 100 based on the control outputs. Powertrain, braking, and steering 150 may include an engine (such as an internal combustion engine or electric motor), a transmission, a differential, axles, wheels, a steering mechanism, and other systems. Vehicle electronics 160 may include an on-board computer, engine management, ignition, communication systems, a car computer, telematics, an in-car entertainment system, and other systems and components. Signaling 170 may include high and low headlights, stop lights, turn signals and backlights, horns and alarms, interior lighting systems, dashboard notification systems, passenger notification systems, radio and wireless network transmission systems, etc. Some of the commands output by AVCS 140 may be delivered directly to powertrain, braking, and steering 150 (or signaling 170), while other commands output by AVCS 140 may be delivered first to vehicle electronics 160, which may generate commands to powertrain, braking, and steering 150 and / or signaling 170.

[0030] In one example, AVCS 140 may receive data from data processing system 120 indicating that an obstacle identified by data processing system 120 should be avoided by slowing the vehicle until a safe speed is reached and then steering the vehicle around the obstacle. AVCS 140 may output commands to powertrain, brakes, and steering 150 (either directly or via vehicle electronics 160) to (1) reduce fuel flow to the engine by changing the throttle setting to lower the engine revolutions per minute (RPM), (2) downshift the drivetrain via the automatic transmission into a lower gear, (3) engage the brake unit (working in coordination with the engine and transmission) to reduce the vehicle's speed until a safe speed is reached, and (4) use the power steering mechanism to implement a steering maneuver until the obstacle is safely bypassed. AVCS 140 may then output commands to powertrain, brakes, and steering 150 to resume the vehicle's previous speed setting.

[0031] FIG. 2 is a flowchart illustrating one embodiment of a method 200 for AV trajectory planning for robust fallback response, according to some implementations of the present disclosure. A processing device having one or more central processing units (CPUs), one or more graphics processing units (GPUs), and / or a memory device communicatively coupled to the CPU and / or GPU may implement method 200 and / or each of its individual functions, routines, subroutines, or operations. Method 200 may be directed to systems and components of a vehicle. In some implementations, the vehicle may be an AV, such as AV 100 of FIG. 1. In some implementations, the vehicle may be a driver-operated vehicle equipped with a driver assistance system, e.g., a Level 2 or Level 3 driver assistance system, that provides limited assistance for certain vehicle systems (e.g., steering, braking, acceleration, etc.) or under limited driving conditions (e.g., highway driving). In certain implementations, a single processing thread may execute method 200. Alternatively, two or more processing threads may execute method 200, with each thread executing one or more individual functions, routines, subroutines, or operations of method 200. In an exemplary embodiment, the processing threads executing method 200 may be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization mechanisms). Alternatively, the processing threads executing method 200 may execute asynchronously with respect to each other. Various operations of method 200 may be performed in a different order (e.g., reversed) compared to the order shown in FIG. 2. Some operations of method 200 may be performed simultaneously with other operations. Some operations may be optional.

[0032] In one implementation, in block 210, the logical processing periodically receives updates to the planned trajectory and updates to the fallback trajectory from a planning system of the AV. The planning system may include the perception and planning system 130. The AV may include the AV 100. In some implementations, the planned trajectory may include a trajectory to a planned position of the AV 100. The fallback trajectory may include a trajectory to a fallback stop position within the environment 101 of the AV 100. In one or more implementations, the trajectory subsystem 132 or the perception and planning system 130 may generate updates to the planned trajectory and fallback trajectory and provide the updates to the data processing system 120.

[0033] As described above, in some implementations, the trajectory subsystem 132 of the AV 100 may generate a planned trajectory from a position of the AV 100 to a planned position. The planned position may include a position along a route to a desired destination of the AV 100. Generating the planned trajectory may include generating an update to a previously generated version of the planned trajectory.

[0034] In some implementations, the recognition and planning system 130 may determine a fallback stop location. The fallback stop location may include a location within the driving environment 101 where it may be safe for the AV 100 to stop. The recognition and planning system 130 may determine the fallback stop location based on one or more stop location factors.

[0035] The stop location factor may include the location of the fallback stop location. The location may include a shoulder within the environment 101 of the AV 100. The location may include a specific portion of the shoulder. The portion of the shoulder may be within a predetermined distance of the road. For example, the portion of the shoulder may be within 1 foot (approximately 0.3 meters) of the road, within 2 feet (approximately 0.6 meters) of the road, within 3 feet (approximately 0.9 meters) of the road, or some other distance. A shorter distance from the road may result in the AV 100 experiencing less lateral force. The portion of the shoulder may be at least a predetermined distance from a barrier next to the shoulder (e.g., a concrete barrier separating the shoulder from non-road portions of the driving environment 101). The predetermined distance may be 1 foot, 2 feet, 3 feet, or some other distance. The location may include a triangular area within a road fork within the environment 101 of the AV 100.

[0036] In some cases, the fallback stopping location may include a center portion of a lane of a road in the environment 101 of the AV 100. For example, the AV 100 may be located in the center lane of a five-lane road where traffic on the road is stopped or moving very slowly. Because it may be impractical to navigate the AV 100 to the shoulder through stopped or slow-moving traffic, in this case, the safe fallback stopping location may be the center of the lane in which the AV 100 is currently traveling. The AVCS 140 may activate one or more components of signaling 170 to indicate that the AV 100 is stopped (e.g., hazard warning flashing lights).

[0037] In some implementations, the stop position factor may include whether the AV 100 can reduce its speed to zero (i.e., the AV 100 can come to a complete stop) at the fallback stop position within a threshold time, which may be 10 seconds, 5 seconds, 3 seconds, 15 seconds, or some other amount of time.

[0038] In some implementations, the recognition and planning system 130 may determine the deceleration value (e.g., -3 m / s 2), determine a stopping distance based on the deceleration, and identify a location that is the determined stopping distance from the current position of AV 100. If the identified location meets one or more stop location factors, recognition and planning system 130 may determine the location as a fallback stop location. In some implementations, recognition and planning system 130 may select multiple deceleration values ​​(e.g., -3 m / s 2 , -4m / s 2 , -5m / s 2 , and -6 m / s 2 ), determine a stopping distance for each deceleration value, and identify one or more locations that are the determined stopping distance from the current position of AV 100. Recognition and planning system 130 may select one of the locations as a fallback stopping location.

[0039] In some implementations, the recognition and planning system 130 may use an artificial intelligence (AI) model to determine the fallback stopping locations. In instances where a description of an implementation refers to an AI model, it should be understood that the AI ​​model may refer to various AI models. For example, the AI ​​model may include an artificial neural network (ANN), which may include multiple nodes (neurons) arranged in one or more layers, and the neurons may be connected to one or more neurons via one or more edges ("synapses"). The synapses may carry signals from one neuron to another, and weights, biases, or other configurations of the nodes or synapses may adjust the value of the signals. The ANN may be trained to adjust the ANN weights or other features. Such training may include inputting trajectory information and other information of one or more objects into the ANN and adjusting the ANN features in response to the ANN output. The ANN may include a deep learning ANN, which may include an ANN with a large number of neurons, synapses, or layers. The AI ​​model may include other types of AI models, such as clustering, decision trees, Bayesian networks, etc.

[0040] In some implementations, the AI ​​model may accept data from the sensing system 110, embeddings based on data from the sensing system 110, or other data as input. The AI ​​model may run on the input data and generate output. The output may include a fallback stop position.

[0041] In one or more implementations, the trajectory subsystem 132 may generate a fallback trajectory from the AV 100 to the fallback stop position. The fallback trajectory may include a spline curve that generates a smooth trajectory from the position of the AV 100 to the fallback stop position. The spline curve may include a cubic curvature parameterized spline. In some implementations, the fallback trajectory may include a piecewise curvature and velocity spline. The piecewise curvature and velocity spline may include quadratic curvature when parameterized by distance. The piecewise curvature and velocity spline may include sextic curvature when parameterized by time. In some implementations, the piecewise curvature may be generated using a piecewise curvature function that is a quadratic function of time.

[0042] In one implementation, block 210 includes receiving an update to a fallback trajectory. The fallback trajectory may include a planned trajectory. For example, an initial portion of the fallback trajectory may include the planned trajectory, and the fallback trajectory may include a trajectory that extends beyond the planned trajectory. Thus, in some implementations, receiving an update to the planned trajectory and an update to the fallback trajectory in block 210 may include receiving an update to the fallback trajectory.

[0043] 3 illustrates an example driving environment 101 overlaid with an example planned trajectory and an example fallback trajectory. The driving environment 101 may include an AV 100. The driving environment 101 may include one or more road lanes 302-1, ..., 302-5. The driving environment 101 may include a shoulder 304 adjacent to the rightmost lane 302-5. The driving environment 101 may include a concrete barrier 306-1 to the left of the leftmost lane 302-1 and another concrete barrier to the right of the shoulder 304. The driving environment 101 may include multiple vehicles 308-1, ..., 308-4, some of which (308-1, ..., 308-3) are traveling in the road lanes 302-1, ..., 302-5 and some of which (308-4) are parked on the shoulder 304.

[0044] As seen in FIG. 3 , the trajectory subsystem 132 of the AV 100 could generate a planned trajectory 310 for the AV 100, represented by a solid arrow in FIG. 3. The perception and planning system 130 could determine a fallback stop location 312, and the trajectory subsystem 132 could generate a fallback trajectory 314 (represented by a dotted line in FIG. 3 ) from the AV 100 to the fallback stop location 312. The trajectory subsystem 132 could generate the fallback trajectory 314 based on the predicted trajectories of other objects in the driving environment 101. For example, the fallback trajectory 314 passes near vehicle 308-3, but the perception and planning system 130 may predict that by the time the AV 100 reaches that position on the fallback trajectory 314, vehicle 308-3 will no longer be at that location.

[0045] In some implementations, the planned trajectory 310 and the fallback trajectory 314 coincide for a predetermined time or a predetermined distance. For example, the fallback trajectory 314 and the planned trajectory 310 may coincide for the first 600 ms of the trajectories 310, 314, after which the two trajectories 310, 314 may diverge. The two trajectories 310, 314 may overlap in cases where the recognition and planning system 130 can regain the ability to generate updates to the planned trajectory 310 while the trajectories overlap, which may prevent the AV 100 from diverging from the planned trajectory 310 if the AV 100 experiences a fault for only a short period of time. As an example, in FIG. 3 , the planned trajectory 310 and the fallback trajectory 314 coincide for the first segment of the two trajectories 310, 314, after which they diverge. In some implementations, the recognition and planning system 130 may adjust the predetermined time or the predetermined distance based on the environment 101 of the AV 100. For example, the recognition and planning system 130 may increase the predetermined time or the predetermined distance in response to an increase in the number of objects in the environment 101. In some implementations, the recognition and planning system 130 may adjust the predetermined time or the predetermined distance based on the speed of the AV 100. For example, the recognition and planning system 130 may increase the predetermined time or the predetermined distance in response to an increase in the speed of the AV 100.

[0046] In some implementations, generating the fallback trajectory 314 may include the trajectory subsystem 132 using guide curves to assist in generating the fallback trajectory 314. Figure 4 is a top-down view of a representation of the driving environment 101, showing the planned trajectory 310, the fallback trajectory 314, and a representation of the guide curves used to generate the fallback trajectory 314. In Figure 4, the AV 100 moves forward by moving from left to right, and the AV 100 moves laterally by moving up and down.

[0047] Point 402 indicates the location of AV 100 at the current time. At this time, trajectory subsystem 132 may not yet have completed generating planned trajectory 310 for AV 100. However, based on previous updates to planned trajectory 310 or based on the current velocity of AV 100, trajectory subsystem 132 may estimate that AV 100 will be located somewhere along line 404 after a threshold time. The threshold time may include a threshold time at which planned trajectory 310 and fallback trajectory 314 may coincide, as described above. trajectory subsystem 132 may estimate that AV 100 will be located at point 406 after the threshold time. trajectory subsystem 132 may generate a guide curve 408 from point 406 to fallback stop position 312. The guide curve may include an estimate of a path that AV 100 may use to travel to fallback stop position 312. The trajectory subsystem 132 may generate the fallback trajectory 314 using the guide curve 408 as a guide.

[0048] In response to the trajectory subsystem 132 generating the planned trajectory 310, the trajectory subsystem 132 may generate a fallback trajectory 314 based on the guide curve 408. This may include the fallback trajectory 314 matching the planned trajectory 310 up to point 410. Point 410 may include the position of the AV 100 on the planned trajectory 310 after the threshold time. The trajectory subsystem 132 may then generate the remaining segment of the fallback trajectory 314 by matching the fallback trajectory 314 with the guide curve 408 until the fallback trajectory 314 reaches the fallback stop position 312. The trajectory subsystem 132, the recognition and planning system 130, or the fallback subsystem 124 may adjust or refine the fallback trajectory 314 to comply with one or more commands (e.g., commands related to deceleration, jerk, etc.), as discussed below.

[0049] In some implementations, the trajectory subsystem 132 may periodically generate the planned trajectory 310. For example, the trajectory subsystem 132 may generate the planned trajectory 310 every 100 milliseconds (ms). The trajectory subsystem 132 may periodically generate the fallback trajectory 314. For example, the trajectory subsystem 132 may generate the fallback trajectory 314 every 100 ms. In some implementations, the trajectory subsystem 132 may generate updates to the planned trajectory 310 at least partially in parallel with updates to the fallback trajectory 314.

[0050] In some implementations, the fallback trajectory 314 may comply with one or more fallback trajectory commands. The fallback trajectory commands may recommend that the fallback trajectory 314 have or exhibit one or more characteristics. The fallback trajectory commands may recommend that the fallback trajectory 314 reduce the velocity of the AV 100 to zero at the fallback stop location 312 within a threshold time. The fallback trajectory commands may recommend that the AV 100 maintain its initial velocity for a predetermined time. After the predetermined time, the fallback trajectory 314 may reduce the velocity of the AV 100. The reduction in velocity may lead to the AV 100 reaching a velocity of zero at the fallback stop location 312. In some implementations, the fallback trajectory 314 may indicate that the AV 100 should maintain its initial velocity for a predetermined time in cases where the recognition and planning system 130 can regain the ability to generate the planned trajectory 310. As an example, AV 100 may be traveling along planned trajectory 310 at 60 miles per hour (MPH) (approximately 97 kilometers per hour (KPH)). Trajectory subsystem 132 may not generate an update to planned trajectory 310 within a threshold time (e.g., 600 ms). In response, AV 100 may autonomously modify its operation according to fallback trajectory 314. Fallback trajectory 314 may indicate that AV 100 should continue traveling at 60 MPH for a second threshold time (e.g., 2 seconds) before beginning to reduce the speed of AV 100. However, trajectory subsystem 132 may generate an update to planned trajectory 310 one second after switching to fallback trajectory 314. AV 100 may then autonomously modify its operation according to the updated planned trajectory 310 without reducing its speed.

[0051] In some implementations, the fallback trajectory command may include a fallback trajectory 314 that is consistent with predetermined motion limits of the AV 100. For example, the fallback trajectory command may indicate that the fallback trajectory 314 does not decelerate (laterally, vertically, or in any other direction) below a deceleration threshold. For example, the deceleration threshold may be -5.0 m / s 2The fallback command may indicate that the fallback trajectory 314 does not jerk (laterally, vertically, or in any other direction) above or below a threshold amount. For example, the jerk amount threshold may be 1.0 m / s 3 The fallback command may indicate that the fallback trajectory 314 does not accelerate (laterally, longitudinally, or in some other direction) above an acceleration threshold.

[0052] In one implementation, the logic processing may adjust the fallback trajectory 314 to match predetermined movement limits of the AV 100. For example, if the fallback trajectory 314 is -5.0 m / s 2 Deceleration below a threshold amount (e.g., -5.5 m / s 2 ), the recognition and planning system 130 may adjust the fallback trajectory 314 so that at no point on the fallback trajectory 314 does the deceleration fall below the threshold (e.g., by moving the fallback stop position 312 further away from the AV 100 to give the AV 100 more time to decelerate). In another example, the fallback trajectory 314 is 1.0 m / s 3 Amount of jerk above a threshold amount (e.g., 1.3 m / s 3 ), the recognition and planning system 130 may adjust the fallback trajectory 314 (e.g., by adjusting the shape of the spline curve to be smoother) so that no points on the fallback trajectory 314 exceed the threshold.

[0053] In some implementations, the trajectory subsystem 132 may generate the fallback trajectory 314 based, at least in part, on a previous version of the planned trajectory 310. For example, the fallback trajectory 314 may match the previous version of the planned trajectory 310 for a predetermined time or a predetermined distance. This may assist the trajectory subsystem 132 in generating an acceptable fallback trajectory 314 because the previous version of the planned trajectory 310 (on which the fallback trajectory 314 may be partially based) was already determined by the recognition and planning system 130 to be an acceptable trajectory.

[0054] 2 , in one implementation, in block 220, logical processing causes AV 100 to operate according to updates to planned trajectory 310. In one implementation, data processing system 120 may receive updates to planned trajectory 310 and updates to fallback trajectory 314. Data processing system 120 may provide updates to planned trajectory 310 to AVCS 140, which may autonomously modify the operation of AV 100 according to the latest update to planned trajectory 310.

[0055] At block 230, in some implementations, logical processing determines that a threshold time has elapsed since the last update to the planned trajectory 310 was received from the planning system of the AV 100. In some implementations, the fallback subsystem 124 may track the time that has elapsed since the data processing system 120 received an update to the planned trajectory 310. In response to a time that exceeds the threshold time, the fallback subsystem 124 may determine that the AV 100 has experienced a failure, and the method 200 may move to block 240. In response to a time that does not exceed the threshold time, the AVCS 140 may return to block 220 and continue to operate the AV 100 in accordance with the updates to the planned trajectory 310.

[0056] In block 240, in one implementation, logical processing autonomously modifies the operation of AV 100 according to the last update of fallback trajectory 314 received from the planning system of AV 100. In some implementations, data processing system 120 may provide fallback trajectory 314 to AVCS 140. Fallback subsystem 124 may provide a notification to AVCS 140 indicating that AVCS 140 will operate AV 100 according to fallback trajectory 314. In one or more implementations, data processing system 120 may also provide planned trajectory 310 to AVCS 140.

[0057] In one or more implementations, block 240 includes modifying operation of AV 100 in response to receiving further data from data processing system 120 or some other system of AV 100. For example, in response to a collision detection subsystem of data processing system 120, perception and planning system 130, or some other system of AV 100 detecting a potential collision on fallback trajectory 314, AVCS 140 may perform a collision avoidance maneuver. The collision avoidance maneuver may include engaging brake 150 components of AV 100, adjusting steering 150 of AV 100, or some other action.

[0058] In some implementations, in block 220, logical processing may cause AV 100 to operate according to the update to fallback trajectory 314 (instead of planned trajectory 310). The fallback trajectory 314 and the planned trajectory 310 may coincide for a predetermined time or a predetermined distance, so that if AVCS 140 operates AV 100 according to the updated fallback trajectory 314 before AV 100 reaches the point where planned trajectory 310 and fallback trajectory 314 diverge, AV 100 will travel along the same trajectory that AV 100 would have operated along according to planned trajectory 310.

[0059] In some implementations, at block 230, logical processing may receive an error notification indicating that AV 100 has experienced a failure and that method 200 should proceed to block 240. In one implementation, fallback subsystem 124 may receive the error notification. Data processing system 120 may send the error notification in response to, for example, a component of sensing system 110 stopping functioning, stopping responding, or sending an error notification to data processing system 120. Data processing system 120 may send the error notification to fallback subsystem 124 in response to recognition and planning system 130 stopping functioning, stopping responding, or sending an error notification to data processing system 120. Data processing system 120 may send the error notification in response to detecting a failure of AV 100 in some other manner.

[0060] FIG. 5 is a flowchart illustrating one embodiment of a method 500 for AV trajectory planning for robust fallback response, according to some implementations of the present disclosure. A processing device having one or more CPUs, one or more GPUs, and / or a memory device communicatively coupled to the CPU and / or GPU may implement method 500 and / or each of its individual functions, routines, subroutines, or operations. Method 500 may be directed to systems and components of a vehicle. In some implementations, the vehicle may be an AV, such as AV 100 of FIG. 1. In some implementations, the vehicle may be a driver-operated vehicle equipped with a driver assistance system, e.g., a Level 4 or Level 3 driver assistance system, that provides limited assistance for certain vehicle systems (e.g., steering, braking, acceleration, etc.) or under limited driving conditions (e.g., highway driving). In certain implementations, a single processing thread may execute method 500. Alternatively, two or more processing threads may execute method 500, each thread executing one or more individual functions, routines, subroutines, or operations of method 500. In an example embodiment, the processing threads performing method 500 may be synchronized (e.g., using semaphores, critical sections, and / or other thread synchronization mechanisms). Alternatively, the processing threads performing method 500 may execute asynchronously with respect to one another. Various operations of method 500 may be performed in a different order (e.g., reversed) compared to the order shown in FIG. 5. Some operations of method 500 may be performed simultaneously with other operations. Some operations may be optional.

[0061] In block 510, the logical processing periodically receives updates to the planned trajectory 310 and updates to one or more fallback trajectories 314-1, ..., 314-n from a planning system of the AV 100. The planning system may include the recognition and planning system 130 of the AV 100. The planned trajectory 310 may include a trajectory to a planned position of the AV 100. Each fallback trajectory 314 of the one or more fallback trajectories 314-1, ..., 314-n may include a trajectory to a respective fallback stop location 312 within the environment 101 of the AV 100.

[0062] In some implementations, periodically receiving updates to the planned trajectory 310 may include functionality similar to some of the functionality of block 210 of method 200 of FIG. 2 . In some implementations, periodically receiving updates to one or more fallback trajectories 314-1, ..., 314-n may include functionality similar to some of the functionality of block 210. For example, one or more of the fallback trajectories 314-1, ..., 314-n may conform to one or more fallback trajectories commands and be adjusted based on a previous version of the planned trajectory 310. However, instead of receiving a single fallback trajectory 314 as discussed in block 210, block 510 may include receiving one or more fallback trajectories 314-1, ..., 314-n. In one or more implementations, the recognition and planning system 130 may identify a predetermined number of fallback stop locations 312-1, ..., 312-n. The predetermined number of fallback stop positions 312-1, ..., 312-n may include 1, 2, 3, 4, 5, or 6 or more fallback stop positions 312-1, ..., 312-n.

[0063] FIG. 6 illustrates another example driving environment 101 overlaid with an example planned trajectory 310 and multiple example fallback trajectories 314-1, ..., 314-3. The driving environment 101 of FIG. 6 may include one or more components of the driving environment 101 of FIG. 3. For example, the driving environment 101 of FIG. 6 may include the AV 100, road lanes 302-1, ..., 302-3, road shoulders 304-1, 304-2, concrete barriers 306-1, 306-2, and vehicles 308-1, ..., 308-3. As can be seen in FIG. 6, the perception and planning system 130 of the AV 100 was able to generate the planned trajectory 310. The recognition and planning system 130 may also determine a plurality of fallback stop locations 312-1, ..., 312-3 and a respective fallback trajectory 314-1, ..., 314-3 for each fallback stop location 312-1, ..., 312-3.

[0064] In some implementations, multiple fallback trajectories 314-1, ..., 314-n may lead to the same fallback stop location 312-1. This may provide AV 100 with multiple options for reaching the same fallback stop location 312-1. In one implementation, each fallback trajectory 314 of the multiple fallback trajectories 314-1, ..., 314-n may lead to a unique fallback stop location 312-1, ..., 312-n. In one or more implementations, the planned trajectory 310 and one or more fallback trajectories 314-1, ..., 314-n may coincide for a predetermined time or a predetermined distance. For example, the fallback trajectories 314-1, ..., 314-n and the planned trajectory 310 may coincide for the first 600 ms of the trajectories 310, 314-1, ..., 314-n, after which the planned trajectory 310 and one or more fallback trajectories 314-1, ..., 314-n may diverge. Different fallback trajectories 314-1, . . . , 314-n may diverge at different times or distances from other fallback trajectories 314-1, . . . , 314-n.

[0065] 5, in block 520, logical processing causes AV 100 to operate according to updates to planned trajectory 310. Block 520 may include functionality similar to the functionality of block 220 of method 200.

[0066] In block 530, logical processing selects one or more of the fallback trajectories 314-1, ..., 314-n fallback trajectories 314 based on selection metrics associated with each fallback trajectory 314. In one implementation, the recognition and planning system 130 may provide the one or more fallback trajectories 314-1, ..., 314-n to the fallback subsystem 124, which may select a fallback trajectory 314 from the one or more fallback trajectories 314-1, ..., 314-n based on selection metrics associated with the selected fallback trajectory 314.

[0067] In some implementations, the fallback subsystem 124 may calculate a selection metric for each fallback trajectory 314 of the one or more fallback trajectories 314-1, ..., 314-n. The fallback subsystem 124 may calculate the selection metric for the fallback trajectory 314 based on one or more selection factors. In one implementation, the selection factor may include the distance of the fallback trajectory 314 to the fallback stop location 312. In some implementations, the selection metric may be higher as the distance to the fallback stop location 312 becomes longer. This may be because a longer distance may allow the AV 100 to decelerate at a lower speed, which may be more comfortable for passengers of the AV 100. Conversely, a shorter distance to the fallback stop location 312 may result in a lower selection metric.

[0068] In some implementations, the selection factors may include whether the fallback trajectory 314 intersects with a predicted trajectory of an object in the environment 101 of the AV 100. A fallback trajectory 314 that intersects the predicted trajectory of the object may include the fallback trajectory 314 and the predicted trajectory of the object overlapping or at least partially matching. In one or more implementations, the trajectory subsystem 132 may generate a predicted trajectory of the object based on the AV 100 operating according to the fallback trajectory 314, the predicted behavior of other objects in the driving environment 101, or other data. In some implementations, the perception and planning system 130 may use an AI model to generate a predicted trajectory for the object. The AI ​​model may use data from the sensing system 110, embeddings based on data from the sensing system 110, the fallback trajectory 314 (or some other trajectory proposed or simulated for the AV 100), or other data as input. The AI ​​model may execute based on the input and generate a predicted trajectory for the object in the driving environment 101.

[0069] In one implementation, the selection metric for the fallback trajectory 314 may be lowered depending on the fallback trajectory 314 and the predicted trajectory of the intersecting object. This may be because there may be a risk of collision between the trajectory of the AV 100 and the trajectory of the object if they intersect. However, in some implementations, the selection metric for the fallback trajectory 314 may not be lowered depending on the trajectory of the fallback trajectory 314 and the predicted trajectory of the intersecting object if they indicate a low or no likelihood of collision.

[0070] In one or more implementations, the selection factor may include lateral movement of the fallback trajectory 314. The lateral movement of the AV 100 may include the AV 100 moving to the left of the AV 100 or moving to the right of the AV 100. In some implementations, the selection metric for a fallback trajectory 314 may be lower for a fallback trajectory 314 that includes lateral movement. The more lateral movement the fallback trajectory 314 includes, the lower the selection metric may be. This may be because lateral movement may cause more anxiety to passengers of the AV 100 and because lateral movement may increase the chance of a collision with other objects in the driving environment 101. In some implementations, the lateral movement may include lateral acceleration / deceleration or jerk of the AV 100.

[0071] In some implementations, the selection factors may include the type of location of the fallback stop location 312. For example, a shoulder may result in a higher selection metric than the center of the roadway lane 302. In some implementations, the selection factors may include whether the AV 100 following the fallback trajectory 314 would result in the trajectory of the AV 100 not complying with traffic laws or predetermined driving practices.

[0072] In one or more implementations, the fallback subsystem 124 may calculate a selection metric for each fallback trajectory 314 based on one or more selection factors. The fallback subsystem 124 may use a cost function that uses the selection factors as inputs. The fallback subsystem 124 may use a weight function that uses the selection factors and weights as inputs. The fallback subsystem 124 may use other functions or operations to calculate the selection metric for each fallback trajectory 314. After calculating the selection metric for each fallback trajectory 314, the fallback subsystem 124 may select the fallback trajectory 314 with the minimum cost, maximum selection metric, etc., as calculated by the cost function. The fallback subsystem 124 may provide the selected fallback trajectory 314 to the data processing system 120. The data processing system 120 may provide the planned trajectory 310 and the selected fallback trajectory 314 to the AVCS 140.

[0073] In some implementations, the fallback subsystem 124 or the recognition and planning system 130 may adjust the selected fallback trajectory 314 (e.g., by modifying the fallback trajectory 314 to match predetermined motion limits of the AV 100, as described above) before providing the fallback trajectory 314 to the AVCS 140. In one or more implementations, the fallback subsystem 124 or the recognition and planning system 130 may adjust one or more of the fallback trajectories 314-1, ..., 314-n before calculating the selection metrics for the one or more fallback trajectories 314-1, ..., 314-n.

[0074] At block 540, the logical processing determines that a threshold time has elapsed since receiving the last update to the planned trajectory 310 from the planning system of the AV 100. Block 540 may include functionality similar to that of block 230 of method 200. At block 550, the logical processing autonomously modifies the operation of the AV 100 according to the last update to the selected fallback trajectory 314 received from the planning system of the AV 100. Block 550 may include functionality similar to that of block 240 of method 200.

[0075] FIG. 7 illustrates a block diagram of an example computing device 700 capable of AV trajectory planning for robust fallback response, according to some implementations of the present disclosure. The example computing device 700 may be connected to other computing devices within a local area network (LAN), an intranet, an extranet, and / or the Internet. The computing device 700 may operate in a server capacity in a client-server network environment. The computing device 700 may be a personal computer (PC), a set-top box (STB), a server, a network router, switch, or bridge, or any device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that device. Furthermore, while only a single example computing device is shown, the term “computer” shall also be considered to include any group of computers that, individually or jointly, execute a set (or sets) of instructions to perform any one or more of the methodologies discussed herein.

[0076] An embodiment of computing device 700 may include a processing unit 702 (also referred to as a processor or CPU), a main memory 704 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM), etc.), a static memory 706 (e.g., flash memory, static random access memory (SRAM), etc.), and a secondary memory (e.g., data storage device 718), which may communicate with each other via a bus 730.

[0077] Processing unit 702 (which may include logic processing 703) represents one or more general-purpose processing units, such as a microprocessor, a CPU, or the like. More specifically, processing unit 702 may be a complex instruction set computer (CISC) microprocessor, a reduced instruction set computer (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor executing other instruction sets, or a processor executing a combination of instruction sets. Processing unit 702 may also be one or more special-purpose processing units, such as a GPU, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. According to one or more aspects of the present disclosure, processing unit 702 may be configured to execute instructions implementing method 200 or 500 for AV trajectory planning for robust fallback response.

[0078] An example of computing device 700 may further include a network interface device 708, which may be communicatively coupled to a network 720. The network interface device 708 may include a network card, a network interface controller, or some other network interface. The network 720 may include a LAN, an intranet, an extranet, the Internet, a modem, a router, a switch, or some other network or network device. In some embodiments, computing device 700 may communicate data with other systems or devices over network 720. An example of computing device 700 may further include a video display 710 (e.g., a liquid crystal display (LCD), a touch screen, or a cathode ray tube (CRT)), an alphanumeric input device 712 (e.g., a keyboard), a cursor control device 714 (e.g., a mouse), and an audio signal generating device 716 (e.g., a speaker).

[0079] Data storage device 718 may include a computer-readable storage medium (or more specifically, a non-transitory computer-readable storage medium) 728 having stored thereon one or more sets of executable instructions 722. According to one or more aspects of the present disclosure, executable instructions 722 may include executable instructions for performing method 200 or 500.

[0080] The executable instructions 722 may also reside, for example, completely or at least partially within the main memory 704 and / or the processing unit 702 during their execution on the computing device 700, with the main memory 704 and the processing unit 702 also constituting computer-readable storage media. The executable instructions 722 may further be transmitted or received over a network via the network interface device 708.

[0081] Although computer-readable storage medium 728 is illustrated in FIG. 7 as a single medium, the term "computer-readable storage medium" should be considered to include a single medium or multiple media (e.g., centralized or distributed databases, and / or associated caches and servers) that store one or more sets of operating instructions. The term "computer-readable storage medium" should also be considered to include any medium capable of storing or encoding a set of instructions for execution by a machine, causing the machine to perform any one or more of the methods described herein. Thus, the term "computer-readable storage medium" should be considered to include, but not be limited to, solid-state memory, and optical and magnetic media.

[0082] In some cases, a particular component of AV 100 (e.g., sensing system 110, data processing system 120, AVCS 140, or other component) may include computing device 700.

[0083] Some portions of the above detailed descriptions are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, understood to be a self-consistent sequence of steps leading to a desired result. The steps require physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0084] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise indicated, and as will be apparent from the discussion that follows, throughout the description, discussions using terms such as "identify," "determine," "adjust," "produce," "compare," "generate," "generate," "perform," "receive," "modify," "select," or the like, will be understood to refer to the actions and processes of a computer system, or similar electronic computing device, that manipulate and transform data represented as physical (electronic) quantities in the computer system's registers and memory into other data similarly represented as physical quantities in the computer system's memory or registers or other such information storage, transmission, or display device.

[0085] Examples of the present disclosure also relate to an apparatus for performing the methods described herein. This apparatus may be specially constructed for the required purposes, or it may be a general-purpose computer system selectively programmed by a computer program stored within the computer system. Such a computer program may be stored on a computer-readable storage medium, such as, but not limited to, any type of disk, including optical disks, CD-ROMs, and magneto-optical disks, read-only memory (ROM), random-access memory (RAM), EPROM, EEPROM, magnetic disk storage media, optical storage media, flash memory devices, other types of machine-accessible storage media, or any type of medium suitable for storing electronic instructions, each coupled to a computer system bus.

[0086] The methods and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems appears as set forth in the description below. Additionally, the scope of the present disclosure is not limited to any particular programming language. It will be understood that a variety of programming languages ​​can be used to implement the teachings of the present disclosure.

[0087] It is to be understood that the above description is intended to be illustrative, and not restrictive. Many other implementations will be apparent to those skilled in the art upon reading and understanding the above description. While the present disclosure describes particular examples, it will be recognized that the systems and methods of the present disclosure are not limited to the examples described herein, but may be modified and practiced within the scope of the appended claims. Accordingly, the specification and drawings should be considered in an illustrative, and not a restrictive, sense. The scope of the present disclosure should, therefore, be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. 1. A method comprising: periodically receiving updates to a planned trajectory and updates to a fallback trajectory from a planning system of an autonomous vehicle (AV), the planned trajectory being a trajectory to a planned position of the AV and the fallback trajectory being a trajectory to a fallback stop position within the AV's environment; operating the AV in accordance with updates to the planned trajectory; and upon determining that a threshold time has elapsed since receiving a last update to the planned trajectory from the planning system of the AV, autonomously modifying operation of the AV according to a last update to the fallback trajectory received from the planning system of the AV.

2. The fallback stop position is a road shoulder within the environment of said AV; a triangular area of ​​road lanes within the AV's environment; or The method of claim 1 , including at least one of a center portion of a lane of a road within the environment of the AV.

3. 10. The method of claim 1, wherein the fallback trajectory reduces the velocity of the AV to zero at the fallback stop position within a threshold time.

4. The method of claim 3 , wherein the threshold time period comprises 10 seconds.

5. The method of claim 1 , wherein the planned trajectory and the fallback trajectory coincide for a predetermined period of time.

6. 10. The method of claim 1, further comprising adjusting the fallback trajectory to conform to predetermined motion limits of the AV.

7. 10. The method of claim 1, wherein the fallback trajectory indicates that the AV will maintain a velocity for a predetermined period of time.

8. 1. A system comprising: Memory and one or more processing units, coupled to the memory; periodically receiving updates to a planned trajectory and updates to one or more fallback trajectories from a planning system of an autonomous vehicle (AV), the planned trajectory being a trajectory to a planned position of the AV, and each fallback trajectory of the one or more fallback trajectories being a trajectory to a respective fallback stop position within an environment of the AV; operating the AV in accordance with updates to the planned trajectory; selecting a fallback trajectory from the one or more fallback trajectories based on a selection metric associated with the respective fallback trajectory; and one or more processing devices configured to perform operations including: upon determining that a threshold time has elapsed since receiving a last update to the planned trajectory from the planning system of the AV, autonomously modifying operation of the AV in accordance with the last update to the selected fallback trajectory received from the planning system of the AV.

9. a fallback stop location among the one or more fallback stop locations of the one or more fallback trajectories includes a portion of a shoulder of a road within an environment of the AV; The system of claim 8 , wherein the portion of the shoulder is within a predetermined distance from a lane of the road.

10. each fallback trajectory of the one or more fallback trajectories: a deceleration amount below a deceleration amount threshold; The system of claim 8 , further comprising generating a trajectory that is free of jerks that exceed a jerk amount threshold.

11. 10. The system of claim 8, wherein each fallback trajectory of the one or more fallback trajectories comprises a trajectory based, at least in part, on a planned trajectory of a previous version of the AV.

12. a selection metric associated with each fallback trajectory; the distance of each fallback trajectory to the fallback stop position; whether the respective fallback trajectories intersect with predicted trajectories of objects in the AV's environment; or The system of claim 8 , based on at least one of the lateral movements of the respective fallback trajectories.

13. The system of claim 8 , wherein the planned trajectory and the selected fallback trajectory coincide for a predetermined period of time.

14. The system of claim 13 , wherein the action further comprises adjusting the predetermined time based on an environment of the AV.

15. 10. The system of claim 8, wherein periodically receiving updates to the planned trajectory and updates to the one or more fallback trajectories comprises receiving updates to the planned trajectory at least partially in parallel with updates to the one or more fallback trajectories.

16. A non-transitory computer-readable storage medium having executable instructions stored thereon, the executable instructions causing a processing unit to: periodically receiving updates to a planned trajectory and updates to one or more fallback trajectories from a planning system of an autonomous vehicle (AV), the planned trajectory being a trajectory to a planned position of the AV, and each fallback trajectory of the one or more fallback trajectories being a trajectory to a fallback stop position within an environment of the AV; operating the AV in accordance with updates to the planned trajectory; selecting a fallback trajectory from the one or more fallback trajectories based on a selection metric associated with the respective fallback trajectory; and one or more processing devices configured to perform operations including: upon determining that a threshold time has elapsed since receiving a last update to the planned trajectory from the planning system of the AV, autonomously modifying operation of the AV according to the last update to the selected fallback trajectory received from the planning system of the AV.

17. one fallback stop position among the one or more fallback stop positions, a roadside within the environment of said AV; or 17. The computer-readable storage medium of claim 16, comprising at least one of a center portion of a lane of a road within an environment of the AV.

18. 17. The computer-readable storage medium of claim 16, wherein the respective fallback trajectories of the one or more fallback trajectories reduce the velocity of the AV to zero at a respective fallback stop within a threshold time.

19. a selection metric associated with each fallback trajectory; the distance of each fallback trajectory to the fallback stop position; whether the respective fallback trajectories intersect with predicted trajectories of objects in the AV's environment; or The computer-readable storage medium of claim 16 , wherein the fallback trajectory is based on at least one of a lateral shift.

20. The executable instructions cause the processing unit to: generating a guide curve; 17. The computer-readable storage medium of claim 16, further comprising: generating a fallback trajectory of the one or more fallback trajectories based on the guide curve.

Images